River and lake object management-oriented knowledge graph autonomous construction and retrieval enhancement method

Abstract

A knowledge graph autonomous construction and retrieval enhancement method for river and lake object management, comprising: based on the structured data table in the water conservancy business system, through the large language model fine-tuned by the river and lake management field knowledge, the correlation between the river and lake management objects is mined, the triple pattern is generated, and the triple is extracted and verified; based on the triple, a knowledge graph is constructed, the comprehensive score of the nodes in the graph is calculated through a weighted scoring mechanism that fuses jurisdiction, space, business and hierarchical correlation, and a multi-hop business subgraph is extracted from the knowledge graph according to the comprehensive score; the business subgraph is information condensed and converted, and converted into a semantic vector through a pre-trained semantic model special for the water conservancy field and stored in a vector database; receiving a user query, converting the query into a query semantic vector, retrieving relevant subgraph knowledge from the vector database as context through similarity calculation, and inputting into the large language model to generate a reply based on professional knowledge.

Description

Technical Field

[0001] This invention relates to a method for autonomous construction and enhanced retrieval of knowledge graphs for river and lake object management. Background Technology

[0002] With the deepening of water conservancy informatization, the demand for intelligent processing of massive amounts of water conservancy data in scenarios such as digital twin watersheds and smart river and lake management is becoming increasingly urgent. Traditional water conservancy data covers basic data, monitoring data, and geospatial data of various river and lake management objects, including rivers, reservoirs and dams, water intakes, and the river chief system. These data are mainly distributed in relational databases and spatial databases, while knowledge platforms also include model calculation results and expert experience, which are mainly stored in graph databases. This characteristic of multi-source heterogeneous data brings enormous challenges to the intelligent processing of water conservancy data.

[0003] Knowledge graphs (KG), as a structured knowledge representation paradigm, can effectively organize the complex relationships between water conservancy objects. However, they suffer from drawbacks such as reliance on human experience in ontology design, high construction costs, and long iteration cycles. Retrieval-enhanced generation (RAG) technology, by integrating external knowledge retrieval and generation models, can improve the accuracy and factuality of question answering, but lacks a deep understanding of professional knowledge and logical reasoning capabilities in the water conservancy field. The fusion of KG and RAG technologies can achieve complementary advantages and become a key path for solving the data association mining and intelligent querying of river and lake management objects. However, the table-structured data in the water conservancy field, surrounding river and lake management objects, exhibits characteristics such as complex business logic, dense professional terminology, and prominent spatial relationships. Traditional technologies struggle to achieve efficient extraction, association construction, and rapid retrieval of table-structured information, hindering the implementation efficiency of smart river and lake management.

[0004] The shortcomings of existing technologies are concentrated in the following aspects: First, the extraction efficiency and accuracy of table structure triples are low: traditional rules rely on manual definition and have poor adaptability; general large models have difficulty understanding foreign key associations and spatial relationship fields between water conservancy tables, which can easily lead to illusions and result in missing or incorrect triples; Second, the adaptability of graph topology construction and subgraph extraction is poor: existing topology construction does not fully explore the hierarchical and spatial relationships of water conservancy objects; subgraph extraction uses general BFS / DFS algorithms, which lack a node priority matching mechanism and cannot accurately match the query needs of water conservancy business, which can easily lead to information redundancy or omissions; Third, the retrieval and question answering effects are not good: the method of generating SQL statements by adding prompt words to large models has problems such as table name / field recognition errors and syntax errors, and is often time-consuming; general knowledge graph retrieval methods do not perform textual semantic enhancement for water conservancy subgraphs, and the generated results are not professional and difficult to support business decisions. Summary of the Invention

[0005] One aspect of this invention is to address the problems of existing technologies (including the BERT+BiLSTM+CRF hybrid model) having a shallow understanding of the structure of water conservancy tables, low automation of triple extraction, and high illusion rate, so as to achieve accurate extraction of water conservancy entities, attributes, and relationships in table structure data.

[0006] Another objective of this invention is to address the problems of existing graph topology construction not being adapted to the characteristics of water conservancy business and insufficient subgraph extraction accuracy, and to achieve dynamic construction of the association relationship of water conservancy objects and accurate extraction of business-oriented subgraphs.

[0007] Another objective of this invention is to address the problems of error-prone SQL generation and the imbalance between efficiency and accuracy in KG-RAG retrieval in existing retrieval technologies. By employing a multi-hop node filtering strategy based on cosine similarity, subgraph compression and efficient retrieval are achieved, thereby improving the accuracy and response speed of question answering in water conservancy scenarios.

[0008] A method for autonomous construction and retrieval enhancement of knowledge graphs for river and lake object management includes: based on structured data tables in a water conservancy business system, using a large language model fine-tuned with knowledge from the river and lake management domain to drive the mining of association patterns between river and lake management objects, generating standardized triplet paradigms, and extracting and verifying high-quality (standard-compliant) triples accordingly; constructing a knowledge graph based on the triples, calculating the comprehensive score of nodes in the graph through a weighted scoring mechanism that integrates jurisdiction, space, business, and hierarchical associations, and extracting multi-hop business subgraphs from the knowledge graph based on the comprehensive scores; performing information condensation and escaping processing on the extracted business subgraphs, converting them into semantic vectors through a pre-trained semantic model specific to the water conservancy domain, and storing them in a vector database; receiving user queries, converting the queries into query semantic vectors, retrieving relevant subgraph knowledge from the vector database as context through similarity calculation, inputting it into the large language model, and generating a response based on professional knowledge.

[0009] In some examples, the process of generating standardized triplet paradigms and extracting and validating high-quality triples based on them includes: parsing the physical definition and structure of the data table; identifying logical relationships between fields within the table and relationships between tables formed through foreign keys or semantic matching based on a water conservancy terminology database and a business rule database; inputting the identified relationships into a large language model that has been fine-tuned by instructions; identifying high-frequency association patterns through unsupervised learning; and generating a structured triplet paradigm that defines subject type, relationship type, object type, and metadata by combining it with river and lake management industry standards; filling the triplet paradigms with the actual record values ​​from the data table to generate an original triplet set; performing logical consistency verification on the original triplet set based on the water conservancy industry rule set; manually confirming any fuzzy triples present during the verification; and outputting the final valid triplet set.

[0010] In some examples, during the generation of the structured triple paradigm, instruction fine-tuning technology is used to adapt the base large language model to the domain. The fine-tuning dataset used contains samples constructed from water conservancy industry standards, business manuals and historical data. The format of the samples is a standardized paradigm output of instructions, inputs and manual annotations.

[0011] In some examples, the process also includes a step of performing paradigm adaptability verification on newly added data tables. Verification metrics include field mapping coverage, data type consistency, and semantic similarity score. If the verification fails, a dynamic optimization process is triggered, driving the large language model to generate new or update existing paradigm rules.

[0012] In some examples, the step of calculating the overall score of nodes in the knowledge graph involves assigning basic weight values ​​to the edges in the knowledge graph. The calculation formula is as follows: (1) in, , , , These represent quantitative relevance scores for jurisdictional, spatial, business, and hierarchical relationships, respectively. to These are the preset weight coefficients for the corresponding association types, and the sum of the weight coefficients is 1.

[0013] In some examples, nodes Relative to the central node Overall retention score The calculation formula is as follows:

[0014] in, Indicates the central node c With nodes i The weight values ​​of the edges connecting them. It is a constant. The commuting distance between the two nodes. The score is based on preset weighting coefficients. The strength of the direct connection was also taken into consideration (by... The density of the topology (CTD) and the global topology density (CTD) are determined.

[0015] In some examples, the process of converting and storing the business subgraph into semantic vectors includes: converting the JSON structure of the business subgraph into high-information-density natural language description text; using an encoder model finely tuned by contrastive learning techniques to encode the natural language description text into fixed-dimensional semantic vectors; associating and storing the semantic vectors, the corresponding natural language description text, and atomic graph metadata in the same set of a vector database; and constructing an efficient approximate nearest neighbor search index for the semantic vector fields.

[0016] In some examples, the collection built in the vector database includes: a primary key field, a semantic vector field, a payload field storing escaped text, and a metadata field storing the atomic graph structure; the semantic vector field supports millisecond-level approximate nearest neighbor search by building a hierarchical navigable small world graph index.

[0017] In some examples, generating a response based on professional knowledge includes: converting the user's natural language query into a query semantic vector using a pre-trained semantic model specific to the water resources field; calculating the cosine similarity between the query semantic vector and the semantic vectors stored in the vector database, and recalling the top N subgraph knowledge fragments with the highest similarity; deduplicating and sorting the recalled knowledge fragments by relevance, and filling them together with the user query into a preset water resources professional prompt template to form a prompt message which is then input into a large language model to generate a professional answer with information source annotation.

[0018] In some examples, the water resources professional prompt template includes system role settings, factual constraints, tracing requirements, and a thought chain guidance structure, which are used to constrain the large language model to reason and answer strictly based on the provided subgraph knowledge.

[0019] This invention constructs a standardized knowledge paradigm system for rivers and lakes, solving the problems of low efficiency and insufficient accuracy in traditional triple extraction. By driving the mining of correlation patterns among river and lake management objects through a large model, and combining industry standards to generate a proprietary paradigm, it avoids the "illusion" problem of general models. Furthermore, through industry rule verification and manual confirmation, it ensures the accuracy and completeness of the triples, significantly reducing the cost of manual intervention. The paradigm system can also dynamically adapt to newly added river and lake management objects.

[0020] This invention establishes a weighted standard system for river and lake management business entities, solving the problem of poor adaptability between graph topology construction and subgraph extraction. The weighted scoring mechanism combines business weights with the degree of node connectivity, enabling subgraph extraction to focus on core related entities and effectively eliminate redundant information. At the same time, the multi-level subgraph construction logic adapts to the hierarchical and networked characteristics of water conservancy business, improving the matching degree between subgraphs and query requirements.

[0021] This invention improves the retrieval and question-answering performance in river and lake management scenarios, solving the problem of poor question-answering results in existing retrieval technologies. It avoids the risk of table name / field recognition errors in SQL generation patterns, achieving efficient and accurate retrieval through subgraph vectorized storage and similarity matching. Furthermore, the dedicated pre-trained model for river and lake management and the subgraph-specific knowledge input ensure the professionalism and factuality of the response results, directly supporting business decision-making.

[0022] This invention enhances its adaptability to the water conservancy industry. The entire process integrates a water conservancy professional knowledge base and industry rule set, ensuring that everything from triple extraction to retrieval and response aligns with water conservancy and river / lake management business scenarios. It can directly address core needs such as digital twin watershed construction and smart river / lake management, covering multiple scenarios including river governance, water resource allocation, and the implementation of the river chief system. Attached Figure Description

[0023] Figure 1 This is a flowchart of a knowledge graph autonomous construction and retrieval enhancement method for river and lake object management according to an embodiment of the present invention.

[0024] Figure 2 This is a flowchart of the autonomous extraction of triples in one embodiment of the present invention.

[0025] Figure 3 This is a schematic diagram comparing the search effects of different search technologies in a typical business scenario of water conservancy and river and lake management according to an embodiment of the present invention. Detailed Implementation

[0026] like Figure 1 As shown, a HydroLite-RAG (Hydro-Map-Lightweight-Retrieval Enhancement Generation) method includes: based on structured data tables in a water conservancy business system, using a large language model fine-tuned with knowledge from the river and lake management domain, driving the mining of association patterns between river and lake management objects, generating standardized triplet paradigms, and extracting and verifying high-quality (standard-compliant) triples accordingly; constructing a knowledge graph based on the triples, calculating the comprehensive score of nodes in the graph through a weighted scoring mechanism integrating jurisdiction, space, business, and hierarchical associations, and extracting multi-hop business subgraphs from the knowledge graph based on the comprehensive scores; performing information condensation and escaping processing on the extracted business subgraphs, and converting them into semantic vectors through a pre-trained semantic model specific to the water conservancy domain, storing them in a vector database; receiving user queries, converting the queries into query semantic vectors, retrieving relevant subgraph knowledge from the vector database as context through similarity calculation, inputting it into the large language model, and generating a response based on professional knowledge. The method is described in detail below.

[0027] Step 1: Automatically extract triples from the water conservancy and river / lake object data table.

[0028] The system inputs data tables of river and lake management objects (river basic information table, river chief responsibility table, water intake monitoring table, reservoir dam operation and maintenance table, etc., including table name, field name, field value, foreign keys between tables, and other related information). Unlike the traditional method that relies on manually defined paradigms, this invention innovatively discovers the correlation patterns between river and lake management objects through a large model, constructs a standardized river and lake knowledge paradigm, and outputs high-quality triples after steps such as preprocessing, river and lake knowledge paradigm generation, extraction of triples from table data, verification and correction, and manual confirmation.

[0029] Step 1.1: Data preprocessing.

[0030] In the preprocessing stage, based on multiple professional knowledge bases such as the water conservancy professional tag set, the water conservancy professional terminology database, the common relationship database of water conservancy database tables, and the business rule database of river and lake management objects, the specific technical means are as follows: First, source data feature extraction is performed. The physical definition information (DDL) of all data tables is obtained by traversing the database connection interface, including table names, field names, data types, field comments, primary key constraints, and foreign key definitions. Next, field semantic mapping is performed. A water resources professional terminology database and a water resources professional tag set are used to semantically align the extracted field names and field comments. A string fuzzy matching algorithm (Levenshtein Distance) is used to calculate the similarity between field comments and water resources professional terms. For example, when a field comment contains keywords such as "river chief," "section," or "reservoir capacity," it is marked as an entity attribute with business meaning, rather than a meaningless system field. Finally, association rules are parsed. Based on a common association database of water resources database tables and a business rule database of river and lake management objects, potential constraints between fields are analyzed. The system automatically scans field naming characteristics to identify non-explicit foreign keys (for example, fields ending in _code or _cd are usually associated with dictionary tables, and fields ending in _id and not primary keys usually imply inter-table associations), and merges these potential associations with explicit foreign key definitions.

[0031] Therefore, intra-table field relationships and inter-table field relationships were extracted. Intra-table field relationships refer to identifying logical relationships between fields within the same table, with a focus on identifying logical relationships between river and lake management objects. These include management responsibility relationships (correspondence between management entities and river / lake management objects, and matching relationships between management entities and responsible areas), river / lake attribute relationships (subordinate relationships between river / lake management objects and hierarchical types, watershed affiliations, and functional positioning), and monitoring data relationships (correspondence between river / lake management objects and monitoring indicators and numerical thresholds). Inter-table field relationships refer to mining the relationships formed between different tables through foreign keys and semantic matching. These mainly include explicit foreign key relationships (foreign key relationships between data tables), implicit semantic relationships (indirect relationships between river / lake management-related entities formed based on field semantic matching), and hierarchical jurisdiction relationships (hierarchical relationships formed based on administrative division codes and river / lake hierarchical codes).

[0032] Step 1.2: Triple normal form extraction.

[0033] A finely tuned large model for river and lake management is introduced (based on a general large model with adjustments made to incorporate data from river and lake management regulations, industry standards, and business manuals). Using the intra-table / inter-table field relationships extracted in the previous steps as a foundation, and combining the large model's ability to mine the relationship patterns of river and lake management objects, a standardized river and lake knowledge paradigm is generated and output in JSON format. This paradigm not only includes the core structure of subject fields (river and lake management object types), relationship types (exclusive relationship definitions), and object fields (related river and lake management object types), but also labels metadata such as the table name, data type, and business scenario tags of the related fields.

[0034] In this step, a large language model based on Instruction Tuning is used to generate a standardized knowledge paradigm for rivers and lakes. First, for the selection of the base large model, the open-source large language model Qwen is adopted, which has general semantic understanding and reasoning capabilities. Then, a special instruction fine-tuning dataset for water conservancy and river-lake management is constructed. In terms of data source, a batch of business-related data is extracted based on unstructured documents such as *Code of Practice for River and Lake Chief System*, *Environmental Quality Standards for Surface Water*, water conservancy business manuals and historical stock database records. Then, in terms of sample construction, samples are constructed in the format of (Instruction, Input, Output). Where Instruction is "Please extract the water conservancy business entities and relational paradigm in the table structure"; Input is the DDL definition, field comments and sampling data of the database table; Output is the standard JSON paradigm manually annotated; Then, LoRA (Low-Rank Adaptation) efficient fine-tuning technology is used to train the base model. By freezing the main parameters of the pre-trained model, only trainable low-rank matrices (Rank=8 or 16) are inserted into the Attention layer of Transformer, which reduces the video memory consumption and injects prior knowledge in the water conservancy field (such as "the river section belongs to the administrative division", "the reservoir contains the dam body") into the model parameters. Finally, the intra-table / inter-table field association relationship extracted in step 1.1 is serialized into a natural language description, which is input into the fine-tuned model as a Prompt, and the model will directly infer and output a standardized JSON knowledge paradigm based on the learned domain knowledge. This paradigm includes subject fields (such as the type of river and lake management object), relationship types (definition of exclusive relationship), object fields and meta-information.

[0035] The generation process driven by the large model can be described as the following process: First, the large model is used to perform unsupervised learning on the preprocessed association relationship data to identify high-frequency association patterns (such as river section-river chief system-responsible body, water intake-water resources zoning-water use index, etc.) (First, perform statistical analysis on the intra-table and inter-table association relationship data output in step 1.1. The system calculates the co-occurrence frequency of different field type combinations (such as "code field-name field", "primary key-foreign key"), and uses the Apriori algorithm to screen out candidate association combinations with confidence higher than a preset threshold (such as 0.8)).

[0036] Next, the high-frequency candidate combinations mentioned above are input into the fine-tuned large model. Utilizing the semantic understanding and inductive capabilities of the large model, semantic clustering is performed on the candidate combinations. For example, the large model identifies that although "river segment code-river segment name" and "reservoir code-reservoir name" have different field names, they both belong to the same semantic pattern of "entity-identifier attribute". Combining this with the input context of river and lake management industry standards, the large model standardizes the clustered patterns (e.g., explicitly naming the identified jurisdictional relationship "river chief system responsibility jurisdiction"), ultimately generating a standardized JSON knowledge paradigm.

[0037] Finally, after generating the basic paradigm library, the large model is used again to verify the paradigm adaptability of the newly added table structure data, dynamically optimize the paradigm rules, and finally form a standardized knowledge paradigm system covering all river and lake management objects, including two major categories: entity-attribute paradigms (such as reservoir dam-dam height-value) and entity-entity paradigms (such as river chief system-responsibility jurisdiction-river section).

[0038] Among them, the normalization fit verification is based on the field mapping coverage. Data type consistency and semantic similarity score The score is based on three indicators.

[0039] Field mapping coverage It calculates the proportion of key attribute fields (such as name, code, and location) defined in the existing paradigm that can be found in the newly added table.

[0040] Data type consistency It verifies whether the data types of the mapped fields are compatible (for example, the normal form requires "water level" to be a floating-point type, which is inconsistent if the table is a character type).

[0041] Semantic similarity score It uses the Embedding model to calculate the vector cosine similarity between table annotations and normalized descriptions.

[0042] The system makes a judgment based on three scoring thresholds (all three indicators are ≥0.8). If all three indicators are higher than the threshold, the adaptation is considered successful and the existing paradigm can be directly reused. Otherwise, it is judged as a new business scenario and a dynamic optimization process is triggered: the newly added table structure is fed back to the large model as a negative sample, driving the model to generate new paradigm rules or update the existing paradigm (such as adding new attribute fields), thereby realizing the self-iteration and expansion of the knowledge paradigm system.

[0043] The core entity types include, but are not limited to, entities related to river and lake management objects such as rivers / lakes, administrative divisions, water conservancy projects (reservoirs / sluices / pumping stations), management entities, governance projects, monitoring stations, and responsible areas. The relationship types mainly include jurisdictional relationships (management entities and river / lake management objects, management entities and responsible areas, etc.), spatial relationships (river / lake management objects and administrative divisions, upstream and downstream relationships between river / lake management objects, river / lake management objects and watershed affiliation, etc.), business relationships (monitoring stations and river / lake management objects), and hierarchical relationships (hierarchical inclusion between administrative divisions, hierarchical affiliation of river / lake objects, etc.). The normal form metadata records the subject field name and its table, relationship type code and business scenario description, object field name and its table, distinguishing field (key field used to uniquely identify entities), and data type constraints to ensure that the normal form has clear business orientation and executability.

[0044] Step 1.3: Extract triples from table data.

[0045] The actual data in each table is matched against the triple normal form, and the specific values ​​of the subject and object are automatically filled in to generate the original set of triples. The extraction process strictly follows the data type constraints and uniqueness requirements of the distinguishing fields in the normal form to ensure the structural consistency and relevance of the original triples. Specifically, this includes constructing unique identifiers (URIs), aligning heterogeneous entities from multiple sources, and handling attribute fusion and conflicts.

[0046] Unique identifiers (URIs) are constructed based on distinguishing fields: These fields are read from the meta-information (e.g., "river code," "administrative division code"), or a globally unique identifier (URI) is constructed using the rule of "namespace + entity type + distinguishing field value." For example, for a river entity, ns:River:500100 is generated; for entities with only a name and no code, the identifier is generated using the name hash value. This ensures that even if multiple records exist, as long as the distinguishing field value is the same, it will always be mapped to a unique graph node in the graph.

[0047] Multi-source heterogeneous entity alignment: To address the issue of inconsistent identifiers for the same entity across different tables (e.g., a river is identified as a standard code in the base table but as a Chinese name in the monitoring table), dictionary mapping alignment and attribute similarity alignment are employed. The dictionary mapping alignment method constructs a "name-code" mapping dictionary (derived from the standard base information table) during the preprocessing stage. During extraction, if the table only contains the name, the system automatically queries the dictionary to convert it to the standard code, thereby generating a unified URI. The attribute similarity alignment method calculates the Jaccard similarity coefficient for key attributes (such as "administrative region," "length," and "latitude and longitude") of entities that cannot be directly mapped. If the similarity exceeds a preset threshold (e.g., 0.8), the entity is considered the same, and an association is automatically established or attribute merging is performed.

[0048] Attribute fusion and conflict handling: When different data sources conflict in their descriptions of the same entity's attributes (e.g., the "dam height" recorded in two tables is inconsistent), a data source confidence grading mechanism is introduced. Data from the "Basic Information Table" or "Ledger Table" (high confidence) is prioritized for retention, while data from the "Temporary Monitoring Table" or "Filing Form" (low confidence) is overwritten to ensure the accuracy of attribute values ​​in the knowledge graph.

[0049] Step 1.4: Verification, correction and manual confirmation.

[0050] Through a dual verification process involving automated validation and manual secondary confirmation, triples with obvious logical errors are eliminated, resulting in the final valid triples. Automated validation involves calling the water conservancy industry rule set to perform logical consistency checks on the original triples, eliminating contradictory or invalid triples. Manual secondary confirmation addresses ambiguous triples that automated validation cannot clearly determine (such as jurisdictional relationships in special areas or the association of newly added water conservancy facilities). A manual review interface is provided, and triples confirmed through this process are included in the set of valid triples, forming the final high-quality (standard-compliant) triples.

[0051] Automated logic verification utilizes the system's built-in water conservancy business rule engine to perform rule matching on the original triples, automatically eliminating erroneous data that violates physical laws or common business sense. Specific verification rules include, but are not limited to, physical attribute limit verification (numerical logic) and river chief system hierarchical jurisdiction matching verification (business logic).

[0052] Rule Example 1.1: Physical Property Limit Verification (Numerical Logic).

[0053] Definition: If the entity type is "reservoir dam", then the value of its attribute "current real-time water level" must be less than or equal to "dam crest elevation" or "check flood level".

[0054] Execution logic: IF (Entity.Type == 'Reservoir' AND Entity.Attr['CurrentWaterLevel']>Entity.Attr['DamCrestElevation']) THEN Status = 'Invalid'. This type of triplet usually originates from sensor malfunction or data entry errors and will be directly rejected by the system.

[0055] Rule Example 1.2: River Chief System Hierarchical Jurisdiction Matching Validation (Business Logic).

[0056] Definition: The administrative level of the management entity must be higher than or equal to the level of the rivers and lakes under its jurisdiction (based on documents and other materials in the project).

[0057] Execution logic: IF (Subject.Type == 'RiverChief' AND Subject.Level == 'VillageLevel' AND Object.Level == 'ProvincialLevel') THEN Status = 'Conflict'. For example, if the extraction results show that "a village-level river chief" is in charge of "the main stream of the Yangtze River (provincial / national level section)", the system judges it as a hierarchical inversion error and automatically marks it as invalid.

[0058] Manual secondary verification is used for fuzzy triples that are judged as "suspected errors" or "confidence level below 0.4" in the automated verification (e.g., the same water intake is associated with two different water intake permit numbers, which may be due to historical changes or errors). The system pushes these triples to the manual review interface. Reviewers use visual tools to confirm, correct, or delete the data. The confirmed data is then merged into the final set of valid triples. Figure 2 A brief process for the autonomous extraction of triples is given.

[0059] Step 2: Graph structure construction and weighted scoring formula subgraph extraction.

[0060] This step involves multiple steps, including graph construction, node edge weight calculation, and multi-hop subgraph extraction, to achieve the functions of knowledge graph construction and subgraph information extraction and compression, ultimately forming multiple subgraph JSONs.

[0061] Step 2.1: Graph topology construction.

[0062] Based on the triples extracted in step 1 above, with entities as nodes and relationships as edges, a graph topology is constructed that includes jurisdictional, spatial, business, and hierarchical relationships. Node attributes include entity type and professional information (such as river / lake classification, management priority, administrative division level, and type of responsible entity). Edge weights are assigned basic weight values ​​based on weighting rules designed according to the jurisdictional, spatial, business, and hierarchical relationships between entities. The formula is as follows: (1) in, This is the jurisdictional relevance, with a value range of [0, 1]. It is assigned a value based on the closeness of the relationship between the management entity and the river / lake management object and the area of ​​responsibility. The more direct the relationship, the higher the value. For spatial correlation, the value range is [0, 1]. The correlation strength is determined based on the geospatial attributes of the river and lake management objects. The data sources are fields such as administrative division code, watershed code, river section code, and coordinate range. For business-related relevance, the value range is [0, 1]. The strength of the association is determined based on the river and lake management objects and the actual management business logic. The data sources are foreign key association fields, project ownership fields, facility binding fields, etc. The correlation is hierarchical, with a value range of [0, 1]. The degree of correlation is determined based on the hierarchical and classification attributes of river and lake management objects. The main data sources include fields such as hierarchical code, hierarchical type, and attribution type. to These are the weighting ratios of these four types of correlations, and their sum is 1.0.

[0063] The specific quantitative calculation rules for each correlation are as follows: (Jurisdiction Relevance) Assigned based on the functional matching degree between the responsible entity and the object. If there is a direct legal jurisdiction relationship (e.g., "district-level river chief" directly corresponds to "district-level river section"), If it is a superior-level supervisory relationship (such as "municipal-level river chief" associated with "district-level river section"), If it is a cross-departmental collaborative relationship (such as the "Environmental Protection Bureau" being associated with "Reservoirs under the jurisdiction of the Water Resources Bureau"), Other weak association cases, .

[0064] (Spatial correlation) is determined based on GIS spatial topology calculations. If the geometric center of a river or lake object is entirely within (inclusive of) the administrative division polygon. If the river's shape crosses the administrative division boundary (intersects). If the two do not overlap, calculate their Euclidean distance. (Unit: meter). If (e.g., 500 meters), then The effect of distance is simulated using logarithmic decay; otherwise... .

[0065] (Business Relevance) is assigned based on the strength of database constraints. Strong constraints mean there is an explicit foreign key in the database or a relationship between intermediate tables. Strong semantics, meaning that the field values ​​match exactly (e.g., the "related river name" in the project table matches the river name in the river table). Weak semantics, meaning matching based on text similarity (Jaccard coefficient) and greater than a threshold of 0.8. .

[0066] (Hierarchical correlation) Based on the depth difference of the tree hierarchy ( Assignment. Parent-child direct hierarchy ( ), generational gap ( ), sibling nodes of the same level .

[0067] Regarding weighting ratios ( to The weighting is based on the analytic hierarchy process (AHP), calculated by a scoring matrix of at least five water resources information technology experts assessing the importance of different association types in search operations (e.g., initially set as follows). ).

[0068] Step 2.2: Calculate the edge weights of nodes.

[0069] Next, for each node, the comprehensive weight of its adjacent nodes is calculated by weighting the commuting time distance (CTD) and the basic weight. The formula is as follows: (2) in, This is the weighting coefficient (default value is 0.6, which can be adjusted according to specific business scenarios). The commuting time distance between nodes reflects the connectivity between nodes in the network. The smaller the value, the closer the connection, and therefore the higher the overall weight, representing a stronger business connection between the node and the central node.

[0070] To address the issue that traditional shortest path algorithms cannot reflect the "multi-dimensional concurrent associations" between water conservancy objects, this step introduces Commute Time Distance (CTD) to calculate the structural density between nodes on the global graph topology constructed in step 2.1.

[0071] In river and lake management, important connections often exhibit "multi-path reinforcement" characteristics. For example, a significant "monitoring station" node and a "river" node typically share a spatial containment edge (SMILE). ), business monitoring edge ( ) and administrative affiliation boundary ( Traditional shortest path algorithms can only capture the edge with the highest weight, ignoring the contributions of other parallel connections. The CTD (Cross-Track Time) metric simulates a random walk model; the more connected paths between nodes and the greater the overall weight, the shorter the round-trip time (CTD). Therefore, introducing CTD can effectively "reward" core nodes with strong spatial, administrative, and business connections, and "penalize" edge nodes with only a single weak connection (such as those with only administrative roles and no actual business dealings), thus accurately selecting neighbors with the highest business coupling to the central node.

[0072] The calculation of CTD is based on the Laplacian matrix of the global graph, and the specific steps are as follows: First, construct the Laplace matrix. Let the adjacency matrix of the graph be... The degree matrix is (A diagonal matrix, where each element is the sum of the weights of the edges connecting the nodes), then the Laplace matrix... Since the graph may not be completely connected, calculation The pseudo-inverse matrix (Moore-Penrose Pseudoinverse) is denoted as Next, calculate the CTD value: node With nodes Commuting time distance between The calculation formula is:

[0073] in, It is the sum of the weights of all edges in the graph. It is a pseudo-inverse matrix Middle Line 1 The elements of the column. Finally, after obtaining the CTD, it is normalized and combined with the base weights from step 2.1. compute nodes Relative to the central node Final overall retention score :

[0074] in, Indicates the central node c With nodes i The weight values ​​of the edges connecting them. To prevent the minimum value where the denominator is zero, this scoring... The strength of the direct connection was also taken into consideration (by... The density of the topology (CTD) and the global topology density (CTD) are determined.

[0075] Step 2.3: Extraction of multi-hop subgraphs.

[0076] First filter the central node The adjacent nodes of the high-scoring nodes (S calculated in step 2.2) form a first-order subgraph. (Default 5, configurable), then using the first-order subgraph node as the new center, repeat the scoring and point selection process to form a second-order subgraph, the jump ends at the preset number of jumps (default 2 jumps), and the final output is a subgraph JSON containing only the core related river and lake management objects.

[0077] Step 3: Subgraph JSON escaping and vectorization representation.

[0078] In this step, a pre-trained large model specifically for river and lake management (reusing the fine-tuned large model from step 1.2) is used. First, the JSON of the selected sub-graphs is escaped to remove redundant symbols and condense core information, increasing information density. After escaping, it is converted into a 768-dimensional semantic vector. An independent Encoder-only pre-trained model is introduced, with an output dimension of 768. This model has a different architecture from the generative model in step 1.2 and focuses on the semantic representation of text. To enable the general embedding model to understand the similarity of water conservancy semantics such as "reservoir capacity" and "flood control level," contrastive learning is used for full parameter fine-tuning. For training data, a (Query, Positive_Doc, Negative_Doc) triple is constructed. Where Query is the water conservancy business problem, Positive_Doc is the escaped text of the corresponding subgraph generated in step 1, and Negative_Doc is the text of other randomly sampled subgraphs; in terms of loss function selection, the InfoNCE Loss loss function is adopted to bring the Query closer to the relevant subgraph vectors and push away the irrelevant subgraph vectors.

[0079] Subsequently, the semantic vectors are associated with the corresponding escaped knowledge content and stored in the Milvus vector database, establishing a vector index-escaped knowledge retrieval structure that supports millisecond-level vector matching and solves the slow retrieval problem caused by the direct storage of full-image vectors in the existing KG-RAG method.

[0080] To achieve millisecond-level retrieval and solve the problem of high latency in multi-hop queries in traditional graph databases, this invention designs an integrated "index-content" storage architecture, which is implemented as follows: In terms of storage design, a Collection is built in the Milvus vector database, containing the following key fields to achieve data self-containment: Primary key field (ID), uniquely identifying the subgraph; Vector field (Embedding), used to store the 768-dimensional semantic vector output in step 3.2, serving as the retrieval entry point; Payload field (Payload_Text), used to store the natural language text generated in step 3.1. This is the contextual material directly input into the large model to generate answers; it is directly retrieved upon retrieval without secondary queries; Metadata field (Meta_Data), storing the JSON structure of the original subgraph, used for front-end visualization or result traceability verification.

[0081] Regarding our index building strategy, we constructed an HNSW (HierarchicalNavigable Small World) index for the Embedding field. (Parameter settings are then configured.) (Number of neighboring nodes) and (Building Depth) While maintaining a recall rate of over 98%, it supports millisecond-level near nearest neighbor (ANN) search on datasets of hundreds of millions. Through this structure, the system completely decouples from the graph database during the online query phase. Compared to traditional methods... In this way, the present invention uses This mechanism of offline pre-computed subgraphs combined with online vector direct retrieval is the core technical means by which this invention solves the efficiency bottleneck of complex hydraulic subgraph retrieval.

[0082] Step 4: Semantic subgraph retrieval and large model generation.

[0083] In practical business scenarios, the process involves query escaping, similarity retrieval, knowledge input, and model response. First, the user's question is converted into a vector. Then, the closest subgraph escaping knowledge set is found through vector matching. Finally, the retrieved subgraph escaping knowledge set is input into the large model to form the final response. Compared to directly using a large model for question answering, this approach enhances the large model's deep understanding of water conservancy expertise and logical reasoning capabilities, avoiding the illusion problem inherent in general large models.

[0084] Step 4.1: User query escaping After receiving the user's query, the query is first converted into a semantic vector using the same pre-trained large model for water conservancy (reusing the instructions in step 1.2 to fine-tune the large model), ensuring that the query vector and the stored subgraph vector have a consistent semantic space. Step 4.2: Similarity search.

[0085] Calculate the cosine similarity between the query vector and the semantic vectors of the subgraphs in the database, and prioritize selecting the vectors with the highest similarity. One as a candidate set ( The value is configurable; the default value is 1024.

[0086] Step 4.3: Large model generation.

[0087] The escaped knowledge content in the candidate set is used as the input model of water conservancy-specific knowledge. The model is guided by prompt templates to perform logical reasoning and content generation based on professional knowledge, so as to ensure that the final response is professional, accurate and logical, and can directly support business decision-making.

[0088] This step transforms the retrieved fragmented subgraph knowledge into structured Prompt input and guides the large model to generate answers using carefully designed prompt templates. The specific implementation includes the following workflow: First, the knowledge context is cleaned and assembled. For the top n candidate escaped knowledge fragments recalled in step 4.2, the following processing logic is executed sequentially to construct a high-quality context: Semantic deduplication. The Jaccard similarity or edit distance between candidate fragments is calculated. If the similarity between two texts is higher than a preset threshold (e.g., 0.8), it is considered redundant information, and only the one with the higher vector similarity score is retained to avoid token waste caused by context repetition; Relevance reordering. The deduplicated knowledge fragments are sorted from high to low according to the vector similarity scores in step 4.2, ensuring that the most core knowledge (e.g., the directly hit river chief information) is at the beginning of the Prompt, conforming to the "primacy effect" attention mechanism of large models; Structured splicing. The selected fragments are spliced ​​in order and indexed. The format is as follows: Context="[Reference Information 1]: ...\n[Reference Information 2]: ...\n". At the same time, a maximum token truncation limit is set (e.g., 4096 tokens) to ensure that the input does not exceed the model context window.

[0089] Next, a water conservancy-specific prompt template was designed. A structured prompt template using role-based design, knowledge constraints, and CoT (Concept of Thought) was adopted. The specific Prompt template is as follows: { System Prompt: You are a professional assistant for water conservancy and river / lake management. Please answer the business questions raised by users based on the provided [Reference Information].

[0090] Constraints: 1. Factual Constraints: Fabricating facts is strictly prohibited. If the answer is not provided in the [Reference Information], please directly answer "This question cannot be answered based on the existing knowledge base," and do not supplement with external common sense. 2. Source tracing requirement: After the key conclusions in the answer, the source of the information must be indicated, for example, "... (Source: Reference Information 1)"; 3. Logical reasoning: When numerical comparisons (such as water level exceeding limits) or hierarchical relationships (such as river chief jurisdiction) are involved, please reason step by step.

[0091] User input: [Reference Information]: {Context} [User Issue]: {Query} answer: } Here, {Context} represents the knowledge concatenated above, and {Query} represents the user's question.

[0092] Finally, the answer is generated and validated, and the completed Prompt is input into the large model (the Qwen online model is selected in this invention). The model is based on the logic of thought chains, first parsing the entity relationships in the knowledge fragment (such as "identify that river section A belongs to area B"), then combining business logic to perform reasoning, and finally outputting a natural language answer including source annotations.

[0093] Figure 3 This diagram illustrates the comparison of retrieval performance of different retrieval technologies in typical business scenarios of water conservancy and river and lake management. The comparison is quantitatively performed from two core dimensions: retrieval time (milliseconds) and retrieval result accuracy (%). The comparison objects include the HydroLite-RAG (water conservancy atlas-lightweight-retrieval enhancement generation) method of this invention and the existing large model + prompt word SQL generation method.

[0094] In terms of retrieval efficiency, existing methods that generate SQL using large models and prompt words require multiple steps, including natural language to SQL syntax parsing, database syntax verification, and cross-table query adaptation, and the time consumption increases significantly in complex relational query scenarios. In contrast, this invention effectively compresses the average retrieval time by using subgraph compression filtering and the millisecond-level index matching capability of the Milvus vector database, which can improve retrieval efficiency by about 40%.

[0095] Regarding the accuracy of results, existing methods for generating SQL using large models and prompt words suffer from issues such as table name / field recognition errors and deviations in cross-table join logic parsing, resulting in an overall accuracy of only about 65%. Furthermore, SQL syntax errors can lead to retrieval failures in some scenarios. This invention, through semantic adaptation of a water conservancy-specific pre-trained model, precise extraction of weighted scoring subgraphs, and enhancement through dedicated knowledge input, achieves a stable retrieval accuracy of 90%, effectively avoiding the illusion problem and lack of professional information issues inherent in existing technologies.

[0096] In summary, the comparative data verifies that the HydroLite-RAG (Hydropower Map-Lightweight-Retrieval Enhancement Generation) method proposed in this invention has dual advantages in retrieval efficiency and result accuracy. It can efficiently support the intelligent query needs in water conservancy and river and lake management scenarios and provide reliable technical support for applications such as digital twin watersheds and smart river and lake management.

Claims

1. A method for autonomous construction and retrieval enhancement of knowledge graphs for river and lake object management, characterized in that, include: Based on the structured data tables in the water conservancy business system, a large language model finely tuned with knowledge from the field of river and lake management is used to drive the mining of the correlation patterns between river and lake management objects, generate standardized triplet paradigms, and extract and verify triplets that meet the standards. A knowledge graph is constructed based on the aforementioned triples, and basic weight values ​​are assigned to the edges in the knowledge graph. The calculation formula is as follows: ,in , , , These represent quantitative relevance scores for jurisdictional, spatial, business, and hierarchical relationships, respectively. to The weight coefficients for the corresponding preset association types are 1; the jurisdictional association relevance R is assigned based on the functional matching degree between the responsible entity and the river and lake management object; the spatial association relevance S is determined based on GIS spatial topology operations, using administrative division codes, basin codes, river segment codes, and coordinate ranges as data foundations, and the association strength is determined according to the Euclidean distance between the river and lake management object and the administrative division, whether they are spatially contained, intersecting, or non-overlapping; the business association relevance B is determined based on the river and lake management object and the actual management business logic, through the data constraint relationship represented by explicit foreign keys or intermediate table associations in the database, field value matching, or text semantic similarity matching; the hierarchical association relevance L is determined based on the hierarchical coding, hierarchical type, and affiliation type of the river and lake management object, through the difference in tree-like hierarchical depth; based on the aforementioned basic weight values... compute nodes Relative to the central node Overall retention score The calculation formula is as follows: ,in, Indicates the central node c With nodes i The basic weight values ​​of the edges connecting them. It is a constant. The commuting distance between the two nodes. The preset weighting coefficients are used; multi-hop business subgraphs are extracted from the knowledge graph based on the comprehensive retention score; The extracted business subgraph's JSON structure is converted into high-information-density natural language description text; a water conservancy-specific encoder model finely tuned using contrastive learning techniques is used to encode the natural language description text into fixed-dimensional semantic vectors; the semantic vectors, the corresponding natural language description texts, and atomic graph metadata are associated and stored in the same set of the vector database, and an efficient approximate nearest neighbor search index is constructed for the semantic vector fields; The system receives user queries, converts the queries into query semantic vectors, retrieves relevant subgraph knowledge from the vector database using similarity calculations as context, inputs it into a large language model, and generates a response based on professional knowledge.

2. The method according to claim 1, characterized in that, The process of generating a standardized triplet paradigm and extracting and validating triplets that conform to the paradigm includes: The physical definition and structure of the data table are analyzed, and based on the water conservancy professional terminology database and business rule database, the logical relationships between fields within the table and the relationships between tables formed by foreign keys or semantic matching are identified. The identified relationships are input into a large language model that has been fine-tuned by instructions. High-frequency association patterns are identified through unsupervised learning. Combined with industry standards for river and lake management, a structured triplet paradigm that defines subject type, relationship type, object type and metadata is generated. The actual record values ​​in the data table are filled into the triplet normal form to generate the original triplet set; the original triplet set is logically consistent with the water conservancy industry rule set, and the fuzzy triplets in the verification are manually confirmed to output the final valid triplet set.

3. The method according to claim 2, characterized in that, In the process of generating the structured triple paradigm, the instruction fine-tuning technique is used to adapt the base large language model to the domain. The fine-tuning dataset used includes samples constructed from water conservancy industry standards, business manuals and historical data. The format of the samples is a standardized paradigm output of instructions, inputs and manual annotations.

4. The method according to claim 2, characterized in that, It also includes a step of verifying the paradigm adaptability of newly added data tables. The verification metrics include field mapping coverage, data type consistency, and semantic similarity score. If the verification fails, a dynamic optimization process is triggered to drive the large language model to generate new or update existing paradigm rules.

5. The method according to claim 1, characterized in that, The collection constructed in the vector database includes: a primary key field, a semantic vector field, a payload field storing escaped text, and a metadata field storing atomic graph structures; the semantic vector field supports millisecond-level approximate nearest neighbor search by constructing a hierarchical navigable small world graph index.

6. The method according to claim 1, characterized in that, The process of generating a response based on professional knowledge includes: converting the user's natural language query into a query semantic vector using a pre-trained semantic model specific to the water conservancy field; calculating the cosine similarity between the query semantic vector and the semantic vectors stored in the vector database, and recalling the top N subgraph knowledge fragments with the highest similarity; deduplicating and sorting the recalled knowledge fragments by relevance, and filling them together with the user query into a preset water conservancy professional prompt template to form prompt information that is input into a large language model to generate a professional answer with information source annotation.

7. The method according to claim 6, characterized in that, The water conservancy professional prompt template includes system role settings, factual constraints, tracing requirements, and a thought chain guidance structure, which are used to constrain the large language model to strictly reason and answer based on the provided subgraph knowledge.

Images