Water conservancy knowledge differentiation retrieval and query decomposition method based on large language model
By constructing a multi-source heterogeneous data retrieval channel and sub-query system in the water conservancy field, and fine-tuning the large language model to achieve query decomposition and path planning, the problem of intelligent decomposition of multi-source heterogeneous data retrieval in the water conservancy field is solved, the retrieval accuracy and system interpretability are improved, and the training cost is reduced.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HOHAI UNIV
- Filing Date
- 2026-01-19
- Publication Date
- 2026-06-05
AI Technical Summary
Existing water conservancy knowledge retrieval technologies face challenges in intelligently decomposing and planning user queries when dealing with multi-source heterogeneous data environments. Furthermore, large language models cannot effectively understand the heterogeneous storage structures and business logic in the water conservancy field, leading to retrieval path errors or logical inversions.
We construct retrieval channels for graph, relational, document, and time-series databases, define subquery types and standard templates, fine-tune a large language model to achieve query decomposition and path planning, employ low-rank adaptation techniques for model training, and design a composite loss function to ensure query accuracy and logical consistency.
It achieves unified semantic alignment and differentiated retrieval of multi-source heterogeneous data, improves retrieval coverage and accuracy, supports intelligent logical decomposition of complex scheduling intentions, enhances the interpretability and decision reliability of the system, and reduces model training costs.
Smart Images
Figure CN122152902A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of water conservancy information technology, specifically involving a method for differentiated retrieval and query decomposition of water conservancy knowledge based on a large language model. Background Technology
[0002] With the deepening of the construction of smart water conservancy systems, the water conservancy industry has accumulated a massive amount of multi-source heterogeneous data resources. These data mainly include four aspects: First, topological network data representing the spatial connectivity of water conservancy facilities, such as the upstream and downstream connections of reservoirs, rivers, and sluice gates within a basin; this type of data is stored in graph databases in graph structure form. Second, attribute data storing various water conservancy engineering technical parameters, including key parameters such as reservoir capacity, flood control level, and design flow; this type of data is organized in relational databases in structured tables. Third, documents recording water conservancy scheduling and operation experience, such as flood control scheduling regulations and reservoir scheduling rules; this type of data exists in document repositories in natural language text form. Finally, real-time data from hydrological monitoring systems, including time-series data such as water level, flow rate, and rainfall; this type of data is stored in time-series databases.
[0003] Existing water resources knowledge retrieval technologies face significant technical bottlenecks when dealing with such multi-source heterogeneous data environments. The primary challenge is that the high heterogeneity of data structures makes it difficult to intelligently decompose and plan user queries. Water resources scheduling decisions often require the integration of information from multiple data sources. For example, querying "how to schedule water resources when a reservoir's water level reaches a specific value" requires simultaneously obtaining information such as the reservoir's key water level parameters, relevant scheduling rules, downstream impact range, and real-time monitoring data. This involves cross-source access to relational databases, document repositories, graph databases, and time-series databases. However, traditional methods require users to explicitly specify which data sources to access and which query methods to use, which is too cumbersome for ordinary users and makes it difficult to accurately express complex information needs.
[0004] Another key challenge is the lack of an intelligent query decomposition mechanism specifically for the water resources sector. Users typically submit complex queries in natural language, but existing systems cannot automatically decompose them into subqueries targeting different data sources, nor can they select appropriate retrieval channels and access strategies for each subquery. Traditional rule-based or template-based methods struggle to cover diverse query scenarios, have high maintenance costs, and poor scalability, failing to meet the urgent needs of water resources operations for intelligent knowledge services.
[0005] Furthermore, despite the breakthroughs achieved by Large Language Models (LLMs) in the field of natural language processing in recent years, their direct application in the water conservancy field still faces significant obstacles. Existing general-purpose large models cannot perceive the differences in heterogeneous storage structures in the water conservancy field and lack the planning ability to handle long-chain business logic. Those skilled in the art find it difficult to generate execution plans containing complex dependencies through simple prompt word engineering, which easily leads to retrieval path errors or logical inversions. Therefore, designing a mechanism that enables large models to both understand natural language intent and accurately plan cross-source retrieval paths like a database optimizer is a key technical challenge that this application urgently needs to address. Summary of the Invention
[0006] Purpose of the invention: To overcome the shortcomings of the prior art, this invention provides a water conservancy knowledge differential retrieval and query decomposition method based on a large language model. By fine-tuning the large language model, it enables it to intelligently decompose user natural language queries into multiple sub-queries, automatically route them to the corresponding retrieval channels, and plan the execution order, thereby providing an accurate query execution scheme for multi-source heterogeneous data retrieval in the water conservancy field.
[0007] Technical solution: The water conservancy knowledge differential retrieval and query decomposition method based on a large language model described in this invention includes the following steps:
[0008] (1) For multi-source heterogeneous data in the water conservancy field, retrieval channels are constructed respectively. The retrieval channels include retrieval channels for graph databases, retrieval channels for relational databases, retrieval channels for document databases and retrieval channels for time series databases, and standardized interface specifications are defined for each channel.
[0009] (2) Define a classification system for subquery types in the field of water conservancy, establish standard templates for each type of subquery, and form a subquery template library;
[0010] (3) Collect training data from historical query records and expert-constructed samples, and have domain experts label the data according to the subquery type classification system to form a labeled dataset by mapping the query decomposition results and channels.
[0011] (4) Design a composite loss function for query decomposition characteristics in the water conservancy field, fine-tune the pre-trained large language model based on the training dataset, force the model to pay attention to the storage characteristics and logical order of heterogeneous data during the training process, so that it has the ability of query decomposition, channel routing and execution planning, and outputs query execution plan.
[0012] Furthermore, the implementation process of step (1) is as follows:
[0013] (11) Access the spatial topology data of water conservancy facilities, use water conservancy engineering entities as graph nodes, use the engineering connection relationship between entities as graph edges, construct the facility topology relationship graph, form a retrieval channel for graph database, support input of entity identifiers and output of related entity sets and relationship paths;
[0014] (12) Access structured attribute data of water conservancy projects, store engineering parameter information of various water conservancy entities, and at the same time build a terminology dictionary in the field of water conservancy to establish a mapping relationship between user natural language expression and standard attribute field name, forming a retrieval channel for relational databases, supporting input of entity identifier and attribute name and output of corresponding attribute value;
[0015] (13) For unstructured documents including scheduling procedures and emergency plans, establish a sparse index based on keywords and a dense index based on semantic vectors, and define an extraction template for "condition-action-condition" rule triples to form a document-type channel that supports inputting query text and outputting relevant document fragments and extracted rule triples.
[0016] (14) Connect to the real-time data interface of the hydrological monitoring system, construct a mapping table between monitoring station identifiers and API request parameters, and form a time-series channel. This channel supports inputting station identifiers and time ranges and outputting monitoring data sequences for the corresponding time periods.
[0017] (15) Define a unified response format specification for the above four types of channels. The output of each channel includes data value, data source identifier, timestamp, data quality mark and confidence initial value field to support subsequent multi-source fusion processing.
[0018] Furthermore, the implementation process of step (2) is as follows:
[0019] (21) Define four basic subquery types based on the four retrieval channels;
[0020] (22) Write a standard template for each type of subquery. The template includes the functional description and typical application scenarios of the query type, the required input parameters and their data types, the optional input parameters and their default values, the data structure of the output results, and annotation examples covering simple and complex cases.
[0021] (23) Based on the single subquery type, define two types of dependency relationships between subqueries: parallel independent relationship where multiple subqueries can be executed simultaneously, and preorder dependency relationship where subsequent subqueries need to use the output of the preceding order as input;
[0022] (24) Based on the above subquery types and dependencies, common composite query patterns in the water conservancy field are summarized; the composite query patterns reflect the domain characteristics of water conservancy scheduling decisions, and a standard decomposition scheme from composite queries to multiple subqueries is predefined to form a composite query decomposition rule base, which serves as a reference for large language models to learn query decomposition.
[0023] Furthermore, the four basic sub-query types are as follows: topological relationship query for graph database retrieval channels, used to query the spatial connection relationships between water conservancy entities; attribute parameter query for relational database retrieval channels, used to query the static attribute parameters of water conservancy entities; rule condition query for document library retrieval channels, used to query the rules and contingency plans in the scheduling procedures; and real-time status query for time series database retrieval channels, used to query the current monitoring data. Each type of sub-query corresponds to one or more retrieval channels.
[0024] Furthermore, the implementation process of step (3) is as follows:
[0025] (31) Extract user query records from the historical query logs of the water conservancy business system, and organize domain experts to construct typical query samples based on actual business scenarios to form the original training sample set;
[0026] (32) The existing samples are expanded by means of synonym rewriting, parameter replacement and progressive complexity to form an expanded training sample set;
[0027] (33) Establish annotation specifications and use the formal tuple method to describe training samples; define each query sample as a binary tuple. ,in For users' raw natural language queries, For the corresponding structured execution plan; execution plan It is further formalized as a triple:
[0028] ;
[0029] (34) A multi-round annotation mechanism is adopted to ensure data quality. First, the annotation personnel perform the initial annotation. Then, the annotation consistency is calculated through cross-validation. For samples with low consistency, a discussion is organized to unify the standards. Finally, senior domain experts review complex query samples and correct unreasonable decomposition schemes.
[0030] (35) Divide the labeled dataset into training set, validation set and test set according to a certain ratio; adopt a stratified sampling strategy in the process of dividing the dataset to ensure that the sample distribution of different query types and complexities in the three datasets is consistent, and avoid training the model to be biased towards specific query patterns.
[0031] Furthermore, the synonym rewriting is a semantically equivalent transformation of the original query; the parameter replacement is to replace parameters in the query that include entity names and attribute names; and the complexity progression is to gradually increase from simple queries to complex queries.
[0032] Furthermore, the annotation specifications in step (33) are to clarify the content that the annotator needs to annotate for each query sample, including the list of subqueries obtained by decomposing the original query and their natural language descriptions, the query type to which each subquery belongs, the target channel and priority that each subquery should access, the dependencies between subqueries and the suggested execution order; for queries that are semantically ambiguous or exceed the system's capabilities, they are marked as needing clarification and explanation of the reasons.
[0033] Furthermore, the implementation process of step (4) is as follows:
[0034] (41) The query planning task is formalized as a sequence-to-sequence generation task. The input sequence is composed of system prompts, a summary of the subquery template library and the user's original query. The output sequence is a query execution plan represented in a predefined JSON format, which includes a list of subqueries, channel mapping, execution order and confidence score.
[0035] (42) According to the input and output definitions, the training set obtained in step (35) is converted into the format required for model training. The input part of each training sample includes task description, subquery type definition, decomposition example and query to be decomposed, and the output part is the standard format query execution plan corresponding to the query.
[0036] (43) Select a pre-trained large language model as the base model and adopt the low-rank fitting parameter efficient fine-tuning method. Insert the low-rank decomposed fitting matrix into the self-attention layer of the model. During training, only update the fitting matrix parameters while keeping the original model parameters frozen, so as to achieve efficient fine-tuning under limited training data.
[0037] (44) Design a composite loss function for query decomposition characteristics in the water conservancy field:
[0038] ;
[0039] ;
[0040] in, The basic sequence generation loss uses the standard cross-entropy loss function to ensure the fluency and accuracy of the generated subquery text description; For input query, For the generated first One token, The sequence length; To improve channel selection loss, a weight penalty is added to the key tokens in the output sequence that involve "retrieval channel identifiers" to ensure that the model routes attribute queries to relational databases rather than time series databases. This is a dependency consistency penalty used to punish logical loops or time-reversed situations in the generated execution plan, ensuring that the generated subquery dependency graph satisfies the directed acyclic graph constraint. , , These are the weighting coefficients for each loss term. This represents the total value of the composite loss function;
[0041] (45) During the training process, the model performance is evaluated on the validation set periodically. Evaluation metrics include the completeness and accuracy of subquery decomposition, the accuracy and recall of channel selection, the accuracy of dependency prediction, and the success rate of the generated query plan in actual execution. The optimal model checkpoint is selected based on the overall performance of the validation set for subsequent deployment.
[0042] Beneficial effects: Compared with the prior art, the beneficial effects of the present invention are as follows:
[0043] 1. Improved retrieval coverage and accuracy of multi-source heterogeneous data: This invention breaks the silo effect of four types of heterogeneous data (graph, text, table, and data) by constructing four standardized channels and sub-query systems, realizing unified semantic alignment and differentiated retrieval of cross-modal data, significantly avoiding the limitations of a single retrieval method, and improving the retrieval coverage and precision of multi-source knowledge in the water conservancy field.
[0044] 2. Intelligent logical decomposition of complex scheduling intentions is realized: structured annotation and fine-tuning training endow large models with the ability to logically decompose and plan dependencies for complex scheduling intentions. The model can transform fuzzy business instructions into executable subquery sequences containing parallel or serial dependencies, effectively solving the problem that complex queries cannot be executed directly.
[0045] 3. Enhanced system interpretability and decision reliability: Unlike the end-to-end black-box model, the channel selection and dependency constraint training of this invention generate query plans with clear execution paths and data sources, which enhances the interpretability of the system process and provides fully traceable data support for high-risk decision-making scenarios such as flood control and disaster relief.
[0046] 4. Reduced deployment cost and training threshold for large-scale domain models: This invention adopts low-rank adaptation (LoRA) fine-tuning technology combined with sample expansion strategy, which greatly reduces the computing cost and data threshold for model training. Under the condition of few samples, the general model can be quickly transferred to water conservancy experts, which is convenient for promotion and deployment in grassroots water conservancy departments. Attached Figure Description
[0047] Figure 1 This is a flowchart of the present invention;
[0048] Figure 2 This is a flowchart of the process of the present invention;
[0049] Figure 3 A schematic diagram of watershed topology modeling for a directed graph G=(V,E);
[0050] Figure 4 Generate a flowchart for the query plan of the large model. Detailed Implementation
[0051] The present invention will now be described in further detail with reference to the accompanying drawings.
[0052] like Figure 1 , Figure 2 As shown, this invention proposes a differentiated retrieval and query decomposition method for water conservancy knowledge based on a large language model, which specifically includes the following steps:
[0053] S1: Data in the water conservancy field is divided into four categories according to storage structure and query characteristics: graph data, relational data, document data, and time series data. An independent retrieval channel and a unified interface specification are established for each type of data.
[0054] S101: The spatial topological relationship data of water conservancy facilities is modeled using a directed graph G=(V,E). Taking the Xiangjiang River basin as an example, the node set V includes water conservancy project entities such as Lishimen Reservoir, Jiangkou Reservoir, and Shuangpai Reservoir, as well as nodes such as the main stream and tributary sections of the Xiangjiang River, control gates, and flood diversion facilities. The edge set E represents the connection relationships between facilities, including upstream and downstream relationships, control relationships, confluence relationships, and flood diversion relationships. For example, the edge (Lishimen Reservoir, downstream section) represents the downstream connection relationship, and the edge (Lishimen Reservoir, XX gate) represents the control relationship, such as... Figure 3 As shown. This channel supports inputting the starting entity and relationship type, and returns a set of related entities and their connection paths through a graph traversal algorithm.
[0055] S102: The structured attribute data of water conservancy projects will be organized and stored in the form of entity-attribute-value triples (e, a, v). For example, for the Lishimen Reservoir, standardized data records will be established such as triples (Lishimen Reservoir, flood limit level, 178.5m), (Lishimen Reservoir, total capacity, 496 million m³), (Lishimen Reservoir, design flow, 3000 m³ / s), (Lishimen Reservoir, dam height, 28.5m), and (Lishimen Reservoir, catchment area, 1560 km²). To enable the mapping of standard field names to natural language queries, a terminology dictionary in the field of water conservancy will be constructed, establishing the correspondence between natural language terms such as "flood limit level" and "flood_limit_level", "total capacity" and "total_capacity", and "design flow" and "design_flow" and database field names.
[0056] S103: For dispatching regulations and emergency plans such as the "Lishimen Reservoir Flood Control Dispatch Regulations" and the "Xiangjiang River Basin Reservoir Group Dispatch Rules," the original natural language text format is preserved, and the documents are segmented according to chapters and paragraphs. Each document segment maintains semantic integrity while recording metadata such as its document and chapter position. Two retrieval mechanisms are established: a sparse index based on keywords and a dense index based on semantic vectors. An extraction template for "condition-action-condition" rule triples is defined to identify and extract the structured representation of dispatching rules from the documents.
[0057] S104: Connects to the real-time data interface of the hydrological monitoring system to obtain time-series data such as water level, flow rate, and rainfall from various monitoring stations. Establishes a mapping table between station identifiers and query parameters. This channel supports inputting station names, monitoring indicators, and time ranges, and returns the monitoring data sequence for the specified time period.
[0058] S105: To ensure consistency in subsequent multi-source fusion processing, a unified response format specification is defined for the above four types of retrieval channels. The output of each channel includes fields such as data content, data source identifier, timestamp, data quality marker, and initial confidence value.
[0059] S2: Define a classification system for sub-query types for water conservancy scheduling operations, write standardized templates for each query type, and summarize the composite query decomposition patterns for typical business scenarios.
[0060] S201: Based on the four types of retrieval channels, four basic sub-query types are defined. The first type is topological relationship query, used to query the spatial connection relationships between water conservancy entities, corresponding to the graph database retrieval channel; the second type is attribute parameter query, used to query the static attribute parameters of water conservancy entities, corresponding to the relational database retrieval channel; the third type is rule condition query, used to query the rules and plans in the scheduling procedures, corresponding to the document library retrieval channel; and the fourth type is real-time status query, used to query the current monitoring data, corresponding to the time series database retrieval channel.
[0061] S202: Develop a detailed, standardized template document for each type of subquery. The template should include a functional description, input parameter specifications, output result specifications, and annotation examples.
[0062] S203: Based on a single subquery, define two types of dependency relationships between subqueries. The first is a parallel independence relationship, which means that multiple subqueries have no data dependency and can be executed simultaneously; the second is a priori dependency relationship, which means that subsequent subqueries need to use the output of the preceding subqueries as input parameters and must be executed in sequence.
[0063] S204: Based on the domain characteristics of water conservancy dispatching decisions, common composite query patterns are summarized. For example, the standard decomposition scheme for the flood control dispatching query "How should the Lishimen Reservoir be dispatched when the water level reaches 180 meters?" is as follows: Subquery 1 queries key water level parameters such as the reservoir's flood control limit water level and high flood control water level (attribute query); Subquery 2 queries the dispatching rules corresponding to a water level of 180 meters (rule query); Subquery 3 queries the facilities downstream of the reservoir that are affected (topology query); Subquery 4 queries the current water level and inflow (time-series query). These four subqueries are parallel and independent. These predefined decomposition patterns serve as reference knowledge for large language models to learn query decomposition.
[0064] S3: Collect data from historical query logs and expert-constructed samples, expand the sample size through various strategies, and have domain experts perform multiple rounds of annotation to form a high-quality supervised training dataset.
[0065] S301: Extract user query records from the historical query logs of the water conservancy business system, covering actual business scenarios such as flood control scheduling, reservoir joint operation, and water conservancy scheduling. At the same time, organize domain experts to construct typical query samples based on actual business needs to form the original training sample set.
[0066] S302: The existing samples are expanded using three strategies: synonym rewriting, parameter replacement, and progressive complexity. Synonym rewriting transforms the original query into a semantically equivalent representation; parameter replacement replaces parameters such as entity names and attribute names in the query; progressive complexity increases the complexity from simple queries to complex queries, forming an expanded training sample set.
[0067] S303: Develop detailed annotation specifications and use a formalized tuple method to describe training samples. Define each query sample as a binary tuple. ,in For users' raw natural language queries, This corresponds to a structured execution plan. Specifically, the execution plan... It is further formalized as a triple:
[0068]
[0069] Taking the question "How should the water level of Lishimen Reservoir be managed when it reaches 180 meters?" as an example, the mathematical expression of its content is as follows:
[0070] 1. Subquery set
[0071] 1) (Attribute Acquisition): ;
[0072] 2) (Rule-based retrieval): ;
[0073] 3) (Topology Analysis): ;
[0074] 4) (Time Series Monitoring): .
[0075] 2. Channel mapping relationship Define mapping function ,in .
[0076] 1) (Relational database channel);
[0077] 2) (Document library channel);
[0078] 3) (Graphic database channel);
[0079] 4) (Time series database channel).
[0080] 3. Dependency Graph In this example, there are no preceding data dependencies between the subqueries, and the edge set... Marked as parallel independent mode, meaning the system can execute concurrently. This significantly reduces response latency.
[0081] S304: A multi-round annotation mechanism is adopted to ensure data quality. First, the annotation personnel perform the initial annotation. Then, the annotation consistency is calculated through cross-validation. For samples with low consistency, a discussion is organized to unify the standards. Finally, senior domain experts review complex query samples and correct unreasonable decomposition schemes.
[0082] S305: Divide the labeled samples into training, validation, and test sets according to a certain ratio. A stratified sampling strategy is used during the partitioning process to ensure that the distribution of samples with different query types and complexities remains consistent across the three datasets, preventing the training model from being biased towards specific query patterns.
[0083] S4: Design a composite loss function for query decomposition characteristics in the water conservancy field. Based on the training dataset, fine-tune the pre-trained large language model, forcing the model to pay attention to the storage characteristics and logical order of heterogeneous data during training, so that it has the ability of query decomposition, channel routing and execution planning, and outputs a query execution plan.
[0084] S401: Define the query planning task as a sequence-to-sequence generation task. The input sequence consists of three parts: system role hints (explaining the model's role and task objectives as a query planning expert in the water resources field), a concise summary of the subquery template library (containing functional descriptions and parameter specifications for four types of subqueries), and the user's original natural language query. The output sequence is a query execution plan represented in JSON format, which includes a list of subqueries, channel mappings, and execution order.
[0085] S402: Convert the labeled dataset into the format required for model training. Each training sample contains an input part and an output part. The input part concatenates the task description, subquery type definition, and query to be decomposed. The output part is the standard format query execution plan corresponding to that query, i.e., the JSON format shown in S303, such as... Figure 4 As shown, all data is uniformly encoded into a token sequence that the model can process, and an appropriate maximum length limit is set.
[0086] S403: A pre-trained large language model is selected as the base model, and a low-rank adaptation (LoRA) parameter-efficient fine-tuning method is adopted. Low-rank decomposed adaptation matrices are inserted into the model's self-attention layer. During training, only the parameters of these adaptation matrices are updated, keeping the original pre-trained model parameters frozen. This approach significantly reduces the number of trainable parameters, achieving efficient fine-tuning with limited training data.
[0087] S404: During training, a language model loss function is used as the basic optimization objective. Simultaneously, the model's domain-specific performance is periodically evaluated on the validation set, including metrics such as subquery decomposition completeness, query type identification accuracy, channel selection accuracy, and dependency prediction accuracy. These evaluation metrics monitor the training process to ensure the model accurately understands query intent within the water resources domain.
[0088] Design a composite loss function for querying decomposition characteristics in the water conservancy field:
[0089] ;
[0090] ;
[0091] in, The basic sequence generation loss uses the standard cross-entropy loss function to ensure the fluency and accuracy of the generated subquery text description; For input query, For the generated first One token, The sequence length; To improve channel selection loss, a weight penalty is added to the key tokens in the output sequence that involve "retrieval channel identifiers" to ensure that the model routes attribute queries to relational databases rather than time series databases. This is a dependency consistency penalty used to punish logical loops or time-reversed situations in the generated execution plan, ensuring that the generated subquery dependency graph satisfies the directed acyclic graph constraint. , , These are the weighting coefficients for each loss term. This represents the total value of the composite loss function.
[0092] This invention does not employ a general text generation loss function, but instead designs a composite loss function oriented towards query decomposition characteristics. This is achieved by introducing channel selection loss (…). ) and dependency consistency loss ( This forces the model to pay attention to the storage characteristics and logical order of heterogeneous data during training. This fine-tuning strategy, which internalizes the data storage structure and business logic dependencies into the model parameters, ensures that the model outputs not just text, but a rigorous execution plan with graph structure attributes.
[0093] S405: Deploy the trained, fine-tuned model in the inference service to perform query decomposition for actual water conservancy business scenarios. Taking the input query "Predict the inflow of each reservoir in the Xiangjiang River basin in the next 24 hours and provide joint scheduling suggestions for reservoirs exceeding flood limits" as an example, the model inference process is as follows:
[0094] Intent parsing: The model first identifies that the query contains three layers of intent: time series prediction, attribute judgment, and rule retrieval.
[0095] Plan generation: The model outputs a structured execution plan, first generating parallel subqueries. (Get the list of reservoirs in the basin, pointing to the graph channel) and (Obtain weather forecasts, pointing to time-series channels); subsequently generate dependent on Subqueries of the results (Query flood control water level, pointing to the relationship channel); finally generate the dependency Subqueries of the results (Retrieve scheduling scheme, pointing to document channel).
[0096] Output: Based on the plan, the system automatically routes to the corresponding channels to obtain data, and finally synthesizes decision recommendations that include the data sources. The above reasoning process verifies that the model has the ability to handle complex dependencies.
[0097] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.
Claims
1. A method for differentiated retrieval and query decomposition of water conservancy knowledge based on a large language model, characterized in that, Includes the following steps: (1) For multi-source heterogeneous data in the water conservancy field, retrieval channels are constructed respectively. The retrieval channels include retrieval channels for graph databases, retrieval channels for relational databases, retrieval channels for document databases and retrieval channels for time series databases, and standardized interface specifications are defined for each channel. (2) Define a classification system for subquery types in the field of water conservancy, establish standard templates for each type of subquery, and form a subquery template library; (3) Collect training data from historical query records and expert-constructed samples, and have domain experts label the data according to the subquery type classification system to form a labeled dataset by mapping the query decomposition results and channels. (4) Design a composite loss function for query decomposition characteristics in the water conservancy field, fine-tune the pre-trained large language model based on the training dataset, force the model to pay attention to the storage characteristics and logical order of heterogeneous data during the training process, so that it has the ability of query decomposition, channel routing and execution planning, and outputs query execution plan.
2. The method for differentiated retrieval and query decomposition of water conservancy knowledge based on a large language model according to claim 1, characterized in that, The implementation process of step (1) is as follows: (11) Access the spatial topology data of water conservancy facilities, use water conservancy engineering entities as graph nodes, use the engineering connection relationship between entities as graph edges, construct the facility topology relationship graph, form a retrieval channel for graph database, support input of entity identifiers and output of related entity sets and relationship paths; (12) Access structured attribute data of water conservancy projects, store engineering parameter information of various water conservancy entities, and at the same time build a terminology dictionary in the field of water conservancy to establish a mapping relationship between user natural language expression and standard attribute field name, forming a retrieval channel for relational databases, supporting input of entity identifier and attribute name and output of corresponding attribute value; (13) For unstructured documents including scheduling procedures and emergency plans, establish a sparse index based on keywords and a dense index based on semantic vectors, and define an extraction template for "condition-action-condition" rule triples to form a document channel that supports inputting query text and outputting relevant document fragments and extracted rule triples. (14) Connect to the real-time data interface of the hydrological monitoring system, construct a mapping table between monitoring station identifiers and API request parameters, and form a time-series channel. This channel supports inputting station identifiers and time ranges and outputting monitoring data sequences for the corresponding time periods. (15) Define a unified response format specification for the above four types of channels. The output of each channel includes data value, data source identifier, timestamp, data quality mark and confidence initial value field to support subsequent multi-source fusion processing.
3. The method for differentiated retrieval and query decomposition of water conservancy knowledge based on a large language model according to claim 1, characterized in that, The implementation process of step (2) is as follows: (21) Define four basic subquery types based on the four retrieval channels; (22) Write a standard template for each type of subquery. The template includes the functional description and typical application scenarios of the query type, the required input parameters and their data types, the optional input parameters and their default values, the data structure of the output results, and annotation examples covering simple and complex cases. (23) Based on the single subquery type, define two types of dependency relationships between subqueries: parallel independent relationship where multiple subqueries can be executed simultaneously, and preorder dependency relationship where subsequent subqueries need to use the output of the preceding order as input; (24) Based on the above subquery types and dependencies, common composite query patterns in the water conservancy field are summarized; the composite query patterns reflect the domain characteristics of water conservancy scheduling decisions, and a standard decomposition scheme from composite queries to multiple subqueries is predefined to form a composite query decomposition rule base, which serves as a reference for large language models to learn query decomposition.
4. The method for differentiated retrieval and query decomposition of water conservancy knowledge based on a large language model according to claim 3, characterized in that, The four basic sub-query types are as follows: topological relationship query for graph database retrieval channels, used to query the spatial connection relationship between water conservancy entities; attribute parameter query for relational database retrieval channels, used to query the static attribute parameters of water conservancy entities; rule condition query for document library retrieval channels, used to query the rules and contingency plans in the scheduling procedures; and real-time status query for time series database retrieval channels, used to query the current monitoring data. Each type of sub-query corresponds to one or more retrieval channels.
5. The method for differentiated retrieval and query decomposition of water conservancy knowledge based on a large language model according to claim 1, characterized in that, The implementation process of step (3) is as follows: (31) Extract user query records from the historical query logs of the water conservancy business system, and organize domain experts to construct typical query samples based on actual business scenarios to form the original training sample set; (32) The existing samples are expanded by means of synonym rewriting, parameter replacement and progressive complexity to form an expanded training sample set; (33) Establish annotation specifications and use the formal tuple method to describe training samples; define each query sample as a binary tuple. ,in For users' raw natural language queries, For the corresponding structured execution plan; execution plan It is further formalized as a triple: ； (34) A multi-round annotation mechanism is adopted to ensure data quality. First, the annotation personnel perform the initial annotation. Then, the annotation consistency is calculated through cross-validation. For samples with low consistency, a discussion is organized to unify the standards. Finally, senior domain experts review complex query samples and correct unreasonable decomposition schemes. (35) Divide the labeled dataset into training set, validation set and test set according to a certain ratio; adopt a stratified sampling strategy in the process of dividing the dataset to ensure that the sample distribution of different query types and complexities in the three datasets is consistent, and avoid training the model to be biased towards specific query patterns.
6. The method for differentiated retrieval and query decomposition of water conservancy knowledge based on a large language model according to claim 5, characterized in that, The synonym rewriting is a semantically equivalent representation transformation of the original query; The parameter replacement refers to replacing parameters in the query, including entity names and attribute names; the progressive complexity refers to gradually increasing from simple queries to complex queries.
7. The method for differentiated retrieval and query decomposition of water conservancy knowledge based on a large language model according to claim 5, characterized in that, The annotation specifications in step (33) are to clarify the content that the annotator needs to annotate for each query sample, including the list of subqueries obtained by decomposing the original query and their natural language descriptions, the query type to which each subquery belongs, the target channel and priority that each subquery should access, the dependencies between subqueries and the suggested execution order; for queries that are semantically ambiguous or beyond the system's capabilities, they are marked as needing clarification and the reasons explained.
8. The method for differentiated retrieval and query decomposition of water conservancy knowledge based on a large language model according to claim 1, characterized in that, The implementation process of step (4) is as follows: (41) The query planning task is formalized as a sequence-to-sequence generation task. The input sequence is composed of system prompts, a summary of the subquery template library and the user's original query. The output sequence is a query execution plan represented in a predefined JSON format, which includes a list of subqueries, channel mapping, execution order and confidence score. (42) According to the input and output definitions, the training set obtained in step (35) is converted into the format required for model training. The input part of each training sample includes task description, subquery type definition, decomposition example and query to be decomposed, and the output part is the standard format query execution plan corresponding to the query. (43) Select a pre-trained large language model as the base model and adopt the low-rank fitting parameter efficient fine-tuning method. Insert the low-rank decomposed fitting matrix into the self-attention layer of the model. During training, only update the fitting matrix parameters while keeping the original model parameters frozen, so as to achieve efficient fine-tuning under limited training data. (44) Design a composite loss function for query decomposition characteristics in the water conservancy field: ； ； in, The basic sequence generation loss uses the standard cross-entropy loss function to ensure the fluency and accuracy of the generated subquery text description; For input query, For the generated first One token, The sequence length; For channel selection loss, a weight penalty is added to the key tokens in the output sequence that involve "retrieval channel identifier" to ensure that the model routes attribute queries to relational databases rather than time series databases; This is a dependency consistency penalty used to punish logical loops or time-reversed situations in the generated execution plan, ensuring that the generated subquery dependency graph satisfies the directed acyclic graph constraint. , , These are the weighting coefficients for each loss term. This represents the total value of the composite loss function; (45) During the training process, the model performance is evaluated on the validation set periodically. Evaluation metrics include the completeness and accuracy of subquery decomposition, the accuracy and recall of channel selection, the accuracy of dependency prediction, and the success rate of the generated query plan in actual execution. The optimal model checkpoint is selected based on the overall performance of the validation set for subsequent deployment.