A question-answering recommendation method and system for flood control dispatching rules based on large language model
By building the flood control scheduling rule map and large language model, the problem of cumbersome scheduling rules review in water conservancy flood control scheduling is solved, and the rapid and accurate rule recommendation is achieved, the efficiency of dispatchers and the scientific decision-making is improved, and the large language model is filled. Application gaps in the field of flood control scheduling.
Patent Information
- Application Number
- CN202410157178.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-02-04
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-02-04
AI Technical Summary
During the water conservancy flood control scheduling process, it is cumbersome to check scheduling rules documents, inefficient efficiency, and lack of application of large language models in scheduling rules recommendations.
Build a flood control scheduling rule map, use the natural language processing capabilities of large language models, quickly and accurately provide scheduling rule recommendations through problem preprocessing, intent analysis, graph query and recommendation generation methods, and fine-tune the training model to improve accuracy.
It greatly improves the work efficiency of dispatchers, ensures the scientificity and accuracy of dispatch decisions, solves the problem of low information retrieval efficiency and accuracy, avoids the 'illusion' problem of large language models in the application of fields, and has high practical value. and commercial value.
Smart Images

Figure CN118069797B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of natural language processing, and in particular to a method and system for recommending question and answer of flood control dispatching rules based on a large language model. Background Art
[0002] In the current flood control dispatching process of water conservancy objects, dispatchers need to consult the official documents of different water conservancy objects to obtain the dispatching rules of different water conservancy objects, so as to carry out dispatching work according to the dispatching rules. The process of consulting the dispatching rule documents and finding the appropriate dispatching rules in combination with the current water level of the water conservancy object, weather forecast, etc. is very cumbersome.
[0003] At present, with the development of large language model technology, large language models have performed well in text processing, especially in understanding natural language, and can accurately obtain content that meets the input prompt requirements in longer texts; at the same time, large language models also have certain reasoning and decision-making capabilities. After special training for vertical fields, they can interact with knowledge bases better and use the domain knowledge recalled from the knowledge base to strengthen and supplement the defects of the model itself. However, large language models have not yet been well combined with the water conservancy field, and there is no precedent for applying the capabilities of large language models to the recommendation of dispatching rules. Summary of the invention
[0004] Purpose of the invention: In order to solve the above problems, the present invention provides a method and system for question-and-answer recommendation of flood control scheduling rules based on a large language model, aiming to solve the problem of cumbersome and inefficient query of scheduling rules in the existing water conservancy flood control scheduling process; by constructing a flood control scheduling rule map and utilizing the powerful natural language processing capabilities of a large language model, fast and accurate question-and-answer recommendation of flood control scheduling rules is achieved, thereby improving the work efficiency of dispatchers and ensuring the scientificity and accuracy of scheduling decisions.
[0005] Technical solution: The method for recommending flood control dispatching rules based on a large language model includes the following steps:
[0006] Extract and collect flood control dispatch rule information from the official dispatch rule documents, and build a flood control dispatch rule map to provide external recallable data for the large language model;
[0007] (2) Filter user questions based on question preprocessing methods to prevent the large language model from answering questions other than scheduling rules;
[0008] (3) Use intent analysis methods to analyze user questions and classify user questions according to intent;
[0009] (4) Use the graph query method to use different model weights according to the intent classification results to generate the corresponding flood control dispatch Neo4j graph-specific CQL query statement for data recall query;
[0010] (5) Using the recommendation generation method to generate rule-containing recommendation answers for the user based on the scheduling information recalled from the graph;
[0011] (6) Fine-tune the training of the large language model to enhance the accuracy of problem solving in each link of the flood control scheduling rule recommendation process and improve the quality of question answering.
[0012] Furthermore, the flood control and dispatching rule map includes water conservancy object nodes, dispatching rule nodes and dispatching rule document tree nodes; wherein the water conservancy object node represents the relevant water conservancy object that supports users to ask questions about dispatching rules; the dispatching rule node is constructed around the water conservancy object node, and is associated with the water conservancy object node, and stores the dispatching rules of the corresponding water conservancy object in the form of natural language text; the document tree node splits the original document into nodes that store content in a tree structure according to chapter and title levels, and contains relevant knowledge of flood control and dispatching such as compilation information, execution authority, organizational guarantees and discharge relationship comparison tables.
[0013] Furthermore, the step (2) comprises the following steps:
[0014] (21) Use the large language model to extract the scheduling target object in the user input task, O = LLM (Prompt OE ,question); Among them, LLM is a general large language model without fine-tuning, Prompt OE is the prompt word used to extract the scheduling target object, question is the task string input by the user, and O is the scheduling target object extraction result;
[0015] (22) Use the large language model to rewrite the multiple expressions of the scheduling target object extracted in the generation step (21) to obtain a possible name set O' of the extracted target object i , where O' i represents the i-th possible name of O;
[0016] (23) According to the possible name set O' of the scheduling target object i The Python script synthesizes CQL query statements for the basic object graph of the watershed, and performs query operations through the py2neo query tool;
[0017] (24) Check the query results. If the scheduling target object exists in the watershed basic object map, and the user task is a scheduling rule recommendation task, the user question is placed in the processing waiting queue of the intent analysis; otherwise, the user is prompted that the scheduling target object does not exist in the watershed basic object map and the recommendation system process is terminated.
[0018] Furthermore, the step (3) comprises the following steps:
[0019] (31) Combined with flood control dispatch recommendation problem set Q i , the problems are divided into three categories: "basic knowledge problems of flood control scheduling", "recommendation problems with specified basin objects and conditions", and "recommendation problems with specified basin objects but no conditions". i When the three categories are represented, "A", "B" and "C" are used respectively, and the flood control dispatch target objects in each problem are marked by manual labeling. i , Flood control dispatch conditions R i And problem classification CL i , get the basis (question, classification) pair: (T i ,CL i ), where T i =Q i -O i -R i , a problem template for removing the scheduling target object and flood control scheduling conditions;
[0020] (32) Using the rewriting ability of the large language model, the large language model is prompted by context to remove the scheduling target object and flood control scheduling conditions from the question template T i The word order and synonyms are changed while maintaining the original meaning, and the corresponding question classification remains unchanged. n synonym templates are generated, and the result is:
[0021] T' x ={T' i1 ,T' i2 ,T' i3 …,T' in} i =LLM(Prompt CH ,T i )
[0022] Among them, LLM is a general large language model without fine-tuning, Prompt CH It is a prompt word for word order change and synonym replacement. i is a problem template for removing the scheduling target object and flood control scheduling conditions, T' i1 ,T' i2 ,T' i3 …,T'in are the generated n synonymous templates, T' x is the final expanded data set template, from which the expanded (question, classification) pair (T' x ,CL x );
[0023] (33) Randomly extract flood control dispatch object O" from the basin basic object library as the flood control dispatch object sequence;
[0024] (34) Utilizing the generative capability of the large language model, through contextual prompts and the FewShot method, the manually annotated flood control dispatch condition features are learned to generate more flood control dispatch conditions that conform to the business logic of flood control dispatch, forming a flood control dispatch condition keyword sequence, specifically:
[0025] R' y = {R' i1 ,R' i2 ,R' i3 …,R' in} i =LLM(Prompt CHR ,S i1 ,S i2 ,S i3 ,T i )
[0026] Among them, Prompt CHR is a prompt word used to learn and generate flood control dispatch conditions, S i1 ,S i2 ,S i3 is the flood control dispatch condition R i Three examples of scheduling conditions randomly selected from ;
[0027] (35) The flood control dispatch object sequence O' and the flood control dispatch condition keyword sequence R' x and question template T' x Combine data sets to automatically generate a large-capacity, high-quality flood control scheduling intention analysis data set D1 = (T", CL), where; T" = O" × R' y ×T' x ;
[0028] (36) Construct a dialogue history for interaction with the large model. In the first dialogue with the role "system", insert the prompt word "Prompt" to guide the large language model to classify the intent. CL; After that, each time the intent of a user question is classified, three "classification question"-"classification result" pairs are randomly extracted from the data set D1 and placed in the "User"-"Assistant" pairs as the three shots answered by the large model, and the user question is placed in a new "User" at the end of the history; the conversation history is sent to the large language model, and the large language model will give the classification result after reasoning.
[0029] Furthermore, the step (4) comprises the following steps:
[0030] (41) Using the dataset template (T i ,CL i ), write a small number of CQL statement templates CP i ; Use the large language model to expand the template and expand the problem set to CP' x ; When combining data sets, the keyword sequence is combined with the CQL statement template to generate CQL to generate training data set D2 = (T", CP, CL), where CP = O" × R' y ×CP' x ;
[0031] (42) Construct a dialogue history for interacting with the large model. In the first dialogue with the role "system", insert the prompt word Prompt to guide the large language model to perform the CQL generation task. CQL ; After that, each time the intent of a user question is classified, three "question"-"question corresponding CQL statement" pairs are randomly extracted from the constructed D2 dataset and put into the "User"-"Assistant" question-answer pair as the three shots answered by the large model, and the user question is put into a new "User" at the end of the history; the conversation history is sent to the large language model, and the large language model will give the CQL generation result after reasoning;
[0032] (43) After the large language model generates CQL statements, it calls the Neo4j graph database query tool to query the dispatch rule graph and retrieve the required flood control dispatch information.
[0033] Furthermore, the step (5) comprises the following steps:
[0034] (51) Using the conversation and understanding capabilities of the large language model, the context prompt specifies that the model should answer strictly based on the dispatch information recalled by the graph. According to the results of intent classification, FS is provided for three different types of questions: "basic common sense questions about flood control dispatch", "recommendation questions with specified basin objects and conditions", and "recommendation questions with specified basin objects but no conditions". A , FS B , FS CThree different presets FewShot let the model answer in a format;
[0035] (52) After the model generates an answer, the original question and answer are simultaneously re-input into the large language model, and the large language model is used to detect and filter the answer to check whether the answer conforms to the scheduling recommendation business logic. If it does not conform to the business logic, the Temperature parameter value of the large language model is increased and a new round of answers is generated until it conforms to the business logic. Finally, the answer is output to the user. If the number of cycles exceeds the specified round, the user is prompted that the answer task cannot be completed.
[0036] Furthermore, the implementation process of step (6) is as follows:
[0037] (61) Use the constructed flood control dispatch intention analysis dataset D1 to fine-tune the pre-trained large language model base to improve the reasoning weight of the large language model dedicated to flood control dispatch intention analysis;
[0038] (62) According to the different classifications in the data set D2, the data set is divided into three parts according to the value of CL, and the pre-trained large language model base is fine-tuned to obtain the inference weights of the large language model specifically for solving the CQL generation of the three types of user problems "A", "B", and "C", so that when the model is called with different weights later, the large language model can improve the ability to generate accurate CQL for different scheduling problems;
[0039] (63) Datasets D1 and D2 are merged to form a new dataset D3, and the pre-trained large language model base is fine-tuned to obtain new inference weights. This weight can simultaneously improve the intent analysis capability of the large language model and the CQL generation capability for the three types of user questions "A", "B", and "C".
[0040] Furthermore, the process of fine-tuning the pre-trained large language model base is as follows:
[0041] For training sets D1, D2, and D3, the training set D x The JSON is processed and converted into the form of historical dialogue, in which the prompt Prompt for guiding the large language model is placed in the first historical dialogue with the role "system"; x Three items are randomly selected from the corresponding question-answer pairs and put into the "User"-"Assistant" question-answer pair as the three shots of the question and the preset large model answer. x Put the training problem in a new "User" at the end of the history, and put D xThe expected answer of the large language model in the training set is put into the "Assistant" later; each question-answer pair in the training set has 1 prompt and 3 random shots; check each training JSON and delete the JSON with repeated shots and training questions.
[0042] The flood control dispatch rule question-answering recommendation system based on a large language model described in the present invention comprises a flood control dispatch rule graph construction module, a question preprocessing module, an intention analysis module, a graph query module, a recommendation generation module and a large model fine-tuning training module; wherein:
[0043] The flood control dispatch rule graph construction module extracts and collects flood control dispatch rule information from the official dispatch rule documents, and constructs a dispatch rule graph to provide external recall data for the large language model.
[0044] The question preprocessing module is used to filter user questions and prevent the large language model from answering questions outside the scheduling rules;
[0045] The intent analysis module classifies user questions according to their intent, and divides the questions into three categories: "basic common sense questions about flood control scheduling", "recommendation questions about specified watershed objects and conditions", and "recommendation questions about specified watershed objects but without conditions";
[0046] The graph query module is used to generate a Cypher statement for graph-related knowledge recall based on the type of question asked by the user and the specific content of the user's question. After executing the Cypher statement, the text in the node is obtained as the question recall content for the large language model to reason.
[0047] In the recommendation generation module, the large language model uses the relevant text information recalled in step 5 and combines it with the user's question to generate rule-based recommended answers for the user.
[0048] The large model fine-tuning training module uses fine-tuning training for small data models with weak performance to enhance the accuracy of problem handling in each link of the flood control scheduling rule recommendation process; the fine-tuning method can adopt full fine-tuning, lora fine-tuning and P-tuning fine-tuning.
[0049] Beneficial effects: Compared with the prior art, the present invention has the following beneficial effects: the present invention performs effective information retrieval on the knowledge graph by parsing questions and generates user-friendly natural language answers, which greatly improves the efficiency and accuracy of information retrieval; the present invention makes full use of the text parsing and generation capabilities of the large language model, so that users can easily and conveniently complete the query and recommendation work related to the dispatching rules, greatly improving the work efficiency of users; the present invention adopts the fine-tuning method of P-tuning for model tuning, which is simpler than re-pre-training or full fine-tuning, and the cost is greatly reduced; through fine-tuning training and designing special prompt words, the system can accurately identify the user's intention and recall relevant flood control dispatching information from the knowledge graph, thereby effectively solving the user's problem, which significantly improves the application ability of the large language model in the field of flood control dispatching; the system is constrained in content generation, avoiding the "hallucination" problem that may occur when the large language model is applied in the field, making the output of the system more accurate and reliable; the present invention not only improves the user's trust in the system and reduces the risk caused by erroneous information, but also fills the gap in the field of intelligent recommendation of flood control dispatching rules, and has extremely high practical value and commercial value. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] Figure 1 This is a schematic diagram of the flood control dispatch rule question-answering system based on the large language model;
[0051] Figure 2 It is a schematic diagram of the map construction process;
[0052] Figure 3 It is a schematic diagram of the process of constructing the intent analysis dataset;
[0053] Figure 4 It is a schematic diagram of the graph query process;
[0054] Figure 5 is a schematic diagram of the recommendation generation process;
[0055] Figure 6 It is a schematic diagram of the large model fine-tuning training process. DETAILED DESCRIPTION
[0056] The present invention will be further described in detail below in conjunction with the accompanying drawings.
[0057] The present invention provides a flood control dispatch rule question-answering recommendation method based on a large language model, comprising the following steps:
[0058] Step 1: Construct a flood control dispatch rule graph. Extract and collect flood control dispatch rule information from the official dispatch rule documents, and construct a dispatch rule graph to provide external recallable data for the large language model.
[0059] The flood control dispatch rule map includes water conservancy object nodes, dispatch rule nodes and dispatch rule document tree nodes.
[0060] Water conservancy object node: represents the relevant water conservancy objects that support users to ask questions about scheduling rules, mainly various reservoirs. There are upstream and downstream relationships between reservoir nodes, and the upstream and downstream relationships are connected.
[0061] Dispatching rule node: This node is built around the water conservancy object node and is associated with it. It stores the dispatching rules of the corresponding water conservancy object in the form of natural language text; the dispatching rule node structure needs to follow the flood control dispatching ontology, which includes key concepts in the fields of dispatching schemes, dispatching objects, triggering objects, dispatching objects, flood protection objects, restriction objects, and control methods; starting from the top-level concept, it is refined layer by layer to determine the hierarchy between key concepts and form a key concept classification system; the association relationship between key concepts is explored to form a concept association framework to build a complete flood control dispatching concept system.
[0062] Document tree node: The original document is split into tree structures according to chapter and title levels to store the content of the node, which contains relevant knowledge of flood control dispatch, such as compilation information, execution authority, organizational guarantee, discharge relationship comparison table, etc. Some documents that cannot extract the first two nodes can be organized in this form.
[0063] Step 2: If Figure 2 As shown, user questions are filtered based on the question preprocessing method to prevent the large language model from answering questions other than the scheduling rules.
[0064] Use the large language model to extract the scheduling target object in the user input task, O = LLM (Prompt OE ,question); Among them, LLM is a general large language model without fine-tuning, Prompt OE is the prompt word for extracting the scheduling target object, question is the task string input by the user, and O is the extraction result of the scheduling target object. Use the large language model to rewrite the various expressions of the extracted scheduling target object to obtain the possible name set O' of the extracted target object i , where O' i Represents the i-th possible name of O. According to the possible name set O' of the scheduling target object i The Python script synthesizes the CQL query statement for the basin basic object map, and performs the query operation through the py2neo query tool. Check the query results. If the scheduling target object exists in the basin basic object map, the user task is the scheduling rule recommendation task, and continue to the next step. Otherwise, the user is prompted that the scheduling target object does not exist in the basin basic object map and the recommendation system process is terminated.
[0065] Step 3: Use the intent analysis method to analyze user questions and classify user questions according to intent, such as Figure 3 As shown, the specific implementation process is as follows:
[0066] S31: First, based on the opinions of water conservancy experts, a set of flood control dispatch recommendation questions that may be raised in various practical scenarios is proposed. i , the problems are divided into three categories: "basic knowledge problems of flood control scheduling", "recommendation problems with specified basin objects and conditions", and "recommendation problems with specified basin objects but no conditions". i When the three categories are represented, "A", "B" and "C" are used respectively, and the flood control dispatch target objects in each problem are marked by manual labeling. i , Flood control dispatch conditions R i And problem classification CL i , get the basis (question, classification) pair: (T i ,CL i ), where T i =Q i -O i -R i , which is a problem template for removing the scheduling target object and flood control scheduling conditions.
[0067] S32: Using the rewriting ability of the large language model, the large language model is prompted by context to remove the scheduling target object and the flood control scheduling condition question template T i The word order and synonyms are changed while the original meaning is maintained, and the corresponding question classification remains unchanged. n synonym templates are generated, and the result is:
[0068] T' x ={T' i1 ,T' i2 ,T' i3 …,T' in} i =LLM(Prompt CH ,T i )
[0069] Among them, LLM is a general large language model without fine-tuning, Prompt CH It is a prompt word for word order change and synonym replacement. i is a problem template for removing the scheduling target object and flood control scheduling conditions, T' i1 ,T' i2 ,T' i3 …,T' in are the generated n synonymous templates, T' xis the final expanded data set template, from which the expanded (question, classification) pair (T' x ,CL x ).
[0070] S33: Randomly extract multiple flood control scheduling objects O" such as reservoirs, sections or measuring stations from the basin basic object library as a flood control scheduling object sequence.
[0071] S34: Using the generative capability of the large language model, through contextual prompts and the FewShot method, learn the manually annotated flood control dispatch condition features, generate more flood control dispatch conditions that meet the business logic of flood control dispatch, and form a flood control dispatch condition keyword sequence, specifically:
[0072] R' y = {R' i1 ,R' i2 ,R' i3 …,R' in} i =LLM(Prompt CHR ,S i1 ,S i2 ,S i3 ,T i );
[0073] Among them, Prompt CHR is a prompt word used to learn and generate flood control dispatch conditions, S i1 ,S i2 ,S i3 is the flood control dispatch condition R i Three examples of scheduling conditions randomly selected from .
[0074] S35: The flood control scheduling object sequence O' and the flood control scheduling condition keyword sequence R' x and question template T' x Combine data sets to automatically generate a large-capacity, high-quality flood control scheduling intention analysis data set D1 = (T", CL), where; T" = O" × R' y ×T' x .
[0075] S36: Construct the conversation history of interaction with the big model, in which the prompt word Prompt is placed in the first conversation of the history with the role "system" to guide the big language model to classify the intent. CL; After that, each time the intent of a user question is classified, three "classification question"-"classification result" pairs are randomly extracted from the data set D1 constructed in S45, and placed in the "User"-"Assistant" pairs as the three shots answered by the large model, and the user question is placed in a new "User" at the end of the history; the conversation history is sent to the large language model, and the large language model will give the classification result after reasoning.
[0076] Step 4: Use the graph query method to use different model weights according to the intent classification results to generate the corresponding flood control dispatch Neo4j graph-specific CQL query statement for data recall query, such as Figure 4 As shown, the implementation process is as follows:
[0077] S41: Using the dataset template constructed in S31 (T i ,CL i ), manually write a small number of CQL statement templates CP i ; Use the large language model in S32 to expand the template and expand the problem set to CP' x When combining the data set in S34, the keyword sequence is also combined with the CQL statement template to generate a CQL training data set D2 = (T", CP, CL), where CP = O" × R' y ×CP' x .
[0078] S42: Construct the conversation history of interaction with the large model, in which the prompt word Prompt for guiding the large language model to perform the CQL generation task is placed in the first conversation of the history with the role "system". CQL ; After that, each time the user question is classified for intent, three "question"-"question corresponding CQL statement" pairs are randomly extracted from the D2 dataset constructed in S41, and put into the "User"-"Assistant" question-answer pair as the three shots answered by the large model, and the user question is put into a new "User" at the end of the history; the conversation history is sent to the large language model, and the CQL generation result will be given after the large language model inference.
[0079] S43: After the large language model generates CQL statements, the dispatch rule graph is queried by calling the Neo4j graph database query tool to recall the required flood control dispatch information.
[0080] Step 5: Use the recommendation generation method to generate a rule-based recommendation answer for the user based on the scheduling information recalled from the graph; Figure 5 As shown, the implementation process is as follows:
[0081] S51: Using the conversation and understanding capabilities of the large language model, the context prompt specifies that the model should answer strictly based on the dispatch information recalled by the graph. According to the results of intent classification, FS is provided for three different types of questions: "Basic common sense questions about flood control dispatch", "Recommendation questions with specified basin objects and conditions", and "Recommendation questions with specified basin objects but no conditions". A , FS B , FS C Three different presets FewShot allows the model to answer in a format.
[0082] S52: After the model generates an answer, the original question and the answer are re-input into the large language model at the same time. The large language model is used to detect and filter the answer to check whether the answer conforms to the scheduling recommendation business logic. If it does not conform to the business logic, the Temperature parameter value of the large language model is increased and a new round of answers is generated until it conforms to the business logic. Finally, the answer is output to the user. If the number of cycles exceeds the specified round, the user is prompted that the answer task cannot be completed.
[0083] Step 6: Fine-tune the training of the large language model to enhance the accuracy of problem handling in each link of the flood control dispatch rule recommendation process and improve the quality of question answering. Figure 6 As shown, the implementation process is as follows:
[0084] S61: Use the flood control dispatch intention analysis dataset D1 constructed in S35 to fine-tune the pre-trained large language model base to improve the inference weight of the large language model specifically for flood control dispatch intention analysis capabilities.
[0085] S62: According to the different classifications in the data set D2 constructed in S42, the data set is divided into three parts according to the value of CL, and the pre-trained large language model base is fine-tuned to obtain the inference weights of the large language model specifically for solving the CQL generation of the three types of user problems "A", "B" and "C". When the model is called with different weights later, the large language model can improve the accurate CQL generation capability for different scheduling problems.
[0086] S63: After the fine-tuning described in S61 and S62 achieves good results, merge the data sets D1 and D2 to form a new data set D3, and fine-tune the pre-trained large language model base to obtain new inference weights, which can simultaneously improve the intent analysis capability of the large language model and the CQL generation capability for the three types of user questions "A", "B" and "C".
[0087] For training sets D1, D2, and D3, the training set D xThe JSON is processed and converted into the form of historical dialogue, in which the prompt Prompt for guiding the large language model is placed in the first historical dialogue with the role "system"; x Three items are randomly selected from the corresponding question-answer pairs and put into the "User"-"Assistant" question-answer pair as the three shots of the question and the preset large model answer. x Put the training problem in a new "User" at the end of the history, and put D x The expected answer of the large language model in the training set is put into the "Assistant" later; in this way, each question-answer pair in the training set has 1 Prompt and 3 random Shots; check each training JSON and delete the JSON with repeated Shots and training questions.
[0088] like Figure 1 As shown, the present invention also proposes a flood control dispatch rule question-answering recommendation system based on a large language model, including a flood control dispatch rule graph construction module, a question preprocessing module, an intention analysis module, a graph query module, a recommendation generation module and a large model fine-tuning training module. Among them:
[0089] The flood control dispatch rule graph construction module extracts and collects flood control dispatch rule information from the official dispatch rule documents, and constructs a dispatch rule graph to provide external recallable data for the large language model.
[0090] The question preprocessing module is used to filter user questions and prevent the large language model from answering questions outside the dispatching rules. For example, if a user asks "What's the weather like today?", the question will be intercepted in the question preprocessing module, and a prompt will be returned saying "This question does not belong to the dispatching rules question, please ask again!" to prompt the user to start asking again. When a question such as "What are the dispatching plans for the Shimen Reservoir in the Tunxi Basin?" is asked, it will be considered to meet the dispatching rules and enter the next intention analysis module.
[0091] The intent analysis module classifies user questions according to intent, and divides the questions into three categories: "Basic knowledge questions about flood control scheduling", "Recommendation questions about specified watershed objects and conditions", and "Recommendation questions about specified watershed objects but without conditions". For example, "What principles should be paid attention to in flood control scheduling?" is the first type of question (referred to as "A" type questions), "The water level of Xia'an Reservoir is 30 meters during the flood season, and the weather forecast predicts large-scale precipitation in the next three days. How should we schedule?" is the second type of question (referred to as "B" type questions), and "I want to know the water level-flow comparison relationship of Lishimen Reservoir" belongs to the third type of question (referred to as "C" type questions).
[0092] The graph query module is used to generate Cypher statements for graph-related knowledge recall based on the type of user questions and the specific content of the user questions. After executing the Cypher statements, the text in the node is obtained as the question recall content for the large language model to reason.
[0093] In the recommendation generation module, the large language model uses the relevant text information recalled in step 5 and combines it with the user's question to generate rule-recommended answers for the user.
[0094] The large model fine-tuning training module is a training method for small-data models with slightly weaker performance. It can enhance the accuracy of problem handling in each link during the flood control scheduling rule recommendation process through fine-tuning training. The fine-tuning method can be full fine-tuning, lora fine-tuning and P-tuning fine-tuning.
[0095] The present invention combines the large language model with the knowledge graph of flood control dispatch rules, and fine-tunes the large language model to enable it to better understand and handle specific problems in the field of flood control dispatch. At the same time, the present invention uses the knowledge graph to verify and enhance the output of the large language model, effectively avoiding the "hallucination" problem that may be generated by the large language model, that is, the output of wrong or fictitious information.
[0096] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be considered as the scope of protection of the present invention.
Claims
1. A question-answering recommendation method for flood control scheduling rules based on a large language model, It is characterized in that The following steps are involved: (1) Extract and collect flood control dispatch rule information from the official dispatch rule documents and construct a flood control dispatch rule graph to provide external recallable data for the large language model; (2) Filter user questions based on question preprocessing methods to prevent the large language model from answering questions other than scheduling rules; (3) Use intent analysis methods to analyze user questions and classify user questions according to intent; (4) Use the graph query method to use different model weights according to the intent classification results to generate the corresponding flood control dispatch Neo4j graph-specific CQL query statement for data recall query; (5) Using the recommendation generation method to generate rule-containing recommendation answers for the user based on the scheduling information recalled from the graph; (6) Fine-tune the training of the large language model to enhance the accuracy of problem handling in each link of the flood control scheduling rule recommendation process and improve the quality of question answering; The step (2) comprises the following steps: (21) Use the large language model to extract the scheduling target object in the user input task, O = LLM (Prompt OE ,question); Among them, LLM is a general large language model without fine-tuning, Prompt OE is the prompt word used to extract the scheduling target object, question is the task string input by the user, and O is the scheduling target object extraction result; (22) Use the large language model to rewrite the multiple expressions of the scheduling target object extracted in the generation step (21) to obtain a possible name set O' of the extracted target object i , where O' i represents the i-th possible name of O; (23) According to the possible name set O' of the scheduling target object i The Python script synthesizes CQL query statements for the basic object graph of the watershed, and performs query operations through the py2neo query tool; (24) Check the query result. If the scheduling target object exists in the watershed basic object map, the user task is a scheduling rule recommendation task, and the user question is placed in the processing waiting queue of the intention analysis; otherwise, the user is prompted that the scheduling target object does not exist in the watershed basic object map and the recommendation system process is terminated; The step (3) comprises the following steps: (31) Combined with flood control dispatch recommendation problem set Q i , the problems are divided into three categories: "basic knowledge problems of flood control scheduling", "recommendation problems with specified basin objects and conditions", and "recommendation problems with specified basin objects but no conditions". i When the three categories are represented, "A", "B" and "C" are used respectively, and the flood control dispatch target objects in each problem are marked by manual labeling. i , Flood control dispatch conditions R i And problem classification CL i , get the basis (question, classification) pair: (T i ,CL i ), where T i =Q i -O i -R i , a problem template for removing the scheduling target object and flood control scheduling conditions; (32) Using the rewriting ability of the large language model, the large language model is prompted by context to remove the scheduling target object and flood control scheduling conditions from the question template T i The word order and synonyms are changed while maintaining the original meaning, and the corresponding question classification remains unchanged. n synonym templates are generated, and the result is: T' x ={T' i1 ,T' i2 ,T' i3 …,T' in } i =LLM(Prompt CH ,T i ) Among them, LLM is a general large language model without fine-tuning, Prompt CH It is a prompt word for word order change and synonym replacement. i is a problem template for removing the scheduling target object and flood control scheduling conditions, T' i1 ,T' i2 ,T' i3 …,T' in are the generated n synonymous templates, T' x is the final expanded data set template, from which the expanded (question, classification) pair (T' x ,CL x ); (33) Randomly extract flood control dispatch object O" from the basin basic object library as the flood control dispatch object sequence; (34) Utilizing the generative capability of the large language model, through contextual prompts and the FewShot method, the manually annotated flood control dispatch condition features are learned to generate more flood control dispatch conditions that conform to the business logic of flood control dispatch, forming a flood control dispatch condition keyword sequence, specifically: R' y ={R' i1 ,R' i2 ,R' i3 …,R' in } i =LLM(Prompt CHR ,S i1 ,S i2 ,S i3 ,T i ) Among them, Prompt CHR is a prompt word used to learn and generate flood control dispatch conditions, S i1 ,S i2 ,S i3 is the flood control dispatch condition R i Three examples of scheduling conditions randomly selected from ; (35) The flood control dispatch object sequence O' and the flood control dispatch condition keyword sequence R' y and question template T' x Combine data sets to automatically generate large-capacity and high-quality flood control scheduling intention analysis data sets D 1 =(T”,CL), where; T”=O”×R' y ×T' x ; (36) Construct a conversation history with the big model. In the first conversation with the role "system", insert the prompt word "Prompt" to guide the big language model to classify the intent. CL ; After that, each time the user question is classified, the dataset D 1 Randomly extract 3 "classification question"-"classification result" pairs, put them into the "User"-"Assistant" pair as the 3 shots answered by the big model, put the user question into a new "User" at the end of the history; send the conversation history to the big language model, and the big language model will give the classification result after reasoning; The step (4) comprises the following steps: (41) Using the dataset template (T i ,CL i ), write a small number of CQL statement templates CP i ; Use the large language model to expand the template and expand the problem set to CP' x ; When combining data sets, combine the keyword sequence with the CQL statement template to generate CQL to generate training data set D 2 =(T”,CP,CL), where CP = O”×R’ y ×CP' x ; (42) Construct a dialogue history for interacting with the large model. In the first dialogue with the role "system", insert the prompt word Prompt to guide the large language model to perform the CQL generation task. CQL ; Each time the user question is classified, the D 2 Randomly extract 3 "question"-"question corresponding CQL statement" pairs from the dataset and put them into the "User"-"Assistant" question-answer pair as 3 shots answered by the big model. Put the user's question into a new "User" at the end of the history. Send the conversation history to the big language model, which will give the CQL generation result after reasoning. (43) After the large language model generates CQL statements, the dispatch rule graph is queried by calling the Neo4j graph database query tool to retrieve the required flood control dispatch information; The step (5) comprises the following steps: (51) Using the conversation and understanding capabilities of the large language model, the context prompt specifies that the model should answer strictly based on the dispatch information recalled by the graph. According to the results of intent classification, FS is provided for three different types of questions: "basic common sense questions about flood control dispatch", "recommendation questions with specified basin objects and conditions", and "recommendation questions with specified basin objects but no conditions". A , FS B , FS C Three different presets FewShot let the model answer in a format; (52) After the model generates an answer, the original question and answer are re-input into the large language model at the same time, and the large language model is used to detect and filter the answer to check whether the answer conforms to the scheduling recommendation business logic; if it does not conform to the business logic, the Temperature parameter value of the large language model is increased and a new round of answers is generated again until it conforms to the business logic, and finally the answer is output to the user. If the number of cycles exceeds the specified round, the user is prompted that the answer task cannot be completed; The implementation process of step (6) is as follows: (61) Using the constructed flood control dispatch intention analysis dataset D 1 Fine-tune the pre-trained large language model base to improve the inference weight of the large language model dedicated to flood control scheduling intention analysis; (62) According to data set D 2 According to the different classifications in the dataset, the dataset is divided into three parts according to the CL value, and the pre-trained large language model base is fine-tuned to obtain the inference weights of the large language model specifically for solving the CQL generation of the three types of user problems "A", "B", and "C". When the model is called with different weights later, the large language model can improve the ability to generate accurate CQL for different scheduling problems; (63) Merge dataset D 1 and D 2 Form a new data set D 3 , fine-tune the pre-trained large language model base to obtain new inference weights, which can simultaneously improve the intent analysis capability of the large language model and the CQL generation capability for three types of user questions: "A", "B", and "C"; The process of fine-tuning the pre-trained large language model base is as follows: For the training set D 1 , D 2 , D 3 , the training set D x The JSON is processed and converted into the form of historical dialogue, in which the prompt Prompt for guiding the large language model is placed in the first historical dialogue with the role "system"; x Three items are randomly selected from the corresponding question-answer pairs and put into the "User"-"Assistant" question-answer pair as the three shots of the question and the preset large model answer. x Put the training problem in a new "User" at the end of the history, and put D x The expected answer of the large language model in the training set is put into the "Assistant" later; each question-answer pair in the training set has 1 prompt and 3 random shots; check each training JSON and delete the JSON with repeated shots and training questions.
2. According to claim 1, a flood control dispatch rule question-answering recommendation method based on a large language model, It is characterized in that The flood control and dispatching rule map includes water conservancy object nodes, dispatching rule nodes and dispatching rule document tree nodes; the water conservancy object nodes represent the relevant water conservancy objects about which users ask questions about dispatching rules; the dispatching rule nodes are constructed around the water conservancy object nodes, and are associated with the water conservancy object nodes, storing the dispatching rules of the corresponding water conservancy objects in the form of natural language text; the document tree nodes split the original document into nodes that store content in a tree structure according to chapter and title levels, and include relevant knowledge about flood control and dispatching, such as compilation information, execution authority, organizational guarantees and discharge relationship comparison tables.
3. A flood control dispatch rule question-answering recommendation system based on a large language model using the method as claimed in any one of claims 1 to 2, It is characterized in that It includes a flood control dispatch rule graph construction module, a problem preprocessing module, an intention analysis module, a graph query module, a recommendation generation module, and a large model fine-tuning training module; among which: The flood control dispatch rule graph construction module extracts and collects flood control dispatch rule information from the official dispatch rule documents and constructs a dispatch rule graph to provide external recallable data for the large language model. The question preprocessing module is used to filter user questions and prevent the large language model from answering questions outside the scheduling rules; The intent analysis module classifies user questions according to their intent, and divides them into three categories: "basic knowledge questions about flood control scheduling", "recommendation questions with specified watershed objects and conditions", and "recommendation questions with specified watershed objects but no conditions"; The graph query module is used to generate a Cypher statement for graph-related knowledge recall based on the type of user questions and the specific content of the user questions. After executing the Cypher statement, the text in the node is obtained as the question recall content for the large language model to reason. In the recommendation generation module, the large language model uses the relevant text information recalled in step 5 and combines it with the user's question to generate rule-based recommended answers for the user; The large model fine-tuning training module uses fine-tuning training for small data volume models with weak performance to enhance the accuracy of problem handling in each link of the flood control scheduling rule recommendation process.
4. According to claim 3, a flood control dispatch rule question-answering recommendation system based on a large language model, It is characterized in that The fine-tuning training methods include full fine-tuning, lora fine-tuning and P-tuning fine-tuning.
