Water conservancy field tool-driven agent construction method based on large language model

Abstract

The invention discloses a water conservancy field tool-driven agent construction method based on a large language model. The method comprises the following steps: respectively aggregating external services and general tools which need to be used by a user into a tool set for a water conservancy project; registering corresponding water conservancy field tools for the registered tool set; performing fine tuning training on the large language model to obtain a fine-tuned model weight, and enhancing the reasoning accuracy of the large language model in a tool set selection stage; analyzing a user problem based on a tool set selection method, and selecting a tool set capable of solving the problem according to the user problem; self-adaptive tool flow construction is carried out according to the intention of a user, and an executable scheme of a tool is constructed; checking a tool stream result, and feeding back a tool stream execution result to the user; and summarizing results by using a large language model, and outputting a plurality of modal formats. According to the method, the forgetting problem occurring when a large language model processes a complex task is solved, and the flexibility and efficiency of tool flow construction are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of natural language processing, and in particular relates to a method for constructing an intelligent agent driven by a tool in the water conservancy field based on a large language model. Background Art

[0002] Large Language Model (LLM) is a natural language processing system with more than 100 billion parameters. It is obtained by training on a large number of corpus data sets and is mainly used in advanced artificial intelligence algorithms for natural language processing tasks such as natural language understanding, text generation, summary generation, and chatbots. Large Language Model Agent (LLM Agent) is a large model application that uses large language models to perform complex tasks. LLMAgent combines LLM with task planners, memories, and tools to perform tasks proposed by users. Among them, LLM acts as a control center or "brain", responsible for managing a series of operations required to complete tasks and respond to user requests, which greatly enhances the large language model, which is reflected in the fact that the Agent system can decompose and plan complex tasks, generate execution plans, and then execute each task in the plan.

[0003] The existing LLM Agent frameworks that support complex task processing, such as Open AI's GPTs and chatGLM3, natively support the function call (tool call) function. The implementation method is: first inject the tool configuration information into the large language model Agent system, and then introduce a special tool identifier when fine-tuning the large model. In the actual reasoning link, the LLM Agent will infer the tool identifier required for the current task. After the tool identifier is generated, the tool call will be triggered to call the tool registered in the system.

[0004] However, research has found that the accuracy of tool calls in the existing LLMAgent framework is only between 30% and 60%. Although the above process can meet the requirements of simple function calls, there are many problems. First, when the number of tools increases to a certain extent, the context becomes longer, and the model is very likely to "forget"; second, for custom tools, if they have never appeared during pre-training, there is a high probability that the tool call cannot be triggered, that is, it is very dependent on large model fine-tuning and has poor adaptability; third, the current function call has very weak support for one tool to call multiple tools, and the call success rate is very low. Although LLMAgent has the ability to handle complex tasks, its ability is far from meeting the reliability requirements of field applications. The root cause is the "illusion" problem of LLM, that is, the generated text does not follow the original text or does not conform to the facts. In the reasoning stage of the LLM Agent system, the "hallucination" problem usually manifests itself as uncontrollable tool calls, that is, the large model uses the wrong tools in the tool flow construction and cannot correctly solve the problem; Fourth, in the construction of tool-driven intelligent bodies in the water conservancy field, there are problems such as lack of data fusion processing and multi-tool fusion operation. The existing intelligent body construction system is mostly biased towards simple, pre-programmed processes, tools cannot be adaptively assembled, and there is a lack of coordination between tools. Summary of the invention

[0005] Purpose of the invention: In order to solve the "forgetting" problem that may occur when a large language model is processing complex tasks, the present invention provides a tool-driven intelligent agent construction method in the water conservancy field based on a large language model.

[0006] Technical solution: The method for constructing an intelligent agent driven by a tool in the water conservancy field based on a large language model according to the present invention comprises the following steps:

[0007] (1) Toolset aggregation stage: First, external services and general tools that users need to use are aggregated into toolsets for water conservancy projects. For each toolset, templates of toolset intent information and prompt strategies are provided for registration and management.

[0008] (2) Tool injection phase: For the registered tool set, register the corresponding water conservancy field tool; the water conservancy field tool is a service or program with clear boundary conditions provided by an external supplier and is used in the form of service call;

[0009] (3) Fine-tune the large language model, obtain the fine-tuned model weights, and enhance the accuracy of the large language model’s reasoning during the toolset selection phase;

[0010] (4) Analyze user problems based on the toolset selection method and select the toolset that can solve the problem according to the user problem;

[0011] (5) Adaptive tool flow construction based on user intent to build an executable solution for the tool;

[0012] (6) Execute the tool flow according to the tool flow executable solution generated in step (5), obtain the final execution result of the tool flow, and then enter step (6);

[0013] (7) Verify the tool flow results, couple the user questions, intent recognition results and the log file of the tool flow execution process with the third prompt strategy template, and use the above contents as the input of the five base models respectively. Each large model is evaluated based on the user's understanding of the user's intention, the control of the tool execution process and the degree of problem resolution. The output results of the five models are summarized. If four or more models give a passing answer, go to step (8); if not, go to step (4);

[0014] (8) Feedback the tool flow execution results to the user; use a large language model to summarize the results and output them in multiple modal formats.

[0015] Furthermore, the toolset prompt strategy template in step (1) includes a prompt strategy for the toolset selection phase and a prompt strategy for the toolflow reasoning phase; the prompt strategy for the toolset selection phase includes flood prediction, intelligent scheduling, flood control planning, and emergency plan generation; the prompt strategy for the toolflow reasoning phase is used in the task planning phase, describing the function of each tool in the toolset, as well as a summary of the input and output information of each tool, so that the large language model can understand and prompt the correct use of the tool.

[0016] Furthermore, the implementation process of step (2) is as follows:

[0017] (21) Write the service configuration file according to the tool content and the service configuration file specification, including tool function description information, tool call request description information, tool parameter extraction prompt strategy and related use cases; the tool function description information is used for the dynamic generation of prompt strategy in the large language model tool flow construction phase, the tool API request description information is used for API call in the tool call phase, and serves as the input and output specifications; the tool parameter extraction prompt strategy and related use cases are used for parameter extraction before the tool execution phase;

[0018] (22) Tool configuration file verification and registration, complete tool configuration file data format verification, service URL accessibility test, and tool usability test;

[0019] If the tool configuration file data format verification passes, then go to step (23); if the tool configuration file data format verification fails, then go back to step (21);

[0020] (23) Dynamically splice and maintain the prompt words of the tool set based on the existing tool function description information.

[0021] Furthermore, the implementation process of step (22) is as follows:

[0022] (221) Tool configuration file data format verification: Use the JSON Schema verification method to verify the data format of the tool configuration file; the verification pass standard is that the file meets the predetermined JSON data structure, all required fields are complete, and the field value type meets the predetermined format requirements;

[0023] (222) Service URL accessibility test: Perform accessibility test on the service URL specified in the configuration file; the test passes if the URL can successfully respond, the response status code is 200, and the returned data format meets the predetermined requirements;

[0024] (223) Tool usability testing: Perform usability testing on the tools specified in the configuration file; the test passing criteria are that the tool can perform the specified functions as expected, no exceptions are thrown during the execution process, and the function execution results meet the expected effects.

[0025] Furthermore, the large language model in step (3) is chatGLM3-6B-8K.

[0026] Furthermore, the implementation process of step (3) is as follows:

[0027] (31) Prepare a domain problem set. For each toolset, collect user questions and aggregate them into a domain problem set Q w ; For the fine-tuning dataset, each of which contains a prompt word p guided by the question instruction 、Question j ∈Q and its corresponding toolset identifier s j ;

[0028] (32) Configure fine-tuning model information, set learning rate, learning rate warm-up ratio, weight decay coefficient, batch size, basic model information, and fine-tuning training set location;

[0029] (33) Fine-tune the large language model and select the Lora mode. After fine-tuning is completed, take out the fine-tuned model weights and use it as the model in the toolset selection stage.

[0030] Furthermore, the implementation process of step (4) is as follows:

[0031] (41) receiving a question input by a user, concatenating the user question with the first prompt strategy template, inputting the fine-tuned large language model, inferring a tool set identifier that the user intends to use, and then verifying the generated identifier;

[0032] If the toolset that the user intends to use is clear in the inference result, that is, there is only one toolset identifier in the output, then go to step (42);

[0033] If the toolset intended to be used in the user's question is unclear or ambiguous in the inference result, that is, zero or more toolset identifiers are generated, the user is required to supplement the information again, and the large language model generates clarification questions, which are fed back to the user and prompted to answer to further clarify the user's intention. Then (41) is repeated until the toolset intended to be used by the user is clear.

[0034] (42) According to the toolset identifier output by the reasoning, select it as the target toolset.

[0035] Furthermore, the implementation process of step (5) is as follows:

[0036] (51) A toolset solves a type of business requirement, which includes several tools that are needed to complete the business requirement. For a user's problem, at least one executable tool chain can be formed through the combination of tools. The toolset selected based on the large language model extracts the functional description information of all tools in the toolset according to the content of the toolset.

[0037] (52) The execution tool adaptively arranges the tasks and adaptively arranges the tool flow to form a tool flow execution list L;

[0038] (53) Verify and check the generated tool flow execution list L to ensure the logical coherence and parameter integrity of the tool flow; specifically, it includes process structure verification, dependency checking, conditional branch verification, exception handling checking, and verification of parameter existence, type, value range and format; if the verification passes, execution is allowed; if it fails, an error report is provided.

[0039] Furthermore, the implementation process of step (52) is as follows:

[0040] (521) Combine the prompt strategies, aggregate the functional description information of each tool and the user's input combination, and dynamically splice them with the second prompt strategy template into a prompt strategy P as the input parameter of the large language model; the prompt strategy P is designed and combined in a thinking chain manner, including model roles, model problem decomposition task prompt strategies, and model tool adaptive tool mapping task prompt strategies; P = concat (P1, P2), where P1 is to guide the large language model to decompose the user problem Q into several sub-problems Q1, Q2, ..., Q through a few-shot and guided prompt strategy. x , each sub-problem Q iThere is a connection and parallel relationship between them; P2 is for adaptive tool mapping; the description information and input and output specifications of the tools in the selected toolset and the basic toolset are taken as input, and the expected output sub-problem Q i The corresponding relationship with the tool identifier; wherein, 0<=i<=x;

[0041] (522) The prompt strategy is used as the input of the large language model to obtain the tool identifier corresponding to each sub-problem to form a tool arrangement table H. For each element h in the table i , by sub-problem Q i and tool identifier s;

[0042] (523) For all elements h in the tool arrangement table H described in (522) i , adaptively load the tool represented by the tool identifier from the starting node, and record the input and output parameters in each step until the last element in the list is processed. Couple the processing results with the information in the tool scheduling table to form the tool flow execution list L.

[0043] Furthermore, the implementation process of step (6) is as follows:

[0044] (61) Maintain a metadata table to record the information generated by tool calls during tool execution. First, extract parameters from user questions as the content in the metadata table;

[0045] (62) For the tool flow execution list L, take out each element in order, then load the corresponding call request information of the current tool from the resource management module, match and load according to the metadata type in each element, and then perform tool call execution; after waiting for the execution to be completed, add the output content to the metadata table, and then repeat this step until all elements are executed.

[0046] Beneficial effects: Compared with the prior art, the beneficial effects of the present invention are: the present invention adopts a semi-adaptive approach to constrain the selection of large language model tools according to the domain business application scenarios, and realizes adaptive assembly skills within its capabilities; it not only avoids the "forgetting" problem that may occur when large language models handle complex tasks, but also eliminates the cumbersome pre-compilation process in traditional methods, and improves the flexibility and efficiency of tool flow construction; compared with the general tool flow calling system that uses a large language model with hundreds of billions of parameters for toolset selection in the tool selection stage, the present invention uses LoRa fine-tuning means to fine-tune the large language model with billions of parameters, and achieves good results in the toolset selection stage, which greatly reduces the model inference cost and achieves the effect of intranet deployment. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 is a flow chart of the present invention;

[0048] Figure 2 A schematic diagram of a tool set prompt word dynamic splicing and maintenance method proposed by the present invention;

[0049] Figure 3 A schematic diagram of the flow chart of the tool set selection method proposed in the present invention;

[0050] Figure 4 This is a schematic diagram of the adaptive tool flow construction method proposed in the present invention. DETAILED DESCRIPTION

[0051] The present invention will be further described in detail below in conjunction with the accompanying drawings.

[0052] like Figure 1 As shown, the present invention proposes a method for constructing an intelligent agent driven by a tool in the water conservancy field based on a large language model, comprising the following steps:

[0053] Step 1: In the toolset aggregation phase, first, external services and general tools that users need to use are aggregated into toolsets for water conservancy projects; for each toolset, templates of toolset intent information and prompt strategies are provided for registration management. Each toolset contains solutions to specific water conservancy needs; each toolset contains solutions to specific business needs.

[0054] For each toolset, two types of information need to be configured:

[0055] 1) Prompt strategy in the toolset selection phase: This includes a description of the specific water conservancy needs that the toolset is designed to address, such as flood prediction, intelligent scheduling, flood control planning, emergency plan generation, etc., as well as some use cases, which are used to enhance its ability to identify and select toolsets in the large language model reasoning phase.

[0056] 2) Prompt strategy for the tool flow reasoning phase: This part of information is used in the task planning phase to describe in detail the functions of each tool in the toolset, as well as a summary of the input and output information of each tool, so that the large language model can understand and prompt the correct use of the tool.

[0057] Step 2: In the tool injection phase, for the registered tool set, register the corresponding water conservancy field tools; tools are services or programs with clear boundary conditions provided by external suppliers, usually used in the form of service calls, which specifically include the following steps:

[0058] (2.1) Write the service configuration file according to the tool content and service configuration file specifications, including but not limited to the tool function description information, tool call request description information, tool parameter extraction prompt strategy and related use cases, specifically:

[0059] Tool function description information: including the purpose of the tool, the overview of the tool input and output, which is used to dynamically generate prompt strategies in the tool flow construction phase. For example, for a tool used for hydrological data analysis, the function description information is "used to analyze river flow, water level changes and rainfall data to predict flood risks and optimize reservoir management"; this description information helps the large language model understand how to use the tool, thereby improving the accuracy and efficiency of task execution.

[0060] Detailed call request description information: including the detailed call request format of the tool, which is used to call the tool in the execution phase; specifically including the call request header and request body, call case information, etc. For example, for a water resource management tool, the call request may include "obtain real-time water level data for a specific river basin" or "calculate water resource consumption within a specific time period".

[0061] Prompt strategy for tool parameter extraction: including parameters that the tool can accept and their explanations and detailed explanations as well as a small number of use cases for tool parameters (used to enhance the ability of the large language model in the reasoning stage), which are used for the large language model to extract user-provided parameters from user questions in the task planning stage. Preferably, for the parameters that the tool can accept and their explanations, since the parameter lists of existing request calls are all in English and some are not interpretable, it is necessary to provide the Chinese meaning of the corresponding parameters to facilitate the large language model to better infer the information provided by the user. For example, for a tool for dam safety monitoring, parameters may include "dam height", "water level", "pressure sensor reading", etc., and Chinese explanations of these parameters need to be provided, such as "dam height: the vertical height of the dam, in meters".

[0062] After completing the configuration file, proceed to step (2.2).

[0063] (2.2) Tool configuration file verification and registration, including but not limited to tool configuration file data format verification, service accessibility testing, and tool usability testing; the purpose of these tests is to ensure that the information in the configuration file is accurate so that the large language model can successfully call and execute external tools, thereby improving the overall system performance and user satisfaction. In more detail:

[0064] Tool configuration file data format verification: Check whether the content in the configuration file meets the predetermined format requirements; format errors may cause the tool to be unable to be loaded or parsed correctly, thus affecting the use of the tool by the large language model; the JSON Schema verification method is used here to verify the data format of the tool configuration file. The verification pass standard is: the file meets the predetermined JSON data structure, all required fields are complete, and the field value type meets the predetermined format requirements.

[0065] Service accessibility test: Verify whether the service call address provided in the configuration file is accessible. This test ensures that the large language model can establish a valid network connection when trying to call the tool; the test passes when the URL can successfully respond, the response status code is 200, and the returned data format meets the predetermined requirements.

[0066] Tool usability testing: Test whether the tool functions normally and can be called through the call case information provided by the tool and verify whether the call output is in the expected output format. The purpose of this test is to confirm whether the tool works as expected and whether it can provide the required support for large language models; the test passing criteria are: the tool can execute the specified function as expected, and no exceptions are thrown during the execution process, and the function execution results meet the expected effect.

[0067] If the tool configuration file data format verification passes, the verification and registration are completed, and the tool can be used to enhance the large language model to handle complex tasks, and proceed to step (2.3); otherwise, repeat the preparation of the configuration file, inject it into the system, and proceed to step (2.1).

[0068] (2.3) Dynamic splicing of tool set prompts. Since services in a tool set may be dynamically added or reduced, it is necessary to dynamically splice and maintain the prompts of the tool set. Dynamic splicing and maintenance needs to be based on existing tool function description information and other content. Dynamic splicing methods include Figure 2 As shown, the tool function description information in the tool configuration file verification and registration phase is obtained, and the tool implementation function F, input parameter brief description information I, and output result brief information O are updated to the tool set prompt strategy, and the syntax of the prompt strategy is adjusted accordingly so that the large language model can accurately understand and use these tools. In this way, the prompt strategy of the tool flow reasoning is kept up to date, ensuring that the large language model can select the appropriate tool according to the actual situation and use it correctly when performing tasks.

[0069] Step 3: In the large model fine-tuning training phase, the large language model is fine-tuned to obtain the model weight after fine-tuning, thereby enhancing the reasoning accuracy of the large language model in the toolset selection phase. The specific implementation is as follows:

[0070] (3.1) Prepare a water conservancy domain problem set. For each tool set, collect user questions and aggregate them into a domain problem set Q w , and then aggregate all datasets into a fine-tuning dataset D FT , for d∈D FT , d consists of three parts, namely the prompt word p instruction 、Question j and its corresponding toolset identifier s j.

[0071] The prompt strategy of question guidance includes task details and toolset identification tasks (toolset identifier, toolset details, etc.). For example, for a water conservancy toolset for water resources management, the problem set may include questions such as "How to predict river flow based on current rainfall?" or "How to optimize the reservoir release strategy to cope with drought?" The prompt words of question guidance may include "Analyze and predict water resources status based on historical hydrological data and meteorological forecasts"; then enter (3.2).

[0072] (3.2) Configure fine-tuning model information. Set the learning rate, learning rate warm-up ratio, weight decay coefficient, batch size, basic model information, and the location of the fine-tuning training set. The preferred learning rate used in this method is 10e-6 and the batch size is 32; then go to (3.3).

[0073] (3.3) Fine-tune the large language model; use the lora fine-tuning method in the instruction fine-tuning method to fine-tune. After the fine-tuning is completed, extract the fine-tuned model weights and use them for the model LLM in the toolset selection stage. PEFT ,Preferably, this method uses four RTX 309024GB cards for fine-tuning training.

[0074] Step 4: Figure 3 As shown in the figure, based on the toolset selection method, user questions are analyzed, user questions are classified according to their intentions, and finally the classification results are obtained. Specifically, the following steps are included:

[0075] (4.1) Receive the question input by the user, and combine the user question with the part of the toolset description in the prompt strategy of all toolset selections in the toolset library to form a toolset prompt strategy. At the same time, randomly select 3 to 5 use cases from the prompt strategies of all toolset selections as few shot examples; specifically, assemble the prompt strategy by the following method:

[0076] Prompt IR =concat(Prompt1,TKSET,RANDOM_SHOTS(TKSHOT))

[0077] Among them, the concat function couples the three, Prompt1 is the first prompt strategy template, TKSET is a list of toolset identifiers, including the following four categories: a) toolset identifiers prepared in the previous stage; b) common toolset identifiers; c) empty identifiers; d) composite identifiers. TKSHOT is a toolset sample. For each group in TKSHOT, it consists of a question-answer pair, that is, the first one is the case question provided by the role "User", and the second one is the toolset selection result provided by the role "Assistant"; the role of the RANDOM_SHOTS function is to randomly select 3 groups of results from the selected samples; then go to step (4.2).

[0078] (4.2) The above content is used as the input of the fine-tuned large language model. The large language model determines the user's intention based on the input and determines whether the current intention can be solved by a tool in a toolset. Specifically, this method determines the target toolset through the following formula:

[0079]

[0080] Among them, LLM PEFT It is the large model after fine-tuning; then go to step (4.3).

[0081] (4.3) If the tool set used by the user’s question is clear in the large language model inference result, that is, tk target_id belongs to category a or b, then go to step P4; if the toolset intended to be used in the user question is unclear or ambiguous in the inference result, that is, tk target_id If it belongs to category c or d, it is considered that the current problem cannot be solved by the selected tool set, and the large language model generates clarification questions, feeds them back to the user and prompts the user to answer to further clarify the user's intention, and then enters step (4.1).

[0082] (4.4) According to the toolset identifier output by the large language model inference, select it as the target toolset tk target , and go to step 5.

[0083] Step 5: Figure 4 As shown, an adaptive tool flow is constructed according to the user's intention to build an executable solution for the tool, which includes the following steps:

[0084] (5.1) A tool set solves a type of business requirement, which includes several tools that are needed to complete the business requirement; for a user's problem, at least one executable tool chain can be formed through the combination of tools. The tool set tk selected based on the large language model in step D target, take out the functional description information of all tools in the toolset; then go to step (5.2).

[0085] (5.2) The execution tool adaptively arranges tasks and adaptively arranges tool flows to form a tool flow execution list L, which specifically includes the following processes:

[0086] S1: Combined prompt strategy, aggregates the functional description information of each tool and the user's input combination, and dynamically splices it with the second prompt strategy template into prompt strategy P as the input parameter of the large language model. Prompt strategy P is designed and combined in the form of thought chain, including model role, model problem decomposition task prompt strategy, model tool adaptive tool mapping task prompt strategy, etc.

[0087] Prompt2 = concat(Prompt 21 ,Prompt 22 )

[0088] Among them, Prompt 21 It uses a few-shot and guided prompt strategy to guide the large language model to decompose the user question Q into several sub-questions Q1, Q2, ..., Qx (0 <= i <= x), and each sub-question Qi is connected and parallel. 22 It is to make adaptive tool mapping; the description information and input and output specifications of the tools owned by the selected toolset and the basic toolset are taken as input, and the corresponding relationship between the expected output sub-problem Qi and the tool identifier is obtained; 0<=i<=x; then enter step S2.

[0089] S2: Take the prompt strategy as the input of the large language model, and get the tool identifier corresponding to each sub-problem to form the tool arrangement table H:

[0090] H=LLM FM (Prompt2,Q)

[0091] Among them, LLM FM The function is to call the base language model, Prompt2 is the prompt strategy generated in step S1, Q is the user's specific question; H is a list of correspondences between sub-problems Qi and tool identifiers, H = {h1, h2, ..., hx}, for hi with 0 <= i <= x, hi = {Qi, s}, Qi is the sub-problem, s is the tool identifier corresponding to the current sub-problem, and then enters step S2.

[0092] S3: For all elements hi in the tool arrangement table H described in step S2, adaptively load the tools represented by the tool identifiers from the starting node, and record the input and output parameters in each step until the last element in the list is processed. Couple the processing results with the information in the tool arrangement table to form a tool flow execution list L.

[0093] (5.3) Verify the generated tool flow execution plan to ensure the logical coherence of the tool flow and the parameter integrity of each step. Specifically, the following verifications need to be completed:

[0094] Process structure verification: Confirm that each step of the tool flow is arranged in a predetermined logical order, and ensure that the transitions between steps are reasonable without logical gaps or jumps.

[0095] Dependency check: Confirm that the execution dependencies of each step are clear and check whether there are circular dependencies or undefined dependencies.

[0096] Conditional branch verification: Check whether the conditional branches in the tool flow cover all possible execution paths to ensure that the tool flow can be executed correctly under different conditions.

[0097] Exception handling check: Confirm that the tool flow contains pre-processing measures for possible exceptions, such as error capture, logging, etc., to ensure that the tool flow can run stably when encountering exceptions.

[0098] Parameter integrity check: perform parameter existence check to ensure that the parameters required for each step are defined and exist, perform parameter type check to confirm that the parameter type is consistent with the expected one, perform parameter value range and format check to ensure that the parameter value is within the allowed range and the format is correct.

[0099] After the above verification, if the tool flow meets all the verification conditions, it is allowed to execute; if there is a verification failure, the system will generate a detailed error report, which includes the specific steps, parameters and error causes of the verification failure, so that the user can make corresponding adjustments and corrections and proceed to step 6.

[0100] Step 6: Execute the tool flow according to the tool flow execution list L generated in step 5 to obtain the final execution result of the tool flow. The specific implementation method is as follows:

[0101] (6.1) Maintain a metadata table T metadata , used to record the information generated by tool calls during tool execution.,First, parameters are extracted from the user questions as the content in the metadata table.

[0102] (6.2) For the tool flow execution list L, take out each piece of information in order, then load the corresponding call request information of the current tool from the resource management module, match and load according to the metadata type in each element, and then perform tool call execution; after waiting for the execution to be completed, add the output content to the metadata table, and then repeat this step until all elements are executed.

[0103] During the execution of the tool flow, the execution status of the tool is monitored in real time to ensure the correctness and efficiency of each tool call. If an abnormal situation is detected, the exception handling mechanism is immediately activated and handled according to the preset exception handling strategy to ensure the continuity and stability of the tool flow.

[0104] After the tool flow is executed, the information in the metadata table is summarized and analyzed to generate a tool flow execution report. The report includes key information such as detailed records of tool calls, execution time, and output results, providing a basis for subsequent tool flow result verification and feedback.

[0105] Step 7: Verify the tool flow results to ensure the effectiveness of the tool execution process. The specific implementation method is as follows: the system collects files of user questions, intent recognition results and tool flow execution logs, and couples the file with the third prompt strategy template to ensure the integrity and consistency of the input data.

[0106] The coupled data is input into five large base models L1, L2, L3, L4, and L5 with 10 billion parameters. Each model independently evaluates the understanding of user intent, the control of tool execution process, and the degree of problem solving. Each large model outputs a pass or fail response based on the evaluation results. The system summarizes and counts the output results of the five models to determine the number of models that passed.

[0107] If the statistical results show that four or more models give passing responses, the tool flow execution result is considered valid, and the system enters step 8 for result feedback.

[0108] If the statistical results show that the number of models that passed is less than four, it is considered that there is a problem with the tool flow execution result, and the system returns to step 4 to re-select the toolset and build the tool flow.

[0109] Step 8: Feedback the tool flow execution results to the user; use the large language model to summarize the results. The final output format has multiple modes, so there will be a combined output:

[0110] For text-based output, a large language model is used to summarize results based on user conversations and tool flow solution execution.

[0111] For form type output, complete the form information, including the header, row and column meanings, and other information.

[0112] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be considered as the scope of protection of the present invention.

Claims

1. A tool-driven intelligent agent construction method in the water conservancy field based on a large language model, characterized in that: The following steps are involved: (1) Toolset aggregation stage: First, external services and general tools that users need to use are aggregated into toolsets for water conservancy projects. For each toolset, templates of toolset intent information and prompt strategies are provided for registration and management. (2) Tool injection phase: For the registered tool set, register the corresponding water conservancy field tool; the water conservancy field tool is a service or program with clear boundary conditions provided by an external supplier and is used in the form of service call; (3) Fine-tune the large language model, obtain the fine-tuned model weights, and enhance the accuracy of the large language model’s reasoning during the toolset selection phase; (4) Analyze user problems based on the toolset selection method and select the toolset that can solve the problem according to the user problem; (5) Adaptive tool flow construction based on user intent to build an executable solution for the tool; (6) Execute the tool flow according to the tool flow executable solution generated in step (5), obtain the final execution result of the tool flow, and then enter step (6); (7) Verify the tool flow results, couple the user questions, intent recognition results and the log file of the tool flow execution process with the third prompt strategy template, and use the above contents as the input of the five base models respectively. Each large model is evaluated based on the user's understanding of the user's intention, the control of the tool execution process and the degree of problem resolution. The output results of the five models are summarized. If four or more models give a passing answer, go to step (8); if not, go to step (4); (8) Feedback the tool flow execution results to the user; use a large language model to summarize the results and output them in multiple modal formats.

2. The method for constructing an intelligent agent driven by a tool in the water conservancy field based on a large language model according to claim 1, characterized in that: The toolset prompt strategy template in step (1) includes prompt strategies for the toolset selection phase and prompt strategies for the toolflow reasoning phase; the prompt strategies for the toolset selection phase include flood prediction, intelligent scheduling, flood control planning, and emergency plan generation; the prompt strategies for the toolflow reasoning phase are used in the task planning phase, describing the functions of each tool in the toolset, as well as a summary of the input and output information of each tool, so that the large language model can understand and prompt the correct use of the tool.

3. The method for constructing an intelligent agent driven by a tool in the water conservancy field based on a large language model according to claim 1, characterized in that: The implementation process of step (2) is as follows: (21) Write the service configuration file according to the tool content and the service configuration file specification, including tool function description information, tool call request description information, tool parameter extraction prompt strategy and related use cases; the tool function description information is used for the dynamic generation of prompt strategy in the large language model tool flow construction phase, and the tool API request description information is used for API call in the tool call phase and serves as the input and output specifications; The prompt strategy and related use cases for tool parameter extraction are used to extract parameters before the tool execution phase; (22) Tool configuration file verification and registration, complete tool configuration file data format verification, service URL accessibility test, and tool usability test; If the tool configuration file data format verification passes, then go to step (23); if the tool configuration file data format verification fails, then go back to step (21); (23) Dynamically splice and maintain the prompt words of the tool set based on the existing tool function description information.

4. The method for constructing an intelligent agent driven by a tool in the water conservancy field based on a large language model according to claim 3 is characterized in that: The implementation process of step (22) is as follows: (221) Tool configuration file data format verification: Use the JSON Schema verification method to verify the data format of the tool configuration file; the verification pass standard is that the file meets the predetermined JSON data structure, all required fields are complete, and the field value type meets the predetermined format requirements; (222) Service URL accessibility test: Perform accessibility test on the service URL specified in the configuration file; the test passes if the URL can successfully respond, the response status code is 200, and the returned data format meets the predetermined requirements; (223) Tool usability testing: Perform usability testing on the tools specified in the configuration file; the test passing criteria are that the tool can perform the specified functions as expected, no exceptions are thrown during the execution process, and the function execution results meet the expected effects.

5. The method for constructing an intelligent agent driven by a tool in the water conservancy field based on a large language model according to claim 1, characterized in that: The large language model in step (3) is chatGLM3-6B-8K.

6. The method for constructing an intelligent agent driven by a tool in the water conservancy field based on a large language model according to claim 1, characterized in that: The implementation process of step (3) is as follows: (31) Prepare a domain problem set. For each toolset, collect user questions and aggregate them into a domain problem set Q w ; For the fine-tuning dataset, each of which contains a prompt word p guided by the question instruction 、Question j ∈Q and its corresponding toolset identifier s j ; (32) Configure fine-tuning model information, set learning rate, learning rate warm-up ratio, weight decay coefficient, batch size, basic model information, and fine-tuning training set location; (33) Fine-tune the large language model and select the Lora mode. After fine-tuning is completed, take out the fine-tuned model weights and use it as the model in the toolset selection stage.

7. The method for constructing an intelligent agent driven by a tool in the water conservancy field based on a large language model according to claim 1, characterized in that: The implementation process of step (4) is as follows: (41) receiving a question input by a user, concatenating the user question with the first prompt strategy template, inputting the fine-tuned large language model, inferring a tool set identifier that the user intends to use, and then verifying the generated identifier; If the toolset that the user intends to use is clear in the inference result, that is, there is only one toolset identifier in the output, then go to step (42); If the toolset intended to be used in the user's question is unclear or ambiguous in the inference result, that is, zero or more toolset identifiers are generated, the user is required to supplement the information again, and the large language model generates clarification questions, which are fed back to the user and prompted to answer to further clarify the user's intention. Then (41) is repeated until the toolset intended to be used by the user is clear. (42) According to the toolset identifier output by the reasoning, select it as the target toolset.

8. The method for constructing an intelligent agent driven by a tool in the water conservancy field based on a large language model according to claim 1, characterized in that: The implementation process of step (5) is as follows: (51) A toolset solves a type of business requirement, which includes several tools that are needed to complete the business requirement. For a user's problem, at least one executable tool chain can be formed through the combination of tools. The toolset selected based on the large language model extracts the functional description information of all tools in the toolset according to the content of the toolset. (52) The execution tool adaptively arranges the tasks and adaptively arranges the tool flow to form a tool flow execution list L; (53) Verify and check the generated tool flow execution list L to ensure the logical coherence and parameter integrity of the tool flow; specifically, it includes process structure verification, dependency checking, conditional branch verification, exception handling checking, and verification of parameter existence, type, value range and format; if the verification passes, execution is allowed; if it fails, an error report is provided.

9. The method for constructing an intelligent agent driven by a tool in the water conservancy field based on a large language model according to claim 8, characterized in that: The implementation process of step (52) is as follows: (521) Combining prompt strategies, aggregating the functional description information of each tool and the user's input combination, and dynamically splicing them with the second prompt strategy template into a prompt strategy P as an input parameter of the large language model; the prompt strategy P is designed and combined in a thinking chain manner, including model roles, model problem decomposition task prompt strategies, and model tool adaptive tool mapping task prompt strategies; P = concat(P1, P2), where P1 is the method that guides the large language model to decompose the user question Q into several sub-questions Q1, Q2, ..., Q through a few-shot and guided prompt strategy. x , each sub-problem Q i There is a connection and parallel relationship between them; P2 is for adaptive tool mapping; the description information and input and output specifications of the tools in the selected toolset and the basic toolset are taken as input, and the expected output sub-problem Q i The corresponding relationship with the tool identifier; wherein, 0<=i<=x; (522) The prompt strategy is used as the input of the large language model to obtain the tool identifier corresponding to each sub-problem to form a tool arrangement table H. For each element h in the table i , by sub-problem Q i and tool identifier s; (523) For all elements h in the tool arrangement table H described in (522) i , adaptively load the tool represented by the tool identifier from the starting node, and record the input and output parameters in each step until the last element in the list is processed. Couple the processing results with the information in the tool scheduling table to form the tool flow execution list L.

10. The method for constructing a tool-driven intelligent agent in the water conservancy field based on a large language model according to claim 1, characterized in that: The implementation process of step (6) is as follows: (61) Maintain a metadata table to record the information generated by tool calls during tool execution. First, extract parameters from user questions as the content in the metadata table; (62) For the tool flow execution list L, take out each element in order, then load the corresponding call request information of the current tool from the resource management module, match and load according to the metadata type in each element, and then perform tool call execution; after waiting for the execution to be completed, add the output content to the metadata table, and then repeat this step until all elements are executed.

Images