Large model active tool calling method and system based on multi-step reasoning
By conducting three-stage training on large language models and building reverse and forward training data sets, the problem of insufficient tool call in large language models in the professional science field is solved, the code executable rate and problem solving efficiency is improved, and the intelligence and effectiveness of cross-domain tool call is realized.
Patent Information
- Application Number
- CN202510934387.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-08
- Publication Date
- 2025-08-05
- Estimated Expiration
- 2045-07-08
AI Technical Summary
The existing large language models lack the awareness and ability to actively call tool libraries in complex inference tasks in the field of professional science, resulting in low code executable rate, insufficient problem-solving efficiency, and limited cross-domain generalization capabilities.
By building a reverse training dataset and a forward training dataset, the large model is trained in three stages, including tool knowledge injection, tool awareness strengthening and cross-domain generalization, improving the model's tool usage ability and cross-domain adaptability.
It significantly improves the code executable rate and problem-solving efficiency of the model in complex inference tasks, realizes the intelligence and effectiveness of tool calls in multiple fields, and improves the accuracy and robustness of the model in multiple inference tasks.
Smart Images

Figure CN120430423A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a large-model active tool calling method and system based on multi-step reasoning. Background Art
[0002] In recent years, large language models (LLMs) have demonstrated significant potential in complex reasoning tasks and have achieved initial success in scientific exploration. However, existing research indicates that current LLMs only have high-school-level capabilities in specialized scientific fields and struggle to solve complex or highly specialized problems. This limitation stems primarily from the following technical flaws: Passive tool invocation mechanisms: Existing methods (such as Chain-of-Thought, Program-of-Thought, and Tool-Integration) rely on LLMs to generate Python code to assist in reasoning. However, these methods only address basic code implementation and fail to motivate the model to proactively invoke advanced tool libraries. For example, when solving the shortest path problem, the model tends to write complex algorithmic code itself rather than invoking mature graph theory libraries in Python (such as NetworkX). This results in a high rate of logical errors and low execution efficiency.
[0003] Lack of tool knowledge and enthusiasm for tool deployment: LLMs lack knowledge of specialized tool libraries (such as those for scientific computing and financial analysis) and lack a decision-making mechanism for when to deploy these tools. Existing technologies lack systematic training for model recognition of tool application scenarios, leading to over-reliance on native reasoning for complex tasks rather than leveraging existing tool resources to efficiently solve problems.
[0004] Weak code execution and generalization capabilities: Traditional tool integration approaches generate code snippets with a low execution rate (especially less than 60% for complex mathematical tasks like MATH), and solutions are difficult to transfer across domains. Methods like retrieval enhancement (RAG) show significant efficiency degradation when processing specialized documents, exposing the flaw of coupling tool retrieval and execution.
[0005] There is an urgent need for a systematic training framework to address the three core problems of LLMs: weak tool awareness, insufficient initiative in calling tools, and limited cross-domain generalization capabilities, so as to improve their ability to solve complex problems in professional fields. Summary of the Invention
[0006] To this end, an embodiment of the present invention provides a large-model active tool calling method and system based on multi-step reasoning, which is used to solve the problem in the existing technology that large language models lack the awareness and ability to actively call tool libraries, resulting in low code execution rate and insufficient problem-solving efficiency in complex reasoning tasks.
[0007] To solve the above problems, an embodiment of the present invention provides a large model active tool calling method based on multi-step reasoning, the method comprising: By collecting tool information, a structured tool description set is generated, and based on the tool description set, executable code samples and corresponding natural language explanations are automatically generated to construct a reverse training dataset; By labeling the necessity of using tools in real-world problems, the large model is prompted to generate a multi-step reasoning path that combines natural language and code, verify and filter the correctness of the reasoning path, and build a positive training dataset; The large model is trained in three stages based on the reverse training dataset and the forward training dataset, including: Phase 1: Using the reverse training dataset to train the basic large model, so as to inject tool usage knowledge and enhance the ability of the large model to use the tool, and obtain the trained large model parameters; Phase 2: Based on the trained large model parameters obtained in the first phase, the forward training dataset is used to train the awareness of the large model usage tool to obtain the trained large model parameters; The third stage: Based on the trained large model parameters obtained in the second stage, multi-domain task data is integrated and the large model is fine-tuned through multi-task learning to achieve cross-domain generalization.
[0008] Preferably, the reverse training data set construction specifically includes: Collect description information of tools in the Python library, including functions, parameters, and usage instructions; Use the description information of the large model optimization tool to generate function labels and concise function summaries to obtain the optimized tool set; Based on the optimized toolset, code snippets are generated to illustrate examples of tool usage, and few-shot learning and filtering mechanisms are used to reduce redundancy and increase the diversity of usage examples.
[0009] Preferably, the method of labeling the necessity of using tools in real-world problems includes: A dual labeling method is used to label the necessity of tool use in real-world problems, including comparative labeling and confidence labeling. The comparative labeling method generates a thinking chain solution and a tool integration solution reasoning for the same problem. When the tool integration solution reasoning accuracy is higher, it is marked as "need tool"; Confidence annotation method: Use large model output tools to call the necessary confidence scores and explanations.
[0010] Preferably, the method for prompting the large model to generate a multi-step reasoning path combining natural language and code includes: Decompose the reasoning task into multiple subtasks, each of which contains natural language reasoning steps and executable code snippets; The output of the code snippet is used as the input of subsequent subtasks until the final answer is generated.
[0011] Preferably, the large model is trained in three stages, and the training objective function of each stage is: ; Where, is the training objective function; is the large model parameter; is the training data set; is the input sequence, Output sequence for the target; The target output sequence tokens; for All previous tokens; Predict probability distributions for the model; is the target sequence length.
[0012] Preferably, the parameters of the large model after training obtained in the first stage are: ; Where, are the parameters of the large model after training obtained in the first stage; are the initial parameters of the large model; is the reverse training loss function; is the reverse training dataset.
[0013] Preferably, the parameters of the large model after training obtained in the second stage are: ; Where, are the parameters of the large model after training obtained in the second stage; are the parameters of the large model after training obtained in the first stage; is the forward training loss function; is the positive training dataset.
[0014] Preferably, the method of integrating multi-domain task data based on the trained large model parameters obtained in the second stage and fine-tuning the large model through multi-task learning to achieve cross-domain generalization includes: make Represents multiple task datasets, each of which corresponds to a specific field or problem type, and combines these task datasets into a unified multi-task objective, whose weighted loss function is: ; Where, is the weighted loss function; is the large model parameter; is the total number of tasks; Index for the task; For the The weight of each task; For the domain datasets; is the input sequence, Output sequence for the target; The target output sequence tokens; for All previous tokens; Predict probability distributions for the model; is the target sequence length; Based on the large model parameters obtained after training in the second stage, by minimizing the weighted loss function To train the large model and finally generate the large model parameters : ; Where, are the parameters of the large model after training obtained in the second stage.
[0015] An embodiment of the present invention further provides a large model active tool calling system based on multi-step reasoning, which is used to implement the large model active tool calling method based on multi-step reasoning described above, specifically comprising: A reverse training data construction module is used to generate a structured tool description set by collecting tool information, and automatically generate executable code samples and corresponding natural language explanations based on the tool description set to construct a reverse training data set; The forward training data construction module is used to label the necessary use of tools in real-world problems, prompt the large model to generate a multi-step reasoning path that combines natural language and code, verify and filter the correctness of the reasoning path, and build a forward training dataset; The large model three-stage training module is used to perform three-stage training on the large model based on the reverse training data set and the forward training data set, including: Phase 1: Using the reverse training dataset to train the basic large model, so as to inject tool usage knowledge and enhance the ability of the large model to use the tool, and obtain the trained large model parameters; Phase 2: Based on the trained large model parameters obtained in the first phase, the forward training dataset is used to train the awareness of the large model usage tool to obtain the trained large model parameters; The third stage: Based on the trained large model parameters obtained in the second stage, multi-domain task data is integrated and the large model is fine-tuned through multi-task learning to achieve cross-domain generalization.
[0016] An embodiment of the present invention also provides an electronic device, which includes a processor, a memory and a bus system, wherein the processor and the memory are connected through the bus system, the memory is used to store instructions, and the processor is used to execute the instructions stored in the memory to implement the above-mentioned large model active tool calling method based on multi-step reasoning.
[0017] It can be seen from the above technical solutions that the present invention has the following beneficial effects: (1) Build a systematic training mechanism for tool usage capabilities: Existing models mostly rely on passive invocation methods such as prompting or code generation, lacking the ability to actively learn tool usage. This invention uses a three-stage training framework (tool knowledge, tool awareness, and multi-task generalization) to enable the model to evolve from "knowing how to use tools" to "knowing when and how to use them," significantly improving the intelligence and effectiveness of tool invocation.
[0018] (2) Significantly improved the accuracy and robustness of the model on various reasoning tasks: Experimental results show that the proposed method (TAT) comprehensively outperforms open source models and existing tool integration solutions on datasets such as mathematics (MATH), science (SciBench), and tabular financial reasoning (DocMath). For example, the accuracy of answering questions on GSM8K reached 79.7%, which is close to that of closed-source models such as GPT-4. At the same time, the average accuracy is significantly higher than that of dedicated methods such as TORA and MathCoder2.
[0019] (3) Comprehensively optimize tool call success rate and code execution capability: The code execution rate of the proposed method (TAT) reaches or exceeds 90% for complex tasks, which is much higher than that of traditional baseline methods. This demonstrates that the model can not only correctly understand the tool functions but also generate structurally correct and semantically clear code, improving verifiability and practical deployment feasibility.
[0020] (4) Support for generalized reasoning capabilities across multiple domains and tasks: Through the multi-task fine-tuning mechanism, the TAT method can transfer the tool usage strategies learned during training to other tasks or domains, which has good scalability and practical deployment value. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] In order to more clearly illustrate the implementation cases of the present invention or the technical solutions in the prior art, the following briefly describes the drawings required for use in the embodiments. By referring to the drawings, the features and advantages of the present invention will be more clearly understood. The drawings are for illustration only and should not be construed as limiting the present invention in any way. Those skilled in the art can derive other drawings based on these drawings without inventive effort. Among them: Figure 1 A flowchart of a large model active tool calling method based on multi-step reasoning provided by the present invention; Figure 2 This is a schematic diagram showing the relationship between the usage and accuracy of tools in different fields in the present invention; Figure 3 A block diagram of a large model active tool calling system based on multi-step reasoning provided by the present invention. DETAILED DESCRIPTION
[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.
[0023] Example 1: In order to solve the problem that the large language model in the prior art lacks the awareness and ability to actively call the tool library, resulting in low code execution rate and insufficient problem solving efficiency in complex reasoning tasks, such as Figure 1 As shown in Figure 2, this paper proposes a large-model active tool invocation method based on multi-step reasoning. This method aims to train large models (LLMs) using forward and reverse constructed datasets, allowing LLMs to learn tool invocation habits and capabilities from the data. This method is called Tool-Awareness Training (TAT). The method includes the following three steps: S1: Generate a structured tool description set by collecting tool information, automatically generate executable code samples and corresponding natural language explanations based on the tool description set, and build a reverse training dataset; S2: By labeling the necessity of using tools in real-world problems, the large model is prompted to generate a multi-step reasoning path that combines natural language and code, verify and filter the correctness of the reasoning path, and build a positive training dataset; S3: Perform three-stage training on the large model based on the reverse training dataset and the forward training dataset, including: Phase 1: Use the reverse training dataset to train the basic large model to inject tool usage knowledge and enhance the large model's ability to use tools, thereby obtaining the trained large model parameters. Phase 2: Based on the trained large model parameters obtained in the first phase, the forward training dataset is used to train the large model's awareness of tool usage and obtain the trained large model parameters. The third stage: Based on the trained large model parameters obtained in the second stage, multi-domain task data is integrated and the large model is fine-tuned through multi-task learning to achieve cross-domain generalization.
[0024] From the above technical solution, it can be seen that the present invention proposes a large-model active tool calling method based on multi-step reasoning. By constructing a reverse training dataset (collecting tool information to generate structured descriptions and code samples, and injecting tool usage knowledge) and a forward training dataset (labeling tool usage necessity labels, generating and verifying multi-step reasoning paths, and enhancing tool usage awareness), the large model is trained in three stages based on these two types of datasets (the first stage uses reverse data to enhance tool usage capabilities, the second stage uses forward data to enhance tool calling awareness, and the third stage integrates multi-domain data to achieve cross-domain generalization), thereby solving the problem of insufficient active tool calling capabilities of large models and improving the code execution rate and problem-solving efficiency of complex reasoning tasks.
[0025] This paper proposes a structured, multi-step reasoning paradigm, CodePlan, that deeply integrates natural language reasoning with executable code. Its core principle is to decompose reasoning tasks into multiple subtasks, each consisting of natural language reasoning steps and executable code snippets. The output of these code snippets serves as the input for subsequent subtasks until the final answer is generated. This paradigm overcomes the limitations of traditional Chain of Thought (CoT) methods, which rely solely on natural language reasoning, and also addresses the lack of systematic code generation in tool integration approaches, achieving the goals of "explicit reasoning logic and structured tool calls."
[0026] In step S1, by collecting tool information, a structured tool description set is generated, and executable code samples and corresponding natural language explanations are automatically generated based on the tool description set to construct a reverse training dataset. .
[0027] Specifically, reverse training data generation automatically generates labeled data in an unsupervised manner. Its core is to use technical means to build paired data between structured tool knowledge and reasoning processes, providing the large model with prior knowledge for tool invocation. This is divided into the following three key steps, each of which is closely linked to form a complete data generation chain: Step 1: Crawling and structuring Python tool library information Use web crawlers to collect tool details for Python libraries (such as those that solve the shortest path problem) from publicly available resources, including raw information such as function descriptions, parameter definitions, and usage examples. For example, crawl the parameters (such as graph structure, starting point, and end point) and usage scenarios of the shortest_path function in the NetworkX library. This information is initially unstructured and requires further processing to form a format that the model can understand.
[0028] Step 2: Use LLMs to optimize tool information and generate structured descriptions Use a large language model to optimize the original crawled information: 1. Function label assignment: Automatically classify each tool (e.g., shortest_path) by its field (e.g., graph theory) and functional category (e.g., path search); 2. Concise description generation: Convert lengthy documents into concise natural language descriptions, such as "shortest_path(G, source, target) is used to calculate the shortest path from source to target in graph G and return a node sequence."
[0029] The optimized information forms a structured data set, providing a standardized knowledge base for the subsequent generation of code samples and questions.
[0030] Step 3: Generate code samples, filter and pair natural language explanations 1. Code Sample Generation and Screening: Based on a structured tool set, code snippets demonstrating tool usage (e.g., complete code for calculating the shortest path using NetworkX) are generated. LLMs are trained through few-shot learning to mimic high-quality code patterns. A screening mechanism is used to remove redundant or repetitive examples. Execution verification is also used to eliminate invalid code with no output, ensuring the executable and diverse nature of the examples.
[0031] 2. Pairing natural language explanations with questions: For the selected code examples, LLMs are used to generate corresponding natural language explanations, detailing the reasoning process of the tool calls (e.g., "Because the problem requires computing the shortest path between two points, the shortest_path function is called, inputting graph data and node coordinates"). The explanations are combined with the code to form a structured "question-code-reasoning" solution, which forms the reverse training dataset.
[0032] This method uses crawlers and LLMs to automatically collect, optimize, and match data, reducing annotation costs and ensuring strong data scalability. The reverse data covers a wide range of tool usage scenarios, helping models learn the tool's "function-parameter-application scenario" mapping, addressing the lack of model-tool knowledge in existing methods. The generated structured data provides high-quality prior knowledge for tool invocation during subsequent model training. For example, in the shortest path problem, the model can directly learn to call mature library functions rather than writing inefficient custom code.
[0033] In step S2, by marking the necessary labels of the tools used in real-world problems, the large model is prompted to generate a multi-step reasoning path that combines natural language and code, verify and filter the correctness of the reasoning path, and build a forward training dataset. .
[0034] Specifically, the forward training data generation phase uses a three-stage process to improve the tool usage awareness of large models. The specific steps and technical details are as follows: Step 1: Labeling real-world problems and determining the necessity of using tools 1. Double labeling method to obtain tool usage labels Method 1: Comparative annotation method of CoT (chain of thought) and tool integration solution Through multiple sampling (e.g., running a large model multiple times to generate different reasoning paths), the accuracy of pure Chain of Thought (CoT) reasoning is compared with that of tool-integrated reasoning. If the tool-integrated approach achieves higher accuracy, the problem is marked as "Needs Tools." For example, in a math word problem, if the accuracy of using the Python sympy library to solve the equation is higher than that of pure text reasoning, the problem is marked as Needs Tools.
[0035] Method 2: LLMs confidence judgment annotation For complex problems (such as cross-disciplinary professional tasks), when comparative annotation methods may be ineffective, it is necessary to use LLMs' own confidence assessment tools. LLMs should output judgment results (such as "need to call a statistical library to calculate probability distribution") and explain the reasoning process, for example, "Because the problem involves multidimensional data correlation analysis, manual calculations are prone to errors, so it is recommended to call the scipy.stats library."
[0036] 2. Mark the target Construct a "question-tool usage label" dataset to clarify the necessity of tool calling in different task scenarios and provide supervision signals for the model to learn "when to call tools".
[0037] Step 2: Prompt LLMs to generate tool-aware reasoning paths 1. Generate CoT and CodePlan solutions CoT Path (Chain of Trace Reasoning): Directly prompts LLMs to deduce the solution to the problem step by step in natural language, such as "First calculate the mean of the data set, then compare the differences between the groups..."
[0038] CodePlan path (structured code + natural language): LLMs are guided by prompts to convert the CoT reasoning process into a structured format of "natural language subtasks + executable code". For example: Subtask 1: “Import data and check for missing values” Code: import pandas as pd;df=pd.read_csv('data.csv'); print(df.isnull().sum()).
[0039] Subtask 2: "Calculate the mean of each column" Code: means = df.mean(); print(means).
[0040] 2. Generate logic The CoT path focuses on the consistency of natural language reasoning, while the CodePlan path emphasizes the executability of tool calls. The combination of the two enables large models to learn both reasoning logic and tool application methods.
[0041] Step 3: Solution Verification and Filtering 1. CoT solution filtering Filtering is performed by comparing the consistency of the final answers of the CoT paths generated by different LLMs. For example, if multiple CoT paths reach the same conclusion (such as "the shortest path length is 5"), the solution is retained; if the conclusions are contradictory, it is eliminated.
[0042] 2. CodePlan solution verification Directly execute the code snippets in the CodePlan and verify that the output matches the problem's expectations. For example, after executing the code to calculate the shortest path, check that the output is consistent with the actual shortest path in the graph structure in the problem. Only retain samples where the code is executable and the results are correct.
[0043] 3. Filter targets Eliminate invalid reasoning (such as logical contradictions) and unexecutable code (such as syntax errors) to ensure the high quality of the forward training dataset and prevent large models from learning incorrect tool call patterns.
[0044] This invention uses dual labeling to cover both simple and complex tasks, enabling large models to independently determine tool requirements based on problem difficulty, avoiding over- or under-calling. CodePlan breaks down the reasoning process into "natural language-to-code" interaction units, forcing the model to clearly understand the contextual logic of tool calls, improving reasoning interpretability and execution accuracy. Validated positive training data directly demonstrates successful examples of "tool call-to-problem solving," helping large models learn the decision-making logic of "why, when, and how" tools are used from data.
[0045] In step S3, the large model is trained in three stages based on the reverse training dataset and the forward training dataset.
[0046] Specifically, based on sufficient training data, this paper proposes a three-stage progressive training strategy to enhance model capabilities. This strategy follows a phased, progressive logic, with each stage building on the previous stage's training results to continuously optimize the model's inference efficiency and accuracy for complex tasks. Each stage focuses on minimizing cross-entropy loss, and through iterative optimization of model parameters, it systematically improves tool invocation capabilities.
[0047] Assume question answering training data set ,in represents the input sequence, represents the corresponding target output sequence ( is the sequence length). The model parameters are expressed as The training objective function of each stage can be uniformly expressed as: ; Where, is the training objective function; is the large model parameter; is the training data set; is the input sequence, Output sequence for the target; The target output sequence tokens; for All previous tokens; Predict probability distributions for the model; is the target sequence length.
[0048] Furthermore, the present invention performs three-stage training on the large model, including: Phase 1: Tool Knowledge Injection Using reverse training dataset Train the basic large model, the initial parameters are , in order to inject tool usage knowledge and enhance the ability of the large model to use the tool, obtain the large model parameters after training, the large model parameters after training obtained in the first stage for: ; Where, is the reverse training loss function.
[0049] Phase 2: Strengthening tool awareness Based on the large model parameters obtained after training in the first stage , using the forward training dataset The awareness of using tools to train large models, obtain the parameters of the large model after training, the parameters of the large model after training obtained in the second stage for: ; Where, is the forward training loss function.
[0050] Stage 3: Cross-task generalization The large model parameters after training obtained in the second stage Based on , rejection sampling and multi-task learning are applied to further enhance adaptability. Specifically, let Represents multiple task datasets, each of which corresponds to a specific field or problem type, and combines these task datasets into a unified multi-task objective, whose weighted loss function is: ; Where, is the weighted loss function; is the large model parameter; is the total number of tasks; Index for the task; For the The weight of each task; For the domain datasets; is the input sequence, Output sequence for the target; The target output sequence tokens; for All previous tokens; Predict probability distributions for the model; is the target sequence length.
[0051] Furthermore, based on the trained large model parameters obtained in the second stage , by minimizing the weighted loss function To train the large model and finally generate the large model parameters : .
[0052] To evaluate the effectiveness of our method (TAT), we conducted a series of experiments on various mathematical, scientific, and tabular reasoning tasks. These tasks comprehensively tested the model's tool usage capabilities, reasoning accuracy, and code executableness. We used the Meta-Llama-3-8B-Instruct model as the backbone model for training, and employed several standard datasets, including GSM8K, MATH, SciBench, and DocMath-Eval.
[0053] The evaluation compared three types of models: closed-source LLMs (such as GPT-4, Claude-2, and Gemini-1.5), open-source LLMs (such as Mistral, GLM, and LLaMA3), and tool-ensemble learning models (such as TORA, MAmmoTH2, and MathCoder2). Evaluation metrics included answer accuracy, code executable rate, and tool usage score (Tool Usage Score). The Tool Usage Score comprehensively considers the initiative and success rate of tool invocation, as well as whether it improves the quality of the final answer.
[0054] TAT improves reasoning accuracy: As shown in Table 1, TAT achieved either optimal or suboptimal accuracy on multiple tasks. It performed particularly well on the math task MATH and the financial spreadsheet task DMSimpShort, reaching 37.7% and 78% respectively, surpassing even several closed-source models, including Claude2. TAT demonstrated superior adaptability to complex tasks, particularly compared to other tool integration methods such as TORA and MathCoder2.
[0055] Table 1 Inference accuracy
[0056] TAT improves code execution: As shown in Table 2, TAT achieves the highest or near-highest code execution success rates across all datasets. For example, on GSM8K, the code execution rate exceeds 90%, while on the more challenging MATH, it remains above 60%. Our model outperforms the baseline on all datasets, achieving the highest code execution rate by effectively handling task complexity.
[0057] Table 2 Code executable and tool usage scores
[0058] TAT demonstrates selective tool invocation: Analysis of tool usage scores in Table 2 shows that TAT avoids unnecessary tool invocation in simple tasks and proactively invokes appropriate tools in complex ones. For example, on DMSimpShort, the tool usage score is 0.0 (indicating that the model actively chooses not to invoke tools), while on Math and SciBench, it demonstrates active and effective tool usage. This selective invocation strategy surpasses the over-reliance on tools found in methods like TORA, achieving a better balance between generalization and accuracy.
[0059] The effectiveness of the phased training strategy was verified: In phased ablation experiments, TAT's three-phase training framework was gradually stripped away. The results are shown in Table 3. The first phase (tool initialization) improved tool usage capabilities but limited reasoning accuracy. The second phase (tool awareness), after introducing selective tool usage, significantly improved the accuracy of simple tasks and effectively reduced redundant calls. The third phase (multi-task fine-tuning) combined the advantages of the first two phases to achieve the best overall performance. For example, on GSM8K, the accuracy of answering questions increased from 41.1% to 79.7%.
[0060] Table 3 Performance of multi-stage training method
[0061] TAT outperforms retrieval enhancement methods in complex tasks such as mathematics: This paper further compared the effectiveness of tool knowledge utilization using few-shot ICL, RAG retrieval enhancement, and TAT fine-tuning. As shown in Table 4, the RAG method performed poorly when processing complex documents, resulting in decreased accuracy. The fine-tuning method of this paper decouples the "tool retrieval" from "tool execution" processes, demonstrating greater adaptability when handling complex reasoning tasks and outperforming the RAG method in both accuracy and efficiency.
[0062] Table 4 Comparison of different tool integration methods
[0063] TAT showed good tool use regulation ability in multi-tasking: Figure 2 Analysis shows that in tasks like GSM8K and DMSimpShort, moderate tool usage leads to the highest accuracy. In SciBench, more frequent tool usage leads to better model performance. These results suggest that improving performance requires properly controlling the model's dependency on tools based on the task type, achieving "tool dependency control."
[0064] In summary, TAT systematically improves the performance of large language models in complex reasoning tasks through a three-stage training strategy from "tool usage, to tool awareness, to cross-task adaptation", and achieves leading results on multiple public datasets, fully demonstrating its potential as a tool-enhanced LLMs training framework.
[0065] Example 2: Figure 3 As shown, the present invention provides a large model active tool calling system based on multi-step reasoning, which is used to implement the large model active tool calling method based on multi-step reasoning of the above embodiment 1, specifically including: The reverse training data construction module 100 is used to generate a structured tool description set by collecting tool information, and automatically generate executable code samples and corresponding natural language explanations based on the tool description set to construct a reverse training data set; The forward training data construction module 200 is used to label the necessity of using tools in real-world problems, prompt the large model to generate a multi-step reasoning path that combines natural language and code, verify and filter the correctness of the reasoning path, and construct a forward training dataset; The large model three-stage training module 300 is used to perform three-stage training on the large model based on the reverse training dataset and the forward training dataset, including: Phase 1: Use the reverse training dataset to train the basic large model to inject tool usage knowledge and enhance the large model's ability to use tools, thereby obtaining the trained large model parameters. Phase 2: Based on the trained large model parameters obtained in the first phase, the forward training dataset is used to train the large model's awareness of tool usage and obtain the trained large model parameters. The third stage: Based on the trained large model parameters obtained in the second stage, multi-domain task data is integrated and the large model is fine-tuned through multi-task learning to achieve cross-domain generalization.
[0066] A large model active tool calling system based on multi-step reasoning in this embodiment is used to implement the aforementioned large model active tool calling method based on multi-step reasoning. Therefore, the specific implementation method of the large model active tool calling system based on multi-step reasoning can be seen in the embodiment part of the large model active tool calling method based on multi-step reasoning in the previous text. For example, the reverse training data construction module 100, the forward training data construction module 200, and the large model three-stage training module 300 are respectively used to implement steps S1, S2, and S3 in the above-mentioned large model active tool calling method based on multi-step reasoning. Therefore, its specific implementation method can refer to the description of the corresponding embodiments of each part. In order to avoid redundancy, it will not be repeated here.
[0067] Embodiment 3: An embodiment of the present invention provides a computer storage medium, which stores a computer software product. The computer software product includes a number of instructions for enabling a computer device to execute the above-mentioned large model active tool calling method based on multi-step reasoning.
[0068] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0069] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0070] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.
[0071] Obviously, the above embodiments are merely examples for clarity of explanation and are not intended to limit the implementation methods. Those skilled in the art will appreciate that other variations or modifications can be made based on the above description. It is not necessary and impossible to enumerate all implementation methods here. Obvious variations or modifications arising therefrom remain within the scope of protection of the present invention.
Claims
1. A large model active tool calling method based on multi-step reasoning, characterized in that: include: By collecting tool information, a structured tool description set is generated, and based on the tool description set, executable code samples and corresponding natural language explanations are automatically generated to construct a reverse training dataset; By labeling the necessity of using tools in real-world problems, the large model is prompted to generate a multi-step reasoning path that combines natural language and code, verify and filter the correctness of the reasoning path, and build a positive training dataset; The large model is trained in three stages based on the reverse training dataset and the forward training dataset, including: Phase 1: Using the reverse training dataset to train the basic large model, so as to inject tool usage knowledge and enhance the ability of the large model to use the tool, and obtain the trained large model parameters; Phase 2: Based on the trained large model parameters obtained in the first phase, the forward training dataset is used to train the awareness of the large model usage tool to obtain the trained large model parameters; The third stage: Based on the trained large model parameters obtained in the second stage, multi-domain task data is integrated and the large model is fine-tuned through multi-task learning to achieve cross-domain generalization.
2. The large model active tool calling method based on multi-step reasoning according to claim 1 is characterized in that: The reverse training data set construction specifically includes: Collect description information of tools in the Python library, including functions, parameters, and usage instructions; Use the description information of the large model optimization tool to generate function labels and concise function summaries to obtain the optimized tool set; Based on the optimized toolset, code snippets are generated to illustrate examples of tool usage, and few-shot learning and filtering mechanisms are used to reduce redundancy and increase the diversity of usage examples.
3. The large model active tool calling method based on multi-step reasoning according to claim 1 is characterized in that: The method of labeling the tools used in real-world problems using necessity labels includes: A dual labeling method is used to label the necessity of tool use in real-world problems, including comparative labeling and confidence labeling. The comparative labeling method generates a thinking chain solution and a tool integration solution for the same problem. When the tool integration solution has a higher reasoning accuracy, it is marked as "needing tool"; Confidence annotation method: Use large model output tools to call the necessary confidence scores and explanations.
4. The large model active tool calling method based on multi-step reasoning according to claim 1 is characterized in that: The method for prompting the large model to generate a multi-step reasoning path that combines natural language and code includes: Decompose the reasoning task into multiple subtasks, each of which contains natural language reasoning steps and executable code snippets; The output of the code snippet is used as the input of subsequent subtasks until the final answer is generated.
5. The large model active tool calling method based on multi-step reasoning according to claim 1 is characterized in that: The large model is trained in three stages, and the training objective function of each stage is: ; Where, is the training objective function; is the large model parameter; is the training data set; is the input sequence, Output sequence for the target; The target output sequence tokens; for All previous tokens; Predict probability distributions for the model; is the target sequence length.
6. The large model active tool calling method based on multi-step reasoning according to claim 1 is characterized in that: The parameters of the large model after training obtained in the first stage are: ; Where, are the parameters of the large model after training obtained in the first stage; are the initial parameters of the large model; is the reverse training loss function; is the reverse training dataset.
7. The large model active tool calling method based on multi-step reasoning according to claim 1 is characterized in that: The parameters of the large model after training obtained in the second stage are: ; Where, are the parameters of the large model after training obtained in the second stage; are the parameters of the large model after training obtained in the first stage; is the forward training loss function; is the positive training dataset.
8. The large model active tool calling method based on multi-step reasoning according to claim 1 is characterized in that: The method of integrating multi-domain task data based on the trained large model parameters obtained in the second stage and fine-tuning the large model through multi-task learning to achieve cross-domain generalization includes: make Represents multiple task datasets, each of which corresponds to a specific field or problem type, and combines these task datasets into a unified multi-task objective, whose weighted loss function is: ; Where, is the weighted loss function; is the large model parameter; is the total number of tasks; Index for the task; For the The weight of each task; For the domain datasets; is the input sequence, Output sequence for the target; The target output sequence tokens; for All previous tokens; Predict probability distributions for the model; is the target sequence length; Based on the large model parameters obtained after training in the second stage, by minimizing the weighted loss function To train the large model and finally generate the large model parameters : ; Where, are the parameters of the large model after training obtained in the second stage.
9. A large model active tool calling system based on multi-step reasoning, characterized in that: The system is used to implement the large model active tool calling method based on multi-step reasoning according to any one of claims 1 to 8, specifically comprising: A reverse training data construction module is used to generate a structured tool description set by collecting tool information, and automatically generate executable code samples and corresponding natural language explanations based on the tool description set to construct a reverse training data set; The forward training data construction module is used to label the necessary use of tools in real-world problems, prompt the large model to generate a multi-step reasoning path that combines natural language and code, verify and filter the correctness of the reasoning path, and build a forward training dataset; The large model three-stage training module is used to perform three-stage training on the large model based on the reverse training data set and the forward training data set, including: Phase 1: Using the reverse training dataset to train the basic large model, so as to inject tool usage knowledge and enhance the ability of the large model to use the tool, and obtain the trained large model parameters; Phase 2: Based on the trained large model parameters obtained in the first phase, the forward training dataset is used to train the awareness of the large model usage tool to obtain the trained large model parameters; The third stage: Based on the trained large model parameters obtained in the second stage, multi-domain task data is integrated and the large model is fine-tuned through multi-task learning to achieve cross-domain generalization.
10. An electronic device, characterized in that: The electronic device includes a processor, a memory and a bus system, the processor and the memory are connected through the bus system, the memory is used to store instructions, and the processor is used to execute the instructions stored in the memory to implement the large model active tool calling method based on multi-step reasoning as described in any one of claims 1 to 8.
Citation Information
Patent Citations
Natural language processing task execution method and device, natural language processing task model training method and device, and medium
CN118536605A
Intelligent question and answer method based on cooperation of large language model and knowledge graph
CN118797017A
Agent-based collaborative reasoning and action large language model learning method
CN119128100A
Big language model logical reasoning method and device, equipment, storage medium and product
CN119378691A
Method and system for synthesizing reasoning data
CN119476479A
Cited By
Intelligent agent evolution method based on GRPO and multi-stage verification
CN120975134A
An agent evolution method based on GRPO and multi-stage verification
CN120975134B
Interpretable financial suggestion generation method and device, electronic equipment and storage medium
CN121902980A
Large language model agent construction method and device based on graph-guided task decomposition
CN122133826A
A method and device for constructing a large language model agent based on graph-guided task decomposition
CN122133826B
