General-purpose tool collaboration and fine-tuning learning system and method for large models

Abstract

This invention provides a general-purpose tool collaboration and refined learning system and method for large models, enhancing their ability to handle complex tasks. The system includes a dynamic tool combination mechanism module, a unified interface module based on language instructions, a general-purpose tool refined learning module, and a comprehensive reasoning module for execution process and result information. The dynamic tool combination mechanism module decomposes the task corresponding to the instruction to be processed into several tool-level sub-tasks. The unified interface module based on language instructions enables collaborative invocation between multiple types of general-purpose tools through a general-purpose tool interface, achieving functional complementarity between different tools. The refined learning module for general-purpose tools fully leverages the proprietary skills of each tool. The comprehensive reasoning module integrates and reasones the execution process information and processing results of several tool-level sub-tasks to obtain the final answer to the task instruction. This system demonstrates significant effectiveness in intelligent question-answering scenarios.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to a general-purpose collaborative and refined learning system and method for large models. Background Technology

[0002] In recent years, pre-trained language models (PLMs) have brought a series of breakthroughs to the fields of natural language processing and artificial intelligence, becoming a mainstream technological paradigm in AI. Currently, large-scale PLMs are considered internationally as the "foundational model" for achieving general artificial intelligence, and are also known domestically as "Big Models" (BMs). Existing large-model technologies, represented by ChatGPT and GPT-4, can perform AI tasks such as article writing, dialogue question answering, and automatic programming. However, their performance in many practical applications is still unsatisfactory, especially in complex task processing, interpretability, credibility, and handling of professional knowledge / skills. Large models are prone to generating factual errors or unreliable text content, and they show significant shortcomings in tasks requiring symbolic knowledge reasoning and data computation. Furthermore, since large models are trained on data collected before a specific point in time, the model knowledge is predetermined and lacks real-time update capabilities. Summary of the Invention

[0003] This invention provides a general-purpose tool collaboration and refined learning system and method for large models, addressing the shortcomings of existing technologies in generating content from large models, such as the generation of factual errors or unreliable text, and significant deficiencies in tasks requiring symbolic knowledge reasoning and data computation. Furthermore, since large models are trained on data collected before specific time points, their knowledge is predetermined and lacks real-time updating capabilities. By enabling collaborative invocation between multiple types of general-purpose tools, functional complementarity between different tools is achieved. Moreover, through refined learning techniques of these general-purpose tools, the proprietary skills of each tool are fully utilized, thereby improving the large model's ability to handle complex tasks. This has shown significant effectiveness in intelligent question-answering scenarios.

[0004] This invention provides a general-purpose tool collaboration and refined learning system for large models, comprising: a dynamic combination mechanism module for different tools connected to the input end and the output end of a large model, used to decompose the task corresponding to the task instruction to be processed into several tool-level sub-tasks through the large model, construct a general-purpose tool call graph based on the several tool-level sub-tasks and a general-purpose tool interface, and establish a dynamic routing mechanism for the general-purpose tools through reinforcement learning and instruction learning; and a unified interface module based on language instructions connected to the input end and the output end of the dynamic combination mechanism module for different tools, used to perform language instruction-based general-purpose tool interface processing according to the several tool-level sub-tasks, the general-purpose tool call graph, and the dynamic routing mechanism. The system includes: a general-purpose tool refinement learning module (whose input is connected to the output of the language instruction-based unified interface module), used to process several tool-level sub-tasks according to the general tools corresponding to the language instruction-based general tool interface, and obtain the processing results of several tool-level sub-tasks; the general-purpose tool refinement learning module is one or more general-purpose tool modules that are updated and maintained in real time; and an execution process and result information integrated reasoning module (whose input is connected to the output of the general-purpose tool refinement learning module and whose output is connected to the input of the large model), used to integrate and reason about the execution process information and processing results of several tool-level sub-tasks to obtain the final answer to the task instruction.

[0005] According to the present invention, a general-purpose tool collaborative and refined learning system for large models is provided. The general-purpose tool refined learning module includes: an interactive web browser tool refined learning module, used to process the pending tool-level subtasks with the tool call type of web browser according to the interactive web browser tool, and obtain the processing result of the pending tool-level subtasks with the tool call type of web browser; a knowledge graph tool refined learning module, used to process the pending tool-level subtasks with the tool call type of knowledge graph according to the knowledge graph tool, and obtain the processing result of the pending tool-level subtasks with the tool call type of knowledge graph; and a tool interface API tool refined learning module, used to process the pending tool-level subtasks with the tool call type of tool interface API according to the API tool, and obtain the processing result of the pending tool-level subtasks with the tool call type of tool interface API.

[0006] According to the present invention, a general-purpose tool collaborative and refined learning system for large models is provided. The refined learning module of the interactive web browser tool includes: a fact retrieval acceleration module, used to decompose the task corresponding to the task instruction to be processed according to a tree structure to obtain several action-level sub-tasks after tree structure decomposition, and predict the retrieval query of several action-level sub-tasks after tree structure decomposition; a fact extraction module, used to extract fact information related to several action-level sub-tasks from the current page text of the interactive web browser, so as to obtain the processing result of the tool-level sub-task to be processed with the tool call type being web browser; the current page of the interactive web browser is retrieved based on the retrieval query of several action-level sub-tasks.

[0007] According to the present invention, a general-purpose tool collaborative and refined learning system for large models is provided. The refined learning module of the interactive web browser tool further includes a visual information fusion module, which is used to input text information and visual information into the large model through a structured input method, so as to generate text information retrieval query, new fact extraction and next action for several action-level sub-tasks.

[0008] According to the present invention, a general-purpose tool collaboration and refined learning system for large models is provided. The refined learning module for knowledge graph tools includes: an initial instruction definition module, used to construct prompts or few-sample examples of serialized atomic operations based on knowledge-based tasks; an atomic operation prediction module, used to predict serialized atomic operations based on the prompts or few-sample examples of the serialized atomic operations and the current state of the knowledge graph, using an atomic operation prediction model of the large model; and a query language conversion and execution module, used to convert the serialized atomic operations into the serialized knowledge graph interactive query language SPARQL; and execute the knowledge graph query language SPARQL to obtain serialized interactive query results, thereby obtaining the processing results of the pending tool-level subtasks with the tool call type of knowledge graph.

[0009] According to the present invention, a general-purpose tool collaborative and refined learning system for large models is provided. The refined learning module of the knowledge graph tool further includes a reward module, which is used to optimize the atomic operation prediction model of the large model based on the incentive model of the large model according to the sequential interactive query results.

[0010] According to the present invention, a general-purpose tool collaboration and refined learning system for large models is provided. The refined learning module for tool interface (API) tools includes: an API customization module based on user needs, used to analyze and collect a set of tool interface APIs related to the task corresponding to the task instruction; an API integration module based on a unified interface, used to construct a unified API repository based on the set of tool interface APIs, the API repository integrating multiple API interfaces that can meet different user needs and establishing an API hierarchy; an API retrieval module based on dynamic needs, used to perform dynamic need API retrieval based on the unified API repository according to the API hierarchy mechanism, and obtain API retrieval results; and a model learning module based on API pre-training, used to perform model learning based on API pre-training according to the API retrieval results, so as to obtain the processing results of the unprocessed tool-level subtasks whose tool call type is tool interface API tools.

[0011] According to the present invention, a general tool collaborative and refined learning system for large models is provided. The tool interface API tool refined learning module further includes a few-shot generalized API tool manual learning module, which is used to update and maintain the unified API repository and optimize the API level. Based on the updated and maintained unified API repository and the optimized API level, few-shot generalized tool manual learning is performed so that the few-shot generalized API tool manual can be used for on-demand retrieval.

[0012] The present invention provides a general tool collaborative and refined learning system for large models, which further includes: a unified interface module based on API instructions, used to understand API tool manuals, obtain easily understandable API tool manual information for large modules, and call the required API tools for the pending tool-level subtasks based on the API tool manual information and the tool call type being a tool interface API tool.

[0013] This invention also provides a general tool collaboration and refined learning method for large models. This method is applicable to the aforementioned general tool collaboration and refined learning system for large models, and includes: a dynamic combination mechanism module for different tools that decomposes the task corresponding to the task instruction to be processed into several tool-level sub-tasks through the large model; constructs a general tool call graph based on the several tool-level sub-tasks and a general tool interface; and establishes a dynamic routing mechanism for general tools through reinforcement learning and instruction learning; a unified interface module based on language instructions that calls the general tool interface based on language instructions according to the several tool-level sub-tasks, the general tool call graph, and the dynamic routing mechanism; a general tool refined learning module that processes the several tool-level sub-tasks according to the general tools corresponding to the general tool interfaces based on language instructions, obtaining the processing results of the several tool-level sub-tasks; the general tool refined learning module is one or more general tool modules that are updated and maintained in real time; and an execution process and result information integrated reasoning module that integrates and reasones the execution process information and processing results of the several tool-level sub-tasks to obtain the final answer to the task instruction.

[0014] This invention provides a general-purpose tool collaboration and refined learning system and method for large models, enhancing their ability to handle complex tasks. The system includes a dynamic tool combination mechanism module, a unified interface module based on language instructions, a general-purpose tool refined learning module, and a comprehensive reasoning module for execution process and result information. The dynamic tool combination mechanism module decomposes the task corresponding to the instruction to be processed into several tool-level sub-tasks. The unified interface module based on language instructions enables collaborative invocation between multiple types of general-purpose tools through a general-purpose tool interface, achieving functional complementarity between different tools. The refined learning module for general-purpose tools fully leverages the proprietary skills of each tool. The comprehensive reasoning module integrates and reasones the execution process information and processing results of several tool-level sub-tasks to obtain the final answer to the task instruction. This system demonstrates significant effectiveness in intelligent question-answering scenarios. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0016] Figure 1 This is a schematic diagram of the structure of the general-purpose collaborative and refined learning system for large models provided by this invention;

[0017] Figure 2This is a schematic diagram of the general framework for tool learning for large models provided by this invention;

[0018] Figure 3 This is a diagram illustrating the overall technical framework of the general-purpose collaborative and refined learning system for large models provided by this invention.

[0019] Figure 4 This is a technical framework diagram of the general-purpose tool refined learning module provided by the present invention;

[0020] Figure 5 The images shown are screenshots of the interactive web browser simulation interface (left) and the set of supported actions (right) provided by this invention.

[0021] Figure 6 This is a framework diagram of the refined learning module for web browser tools provided by the present invention;

[0022] Figure 7 This is a technical framework diagram of the refined learning module for web browser tools that considers accelerating fact retrieval and integrating visual information, provided by the present invention.

[0023] Figure 8 This is an example of knowledge graph retrieval based on atomic serialization operations provided by the present invention (the left figure is an example of process record and the right figure is an example of intermediate steps in query operation);

[0024] Figure 9 This is a technical framework diagram of the refined learning module for knowledge graph tools provided by this invention;

[0025] Figure 10 This is a technical framework diagram of the API-oriented tool refinement learning module provided by the present invention;

[0026] Figure 11 The annotation platform provided by this invention unifies the annotation record diagrams used by various tools;

[0027] Figure 12 This is a screenshot of the interface of the tool learning data annotation platform provided by the present invention;

[0028] Figure 13 This is a flowchart illustrating the collaborative and refined learning method for general-purpose tools aimed at large models provided by this invention. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0030] Given that existing large-scale modeling techniques still fall short of expectations in many practical applications, particularly in complex task processing, interpretability, and credibility, large models suffer from a severe "hallucination" phenomenon when generating content. This means they are prone to generating factually incorrect, toxic, or unreliable text, and they significantly underperform in tasks requiring symbolic reasoning and data computation. Furthermore, large models are trained on data collected before specific time points, meaning their knowledge is predetermined and they lack real-time update capabilities.

[0031] Currently, AI machine systems lack the effective learning, reasoning, prediction, and planning capabilities of humans and animals. It is believed that "how to enable machines to learn proactively, observe and represent the world, learn to predict, and how to enable machines to execute long-term predictions and long-term plans by breaking down complex behaviors into lower-level behavioral sequences" is the main bottleneck in building autonomous machine intelligence.

[0032] To address the technical problems existing in current technologies, this invention draws on human cognitive behavior in using tools to construct a general-purpose tool collaboration and refined learning system technology for large models. This system enables large models to learn and use various general-purpose tools, leveraging their potential to help solve complex tasks. By teaching large models to collaboratively invoke and refine the use of external tools, the invention aims to further enhance the intelligence level of current large-scale models. This invention revolves around the overall framework of the general-purpose tool collaboration and refined learning system for large models, including a dynamic combination mechanism module 1 for different tools, a unified interface module 2 based on language instructions, a general-purpose tool refined learning module 3, and a comprehensive reasoning module 4 for execution process and result information. The technical framework of the general-purpose tool refined learning module 3 is described in detail. Finally, the general-purpose tool collaboration and refined learning platform for large models is introduced, demonstrating significant effectiveness in intelligent question-answering scenarios.

[0033] Please refer to Figure 1 , Figure 1 This is a schematic diagram of the structure of the general-purpose collaborative and refined learning system for large models provided by the present invention.

[0034] Please refer to Figure 2 , Figure 2 This invention provides a general framework diagram for tool learning for large models.

[0035] Please refer to Figure 3 , Figure 3 This is a diagram illustrating the overall technical framework of the general-purpose collaborative and refined learning system for large models provided by this invention.

[0036] This invention provides a general-purpose collaborative and refined learning system for large models, comprising:

[0037] The module 1, which connects the input end to the output end of the large model, is a dynamic combination mechanism module for different tools. It is used to decompose the task corresponding to the task instruction to be processed into several tool-level sub-tasks through the large model, construct a general tool call graph based on several tool-level sub-tasks and general tool interfaces, and establish a dynamic routing mechanism for general tools through reinforcement learning and instruction learning.

[0038] The input end is connected to the output end of the dynamic combination mechanism module 1 of different tools, which is a unified interface module 2 based on language instructions. This module is used to call the general tool interface based on language instructions according to several tool-level sub-tasks, a general tool call graph, and a dynamic routing mechanism.

[0039] The input end is connected to the output end of the language instruction-based unified interface module 2. The general tool refinement learning module 3 is used to process several tool-level sub-tasks according to the general tools corresponding to the language instruction-based general tool interface, and obtain the processing results of several tool-level sub-tasks. The general tool refinement learning module is one or more general tool modules that are updated and maintained in real time.

[0040] The input end is connected to the output end of the general tool refined learning module 3, and the output end is connected to the input end of the large model. The execution process and result information integrated reasoning module 4 is used to integrate and reason about the execution process information and processing results of several tool-level sub-tasks to obtain the final answer to the task instruction.

[0041] Specifically, a collaborative learning system with multiple tools: how to correctly invoke the capabilities of various tools and execute them collaboratively to solve complex tasks, and how to comprehensively reason and summarize the information obtained from multiple tools in combination with the model's own knowledge.

[0042] The multi-source nature of information in tool learning necessitates that models synthesize and organize the complex information they acquire and apply it to specific task scenarios. Different tools possess unique advantages in solving specific tasks. By enabling large models to master multiple tools, they can solve more complex tasks and improve their reasoning capabilities and intelligence. Key technical challenges to address include: dynamic combination modules of different tools, robust comprehensive reasoning of execution processes and results, and the implementation of multi-type tool interaction based on tool interfaces.

[0043] The system will be built upon two advanced tools: knowledge graphs (such as Wikidata, using the SPARQL query language) and web search engines, combined with modules for dynamic composition and integrated reasoning. The system will be divided into four modules: a dynamic composition module 1 for different tools; a unified interface module based on language instructions 2; a refined learning module for general tools 3; and a module for integrated reasoning based on execution process and result information 4. The specific scheme is as follows:

[0044] (1) Dynamic combination of different tools Module 1: In terms of dynamic combination of modules, a dynamic routing mechanism is adopted: 1) Complex tasks are decomposed into sub-tasks through a large model, and a call graph structure or flowchart between tools is constructed. A planning graph is defined as a set of vertices and edges. Node v can consist of task instructions, the original problem, sub-problems, tool descriptions, etc., and ε represents the edges between nodes. The starting node of an edge is a prerequisite for a reasoning step, while the ending node is the output of the reasoning step, i.e., an intermediate conclusion or answer. Then, reinforcement learning and instruction learning are used to model the dynamic routing mechanism of the modules. 2) Graph Neural Networks (GNNs) can also be used to model the planning graph structure between tools. By learning the relationships and interactions between tools through the GNN model, the dynamic combination of modules can be determined, enabling more accurate prediction of interactions between tools, thus achieving better task decomposition and combination. Secondly, reinforcement learning methods can be used to optimize the dynamic combination of modules. By introducing reward functions and policy networks, the large model can dynamically select the optimal tool combination based on different task requirements. This method can more flexibly respond to different task requirements and tool combination methods, thereby achieving better task efficiency and accuracy. Additionally, the instruction-based learning method allows users to directly provide instructions to the model, telling it how to combine tools to complete the task.

[0045] (2) Based on a unified language instruction interface module 2: This module implements multi-type tool interaction based on a unified interactive cognitive interface, enabling interaction between the large model and various tools. Here, a collaborative learning framework for a large model tool that uses multiple types of tools simultaneously is constructed based on single-tool learning models (BM-Web, BM-KG, BM-API, etc.). The main intelligent agent large model interacts with each single-tool learning model through human language instructions. Sub-tasks are obtained through a dynamic module combination process. Then, tool language instructions are constructed according to tool descriptions and task types. The single-tool learning models execute output feedback according to customized language instructions. The interaction is simple, and the fine-grained manipulation of tools by single-tool learning models can unleash the potential of individual tools. However, this is heavily dependent on the instruction understanding ability of the single-tool learning models and the task decomposition ability of the main model large model.

[0046] (3) General Tool Refinement Learning Module 3: How to define the model-tool interaction interface to adapt the model to specific tools, understand the meaning of the tools' executable operations, and thus output the correct decision sequence in complex environments. Tool learning needs to be grounded in specific yet universal tools. Here, it can be, but is not limited to, selecting Web browsers, knowledge graphs, and tool interface APIs as representative general tools to provide a system design scheme, realizing the large model's refined learning and use of these general tools. The key issues to be solved include: the tool refinement learning module BM-Web based on an interactive Web browser for complex problem decomposition, the tool refinement learning module BM-KG based on sequential knowledge graph query, and the tool refinement learning module BM-API oriented towards tool interface APIs.

[0047] (4) Execution Process and Result Information Integrated Reasoning Module 4: In the use of multiple types of tools in a large model, the accuracy and robustness of information results are crucial. Therefore, a series of measures are needed to improve the reliability and stability of the model's reasoning and execution processes. First, existing fact-verification datasets can be used to train the model to identify and judge the authenticity of tool feedback information, thus improving the model's accuracy and credibility. During execution, feedback control and error correction mechanisms can be used to correct errors and ignore irrelevant information to ensure the model's accuracy. For example, if the model's actions lead to negative rewards, the error correction mechanism can adjust the strategy to prevent similar actions from occurring in the future. This can minimize the possibility of erroneous decisions by the model. Finally, by combining a self-consistency thought chain to generate multiple combined process executions and reasoning, and then voting on the output results or verifying the model's selection results, the robustness and reliability of information can be further improved, making the model more universal and adaptable.

[0048] The general-purpose tool collaborative and refined learning system for large models has the following innovations: 1) A new multi-type tool collaborative learning technology framework, consisting of four parts: a dynamic combination module 1 of different tools, a unified interface module based on language instructions 2, a general-purpose tool refined learning module 3, and a comprehensive reasoning module for execution process and result information 4. 2) A multi-tool collaborative technology framework based on a language instruction interface has been designed.

[0049] Unless otherwise specified, the "large model" in this patent refers to a large-scale pre-trained language model.

[0050] In summary, the general-purpose tool collaboration and refined learning system for large models of the present invention achieves functional complementarity between different tools by enabling collaborative invocation among multiple types of general-purpose tools, and fully leverages the proprietary skills of the tools through refined learning technology of general-purpose tools, thereby improving the ability of large models to handle complex tasks. It has shown significant effects when applied to intelligent question answering scenarios.

[0051] Based on the above embodiments:

[0052] Please refer to Figure 4 , Figure 4 This is a technical framework diagram of the refined learning module of the general tool provided by the present invention.

[0053] In a preferred embodiment, the general tool refinement learning module 3 includes: an interactive web browser tool refinement learning module, used to process the pending tool-level subtasks of the tool call type being web browser based on the interactive web browser tool, and obtain the processing results of the pending tool-level subtasks of the tool call type being web browser; a knowledge graph tool refinement learning module, used to process the pending tool-level subtasks of the tool call type being knowledge graph based on the knowledge graph tool, and obtain the processing results of the pending tool-level subtasks of the tool call type being knowledge graph; and a tool interface API tool refinement learning module, used to process the pending tool-level subtasks of the tool call type being tool interface API based on the API tool, and obtain the processing results of the pending tool-level subtasks of the tool call type being tool interface API.

[0054] For tools designed for interactive web browsers, to break down barriers between different web pages, we will build a unified visual interface based on the browser-rendered graphical interface and a unified code text interface based on HTML. After defining the interaction interface between the model and the tool, we will collect human behavior data on browser usage and fine-tune the pre-trained model to learn and simulate human behavior, enabling it to use the tool intelligently. For knowledge graphs, we will build a human-machine collaborative annotation platform to record human behavior sequences when using these two types of tools. Based on a certain amount of behavior sequences, we will train a large model and then perform reinforcement learning training. For tools presented as APIs, we will build a platform that supports API input and runs the model's API calls in an isolated environment, presenting the results in a human-readable format, allowing humans to collaborate with the model to understand and call the API. This approach will also collect more human behavior data to provide the model with data for imitation and learning.

[0055] Please refer to Figure 5 , Figure 5 The screenshots (left) and the set of actions supported by the interactive web browser simulation interface provided by this invention are shown in the present invention (right).

[0056] Please refer to Figure 6 , Figure 6 This is a framework diagram of the refined learning module for web browser tools provided by the present invention.

[0057] In a preferred embodiment, the refined learning module of the interactive web browser tool includes: a fact retrieval acceleration module, used to decompose the task corresponding to the task instruction to be processed according to a tree structure, to obtain several action-level sub-tasks after tree structure decomposition, and to predict the retrieval query of several action-level sub-tasks after tree structure decomposition; a fact extraction module, used to extract factual information related to several action-level sub-tasks from the current page text of the interactive web browser, so as to obtain the processing result of the tool-level sub-task to be processed with the tool call type being web browser; the current page of the interactive web browser is retrieved based on the retrieval query of several action-level sub-tasks.

[0058] As a preferred embodiment, the refined learning module of the interactive web browser tool also includes a visual information fusion module, which is used to input textual and visual information into the large model through a structured input method to generate text information retrieval queries, new fact extraction, and next actions for several action-level sub-tasks.

[0059] Specifically, the learning process for the interactive web browser tool BM-Web includes: constructing a search engine user interface, defining key interactive elements during search engine usage, such as search buttons, page scrolling, and page clicks, and the impact of these interactions on the current interface, such as clicking a control to access a specific page. Based on the defined interaction interface between the large model and the tool, human behavior data regarding browser usage is collected. Specifically, in one embodiment, two groups of participants are recruited. One group (the requesters) is asked to formulate a series of requirements and provide natural language-based instructions, such as "Search why the sky is blue?". The other group (the implementers) then needs to perform the corresponding operations using the constructed search engine user interface according to these instructions. After collecting human behavior data, the pre-trained model is fine-tuned to learn and simulate human behavior, thereby enabling it to use the tool intelligently. Specifically, the model's input is S. t The model uses the following information: original question, current query, action set, window set, and fact set. Based on this, it predicts the next decision.

[0060] Learning about web browser tools mainly includes four basic components:

[0061] (1) Action prediction: Based on the current state, predict the probability of an action in the action space;

[0062] Predict the next action to be performed. Taking the action Search as an example, suppose {x1,…,x} N} is a tokenized sequence of the action name Search, where x i This represents a specific marker. The probability of a search can be decomposed as follows:

[0063]

[0064] Wherein, P(Search|S t The output is the sequence of search terms: Search = {x1, ..., x}. N The predicted probability of}, P(x1|S t ) represents a given state S t Given that the first word is x1, the predicted probability, P(x i |S t ,x1,…,x i-1 ) represents a given state S t The i-th word following the first i-1 words is x. i The predicted probability, S t Let x represent the current state at time t. i The term represents a word or token, where N indicates that the length of the corresponding Search query word sequence is N.

[0065] During the reasoning process, the action with the highest probability of execution is selected on the interface.

[0066] (2) Query generation: Generate a subtask retrieval query based on the current state;

[0067] Generate search statements Search engines (such as Bing) are used to perform text generation tasks.

[0068]

[0069] Wherein, P(Q) t+1 |S t ) represents a given state S t Generate search query statement Q under the given conditions t+1 The predicted probability, P(q1|S t ) represents a given state S t The probability of generating the first token q1 under the given conditions is P(q). i |S t ,q1,…,q i-1 ) represents a given state S t When predicting the i-th word as q with the first i-1 tokens. i The predicted probability, S t Let q represent the environment state at time step t. i Represents the i-th token of the question statement, |Q t+1 | indicates the length of the token sequence in the question statement.

[0070] (3) Fact Extraction Module: Extracts information related to the question from the current page;

[0071] Assuming the window is currently displaying search results in browsing mode... The goal of this step is to extract the query results W t Extract a supporting fact f = {w i ,…,w j}, where 1≤i≤j≤|w t |. Only generate a given S t The first and last few Ns of f f Characters, by maximizing conditional probability In this implementation, [s] and [e] are denoted as the start and end special characters of the starting fact f, respectively. During the reasoning process, after decoding the start and end markers, W can be located through text matching. t Find the desired sequence, with a preset quantity N. f It can be 10.

[0072] (4) Answer generation: The answer is synthesized based on the collected set of facts and the original question.

[0073] The integration module is responsible for assembling a series of supporting facts into a coherent answer. During training, the optimized model is based on the original question (Q0) and the given supporting facts (f1,...,f...). N Generate the answer (Answer) and maximize P(Answer|Q0,f1,…,f N ).

[0074] Please refer to Figure 7 , Figure 7 This invention provides a technical framework diagram for a web browser tool's refined learning module that considers accelerating fact retrieval and integrating visual information.

[0075] Considering that visual information such as webpage layout and color plays a crucial role in information retrieval when people browse and obtain information on webpages, this embodiment further considers visually enhanced information retrieval and efficient text information retrieval, implemented as follows:

[0076] Visual information fusion module: Given text input x text and visual input X vision The probability of synthesizing the target text Y based on textual and visual information is:

[0077]

[0078] here For multimodal pre-trained language models (M-scale models), such as Multimodal-CoT, KOSMOS-1, etc., or other pre-trained models that support structured input, such as CPM-3, ChatGPT, etc., visual information such as font, color, and layout, along with text information, are input into the model through structured input.

[0079] in, Indicates the given input text X text and visual input information X vision The predicted probability of generating Y. Y indicates i Let Y represent the i-th token.

[0080] Fact Retrieval Acceleration Module: Considering that using a chain-based retrieval method, which sequentially performs information retrieval, action execution, fact acquisition, and information synthesis, results in slow answer response speed, and that excessively long historical state information increases the model's inference cost and easily exceeds the model's maximum input length, this embodiment replaces the chain-based decomposition with a tree structure decomposition of the original problem. A divide-and-conquer approach is adopted for each node state, and multiple queries are predicted simultaneously for parallel searching to obtain fact information.

[0081] In summary, the refined learning module of the interactive web browser tool has the following innovations: 1) It is the first to integrate web-related visual information to improve the web text retrieval effect in the web browser using BM; 2) In response to the problem of excessively long sequences in fact extraction, it provides an efficient fact extraction module that generates the first and last sequences of the span; 3) It proposes to decompose the problem into a tree structure using a box and perform divide-and-conquer parallel processing to achieve more efficient BM-Web technology.

[0082] Please refer to Figure 8 , Figure 8 This invention provides an example of knowledge graph retrieval based on atomic serialization operations (the left figure is an example of process recording and the right figure is an example of intermediate steps in the query operation).

[0083] Please refer to Figure 9 , Figure 9 This is a technical framework diagram of the refined learning module for knowledge graph tools provided by this invention.

[0084] As a preferred embodiment, the knowledge graph-oriented tool refinement learning module includes: an initial instruction definition module, used to construct hints or few-sample examples of serialized atomic operations based on knowledge-based tasks; an atomic operation prediction module, used to predict serialized atomic operations based on the hints or few-sample examples of serialized atomic operations and the current state of the knowledge graph, using a large-scale atomic operation prediction model; a query language conversion and execution module, used to convert serialized atomic operations into the serialized knowledge graph interactive query language SPARQL; and to execute the knowledge graph query language SPARQL to obtain serialized interactive query results, thereby obtaining the processing results of the tool-level subtask to be processed with the tool call type of knowledge graph.

[0085] As a preferred embodiment, the knowledge graph tool's refined learning module also includes a reward module, which optimizes the atomic operation prediction model of the large model based on the incentive model of the large model according to the sequential interactive query results.

[0086] Specifically, this embodiment will construct a refined learning platform for knowledge graph tools. On this platform, annotators can complete complex knowledge base queries simply by clicking. The background records the results of human observation data and the process of sequence operations, which are then provided to the model for training. Subsequently, this platform can be used to recreate the model's operations, allowing human annotators to help adjust the behavioral sequences.

[0087] KoPL is a programming language designed for complex reasoning problems. KoPL summarizes knowledge in a knowledge graph into seven categories (entities, concepts, attributes, relations, attribute facts, relational facts, and descriptive facts). Based on this, it abstracts 14 knowledge operation functions (such as Find, FilterConcept, FilterStr, Relate, And, Or, etc.) and 13 query functions (such as QueryName, QueryAttr, QueryRelation, SelectionAmong, etc.). Regarding the construction of atomic operations, this embodiment first borrows KoPL's function settings and uses them as atomic operations. Through these atomic operations, various complex reasoning tasks based on knowledge graphs can be basically implemented.

[0088] In order for large models to learn and imitate human knowledge query behavior, it is necessary to obtain knowledge query behavior data based on atomic operations. (1) First, based on the defined atomic operations, a simulation annotation platform for knowledge query is constructed, and behavioral data and intermediate results are recorded by manual annotation. For the source of the annotated data, this type of data is often defined on a very small subset of the knowledge graph, a specific knowledge graph, or strictly defined according to a given knowledge graph. This questioning method is quite different from the actual questioning. This embodiment will consider combinatorial generalization and zero-shot generalization to construct high-quality question-and-answer data. (2) Since the manual annotation process based on atomic operations is relatively complex, it can only annotate a small amount of data. The annotation data can be augmented by paraphrasing the questions.

[0089] Regarding model training, BM-KG is trained using a behavior imitation learning optimization method on the collected atomic operation behavior data of knowledge graph queries.

[0090] (1) Initial instruction definition module. Based on the user's question, construct hints or few-sample examples of atomic operations as task language instructions. This approach helps large models understand atomic operations.

[0091] (2) The atomic operation prediction module, as an Actor in the imitation learning, performs atomic operation prediction based on the instruction input and the current state at time t.

[0092] (3) The query language conversion and execution module converts atomic operations into the knowledge graph query language SPARQL. Since atomic operations are fixed, this can be achieved through a predefined template method. Then, the query language is executed to obtain the current query results.

[0093] (4) The reward module scores the generated operation sequence based on the current execution result and action, and uses it as a reward for optimizing the atomic operation model.

[0094] In summary, the refined learning module for knowledge graph tools has the following innovations: by designing serialized atomic decomposition, a knowledge question-and-answer method for serialized knowledge graph interactive queries is realized. Adjustments are made based on the intermediate states obtained from the serialized interactive queries, which can avoid questions that cannot be answered due to errors or lack of answers in any intermediate steps.

[0095] Please refer to Figure 10 , Figure 10 This is a technical framework diagram of the API-oriented tool refinement learning module provided by the present invention.

[0096] As a preferred embodiment, the API-oriented tool refinement learning module includes: an API customization module based on user needs, used to analyze and collect a set of APIs related to the task corresponding to the task instructions; an API integration module based on a unified interface, used to build a unified API repository based on the set of APIs, the API repository integrating multiple API interfaces that can meet different user needs, and establishing an API hierarchy; an API retrieval module based on dynamic needs, used to perform dynamic API retrieval based on the unified API repository according to the API hierarchy mechanism, and obtain API retrieval results; and a model learning module based on API pre-training, used to perform model learning based on API pre-training based on API retrieval results, so as to obtain the processing results of the tool-level subtasks to be processed where the tool call type is a tool interface API tool.

[0097] As a preferred embodiment, the API tool refinement learning module also includes a few-shot generalization API tool manual learning module, which is used to update and maintain a unified API repository and optimize the API level. Based on the updated and maintained unified API repository and optimized API level, few-shot generalization tool manual learning is performed so that on-demand retrieval can be performed using the few-shot generalization API tool manual.

[0098] Specifically, with the continuous development of technology and the increasing complexity and diversity of application scenarios, many problems require specific API interfaces to solve. For example, for problems involving systems of equations like "2x^2=3y-1 and x+y=10xy", specific equation-solving API interfaces are needed; to create a PowerPoint presentation about tool learning and download images from examples.com, specific image download API interfaces are required; and to convert PDF files to Word documents, specific document conversion API interfaces are needed. Furthermore, downloading photos with specific themes from Instagram, such as glaciers or aurora borealis, also requires specific API interfaces.

[0099] Due to the high flexibility and customization of these APIs, it is difficult to unify them using a single tool. To address this issue, this embodiment proposes the idea of ​​directly modeling specific API interfaces. By analyzing and collecting user requirements, a set of APIs needed by users was obtained, and a unified API repository was built based on the analysis results. This repository integrates various API interfaces to meet different user needs. We also established a strict API hierarchy so that the model can retrieve or use each sub-API.

[0100] For each API, its description, parameters, and return value are constructed, and sample examples are provided to help the model learn the API's usage and purpose. For general APIs, labeled data is also built, allowing the model to call the API more accurately through training. This allows the model to easily call specific API interfaces according to user needs, thereby solving a wide variety of problems. The API repository is continuously updated and maintained to ensure it includes the latest and best API interfaces. The API layer is also constantly evolving to adapt to different application scenarios and user needs. Simultaneously, efforts are being made to improve API documentation and usage guides so that users can more easily use and call the model, access these API interfaces, and provide users with a flexible, efficient, and easy-to-use API platform to more easily solve various problems.

[0101] To enable the refined learning module of the tool interface API to generalize instructions across tasks, it is necessary to construct a zero-shot or few-shot instruction learning method for the tool interface API. Considering that commonly used tools such as knowledge graphs, web browsers, and databases often provide rich and complete interface documentation and usage examples, an instruction data method based on the tool manual is adopted. Addressing the issue that the large number of tool APIs and examples makes it difficult to input all of them into large models through instruction definitions due to the limited input length of large models, an on-demand retrieval support approach is adopted. This is achieved using retrieval techniques such as dense vector retrieval, differentiable search index (DSI), or hierarchical navigable small world (HNSW) algorithms. Utilizing the inherent hierarchical relationships of the API helps achieve efficient and accurate retrieval.

[0102] The following example illustrates how comprehensive API calls are performed in the system. A user request is: "Help me create a PPT about tool learning, where images are downloaded from example.com and then converted to PDF." Analysis reveals that this request requires three APIs: an API for creating the PPT, an API for searching and downloading images, and an API for format conversion. These three APIs will be integrated into our BM-API. In a real user scenario (using this problem as an example), the model will perform an API search, obtaining matching API functions. If the function appeared during model training, the model can call the API with high accuracy. If the function did not appear during model training, the model needs to learn how to use the API by reading the API manual and sample examples. The model then calls the three required APIs sequentially to obtain the final "tool learning.pdf" file.

[0103] In summary, the API-oriented refined learning module has the following innovations: it proposes to construct API calling language instructions based on API manual descriptions and examples, realizes API tool learning technology with few samples and zero samples, and combines it with efficient API retrieval to realize the calling and use of large-scale APIs, applicable to any tool with an API manual.

[0104] As a preferred embodiment, it further includes: a unified interface module based on API instructions, used to understand the API tool manual, obtain easily understandable API tool manual information for large modules, and call the required API tools for the pending tool-level subtasks based on the API tool manual information and the tool call type being the tool interface API tool.

[0105] Specifically, a unified interface module based on tool API instructions is used. Considering that Wikidata's SPARQL query language, web search engines, and various programming languages ​​all provide rich interfaces, large models can directly call tool API interfaces to perform operations on the tools. First, API manual understanding is achieved using ChatGPT or rule-based automatic parsing techniques to extract relevant information (such as input parameters, output formats, and example code snippets). This information is then converted into a machine-readable format easily understood by the large model through a defined schema. Second, API retrieval automatically selects suitable APIs based on their relevance and compatibility with the current task. This tool API module avoids building a separate tool learning model for each tool to accept language instructions, resulting in better scalability and flexibility. For model optimization, both methods leverage interactive learning, using environmental feedback to reinforce learning.

[0106] Please refer to Figure 11 , Figure 11 The annotation platform provided by this invention unifies the annotation record diagrams used by various tools.

[0107] Please refer to Figure 12 , Figure 12 The interface diagram of the data annotation platform for the tool provided by this invention.

[0108] A Tool Learning Platform (TLP) was developed based on the Tool Learning Framework. This platform aims to provide an efficient and interactive learning environment, enabling users to easily learn and use various tools, such as web browsers, knowledge graphs, structured databases, and large model tools. The platform has the following characteristics: (1) It is a three-way interactive platform, where tools, models, and humans can interact and learn from each other; (2) The platform has an executable environment for the tools, allowing tool calls and feedback to be presented to the model and humans in real time; (3) The platform is scalable, allowing human annotators to perform a series of actions, including clicking, searching, and other browsing actions, or to make function calls to complete more complex tasks; (4) It is a multi-step execution environment, where the output of each step can be read by the subsequent model to take subsequent actions; (5) Automatic recording. The operations performed by the model and humans on the platform are fully recorded as real human data for subsequent model training. The results of tool calls generated by the model are directly presented to the model and humans for interactive learning. The corrected sequences predicted by the model are recorded for continuous model training.

[0109] In summary, the tool learning platform has the following innovations: it constructs a multi-type tool learning platform for large models, which can provide an efficient and interactive learning environment, enabling users and business leaders to easily learn and use various tools.

[0110] Please refer to Figure 13 , Figure 13 This is a flowchart illustrating the collaborative and refined learning method for general-purpose tools aimed at large models provided by this invention.

[0111] This invention also provides a general tool collaboration and refined learning method for large models, applicable to the aforementioned general tool collaboration and refined learning system for large models, comprising:

[0112] 1301: The dynamic combination mechanism module of different tools decomposes the task corresponding to the task instruction to be processed into several tool-level sub-tasks through a large model, constructs a general tool call graph based on several tool-level sub-tasks and general tool interfaces, and establishes a dynamic routing mechanism for general tools through reinforcement learning and instruction learning.

[0113] 1302: The unified interface module based on language instructions calls the general tool interface based on language instructions according to several tool-level subtasks, a general tool call graph, and a dynamic routing mechanism;

[0114] 1303: The general tool refinement learning module processes several tool-level subtasks based on the general tools corresponding to the language instruction-based general tool interface, and obtains the processing results of several tool-level subtasks;

[0115] 1304: The general tool refinement learning module is one or more general tool modules that are updated and maintained in real time; the execution process and result information integrated reasoning module integrates and reasons the execution process information and processing results of several tool-level sub-tasks to obtain the final answer to the task instruction.

[0116] For an introduction to the general-purpose collaborative and refined learning method for large models provided by this invention, please refer to the above system embodiments; the invention itself will not be described in detail here.

[0117] The main innovations of this invention are as follows: 1) This invention constructs an effective framework for collaborative and refined learning of general tools, which includes a dynamic combination mechanism module 1 for different tools, a unified interface module 2 based on language instructions, a refined learning module for general tools 3, and a comprehensive reasoning module for execution process and result information 4; 2) A new collaborative learning framework for multiple types of tools is proposed, namely, a multi-tool collaborative learning framework based on language instruction interfaces and API instruction interfaces; 3) A multi-type tool learning platform for BM is constructed, which can provide an efficient and interactive learning environment through which users and BM can easily learn and use various tools; 4) BM tool learning technology for refined operation of general tools is constructed, including BM-Web, a tool refined learning technology based on an interactive web browser that integrates visual information, BM-KG, a tool refined learning technology based on sequential knowledge graph query, and BM-API, an API tool refined learning method that supports few-shot / zero-shot learning.

[0118] The general-purpose tool for building large models, the collaborative and refined learning system of this invention, has significant practical value:

[0119] The constructed large-model tool learning system utilizes predefined tools to extend the task processing capabilities of large models. Combining the execution results of these tools with the generated results of the large model improves the accuracy of model generation. Optimizing tools for specific domains allows the model to better leverage these tools to solve tasks, enhancing the practicality and operability of the research. Compared to using only a general-purpose language model, the constructed large-model tool learning system offers the following advantages: avoiding over-reliance on memory and enhancing real-time update capabilities; superior performance in specific domains; providing new opportunities for previously unsolvable tasks; supporting more natural human-computer interaction; improving model interpretability and credibility; enhancing model robustness; and improving low-resource language understanding capabilities. Therefore, the constructed large-model tool learning system can realize new task-solving methods and provide an effective approach to solving problems such as the "illusion" of large models.

[0120] This invention constructs a general tool collaboration and refined learning system for large models, enabling large models to mimic human cognitive behaviors when using tools. It constructs methods such as complex task decomposition, reasoning planning, and interactive feedback, thereby achieving autonomous cognitive learning capabilities for large models.

[0121] This invention draws on and learns from human cognitive behavior in using tools, constructs a tool learning model for general tools such as web browsers, knowledge graphs, and tool APIs, and conducts practical verification on application tasks such as intelligent question answering and task modeling in complex scenarios, which helps to improve the ability of the large model to solve complex tasks in practical applications.

[0122] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A general-purpose collaborative and refined learning system for large models, characterized in that, include: The module for dynamic combination of different tools connected to the input end and the output end of the large model is used to decompose the task corresponding to the task instruction to be processed into several tool-level sub-tasks through the large model, construct a general tool call graph based on several tool-level sub-tasks and general tool interface, and establish a dynamic routing mechanism for general tools through reinforcement learning and instruction learning. The input end is connected to the output end of the dynamic combination mechanism module of the different tools. The unified interface module based on language instructions is used to call the general tool interface based on language instructions according to several tool-level sub-tasks, the general tool call graph and the dynamic routing mechanism. The input end of the general tool refinement learning module is connected to the output end of the language instruction-based unified interface module. It is used to process several tool-level sub-tasks according to the general tools corresponding to the language instruction-based general tool interface, and obtain the processing results of several tool-level sub-tasks. The general tool refinement learning module is one or more general tool modules that are updated and maintained in real time. The input end is connected to the output end of the general tool refined learning module, and the output end is connected to the input end of the large model. The execution process and result information integrated reasoning module is used to integrate and reason about the execution process information and processing results of several tool-level sub-tasks to obtain the final answer of the task instruction. The general-purpose tool's refined learning module includes: The interactive web browser tool fine-grained learning module is used to process the pending tool-level subtasks with the tool call type of web browser based on the interactive web browser tool, and obtain the processing results of the pending tool-level subtasks with the tool call type of web browser. The Knowledge Graph Tool Refinement Learning Module is used to process the pending tool-level subtasks whose tool call type is Knowledge Graph based on the Knowledge Graph Tool, and obtain the processing results of the pending tool-level subtasks whose tool call type is Knowledge Graph. The Tool Interface API Tool Refinement Learning Module is used to process the pending tool-level subtasks whose tool call type is Tool Interface API Tool, based on the API Tool, and obtain the processing results of the pending tool-level subtasks whose tool call type is Tool Interface API Tool.

2. The general-purpose collaborative and refined learning system for large models according to claim 1, characterized in that, The refined learning module of the interactive web browser tool includes: The fact retrieval acceleration module is used to decompose the task corresponding to the task instruction to be processed according to the tree structure to obtain several action-level sub-tasks after the tree structure decomposition, and predict the retrieval query of several action-level sub-tasks after the tree structure decomposition. The fact extraction module is used to extract factual information related to several action-level subtasks from the current page text of the interactive web browser, so as to obtain the processing result of the pending tool-level subtask with the tool call type of web browser; the current page of the interactive web browser is retrieved based on the retrieval query of several action-level subtasks.

3. The general-purpose collaborative and refined learning system for large models according to claim 2, characterized in that, The refined learning module of the interactive web browser tool also includes a visual information fusion module, which is used to input textual and visual information into the large model through a structured input method to generate text information retrieval queries, new fact extraction, and next actions for several action-level sub-tasks.

4. The general-purpose collaborative and refined learning system for large models according to claim 1, characterized in that, The knowledge graph tool's refined learning module includes: The initial instruction definition module is used to build hints or few-sample examples of serialized atomic operations based on knowledge-based tasks; The atomic operation prediction module is used to predict the serialized atomic operations based on the hints or few-sample examples of the serialized atomic operations and the current state of the knowledge graph, using the atomic operation prediction model of the large model. The query language conversion and execution module is used to convert the serialized atomic operations into the serialized knowledge graph interactive query language SPARQL; execute the knowledge graph query language SPARQL to obtain the serialized interactive query results, so as to obtain the processing results of the pending tool-level subtasks with the tool call type of knowledge graph.

5. The general-purpose collaborative and refined learning system for large models according to claim 4, characterized in that, The knowledge graph tool's refined learning module also includes a reward module, which is used to optimize the atomic operation prediction model of the large model based on the incentive model of the large model according to the sequential interactive query results.

6. The general-purpose collaborative and refined learning system for large models according to claim 1, characterized in that, The tool interface API tool refinement learning module includes: The API customization module based on user needs is used to analyze and collect a set of tool-oriented interface APIs related to the task corresponding to the task instruction. The API integration module based on a unified interface is used to build a unified API repository based on the set of tool-oriented APIs. The API repository integrates a variety of API interfaces that can meet different user needs and establishes an API hierarchy. The API retrieval module based on dynamic requirements is used to perform dynamic API retrieval based on the unified API repository according to the API hierarchy mechanism, and obtain API retrieval results. The model learning module based on API pre-training is used to perform model learning based on API pre-training according to the API retrieval results, so as to obtain the processing results of the tool-level subtasks to be processed for the tool call type of the tool interface API tool.

7. A general-purpose tool collaborative and refined learning system for large models according to claim 6, characterized in that, The API tool refinement learning module also includes a few-shot generalized API tool manual learning module, which is used to update and maintain the unified API repository and optimize the API level. Based on the updated and maintained unified API repository and the optimized API level, few-shot generalized tool manual learning is performed so that the few-shot generalized API tool manual can be used for on-demand retrieval.

8. A general-purpose collaborative and refined learning system for large models according to any one of claims 1 to 7, characterized in that, Also includes: The unified interface module based on API instructions is used to understand the API tool manual, obtain easily understandable API tool manual information for large modules, and call the required API tools for the pending tool-level subtasks based on the API tool manual information and the tool call type is the tool interface API tool.

9. A general-purpose collaborative and refined learning method for large models, characterized in that, This method is applicable to the general-purpose collaborative and refined learning system for large models as described in any one of claims 1 to 8, comprising: The dynamic combination mechanism module of different tools decomposes the task corresponding to the task instruction to be processed into several tool-level sub-tasks through a large model, constructs a general tool call graph based on several tool-level sub-tasks and general tool interface, and establishes a dynamic routing mechanism for general tools through reinforcement learning and instruction learning. The unified interface module based on language instructions calls the general tool interface based on language instructions according to several tool-level subtasks, the general tool call graph, and the dynamic routing mechanism. The general tool refinement learning module processes several tool-level subtasks based on the general tools corresponding to the language instruction-based general tool interface, and obtains the processing results of several tool-level subtasks; the general tool refinement learning module is one or more general tool modules that are updated and maintained in real time; the general tool refinement learning module includes an interactive web browser tool refinement learning module, a knowledge graph tool refinement learning module, and a tool interface API tool refinement learning module; The refined learning module of the interactive web browser tool processes the pending tool-level subtasks with the tool call type of web browser based on the interactive web browser tool, and obtains the processing results of the pending tool-level subtasks with the tool call type of web browser. The knowledge graph tool's refined learning module processes the pending tool-level subtasks whose tool call type is knowledge graph, based on the knowledge graph tool, and obtains the processing results of the pending tool-level subtasks whose tool call type is knowledge graph. The API tool refinement learning module processes the pending tool-level subtasks whose tool call type is API tool, based on the API tool, and obtains the processing results of the pending tool-level subtasks whose tool call type is API tool. The execution process and result information integration reasoning module integrates and reasons the execution process information and processing results of several tool-level sub-tasks to obtain the final answer to the task instruction.

Images