Tool calling method and device, electronic equipment and storage medium

Abstract

The invention discloses a tool calling method and device, electronic equipment and a storage medium. According to the specific implementation scheme, the method comprises the steps that a to-be-completed task is analyzed through a large language model in the tool calling method, at least one to-be-used tool needed for executing the to-be-completed task is selected from a tool library, and an initial tool candidate set is obtained; determining a tool calling sequence based on the initial tool candidate set and the tree search framework by utilizing the large language model; performing quantitative evaluation on the tool calling sequence, and determining a quantitative evaluation result corresponding to the tool calling sequence; and backtracking the quantitative evaluation result to nodes of the tree search framework, and updating reward values and upper confidence boundaries of the nodes of the tree search framework. Complex tasks to be completed are understood, intelligent decisions are made when the tools in the tool library are called, and multiple tools to be used are automatically called to execute the complex tasks to be completed.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a tool invocation method, apparatus, electronic device, and storage medium. Background Technology

[0002] In the field of tool invocation, large language models have powerful language reasoning capabilities. However, in existing tool invocation methods, large language models often complete specific functions by calling a single external tool, such as generating SQL statements from natural language to access a database, or calling a calculator to perform mathematical reasoning.

[0003] However, in practical applications, the following limitations arise: Firstly, large language models themselves do not possess the ability to directly manipulate external environments or tools; secondly, many tasks require interaction with multiple tools, such as database queries, computation engine calls, image recognition interfaces, search engines, and automated script execution systems, all of which require the coordinated execution of multiple tools. Furthermore, user needs are often complex and varied, frequently involving the collaborative invocation of multiple sub-tasks and tools. Summary of the Invention

[0004] This invention provides a tool invocation method, apparatus, electronic device, and storage medium to improve the success rate of tool invocation by invoking multiple tools to perform complex tasks.

[0005] According to one aspect of the present invention, a tool invocation method is provided, comprising:

[0006] The task to be completed is parsed by the large language model in the tool invocation method, and at least one tool to be used to execute the task is selected from the tool library to obtain an initial tool candidate set. The tree search frame corresponding to the initial tool candidate set is determined. The number of initial tool candidate sets is at least one. The tool library stores text descriptions indicating the tools to be used. The root node of the tree search frame indicates the user instruction text. The nodes in the tree search frame other than the root node indicate the text concatenated by the text indicated by the parent node of the node and the selected tool to be used. The edge of the tree search frame indicates the next tool to be used selected from the initial tool candidate set starting from the current node of the tree search frame.

[0007] Using the large language model, based on the initial tool candidate set and the tree search framework, a tool invocation sequence is determined, wherein the tools to be used in the tool invocation sequence are used to execute the task to be completed;

[0008] The tool call sequence is quantitatively evaluated to determine the quantitative evaluation result corresponding to the tool call sequence;

[0009] The quantitative evaluation results are backtracked to the nodes of the tree search framework to update the reward value and upper confidence bound of the nodes of the tree search framework.

[0010] According to another aspect of the present invention, a tool calling device is provided, comprising:

[0011] The parsing module is used to parse the task to be completed through the large language model in the tool invocation method, select at least one tool to be used to execute the task from the tool library, obtain an initial tool candidate set, and determine the tree search frame corresponding to the initial tool candidate set. The number of initial tool candidate sets is at least one. The tool library stores text descriptions indicating the tools to be used. The root node of the tree search frame indicates the user instruction text. The nodes in the tree search frame other than the root node indicate the text concatenated by the text indicated by the parent node of the node and the selected tool to be used. The edges of the tree search frame indicate the next tool to be used selected from the initial tool candidate set starting from the current node of the tree search frame.

[0012] The determination module is used to determine a tool invocation sequence based on the large language model, the initial tool candidate set, and the tree search framework, wherein the tool invocation sequence contains the tools to be used to execute the task to be completed;

[0013] An evaluation module is used to perform quantitative evaluation on the tool call sequence and determine the quantitative evaluation result corresponding to the tool call sequence;

[0014] The update module is used to backtrack the quantitative evaluation results to the nodes of the tree search framework, update the reward value of the nodes of the tree search framework and the upper confidence boundary value evaluation module update module.

[0015] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0016] At least one processor; and

[0017] A memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the tool invocation method described in any embodiment of the present invention.

[0019] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the tool invocation method described in any embodiment of the present invention.

[0020] The technical solution of this invention involves parsing the task to be completed using a large language model in the tool invocation method, selecting at least one tool from a tool library required to execute the task, and obtaining an initial tool candidate set. Using the large language model, based on the initial tool candidate set and the tree search framework, a tool invocation sequence is determined. The tool invocation sequence is then quantitatively evaluated to determine the corresponding quantitative evaluation result. The quantitative evaluation result is then backtracked to the nodes of the tree search framework, updating the reward value and upper confidence bound of the nodes. This achieves an understanding of complex tasks to be completed, intelligent decision-making when invoking tools from the tool library, and automated invocation of multiple tools to execute complex tasks.

[0021] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart of a tool invocation method provided in Embodiment 1 of the present invention;

[0024] Figure 2 This is a flowchart of a task decomposition method according to Embodiment 1 of the present invention;

[0025] Figure 3 This is a flowchart of a method for determining quantitative evaluation results according to Embodiment 2 of the present invention;

[0026] Figure 4 This is a schematic diagram of a tool calling device according to Embodiment 3 of the present invention;

[0027] Figure 5 This is a block diagram of an electronic device provided according to Embodiment 4 of the present invention. Detailed Implementation

[0028] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0029] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0030] Example 1

[0031] S110. The task to be completed is parsed through the large language model in the tool call method, and at least one tool to be used is selected from the tool library to execute the task to be completed, so as to obtain an initial tool candidate set and determine the tree search framework corresponding to the initial tool candidate set.

[0032] The initial tool candidate set has at least one number, the tool library stores text descriptions indicating the tools to be used, the root node of the tree search frame indicates the user instruction text, the nodes in the tree search frame other than the root node indicate the text composed of the text indicated by the parent node of the node and the selected tool to be used, and the edges of the tree search frame indicate the next tool to be used selected from the current node of the tree search frame in the initial tool candidate set.

[0033] In this embodiment, the tool invocation method can be understood as a method for invoking a tool to be used, and the tool invocation method may include a large language model. The task to be completed can be understood as a task that needs to be completed by an electronic device; executing the task can be accomplished through a combination of various tools to be used. The tool library can be understood as a collection of text or vectors corresponding to the tools to be used; the tools to be used can be invoked using the text or vectors corresponding to the tools. A tool to be used can be understood as a tool used to perform a single task. The initial tool candidate set can be understood as a collection of text or vectors corresponding to at least one tool to be used; a tool in one initial tool candidate set can execute the task to be completed.

[0034] Specifically, automatically scanning and identifying available tools in electronic devices can be achieved by collecting metadata about these tools through a standardized interface. A unique identifier is assigned to each tool, and a vector corresponding to that tool is generated based on its metadata. This vector is then stored in a tool library. For a given task, a large language model within the tool invocation method parses the task to generate multiple tool search terms. The most relevant tools retrieved from these search terms are then combined to form an initial tool candidate set. All tools in the initial candidate set can be combined to execute the task.

[0035] For example, the tools to be used may include: topology analysis tools, real-time query tools for GIS platforms, cached tool sequences, etc. The task to be completed may be: overload risk analysis of substation A. When constructing a tree search framework for the initial tool candidate set, a tree search framework based on a Monte Carlo tree search framework can be constructed. The root node of the tree search framework indicates the user instruction text; nodes in the tree search framework other than the root node indicate text composed of the text indicated by the parent node of the root node and the selected tool to be used; and the edges of the tree search framework represent the next tool to be used selected from the initial tool candidate set starting from the current node of the tree search framework.

[0036] S120. Using the large language model, based on the initial tool candidate set and the tree search framework, determine the tool invocation sequence.

[0037] The tool call sequence includes the tools to be used to execute the task to be completed.

[0038] In this embodiment, the tool invocation sequence can be understood as a sequence of tools to be used in the initial tool candidate set for performing the task to be completed. The tool invocation sequence can indicate the order in which each tool to be used is used.

[0039] Specifically, using the large language model in the tool invocation method, a weighted combination value is obtained by combining the predicted probability of each candidate tool being invoked by the large language model with the upper confidence bound value of the corresponding node in the tree search framework. Based on this weighted combination value, an existing tool invocation path in the tree search framework is selected as the tool invocation sequence. If the task to be completed is not resolved after the tool corresponding to the tool invocation path is invoked, a new child node is created on the node of the tool invocation path to obtain the tool invocation sequence. The child node stores the text concatenated from the text indicated by the parent node corresponding to the child node and the selected tool.

[0040] S130. Quantitatively evaluate the tool call sequence and determine the quantitative evaluation result corresponding to the tool call sequence.

[0041] In this embodiment, the quantitative evaluation result can be understood as a comprehensive evaluation result of the resources required to perform the task to be completed by the initial set of tool candidates.

[0042] Specifically, the tool call sequence can be quantitatively evaluated in multiple dimensions, including assessments of task completion, time efficiency, and resource consumption. This evaluation, combined with the synergistic effects between the tools to be used within the tool call sequence, yields the quantitative evaluation results for the initial tool candidate set. For each initial candidate tool set, a tree search framework can be constructed, which can be understood as an algorithmic framework for finding the optimal solution. When determining the quantitative evaluation results, the synergistic effects between the tools to be used in the initial candidate set must also be considered. Positive synergy can improve the quantitative evaluation results, while negative synergy can lead to database lock conflicts and reduce execution efficiency.

[0043] S140. The quantitative evaluation result is backtracked to the nodes of the tree search framework, and the reward value and upper confidence boundary value of the nodes of the tree search framework are updated.

[0044] Specifically, after determining the quantitative evaluation result corresponding to the execution tool call sequence, the quantitative evaluation result can be traced back to each node of the tree search framework to update the reward value and upper confidence boundary value of the node of the tree search framework.

[0045] For example, the improved UCT algorithm (Upper Confidence Interval algorithm) is an algorithm used in tree search frameworks, mainly for solving decision problems with uncertainty and randomness. Its calculation formula is as follows: In the formula Denotes the upper confidence threshold, where The fusion coefficient is adjustable and is set to 0.7.

[0046] The technical solution of this invention involves parsing the task to be completed using a large language model in the tool invocation method, selecting at least one tool from a tool library required to execute the task, and obtaining an initial tool candidate set. Using the large language model, based on the initial tool candidate set and the tree search framework, a tool invocation sequence is determined. The tool invocation sequence is then quantitatively evaluated to determine the corresponding quantitative evaluation result. The quantitative evaluation result is then backtracked to the nodes of the tree search framework, updating the reward value and upper confidence bound of the nodes. This achieves an understanding of complex tasks to be completed, intelligent decision-making when invoking tools from the tool library, and automated invocation of multiple tools to execute complex tasks.

[0047] Based on the above embodiments, modified embodiments of the above embodiments are proposed. It should be noted that, in order to keep the description brief, only the differences from the above embodiments are described in the modified embodiments.

[0048] In one embodiment, determining the tool invocation sequence using the large language model, based on the initial tool candidate set and the tree search framework, includes:

[0049] The predicted probability of each candidate tool being invoked by the large language model is weighted and combined with the upper confidence bound of the corresponding node in the tree search framework to obtain a weighted combination value. Based on the weighted combination value, the existing tool invocation path in the tree search framework is selected as the tool invocation sequence.

[0050] If the task to be completed is not resolved after the tool corresponding to the tool call path is called, a new child node is created on the node of the tool call path to obtain an updated tool call sequence. The child node stores the text concatenated by the text indicated by the parent node corresponding to the child node and the selected tool to be used.

[0051] For example, the formula for calculating the weighted combination value is: In the formula This represents the upper confidence bound of the corresponding node in the tree search framework. This represents the predicted probability of each candidate tool being invoked by the large language model. When a task needs to be executed, an existing tool invocation path in the tree search framework can be selected based on the weighted combination value. The path with the highest weighted combination value can be chosen as the tool invocation path. If the task is not resolved after the tool corresponding to the tool invocation path is invoked, it means that the tool invocation path does not meet the conditions for the task. In this case, the tool invocation path can be updated by creating a new child node on the node of the tool invocation path, resulting in a new tool invocation path. The process continues to determine whether the new tool invocation path can resolve the task until a tool invocation sequence that can resolve and execute the task is obtained.

[0052] In one embodiment, the determination operation of the tool library includes:

[0053] Obtain the tool to be used, and the metadata corresponding to the tool to be used, wherein the metadata indicates the function of the tool to be used;

[0054] Using an embedding generative model, the metadata is used to generate a semantic embedding vector corresponding to the tool to be used;

[0055] The semantic embedding vector corresponding to the tool to be used is stored in the tool library.

[0056] In this embodiment, metadata can be understood as information indicating the function of the tool to be used. Metadata may include the API interface, input / output format, execution time, etc. of the tool to be used.

[0057] Specifically, the process involves acquiring the tools to be used and exporting their corresponding metadata from the database. Using an embedding generative model, semantic embedding vectors corresponding to the tools are generated from the metadata, and all information provided by the metadata is stored in these semantic embedding vectors. Finally, the semantic embedding vectors for each tool are stored in a tool library. Therefore, the tool library is a text file containing information about each tool, including its name, parameter descriptions, output format, usage examples, and information indicated by the metadata.

[0058] In one embodiment, the method of parsing the task to be completed through a large language model in the tool invocation method, and selecting at least one tool to be used from the tool library to execute the task to be completed, obtains an initial tool candidate set, including:

[0059] The task to be completed is parsed by the large language model in the tool call method, and the task to be completed is decomposed into at least one parsing task.

[0060] For each parsing task, generate at least one tool search term;

[0061] The most relevant tools to be used, retrieved from the tool library using the respective tool search terms, are collected to form an initial tool candidate set.

[0062] In this embodiment, the parsing task can be understood as a subtask after the task to be completed is decomposed, and each parsing task can be completed by at least one tool to be used.

[0063] Specifically, the task to be completed is parsed using a large language model, decomposing it into at least one parsing task, ensuring that each parsing task can be completed by at least one tool. For each parsing task, a tool to be used to execute the parsing task is called from a tool library, and the tool library can contain at least one tool that meets the specified conditions. Therefore, the called tools can be arbitrarily combined to obtain multiple initial candidate sets of tools capable of completing the task.

[0064] For example, Figure 2 This is a flowchart of a task decomposition method according to Embodiment 1 of the present invention. Figure 2 As shown, the task to be completed can be parsed using a large language model, breaking it down into multiple parsing tasks, such as Task 1: querying data, and Task 2: predicting load. Next, tools to be used to execute Task 1 and Task 2 are selected from the tool library. For example, the tool for Task 1 is a tool for querying data from the database, and the result of this tool's invocation is placed in Result 1. Similarly, the tool for Task 2 is a tool for predicting load using a predictive model, and the result of this tool's invocation is placed in Result 2. The large language model then aggregates the text from Result 1 and Result 2 to form the final response text for the complex task, which is placed in Result 3.

[0065] In one embodiment, after using the initial tool candidate set corresponding to the target quantification evaluation result as the tool set for performing the task to be completed and the upper confidence boundary, the method further includes:

[0066] The tasks to be completed and the tool call sequences for executing the tasks to be completed are stored in the training dataset;

[0067] The large language model is updated using the training dataset.

[0068] In this embodiment, the training dataset can be understood as a dataset used to train a large language model. The training dataset may contain tasks to be completed and toolsets for performing those tasks.

[0069] Specifically, after determining the toolset for executing the task, both the task and the toolset can be stored in the training dataset. The training dataset can then be used to optimize the large language model by adjusting its parameters.

[0070] For example, the tool call sequence of the toolset executing the task to be completed can be determined. Starting from the root node to the child node trajectory of the tool call sequence, a supervised fine-tuning algorithm can be used to train a large language model. This can involve adjusting the learning rate of the large language model or the hyperparameters of the optimizer. When the parameters of the large language model being fine-tuned are too large, low-rank fine-tuning can be performed on the large language model.

[0071] Example 2

[0072] Figure 3 This is a flowchart of a method for determining quantitative evaluation results according to Embodiment 2 of the present invention. This embodiment focuses on the method for determining quantitative evaluation results described in the above embodiment. Figure 3 As shown, the method includes:

[0073] S210. The task to be completed is parsed through the large language model in the tool call method, and at least one tool to be used is selected from the tool library to execute the task to be completed, so as to obtain an initial tool candidate set and determine the tree search framework corresponding to the initial tool candidate set.

[0074] S220. Using the large language model, based on the initial tool candidate set and the tree search framework, determine the tool invocation sequence.

[0075] S230. Quantitatively evaluate the task completion rate corresponding to the tool call sequence to obtain the quantified result of task completion.

[0076] In this embodiment, the task completion metric result can be understood as a quantitative score indicating the degree of task completion when the task to be completed is executed using the initial set of tool candidates.

[0077] For example, a quantitative assessment of task completion could be as follows: if the prediction accuracy is ≥90%, the task completion quantification result is awarded 1 point; otherwise, points are deducted proportionally.

[0078] S240. Quantitatively evaluate the time efficiency corresponding to the tool call sequence to obtain the time efficiency quantification result.

[0079] In this embodiment, the time efficiency quantification result can be understood as a quantitative score indicating the time efficiency when performing the task to be completed using the initial set of tool candidates.

[0080] For example, a quantitative evaluation of time efficiency can be: if the total time taken for a tool call sequence is less than 2 seconds, the time efficiency quantification result is 1 point; if it exceeds the time limit, the score decreases linearly.

[0081] S250. Quantitatively evaluate the resource consumption corresponding to the tool call sequence to obtain the resource consumption quantification result.

[0082] In this embodiment, the resource consumption quantification result can be understood as a quantified score indicating the resources consumed when performing the task to be completed using the initial set of tool candidates.

[0083] For example, a quantitative assessment of resource consumption could be: if the GPU utilization rate exceeds 50%, the quantitative result of resource consumption would be deducted by 0.2 points.

[0084] S260. Calculate the quantitative evaluation result corresponding to the tool call sequence based on the collaborative result corresponding to the tool call sequence, the task completion quantification result, the time efficiency quantification result, and the resource consumption quantification result.

[0085] The collaborative result indicates the result of the mutual influence between the tools to be used in the tool invocation sequence.

[0086] In this embodiment, the collaboration result indicates the result of the mutual influence between the tools to be used in the tool call sequence. The collaboration result is calculated based on the collaboration effect between the tools to be used.

[0087] Specifically, based on the synergistic effect between the tools to be used corresponding to the tool invocation sequence, the synergistic result corresponding to the tool invocation sequence is determined. Based on the above-mentioned task completion quantification result, time efficiency quantification result, and resource consumption quantification result, the quantitative evaluation result corresponding to the initial tool candidate set is calculated.

[0088] Optionally, calculating the quantitative evaluation result corresponding to the tool call sequence based on the collaboration result, the task completion metric result, the time efficiency metric result, and the resource consumption metric result corresponding to the tool call sequence includes:

[0089] Determine each tool to be used corresponding to the tool call sequence;

[0090] The collaboration results among the tools to be used are detected, including positive collaboration and negative collaboration.

[0091] When the collaboration result is positive collaboration, the sum of the task completion quantification result, the time efficiency quantification result, and the resource consumption quantification result is used as the quantitative evaluation result corresponding to the tool call sequence.

[0092] If the collaboration result is negative, the initial tool candidate set corresponding to the tool call sequence is marked as an alarm state, indicating that the initial tool candidate set cannot execute the task to be completed.

[0093] In this embodiment, the alarm status can be understood as indicating that there is a conflict between the tools to be used in the initial tool candidate set, which may lead to the failure or inefficiency of the tool to be called.

[0094] For example, when detecting the collaborative results between the tools to be used, such as tool A and tool B, tool A and tool B can be scored separately to obtain a score result. and Then, the collaborative working state of tool A and tool B is scored to obtain the score results. .when ( For calibration value When collaboration results are positive, the sum of the quantified results of task completion, time efficiency, and resource consumption can be used as the quantitative evaluation result corresponding to the initial tool candidate set. When a collaboration result is negative, the initial tool candidate set corresponding to the tool call sequence may cause the tool call to be completed to fail or be inefficient. Therefore, the initial tool candidate set is marked as an alarm state.

[0095] S270. The quantitative evaluation result is backtracked to the nodes of the tree search framework, and the reward value and upper confidence boundary value of the nodes of the tree search framework are updated.

[0096] The technical solution of this invention quantitatively evaluates the task completion rate corresponding to the tool call sequence to obtain a quantified task completion result; quantitatively evaluates the time efficiency corresponding to the tool call sequence to obtain a quantified time efficiency result; and quantitatively evaluates the resource consumption corresponding to the tool call sequence to obtain a quantified resource consumption result. Based on the collaboration result, the quantified task completion result, the quantified time efficiency result, and the quantified resource consumption result, a quantitative evaluation result corresponding to the tool call sequence is calculated. By determining the tree search framework corresponding to the initial tool candidate set, a comprehensive understanding of complex tasks to be completed is achieved. Furthermore, by determining the quantified evaluation result through the tool call sequence corresponding to the tree search framework, intelligent decision-making is realized when calling tools from the tool library.

[0097] Example 3

[0098] Figure 4 This is a schematic diagram of a tool calling device according to Embodiment 3 of the present invention. Figure 4 As shown, the device includes:

[0099] The parsing module 310 is used to parse the task to be completed through the large language model in the tool call method, select at least one tool to be used to execute the task to be completed from the tool library, obtain an initial tool candidate set, and determine the tree search frame corresponding to the initial tool candidate set. The number of initial tool candidate sets is at least one. The tool library stores text descriptions indicating the tools to be used. The root node of the tree search frame indicates the user instruction text. The nodes in the tree search frame other than the root node indicate the text concatenated by the text indicated by the parent node of the node and the selected tool to be used. The edge of the tree search frame indicates the next tool to be used selected from the initial tool candidate set starting from the current node of the tree search frame.

[0100] The determination module 320 is used to determine a tool invocation sequence based on the large language model, the initial tool candidate set, and the tree search framework, wherein the tool invocation sequence contains the tools to be used to perform the task to be completed;

[0101] Evaluation module 330 is used to perform quantitative evaluation on the tool call sequence and determine the quantitative evaluation result corresponding to the tool call sequence;

[0102] The update module 340 is used to backtrack the quantitative evaluation results to the nodes of the tree search framework and update the reward value and upper confidence boundary value of the nodes of the tree search framework.

[0103] The tool invocation device provided in this embodiment of the invention utilizes a parsing module to parse the task to be completed using a large language model in the tool invocation method, and selects at least one tool to be used from the tool library to execute the task, obtaining an initial tool candidate set; a determining module uses the large language model to determine the tool invocation sequence based on the initial tool candidate set and the tree search framework; an evaluation module performs a quantitative evaluation on the tool invocation sequence to determine the quantitative evaluation result corresponding to the tool invocation sequence; and an updating module backtracks the quantitative evaluation result to the nodes of the tree search framework, updating the reward value and upper confidence bound value of the nodes of the tree search framework. Through the cooperation between the modules, the device achieves an understanding of complex tasks to be completed, intelligent decision-making when invoking tools from the tool library, and automated invocation of multiple tools to be used to execute complex tasks.

[0104] In one embodiment, the determining module 320 is specifically used for:

[0105] The predicted probability of each candidate tool to be used being invoked in the large language model is weighted and combined with the upper confidence bound of the corresponding node in the tree search framework to obtain a weighted combination value. Based on the weighted combination value, the existing tool invocation path in the tree search framework is selected as the tool invocation sequence.

[0106] If the task to be completed is not resolved after the tool corresponding to the tool call path is called, a new child node is created on the node of the tool call path to obtain an updated tool call sequence. The child node stores the text concatenated by the text indicated by the parent node corresponding to the child node and the selected tool to be used.

[0107] In one embodiment, the evaluation module 330 includes:

[0108] The first evaluation unit is used to quantitatively evaluate the task completion degree corresponding to the tool call sequence and obtain the task completion quantification result.

[0109] The second evaluation unit is used to quantitatively evaluate the time efficiency corresponding to the tool call sequence and obtain the time efficiency quantification result.

[0110] The third evaluation unit is used to quantitatively evaluate the resource consumption corresponding to the tool call sequence and obtain the quantitative result of resource consumption.

[0111] The calculation unit is used to calculate the quantitative evaluation result corresponding to the tool call sequence based on the collaborative result corresponding to the tool call sequence, the task completion quantification result, the time efficiency quantification result, and the resource consumption quantification result.

[0112] The collaboration result indicates the result of the interaction between the tools to be used in the initial tool candidate set.

[0113] In one embodiment, the computing unit is specifically used for:

[0114] Determine each tool to be used corresponding to the tool call sequence;

[0115] The collaboration results among the tools to be used are detected, including positive collaboration and negative collaboration.

[0116] When the collaboration result is positive collaboration, the sum of the task completion quantification result, the time efficiency quantification result, and the resource consumption quantification result is used as the quantitative evaluation result corresponding to the tool call sequence.

[0117] If the collaboration result is negative, the initial tool candidate set corresponding to the tool call sequence is marked as an alarm state, indicating that the initial tool candidate set cannot execute the task to be completed.

[0118] In one embodiment, the parsing module 310 includes a tool library determination operation, specifically used for:

[0119] Obtain the tool to be used, and the metadata corresponding to the tool to be used, wherein the metadata indicates the function of the tool to be used;

[0120] Using an embedding generative model, the metadata is used to generate a semantic embedding vector corresponding to the tool to be used;

[0121] The semantic embedding vector corresponding to the tool to be used is stored in the tool library.

[0122] In one embodiment, the parsing module 310 includes:

[0123] The task to be completed is parsed by the large language model in the tool call method, and the task to be completed is decomposed into at least one parsing task.

[0124] For each parsing task, generate at least one tool search term;

[0125] The most relevant tools to be used, retrieved from the tool library using the respective tool search terms, are collected to form an initial tool candidate set.

[0126] In one embodiment, the tool invocation device further includes a storage module, specifically used for:

[0127] The tasks to be completed and the tool call sequences for executing the tasks to be completed are stored in the training dataset;

[0128] The large language model is updated using the training dataset.

[0129] The tool invocation device provided in this embodiment of the invention can execute the tool invocation method provided in any embodiment of the invention. Through the cooperation and coordination between the modules, the tool for executing the task is invoked, and it has the corresponding functional modules and beneficial effects of the execution method.

[0130] Example 4

[0131] According to embodiments of the present invention, the present invention also provides an electronic device, a computer-readable storage medium, and a computer program product.

[0132] Figure 5This is a block diagram of an electronic device according to Embodiment 4 of the present invention, which implements the tool invocation method described in the embodiments of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0133] like Figure 5 As shown, the electronic device 410 includes at least one processor 411 and a memory, such as a read-only memory (ROM) 412 or a random access memory (RAM) 413, communicatively connected to the at least one processor 411. The memory stores computer programs executable by the at least one processor. The processor 411 can perform various appropriate actions and processes based on the computer program stored in the ROM 412 or loaded from storage unit 418 into the RAM 413. The RAM 413 may also store various programs and data required for the operation of the electronic device 410. The processor 411, ROM 412, and RAM 413 are interconnected via a bus 414. An input / output (I / O) interface 415 is also connected to the bus 414.

[0134] Multiple components in the electronic device are connected to the I / O interface 415, including: an input unit 416, such as a keyboard, mouse, etc.; an output unit 417, such as various types of displays, speakers, etc.; a storage unit 418, such as a disk, optical disk, etc.; and a communication unit 419, such as a network card, modem, wireless transceiver, etc. The communication unit 419 allows the electronic device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0135] Processor 411 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 411 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 411 performs the various methods and processes described above, such as tool invocation methods.

[0136] In some embodiments, the tool invocation method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 418. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 410 via ROM 412 and / or communication unit 419. When the computer program is loaded into RAM 413 and executed by processor 411, one or more steps of the tool invocation method described above may be performed. Alternatively, in other embodiments, processor 411 may be configured to execute the tool invocation method by any other suitable means (e.g., by means of firmware).

[0137] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0138] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0139] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0140] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0141] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0142] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0143] In some embodiments, the computer program product includes a computer program that, when executed by a processor, implements the tool invocation method provided in the embodiments of the present invention.

[0144] The technical solution of this invention includes a tool invocation method, apparatus, electronic device, and storage medium. The tool invocation method uses a large language model to parse the task to be completed and selects at least one tool from a tool library required to execute the task, obtaining an initial tool candidate set. Using the large language model, based on the initial tool candidate set and a tree search framework, a tool invocation sequence is determined. The tool invocation sequence is quantitatively evaluated to determine the corresponding quantitative evaluation result. The quantitative evaluation result is then backtracked to the nodes of the tree search framework, updating the reward value and upper confidence bound of the nodes. This achieves understanding of complex tasks to be completed, intelligent decision-making when invoking tools from the tool library, and automated invocation of multiple tools to execute complex tasks.

[0145] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0146] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A tool invocation method, characterized by, include: The task to be completed is parsed by the large language model in the tool invocation method, and at least one tool to be used to execute the task is selected from the tool library to obtain an initial tool candidate set. The tree search frame corresponding to the initial tool candidate set is determined. The number of initial tool candidate sets is at least one. The tool library stores text descriptions indicating the tools to be used. The root node of the tree search frame indicates the user instruction text. The nodes in the tree search frame other than the root node indicate the text concatenated by the text indicated by the parent node of the node and the selected tool to be used. The edge of the tree search frame indicates the next tool to be used selected from the initial tool candidate set starting from the current node of the tree search frame. Using the large language model, based on the initial tool candidate set and the tree search framework, a tool invocation sequence is determined, wherein the tools to be used in the tool invocation sequence are used to execute the task to be completed; The tool call sequence is quantitatively evaluated to determine the quantitative evaluation result corresponding to the tool call sequence; The quantitative evaluation results are backtracked to the nodes of the tree search framework to update the reward value and upper confidence bound of the nodes of the tree search framework.

2. The method of claim 1, wherein, The process of determining the tool invocation sequence using the large language model, based on the initial tool candidate set and the tree search framework, includes: The predicted probability of each candidate tool being invoked by the large language model is weighted and combined with the upper confidence bound of the corresponding node in the tree search framework to obtain a weighted combination value. Based on the weighted combination value, the existing tool invocation path in the tree search framework is selected as the tool invocation sequence. If the task to be completed is not resolved after the tool corresponding to the tool call path is called, a new child node is created on the node of the tool call path to obtain an updated tool call sequence. The child node stores the text concatenated by the text indicated by the parent node corresponding to the child node and the selected tool to be used.

3. The method of claim 1, wherein, The step of quantitatively evaluating the tool call sequence and determining the quantitative evaluation result corresponding to the tool call sequence includes: The task completion rate corresponding to the tool call sequence is quantitatively evaluated to obtain the quantified result of task completion. The time efficiency corresponding to the tool call sequence is quantitatively evaluated to obtain the time efficiency quantification result; The resource consumption corresponding to the tool call sequence is quantitatively evaluated to obtain the quantitative result of resource consumption. Based on the collaborative results corresponding to the tool call sequence, the quantified results of task completion, the quantified results of time efficiency, and the quantified results of resource consumption, calculate the quantitative evaluation results corresponding to the tool call sequence. The collaboration result indicates the result of the interaction between the tools to be used in the initial tool candidate set.

4. The method according to claim 3, characterized in that, The step of calculating the quantitative evaluation result corresponding to the tool call sequence based on the collaboration result, the task completion metric result, the time efficiency metric result, and the resource consumption metric result includes: Determine each tool to be used corresponding to the tool call sequence; The collaboration results among the tools to be used are detected, including positive collaboration and negative collaboration. When the collaboration result is positive collaboration, the sum of the task completion quantification result, the time efficiency quantification result, and the resource consumption quantification result is used as the quantitative evaluation result corresponding to the tool call sequence. If the collaboration result is negative, the initial tool candidate set corresponding to the tool call sequence is marked as an alarm state, indicating that the initial tool candidate set cannot execute the task to be completed.

5. The method according to claim 1, characterized in that, The determination operation of the tool library includes: Obtain the tool to be used, and the metadata corresponding to the tool to be used, wherein the metadata indicates the function of the tool to be used; Using an embedding generative model, the metadata is used to generate a semantic embedding vector corresponding to the tool to be used; The semantic embedding vector corresponding to the tool to be used is stored in the tool library.

6. The method according to claim 1, characterized in that, The method involves parsing the task to be completed using a large language model and selecting at least one tool from the tool library required to execute the task, resulting in an initial tool candidate set, including: The task to be completed is parsed by the large language model in the tool call method, and the task to be completed is decomposed into at least one parsing task. For each parsing task, generate at least one tool search term; The most relevant tools to be used, retrieved from the tool library using the respective tool search terms, are collected to form an initial tool candidate set.

7. The method according to claim 1, characterized in that, After backtracking the quantitative evaluation result to the nodes of the tree search framework and updating the reward value and upper confidence bound of the nodes of the tree search framework, the method further includes: The tasks to be completed and the tool call sequences for executing the tasks to be completed are stored in the training dataset; The large language model is updated using the training dataset.

8. A tool calling device, characterized in that, include: The parsing module is used to parse the task to be completed through the large language model in the tool invocation method, select at least one tool to be used to execute the task from the tool library, obtain an initial tool candidate set, and determine the tree search frame corresponding to the initial tool candidate set. The number of initial tool candidate sets is at least one. The tool library stores text descriptions indicating the tools to be used. The root node of the tree search frame indicates the user instruction text. The nodes in the tree search frame other than the root node indicate the text concatenated by the text indicated by the parent node of the node and the selected tool to be used. The edges of the tree search frame indicate the next tool to be used selected from the initial tool candidate set starting from the current node of the tree search frame. The determination module is used to determine a tool invocation sequence based on the large language model, the initial tool candidate set, and the tree search framework, wherein the tool invocation sequence contains the tools to be used to execute the task to be completed; An evaluation module is used to perform quantitative evaluation on the tool call sequence and determine the quantitative evaluation result corresponding to the tool call sequence; An update module is used to backtrack the quantitative evaluation results to the nodes of the tree search framework and update the reward value and upper confidence bound of the nodes of the tree search framework.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor to enable the at least one processor to perform the tool invocation method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the tool invocation method of any one of claims 1-7.

Images