Tool calling method and device based on large language model

Abstract

The present application discloses a tool calling method based on a large language model, the method comprising: obtaining a question text, the question text including task information; obtaining a first set of candidate tools that match the question text with a preset tool library; screening the first set of candidate tools based on currently available local hardware resources and / or currently available online resources of a cloud application programming interface (API) to obtain a second set of candidate tools; determining a target tool that matches the task information from the second set of candidate tools based on the large language model, generating a tool calling instruction based on the target tool; selecting a target tool to be called based on the tool calling instruction, and obtaining an output result of the target tool; and generating a reply text corresponding to the large language model based on the question text and the output result. The present application can combine the resource usage of tools on multiple platforms to obtain tools with a higher degree of matching, thereby achieving efficient resource utilization and dynamic scheduling of task tools.

Description

Technical Field

[0001] The present application relates to the technical field of large language models, and in particular to a tool calling method and device based on a large language model. Background Art

[0002] The rapid development of artificial intelligence (AI) technology has demonstrated powerful text generation capabilities in the field of natural language processing, enabling answers to questions or queries. However, faced with complex task demands, single platforms or large, single-modal language models often struggle to provide accurate and comprehensive responses. Summary of the Invention

[0003] The purpose of this application is to provide a tool calling method based on a large language model, combining the resource usage of tools on multiple platforms to obtain tools with higher matching degrees, achieve efficient resource utilization and dynamic scheduling of task tools.

[0004] In a first aspect, the present application provides a tool calling method based on a large language model, the method comprising:

[0005] Obtaining a question text, wherein the question text includes task information;

[0006] Obtaining a first candidate tool set that matches the question text with a preset tool library, wherein the tool library includes a plurality of tools;

[0007] Based on currently available local hardware resources and / or currently available online resources of a cloud application programming interface (API), the first candidate tool set is screened to obtain a second candidate tool set;

[0008] determining a target tool matching the task information from the second candidate tool set based on the large language model, and generating a tool calling instruction based on the target tool;

[0009] Selecting a target tool to be called based on the tool calling instruction, so that the target tool executes the task corresponding to the task information, and obtaining an output result of the target tool;

[0010] Based on the question text and the output results, the large language model generates the answer text corresponding to the question text.

[0011] Optionally, the method further includes a training method for a large language model, the training method including:

[0012] Constructing a first problem sample and determining a plurality of first tools that need to be called in the first problem sample;

[0013] Split the first problem sample according to graph logic and / or tree logic to obtain multiple sub-problems in the first problem sample;

[0014] logically linking the plurality of first tools according to the plurality of sub-problems to obtain a tool chain;

[0015] The tool chain is disassembled in a defective manner to obtain a first tool sample set with multiple permutations and combinations;

[0016] The large language model is trained using the first question sample and a plurality of first tool sample sets.

[0017] Optionally, the method further includes a training method for a large language model, the training method including:

[0018] Constructing a second problem sample and determining a plurality of second tools that need to be called in the second problem sample;

[0019] Performing a plurality of permutations and combinations of the plurality of second tools in a defective manner, and / or adding a plurality of tools unrelated to the plurality of second tools, to obtain a plurality of second tool sample sets;

[0020] The large language model is trained using the second question sample and a plurality of second tool sample sets.

[0021] Optionally, the tools include local tools and online tools corresponding to the cloud application programming interface (API). Based on currently available local hardware resources and / or currently available online resources of the cloud API, the first candidate tool set is screened to obtain a second candidate tool set, including:

[0022] If the currently available local hardware resources are less than the first threshold, the local tools whose hardware resources are greater than the second threshold are removed from the first candidate tool set to obtain a second candidate tool set;

[0023] and / or,

[0024] If the current online resources available for the cloud API are less than the third threshold, the cloud APIs corresponding to the online tools whose computing resources are greater than the fourth threshold in the first candidate tool set are eliminated to obtain a second candidate tool set, where the current online resources available for the cloud API include computing resources and bandwidth resources.

[0025] Optionally, the tools include local tools and online tools corresponding to the cloud application programming interface (API). Based on currently available local hardware resources and / or currently available online resources of the cloud API, the first candidate tool set is screened to obtain a second candidate tool set, including:

[0026] If there are local tools and online tools with the same functions in the first candidate tool set, and if the current locally available hardware resources meet the hardware resources required by the local tools, the cloud API corresponding to the online tool is removed from the first candidate tool set to obtain the second candidate tool set.

[0027] Optionally, the tool library stores tool information of the tools, including type, name, function, tool parameters, description information, and requirements. Obtaining a first set of candidate tools that match the question text with a preset tool library includes:

[0028] Get the text embedding vector of the question text;

[0029] Get the tool embedding vector of the tool description of each tool;

[0030] The retriever calculates the similarity between the text embedding vector and the tool embedding vector of each tool, and determines the K tools with the highest similarity as the first candidate tool set.

[0031] Optionally, the tool library stores tool information of the tools, including type, name, function, tool parameters, description information, and requirements. Determining a target tool associated with the task information from the second set of candidate tools based on the large language model and generating a tool call instruction based on the target tool include:

[0032] Structuring the tool information of each tool to obtain the corresponding Json description information of each tool;

[0033] Add the Json description information of each tool in the second candidate tool set to the initial prompt word template to obtain an adjusted prompt word template, wherein the initial prompt word template includes the system prompt word and the task type template;

[0034] Based on the adjusted prompt word template, the information of each tool in the second candidate tool set, and the question text, the large language model determines the target tool that matches the task information;

[0035] A tool calling instruction is generated based on the tool information of the target tool, wherein the tool calling instruction includes a tool name and tool parameters.

[0036] Optionally, selecting a target tool to be called based on the tool calling instruction so that the target tool performs the task corresponding to the task information and obtaining an output result of the target tool includes:

[0037] Generate structured function call instructions based on tool call instructions, and output the structured function call instructions to the FastAPI platform;

[0038] The FastAPI platform uses asynchronous monitoring to monitor structured function call instructions;

[0039] The FastAPI platform selects the target tool to call based on the structured function call instructions;

[0040] The FastAPI platform monitors the execution status of the target tool in real time to obtain the output results of the target tool.

[0041] In a second aspect, the present application provides a tool calling device based on a large language model, the device comprising:

[0042] An acquisition module, used for acquiring question text, wherein the question text includes task information;

[0043] A retrieval module is used to obtain a first set of candidate tools that match the question text with a preset tool library, wherein the tool library includes a plurality of tools;

[0044] A screening module, configured to screen the first candidate tool set based on currently available local hardware resources and / or currently available online resources of a cloud application programming interface (API) to obtain a second candidate tool set;

[0045] a language model module, configured to determine a target tool matching the task information from the second candidate tool set based on the large language model, and generate a tool calling instruction based on the target tool;

[0046] A tool calling module is used to select a target tool to be called based on a tool calling instruction, so as to make the target tool execute the task corresponding to the task information and obtain the output result of the target tool;

[0047] The language model module is also used to generate a response text corresponding to the question text based on the question text and the output result.

[0048] In a second aspect, the present application provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the tool calling method based on the large language model as described above are implemented.

[0049] This application uses a tool scheduling strategy that uses a mixed call of local tools and cloud-based online tools based on the obtained question text and the matching first candidate tool set, combined with the resource usage of tools on multiple platforms, to further screen the first candidate tool set to obtain a better second candidate tool set. Based on the large language model, the target tool that matches the task information is determined from the second candidate tool set to realize the call of the target tool. By screening the tools at multiple levels, not only can the tool that better matches the task information be obtained, but the task tools and resources can also be dynamically combined to achieve efficient resource utilization and dynamic scheduling of task tools, thereby achieving the best balance between performance and cost, enhancing the ability to respond to task requirements, and improving the adaptability and scalability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1A first flow chart of a tool calling method based on a large language model provided in an embodiment of the present application;

[0051] Figure 2 A second flow chart of the tool calling method based on a large language model provided in an embodiment of the present application;

[0052] Figure 3 A third flow chart of a tool calling method based on a large language model provided in an embodiment of the present application;

[0053] Figure 4 A fourth flow chart of a tool calling method based on a large language model provided in an embodiment of the present application;

[0054] Figure 5 A fifth flow chart of a tool calling method based on a large language model provided in an embodiment of the present application;

[0055] Figure 6 A sixth flow chart of a tool calling method based on a large language model provided in an embodiment of the present application;

[0056] Figure 7 A system block diagram of a tool calling device based on a large language model provided in an embodiment of the present application;

[0057] Figure 8 A system block diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0058] The present application will be described in detail below in conjunction with the specific embodiments shown in the accompanying drawings, but these embodiments do not limit the present application. Structural, methodological, or functional changes made by ordinary technicians in this field based on these embodiments are included in the scope of protection of the present application.

[0059] Please refer to Figure 1 , an embodiment of the present application provides a tool calling method based on a large language model, the method comprising steps S101-S106.

[0060] S101: Obtain a question text, wherein the question text includes task information.

[0061] For example, a question text input by a user can be obtained based on human-computer interaction. The question text is used to represent the question. The question text can be information in any one or more modalities, such as text, image, and audio.

[0062] The question text includes task information, which is text information based on natural language description of the task to be performed subsequently. For example, "Please translate the following sentence into English" is a text translation task. In subsequent processing, the model or tool corresponding to the text translation task is retrieved.

[0063] S102: Obtain a first set of candidate tools that match the question text with a preset tool library, wherein the tool library includes a plurality of tools.

[0064] The tool library includes multiple tools. The tools include local tools and online tools corresponding to the cloud application programming interface (API). Tools can also be understood as models, which is not limited here. The cloud application programming interface (API) can be understood as a functional service interface deployed on the cloud platform, usually presented in HTTP / HTTPS style, supporting remote calls of models or systems to complete tasks. For example, tools may include text question-and-answer tools, counter tools, network knowledge retrieval tools, target detection, image recognition, image segmentation, text maps, etc. Tools can be local model encapsulations, or remote APIs (such as Baidu Encyclopedia API, Open API calls, etc.) to meet the needs of different fields or different tasks.

[0065] Based on the question text, a search is performed in the tool library to obtain various tools that match the question text, and the various tools are used to form a first candidate tool set.

[0066] S103 , based on currently available local hardware resources and / or currently available online resources of a cloud application programming interface (API), the first candidate tool set is screened to obtain a second candidate tool set.

[0067] Based on the obtained first candidate tool set, the available resources locally and in the cloud are dynamically monitored. According to the current hardware resources available locally and / or the current online resources available through the cloud application programming interface (API), each tool in the first candidate tool set is further screened to obtain better tools to form a second candidate tool set, thereby achieving efficient utilization of local and cloud resources.

[0068] S104 : determining a target tool that matches the task information from the second candidate tool set based on the large language model, and generating a tool calling instruction based on the target tool.

[0069] The large language model can be understood as a pre-trained large language model. Based on this large language model, semantic understanding of the question text is performed. After determining the second set of candidate tools, since there are still a large number of candidate tools, the large language model is used to filter the candidate tools to identify target tools that match the question text. Therefore, based on the large language model's semantic understanding of the question text, a target tool that matches the task information is identified from the second set of candidate tools. Based on the target tool's tool information, a tool call instruction is generated.

[0070] The number of target tools can be one or more, depending on the number of tools required by the question text. Some tasks require a target tool. For example, if the question is "What's the weather like today?", the target tool is the weather tool. If the question is "Recognize text in an image and translate it into English," the target tools are both the image recognition tool and the translation tool.

[0071] For example, InternVL2.0 is used as the foundation of a large language model, and an autoregressive approach is used to achieve semantic understanding. This allows the multimodal large language model trained on this foundation to predict the next token based on its own previously generated tokens, forming a sequential language output.

[0072] Exemplarily, the above-mentioned autoregressive method can adopt the following autoregressive function, so that the large language model base can be trained by the following loss function, so that the output is closer to the natural distribution of real data.

[0073] ;

[0074] Represents the predicted output (token) at time step t, Represents the output of the previous time step (Token). Represents the model parameters of the large language model, which can be a vector or a set of parameters that need to be learned through training data, representing the mapping from the observed historical sequence to the next output distribution. Given all previous observations Under the conditions of and model parameters θ, it is observed that The conditional probability of all observations before time step t is input into a large language model with parameter θ, and the output of the current time step t is predicted based on the large language model The probability distribution of . In the training process of the large language model, by minimizing the loss function , maximizing the log-likelihood of predicting the true target at all time steps, thereby approximating the true data distribution. After the large language model is trained, the trained parameters θ are obtained. During inference, based on the trained model parameters θ, the large language model can predict new input sequences in an autoregressive manner.

[0075] S105 , selecting a target tool to be called based on the tool calling instruction, so that the target tool executes the task corresponding to the task information, and obtains an output result of the target tool.

[0076] According to the tool calling instruction, a general target tool is selected, the target tool executes the task information in the problem text, and outputs the result.

[0077] S106: Based on the question text and the output result, the large language model generates a response text corresponding to the question text.

[0078] The large language model integrates the question text and output results, adjusts or supplements the output content, and outputs the answer text corresponding to the question text.

[0079] In an embodiment of the present application, a first candidate tool set that matches the question text is obtained, and combined with the resource usage of tools on multiple platforms, a tool scheduling strategy that mixes local tools with online tools in the cloud is adopted to further screen the first candidate tool set to obtain a better second candidate tool set. Based on the large language model, a target tool that matches the task information is determined from the second candidate tool set to implement the call of the target tool. By screening tools at multiple levels, not only can tools that better match the task information be obtained, but task tools and resources can also be dynamically combined to achieve efficient resource utilization and dynamic scheduling of task tools, thereby achieving the best balance between performance and cost.

[0080] One embodiment of the present application, such as Figure 2 As shown, the method also includes a training method for a large language model, and the training method includes the steps of:

[0081] S201, constructing a first problem sample, and determining a plurality of first tools that need to be called in the first problem sample;

[0082] S202: Split the first question sample according to graph logic and / or tree logic to obtain multiple sub-questions of the first question sample;

[0083] S203, logically linking the multiple first tools according to the multiple sub-problems to obtain a tool chain;

[0084] S204, disassembling the tool chain in a defective manner to obtain a first tool sample set with multiple permutations and combinations;

[0085] S205 : Training a large language model using the first question sample and a plurality of first tool sample sets.

[0086] In an embodiment of the present application, during the training process of a large language model, the logical order of the tools in the sample tool set is shuffled to eliminate the influence of the model on the tool calling preference, while also avoiding the model's dependence on the order and position of the input sequence. Different tool combinations are generated through random sampling, random shuffling, etc. to enhance the diversity of tool sample combinations, improve the semantic understanding and robustness of the large language model for tools, and increase the model's ability to refuse to answer.

[0087] For example, a first problem sample is constructed and then split according to graph logic and / or tree logic to obtain multiple sub-problems within the first problem sample. Each sub-problem corresponds to a tool, which can be used to determine the multiple first tools that need to be called for the first problem sample. These multiple first tools are logically linked according to the multiple sub-problems to obtain a tool chain. A tool chain represents a combination of multiple tools in which the output of the previous tool serves as the input of the subsequent tool. For example, if the first problem sample is "Help me identify this image and translate it into English," the tool chain called is: image recognition tool → text translation tool. The tool chain is disassembled in a defective manner to obtain a plurality of permutations and combinations of first tool sample sets. For example, a first tool sample set is constructed that includes all the first tools in the tool chain, such as the first tool sample set is {image recognition, text translation, search engine}. After model training, the large language model can output normal tool call results; a first tool sample set includes part of the first tools in the tool chain, such as the first tool sample set is {image recognition, search engine}. After model training, the large language model can output "the tool is missing in the tool set and cannot be called and returned"; a first tool sample set does not include all the first tools in the tool chain, such as the first tool sample set is {search engine}. After model training, the large language model can output "there is no executable tool for the current task", which can realize the semantic understanding ability of the large language model, improve the accuracy of the large language model, and enhance the large language model's ability to handle tool missing and refuse to answer.

[0088] One embodiment of the present application, such as Figure 3 As shown, the method also includes a training method for a large language model, and the training method includes the steps of:

[0089] S301, constructing a second problem sample, and determining a plurality of second tools that need to be called in the second problem sample;

[0090] S302, performing a plurality of permutations and combinations of the plurality of second tools in a missing manner, and / or adding a plurality of tools unrelated to the plurality of second tools, to obtain a plurality of second tool sample sets;

[0091] S303: Training the large language model using the second question sample and multiple second tool sample sets.

[0092] Construct a second problem sample, determine multiple second tools that need to be called in the second problem sample, perform multiple permutations and combinations of the multiple second tools in a missing manner, and / or add multiple tools that are not related to the multiple second tools to obtain multiple second tool sample sets.

[0093] For example, a second tool sample set may be constructed from multiple second tools, a second tool sample set may include multiple second tools and other tools other than the multiple second tools, and a second tool sample set may be missing at least one second tool. In this embodiment, by constructing multiple tool sample sets and considering data on combinations of tool inclusion, missingness, and redundancy in the tool sets, the large language model is trained to improve its ability to understand tool semantics and achieve generalization for unknown tools.

[0094] One embodiment of the present application, such as Figure 4 As shown, the tool library stores tool information of tools, including type, name, parameter description information and requirements. The process of obtaining a first candidate tool set that matches the question text with the preset tool library includes the following steps:

[0095] S401, obtaining a text embedding vector of the question text;

[0096] S402, obtaining a tool embedding vector of a tool description of each tool;

[0097] S403: Calculate the similarity between the text embedding vector and the tool embedding vector of each tool through the retriever, and determine the K tools with the highest similarity as the first candidate tool set.

[0098] Obtain the question text entered by the user and determine the text embedding vector for the question text. Text embedding is a technique that converts text content (such as words, sentences, or paragraphs) into a dense vector of fixed length. This technique captures the semantic and contextual information of the text, ensuring that semantically related words are close together in the embedding space and unrelated words are farther apart. The semantic relationship between words can be represented by the distance between the vectors. For example, an embedding model can be used to represent the question text in vector form. For example, the embedding model can be a BGE Embedding model, which uses the BGE Embedding model to perform a vector representation of the question text and obtain a text embedding vector for the question text.

[0099] For example, an embedding model is used to obtain a tool embedding vector of a tool description. The embedding model may be a BGE Embedding model.

[0100] The retriever calculates the similarity between the text embedding vector and the tool embedding vector of each tool, and determines the K tools with the highest similarity as the first candidate tool set, thereby determining the tool set that is semantically relevant to the task information in the question text. For example, the similarity can be cosine similarity. For example, the retriever is a BGE Ranker re-ranking model.

[0101] Assuming that the task-related text of task A includes question text and prompt words, the tool description T of the task-related text is input into the BGE Ranker rearrangement model, and BGE encoding is performed to obtain the text embedding vector. The tool description is input into the BGE Ranker rearrangement model, BGE encoding is performed to obtain the tool embedding vector, the cosine similarity between the text embedding vector and the tool embedding vector is calculated, and the Top-K tools with the highest cosine similarity are determined.

[0102] An example is as follows:

[0103]

[0104]

[0105]

[0106] Top-K Tools

[0107] in, The text embedding vector representing the task-related text T, Characterization Toolset The tool embedding vector corresponding to the tool description. For each tool in the Top-K tools, a unique hash value is generated for the tool description of the tool, and the hash value is used as the key (key) and the corresponding tool identifier as the value (Value) to construct a key-value pair for the tool, and the key-value pair is stored in an efficient search structure hash table (Hash Map) or key-value memory (Key-Value Store). The tool identifier is used to represent the tool name. When the system needs to quickly identify or obtain detailed information about the tool based on the tool description, it can calculate the hash value of the tool description to be queried, and search in the search structure hash table or key-value memory based on the calculated hash value. When the matching hash value is queried, the corresponding tool identifier or tool-related detailed information can be quickly retrieved, so that the tool associated with the task can be retrieved more quickly.

[0108] By calculating the similarity between the semantic vector of task information and the semantic vector of tool information in the tool library, a high-confidence tool set can be generated, which will greatly reduce the context occupancy of the subsequent large language model and increase the diversity of tool types in the context of the large language model.

[0109] Exemplarily, the tool embedding vector of each tool is hash-encoded to achieve more efficient retrieval.

[0110] In one embodiment of the present application, the tools include local tools and online tools corresponding to the cloud application programming interface (API). Based on the current hardware resources available locally and / or the current online resources available to the cloud API, the first candidate tool set is screened to obtain a second candidate tool set, including: if the current hardware resources available locally are less than a first threshold, the local tools in the first candidate tool set whose hardware resources are greater than the second threshold are eliminated to obtain a second candidate tool set.

[0111] For example, hardware resources may include graphics processor resources, which include graphics processor memory usage, total graphics memory, and graphics processor utilization. Graphics processor resources may be determined based on the platform, and the first and second thresholds may be determined based on actual conditions or may be dynamic thresholds. A command may be used to obtain the current graphics processor resource usage of the platform, thereby obtaining the currently available graphics processor resources.

[0112] In one embodiment of the present application, the tools include local tools and online tools corresponding to cloud application programming interfaces (APIs). Based on the currently available local hardware resources and / or the currently available online resources of the cloud APIs, a first set of candidate tools is screened to obtain a second set of candidate tools, including: if the currently available online resources of the cloud APIs are less than a third threshold, the cloud APIs corresponding to the online tools in the first set of candidate tools whose computing resources are greater than a fourth threshold are removed to obtain the second set of candidate tools, wherein the currently available online resources of the cloud APIs include computing resources and bandwidth resources. The third and fourth thresholds can be determined based on actual conditions or can be dynamic thresholds.

[0113] For example, the online resources available to the cloud API can be queried through the API status interface provided by the cloud, and computing resources, bandwidth resources, etc. can be obtained. For example, if the computing resources of the online tool corresponding to a cloud API exceed 20% of the online resources available to the cloud API, the cloud API corresponding to the online tool will be removed from the second candidate tool set.

[0114] In one embodiment of the present application, the tools include local tools and online tools corresponding to cloud application programming interfaces (APIs). Based on the current hardware resources available locally and / or the current online resources available to the cloud APIs, the first candidate tool set is screened to obtain a second candidate tool set, including: if there are local tools and online tools with the same functions in the first candidate tool set, and if the current hardware resources available locally meet the hardware resources required by the local tools, the cloud APIs corresponding to the online tools are removed from the first candidate tool set to obtain the second candidate tool set. If two tools in the second candidate tool set can achieve the same function, it is considered a functional conflict, and the local tool is used first to reduce the tool overhead and increase the real-time performance of the tool.

[0115] In this embodiment, based on the local tool resource occupancy and the online tool resource occupancy corresponding to the cloud API, and the resource occupancy of tools on multiple platforms, the tool with the optimal computing resources can be dynamically selected according to task requirements to achieve the best balance between computing performance and cost, so as to achieve efficient resource utilization and dynamic scheduling of tasks.

[0116] One embodiment of the present application, such as Figure 5 As shown, determining a target tool associated with the task information from the second candidate tool set based on the large language model, and generating a tool call instruction based on the target tool, includes the following steps:

[0117] S501, structure the tool information of each tool to obtain the Json description information corresponding to each tool;

[0118] S502: Add the Json description information of each tool in the second candidate tool set to the initial prompt word template to obtain an adjusted prompt word template, wherein the initial prompt word template includes a system prompt word and a task type template;

[0119] S503, based on the adjusted prompt word template, the information of each tool in the second candidate tool set, and the question text, the large language model determines a target tool that matches the task information;

[0120] S504 : Generate a tool calling instruction based on the tool information of the target tool, wherein the tool calling instruction includes a tool name and tool parameters.

[0121] The tool call instruction can include type type, tool definition function, function includes tool name name, tool description description, parameter parameter, required type required, etc.

[0122] Assuming that the system prompt word in S502 can be represented by SP, the task type template is represented by Template(T), Template(T) can be set as a fixed template according to the task type, and the second candidate tool set is represented by FL, then the adjusted prompt word template P(T) can be expressed by the following formula:

[0123] P(T) =SP+Template(T)+FL.

[0124] For example, the tool information of each tool is structured to obtain the corresponding JSON description information of each tool. The tool information includes type, name, parameter description information and requirements. The tool information is parameterized and described in JSON format.

[0125] For example, using the semantic-sam and clip tools as examples, each tool's description format can use JSON format, including type, function, name, description, parameters, and required fields. For example, type represents the tool function type; function represents the function of the tool called by the tool, and function can include name and description, where name can be the tool name and description can be a description of the tool's core functionality; parameters represent the parameters included in the tool, assuming the parameters include image and boxes, where image represents the path of the input image, in string format (such as a local path or URL); boxes represents the coordinates of the target area, in string format or JSON array format (such as [[x1,y1,x2,y2], ...]); required indicates that the image and boxes parameters must be provided when calling.

[0126] For example, the description is "According to the coordinate information in the input boxes parameter, the image is segmented, and then different fine-grained segmentation results are output." For another example, the description is "The path to the image used to pass to the model as input."

[0127] The JSON description of each tool in the second candidate tool set is added to the initial prompt word template to generate an adjusted prompt word template. The initial prompt word template includes the system prompt word and the task type template. Based on the adjusted prompt word template, the information about each tool in the second candidate tool set, and the question text, the large language model determines the target tool associated with the task information. Based on the target tool's tool information, a tool call instruction is generated. The tool call instruction may include the tool name and tool parameters.

[0128] In this embodiment, the Json description information of the tool is added to the system prompt word. Under the guidance of the system prompt word, the large language model can better understand the tool semantic information, so as to obtain the target tool that better matches the task information.

[0129] One embodiment of the present application, such as Figure 6 As shown, selecting a target tool to be called based on a tool calling instruction so that the target tool performs the task corresponding to the task information and obtains the output result of the target tool includes:

[0130] S601, generating a structured function call instruction based on the tool call instruction, and outputting the structured function call instruction to the FastAPI platform;

[0131] S602, the FastAPI platform monitors the structured function call instruction in an asynchronous monitoring manner;

[0132] S603, the FastAPI platform selects a target tool to be called according to the structured function call instruction;

[0133] S604, the FastAPI platform monitors the execution status of the target tool in real time to obtain the output result of the target tool.

[0134] In this embodiment, the FastAPI framework is used as a monitoring mechanism for tool calls. The local model is suspended in the background through the framework, and tool calls are implemented through port access and callback mechanisms. For cloud APIs, access is performed through HTTP and other means. Based on the FastAPI framework, standardized tool interfaces, real-time status monitoring (supporting WebSocket and polling), dynamic tool calls and result returns, tool registration and discovery, error handling and retry mechanisms, as well as performance monitoring and logging can be implemented. This mechanism realizes efficient tool call and status management driven by task requirements. Local tools are deployed in this framework and called through asynchronous monitoring.

[0135] For example, the asynchronous monitoring method can be expressed by the following formula:

[0136] , Callback ;

[0137] Among them, D i Represents the i-th tool, Callback represents the callback function, and AsyncListen() represents the asynchronous listening function.

[0138] Exemplarily, a structured function call instruction is generated based on the tool call instruction, and the structured function call instruction is output to the FastAPI platform. A structured function call instruction in the function calling format can be used, and the structured function call instruction can include API_name and API_params, where API_name can be the tool name and API_params can be the tool parameter name. API_params can also be in the form of a dictionary containing tool parameter names and values. Function calling is a mechanism in which a large language model generates an executable function description in a specific format (structured (Json)). Through this mechanism, the large language model can call external tools, APIs, etc. to complete tasks that it cannot complete through text. The FastAPI platform serves as the tool call backend in the function calling mechanism of the large language model. By parsing the structured tool request, it dynamically calls the registered tool function and returns the output result of the tool call. Based on the question text and the output result, the large language model generates the answer text corresponding to the question text.

[0139] For example, a user asks, "Please tell me what animals are in this image?" The tool called is grounding_dino. The structured function call format is {"API_Name":"grounding_dino","API_Name":{"image_path":" / data / cats.jpg","prompt":"a cat on the table"}}. The grounding_dino call returns: "A cat was found." Based on the call result, the large language model generates a response: "According to the detection results, this image contains a cat, and the cat is on the table."

[0140] Based on the same inventive concept, an embodiment of the present application also provides a tool calling device based on a large language model. The implementation solution for solving the problem provided by this device is similar to the implementation solution recorded in the above method. Therefore, the specific limitations in the embodiments of one or more tool calling devices based on a large language model provided below can be found in the above limitations on the tool calling method based on a large language model, and will not be repeated here.

[0141] like Figure 7 As shown, the present application provides a tool calling device based on a large language model, the device comprising:

[0142] An acquisition module 701 is used to acquire a question text, wherein the question text includes task information;

[0143] A retrieval module 702 is configured to obtain a first set of candidate tools that match the question text with a preset tool library, wherein the tool library includes a plurality of tools;

[0144] A screening module 703 is configured to screen the first candidate tool set based on currently available local hardware resources and / or currently available online resources via a cloud application programming interface (API) to obtain a second candidate tool set;

[0145] A language model module 704 is configured to determine a target tool that matches the task information from the second candidate tool set based on the large language model, and generate a tool call instruction based on the target tool;

[0146] A tool calling module 705 is used to select a target tool to be called based on the tool calling instruction, so as to make the target tool execute the task corresponding to the task information and obtain the output result of the target tool;

[0147] The language model module 704 is further configured to generate a response text corresponding to the question text based on the question text and the output result.

[0148] Optionally, the language model module 704 is specifically configured to:

[0149] Constructing a first problem sample and determining a plurality of first tools that need to be called in the first problem sample;

[0150] Split the first problem sample according to graph logic and / or tree logic to obtain multiple sub-problems in the first problem sample;

[0151] logically linking the plurality of first tools according to the plurality of sub-problems to obtain a tool chain;

[0152] The tool chain is disassembled in a defective manner to obtain a first tool sample set with multiple permutations and combinations;

[0153] The large language model is trained using the first question sample and a plurality of first tool sample sets.

[0154] Optionally, the language model module 704 is specifically configured to:

[0155] Constructing a second problem sample and determining a plurality of second tools that need to be called in the second problem sample;

[0156] Performing a plurality of permutations and combinations of the plurality of second tools in a defective manner, and / or adding a plurality of tools unrelated to the plurality of second tools, to obtain a plurality of second tool sample sets;

[0157] The large language model is trained using the second question sample and a plurality of second tool sample sets.

[0158] Optionally, the tools include local tools and online tools corresponding to the cloud application programming interface (API). The screening module 703 is specifically used to:

[0159] If the currently available local hardware resources are less than the first threshold, the local tools whose hardware resources are greater than the second threshold are removed from the first candidate tool set to obtain a second candidate tool set;

[0160] and / or,

[0161] If the current online resources available for the cloud API are less than the third threshold, the cloud APIs corresponding to the online tools whose computing resources are greater than the fourth threshold in the first candidate tool set are eliminated to obtain a second candidate tool set, where the current online resources available for the cloud API include computing resources and bandwidth resources.

[0162] Optionally, the tools include local tools and online tools corresponding to the cloud application programming interface (API). The screening module 703 is specifically used to:

[0163] If there are local tools and online tools with the same functions in the first candidate tool set, and if the current locally available hardware resources meet the hardware resources required by the local tools, the cloud API corresponding to the online tool is removed from the first candidate tool set to obtain the second candidate tool set.

[0164] Optionally, the tool library stores tool information of the tool, including type, name, function, tool parameters, description information and requirements. The retrieval module 702 is specifically used to:

[0165] Get the text embedding vector of the question text;

[0166] Get the tool embedding vector of the tool description of each tool;

[0167] The similarity between the text embedding vector and the tool embedding vector of each tool is calculated by the retriever, and the K tools with the highest similarity are determined as the first candidate tool set.

[0168] Optionally, the tool library stores tool information of the tools, including type, name, function, tool parameters, description information and requirements. The language model module 704 is specifically used to:

[0169] Structuring the tool information of each tool to obtain the corresponding Json description information of each tool;

[0170] Add the Json description information of each tool in the second candidate tool set to the initial prompt word template to obtain an adjusted prompt word template, wherein the initial prompt word template includes the system prompt word and the task type template;

[0171] Based on the adjusted prompt word template, the information of each tool in the second candidate tool set, and the question text, the large language model determines the target tool that matches the task information;

[0172] A tool calling instruction is generated based on the tool information of the target tool, wherein the tool calling instruction includes a tool name and tool parameters.

[0173] Optionally, the tool calling module 705 is specifically configured to:

[0174] Generate structured function call instructions based on tool call instructions, and output the structured function call instructions to the FastAPI platform;

[0175] The FastAPI platform uses asynchronous monitoring to monitor structured function call instructions;

[0176] The FastAPI platform selects the target tool to call based on the structured function call instructions;

[0177] The FastAPI platform monitors the execution status of the target tool in real time to obtain the output results of the target tool.

[0178] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements any of the above-mentioned tool calling methods based on a large language model.

[0179] Figure 8 This is a hardware structure diagram of a computer device provided in an embodiment of the present application. Figure 8 The computer device shown includes: a processor 801, a communication interface 802, a memory 803 and a communication bus 804. The processor 801, the communication interface 802 and the memory 803 communicate with each other via the communication bus 804. Figure 8 The connection method between the processor 801, the communication interface 802, and the memory 803 shown is merely exemplary. During implementation, the processor 801, the communication interface 802, and the memory 803 may also be connected to each other in communication with each other using other connection methods besides the communication bus 804.

[0180] The memory 803 can be used to store computer programs, which may include instructions and data to implement the steps of any of the above-mentioned tool calling methods based on a large language model. In the embodiment of the present application, the memory 803 can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), flash memory, optical storage, and registers. The memory 803 may include a hard disk and / or memory.

[0181] The processor 801 may be a general-purpose processor. A general-purpose processor may be a processor that performs specific steps and / or operations by reading and executing a computer program (e.g., a computer program) stored in a memory (e.g., memory 803). The general-purpose processor may use data stored in the memory (e.g., memory 803) during the execution of the steps and / or operations. The general-purpose processor may be, for example, but not limited to, a central processing unit (CPU). In addition, the processor 801 may also be a special-purpose processor. A special-purpose processor may be a processor specially designed to perform specific steps and / or operations. A special-purpose processor may be, for example, but not limited to, an ASIC and an FPGA. In addition, the processor 801 may also be a combination of multiple processors, such as a multi-core processor.

[0182] Communication interface 802 may include input / output (I / O) interfaces, physical interfaces, and logical interfaces for interconnecting components within a network device, as well as interfaces for interconnecting the network device with other devices (e.g., network devices). The communication network may be Ethernet, a radio access network (RAN), a wireless local area network (WLAN), or the like. Communication interface 802 may be a module, circuit, transceiver, or any other device capable of communication.

[0183] During implementation, each step of the above method can be completed by an integrated logic circuit of the hardware in the processor 801 or by instructions in the form of software. The method disclosed in conjunction with the embodiments of the present application can be directly embodied as being executed by a hardware processor, or can be executed by a combination of hardware and software modules in the processor. The software module can be located in a storage medium mature in the art, such as a random access memory flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 803, and the processor 801 reads the information in the memory 803 and completes the steps of the above method in combination with its hardware. To avoid repetition, it will not be described in detail here.

[0184] Although the preferred embodiments of the present application have been disclosed for illustrative purposes, those skilled in the art will appreciate that various modifications, additions and substitutions are possible, without departing from the scope and spirit of the application as disclosed in the accompanying claims.

Claims

1. A tool calling method based on a large language model, characterized in that: The method comprises: Obtaining a question text, wherein the question text includes task information; Obtaining a first set of candidate tools that match the question text with a preset tool library, wherein the tool library includes a plurality of tools; Based on currently available local hardware resources and / or currently available online resources of a cloud application programming interface (API), the first candidate tool set is screened to obtain a second candidate tool set; determining a target tool matching the task information from the second candidate tool set based on the large language model, and generating a tool calling instruction based on the target tool; Selecting a target tool to be called based on the tool calling instruction, so that the target tool performs the task corresponding to the task information, and obtaining an output result of the target tool; Based on the question text and the output result, the large language model generates a response text corresponding to the question text; The method further includes a method for training the large language model, and the training method includes: Constructing a first problem sample, and determining a plurality of first tools that need to be called in the first problem sample; Split the first question sample according to graph logic and / or tree logic to obtain multiple sub-questions in the first question sample; logically associating the plurality of first tools according to the plurality of sub-problems to obtain a tool chain; Disassembling the tool chain in a defective manner to obtain a first tool sample set in multiple permutations and combinations; The large language model is trained using the first question sample and a plurality of first tool sample sets.

2. The tool calling method based on a large language model according to claim 1, characterized in that: The method further includes a training method for the large language model, wherein the training method includes: Constructing a second problem sample, and determining a plurality of second tools that need to be called in the second problem sample; Performing a plurality of permutations and combinations of the plurality of second tools in a defective manner, and / or adding a plurality of tools unrelated to the plurality of second tools, to obtain a plurality of second tool sample sets; The large language model is trained using the second question sample and multiple second tool sample sets.

3. The tool calling method based on a large language model according to claim 1, characterized in that: The tools in the tool library include local tools and online tools corresponding to the cloud application programming interface (API). Based on the current available local hardware resources and / or the current available online resources of the cloud API, the first candidate tool set is screened to obtain a second candidate tool set, including: If the currently available local hardware resources are less than the first threshold, remove the local tools whose hardware resources are greater than the second threshold from the first candidate tool set to obtain a second candidate tool set; and / or, If the current online resources available for the cloud API are less than the third threshold, the cloud APIs corresponding to the online tools whose computing resources are greater than the fourth threshold in the first candidate tool set are eliminated to obtain a second candidate tool set, where the current online resources available for the cloud API include computing resources and bandwidth resources.

4. The tool calling method based on a large language model according to claim 1, characterized in that: The tools in the tool library include local tools and online tools corresponding to the cloud application programming interface (API). Based on the current available local hardware resources and / or the current available online resources of the cloud API, the first candidate tool set is screened to obtain a second candidate tool set, including: If there are local tools and online tools with the same functions in the first candidate tool set, and if the current locally available hardware resources meet the hardware resources required by the local tools, the cloud API corresponding to the online tool is removed from the first candidate tool set to obtain the second candidate tool set.

5. The tool calling method based on a large language model according to claim 1, characterized in that: The tool library stores tool information of the tools, including type, name, function, tool parameters, description information, and requirements. Acquiring a first set of candidate tools that matches the question text with a preset tool library includes: Obtaining a text embedding vector for the question text; Get the tool embedding vector of the tool description of each tool; The similarity between the text embedding vector and the tool embedding vector of each tool is calculated by a retriever, and K tools with the highest similarity are determined as a first candidate tool set.

6. The tool calling method based on a large language model according to claim 1, characterized in that: The tool library stores tool information of the tools, the tool information including type, name, function, tool parameters, description information, and requirements. Determining a target tool associated with the task information from the second candidate tool set based on the large language model, and generating a tool call instruction based on the target tool include: Structuring the tool information of each tool to obtain the Json description information corresponding to each tool; Adding the Json description information of each tool in the second candidate tool set to the initial prompt word template to obtain an adjusted prompt word template, wherein the initial prompt word template includes a system prompt word and a task type template; Based on the adjusted prompt word template, information about each tool in the second candidate tool set, and the question text, the large language model determines a target tool that matches the task information; A tool calling instruction is generated based on the tool information of the target tool, wherein the tool calling instruction includes a tool name and tool parameters.

7. The tool calling method based on a large language model according to claim 6, characterized in that: Selecting a target tool to be called based on the tool calling instruction so that the target tool performs the task corresponding to the task information and obtaining an output result of the target tool includes: Generating a structured function call instruction based on the tool call instruction, and outputting the structured function call instruction to the FastAPI platform; The FastAPI platform monitors the structured function call instruction in an asynchronous monitoring manner; The FastAPI platform selects a target tool to call according to the structured function call instruction; The FastAPI platform monitors the execution status of the target tool in real time to obtain the output results of the target tool.

8. A tool calling device based on a large language model, characterized in that: The device comprises: An input module, configured to obtain a question text, wherein the question text includes task information; A retrieval module is used to obtain a first set of candidate tools that match the question text with a preset tool library, wherein the tool library includes a plurality of tools; a screening module, configured to screen the first candidate tool set based on currently available local hardware resources and / or currently available online resources of a cloud application programming interface (API) to obtain a second candidate tool set; a language model module, configured to determine a target tool matching the task information from the second candidate tool set based on the large language model, and generate a tool calling instruction based on the target tool; A tool calling module is used to select a target tool to be called based on the tool calling instruction, so as to make the target tool perform the task corresponding to the task information and obtain an output result of the target tool; The language model module is further configured to generate a response text corresponding to the question text based on the question text and the output result; The language model module specifically includes a training method for the large language model, and the training method includes: Constructing a first problem sample, and determining a plurality of first tools that need to be called in the first problem sample; Split the first question sample according to graph logic and / or tree logic to obtain multiple sub-questions in the first question sample; logically associating the plurality of first tools according to the plurality of sub-problems to obtain a tool chain; Disassembling the tool chain in a defective manner to obtain a first tool sample set in multiple permutations and combinations; The large language model is trained using the first question sample and a plurality of first tool sample sets.

9. A computer device comprising a memory and a processor, characterized in that: The memory stores a computer program, and is characterized in that when the processor executes the computer program, the steps of the tool calling method based on a large language model according to any one of claims 1 to 7 are implemented.

Images