Large model tool calling method, model training data processing method and intelligent vehicle

Abstract

The application relates to a large model tool calling method, a model training data processing method and an intelligent vehicle. The method comprises the following steps: acquiring a model task instruction as model input information for this round of thinking, inputting the model input information into a tool exploration unit of a large model multiple times, outputting candidate calling tools and tool calling suggestions, recalling multiple recalled calling tools from a preset calling tool set, inputting the tool calling suggestions, the multiple recalled calling tools and the model input information into a tool decision unit of the large model, outputting a target calling tool, acquiring calling feedback data of the target calling tool, splicing the calling feedback data into model input information for the next round of thinking, returning to the step of inputting the model input information into the tool exploration unit of the large model multiple times, and continuing until the target calling tool output by the tool decision unit can complete a target task. The method can improve the tool calling accuracy of a large language model.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method for calling large model tools, a method for processing model training data, intelligent vehicles, readable storage media, and computer program products. Background Technology

[0002] When dealing with inference tasks, Large Language Models (LLMs) need to select usable tools from a large library of tools due to the complexity and diversity of user queries.

[0003] In traditional technologies, large language models often rely on their fundamental model capabilities to invoke tools, directly incorporating all massive amounts of tool information into the prompts for user questions. This method of tool invocation exceeds the effective processing capacity of large language models, causing them to spend too much time considering user questions and reducing their efficiency in selecting tools. Consequently, large language models cannot accurately invoke relevant tools, affecting their ability to accurately solve user problems.

[0004] Therefore, traditional techniques suffer from low accuracy in calling tools from large language models. Summary of the Invention

[0005] Based on this, it is necessary to provide a method for calling large language model tools, a method for processing model training data, intelligent vehicles, readable storage media, and computer program products that can improve the accuracy of tool calls in large language models, in response to the above-mentioned technical problems.

[0006] Firstly, this application provides a method for invoking large model tools, the method comprising the following steps:

[0007] Obtain model task instructions used to instruct the large model to perform corresponding target tasks, and use them as model input information for this round of consideration. Input the model input information multiple times into the tool exploration unit of the large model, output at least one candidate tool, and obtain tool invocation suggestions from the tool exploration unit.

[0008] From a preset set of calling tools, recall multiple recalled calling tools that match each of the candidate calling tools; input the tool calling suggestions, the multiple recalled calling tools, and the model input information into the tool decision unit of the large model, output at least one target calling tool, and obtain the tool calling suggestions from the tool decision unit; the target calling tool is the calling tool among the multiple recalled calling tools;

[0009] If the target calling tool cannot complete the target task, the calling feedback data of the target calling tool is obtained, and the model thinking process data of the tool decision unit, the calling feedback data, and the model task instructions are concatenated into the model input information for the next round of thinking. Then, the step of inputting the model input information into the tool exploration unit of the large model multiple times is returned to be executed until the target calling tool output by the tool decision unit can complete the target task.

[0010] In one embodiment, the step of repeatedly inputting the model input information into the tool exploration unit of the large model and outputting at least one candidate invocation tool includes:

[0011] The model input information is input into the tool exploration unit of the large model;

[0012] The tool exploration unit parses the model task instructions in the model input information to generate model parsing results; the model parsing results include at least one initial invocation tool.

[0013] Return to the tool exploration unit that inputs the model input information into the large model until the number of model parsing results meets a preset threshold, and use each of the initial calling tools in the multiple model parsing results as the at least one candidate calling tool.

[0014] In one embodiment, each model parsing result further includes model thinking process data; the process of obtaining tool invocation suggestions from the tool exploration unit includes:

[0015] Obtain the model thinking process data from the analysis results of each model;

[0016] Input the thinking process data of each model into the large model;

[0017] The summary function of the large model is invoked to summarize the thinking process data of each model and obtain the tool invocation suggestions.

[0018] In one embodiment, the step of recalling multiple recalled invocation tools that match each of the candidate invocation tools from a preset invocation tool set includes:

[0019] Obtain tool description information for each of the at least two supported calling tools in the calling toolset;

[0020] Based on the tool description information of each of the supported calling tools, obtain the semantic similarity between each of the supported calling tools and each of the candidate calling tools;

[0021] Based on the semantic similarity, multiple recalled calling tools are recalled from each of the supported calling tools.

[0022] In one embodiment, obtaining the invocation feedback data of the target invocation tool includes:

[0023] According to the target task, the target invocation tool is invoked through the large model to obtain real invocation feedback data, and the real invocation feedback data is used as the invocation feedback data;

[0024] And / or,

[0025] According to the target task, the target calling tool is simulated through the large model to obtain simulated calling feedback data, and the simulated calling feedback data is used as the calling feedback data.

[0026] In one embodiment, the method further includes:

[0027] Perform the invocation operation on each of the target invocation tools to obtain the data resources returned by each of the target invocation tools;

[0028] Based on whether the data resources returned by each of the target calling tools meet the data resource acquisition requirements of the target task, it is determined whether the target calling tool can complete the target task.

[0029] Secondly, this application provides a method for processing model training data, the method comprising:

[0030] Based on the model task instructions and the corresponding recalled calling tools, a tool is constructed to retrieve model training data; the recalled calling tools are multiple calling tools that match each candidate calling tool from a preset calling tool set; the candidate calling tools are the calling tools output by the tool exploration unit of the target large model when processing the model task instructions; the target large model is the large model in any of the above-described large model tool calling methods;

[0031] Based on the model thinking process data, target tool invocation, and corresponding tool invocation suggestions output by the tool decision unit of the target large model, construct tool agent training data;

[0032] The output tool calls up training data; the tool calling up training data includes the model training data retrieved by the tool and the tool agent training data.

[0033] Thirdly, this application also provides an intelligent vehicle, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described large model tool invocation method.

[0034] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0035] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method.

[0036] The aforementioned large-scale model tool invocation method, model training data processing method, intelligent vehicle, readable storage medium, and computer program product acquire model task instructions used to instruct the large model to perform corresponding target tasks as model input information for the current round of thinking. This model input information is then repeatedly input into the tool exploration unit of the large model, outputting at least one candidate invocation tool and obtaining tool invocation suggestions from the tool exploration unit. From a preset invocation tool set, multiple recalled invocation tools matching each candidate invocation tool are recalled. The tool invocation suggestions, multiple recalled invocation tools, and model input information are input into the tool decision unit of the large model, outputting at least one target invocation tool and obtaining tool invocation suggestions from the tool decision unit. The target invocation tool is one of the multiple recalled invocation tools. If the target invocation tool cannot complete the target task, the invocation feedback data of the target invocation tool is acquired. The model thinking process data, invocation feedback data, and model task instructions from the tool decision unit are concatenated into the model input information for the next round of thinking. The process then returns to the step of repeatedly inputting the model input information into the tool exploration unit of the large model until the target invocation tool output by the tool decision unit can complete the target task. This scheme, based on the initial screening of the large model tool exploration unit and the formal selection of the tool decision unit, realizes a two-stage, multi-round iterative feedback of the large model calling tools for the retrieval target. Furthermore, it incorporates the thinking chain data of the model's thinking process into the retrieval process of the target calling tool, ensuring a high degree of similarity and matching between the target calling tool and the target task corresponding to the model's task instructions. This achieves dynamic optimization and precise screening of the large model's calling tools, effectively improving the accuracy of the large language model's tool calling. It solves the problem of low accuracy in tool calling by traditional techniques for large language models, achieving the beneficial effect of improving the accuracy of tool calling by large language models. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is an application environment diagram of a large model tool invocation method in one embodiment;

[0039] Figure 2 This is a flowchart illustrating a method for invoking a large model tool in one embodiment;

[0040] Figure 3 This is a flowchart illustrating another method for invoking a large model tool in one embodiment;

[0041] Figure 4 This is a flowchart illustrating a model training data processing method in one embodiment;

[0042] Figure 5 This is a structural block diagram of a large model tool invocation device in one embodiment;

[0043] Figure 6 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0045] The large model tool invocation method provided in this application embodiment can be applied to, for example... Figure 1In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104, or it can be located in the cloud or on another network server. Server 104 can acquire model task instructions used to instruct the large model to perform corresponding target tasks, which serve as model input information for this round of thinking. It inputs the model input information multiple times into the tool exploration unit of the large model, outputting at least one candidate calling tool and obtaining tool calling suggestions from the tool exploration unit. Server 104 can recall multiple recalled calling tools matching each candidate calling tool from a preset calling tool set. It inputs the tool calling suggestions, multiple recalled calling tools, and model input information into the tool decision unit of the large model, outputting at least one target calling tool and obtaining tool calling suggestions from the tool decision unit. The target calling tool is the calling tool among the multiple recalled calling tools. If the target calling tool cannot complete the target task, server 104 can acquire the calling feedback data of the target calling tool, concatenate the model thinking process data, calling feedback data, and model task instructions from the tool decision unit into model input information for the next round of thinking, and return to execute the step of inputting the model input information multiple times into the tool exploration unit of the large model until the target calling tool output by the tool decision unit can complete the target task. Among them, terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, drones, low-altitude aircraft, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, projection devices, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc. Server 104 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0046] In one exemplary embodiment, such as Figure 2 As shown, a method for calling large model tools is provided. This embodiment applies this method to... Figure 1 The method is illustrated using server 104 as an example. In this embodiment, the method includes the following steps S202 to S206. Wherein:

[0047] Step S202: Obtain the model task instruction used to instruct the large model to perform the corresponding target task, as the model input information for this round of consideration, input the model input information to the tool exploration unit of the large model multiple times, output at least one candidate calling tool, and obtain the tool calling suggestion from the tool exploration unit.

[0048] The tool exploration unit can be a guessing exploration module of the large model for calling tools, that is, the exploration area. It can be used to quickly retrieve multiple candidate calling tools that may be applicable to the task from a large number of tools according to the model task instructions. Combining the task requirements of the model task instructions and the functional characteristics of the candidate calling tools, it summarizes and generates tool calling suggestions including data from the model thinking process of the tool exploration unit, such as the tool type and applicable scenarios of the candidate calling tools, for use by the tool decision unit.

[0049] The model task instructions can be instructions obtained from parsing user queries or system descriptions, which can be provided to the large model to perform the corresponding target task, such as "query the weather in city A today". Since user queries are often complex and diverse, such as "who is the director of this year's highest-grossing film, please play this film", it is necessary to specify the highest-grossing film this year, the director of the film, and the task requirement of playing this film. Therefore, for such complex problems, it is often necessary to use the large model to parse and consider the model task instructions in multiple rounds.

[0050] In practical applications, server 104 can parse the original question (query) directly input by the user through voice or text to obtain the target task with clear requirements for instructing the large model to perform. It can then obtain the model task instruction for the corresponding target task and use the model task instruction as the model input information for the large model's current round of thinking. The model input information is input to the tool exploration unit of the large model multiple times. Through the tool exploration unit, the model task instruction in the model input information is parsed multiple times, and tool calling suggestions and at least one candidate calling tool can be output.

[0051] For example, for the model task instruction "Query the weather in city A today", its instruction keywords can be clearly identified as "query", "city A", "today", and "weather". This model task instruction is used as the model input information for this round of thinking of the large model and input into the tool exploration unit of the large model. The keyword "weather" can be used to quickly identify "meteorological tools", thereby outputting candidate tools such as "weather API (Application Programming Interface)", "meteorological data service API", "map data API", and "weather station website". Combining the task requirements of the model task instruction (such as prioritizing APIs for high real-time requirements) and the functional characteristics of the candidate tools (such as response speed, data accuracy, query relevance, etc.), tool calling suggestions including the thinking and analysis process of the large model are generated, such as prioritizing the use of the "weather API" of third-party platforms.

[0052] Step S204: Recall multiple recalled calling tools that match each candidate calling tool from a preset calling tool set; input the tool calling suggestions, multiple recalled calling tools and model input information into the tool decision unit of the large model, output at least one target calling tool, and obtain the tool calling suggestions from the tool decision unit.

[0053] Among them, the recalled calling tools can be calling tools that are highly related to the task requirements of the target task corresponding to the model input information and match the functions of each candidate calling tool; the target calling tool can be the calling tool among multiple recalled calling tools.

[0054] The tool decision unit can be the recall and invocation module of the large model for calling tools, that is, the formal area. It can be used to accurately select the target calling tool that can perform the target task from a large number of tools based on the tool invocation suggestions of the tool exploration unit, the recalled calling tools, and the model input information. It can also generate tool invocation suggestions that include the model thinking process data of the tool decision unit by combining the task requirements of the model task instructions and the functional characteristics of the target calling tool.

[0055] For example, the preset tool set can be a pre-built tool knowledge base, which contains metadata information of the calling tools, such as tool name, tool type (e.g., API, database, script, etc.), function description, output format, performance indicators (e.g., data timeliness, response time), tool constraints (e.g., "requires application for API Key (Application Programming Interface Key)"), etc.

[0056] In a specific implementation, server 104 can retrieve multiple recalled calling tools that match each candidate calling tool from a preset calling tool set based on semantic similarity retrieval from each candidate calling tool based on the candidate calling tools in the tool exploration unit. Then, the tool calling suggestions, multiple recalled calling tools and model input information are input to the tool decision unit of the large model. The tool decision unit can output at least one target calling tool that can execute the target task corresponding to the model task instruction, and obtain the tool calling suggestions from the tool decision unit.

[0057] Continuing with the model task instruction "Query the weather conditions of city A today" as an example, based on the candidate calling tools initially retrieved by the tool exploration unit of the large model, such as "weather API", "meteorological data service API", "map data API", and "weather station official website", multiple semantically related or functionally similar calling tools (such as the "weather API" of various third-party platforms) are recalled from the preset calling tool set. Then, the tool calling suggestions (such as giving priority to the use of the "weather API" of third-party platforms), multiple recalled calling tools and model input information are input into the tool decision unit of the large model, which can output target calling tools such as "Weather API of platform A", "Weather API of platform B", and "Weather API of platform C".

[0058] Step S206: If the target calling tool cannot complete the target task, obtain the calling feedback data of the target calling tool, and combine the model thinking process data, calling feedback data and model task instructions of the tool decision unit into the model input information for the next round of thinking. Then, return to execute the step of inputting the model input information into the tool exploration unit of the large model multiple times until the target calling tool output by the tool decision unit can complete the target task.

[0059] Optionally, the situation where the target calling tool cannot complete the target task may refer to the situation where the calling result returned by the target calling tool cannot cover the task requirements of the target task.

[0060] In the specific implementation, when the target calling tool cannot complete the task requirements of the target task, the server 104 can obtain the calling feedback data of the target calling tool, and concatenate the model thinking process data of the tool decision unit, the calling feedback data, and the model task instructions into the model input information for the next round of thinking. Then, it returns to execute the step of inputting the model input information into the tool exploration unit of the large model multiple times until the target calling tool output by the tool decision unit can complete the target task.

[0061] For example, for the model task instruction "Query the weather conditions of city A today", if the user also needs to query the typhoon situation, but the target calling tool does not provide the typhoon forecast for city A today, that is, the target calling tool cannot fulfill the task requirements of the target task corresponding to the model task instruction, then it is necessary to conduct the next round of thinking and exploration through the feedback of calling the target calling tool, until the tool decision unit outputs a target calling tool that can complete the typhoon forecast.

[0062] For example, for the model task instruction "Who is the director of this year's highest-grossing film? Please play this film", if the target calling tool only has a search tool, it can only search for this year's highest-grossing film and its director, but cannot fulfill the task requirement of playing the film. Therefore, it is necessary to carry out the next round of thinking and exploration until the target calling tool that can play the film is output.

[0063] The aforementioned large-scale model tool invocation method obtains model task instructions used to instruct the large-scale model to perform corresponding target tasks as model input information for the current round of consideration. This model input information is then repeatedly input into the tool exploration unit of the large-scale model, outputting at least one candidate invocation tool and obtaining tool invocation suggestions from the tool exploration unit. From a pre-defined set of invocation tools, multiple recalled invocation tools matching each candidate tool are retrieved. The tool invocation suggestions, multiple recalled invocation tools, and model input information are input into the tool decision unit of the large-scale model, outputting at least one target invocation tool and obtaining tool invocation suggestions from the tool decision unit. The target invocation tool is one of the multiple recalled invocation tools. If the target invocation tool cannot complete the target task, invocation feedback data from the target invocation tool is obtained. The model thinking process data, invocation feedback data, and model task instructions from the tool decision unit are then concatenated to form the next round. The model input information is processed and returned to the tool exploration unit of the large model multiple times until the target calling tool output by the tool decision unit can complete the target task. Based on the initial screening by the tool exploration unit and the formal selection by the tool decision unit, the large model realizes a two-stage, multi-round iterative feedback of the target calling tool retrieval. Furthermore, the thinking chain data of the model thinking process is incorporated into the target calling tool retrieval process, which also ensures a high degree of similarity and matching between the target calling tool and the target task corresponding to the model task instruction. This enables dynamic optimization and precise screening of the large model calling tools, effectively improving the accuracy of the large language model's tool calling. It solves the problem of low accuracy of large language model tool calling in traditional technologies, and achieves the beneficial effect of improving the accuracy of tool calling in large language models.

[0064] In an exemplary embodiment, inputting model input information multiple times into the tool exploration unit of a large model and outputting at least one candidate calling tool includes: inputting model input information into the tool exploration unit of a large model; parsing the model task instructions in the model input information through the tool exploration unit to generate model parsing results; returning to the tool exploration unit that inputs model input information into the large model until the number of model parsing results meets a preset number threshold, and using each initial calling tool in the multiple model parsing results as at least one candidate calling tool.

[0065] The model parsing results may include at least one initial invocation tool.

[0066] In a specific implementation, server 104 can input model input information into the tool exploration unit of the large model. The tool exploration unit parses the model task instructions in the model input information and generates model parsing results, including initial calling tools and model thinking process data with reasoning records, through semantic parsing, text recognition, structured processing, etc. The server returns the input information to the tool exploration unit of the large model until the number of model parsing results meets a preset threshold, which may include the minimum number of candidate calling tools, the maximum number of iterations of cyclic parsing, etc. Each initial calling tool in multiple model parsing results is used as at least one candidate calling tool.

[0067] For example, the tool exploration unit (exploration area) can use N iterations of the Large Language Model (LLM) to parse the task requirements of the model task instruction corresponding to the user's query. Each parsing outputs a model parsing result including at least one initial tool invocation and a thought analysis process. After N iterations of the Large Language Model, M candidate tools can be output (M>N). In other words, the tool exploration unit (exploration area) is a "preliminary filter" of the large model, capable of quickly identifying potentially applicable candidate tools and providing reasonable tool invocation suggestions by parsing the semantics and intent of the task instruction.

[0068] The technical solution of this embodiment uses a tool exploration unit of a large model to parse the model task instructions in the model input information, which can obtain model parsing results that match the target task. Furthermore, by using a preset data threshold for the model parsing results, the tool exploration unit can control the process of selecting candidate calling tools, thereby providing a diverse and high-quality decision basis for the tool decision unit (formal area) to select the optimal calling tool for the target task.

[0069] In an exemplary embodiment, each model parsing result also includes model thinking process data; obtaining tool call suggestions from the tool exploration unit includes: obtaining model thinking process data from each model parsing result; inputting each model thinking process data into the large model; calling the summary function of the large model to summarize the thinking process data of each model and obtain tool call suggestions.

[0070] Among them, model thinking process data can be the thinking and analysis process of logical reasoning of large models, that is, thought chain (COT) data.

[0071] Among them, the summary function of the large model refers to the ability of the large language model (LLM) to condense long texts or complex information into concise and accurate summaries, retaining the core content and key information. It can be used to summarize and refine the thinking and analysis process of the large model (i.e., the thought chain (COT) data), as well as to summarize and refine various contents such as model task instructions, model input information, tool call suggestions, tool calls, text and image information, and audio and video information.

[0072] In practical implementation, since the thinking and analysis process of multiple large language models is often lengthy, it is necessary to summarize these lengthy thinking and analysis processes to extract concise tool invocation suggestions. Therefore, for the model parsing results output during multiple parsing sessions, server 104 can obtain model thinking process data from each model parsing result by performing structured field parsing or natural language processing on the reasoning information of the model parsing results. It then integrates the thinking process data of each model and inputs the integrated model thinking process data into the large model. By calling the summary function of the large model, it can summarize the thinking process data of each model by extracting common reasoning logic, evaluating conflict points in tool invocation, and selecting the optimal tool, thereby obtaining tool invocation suggestions.

[0073] Similarly, the thinking and analysis process of the tool decision-making unit can also be summarized by calling the summary function of the large model, which summarizes the model thinking process data in the thinking and analysis process of the tool decision-making unit.

[0074] The technical solution of this embodiment, by summarizing and integrating the model thinking process data in the model analysis results, can obtain more accurate and concise tool calling suggestions, thereby refining the lengthy and scattered model inference data into core suggestions that focus on guiding tool calling, realizing the synthesis of the large model's thinking chain, and improving the accuracy of the large model in selecting the target calling tool.

[0075] In an exemplary embodiment, recalling multiple recalled calling tools that match each candidate calling tool from a preset calling tool set includes: obtaining tool description information of each supportable calling tool from at least two supportable calling tools in the calling tool set; obtaining the semantic similarity between each supportable calling tool and each candidate calling tool based on the tool description information of each supportable calling tool; and recalling multiple recalled calling tools from each supportable calling tool based on the semantic similarity.

[0076] Among them, the supported calling tools can be calling tools that meet basic filtering conditions (such as functional domain relevance and no requirement constraint conflicts) from a preset set of calling tools.

[0077] The tool description information may include vectorized metadata information for each calling tool, such as tool name, tool type, function description, output format, performance indicators, and tool constraints.

[0078] In a specific implementation, server 104 can obtain the tool description information of each supportable tool from the metadata information such as the tool name and function description of each supportable tool in the tool set; based on the tool description information of each supportable tool, cosine similarity can be used for vectorized calculation to obtain the semantic similarity between each supportable tool and each candidate tool; based on the semantic similarity, multiple recalled tools with the highest functional domain similarity to each candidate tool are recalled from among the supportable tools.

[0079] For example, the tool exploration unit (exploration area) provides M tools that may not be supported in the current domain or agent. Therefore, it is necessary to pre-build a tool knowledge base based on the currently supported tools. From the tool knowledge base, the tool description information of each supported calling tool can be obtained. Based on the semantic similarity between the tool names of each supported calling tool and each candidate calling tool, S similar calling tools with similar functions can be recalled from the tool knowledge base for each of the M tools. The number S is unlimited and can be a fixed number or an indefinite number of recalled calling tools obtained based on the semantic similarity score threshold between the two. Finally, M*S tools can be recalled, which generates data for training the tool_retrieval model (tool retrieval model). There are a total of M*S data pairs, and the model input is the same query.

[0080] Furthermore, the tool decision unit (formal area) inputs the tool invocation suggestions from the exploration area, the final recalled invocation tools, and the model input information (user query) into an LLM, and outputs multiple target invocation tools and the final tool invocation suggestions, thus generating the data for the Tools Agent's current round of thinking and analysis.

[0081] The technical solution of this embodiment obtains tool description information of each of the at least two supported calling tools from the calling tool set, and obtains the semantic similarity between each supported calling tool and each candidate calling tool based on the tool description information. This effectively utilizes the tool description information of each supported calling tool to obtain the semantic similarity of each supported calling tool, achieving accurate semantic understanding of the recalled tools. Then, based on the semantic similarity, multiple recalled calling tools are recalled from each supported calling tool. This realizes the use of the semantic similarity of each supported calling tool to recall calling tools with a higher degree of matching with the task requirements of the target task, effectively improving the recall accuracy of calling tools.

[0082] In an exemplary embodiment, obtaining the call feedback data of the target calling tool includes: calling the target calling tool through a large model according to the target task, obtaining real call feedback data, and using the real call feedback data as call feedback data; and / or, simulating the call to the target calling tool through a large model according to the target task, obtaining simulated call feedback data, and using the simulated call feedback data as call feedback data.

[0083] Among them, the call feedback data can be an objective record of whether the call result returned by the target calling tool covers the task requirements of the target task, including information such as the data resources required by the target task, the performance status of the call process, and the matching degree between the data resources and the target task.

[0084] Among them, the actual call feedback data can be obtained by actually triggering the execution of the target calling tool through a large model, and obtaining the feedback data of the target calling tool in the actual task requirement environment of the target task.

[0085] Simulated feedback data can be generated by using historical data to generate simulated feedback data, which provides the data resources needed to simulate the task requirements of a target task through virtual simulation tools with large models.

[0086] In a specific implementation, server 104 can invoke a target invocation tool through a large model based on the target task. The target invocation tool executes the target task, and the server can obtain the actual invocation feedback data of the target invocation tool in accordance with the actual task requirements of the target task, and use the actual invocation feedback data as the invocation feedback data; and / or, based on the target task, the server can simulate the invocation of the target invocation tool through a virtual simulation tool of the large model, and can use historical data to obtain the simulated invocation feedback data corresponding to the task requirements of the target task, and use the simulated invocation feedback data as the invocation feedback data.

[0087] For example, continuing with the model task instruction "Query the weather conditions of city A today", a real call requires requesting a specific weather API and obtaining the weather data returned by the weather API as the real call feedback data; a simulated call requires obtaining a virtual weather forecast based on historical data through simulation calculations as the simulated call feedback data.

[0088] The technical solution of this embodiment obtains the call feedback data of the target calling tool by both real and simulated calls, which balances realism and flexibility. The obtained call feedback data can be used as input for the next round of thinking of the large model, driving the tool decision unit to further adjust the target calling tool selection parameters, and providing high-quality feedback data for the large model to accurately call tools that are highly relevant to the target task.

[0089] In an exemplary embodiment, the method further includes: performing invocation operations on each target invocation tool to obtain data resources returned by each target invocation tool; and determining whether the target invocation tool can complete the target task based on whether the data resources returned by each target invocation tool meet the data resource acquisition requirements of the target task.

[0090] In practical implementation, if the call results returned by the target calling tools selected by the large model retrieval meet the task requirements corresponding to the target task, then the tool calling task of the large model can be terminated. Therefore, server 104 can execute the call operation on each target calling tool to obtain the data resources returned by each target calling tool; based on whether the data resources returned by each target calling tool meet the data resource acquisition requirements of the target task, it can be determined whether the target calling tool can complete the target task.

[0091] For example, continuing with the model task instruction "Query today's weather in city A," if the user also needs information about typhoons, the results returned by the large model's calling tool should meet the task requirement of including typhoon forecasts. Suppose the target calling tool retrieved by the large model only returns data resources such as today's temperature, precipitation, humidity, and UV index for city A, but not data resources related to typhoon forecasts. This indicates that the target calling tool has failed to complete the target task, and it is necessary to return to the steps for when the target calling tool cannot complete the target task.

[0092] The technical solution of this embodiment determines whether the target calling tool can complete the target task by executing the call to obtain the data resources returned by the target calling tool. This allows the completion of the target task to have a clear end mark, ensuring the reliability of the target calling tool and the accuracy of the target task completion, thereby realizing a closed loop of the entire process from tool call to task completion verification.

[0093] For example, in specific applications, with Figure 3 For example, this application provides a method for calling large-scale model tools. The in-vehicle large-scale language model can preset a list of functions, such as obtaining the in-vehicle temperature, obtaining the air conditioning setting temperature, and setting the window position, clearly defining the corresponding preset tool set. The large-scale language model can identify the corresponding function based on the user's vague intent. For example, when a passenger in the car expresses vague intents such as "Is it a bit stuffy inside the car?", "Will the children in the back seat be hot?", or "I'm so hot, extremely hot," the in-vehicle large-scale language model can identify the function that requires obtaining the in-vehicle temperature. Therefore, the in-vehicle large-scale language model, based on the large-scale model tool calling method provided in this application, can accurately retrieve the calling tool. This includes:

[0094] When responding to user intent, such as Figure 3In the example of "I'm so hot, extremely hot," the exploration zone of the in-vehicle large language model, also known as the tool exploration unit, initiates multiple rounds of thinking cycles in response to this model task instruction. The exploration zone attempts to obtain environmental information such as the vehicle's interior temperature, air conditioning temperature, and ultraviolet radiation intensity through "Exploration Path 1," using this information as the model task instruction invoked by the tool, and then using this model task instruction as the model input information for the in-vehicle large language model's current round of thinking.

[0095] The in-vehicle large language model, based on these model task instructions, inputs model input information multiple times into the tool exploration unit, generating different execution exploration paths. Each path corresponds to different tool combination logic, thus obtaining multiple candidate tools and tool invocation suggestions. The exploration area of ​​the in-vehicle large model combines multiple COT function lists to perform thought chain deduction on the candidate tools and tool invocation suggestions of multiple exploration paths. It finds semantically similar function tools from the knowledge base (i.e., the pre-set tool invocation set). Through the combination of "thought chain (COT) + tool (Tool)," it filters duplicate function lists, realizing the screening, deduplication, and rationality verification of tool invocation to avoid calling invalid / duplicate tools. At the same time, the in-vehicle large model also needs to summarize the LLM thinking analysis process based on the thought chain data (COT) of the exploration area to obtain core and refined tool invocation suggestions.

[0096] Thus, through N rounds of thinking and summarizing on tool calls, the formal area of ​​the vehicle-mounted large model LLM, namely the tool decision unit, obtains the thought chain (COT) for optimizing tool call suggestions, as well as multiple target call tools (Tool) that are matched with each candidate call tool and recalled from the preset call tool set.

[0097] When the target calling tool can meet the task requirements of the target task corresponding to the model task instruction, it indicates that the task can be started (task completed), the vehicle-mounted large model ends its thinking, and the tool calling process terminates. When the target calling tool does not meet the task requirements of the target task (i.e., the "No" branch), the calling feedback data of the target calling tool is obtained through actual and simulated calls. The obtained calling feedback data is used as the input for the next round of thinking of the large model and sent back to the "exploration area" to start the next round of N+1 thinking, re-optimize the tool calling path, and obtain a more suitable target calling tool.

[0098] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0099] Based on the same inventive concept, this application also provides a model training data processing method. The solution provided by this model training data processing method is similar to the solution described above. Therefore, the specific limitations of one or more model training data processing method embodiments provided below can be found in the limitations of the large model tool calling method described above, and will not be repeated here.

[0100] In one exemplary embodiment, such as Figure 4 As shown, a method for processing model training data is also provided, including:

[0101] Step S402: Based on the model task instructions and the corresponding recalled tools, construct the tool to retrieve model training data.

[0102] Among them, the recalled calling tools are multiple calling tools that match each candidate calling tool from a preset calling tool set; the candidate calling tools are the calling tools output by the tool exploration unit of the target large model when processing the model task instructions; the target large model is the large model in any of the above-mentioned large model tool calling methods.

[0103] Step S404: Based on the model thinking process data output by the tool decision unit of the target big model, the target calling tool, and the corresponding tool calling suggestions, construct the tool agent training data;

[0104] Step S406: Output the tool to call the training data; the tool calling the training data includes the tool retrieves the model training data and the tool agent training data.

[0105] In its implementation, server 104 can generate training data for a tool retrieval model (tool_retrieval model) by obtaining model task instructions and corresponding recalled tools from the tool exploration unit in the target large model; and generate training data for a tool agent (Tools Agent) by obtaining model thinking process data (COT, thought chain), target calling tools, and corresponding tool calling suggestions from the tool decision unit in the target large model; then, the tool retrieval model training data and the tool agent training data are combined into tool calling training data, which can then be input into the target large model. The large model using the above-mentioned combined tool calling training data can filter out potentially usable tools from a massive number of tools and provide accurate calling tools through a rigorous thought chain, effectively improving the accuracy of tool calling training data in the large language model.

[0106] Based on the same inventive concept, this application also provides a large model tool invocation apparatus for implementing the large model tool invocation method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more embodiments of the large model tool invocation apparatus provided below can be found in the limitations of the large model tool invocation method described above, and will not be repeated here.

[0107] In one exemplary embodiment, such as Figure 5 As shown, a large model tool invocation device is provided, including:

[0108] The exploration module 510 is used to obtain model task instructions for instructing the large model to perform corresponding target tasks, which serve as model input information for this round of thinking. The model input information is input to the tool exploration unit of the large model multiple times, outputting at least one candidate tool for invocation, and obtaining tool invocation suggestions from the tool exploration unit.

[0109] The decision module 520 is used to recall multiple recalled calling tools that match each candidate calling tool from a preset set of calling tools; input the tool calling suggestions, multiple recalled calling tools and model input information into the tool decision unit of the large model, output at least one target calling tool, and obtain the tool calling suggestions from the tool decision unit; the target calling tool is the calling tool among the multiple recalled calling tools;

[0110] Feedback module 530 is used to obtain the call feedback data of the target calling tool when the target calling tool cannot complete the target task. It combines the model thinking process data of the tool decision unit, the call feedback data and the model task instructions into the model input information for the next round of thinking, and returns to execute the step of inputting the model input information into the tool exploration unit of the large model multiple times until the target calling tool output by the tool decision unit can complete the target task.

[0111] In one embodiment, the exploration module 510 is further configured to input model input information to the tool exploration unit of the large model; parse the model task instructions in the model input information through the tool exploration unit to generate model parsing results; the model parsing results include at least one initial calling tool; return to the tool exploration unit that inputs model input information to the large model until the number of model parsing results meets a preset number threshold, and use each initial calling tool in the multiple model parsing results as at least one candidate calling tool.

[0112] In one embodiment, the exploration module 510 is also used to obtain model thinking process data from the analysis results of each model; input the model thinking process data into the large model; call the summary function of the large model to perform a summary operation on the model thinking process data and obtain tool call suggestions.

[0113] In one embodiment, the decision module 520 is further configured to obtain tool description information of each of the at least two callable tools in the tool set; obtain the semantic similarity between each callable tool and each candidate callable tool based on the tool description information of each callable tool; and recall multiple recalled callable tools from each callable tool based on the semantic similarity.

[0114] In one embodiment, the feedback module 530 is further configured to, according to the target task, call the target calling tool through the large model to obtain real calling feedback data, and use the real calling feedback data as calling feedback data; and / or, according to the target task, simulate calling the target calling tool through the large model to obtain simulated calling feedback data, and use the simulated calling feedback data as calling feedback data.

[0115] In one embodiment, the apparatus is further configured to perform invocation operations on each target invocation tool to obtain data resources returned by each target invocation tool; and to determine whether the target invocation tool can complete the target task based on whether the data resources returned by each target invocation tool meet the data resource acquisition requirements of the target task.

[0116] Based on the same inventive concept, this application also provides a model training data processing apparatus, the apparatus comprising:

[0117] The tool retrieval model training data module is used to construct tool retrieval model training data based on model task instructions and corresponding recalled calling tools; the recalled calling tools are multiple calling tools that match each candidate calling tool from a preset calling tool set; the candidate calling tools are calling tools output by the target large model tool exploration unit when processing the model task instructions; the target large model is the large model in any of the above-mentioned large model tool calling methods.

[0118] The tool agent training data construction module is used to construct tool agent training data based on the model thinking process data output by the tool decision unit of the target large model, the target calling tools, and the corresponding tool calling suggestions.

[0119] The tool calls training data output module is used to output the tool calls training data; the tool calls training data includes the tool retrieves model training data and the tool agent training data.

[0120] The modules in the aforementioned large model tool calling device and model training data processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0121] In one exemplary embodiment, a computer device is provided, which may be a terminal or an intelligent vehicle, and its internal structure diagram may be as follows. Figure 6As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a large-scale tool invocation method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0122] Those skilled in the art will understand that the structure shown in Figure 6 is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0123] In one embodiment, an intelligent vehicle is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described large model tool invocation method in various embodiments.

[0124] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described large model tool invocation method embodiments.

[0125] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the above-described method for invoking a large computational model tool.

[0126] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0127] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic resistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence processors, etc., and are not limited to these.

[0128] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0129] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for calling a large model tool, characterized in that, The method includes: The system acquires model task instructions to instruct the large model to perform corresponding target tasks, which serve as model input information for this round of thinking. This model input information is then repeatedly input into the tool exploration unit of the large model, outputting at least one candidate calling tool and obtaining tool calling suggestions from the tool exploration unit. The at least one candidate calling tool includes the initial calling tool from multiple model parsing results output by the tool exploration unit. Each model parsing result includes at least one initial calling tool and model thinking process data. Each model parsing result is generated by the tool exploration unit each time it parses the model task instructions from the model input information. The tool calling suggestions are obtained by summarizing the model thinking process data from each model parsing result using the large model's summary function. From a preset set of calling tools, recall multiple recalled calling tools that match each of the candidate calling tools; input the tool calling suggestions, the multiple recalled calling tools, and the model input information into the tool decision unit of the large model, output at least one target calling tool, and obtain the tool calling suggestions from the tool decision unit; the target calling tool is the calling tool among the multiple recalled calling tools; If the target calling tool cannot complete the target task, the calling feedback data of the target calling tool is obtained, and the model thinking process data of the tool decision unit, the calling feedback data, and the model task instructions are concatenated into the model input information for the next round of thinking. Then, the step of inputting the model input information into the tool exploration unit of the large model multiple times is returned to be executed until the target calling tool output by the tool decision unit can complete the target task.

2. The method according to claim 1, characterized in that, The tool exploration unit that inputs the model input information multiple times into the large model and outputs at least one candidate invocation tool includes: The model input information is input into the tool exploration unit of the large model; The tool exploration unit parses the model task instructions in the model input information to generate model parsing results; the model parsing results include at least one initial invocation tool. Return to the tool exploration unit that inputs the model input information into the large model until the number of model parsing results meets a preset threshold, and use each of the initial calling tools in the multiple model parsing results as the at least one candidate calling tool.

3. The method according to claim 2, characterized in that, Each model parsing result also includes model thinking process data; the tool invocation suggestions for obtaining the tool exploration unit include: Obtain the model thinking process data from the analysis results of each model; Input the thinking process data of each model into the large model; The summary function of the large model is invoked to summarize the thinking process data of each model and obtain the tool invocation suggestions.

4. The method according to claim 1, characterized in that, The step of recalling multiple recalled invocation tools that match each of the candidate invocation tools from a preset invocation tool set includes: From the set of tools that can be called, obtain the tool description information of each of the tools that can be called; Based on the tool description information of each of the supported calling tools, obtain the semantic similarity between each of the supported calling tools and each of the candidate calling tools; Based on the semantic similarity, multiple recalled calling tools are recalled from each of the supported calling tools.

5. The method according to claim 1, characterized in that, The step of obtaining the call feedback data of the target calling tool includes: According to the target task, the target invocation tool is invoked through the large model to obtain real invocation feedback data, and the real invocation feedback data is used as the invocation feedback data; And / or, According to the target task, the target calling tool is simulated through the large model to obtain simulated calling feedback data, and the simulated calling feedback data is used as the calling feedback data.

6. The method according to claim 1, characterized in that, The method further includes: Perform the invocation operation on each of the target invocation tools to obtain the data resources returned by each of the target invocation tools; Based on whether the data resources returned by each of the target calling tools meet the data resource acquisition requirements of the target task, it is determined whether the target calling tool can complete the target task.

7. A method for processing model training data, characterized in that, The method includes: Based on the model task instructions and the corresponding recalled calling tools, a tool is constructed to retrieve model training data; the recalled calling tools are multiple calling tools that match each candidate calling tool from a preset calling tool set; the candidate calling tools are calling tools output by the tool exploration unit of the target large model when processing the model task instructions; the target large model is the large model in the large model tool calling method according to any one of claims 1 to 6; Based on the model thinking process data, target tool invocation, and corresponding tool invocation suggestions output by the tool decision unit of the target large model, construct tool agent training data; The output tool calls up training data; the tool calling up training data includes the model training data retrieved by the tool and the tool agent training data.

8. An intelligent vehicle, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images