Methods, computing devices, and storage media for large language model oriented multi-tool invocation

Abstract

Embodiments of the present application relate to a method, a computing device and a storage medium for large language model-oriented multi-tool invocation. The method comprises: receiving an input request of a user, parsing via a predetermined large language model to obtain an original intention of the user; based on the original intention of the user, determining a candidate tool list from a pre-registered tool set via the predetermined large language model, determining at least one target tool from the candidate tool list, generating a tool invocation scheme about the original intention of the user via the predetermined large language model, the tool invocation scheme being associated with the at least one target tool; based on the tool invocation scheme, running the at least one target tool in a predetermined system environment; inputting a running result of the at least one target tool into the predetermined large language model, and determining response information about the input request. Thus, the business scenarios of a large number of heterogeneous APIs can be adapted, the risk of large model context overflow is reduced, and the accuracy and stability of the output result are improved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of data processing, and more specifically to a method, computing device, and storage medium for multi-tool invocation for large language models. Background Technology

[0002] With the popularization of Large Language Model (LLM) technology, intelligent interaction systems based on LLM have been widely used in various data query and analysis scenarios. Model Context Protocol (MCP), as a mainstream general interface protocol, enables the connection between large models and external tools and data sources, and has become an important foundation of the AIAgent ecosystem.

[0003] However, existing multi-tool invocation schemes for large language models can lead to problems when providing data services in specific fields (such as finance and accounting). These fields have a large amount of historical data and involve a wide variety of tools. During the data service provision process, the large amount of data returned by each tool can easily cause problems such as high computational pressure and context overflow in the large language model. This may result in problems such as low parsing accuracy, inability to correctly understand user intent, and uncontrollable invocation results. Consequently, the output results may deviate from user needs, have poor stability, and low accuracy.

[0004] In summary, the existing methods for calling multiple tools for large language models have the following shortcomings: they cannot adapt to business scenarios with massive heterogeneous APIs, have a high risk of model context overflow, and produce inaccurate output results. Summary of the Invention

[0005] To address the aforementioned issues, this invention provides a method, computing device, and storage medium for multi-tool invocation of large language models, adapting to business scenarios with massive heterogeneous APIs, reducing the risk of context overflow in large models, and improving the accuracy and stability of output results.

[0006] According to a first aspect of the present invention, a method for multi-tool invocation oriented to a large language model is provided, comprising: receiving an input request from a user and parsing it via a predetermined large language model to obtain the user's original intent; determining a list of candidate tools from a pre-registered lightweight tool set via the predetermined large language model based on the user's original intent; determining at least one target tool from the list of candidate tools based at least on the user's original intent, so as to generate a tool invocation scheme for the user's original intent via the predetermined large language model, the tool invocation scheme being associated with at least one target tool; running at least one target tool in a predetermined system environment based on the tool invocation scheme; and inputting the running result of the at least one target tool into the predetermined large language model to determine response information regarding the input request.

[0007] In some embodiments, determining a candidate tool list from a pre-registered lightweight toolset via a predetermined large language model based on the user's original intent includes: loading the descriptive information of the tools in the pre-registered lightweight toolset into the predetermined large language model; and determining the candidate tool list via the predetermined large language model based on the user's original intent and the descriptive information of the tools.

[0008] In some embodiments, determining at least one target tool from a list of candidate tools based at least on the user's original intent, so as to generate a tool invocation scheme about the user's original intent via a predetermined large language model, includes: loading detailed information of each candidate tool involved in the candidate tool list into the predetermined large language model, so that the predetermined large language model matches at least one target tool based on the user's original intent and contextual information about the input request; and determining the parameter passing relationship between the relevant parameters of the matched at least one target tool and the corresponding target tool, so that the predetermined large language model generates a tool invocation scheme including invocation order and parameter flow logic.

[0009] In some embodiments, running at least one target tool in a predetermined system environment based on a tool invocation scheme includes: generating executable invocation code corresponding to the matched at least one target tool via a predetermined large language model based on the tool invocation scheme; and determining the running order of the matched at least one target tool based on the parameter passing relationship between the relevant parameters of the matched at least one target tool and the corresponding target tool, so as to run the matched at least one target tool in the predetermined system environment; wherein the running order includes parallel running or serial running.

[0010] In some embodiments, inputting the execution result of at least one target tool into a predetermined large language model includes: in response to the serial execution order of the matched at least one target tool, directly transmitting the output result of the first-run target tool to the adjacent target tool that runs later; and sequentially completing the execution of all matched target tools in a predetermined system environment, and inputting only the output result of the last-run target tool into the predetermined large language model.

[0011] In some embodiments, running at least one target tool in a predetermined system environment based on a tool invocation scheme includes: in response to matching multiple target tools, concatenating the multiple matched target tools based on the running order and parameter flow logic of the multiple matched target tools to generate a composite target tool; and running the generated composite target tool in the predetermined system environment.

[0012] In some embodiments, running at least one target tool in a predetermined system environment based on a tool invocation scheme further includes: in response to no target tool being matched, determining business logic about the input request based on the user's original intent and contextual information about the input request; for the business logic, writing a new target tool and corresponding executable invocation code based on each tool in a pre-lightweight registered tool set via a predetermined large language model; and running the newly written target tool in the predetermined system environment.

[0013] In some embodiments, the method for multi-tool invocation for large language models further includes: obtaining time keywords and their types based on the user's original intent and contextual information about the input request, including precise time keywords, relative time keywords, fuzzy time keywords, and / or financial statement time keywords; matching a time recognition scheme to the type of time keyword; and parsing a standardized target time parameter adapted to the business scenario corresponding to the input request based on the matched time recognition scheme, and substituting the standardized target time parameter into the tool invocation scheme.

[0014] In some embodiments, the time recognition scheme for type matching of time keywords includes one or more of the following: for precise time keywords, performing standardized format parsing; for relative time keywords, performing conversion based on the current system time; for financial report time keywords, performing query and escaping based on the corresponding subject; and for fuzzy time keywords, performing differentiated semantic mapping based on the corresponding business scenario.

[0015] According to a second aspect of the invention, a computing device is provided, the computing device comprising: at least one processing unit; at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions, when executed by the at least one processing unit, causing the device to perform steps according to the method of the first aspect.

[0016] According to a third aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a machine, implements the method according to the first aspect.

[0017] As described above, the traditional methods for calling multiple tools for large language models have the following shortcomings: they cannot adapt to business scenarios with massive heterogeneous APIs, have a high risk of model context overflow, and produce inaccurate output results.

[0018] To at least partially address one or more of the aforementioned problems and other potential issues, an exemplary embodiment of the present invention proposes a method for multi-tool invocation for a large language model. This method involves receiving a user's input request, parsing it via a predetermined large language model to obtain the user's original intent, then filtering candidate tools from a pre-registered tool set based on the original intent and determining at least one target tool, further generating a corresponding tool invocation scheme associated with the at least one target tool; running the at least one target tool in a predetermined system environment based on the tool invocation scheme; and inputting the running results of the at least one target tool into the predetermined large language model to determine response information regarding the input request.

[0019] Thus, the above solution establishes an automatic association mechanism between user requests, tool selection, tool execution, and result generation, realizing the collaborative processing of large language models and external tools, and improving the automated processing capability, tool invocation accuracy, and response effectiveness in complex task scenarios.

[0020] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0021] The above and other features, advantages, and aspects of the various embodiments of the present invention will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements.

[0022] Figure 1A schematic diagram of a system for implementing a method for multi-tool invocation for a large language model according to an embodiment of the present invention is shown.

[0023] Figure 2 A flowchart of a method for multi-tool invocation for large language models according to an embodiment of the present invention is shown.

[0024] Figure 3 A flowchart of a method for generating new tools according to an embodiment of the present invention is shown.

[0025] Figure 4 A flowchart of a method for determining time parameters according to an embodiment of the present invention is shown.

[0026] Figure 5 A block diagram of an electronic device according to an embodiment of the present invention is shown. Detailed Implementation

[0027] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present invention, including various details to aid understanding. These details should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0028] The term "comprising" and its variations as used herein signify open inclusion, i.e., "including but not limited to". Unless otherwise stated, the term "or" means "and / or". The term "based on" means "at least partially based on". The terms "one example embodiment" and "one embodiment" mean "at least one example embodiment". The term "another embodiment" means "at least one additional embodiment". The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0029] Figure 1 A schematic diagram of a system 100 for implementing a method for multi-tool invocation for a large language model according to an embodiment of the present invention is shown. Figure 1 As shown, system 100 includes computing device 110, large language model 130, network 140, user device 150, and toolset 160.

[0030] Regarding user device 150, it carries an application configured to receive user input requests, parse the user input requests to obtain the user's original intent, and further implement a multi-tool invocation method for large language models to determine response information for the input request, thereby completing the response to the user request. User requests may include, for example, asking questions, making queries (e.g., querying information, data from various industries, etc.), chatting, or requests to execute other related applications or tools.

[0031] Regarding user devices 150, such as mobile phones, laptops, tablets, desktops, and other devices.

[0032] Regarding large language models 130, examples include the Qwen model, ChatGPT model, DeepSeek series models, Qwen series models, GLM series models, and other large language models.

[0033] The large language model 130 can be deployed on a cloud server or on a computing device 110. When the large language model 130 is deployed on a cloud server outside the computing device 110, it can interact with the large language model 130 and the toolset 160 via network 140.

[0034] The computing device 110 receives input information from the user. The user can directly operate the computing device 110 to input information, or the user can input information through the user device 150 and send the user's input information to the computing device 110 through the user device 150.

[0035] The computing device 110 may have one or more processing units, including dedicated processing units such as graphics processing units (GPUs), field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), general-purpose computing on graphics processing units (GPGPUs), and general-purpose processing units such as CPUs. Additionally, one or more virtual machines may run on each computing device 110. In some embodiments, the computing device 110 includes, for example, an original intent parsing module 112, a candidate tool list determination module 114, a tool invocation scheme generation module 116, a tool execution module 118, and a response information determination module 120.

[0036] The original intent parsing module 112 is used to receive the user's input request and parse it through a predetermined large language model to obtain the user's original intent.

[0037] The candidate tool list determination module 114 is used to determine a candidate tool list from a pre-lightly registered set of tools based on the user's original intent via a pre-defined large language model.

[0038] Tool invocation scheme generation module 116 is used to determine at least one target tool from a list of candidate tools based at least on the user's original intent, so as to generate a tool invocation scheme about the user's original intent via a predetermined large language model, the tool invocation scheme being associated with at least one target tool.

[0039] Tool execution module 118 is used to run at least one target tool in a predetermined system environment based on a tool invocation scheme.

[0040] The response information determination module 120 is used to input the running results of at least one target tool into a predetermined large language model in order to determine response information about the input request.

[0041] Figure 2 A flowchart of a method 200 for multi-tool invocation for a large language model according to an embodiment of the present invention is shown. Method 200 may be derived from, for example... Figure 1 The computing device 110 shown can be used for execution, and can also be used in Figure 5 The method is performed at the illustrated electronic device 500. It should be understood that method 200 may also include additional steps not shown and / or the steps shown may be omitted, and the scope of the invention is not limited in this respect.

[0042] In step 202, the computing device 110 receives the user's input request and parses it using a predetermined large language model to obtain the user's original intent.

[0043] User input can be one or more of the following: text, voice, video, audio, or image. It should be understood that user input may be colloquial, incomplete, vague, non-standard, or context-dependent. For example, a user inputting "Company C's market value surged last year? Can you tell me how it's doing this year?" suffers from multiple problems, including vague referencing, abbreviated timeframes, and implicit business requirements.

[0044] Regarding obtaining the user's original intent, the user's input information may be ambiguous, multimodal, or relatively colloquial. By parsing the user's input information through a pre-defined large language model, it is possible to uniformly translate multimodal content, complete incomplete sentences, resolve ambiguous references, standardize colloquial expressions, and complete semantics based on contextual relationships. From this, the user's true business needs can be accurately extracted, thereby obtaining the user's original intent. Furthermore, the large language model can output structured or pre-formatted original intent information for use in subsequent tool filtering stages.

[0045] In addition, the large language model can also extract key parameter materials such as various time keywords, enterprise entities, and industry concepts from the input content during the parsing process, providing basic data support for subsequent differentiated time recognition, candidate tool matching, and call scheme generation.

[0046] In step 204, computing device 110 determines a list of candidate tools from a pre-registered lightweight toolset based on the user's original intent via a predetermined large language model.

[0047] In some embodiments, the computing device 110 loads the descriptive information of tools from a pre-lightweight registered toolset into a predetermined large language model; the computing device 110 determines a list of candidate tools based on the user's original intent and the descriptive information of the tools via the predetermined large language model.

[0048] Regarding pre-registered lightweight tools, it should be understood that this differs from the traditional large-scale tool invocation method, which requires full registration of all tool parameters, interface fields, parameter passing rules, complete documentation, and other information. In contrast, this invention employs a lightweight tool registration mechanism adapted to the MCP protocol. For example, each MCP tool registered in the MCP management platform is encapsulated, generating corresponding Python functions. These Python functions include, for example, a function name (e.g., the tool name), a function description (e.g., a tool description), and detailed function information (e.g., including full information about the tool such as inputs, outputs, and fields).

[0049] The tool description, for example, is a concise and structured description used in the initial tool selection phase. It is very small (e.g., less than 200 words) and only includes the tool name, core capabilities, applicable scenarios, tool classification, and a brief functional overview. The description information does not include any specific call parameters, code logic, or interface details; it is only used for semantic matching and coarse-grained filtering. Complete and detailed tool information is not loaded during the persistent phase, thus achieving a layered loading mechanism of "persistent lightweight loading and on-demand loading of details."

[0050] During the initialization phase, when the tool loads the pre-defined large language model, it only loads lightweight metadata such as the tool's unique identifier, tool description, and business scenario tags. It does not input all parameters such as complex parameter structures, field definitions, request bodies, return formats, and detailed parameter passing logic, thereby obtaining a pre-registered lightweight toolset.

[0051] For example, in financial data usage scenarios, all financial API tools and data processing tools can be uniformly incorporated into the tool set in a lightweight form and loaded into a pre-defined large language model. This supports the batch lightweight registration of massive heterogeneous financial APIs and prevents the expansion of the model's resident context due to the increase in the number of tools, thus saving the model's context and maintaining the integrity of the tools.

[0052] Regarding the determination of the candidate tool list for the pre-defined large language model, for example, all function names and function descriptions, along with the context of the user's historical dialogue, are used as background knowledge to generate prompt words to be input into the pre-defined large language model. After receiving the user's input, the pre-defined large language model will output a list of tools to be used according to the user's intention and the tools that have been lightweight registered in the large language model, that is, determine the candidate tool list.

[0053] For example, if a user inputs "analyze the market capitalization trend of Company C this year and the views of recent institutional research reports," the pre-defined large language model parses the input to obtain the standardized original intent as: obtain the market capitalization change data of the target company this year and obtain the analysis content of the corresponding recent institutional research reports. Then, the pre-defined large language model only loads the brief information of all tools (i.e., using a pre-registered lightweight tool set) for global semantic matching. Through the scenario tags and function descriptions in the briefs, it quickly filters out the "company market capitalization query tool" and "institutional research report retrieval tool" with the highest matching degree, excluding irrelevant tools such as news announcement tools, fund flow tools, and short-term market data broadcasting tools, and quickly generates a candidate tool list. The above solution, through lightweight tool registration, allows the brief information of tools to reside lightly in the pre-defined large model for initial tool screening, solving the technical pain points of the traditional full-scale registration model: the larger the tool library, the higher the context usage, and the slower the screening speed. While ensuring the accuracy of the initial screening, it greatly reduces the memory overhead of the large model, improves the tool screening efficiency and system stability in scenarios with massive financial tools, and avoids invalid tools from entering subsequent processes, improving the overall query accuracy of the call chain.

[0054] In step 206, computing device 110 determines at least one target tool from a list of candidate tools based at least on the user's original intent, so as to generate a tool invocation scheme for the user's original intent via a predetermined large language model, the tool invocation scheme being associated with at least one target tool.

[0055] In some embodiments, the computing device 110 loads details of each candidate tool in the candidate tool list into a predetermined large language model so that the predetermined large language model matches at least one target tool based on the user's original intent and contextual information about the input request.

[0056] Continuing with the example above, the candidate tool list includes three tools: a market capitalization query tool, a research report retrieval tool, and a report analysis tool. The large language model loads the complete parameter details of these three tools from the candidate tool list and performs precise matching based on the user's intent (the subject "Company C"), the time dimension "this year", and the business dimension "trend and institutional views". This confirms that the market capitalization query tool and the research report retrieval tool are both necessary target tools.

[0057] Therefore, the above solution initially loads only a lightweight introduction during the screening stage. Only after entering the precise matching stage does it dynamically retrieve the complete parameter list, input parameter fields, data format, constraints, output structure, dependency parameters, and other detailed information of each tool for the subset of candidate tools as needed. This avoids keeping all tool details in the model and enables resource consumption on demand.

[0058] In some embodiments, computing device 110 determines the parameter transfer relationship between the relevant parameters of at least one matched target tool and the corresponding target tool, so as to generate a tool calling scheme containing the calling order and parameter flow logic of a predetermined large language model.

[0059] Regarding the tool invocation scheme, it is not a single tool instruction. For example, it includes: the list of selected target tools, the tool execution order, the parameter mapping relationship between tools, the rules for completing required parameters, the data output flow path, and the serial and parallel execution strategy.

[0060] For example, if a user's requirement is "to filter corresponding negative research reports and summarize the risks based on Company C's market capitalization fluctuations this year," then there is a strong business dependency between the tools: market capitalization fluctuation data needs to be obtained first, and then the abnormal market capitalization range parameter needs to be passed to the research report retrieval tool to limit the research report filtering scope. In this case, the big data model automatically identifies the parameter flow relationship: the output parameters (time interval, fluctuation range) of the market capitalization tool are the input dependent parameters of the research report tool, and the big data model generates a complete calling scheme based on this, including a serial calling sequence and automatic parameter flow across tools. If the two tools have no data dependency, such as simultaneously querying market capitalization and industry valuation, then a parallel calling scheme is generated for these two tools.

[0061] Therefore, the above solution solves the problem of resource waste in traditional full loading by combining lightweight registration and dynamic full loading of tools; by automatically identifying the parameter dependencies and flow relationships of multiple tools, it solves the technical defects of traditional independent multi-tool calls, inability to automatically chain parameters, and the need for manual orchestration, realizing intelligent and automated orchestration of complex multi-tool links, and greatly improving the adaptability and calling accuracy of complex business scenarios.

[0062] For example, to further address the context capacity limitation of large language models, this solution also supports manual tool specification. For instance, based at least on the user's original intent, the user's preferred tool is identified, a tool call tag is generated, and input into the large language model, so that the tool corresponding to the tool call tag appears in the candidate tool list, and the tool corresponding to the tool call tag is determined as a tool in at least one target tool.

[0063] For example, users can specify one or more target tools in their input using the @ symbol, ensuring that the request only uses the specified tools and preventing the large language model from arbitrarily calling irrelevant tools. For instance, when a user explicitly needs "financial statement comparison analysis," they can force the use of only financial statement tools by inputting "@financial statement query tool, @financial indicator calculation tool," thus avoiding the accidental invocation of other tools.

[0064] In some embodiments, tools that match naming rules or function tags are batch-filtered using regular expression matching patterns. For example, the regular expression " _financial_report "Batch filter all financial reporting tools and automatically filter non-financial analysis tools to achieve precise, scenario-based filtering of a large number of tools and improve the efficiency of calling complex tool libraries. For example, specific tools can be removed in batches using prompts."

[0065] Meanwhile, this solution supports limiting the number of interactions with the large language model and / or the maximum number of calls to a specified tool in custom prompts, and imposes hard boundary constraints on the hierarchy and number of loop calls in multi-turn chained calls. For example, it can configure a maximum of 3 interactions in a single-turn dialogue and a maximum of 2 calls to a specified tool, thereby avoiding issues such as context overflow and response timeouts caused by multi-level nested calls in the large language model.

[0066] Therefore, this invention achieves a precise and controllable calling mode for MCP tools by implementing a customizable prompt word constraint mechanism, refined tool filtering rules, and controlling the number of calls. This significantly improves the accuracy and stability of calls under complex business tool clusters and effectively avoids technical problems such as exceeding the context limit of large language models and invalid calls.

[0067] In step 208, computing device 110 runs at least one target tool in a predetermined system environment based on a tool invocation scheme.

[0068] In some embodiments, running at least one target tool in a predetermined system environment includes: computing device 110 generating executable invocation code corresponding to the matched at least one target tool via a predetermined large language model based on a tool invocation scheme.

[0069] Continuing the example above, the market capitalization query tool and research report retrieval tool are identified from the candidate tool list. Both are essential target tools. Detailed information about these two tools is obtained (e.g., complete parameter list, input fields, data format, constraints, output structure, dependent parameters, etc.). This detailed information (i.e., the corresponding function details) is combined with user input, the user's original intent, and historical context to generate new prompts via prompt engineering. These newly generated prompts are then input into a predefined large language model, which generates a tool invocation scheme. Finally, executable invocation code corresponding to the market capitalization query tool and research report retrieval tool is generated based on the tool invocation scheme. For example, the large language model generates Python code to implement the invocation of these two tools. The generated executable invocation code is then input into an independent environment (e.g., a Python environment) that integrates all tools (and their corresponding functions) for execution.

[0070] In some embodiments, the computing device 110 determines the running order of the at least one matched target tool based on the parameter transfer relationship between the relevant parameters of the at least one matched target tool and the corresponding target tool, so as to run the at least one matched target tool in a predetermined system environment; wherein, the running order includes parallel running or serial running.

[0071] Regarding parameter passing relationships, for example, the trading date, price change data, and trading volume data output by the "Individual Stock Market Data Tool" can be used as input parameters for the "Market Anomaly Analysis Tool"; market data is a prerequisite parameter, and the analysis results are a subsequent output, forming a clear parameter passing link between the two.

[0072] Regarding the determination of the execution order, for tools that are independent and have no parameter dependencies, such as "individual stock market value query" and "industry index query", the large language model determines that they run in parallel, so it initiates calls simultaneously to improve response speed. For tool chains with dependencies, such as the execution logic of "basic data query → abnormal indicator extraction → risk rating calculation", the large language model determines that they run sequentially. In this case, it is necessary to ensure that the business logic is correct and the parameter passing is effective, and use the output of the previous tool as the input of the next tool to ensure the correct operation of the tools.

[0073] Therefore, the above solution eliminates the cost of manual coding adaptation by automatically generating executable code through a large language model; it balances query response speed and the correctness of complex link logic through an adaptive serial-parallel scheduling mechanism; and it relies on an independent system environment to completely isolate the computation of massive business data and interface flow from the model context, thus solving the core pain points of large model window overflow and excessive computing power consumption from the runtime layer.

[0074] In some embodiments, in response to matching multiple target tools, the computing device 110 concatenates the multiple target tools based on the running order and parameter flow logic of the matched multiple target tools to generate a composite target tool; and runs the generated composite target tool in a predetermined system environment.

[0075] For example, in a traditional business scenario, there might be a need to diagnose the performance of the CSI 300 constituent stocks. In this scenario, the business would typically be implemented using two APIs: the first API retrieves the constituent stocks based on the index, and the second API receives a list of stocks, checks their financial performance one by one, and finally outputs a statistical table. Under the MCP framework of this solution, this business scenario is replicated in a large language model application, i.e., it is registered as a tool for diagnosing the performance of index constituent stocks. This tool serially mounts two existing APIs (for example, serially mounting the APIs of the two functions mentioned above, merging them into one). Therefore, from the perspective of the MCP layer, only one API is serving.

[0076] Therefore, the above solution can automatically combine and encapsulate multiple discrete and independent API tools into a single composite tool according to business logic and parameter dependencies. Only a single unified service entry point is exposed at the MCP protocol service layer, eliminating the need to mount and manage multiple original APIs individually, greatly simplifying API mounting, operation and maintenance, and interface management costs. For example, it can adapt to large-scale deployment scenarios involving massive amounts of discrete APIs in the financial sector.

[0077] In step 210, the computing device 110 inputs the running results of at least one target tool into a predetermined large language model in order to determine response information regarding the input request.

[0078] In some embodiments, in response to the sequential execution order of at least one matched target tool, the computing device 110 directly transmits the output of the first-run target tool to the adjacent target tool that runs later.

[0079] For example, regarding the serial data flow: raw financial statement data acquisition tool → financial statement indicator cleaning tool → year-on-year financial statement analysis tool. In traditional solutions, the results of each step are sent back to the large model, and the superposition of multi-level data easily leads to overflow. In this solution, the massive amount of structured data output by the raw financial statement data acquisition tool is directly transmitted point-to-point to the cleaning tool, and the cleaned structured indicator data is directly transmitted to the analysis tool. The entire data flow is a closed loop within the tool's internal flow, without passing through the context of the large model.

[0080] In some embodiments, the computing device 110 sequentially runs all matching target tools in a predetermined system environment, and only inputs the output of the last run target tool into a predetermined large language model.

[0081] In the above solution, the mechanism of direct parameter transmission between serial tools, isolation of intermediate data, and input of only the final result into the model completely solves the problem of large model context window overflow caused by massive intermediate data in multi-API chain calls in business scenarios. This significantly reduces the pressure of model data processing and transmission overhead, while ensuring the data integrity and logical coherence of multi-level tool linkage calls. It is also compatible with API calls that were not considered for compatibility with large models in the initial design.

[0082] Furthermore, regarding tool output configuration, in some business scenarios (such as financial scenarios), there are massive amounts of existing APIs. The output data formats of APIs developed by different vendors and at different times are chaotic, including plain text, custom tables, unstandardized fields, semi-structured JSON, and non-standard messages, among other heterogeneous formats. Directly inputting these into large language models can lead to compatibility issues such as difficulty in model understanding, loss of key information recognition, and decreased inference accuracy. This solution uses a unified and standardized output mechanism to convert all tool-returned data into Markdown format, providing a three-tiered conversion scheme to adapt to API output data of varying complexity. Specifically, it includes three conversion implementation methods.

[0083] The first method is to directly convert the API's output into Markdown, and add data descriptions and explanations before and after it. This includes basic information such as the dataset's name, description, source, and time, as well as metadata information such as field names, types, and field descriptions.

[0084] The second approach is to define a new Markdown template and reference data from the original API output.

[0085] The third method involves customizing the script to process the results and generate Markdown. Using a scripting language (such as JavaScript), the output of the original API is processed more freely and complexly to create new data, which is then output as Markdown for use by larger models.

[0086] Therefore, the above solution covers all existing API heterogeneous data scenarios through a three-layer hierarchical conversion mechanism, with strong adaptability. It eliminates the need to develop separate adaptation logic for each type of API, reducing the cost of API access and transformation. Furthermore, it outputs a unified Markdown standard format, relying on Markdown's built-in heading, table, list, and annotation syntax to enhance the large model's sensitivity to business data (e.g., numbers, indicators, and entity information in financial business), improving the accuracy of model data inference and analysis. In addition, it can utilize the metadata and field descriptions attached to Markdown to provide large models with auxiliary information on data sources and field definitions, reducing the model's misunderstanding of business data.

[0087] Figure 3 A flowchart of a method 300 for generating a new tool according to an embodiment of the present invention is shown. Method 300 may be derived from, for example... Figure 1 The computing device 110 shown can be used for execution, and can also be used in Figure 5 The method is performed at the illustrated electronic device 500. It should be understood that method 300 may also include additional steps not shown and / or the steps shown may be omitted, and the scope of the invention is not limited in this respect.

[0088] In step 302, if the computing device 110 does not match the target tool, the business logic regarding the input request is determined based on the user's original intent and contextual information about the input request.

[0089] For example, a user's input request might be: "Retrieve all trading days in the past year where the monthly decline exceeded 15%, then retrieve all company announcements that occurred on that day and statistically analyze the distribution of announcement types." After traversing the lightweight registered tool set via systems, computing devices 110, or other electronic devices that utilize this tool's calling method, it was found that only "individual stock quote query tools" and "company announcement search tools" were included. There was no composite tool capable of automatically filtering trading days with sharp declines, matching announcements for those dates, and performing categorized statistics; therefore, the target tool was not found.

[0090] Continuing with the above example, the pre-defined large language model automatically decomposes the complete business logic based on context as follows: First, obtain the daily price change data for the whole year through market data tools; second, filter out abnormal trading days with price drops exceeding the threshold; third, batch input the abnormal dates as parameters to retrieve the corresponding announcements; fourth, classify, count, and aggregate the massive amount of returned announcements to form the final analysis results.

[0091] In step 304, computing device 110, for the business logic, writes matching new target tools and corresponding executable call code based on each tool in the pre-lightweight registered toolset using a predetermined large language model.

[0092] Continuing with the example above, the pre-defined large language model automatically writes a completely new composite tool code based on the capabilities of existing market data query tools and announcement retrieval tools. Internally, the code first calls the market data tool to retrieve daily data for the entire year, then uses a local script to perform threshold filtering, generating a batch list of dates that meet the criteria. Next, it iteratively inputs the data into the announcement interface to complete batch retrieval. Finally, it performs aggregation statistics, classification counting, and result normalization on the massive amount of structured announcement data. This new tool is entirely dynamically generated by the large language model, requiring no manual development or prior registration, and can complete the user's composite analysis needs in a closed loop. Simultaneously, all intermediate processes of massive data filtering, iterative calls, and batch aggregation run within the tool space and / or an independent system environment, without consuming the context capacity of the large language model, further avoiding the problem of exceeding the limits of the large language model's ultra-high traffic data flow.

[0093] In step 306, computing device 110 runs the newly written target tool in a predetermined system environment.

[0094] Continuing with the above example, computing device 110 runs the newly generated "sales crash day + corresponding announcement statistics" composite tool in the predetermined system environment. It completes the annual data traversal, threshold filtering, batch date parameter passing, multi-round announcement retrieval, and announcement type statistics. Finally, it outputs the normalized statistical conclusions and detailed data, and inputs the output results into the predetermined large language model to generate user-readable response content.

[0095] Therefore, when the pre-set tools cannot cover the complex, niche, and composite needs of users, the above solution can rely on the capabilities of existing lightweight tools to autonomously deduce business logic and dynamically write new tools and executable code, enabling the creation of tools and autonomous closed-loop execution even without pre-set tools. At the same time, by using the offline code running mode to carry out ultra-large batch data operations and multi-level tool nesting and flow, it avoids the context capacity limitations of large language models, solves the technical defects of traditional tool calling system capabilities being fixed, scenario coverage being limited, big data flow overflow, and complex chain tasks being unable to be automatically executed, and greatly improves the scenario generalization ability of MCP tool calling Agent.

[0096] Figure 4 A flowchart of a method 400 for determining time parameters according to an embodiment of the present invention is shown. Method 400 may be derived from, for example... Figure 1 The computing device 110 shown can be used for execution, and can also be used in Figure 1 The system 100 shown is in Figure 5 The method is performed at the illustrated electronic device 500. It should be understood that method 400 may also include additional steps not shown and / or the steps shown may be omitted, and the scope of the invention is not limited in this respect.

[0097] In step 402, the computing device 110 obtains time keywords and their types based on the user's original intent and contextual information about the input request. The time keywords include precise time keywords, relative time keywords, fuzzy time keywords, and / or financial statement time keywords.

[0098] In step 404, the computing device 110 matches the time identification scheme according to the type of the time keyword.

[0099] Regarding time keywords, the questions that the MCP client can usually receive, and the time keywords that can be decomposed through natural language grammar analysis using a pre-defined large language model, are divided into four categories in this solution: precise time keywords, relative time keywords, fuzzy time keywords, and financial statement time keywords.

[0100] In some embodiments, standardized format parsing is performed for precise time keywords.

[0101] Regarding precise time keywords, for example, for deterministic and precise time expressions containing specific years, quarters, months, and dates (such as December 2025, the third quarter of 2024, etc.), based on a large language model, various colloquial abbreviations and non-standard written time texts are uniformly standardized and converted into the YYYYMMDD standard structured time format, and the output is a standard time field that can be universally recognized by various tools.

[0102] In some embodiments, relative time keywords are converted by combining the current system time.

[0103] Regarding relative time keywords, such as explicit relative time keywords like "last week," "last year," "last three months," and "last year's same period last year," automatic time offset conversion is performed based on the large language model combined with the system's current base time to obtain a standard structured time format expression. For example, if the system's current base time is October 1, 2025, explicit relative time keywords such as "last six months," "last week," "last year's same period last year," and "last three months" entered by the user are converted into time offsets: "last six months" is converted to April 1, 2025 to October 1, 2025, and "last week" is converted to September 24, 2025 to October 1, 2025, generating absolute time interval parameters that can be directly adapted to various MCP tool interfaces.

[0104] In some embodiments, ambiguous time keywords are escaped based on the context.

[0105] Regarding fuzzy time keywords, such as "recently", "latest", "recent" and other fuzzy time keywords, different escapes are performed according to the context semantics. For example, in the news and information scenario, "latest" is mapped to the information data released in the last three days and sorted in reverse chronological order; in the financial report query scenario, "latest" is mapped to the latest financial report released by the target listed company; in the research report analysis scenario, "recent" is mapped to the research report data released by institutions within the last month, so as to achieve precise adaptation of the fuzzy time semantics to the scenario.

[0106] In some embodiments, for financial report time keywords, query and escape are combined with the corresponding entity.

[0107] Regarding financial report time keywords, due to the different fiscal year definitions and financial report disclosure cycles of different listed companies, there are different rules and a general time conversion logic cannot be used. For example, after a large language model recognizes financial report time keywords such as "First Quarterly Report for 25th Year" and "Half-Year Report for 2024", it binds the time parameter to the asset entity of the corresponding listed enterprise, retrieves the preset fiscal year definition rule of the enterprise, and accurately translates the financial report cycle statement into the standard natural year time interval.

[0108] For example, when the user inputs "View the data of the First Quarterly Report for 25th Year of CATL", the large language model recognizes the financial report period time keyword and the asset entity "CATL", retrieves the enterprise's fiscal year rule, and accurately converts the financial report quarter cycle into the corresponding standard natural year month and time interval, ensuring that the financial report time caliber is completely consistent with the disclosure rules of the listed company.

[0109] In addition, regarding the recognition of entities, an entity itself has multiple expressions such as name, code, alias (synonym, abbreviation), full name, etc. When constructing an indexed database, multiple expressions of the same entity are fielded and stored in the corresponding vector library. For example, "Moutai, Kweichow Moutai, 600519, Kweichow Moutai Co., Ltd., Kweichow Moutai Group" are all different expressions of the same entity. When performing entity recognition, access the database, collect information such as the name, abbreviation, full name, code, alias of the entity, first based on the natural language recognized by the large language model, obtain the initial entity name from the syntactic analysis, and perform vector similarity retrieval based on the initial entity name, and take the top N as candidate entity names. For the N candidate entity names obtained, in combination with the user's original intention, they are screened by a predetermined arbitration large model (for example, a predetermined large language model) to obtain the final screening result as the entity recognition result.

[0110] In step 406, the computing device 110 analyzes and obtains a standardized target time parameter adapted to the service scenario corresponding to the input request based on the matched time recognition scheme, and substitutes the standardized target time parameter into the tool call scheme.

[0111] Continuing the example above, when a user requests to "view CATL's Q3 2024 financial report, stock price trend over the past six months, and the latest institutional research reports," the parameters are standardized layer by layer using a large language model: the precise time "Q3 2024" is normalized into a standard time range for financial reports; the relative time "over the past six months" is dynamically converted into a fixed market statistics range; the ambiguous time "latest" is mapped to a corresponding time range based on the differences in research report scenarios; and a two-layer entity identification mechanism accurately identifies the asset entity "CATL." Finally, all standardized time parameters and standardized asset entity parameters are uniformly substituted into the calling parameters of the corresponding financial report query tool, market trend tool, and research report retrieval tool to facilitate the generation and execution of tool calling schemes.

[0112] Therefore, the above solution achieves systematic compatibility optimization for core subject-type parameters and business enumeration-type parameters in business scenarios (such as financial business and supply chain business). By distinguishing four types of differentiated time parsing logics—precise time, relative time, fuzzy relative time, and financial report-specific time—and combining a two-layer asset subject identification mechanism of vector retrieval + arbitration big model, it solves various technical problems such as poor adaptability of traditional big model single parsing rules, confusing interpretation of fuzzy time, lack of specific conversion rules for financial report cycles, errors in identifying ambiguous enterprise subjects, and incompatibility of parameters from heterogeneous tools. It achieves standardization, normalization, and scenario-based conversion of all parameters, avoiding MCP tool call errors, data filtering deviations, and result distortion from the parameter source, and significantly improving the parameter compatibility, scenario adaptability, and business processing accuracy of the multi-tool call system.

[0113] Figure 5 A schematic step diagram of an example electronic device 500 that can be used to implement embodiments of the contents of this specification is shown. For example, as Figure 1 The computing device 110 shown can be implemented by an electronic device 500. As shown, the electronic device 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) 502 or loaded from a storage unit (RAM) 508 into a random access memory (RAM) 503. The RAM 503 can also store various programs and data required for the operation of the electronic device 500. The CPU 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0114] Multiple components in electronic device 500 are connected to input / output I / O interface 505, including: input unit 506, such as keyboard, mouse, microphone, etc.; output unit 507, such as various types of monitors, speakers, etc.; storage unit 508, such as disk, optical disk, etc.; and communication unit 509, such as network card, modem, wireless transceiver, etc. Communication unit 509 allows device 500 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0115] The various processes and procedures described above, such as methods 300 to 400, can be executed by the central processing unit CPU 501. For example, in some embodiments, methods 300 to 400 can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 508. In some embodiments, part or all of the computer program can be loaded and / or installed on device 500 via read-only memory ROM 502 and / or communication unit 509. When the computer program is loaded into random access memory RAM 503 and executed by the central processing unit CPU 501, one or more actions of methods 300 to 400 described above can be performed.

[0116] This invention relates to methods, apparatus, systems, electronic devices, computer-readable storage media, and / or computer program products. The computer program product may include computer-readable program instructions for performing various aspects of the invention.

[0117] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0118] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge computing devices. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to computer-readable storage media within the respective computing / processing device.

[0119] The computer program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions. This electronic circuitry can execute the computer-readable program instructions to implement various aspects of the invention.

[0120] Various aspects of the present invention are described herein with reference to flowchart illustrations and / or step diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each step in the flowchart illustrations and / or step diagrams, as well as combinations of steps in the flowchart illustrations and / or step diagrams, can be implemented by computer-readable program instructions.

[0121] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more steps of the flowchart and / or diagram of steps. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more steps of the flowchart and / or diagram of steps.

[0122] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more steps of a flowchart and / or a diagram of steps.

[0123] The flowcharts and step diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each step in the flowchart or step diagram may represent a module, segment, or part of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions indicated in the step may occur in a different order than those indicated in the drawings. For example, two consecutive step steps may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each step in the step diagram and / or flowchart, and combinations of steps in the step diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0124] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for multi-tool invocation for large language models, characterized in that, The method includes: Receive user input requests and parse them using a predefined large language model to obtain the user's original intent; Based on the user's original intent, a list of candidate tools is determined from a pre-registered lightweight toolset via the pre-defined large language model; the pre-registered lightweight toolset includes descriptive information about the tools, which at least does not include specific invocation parameters for the tools. Based at least on the user's original intent, at least one target tool is identified from a list of candidate tools in order to generate a tool invocation scheme for the user's original intent via the predetermined large language model, the tool invocation scheme being associated with the at least one target tool; Based on the tool invocation scheme, run the at least one target tool in a predetermined system environment; and The results of the operation of the at least one target tool are input into the predetermined large language model in order to determine response information regarding the input request.

2. The method according to claim 1, characterized in that, Based on the user's original intent, a list of candidate tools is determined from a pre-registered lightweight toolset via the pre-defined large language model, including: The descriptive information of the tools in the pre-registered lightweight toolset is loaded into the pre-defined large language model; and Based on the user's original intent and the tool's brief information, a list of candidate tools is determined via the predetermined large language model.

3. The method according to claim 1, characterized in that, Based at least on the user's original intent, identifying at least one target tool from a list of candidate tools, and generating a tool invocation scheme about the user's original intent via the predetermined large language model, includes: The detailed information of each candidate tool in the candidate tool list is loaded into the predetermined large language model, so that the predetermined large language model matches at least one target tool based on the user's original intent and contextual information about the input request; and Determine the parameter passing relationship between the relevant parameters of at least one matched target tool and the corresponding target tool, so that the predetermined large language model can generate a tool calling scheme that includes the calling order and parameter flow logic.

4. The method according to claim 3, characterized in that, Based on the tool invocation scheme, running the at least one target tool in a predetermined system environment includes: Based on the tool invocation scheme, executable invocation code corresponding to at least one matched target tool is generated via the predetermined large language model; and based on the relevant parameters of the at least one matched target tool and the parameter passing relationship between the corresponding target tool, the execution order of the at least one matched target tool is determined so as to run the at least one matched target tool in a predetermined system environment; wherein, the execution order includes parallel execution or serial execution.

5. The method according to claim 3 or 4, characterized in that, Inputting the results of the execution of at least one target tool into the predetermined large language model includes: In response to the fact that the execution order of at least one matched target tool is sequential, the output of the first-running target tool is directly transmitted to the adjacent target tool that runs later; and In a predetermined system environment, all matching target tools are executed sequentially, and only the output of the last executed target tool is input into the predetermined large language model.

6. The method according to claim 1, characterized in that, Based on the tool invocation scheme, running the at least one target tool in a predetermined system environment includes: In response to the matching of multiple target tools, based on the execution order and parameter flow logic of the matched target tools, the multiple matched target tools are concatenated to generate a composite target tool; and The generated composite target tool is run in a predetermined system environment.

7. The method according to claim 1, characterized in that, Based on the tool invocation scheme, running the at least one target tool in the predetermined system environment further includes: In response to the failure to match the target tool, business logic regarding the input request is determined based on the user's original intent and contextual information about the input request; For the aforementioned business logic, based on the tools in the pre-registered lightweight toolset using the predetermined large language model, a new target tool and its corresponding executable calling code are written; and Run the newly written target tool in the predetermined system environment.

8. The method according to claim 1, characterized in that, Also includes: Based on the user's original intent and contextual information about the input request, time keywords and their types are obtained, including precise time keywords, relative time keywords, fuzzy time keywords, and / or financial statement time keywords; A time identification scheme is proposed based on the type matching of the aforementioned time keywords; as well as Based on the matched time recognition scheme, the standardized target time parameter adapted to the business scenario corresponding to the input request is obtained by parsing, and the standardized target time parameter is substituted into the tool invocation scheme.

9. The method according to claim 8, characterized in that, The time identification scheme for type matching of the aforementioned time keywords includes one or more of the following: For precise time keywords, perform standardized format parsing; For relative time keywords, the conversion is performed based on the current system time; For fuzzy time keywords, perform differentiated semantic mapping based on the corresponding business scenario; as well as Search and escape keywords related to financial report time periods, combined with the corresponding entities.

10. A computing device, comprising: At least one processing unit; At least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions, when executed by the at least one processing unit, causing the device to perform the steps of the method according to any one of claims 1 to 9.

11. A computer-readable storage medium having a computer program stored thereon, the computer program implementing the method according to any one of claims 1 to 9 when executed by a machine.

Images