Tool calling method based on code form

Abstract

The invention discloses a tool calling method based on a code form, and the method comprises the steps: 1, packaging the use of a tool into a function form, and formulating a calling document according to the function form; 2, compiling a code logic fragment called by a tool; 3, introducing an error feedback mechanism based on feedback of the instant compiler; and 4, calling the next tool by utilizing the tool response content prompt. According to the method, a code interaction mechanism is constructed, a task is disassembled into a structured code instruction, a code analysis and dynamic execution module is combined, deep fusion of the task planning capacity and the tool function execution capacity of a large language model is achieved, and the tool calling precision and the parameter optimization efficiency are remarkably improved. The framework supports integration of multiple types of tools including a data analysis tool, a professional software interface and an automatic script, and system robustness in a complex task scene is enhanced through a standardized interface protocol and a result feedback mechanism.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and natural language processing technology, and in particular relates to a tool invocation method based on code. Background Technology

[0002] In practical applications of Large Language Models (LLMs), tool calls have become an important means of extending model capabilities and completing complex tasks. Current mainstream solutions mostly adopt a JSON-based structured call format to achieve functions such as data processing and API interaction.

[0003] However, existing JSON tool calling mechanisms still have the following limitations in practical applications: 1. Low inference efficiency in request-intensive tasks: When the model needs to frequently call the same tool, the traditional JSON calling method, which only supports passing parameters once each time, requires repeated interactions, resulting in a long inference chain and rapid accumulation of context. This not only reduces interaction efficiency but also easily leads to illusions and performance degradation caused by long contexts. 2. Truncation of excessively long response information: The return content of JSON calls is transmitted in plain text form. If the tool response content is long (such as structured tables or long text results), the model receiving end often needs to truncate the content. This fixed-length truncation mechanism can easily cause the loss of key fields or core information, affecting the accuracy of subsequent inference and task execution. Summary of the Invention

[0004] The purpose of this invention is to solve the problems of low inference efficiency and truncation of extremely long response information in request-intensive tasks, and to propose a tool invocation method based on code.

[0005] Tool invocation methods based on code, including:

[0006] Step 1: Encapsulate the use of the tool into functions and create a calling document accordingly;

[0007] Step 2: Write the code logic snippet for the tool call;

[0008] Step 3: Introduce an error feedback mechanism based on feedback from the just-in-time compiler;

[0009] Step 4: Use the tool response prompts to invoke the next tool.

[0010] The significant advancement of this invention compared to existing technologies lies in:

[0011] 1. Security and accuracy assurance mechanism: The framework performs strict syntax verification and security review on the calling code generated by the language model to prevent risks such as tool call failure, permission overreach or user information leakage caused by code logic errors or potential security vulnerabilities, and ensure the stability and reliability of the entire calling process.

[0012] 2. Efficient inference for request-intensive instructions: For scenarios that require frequent calls to the same tool, the framework supports encapsulating multiple sets of call parameters into a list structure and generating batch call code at once through a loop structure, thereby significantly reducing the number of interaction rounds and context length of the model, improving inference efficiency and reducing token consumption.

[0013] 3. Caching mechanism for ultra-long response content: For ultra-long response content returned by the tool, the framework uses the natural caching mechanism of variables to cache the content in the intermediate storage area and allows the model to request specified fragments on demand, thereby effectively avoiding the problem of loss of key information due to truncation and improving the integrity and utilization efficiency of the response.

[0014] 4. High versatility and scalability: The framework has excellent generalization capabilities, supporting rapid adaptation to various heterogeneous tools and API interfaces. Integrating new tools requires no significant modification to the underlying logic; only interface descriptions or examples are needed to achieve smooth integration and effective invocation of new tools.

[0015] In summary, this invention constructs an efficient, secure, and universal tool invocation framework for large language models by converting natural language instructions into executable code and combining code execution feedback for intelligent correction and security control. This significantly improves the execution capability and practical value of language models in complex tasks.

[0016] To more clearly illustrate the functional characteristics and structural parameters of the present invention, further explanation is provided below in conjunction with the accompanying drawings and specific embodiments. Attached Figure Description

[0017] Figure 1 This is a flowchart of the method provided in the embodiments of this application;

[0018] Figure 2 This is a pseudocode illustration provided in an embodiment of this application;

[0019] Figure 3 These are comparative diagrams provided in the embodiments of this application. Detailed Implementation

[0020] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] To gain a deeper understanding of this invention, the specific embodiments of the invention will be described in further detail below with reference to the accompanying drawings and specific steps. The following embodiments are used to illustrate the invention and are not intended to limit the scope of the invention. Rather, these embodiments are provided to enable a more thorough and comprehensive understanding of the disclosure of this application. This embodiment is a tool invocation framework based on code, such as... Figure 1 As shown, the method includes the following steps:

[0022] Step 1: Encapsulate the use of the tool into functions and create a calling document accordingly.

[0023] First, it's essential to fully understand the tool's functions and usage. For tools used in different scenarios, such as data processing tools, weather query tools, and network request tools, clearly define the input parameters, output results, and specific operations to be performed. For example, if you want to encapsulate a weather query tool, you need to clarify that its input parameters may include geographical location (such as city name or latitude and longitude) and query time (such as current or future date), while the output results may include weather information at the corresponding time point, such as temperature, probability of rainfall, wind speed, etc.

[0024] Next, based on the analysis of the tool's functionality, a function interface adapted for weather queries is defined. The function's input parameters should correspond to the query conditions required by the tool, and the return value should cover complete weather information or provide data in a structured form for subsequent use by the model. Finally, the specific logic for calling the weather query tool is encapsulated within the function. This may involve steps such as requesting the weather service API, constructing the query URL, setting request parameters, and parsing the returned results. Simultaneously, it is necessary to handle potential exceptions to ensure the function's robustness. For example, when requesting weather information, if the specified city is invalid or the network request fails, the relevant exceptions should be caught, and reasonable error messages or default values ​​should be returned to ensure the overall process is controllable and stable.

[0025] As a crucial component of system design, the tool invocation documentation must clearly define the interaction methods between the Language Large Model (LLM) and external tools. This documentation not only provides technical reference for developers but also conveys the tool's invocation logic, parameter specifications, and return processing mechanisms to the LLM in a structured, machine-readable format. The tool invocation documentation includes the following sections:

[0026] 1. Document Overview: At the beginning of the document, provide an overview of the tool usage, including the tool's name, a brief description of its functions, and applicable scenarios;

[0027] For example, a document overview for a weather query tool could be written as follows: This tool is used to query weather information for a specified city at a specific time. It is suitable for intelligent task scenarios that require weather condition awareness, such as itinerary planning, event scheduling, and environmental awareness-driven behavior selection.

[0028] 2. Function Description: Provide a detailed description of each encapsulated function, including the function name, functional description, parameter list, and return value description;

[0029] Taking the query_weather function as an example, the documentation can be described as follows:

[0030] Function name: query_weather

[0031] Function Description: Queries the weather conditions for a specified city on a specified date.

[0032] Parameter list:

[0033] city: String type, the name of the target city to query (supports Chinese and English).

[0034] date: String type, query date (format is "YYYY-MM-DD", such as "2025-06-01").

[0035] Return value description:

[0036] When the query is successful, a dictionary structure containing information such as weather status, maximum / minimum temperature, and wind speed is returned.

[0037] When a query fails, return an error message or None, and log the exception for troubleshooting.

[0038] 3. Sample Code: Provides sample code for function calls, allowing users to quickly understand how to use the tool.

[0039] Step 2: Write code snippets for tool calls

[0040] In the code-based tool invocation framework described in this invention, the invocation logic generated by the language model no longer adopts the traditional structured JSON or function signature format, but instead directly describes the tool invocation in the form of complete executable code snippets. This approach not only enhances the expressiveness and flexibility of the model, but also inherently possesses the structural advantage of handling request-intensive tasks.

[0041] Specifically, if a user wants to query the average temperature of a location for the next week, the traditional structured JSON tool call method has significant limitations. Since each call can only pass a single parameter, the model has to repeatedly call the same tool, leading to a significant increase in interaction frequency. This not only severely impacts efficiency but also easily causes a rapid accumulation of context length, resulting in problems such as model illusion, information loss, and decreased inference efficiency. In contrast, the code-based tool call method allows the language model to directly construct call code containing loop structures (such as for statements) to automate batch requests for multi-day temperature information. Instead of the model progressively expanding all call instructions, it generates a complete logical fragment at once, enabling the model to schedule the tool to execute a series of operations within a unified context and complete data integration and post-processing locally. For example, to query the temperature for the next seven days, the model can directly generate... Figure 2 pseudocode:

[0042] This code snippet, through simple loops and variable maintenance, completes seven tool calls within a single interaction and summarizes the final result by averaging. Compared to traditional multi-turn interaction methods, code-based calls offer stronger abstraction and composability, reducing the number of request rounds and token consumption, while also mitigating the risk of interference from intermediate context states, effectively improving the stability and robustness of complex task processing. Furthermore, this approach facilitates subsequent task expansion (such as weighted averaging and logical judgments), providing the language model with greater control space and inference flexibility.

[0043] In summary, the technical advantages of code-based tool invocation methods are reflected in the following aspects:

[0044] 1. No explicit design required, naturally supports batch processing capabilities: Since the code itself has the semantic characteristics of expressing loops and flow control, the language model can directly realize multi-parameter calls to the same tool through natural generation, without the need for additional modules or hints of structural constraints.

[0045] 2. Reduce model interaction rounds and prompt redundancy: Multiple requests are constructed through a single code call, avoiding the overhead of independent interaction required for each call in traditional tool calling methods, effectively compressing context length and inference path.

[0046] 3. Reduce token consumption and context pollution: The function template and parameter list only need to be passed in once in the loop structure, which replaces the repetitive structure description required by multiple function calls, significantly reducing token usage and model memory load.

[0047] 4. Enhance the universality and generalization of task expression: This mechanism can be naturally transferred to other similar "multi-objective batch processing" tasks, such as multi-file operations, multi-data cleaning, multi-object scoring, etc., and has good cross-task adaptability.

[0048] Step 3: Introduce an error feedback mechanism based on feedback from the just-in-time compiler.

[0049] To further improve the execution correctness and system robustness of the tool invocation code, this implementation method performs real-time testing on the generated code based on a just-in-time compiler or static code analysis tool before the language model generates and executes the code, and introduces an error feedback mechanism in conjunction with feedback information. This step includes the following sub-steps:

[0050] Step 31: Integrate the just-in-time compiler;

[0051] Integrate a code execution environment with just-in-time (JIT) compilation capabilities into the system, such as Python's `exec()` function with a syntax checking module, Node.js's JIT compilation mechanism, or other execution engines that can dynamically analyze code. The compiler can perform just-in-time compilation and semantic analysis on the code before execution, detecting potential problems such as syntax errors, parameter type mismatches, undefined variables, and function call exceptions, ensuring that the code conforms to execution specifications before officially calling external tools.

[0052] Step 32: Error feedback collection;

[0053] When the just-in-time compiler detects errors or potential risks in the code, the system will automatically collect error information, including: error types such as SyntaxError, TypeError, and NameError; the location of the error, including the line number and code segment; and error messages, including a detailed description of the exception. This feedback information will serve as an important basis to guide the language model in correcting and regenerating the original code.

[0054] Step 33: Classify and process different error types separately;

[0055] Based on the collected error feedback, the system categorizes the error messages by type and formulates corresponding handling strategies:

[0056] For syntax errors: Instruct the language model to correct code spelling, bracket pairing, and indentation standards;

[0057] For parameter errors: If the parameter type, number, or order is incorrect, the tool will prompt the model to adjust the parameter calling format according to the tool calling document specifications;

[0058] For call exceptions: calling an undefined function, misspelling a function name, or missing library imports, the system will automatically refer to the calling documentation to complete or correct the calling code;

[0059] For runtime errors, such as non-existent file paths or out-of-bounds indexes, the abnormal scenario will be logged and fed back to the language model to add exception handling logic or input validity validation.

[0060] Step 34: Perform automatic correction and regeneration;

[0061] Based on the categorized error feedback, two methods were used for correction:

[0062] The first type is automatic correction: for simple errors, including common syntax errors, incorrect number of parameters, and misspelled function names, the errors are directly corrected and then submitted to the just-in-time compiler for verification.

[0063] The second method is guided regeneration: For complex errors or multiple problems, a prompttrefinement will be automatically generated to guide the language model to regenerate the calling code that conforms to the specification based on the original task requirements and error feedback;

[0064] Step 35: Multiple rounds of feedback iteration;

[0065] When there are multiple complex errors or dependent context adjustments, a multi-round feedback iteration mechanism will be used to repeatedly execute the process of "compiler detection - error collection - classification and processing - correction / regeneration" until the real-time compiler confirms that the code has passed all syntax and calling specification checks before entering the subsequent actual tool call execution stage.

[0066] Through the above mechanism, the execution correctness of the code generated by the language model can be significantly improved, the probability of task failure due to calling errors can be reduced, and the stability and reliability of the system can be enhanced in complex and ever-changing environments.

[0067] Step 4: Use the tool response prompts to invoke the next tool;

[0068] In practical applications, complex tasks often involve cascading calls of multiple tools, where the parameters of tool B depend on the response of tool A. Therefore, in the tool invocation process driven by a Large Language Model (LLM), the output of previous tools must be accurately passed to subsequent tools to ensure the correctness of the call chain and the continuity of the task flow. Compared to the traditional JSON tool invocation mechanism, the code-based tool invocation framework proposed in this invention has inherent advantages in handling tool response results and supporting cascading calls of multiple tools.

[0069] Specifically, in the tool invocation process in code form, the tool's response results are automatically parsed and stored in the execution environment as pre-assigned structured variables. These response results are stored in a JSON-like data structure. When the tool is invoked, the model assigns these data to predefined variables or fields through code logic. This means that regardless of whether the response content is a string, a number, or a more complex nested structure, the model can directly access and use this data through standard programming logic such as field references, conditional judgments, and loop iterations.

[0070] It's worth noting that even if the tool's response data is returned in JSON format, truncation may still occur in some extreme cases if the response is very long, especially when the response exceeds the model's context limits. However, unlike traditional structured JSON tool calls where truncation causes direct information loss, code-based tool calls store response data in a cache and ensure data integrity through runtime code retrieval mechanisms. Even if the prompt is partially truncated in some cases, the entire response data can still be accessed through fields and fully referenced and used in subsequent logic. In this way, the response content is not just confined to the model-generated prompt text but is persisted in the execution environment, enabling data to flow and be used across calls and steps, avoiding information loss or inconsistency.

[0071] This mechanism ensures that all information returned by the tool is fully utilized at each step, regardless of whether the response is truncated or partially missing, making it particularly suitable for complex, dynamic, and interactive tasks.

[0072] Therefore, for responses with different lengths, the following response processing strategy is adopted:

[0073] 1. For tools with short response content, directly embed the complete response into the model hint context for the model to explicitly parse;

[0074] 2. For tools with excessively long response content, the hints only include partial fields or structural fragments, which are provided to the model as "structural and semantic references"; the complete data of the response results are still stored in local variables, which are dynamically obtained by the model when the code logic is called during subsequent code execution; for example, the model can generate a for loop to iterate through a list of fields in the response, or make conditional judgments based on the content of a specific field. Even if the specific data under that field is not expanded in the context, its true value can still be accessed from the cache and used for inference.

[0075] In contrast, traditional JSON call mechanisms require response content to be returned as text and embedded in the language model context. Once the context limit is exceeded, the response will be truncated, resulting in the loss of important fields. This affects the calling logic and judgment conditions of subsequent tools, and in severe cases, it can even interrupt the entire task chain.

Claims

1. A tool invocation method based on code, characterized in that, The method includes: Step 1: Encapsulate the use of the tool into functions and create a calling document accordingly; Step 2: Write the code logic snippet for the tool call; Step 3: Introduce an error feedback mechanism based on feedback from the just-in-time compiler; Step 4: Use the tool response prompts to invoke the next tool.

2. The method according to claim 1, characterized in that, Step 1: Encapsulate the use of the tool into functions, and create a calling document accordingly, including: For tools used in different scenarios, clearly define the input parameters, output results, and specific operations to be performed; Based on the analysis of the tool's functions, define an adapted function interface; the function's input parameters should correspond to the query conditions required by the tool, and the return value should cover complete information or provide data in a structured form for easy use by the model later; The tool call documentation includes the following sections: Document Overview: The document begins with an overview of the tool usage, including the tool's name, a brief description of its functions, and applicable scenarios. Function description: Provides a detailed description of each encapsulated function, including the function name, functional description, parameter list, and return value description; Example code: Provides example code for function calls to help users quickly understand how to use the tool.

3. The method according to claim 2, characterized in that, Step 2: Write the code logic snippet for the tool call, including: Tool calls are described in the form of complete executable code snippets; the language model directly constructs call code containing loop structures to realize automated batch requests for multi-day temperature information; by generating complete logic snippets at once, the model can schedule tools to perform a series of operations in a unified context and complete data integration and post-processing locally; The loop structure only requires passing the function template and parameter list once, replacing the repetitive structure descriptions needed for multiple function calls.

4. The method according to claim 3, characterized in that, Step 3: Introduce an error feedback mechanism based on feedback from the just-in-time compiler, including: Step 31: Integrate the just-in-time compiler; The system integrates a code execution environment with just-in-time (JIT) compilation capabilities; the compiler can perform just-in-time compilation and semantic analysis on the code before execution, detecting potential problems such as syntax errors, parameter type mismatches, undefined variables, and function call exceptions. Step 32: Error feedback collection; When the just-in-time compiler detects errors or potential risks in the code, the system will automatically collect error information, including: error types such as SyntaxError, TypeError, and NameError; the location of the error, including the line number and code segment; and error message information, including a detailed description of the exception. Step 33: Classify and process different error types separately; Based on the collected error feedback, the system categorizes the error messages by type and formulates corresponding handling strategies: For syntax errors: Instruct the language model to correct code spelling, bracket pairing, and indentation standards; For parameter errors: If the parameter type, number, or order is incorrect, the tool will prompt the model to adjust the parameter calling format according to the tool calling document specifications; For call exceptions: calling an undefined function, misspelling a function name, or missing library imports, the system will automatically refer to the calling documentation to complete or correct the calling code; For runtime errors, such as non-existent file paths or out-of-bounds indexes, the abnormal scenario will be logged and fed back to the language model to add exception handling logic or input validity validation. Step 34: Perform automatic correction and regeneration; Based on the categorized error feedback, two methods were used for correction: The first type is automatic correction: for simple errors, including common syntax errors, incorrect number of parameters, and misspelled function names, the errors are directly corrected and then submitted to the just-in-time compiler for verification. The second method is guided regeneration: For complex errors or multiple problems, rewrite prompts will be automatically generated to guide the language model to regenerate calling code that conforms to the specifications based on the original task requirements and error feedback; Step 35: Multiple rounds of feedback iteration; When there are multiple complex errors or dependent context adjustments, a multi-round feedback iteration mechanism will be used to repeatedly execute the "compiler detection - error collection - classification and processing - correction / regeneration" process until the just-in-time compiler confirms that the code has passed all syntax and calling specification checks, and then it will enter the subsequent actual tool call execution stage.

5. The method according to claim 4, characterized in that, Step 4: Use the tool's response prompts to invoke the next tool; including: The output of previous tools is accurately passed to subsequent tools to ensure the correctness of the call chain and the continuity of the task flow; In the code-based tool invocation process, the tool's response results are automatically parsed and stored in the execution environment as pre-assigned structured variables. These response results are stored in a JSON-like data structure. When the tool is invoked, the model assigns these data to predefined variables or fields through code logic. The code-based tool invocation stores the response data in a cache and ensures data integrity through the runtime reading mechanism of the code. Even if the prompt is truncated in some cases, the entire response data can still be fully referenced and used in subsequent logic through field access. The response content does not only remain in the prompt text generated by the model but is persisted in the execution environment, enabling data to flow and be used across calls and steps, avoiding information loss or inconsistency. This mechanism ensures that all information returned by the tool is fully utilized at each step. For responses with different lengths, the following response processing strategy is adopted: For tools with short response content, the complete response is directly embedded in the model hint context for the model to parse explicitly; For tools with excessively long response content, the prompts only include partial fields or structural fragments, which are provided to the model as "structural and semantic references"; the complete data of the response results are still stored in local variables, which are dynamically obtained by the model calling the code logic during subsequent code execution.

Images