Parameter invocation method for model, and computing device, electronic device and storage medium
By using the Hierarchical Tool Error Checklist (HiTEC) framework, combined with global and local checklists, the problem of inaccurate parameter filling when calling external tools in large language models is solved, achieving efficient and accurate parameter calling, and improving system stability and user experience.
Patent Information
- Application Number
- PCT/CN2026/084106
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2025-05-12
- Filing Date
- 2026-03-17
- Publication Date
- 2026-11-19
AI Technical Summary
When calling external tools or APIs, large language models suffer from inaccurate parameter filling, resulting in a high failure rate. Existing methods consume a lot of resources and have unstable feedback mechanisms, making it difficult to cover the specific needs of most tools.
The Hierarchical Error Checklist (HiTEC) framework is adopted, which combines global and local checklists. The parameters are initially verified through the global checklist and then deeply corrected through the local checklist to ensure that the calling parameters comply with general and specific rules.
It significantly improves the accuracy and robustness of model parameter calls, reduces the call failure rate, and enhances system efficiency and user experience.
Smart Images

Figure CN2026084106_19112026_PF_FP_ABST
Abstract
Description
The model's parameter calling methods, computing devices, electronic devices, and storage media. Technical Field
[0001] This disclosure relates to large model technology and the field of parameter calling, specifically to a method for calling parameters of a model, a computing device, an electronic device, and a storage medium. Background Technology
[0002] Large Language Models (LLMs) require calls to external tools or Application Programming Interfaces (APIs) to perform specific tasks, but inaccurate parameter filling is a major obstacle. Traditional methods such as rule-based hints and simulated feedback can alleviate this to some extent, but they are resource-intensive, have unstable feedback mechanisms, and struggle to cover the specific needs of most tools. This results in a high failure rate and limits the model's usability.
[0003] There is currently no effective solution to the above problems. Summary of the Invention
[0004] This disclosure provides a method for calling parameters of a model, a computing device, an electronic device, and a storage medium to at least solve the technical problem of low accuracy in calling model parameters.
[0005] According to one aspect of the present disclosure, a parameter generation method is provided, comprising: in response to receiving input data of a target model, inputting the input data and a global detection list into the target model, generating initial calling parameters using the target model, wherein the initial calling parameters are used to represent calling parameters for at least one calling object, and the global detection list is used to represent a list for error correction of calling parameters for any calling object; retrieving a local detection list corresponding to at least one calling object, wherein the local detection list is used to represent a list for error correction of calling parameters for at least one calling object; inputting the initial calling parameters and the local detection list into the target model, and performing error correction on the initial calling parameters using the target model to obtain target calling parameters.
[0006] According to one aspect of the present disclosure, a parameter generation method is provided, comprising: responding to an input instruction applied to an operation interface, displaying input data of a target model and a global detection list on the operation interface, wherein the global detection list is used to represent a list of error corrections performed on the call parameters of any call object; responding to a processing instruction applied to the operation interface, displaying target call parameters on the operation interface, wherein the target call parameters are obtained by inputting initial call parameters and a local detection list into the target model and performing error corrections on the initial call parameters using the target model, and the local detection list is a local detection list of at least one call object, wherein the initial call parameters are generated by inputting input data and the global detection list into the target model and performing error corrections on the call parameters of at least one call object, and the initial call parameters are used to represent call parameters for calling at least one call object, and the local detection list is used to represent a list of error corrections performed on the call parameters of at least one call object.
[0007] According to one aspect of the present disclosure, a parameter generation method is provided, comprising: acquiring input data and a global detection list by calling a first interface, wherein the first interface includes a first parameter, and the parameter value of the first parameter includes the input data and the global detection list; inputting the input data and the global detection list into a target model, and generating initial calling parameters using the target model, wherein the initial calling parameters are used to represent calling parameters for calling at least one calling object, and the global detection list is used to represent a list for error correction of calling parameters for any calling object; retrieving a local detection list corresponding to at least one calling object, wherein the local detection list is used to represent a list for error correction of calling parameters for at least one calling object; inputting the initial calling parameters and the local detection list into the target model, and performing error correction on the initial calling parameters using the target model to obtain target calling parameters; and outputting the target calling parameters by calling a second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter includes the target calling parameters.
[0008] According to another aspect of the present disclosure, a computing device is also provided, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods in various embodiments of the present disclosure when it runs.
[0009] According to another aspect of the present disclosure, an electronic device is also provided, including: a memory storing an executable program; and a processor connected to the memory via a bus for running the program, wherein the program executes the methods in various embodiments of the present disclosure during runtime.
[0010] According to another aspect of the embodiments of the present disclosure, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored executable program, wherein, when the executable program is executed, it controls the device where the computer-readable storage medium is located to perform the methods of the various embodiments of the present disclosure.
[0011] According to another aspect of the embodiments of this disclosure, a computer program product is also provided, including a computer program that, when executed by a processor, implements the methods of various embodiments of this disclosure.
[0012] According to another aspect of the embodiments of this disclosure, a computer program product is also provided, including a non-volatile computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods in various embodiments of this disclosure.
[0013] According to another aspect of the embodiments of this disclosure, a computer program is also provided, which, when executed by a processor, implements the methods of the various embodiments of this disclosure.
[0014] In this embodiment of the disclosure, in response to receiving input data from the target model, the input data and a global detection list are input into the target model. Initial calling parameters are generated using the target model, wherein the initial calling parameters represent calling parameters for at least one calling object, and the global detection list represents a list for error correction of calling parameters for any calling object. Local detection lists corresponding to at least one calling object are retrieved, wherein the local detection lists represent a list for error correction of calling parameters for at least one calling object. The initial calling parameters and the local detection lists are input into the target model, and the target model is used to perform error correction on the initial calling parameters to obtain the target calling parameters, thereby improving the accuracy of parameter calling generation in the model. It is worth noting that, due to the adoption of… By employing a combined global and local error checking and correction strategy, the target model can ensure the correctness of call parameters from both a broad global perspective and a specific local perspective when generating call parameters. This directly improves the quality of call parameters. When generating call parameters, the target model can avoid common errors at the global level while refining the parameter details of specific call objects at the local level. This ensures that the call can be checked and improved based on a comprehensive error knowledge base. This comprehensive and progressively refined error correction strategy improves the quality of generated call parameters and fundamentally solves the technical problem of low call accuracy caused by insufficient error correction in the parameter calling process of previous models. It improves the accuracy of model parameter calling and thus solves the technical problem of low accuracy of model parameter calling.
[0015] It is worth noting that the above general description and the following detailed description are merely for illustrative and explanatory purposes and do not constitute a limitation thereof. Attached Figure Description
[0016] The accompanying drawings, which are included to provide a further understanding of this disclosure and form part of this disclosure, illustrate exemplary embodiments of the present disclosure and are used to explain the disclosure, but do not constitute an undue limitation of the disclosure. In the drawings:
[0017] Figure 1 is a schematic diagram of a model application scenario according to an embodiment of the present disclosure;
[0018] Figure 2 is a flowchart of a parameter generation method according to an embodiment of the present disclosure;
[0019] Figure 3 is a schematic diagram of the error checking process under the corresponding framework of a hierarchical tool error checking table according to an embodiment of the present disclosure;
[0020] Figure 4 is a flowchart of a parameter generation method according to an embodiment of the present disclosure;
[0021] Figure 5 is a flowchart of a parameter generation method according to an embodiment of the present disclosure;
[0022] Figure 6 is a schematic diagram of a parameter generation apparatus according to an embodiment of the present disclosure;
[0023] Figure 7 is a schematic diagram of a parameter generation apparatus according to an embodiment of the present disclosure;
[0024] Figure 8 is a schematic diagram of a parameter generation apparatus according to an embodiment of the present disclosure;
[0025] Figure 9 is a flowchart of a parameter generation method according to an embodiment of the present disclosure;
[0026] Figure 10 is a structural block diagram of a computing device according to an embodiment of the present disclosure;
[0027] Figure 11 is a structural block diagram of an electronic device according to an embodiment of the present disclosure. Detailed Implementation
[0028] To enable those skilled in the art to better understand the present disclosure, the technical solutions of the present disclosure will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present disclosure, and not all embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present disclosure.
[0029] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0030] The technical solution disclosed herein is primarily implemented using large-scale model technology. Here, "large-scale model" refers to a deep learning model with a massive number of parameters, typically containing hundreds of millions, tens of billions, hundreds of billions, trillions, or even tens of trillions of parameters. Large-scale models, also known as foundation models, are pre-trained using large-scale unlabeled corpora to produce pre-trained models with hundreds of millions of parameters. These models are adaptable to a wide range of downstream tasks and exhibit good generalization ability. Examples include Large Language Models (LLMs) and multi-modal pre-training models.
[0031] It's important to note that in practical applications, large models can be fine-tuned using a small number of samples after pre-training, allowing them to be applied to various tasks. For example, large models can be widely used in Natural Language Processing (NLP), computer vision, and speech processing. Specifically, they can be applied to computer vision tasks such as Visual Question Answering (VQA), Image Captioning (IC), and Image Generation, as well as NLP tasks such as text-based sentiment classification, text summarization, and machine translation. Therefore, the main application scenarios for large models include, but are not limited to, digital assistants, intelligent robots, search, online education, office software, e-commerce, and intelligent design.
[0032] First, some nouns or terms that appear in the description of the embodiments of this disclosure shall be interpreted as follows:
[0033] Large Language Model (LLM) is a deep learning model with a large number of parameters. It is mainly used to handle natural language understanding and generation tasks. It can learn from massive amounts of text data to perform various natural language processing tasks such as language translation, text generation, and question answering.
[0034] Hierarchical Tool Error Checklist (HiTEC) is a framework for diagnosing and correcting tool call errors. It divides the error list into two levels: global and local. It is used to comprehensively identify and correct parameter filling errors that may occur when large language models call tools or application programming interfaces (APIs), thereby improving the accuracy and robustness of the calls.
[0035] In-Context Learning (ICL), in Natural Language Processing, models generate output directly by understanding the context of the input, without requiring additional training or fine-tuning. In the HiTEC framework, ICL is used to embed an error list directly into prompts, correcting errors on the spot.
[0036] Model improvement methods (Kahneman-Tversky Optimization, or KTO for short) are model improvement methods based on behavioral economics and psychology theories. They use model error instances as negative samples and fine-tune them through preference improvement methods to enhance the model's performance when faced with errors or non-ideal inputs, thereby improving its adaptability and stability.
[0037] Direct Preference Optimization (DPO) is a method for evaluating and improving model output. It improves model performance by directly comparing multiple candidate outputs generated by the model and selecting the better option for learning. This method is particularly suitable for improving generative models, effectively guiding the model to learn preferences and improving output quality.
[0038] Currently, large language models face parameter filling errors during tool invocation, which severely impacts their accuracy and reliability. Existing tool learning methods typically rely on extensive real-world interactions to improve invocation accuracy, which is not only resource-intensive but can also lead to instability issues. Furthermore, most tool invocation errors are common and predictable types.
[0039] Currently, in research on the interaction between large language models and external tools, common approaches mainly rely on the following strategies:
[0040] Prompt engineering based on rules or a few examples guides the model to call tools through preset prompts or a few interactive examples, but this method can only cover some common errors and is difficult to adapt to the diversity of tools and changes in context.
[0041] The feedback mechanism that simulates real tool calls continuously adjusts the call parameters by simulating execution and obtaining actual feedback. Although it can gradually improve accuracy, it has problems such as high resource consumption, high call costs, and unstable feedback delays.
[0042] Single error correction strategies, with some methods focusing on missing parameters or format errors, fail to simultaneously address both globally common errors and tool-specific errors, resulting in insufficient error correction capabilities when facing complex calling scenarios.
[0043] Therefore, existing solutions have significant limitations in terms of the comprehensiveness of error identification, robustness of invocation, and resource efficiency, and there is an urgent need for a systematic and hierarchical error diagnosis and correction method.
[0044] To address the aforementioned issues, this disclosure proposes a novel framework, the Hierarchical Tool Error Checklist (HiTEC), which improves the accuracy of tool invocation by reducing reliance on real-world interactions through a systematic error diagnosis and correction mechanism.
[0045] According to embodiments of this disclosure, a method for calling parameters of a model is provided. It should be noted that the steps shown in the flowcharts in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowcharts, in some cases, the steps shown or described may be executed in a different order than that shown here.
[0046] Considering the large number of model parameters in large models and the limited computing resources of mobile terminals, the method provided in this disclosure can be applied to the application scenario shown in Figure 1, but is not limited thereto. Figure 1 is a schematic diagram of a model application scenario according to an embodiment of this disclosure. In the application scenario shown in Figure 1, the large model is deployed on server 10. Server 10 can connect to one or more client devices 20 through a local area network (LAN), a wide area network (WAN), an internet connection, or other types of data networks. The client devices 20 may include, but are not limited to, smartphones, tablets, laptops, PDAs, personal computers, smart home devices, and in-vehicle devices. The client devices 20 can interact with users through a graphical user interface to call the large model, thereby implementing the method provided in this disclosure.
[0047] In this embodiment of the disclosure, the system consisting of a client device and a server can perform the following steps: The client device generates input data. The server inputs the input data and a global detection list into the target model, and uses the target model to generate initial call parameters; retrieves a local detection list corresponding to at least one calling object; inputs the initial call parameters and the local detection list into the target model, and uses the target model to perform error correction on the initial call parameters to obtain the target call parameters.
[0048] It should be noted that with the rapid development of high-performance computing units, the methods provided in this disclosure can also be applied to integrated model machines in other application scenarios. In one optional embodiment, the integrated model machine has multiple built-in models. Users can select one model to adjust as needed to obtain their own model. The high-performance computing unit built into the integrated model machine can then directly call the adjusted model to execute the methods provided in this disclosure. In another optional embodiment, the large integrated model machine has a pre-trained model built-in. Therefore, the high-performance computing unit built into the integrated model machine can directly call this model to execute the methods provided in this disclosure.
[0049] Furthermore, when users need to train their own models, they can upload their own datasets via the client. These datasets are then sent to the server, allowing the server to adjust the pre-trained model using the dataset to obtain the user's customized model, which can then be deployed to the production environment. To facilitate users' model adjustment needs, the server provides complete adjustment tools, development frameworks, and processes, supporting multiple adjustment strategies. This allows the adjusted model to better adapt to different application domains and achieve a high degree of customization.
[0050] Under the above operating environment, this disclosure provides a parameter generation method as shown in Figure 2. Figure 2 is a flowchart of the parameter generation method according to an embodiment of this disclosure. As shown in Figure 2, the method may include the following steps:
[0051] Step S202: In response to receiving input data from the target model, input data and global detection list are input into the target model, and initial calling parameters are generated using the target model.
[0052] The initial call parameters are used to represent the call parameters for calling at least one call object, and the global check list is used to represent the list of call parameters for error correction of any call object.
[0053] The input data mentioned above refers to the instructions sent by the user or system to the target model, requesting the model to perform a specific task or obtain certain information. Input data is information provided to the target model by the user or system, usually expressed in natural language, used to instruct the model to perform a specific task or invoke a tool. Input data is the initial information that triggers the target model to generate invocation parameters. It contains user intent and needs, and the model generates invocation parameters based on this information.
[0054] The target model described above is used to generate the call parameters required for interaction with external tools or services, and based on these parameters, invokes the relevant processing tools to output the desired content. The target model understands natural language instructions and generates corresponding call parameters for interaction with external tools or services. It receives user instructions (input data) and error lists (global and local detection lists), and through analysis and learning, generates the parameters required to invoke external tools. The model's intelligence ensures that the generated call parameters conform to both general rules and the requirements of specific tools.
[0055] The initial call parameters described above are the initial version generated by the target model based on the input data and the global detection list. Although these parameters have undergone preliminary verification, they may still contain some object-specific errors and require further correction. Call parameters are a set of parameters generated by the target model for interacting with the call object, containing specific information required by external tools or services to perform a particular task. Call parameters act as a bridge between the model and the call object; accurate call parameters ensure correct task execution and prevent call failures.
[0056] The aforementioned calling object refers to a tool, service, or interface external to the target model, possessing specific functions capable of receiving and processing calling parameters and executing the operations required by the user. The calling object is the receiver of the calling parameters generated by the target model; the parameters generated by the model must conform to the calling object's calling format and rules to ensure successful calling. The aforementioned "any calling object" can be all calling objects, or one or more arbitrarily selected calling objects. There is no specific limitation on the term "any calling object" here; it can be a calling object determined based on actual needs.
[0057] The aforementioned global checklist is a set of general error-checking rules applicable to different calling objects. It includes basic error types that are prone to occur during the generation of call parameters, such as incorrect parameter format and missing required parameters. The global checklist serves as guidance in the early stages of generating call parameters, ensuring the basic correctness of the parameters and avoiding common errors. The global checklist can be a rule database, containing general checkpoints such as "verify the correct date format" and "check the existence of required parameters," but it is not limited to these. This is merely an example and not a specific limitation.
[0058] When the target model receives input data from the user, it considers both the input data and the contents of the global checklist to generate initial call parameters. The global checklist acts as a guide, serving as a general checklist containing numerous basic error types related to parameter generation, such as incorrect parameter formatting, missing necessary parameters, and parameter values outside the acceptable range. By combining this with the global checklist early in the generation process, the model can proactively avoid these common errors when generating call parameters, laying a solid foundation.
[0059] Step S204: Retrieve at least one local detection list corresponding to the calling object.
[0060] The local detection list is used to represent a list of call parameters for error correction of at least one calling object.
[0061] The aforementioned calling objects refer to the external systems, services, or tools that the target model needs to call, such as weather information interfaces, map service interfaces, etc. Each of them has a unique set of parameter rules and calling logic.
[0062] Compared to the global checklist, the aforementioned local checklist focuses more on the detailed requirements of specific calling objects. It includes validation standards for unique error types and calling rules for each calling object. This checklist ensures that calling parameters not only meet basic format requirements, but more importantly, it guarantees that the parameters conform to the specific functional requirements and usage specifications of the calling object.
[0063] A local checklist is a set of error-checking rules for a specific calling object. It includes the object's unique parameter requirements and error types, such as the accuracy of parameter names and the range of parameter values. In the later stages of generating calling parameters, the local checklist performs more detailed checks and corrections on the parameters to ensure they meet the requirements of the specific calling object. The local checklist can be a configuration file that details the specific parameter rules and error patterns for each calling object. For example, the "location" parameter must be the full city name, not an abbreviation or shortened form; this is just an example and not a specific limitation.
[0064] The aforementioned partial detection list can be used to detect one-to-one with the calling object, or it can be used to detect multiple calling objects.
[0065] In one optional embodiment, a corresponding local checklist can be invoked for the object to be invoked. This checklist is not only an error checklist but also a precise capture of the specific needs of the invoking object. Compared to a global checklist, a local checklist focuses more on identifying and correcting errors specific to a particular tool or service, such as the accurate spelling of parameter names, the valid range of parameter values, and the combined dependencies of certain parameters. Through this meticulous check, the system can ensure that the generated invoking parameters meet both general standards and the personalized requirements of the specific invoking object, thereby improving the accuracy and success rate of the invoking.
[0066] By invoking a local checklist, unique errors of specific call objects that are not covered by the global checklist can be identified and corrected. This process ensures that the call parameters are not only correct in format and completeness, but also closely match the functional requirements of the call object. Ultimately, this two-tiered error correction strategy significantly improves the quality of model-generated call parameters, reduces the call failure rate, and enhances the stability and efficiency of interaction with external tools.
[0067] Step S206: Input the initial call parameters and local detection list into the target model, and use the target model to perform error correction on the initial call parameters to obtain the target call parameters.
[0068] The target call parameters mentioned above are the final, accurate set of call parameters after global and local error correction. They are used to effectively interact with specific call objects and ensure that external tools or services are called correctly.
[0069] After the target model generates initial calling parameters based on the global detection list, the system then inputs these initial calling parameters along with the corresponding local detection lists back into the target model. The purpose of this round of input and processing is to perform a second stage of error correction, namely, a deep check and correction of the calling rules for a specific calling object. Based on the detailed rules in the local detection lists, the target model identifies and corrects any errors that do not conform to the specific requirements of the calling object, ultimately producing a set of target calling parameters that have undergone both global and local correction.
[0070] The local checklist includes error types and calling rules related to specific calling objects. By performing secondary verification on the initial calling parameters, it can be ensured that these parameters meet the specific needs of the calling object. This process not only enhances the accuracy of the model's generated calling parameters but also enables the model to self-correct when facing complex calling scenarios by introducing negative sample generation and preference-based fine-tuning methods.
[0071] Through the steps described above, a deep improvement was achieved in the calling parameters, ensuring that while meeting general format requirements, the personalized rules of specific calling objects were also strictly followed. This dual-correction mechanism significantly improves the robustness of the model's interaction with external tools, reduces call failures caused by parameter errors, thereby improving calling efficiency and enhancing the overall system performance and user experience.
[0072] For example, suppose the target model needs to call a translation API for a user to translate a piece of Chinese text into English. First, the model generates preliminary call parameters using a global checklist, including "source_language" (source language) and "target_language" (target language). Then, a local checklist is created corresponding to the translation API. This checklist may specifically emphasize that the "source_language" and "target_language" parameters need to be standard language codes, not the full names or abbreviations of the languages. Furthermore, the checklist may require that the "source_text" parameter cannot contain special characters to avoid call failure. The target model processes these initial call parameters along with the local checklist, identifying that the "source_language" and "target_language" parameters use the full names of the languages (e.g., "Chinese" and "English"), while the standard codes (should be "zh" and "en") are correct. The model will automatically correct these errors, while also checking the "source_text" parameter for any special characters and filtering or replacing them accordingly. Through this process, the final generated target call parameters conform to the calling specifications of the translation API, enabling successful API calls and high-quality text translation tasks. This achieves accurate tool calls, improving work efficiency and user experience. This example fully demonstrates the benefits of deep correction through a partial checklist, ensuring the comprehensiveness and accuracy of call parameters, avoiding common errors in tool calls, and improving the overall stability and performance of the system.
[0073] First, a global checklist is used to perform an initial check on the call parameters, eliminating most basic errors and laying a solid foundation for subsequent calls. Then, by calling the local checklists corresponding to each tool, the checks are further refined to ensure that the parameters not only conform to general rules but also meet the needs of the specific caller. This layered error checking and correction mechanism significantly improves the accuracy of call parameters and the success rate of calls, reduces call failures caused by incorrect parameters, and thus improves the efficiency of the entire interaction process and the user experience.
[0074] In this embodiment of the disclosure, in response to receiving input data from the target model, the input data and a global detection list are input into the target model. Initial calling parameters are generated using the target model, wherein the initial calling parameters represent calling parameters for at least one calling object, and the global detection list represents a list for error correction of calling parameters for any calling object. Local detection lists corresponding to at least one calling object are retrieved, wherein the local detection lists represent a list for error correction of calling parameters for at least one calling object. The initial calling parameters and the local detection lists are input into the target model, and the target model is used to perform error correction on the initial calling parameters to obtain the target calling parameters, thereby improving the accuracy of parameter calling generation in the model. It is worth noting that, due to the adoption of… By employing a combined global and local error checking and correction strategy, the target model can ensure the correctness of call parameters from both a broad global perspective and a specific local perspective when generating call parameters. This directly improves the quality of call parameters. When generating call parameters, the target model can avoid common errors at the global level while refining the parameter details of specific call objects at the local level. This ensures that the call can be checked and improved based on a comprehensive error knowledge base. This comprehensive adjustment and progressively refined error correction strategy improves the quality of generated call parameters and fundamentally solves the technical problem of low call accuracy caused by insufficient error correction in the parameter calling process of previous models. It improves the accuracy of model parameter calling and thus solves the technical problem of low accuracy of model parameter calling.
[0075] In the above embodiments of this disclosure, input data and a global detection list are input into the target model, and the target model is used to generate initial call parameters. This includes: performing intent recognition on the input data to obtain intent call information, wherein the intent call information is used to indicate the call object type of the target model; and using the global detection list to guide the target model to generate initial call parameters according to the intent call information, wherein the guidance is used to indicate that the target model avoids the error information listed in the global detection list during the process of generating initial call parameters.
[0076] The intent recognition process described above is the core process by which the target model analyzes input data to understand the user's intent or request in order to determine what type of service or tool to invoke. The intent invocation information mentioned above is the result of the intent recognition process, indicating the type of object the target model will invoke, such as a weather query service or a map navigation tool.
[0077] The above guidance refers to the process by which the target model follows the rules of the global checklist when generating call parameters, avoiding the error messages listed in the checklist.
[0078] First, the system receives the user's natural language command (input data), and the target model identifies the user's intent to determine the type of operation the user wants to perform (the type of object to invoke). Then, based on this intent information, the system guides the target model through a global checklist, enabling it to proactively avoid common errors listed in the checklist when generating invocation parameters, such as incorrect parameter formats, missing required fields, and incorrectly spelled parameter names. This process ensures that even in complex queries or scenarios requiring the invocation of multiple tools, the initial invocation parameters generated by the model are generally correct and conform to invocation specifications.
[0079] By using a global checklist to guide the model in generating initial call parameters, not only is the relevance and accuracy of parameter generation enhanced, but the model's adaptability and robustness in complex calling scenarios are also improved. This strategy reduces the workload of error correction in subsequent stages, saves system resources, and lowers the call failure rate caused by parameter errors, ultimately improving user satisfaction and overall system efficiency.
[0080] In the above embodiments of this disclosure, initial calling parameters and a local detection list are input into the target model, and the target model is used to correct errors in the initial calling parameters to obtain target calling parameters. This includes: inputting initial calling parameters and a local detection list into the target model; using the target model to identify errors in the initial calling parameters to obtain error information in the initial calling parameters; generating correction information corresponding to the error information based on the local detection list; and correcting errors in the initial calling parameters based on the correction information to obtain target calling parameters.
[0081] The aforementioned error identification is used to indicate that in the second interaction, the target model reviews the initially generated call parameters to identify whether there are any errors that do not conform to the rules in the local detection list. It should be noted that the target model can be used to perform error identification on the initial call parameters to obtain the identification result. This result indicates whether there are any errors in the initial call parameters. If errors are found, the subsequent step of generating correction information corresponding to the errors based on the local detection list is executed. If no errors are found, the initial call parameters are determined to be the target call parameters.
[0082] The above correction information is based on the description and guidance of error types in the local detection list, and the specific guidance information generated by the system to correct errors in the initial call parameters, such as suggested correct parameter values and parameter formats.
[0083] The system inputs the initial call parameters along with a local checklist into the target model to further correct errors in the parameters. First, the model performs deep error detection based on the local checklist, checking for errors specific to the calling object, such as misspelled parameter names or out-of-range parameter values. Once an error is detected, the system generates corresponding correction information based on the local checklist. This correction information includes specific guidance on how to correct the error, such as how to adjust parameter values or formats. Subsequently, the model uses this correction information to precisely correct errors in the initial call parameters, ultimately obtaining target call parameters that meet the requirements of the calling object.
[0084] The above process combines the model's intelligent recognition capabilities with the standardization of the local detection list. Through multiple rounds of interaction and correction, it ensures high accuracy of the calling parameters and a high success rate of the call. Compared to relying solely on once-generated calling parameters, this two-stage detection and correction mechanism can more comprehensively cover various types of errors, significantly improving the system's adaptability and robustness in complex query and multi-tool calling scenarios.
[0085] By introducing a local detection list and a two-round interaction mechanism, the accuracy and relevance of the model's generated call parameters are effectively enhanced. In practice, it significantly reduces call failures caused by parameter errors, improves user satisfaction and product intelligence, and reduces reliance on expensive real-world interactions, thus saving costs. For the interaction of large language models with external tools, this approach is a breakthrough, improving the system's performance in complex environments and enhancing the model's self-learning and self-correcting capabilities.
[0086] In the embodiments disclosed above, the method further includes: obtaining a first call error type of the target model, wherein the first call error type represents the error type generated by the target model during the call to any calling object; constructing a structured description template corresponding to the first call error type, wherein the structured description template describes the call error type using a structured language; extracting error prevention conditions corresponding to the first call error type, wherein the error prevention conditions represent the conditions that the target model needs to meet during parameter generation; and generating a global detection list based on the first call error type, the structured description template, and the error prevention conditions.
[0087] The first call error type mentioned above refers to the specific error types that the target model may encounter when calling any tool or application interface. These error types reflect common reasons for call failure, such as misspelled parameter names, missing required parameters, mismatched parameter types, and non-standard formats.
[0088] The structured description template mentioned above is a standardized document or data format used to describe and record the specific characteristics of the first call error type, such as the context in which the error occurred, the specific manifestation of the error, and the recommended solution, so as to facilitate model understanding and learning.
[0089] The aforementioned error prevention conditions are a series of conditions or rules that need to be met during the parameter generation process in order to avoid specific error types. These conditions guide the model on how to generate parameters to avoid errors.
[0090] First, the system collects and analyzes the error types encountered by the target model when calling different tools or APIs. These errors may stem from defects in the model's own parameter generation or from a mismatch between the parameters and the requirements of the calling object. For each error type, the system constructs a corresponding structured description template. These templates use structured language to record the characteristics of the error in detail, such as the context in which the error occurred, the specific manifestation of the error, the potential impact of the error, and suggested preventive measures. From the structured description templates, the system further extracts the conditions that the model must follow when generating call parameters to prevent errors from occurring. Finally, by integrating the first call error type, the structured description templates, and the error prevention conditions, a global checklist is generated. This checklist contains all error types that the model should avoid when generating call parameters and their preventive conditions, serving as a guide for the model to generate and validate call parameters.
[0091] By systematically identifying and recording common error types in the parameter generation process of the target model, a rule framework is constructed to guide the model in avoiding these errors. A global checklist, as a product of this process, provides the model with a comprehensive guide to error verification and prevention, ensuring high-quality generation of call parameters, reducing the possibility of call failures, and thus improving the efficiency and robustness of interaction with external tools.
[0092] First, the primary error types were identified from historical call data, such as misspelled parameter names and missing required parameters. Then, a structured description template was built to record the characteristics of each error type in detail, providing a clear path for the model to learn and understand errors. The extraction of error prevention conditions set hard rules for the model when generating call parameters, preventing the recurrence of errors. Finally, the generation of a global checklist integrated all information, becoming an important reference in the model's call parameter generation process and ensuring the accuracy of the calls.
[0093] The global checklist generated through the above steps significantly enhances the relevance and accuracy of the model's generated call parameters. The model not only avoids common formatting errors and missing required fields, but also prevents more complex and specific error types, such as misspelled parameter names and valid ranges of parameter values. This not only improves the efficiency and success rate of model interaction with external tools, but also reduces reliance on expensive real-world interactions, saving costs. In tool calls, this method provides a systematic and hierarchical approach to error diagnosis and correction, improving the overall system's intelligence level.
[0094] It should be noted that the global and local detection lists mentioned above can be created manually, through models, systems, etc. There are no restrictions on the methods used to create the global and local detection lists.
[0095] In the above embodiments of this disclosure, obtaining the first call error type of the target model includes: obtaining historical error data of the target model, wherein the historical error data is used to represent error data generated during the historical call process of the target model; classifying the historical error data to obtain an initial call error type, and outputting the initial call error type; receiving feedback information on the initial call error type, and adjusting the initial call error type based on the feedback information to obtain the first call error type.
[0096] The aforementioned historical error data records failures of the target model in the past when it interacted with external tools or APIs due to parameter generation errors. This data contains detailed information and context of the errors.
[0097] The aforementioned initial call error types are a set of error types derived from preliminary analysis and classification of historical error data, which have not yet been adjusted based on expert or system feedback.
[0098] The feedback information mentioned above is a revised opinion on the classification of initial call error types from expert evaluation or automatic system analysis, which is used to more accurately define the error types and make them more consistent with the actual call situation.
[0099] The system collects records of failures caused by parameter generation errors when the target model calls different tools or services. This data may come from logs, user feedback, or automated test reports. Historical error data is analyzed to identify and categorize common reasons for call failures, forming initial call error types. Classification may be based on the frequency and type of errors (e.g., missing parameters, format errors) and the specific requirements of the tool call. The initially categorized call error types are presented to experts or reviewed by automated systems, collecting feedback, including suggestions for correcting the classifications, newly discovered error types, or the removal of existing error types. Based on the received feedback, the initial call error types are adjusted, classification errors are corrected, and error types are added or removed, ultimately forming a more accurate set of initial call error types.
[0100] Obtaining the first call error type is a dynamic process based on the model's historical error records. It aims to continuously refine the error type definition through analysis and feedback, thereby better guiding the model to avoid these errors in future calls. This process combines data analysis with intelligent feedback from humans or the system, forming an iterative loop that continuously improves the model's call success rate and robustness.
[0101] The target model has accumulated a number of error cases during historical calls, such as incorrect parameter names and incorrect parameter formats. By collecting and classifying this historical error data, we can initially identify the typical error types encountered by the model during the call process. These initially classified error types are then submitted to the technical team or reviewed through an automated system to gather feedback and make necessary corrections. For example, it may be discovered that some error types are improperly categorized, or that newly emerging error types are not covered. Finally, the adjusted error types will be integrated into the global checklist within the HiTEC framework to guide the generation of future call parameters and prevent recurrence.
[0102] By systematically collecting and analyzing historical error data, and combining this with a feedback mechanism for iterative improvement of error type definitions, the accuracy and robustness of the model in generating call parameters are enhanced. Specifically, it enables the model to more comprehensively understand the errors that are prone to occur during past calls and proactively avoid them in subsequent calls, thereby significantly reducing the possibility of call failures, improving the model's intelligence level and user experience, and also saving the additional costs incurred due to call errors.
[0103] It should be noted that after receiving feedback information on the initial call error type, if there is a need for adjustment, the initial call error type can be adjusted according to the feedback information; if there is no need for adjustment, the initial call error type can be directly determined as the first call error type.
[0104] In the embodiments disclosed above, the method further includes: obtaining object attributes of at least one calling object and a second calling error type of the target model, wherein the second calling error type is used to represent different error types generated during the process of the target model calling at least one calling object; constructing an example sample library of the second calling error type, wherein the example sample library contains a first error example sample of the second calling error type and a first correct example sample of the second calling error type; generating an error description corresponding to the second calling error type and obtaining error correction guidance for the second calling error type, wherein the error correction guidance is used to guide the target model to correct errors in the calling parameters; and generating a local detection list based on object attributes, the second calling error type, the example sample library, the error description, and the error correction guidance.
[0105] The aforementioned object attributes define the characteristics and parameter requirements of the calling object, including but not limited to functional descriptions, input parameter formats, and output data types, and form the basis for constructing a local detection list.
[0106] The second type of call error mentioned above refers to specific error types encountered by the target model when calling a specific tool or service. Compared to the first type of call error, these errors are more specific and closely related to the attributes of the specific calling object.
[0107] The example sample library mentioned above is a database containing correct and incorrect call examples, used to train or guide models to understand the correct way to call a particular tool and common calling errors.
[0108] The first erroneous example sample and the first correct example sample mentioned above are crucial components in the process of building the local detection list. They are mainly used to train and guide the model to identify and correct error types specific to the calling object.
[0109] The first set of error examples mentioned above refers to instances of call failures caused by errors in the parameter generation process when invoking a specific tool or service. These examples are typically compiled from errors recorded in the model's historical call logs, or they may be intentionally designed to demonstrate the specific manifestations of particular call error types (such as misspelled parameter names, missing required parameters, etc.). By analyzing these error examples, the model can learn the specific error patterns that may be encountered during parameter generation, thereby enabling it to more effectively identify and avoid these errors in future calls.
[0110] The first correct example sample mentioned above, in contrast to the first incorrect example sample, refers to the result example where the parameters are correctly generated and the call is successful under the same calling scenario. These examples are also based on historical calling data or designed manually, and can accurately follow the parameter requirements and format specifications of the calling object. They are the goals that the model should pursue when generating parameters. By comparing and learning from correct and incorrect example samples, the model can gradually master the method of generating parameters that meet the requirements of the calling object, thereby improving the success rate of the call.
[0111] For example, the target model is learning how to correctly call a translation API that requires parameters including "source_text" (source text) and "target_language" (target language). During the construction of the local detection inventory: the first set of error sample samples might contain examples of parameters the model has previously generated, such as misspelling "source_text" as "source_textt" or setting the "target_language" parameter to invalid code (e.g., "zhn" instead of "zh"). These error sample samples reflect the specific types of errors the model might encounter in parameter generation. Similarly, some examples of correctly generated parameters and successful API calls will be collected or designed. For example, a correct call parameter might be {"source_text":"Hello","target_language":"zh"}, where the naming and format of "source_text" and "target_language" conform to the API requirements; the model should learn this correct parameter generation pattern.
[0112] By comparing and learning from these correct and incorrect example samples, the model can more accurately identify and correct specific error types during parameter generation, such as misspelled parameter names or incorrect parameter values, thereby improving the accuracy and success rate of calling translation APIs. This example-based learning method is a crucial step in building and maintaining a local checklist. It ensures that the model can specifically avoid known error types when dealing with specific call objects, improving its intelligent calling capabilities and robustness.
[0113] First, analyze the relevant documentation of the invoked objects to extract their object attributes and potential invocation limitations. Simultaneously, collect the secondary invocation error types generated by the target model when invoking these objects. Based on these secondary invocation error types, construct a sample library containing both error and correct examples to guide the model in identifying and correcting errors. Write detailed error descriptions for different secondary invocation error types and formulate guidelines for correcting these errors. Integrate object attributes, secondary invocation error types, sample library, error descriptions, and correction guidelines to generate a localized checklist specific to the invoked object.
[0114] The construction of local detection lists is tailored to specific tools and scenarios, based on a deep understanding of the tools (e.g., through studying relevant documentation) and analysis of past model call errors. For example, if the target model frequently encounters errors such as incorrectly formatted date parameters when calling a weather API, the local detection list will not only include the object attribute information of the weather API, but also collect and label examples of these errors (first error example samples) and examples of how to correctly input date parameters (first correct example samples). Combined with error descriptions and correction guidance, the final generated local detection list will guide the model to generate call parameters more accurately when calling the weather API, reducing the probability of errors.
[0115] By constructing a localized detection checklist containing a sample library of second-invocation error types, error descriptions, and correction guidelines, the target model can more accurately identify and correct errors when generating invocation parameters. This has a significant positive effect on improving the accuracy of model interaction with external tools and enhancing the user experience. Specifically, it avoids common tool-specific errors in model parameter generation, reduces the number of invocation failures, and thus improves efficiency and robustness.
[0116] It should be noted that, depending on actual needs, one or more of the following can be selected to generate a partial detection list: object attributes, second-invocation error type, sample library, error description, and error correction guidelines. Alternatively, a partial detection list can be generated directly based on object attributes, second-invocation error type, sample library, error description, and error correction guidelines.
[0117] In the embodiments disclosed above, the method further includes: acquiring an input sample; constructing a second erroneous example sample and a second correct example sample corresponding to the input sample based on a global detection list and / or a local detection list; inputting the input sample into a target model and generating sample calling parameters using the target model; constructing a loss function based on the sample calling parameters, the second erroneous example sample, and the second correct example sample; and updating the model parameters of the target model using the loss function.
[0118] The input samples mentioned above are example data used to verify and improve model performance during model training. They can be user queries, calling commands, etc., used to simulate actual calling scenarios.
[0119] The aforementioned second incorrect example sample and second correct example sample are derived from the input sample and, in combination with the global detection list and / or local detection list, form a set of examples constructed by the system for training the model to identify and distinguish between correct and incorrect call parameters.
[0120] The sample call parameters mentioned above are parameters generated by the target model after receiving the input sample, used to call tools or services.
[0121] The loss function mentioned above can be the Kahneman-Tversky Optimization (KTO) loss function, which measures the difference between the call parameters generated by the model and the correct samples, and guides the adjustment of model parameters to improve call accuracy.
[0122] Obtain input samples representing various invocation requests the model might encounter. Based on the global and / or local detection lists, construct corresponding correct and incorrect invocation parameter examples (a second correct example sample and a second incorrect example sample) for each input sample. Input the input samples into the target model to generate sample invocation parameters. Compare the model-generated sample invocation parameters with the constructed second correct example samples, calculate the loss, and construct the KTO loss function. Use the KTO loss function to guide the update of model parameters, improving the model's accuracy and robustness in invocation parameter generation.
[0123] By constructing and utilizing examples of correct and incorrect invocation parameters, the target model is trained and improved, enabling it to generate invocation parameters more accurately when encountering similar inputs. The KTO loss function plays a central role in this process, measuring the gap between the model's generated parameters and the correct parameters, driving adjustments to the model's parameters, and thus gradually improving the model's accuracy in invoking tools.
[0124] By introducing the KTO loss function, the model's learning and self-correcting capabilities in generating call parameters are enhanced. Specifically, it enables the model to gradually reduce errors and improve call accuracy and overall system performance by updating parameters guided by the loss function based on specific call error types and scenarios. This approach not only improves the model's intelligence but also reduces resource waste caused by call failures, thereby increasing user satisfaction.
[0125] In the above embodiments of this disclosure, a loss function is constructed based on sample call parameters, a second erroneous example sample, and a second correct example sample, including: determining a first similarity between the sample call parameters and the second erroneous example sample, and determining a second similarity between the sample call parameters and the second correct example sample; determining a first product of the first similarity and a first weight, and determining a second product of the second similarity and the second weight, wherein the first weight is less than the second weight; and constructing a loss function based on the first product, the second product, and the training objective of the target model.
[0126] The first and second similarity scores mentioned above measure the degree of similarity between the sample call parameters generated by the target model and the second incorrect and second correct example samples, respectively. Similarity scores may be calculated based on various metrics, such as string similarity and parameter matching.
[0127] The first and second weights mentioned above are weight coefficients used to adjust the contribution of different components in the loss function. Typically, the second weight is greater than the first weight, meaning the system is more inclined to penalize deviations from correct example samples rather than similarities to incorrect example samples.
[0128] The similarity calculation described above is used to determine the first similarity between the sample call parameters and the incorrect example sample, and the second similarity with the correct example sample.
[0129] The aforementioned weighted adjustment involves applying a first weight and a second weight to weight the first similarity and the second similarity, forming a first product and a second product.
[0130] The loss function described above is designed based on the first product, the second product, and the training objective of the model. This loss function can guide the direction of model parameter updates, thereby improving the quality of parameter calls.
[0131] The aforementioned model parameter updates utilize a constructed loss function and a specific learning algorithm to update the internal parameters of the target model, thereby reducing the value of the loss function and improving the accuracy of parameter generation.
[0132] By quantifying the distance between the parameters generated by the model and correct / incorrect examples, and by weighting the contributions of these similarities, a loss function is constructed to guide the iterative updates of the model parameters. In this way, the model can distinguish between correct and incorrect call parameters during training, thereby reducing the probability of errors during actual calls and improving the overall success rate of tool calls.
[0133] To improve the accuracy and robustness of the target model's interaction with external tools, a negative sample generation and KTO improvement method are introduced by constructing global and local error lists and designing a two-stage feedback correction mechanism. This involves the construction and use of the loss function. The steps described above, within this context, specifically illustrate how to utilize second erroneous and second correct example samples to construct the loss function to guide model parameter updates. By assigning greater weight to correct examples, the system encourages the model to move towards generating correct parameters, while simultaneously penalizing proximity to erroneous examples to prevent the model from producing similar errors.
[0134] By constructing the loss function using the methods described above, the model can be effectively guided in parameter generation, reducing invocation errors and improving invocation accuracy and robustness. This loss function design strategy helps the model prioritize learning patterns extracted from correct examples while avoiding imitating incorrect invocation behaviors. This allows for rapid improvement of the model's tool invocation capabilities with limited training data, reducing model training costs and enhancing user satisfaction and experience.
[0135] This disclosure proposes a Hierarchical Tool Error Checklist (HiTEC), whose core idea is to construct a two-layer error list to achieve systematic diagnosis and efficient correction of errors during the invocation of large language model tools.
[0136] Construct a global error list and design a unified global error template for common errors in tool calls (such as incorrect tool name, missing required parameters, mismatched parameter types, non-standard formatting, redundant information, etc.). Embed the global error list into the user's initial query to guide the model to avoid common errors in advance when generating tool calls.
[0137] Build a local error list. For each specific tool, a detailed local error list can be designed based on its function description, parameter requirements, and use case. The local error list includes: correct query examples, erroneous call output, error messages, and improvement suggestions, ensuring that tool-specific errors are corrected in a targeted manner.
[0138] The dual-round interactive correction mechanism works as follows: In the first round, the model generates tool call results based on the initial prompts, which may contain some errors. In the second round, the initial output is reviewed and corrected using a local error list to ensure that the required parameters, parameter types, and formats all meet the requirements.
[0139] Two deployment methods are available: Hierarchical Tool Error Checklist with Instant Contextual Learning (HiTEC-ICL), which embeds global and local error lists directly into the prompts for immediate error correction without fine-tuning; and Hierarchical Tool Error Checklist with Deep Error Optimization (HiTEC-DEO), which generates high-quality negative samples from the error list and employs a preference-based deep fine-tuning method to further enhance the model's self-identification and error correction capabilities during tool calls, achieving a higher level of call accuracy.
[0140] Figure 3 is a schematic diagram of the error checking process under a hierarchical tool error checking table framework according to an embodiment of the present disclosure. As shown in Figure 3, the framework is constructed by two levels of error checking tables. The global error checking table is used to capture errors that most tools will encounter, while the local error checking table is used to focus on errors and contextual faults of specific tools.
[0141] As shown in Figure 3, for the global error checklist, historical tool call feedback can be performed first. Data collected from historical tool calls can be filtered and summarized to extract common error types and patterns, providing basic data for building the global error checklist. Based on the basic data, a unified error template can be designed to cover common errors encountered in tool calls, such as misidentification of tool names, omission of required parameters, and mismatched parameter types.
[0142] As shown in Figure 3, tool metadata, which includes the functional description, parameter requirements, and usage scenarios of specific tools, is a key source of information for constructing a local error checklist. It can be generated using LLM-based Local Error Checklist Generation. By leveraging the powerful understanding and generation capabilities of the LLM, the usage specifications, parameter requirements, and common errors of a specific tool are automatically analyzed and summarized, thus constructing a detailed error checklist for that tool. The LLM can output elements such as queries, tool calling outputs, error messages, and thought-of-errors. These elements together constitute the dynamic feedback and learning mechanism in the HiTEC framework. Queries drive the generation of tool calls; error messages in the tool calling outputs facilitate the construction and updating of the error checklist; and thought-of-errors help the model learn and improve itself. During the dynamic improvement process based on these elements, incorrect calling examples can be constructed. These examples serve as negative samples to guide model fine-tuning and improve error correction strategies. The entire process is a closed loop; through continuous feedback and adjustment, the model's accuracy and robustness in tool calling are improved. A local error checklist can be obtained from the example of the error call.
[0143] As shown in Figure 3, Hierarchical Error Checking Table (HiTEC) Enhanced Context Learning (Enhanced ICL) refers to the ability of the model to utilize contextual information to improve its performance when generating calling instructions. The role of Function Calling Negative Examples in the HiTEC framework is mainly in model training and performance improvement. When the calling instructions generated by the model do not match the expectations, policy-based fine-tuning (KTO-based Fine Tuning) can be used to enable the model to make more accurate decisions when encountering similar errors, thereby improving the model's performance and robustness.
[0144] This disclosure enables systematic error diagnosis, employing a hierarchical error list that separates common errors from tool-specific errors. It comprehensively covers various error scenarios, avoiding the omission of errors that occurred when relying solely on a single prompt or feedback mechanism. It covers both general and tool-specific errors.
[0145] The dual-round interactive feedback mechanism generates preliminary call results in the first round and makes targeted corrections using a local error list in the second round, making the call output more accurate and significantly reducing the risk of failure due to missing parameters or format errors.
[0146] HiTEC-KTO employs a negative sample generation and fine-tuning strategy. By constructing erroneous instances and performing preference-based fine-tuning, the model gains self-correcting capabilities when facing complex calling scenarios, while reducing reliance on expensive real-world interactive feedback.
[0147] With dynamic adaptability and efficient deployment, the global and local lists can be flexibly customized according to different tools and scenarios. It is suitable for immediate scenarios that do not require tuning (HiTEC-ICL) and can further improve performance through fine-tuning (HiTEC-KTO). Experiments on multiple public datasets show that this disclosure has higher parameter filling accuracy, and the overall tool invocation success rate and robustness are significantly better than traditional methods. This disclosure not only theoretically achieves comprehensive error identification and efficient correction, but also effectively reduces resource consumption and improves the model's tool invocation capability and overall system performance in practice.
[0148] A hierarchical error list mechanism is proposed, employing a two-tiered structure of a global error list and a local error list to systematically identify and correct various errors during tool invocation. A two-round interactive correction strategy is designed, employing a two-stage interactive process that combines initial output with subsequent feedback correction to achieve dynamic error correction, significantly improving the accuracy of tool invocation. A negative sample generation and KTO fine-tuning method utilizes the error list to generate high-quality negative samples and employs KTO for fine-tuning, enabling the model to proactively identify and correct errors when facing actual invocations. An efficient adaptation and deployment strategy provides a tuning-free instant error correction scheme (HiTEC-ICL) and a fine-tuning-based deep improvement scheme (HiTEC-KTO) to meet the tool invocation needs of different scenarios, achieving dual improvements in resources and performance.
[0149] The local error checklist contains the following components: tool information, a simulated query for each error type, function call output that can invoke the corresponding error, an error message describing the error, and an error reflection approach specifying how to correct such errors. The local error checklist is a fault prevention and identification system specifically designed for the unique functionality, parameter requirements, and relevant context of each tool. Its existence compensates for the limitations of the global error checklist in terms of coverage. While the latter provides an overview of common error types, it struggles to be comprehensive and detailed for the specific errors of each tool. The local checklist addresses this challenge, ensuring that errors, no matter how subtle or specific, can be detected and corrected promptly during tool invocation.
[0150] The tool information described above for each local error checklist begins with a thorough understanding of the tool itself, including but not limited to its functional description, parameter list, and calling syntax. This information forms an important reference when the model generates calling instructions, ensuring that the selection and use of tools meet the expected goals.
[0151] The aforementioned list of simulated queries and error outputs includes simulated queries for each error type, along with examples of function call outputs that can trigger these specific errors. These examples not only form the basis for error prevention but also provide the model with learning material for error identification, helping it understand how errors occur in different contexts.
[0152] The error messages and reflection suggestions described above provide a detailed description of each specific error, along with reflective ideas on how to correct it. This is a crucial channel for the model to receive feedback and make corrections after generating error calls, ensuring that every mistake can be transformed into an opportunity for self-improvement.
[0153] To construct such a highly customized local error checklist, preliminary tool research and information organization are crucial. This includes in-depth analysis of tool functions, precise understanding of parameter requirements, and a comprehensive understanding of the tool's invocation context. Appropriately integrating this information into the error checklist forms a complete system containing tool information, error triggering examples, error descriptions, and correction strategies.
[0154] The local error checklist is presented in a clear structure, where each error is recorded and categorized in detail. The model can learn and adjust based on this information to avoid repeating the same problems in future calls. This localized, granular error management strategy enhances the adaptability and accuracy of the large model during tool calls.
[0155] Local error checklists provide the model with tool-specific error prevention and correction guidelines through in-depth analysis and precise description of tool characteristics. They not only cover common tool errors but also highlight tool-specific differences, enabling the model to handle various specific errors more effectively in complex calling environments, thus improving the efficiency and quality of tool calls. Combining the comprehensiveness of global error checklists with the specificity of local checklists forms a robust error management framework.
[0156] The Global Error Checklist is designed to address the widespread and impactful errors encountered by large models when invoking tools. It focuses not only on specific queries or tools but also on common failure modes during tool invocation and prevalent issues related to tool selection and usage. The Global Error Checklist carefully selects eight key error types based on past model-tool interaction experience, representing typical obstacles encountered in tool interaction and aiming to cover various potential errors throughout the entire process from tool selection to parameter filling. However, it should be noted that the eight key error types listed here are merely illustrative; the specific error types and their number can be adjusted according to actual needs and are not limited here.
[0157] At the tool level, the list first considers tool name recognition errors (Error 0) and tool quantity errors (Error 7). These two error types reflect the confusion that the model may have when selecting or understanding tool names, as well as the inaccurate judgment of the number of tools during the invocation process. These problems directly affect the correct selection and invocation of tools.
[0158] The parameter-level error checking is more detailed, covering common issues such as missing parameters (Error 1), parameter type mismatch (Error 2), empty or inappropriate parameter values (Error 3), and redundant parameters (Error 4). These error types aim to ensure that the model can accurately fill in all required parameters, use the correct parameter types, avoid invalid or missing parameter values, and eliminate unnecessary redundant parameters when generating calling instructions, thereby improving the accuracy and efficiency of the call.
[0159] In addition, the global error checklist also focuses on the conciseness and relevance of the output information (Error 6), ensuring that the call results do not contain any redundant or irrelevant information, and preventing invalid tool call output formats (Error 5) to avoid problems such as inconsistent syntax. If these problems are not handled properly, they may interfere with subsequent data processing and analysis processes.
[0160] This comprehensive and detailed error list provides a solid foundation for the interaction between the large model and the tool, ensuring that the system can quickly identify and correct errors when encountering various calling scenarios, significantly improving the reliability and efficiency of model-tool interaction. This checklist is designed as a global error prevention mechanism, covering and reducing common errors in the tool calling process, ensuring the smooth operation of the entire calling flow, and ultimately improving user experience and overall system performance.
[0161] The global error checklist builds a more robust and efficient model-tool interaction system by systematically diagnosing and correcting common errors in tool calls. It not only includes accuracy checks on tool selection and naming but also delves into the details of parameter filling, ensuring the simplicity and correct formatting of the call output. It is a comprehensive error prevention and correction framework.
[0162] Local and global error checklists together construct a multi-layered error prevention system. The global checklist covers basic and broad error types, setting minimum call standards for the model; while the local checklists focus on discovering and correcting the unique errors of each tool, ensuring that the model can make more accurate and effective call decisions when facing specific tools. The two complement each other, jointly driving the robustness and accuracy of model-tool interactions to new heights.
[0163] This disclosure effectively improves the accuracy of interaction between large models and external tools through a hierarchical error list. In real-world projects, it reduces the risk of call failures caused by incorrect parameter selection and parameter name filling, thereby enhancing product intelligence and user experience while saving costs. By constructing global and local error lists, a two-stage feedback correction mechanism can be designed, along with the introduction of negative sample generation and related improvement methods. This is applicable to external large model API calls as well as the fine-tuning of private large models.
[0164] In the context learning (ICL) model, global and local error check tables can be integrated into the tool invocation process of large language models (LLMs) to ensure that tool invocations are not only efficient but also accurate. This enhances the effectiveness of tools by preventing and promptly diagnosing errors within the context of model-generated tool invocation instructions.
[0165] Embedded in the initial query, the global error check table is embedded into the inference process the moment the user initiates the query. This proactive integration helps the model avoid common invocation problems from the source, such as incorrect tool name recognition or omission of required parameters. By implementing error prevention at the beginning of the inference process, the system significantly improves the accuracy and reliability of tool invocation, reduces invocation failures caused by common errors, and provides users with a more stable and satisfactory service experience.
[0166] To further enhance the accuracy of tool invocation, a two-round interaction mechanism was designed for targeted correction using a local error checklist. In the first round, the model generates tool invocation instructions based on the initial query; in the second round, guided by the local error checklist, the initially generated invocation instructions undergo in-depth review and correction, particularly addressing parameter filling errors and tool-specific anomalies.
[0167] After the first round of calls, the local error checklist plays a crucial role in ensuring that each parameter meets the tool's requirements, correcting potential tool-specific errors, and thus improving the overall inference results.
[0168] In the Key-to-Optimize (KTO) model, error checking and correction are instantaneous, requiring no additional training or tuning cycles. This provides users with rapid response while ensuring the accuracy of tool calls. By combining global and local error lists, the entire approach not only prevents common errors but also addresses specific issues in tool calls, achieving comprehensive improvements from macro to micro levels.
[0169] In summary, the HiTEC-ICL strategy, by proactively integrating a global error check table into the context and utilizing a local error check table for targeted correction during the two rounds of interaction, successfully and significantly improves the accuracy and reliability of LLM tool calls without tuning. This approach is not only suitable for real-time task processing but also ensures the model's adaptability to complex tool call scenarios, greatly enhancing the tool's practical value and user experience.
[0170] For negative example generation, a negative example generation strategy can be adopted to further enhance the accuracy and stability of the large language model (LLM) in tool calls. This process cleverly constructs a series of negative examples targeting specific errors by combining correctly formatted tool call outputs with corresponding local error checking tables. These examples are designed to simulate real-world call failure scenarios.
[0171] Starting with standardized tool call examples and referencing the error types detailed in the local error checklist, including but not limited to incorrect parameter types, empty parameter values, and redundant parameters, these errors are introduced with specific prompts to create outputs highly similar to real-world failed calls. The negative examples generated in this way not only encompass all the characteristics of erroneous calls but also simulate various problems that might be encountered in actual tool call tasks, providing valuable training material for the model to identify and correct errors in complex situations.
[0172] We can pair negative examples with their corresponding correct tool call outputs to construct a dataset specifically for model fine-tuning, named the Paired Tool Call (PTC) dataset. Each pair of samples (correct and incorrect tool calls) in the PTC dataset is designed to improve the model's ability to identify, avoid, and correct errors. Fine-tuning the model using the PTC dataset aims to teach it how to avoid common errors when generating tool calls and how to effectively correct errors when they occur. The ultimate goal is to improve the model's reliability and consistency in tool calls, making its behavior more accurate and efficient.
[0173] Through this series of negative example generation and model fine-tuning, not only is the model's ability to recognize and handle errors enhanced, but its calling behavior is also improved. This ensures that when faced with complex tool calling scenarios, the model can execute tasks more confidently and accurately, thereby greatly improving user experience and overall system performance.
[0174] When faced with the PTC dataset, which consists of paired data, an intuitive fine-tuning option is to use the Direct Preference Improvement (DPO) method. DPO is an effective strategy to improve the model's preference decision-making ability in data comparisons. The difference between positive and negative samples in the PTC dataset often lies in only a few tokens. This causes the gradient of the DPO loss function to approach zero, and the vanishing gradient phenomenon limits the model's ability to learn from positive and negative sample comparisons, resulting in a weak update signal. During fine-tuning, the probability of the correct token tends to decrease, which, compared to the reference model (i.e., the initial model before fine-tuning), means that the model's accuracy in tool calls may decrease rather than increase.
[0175] To overcome the limitations of DPO on the PTC dataset, this disclosure introduces a bias improvement strategy (Kahneman-Tversky, or KTO for short). KTO rearranges the derivative of the loss function, enabling the model to learn more effectively from both positive and negative samples.
[0176] The method of using KTO to fine-tune large language models (LLMs) on PTC-type datasets can be called HiTEC-KTO. The KTO loss function can be defined in a paired format to specify how to improve between correct and incorrect tool call pairs, ensuring the model learns the correct tool calls more accurately and efficiently, while avoiding the limitations of the DPO method when dealing with PTC datasets. The core of this method is to construct a loss function that can distinguish between positive (correct tool calls) and negative (incorrect tool calls) samples to guide the model's fine-tuning process, making the model more inclined to generate correct tool calls while reducing the generation of incorrect ones.
[0177] This pairing format not only benefits the model's ability to identify and learn subtle differences (the nuanced distinction between correct and incorrect tool calls), but it is also particularly well-suited for handling PTC-type training data, where each sample consists of a pair of tool call sequences with subtle differences—one correct and the other incorrect. By applying the KTO loss function to the paired samples, the model can more sensitively capture these subtle differences, thereby progressively improving the accuracy and robustness of its tool calls during fine-tuning. The expression for the KTO loss function is as follows:
[0178] LKTO(x,yw,yl;θ)=λw-λwσ(β(rθ(x,yw)-z0))+λl-λlσ(β(z0-rθ(x,yl)));
[0179] In this model, LKTO(x,yw,yl;θ) measures the difference between the model's expectation and the expected value for correct tool calls yw and incorrect tool calls yl given input x, and θ is a constant. x represents the user input or query, which is the basic information received by the model. yw represents the correct tool call sequence, i.e., the target output. yl represents the incorrect tool call sequence. λw and λl are the weight coefficients of the loss terms for correct and incorrect calls, respectively, controlling the proportion of positive and negative feedback in the loss function. σ represents the set of model parameters, which is the improvement target. β is a sensitivity or slope parameter used to amplify or reduce the output of the function, affecting the sensitivity of the loss function to small differences. rθ(x,yw) represents the model's scoring function, used to evaluate the quality score of the model generating call yw given input x. rθ(x,yl) represents the model's scoring function, used to evaluate the quality score of the model generating call yl given input x. z0 is a benchmark or reference point used to distinguish the relative scores of good and bad calls. The model's goal is to push the score of correct calls above z0 and suppress the score of incorrect calls below z0.
[0180] The KTO loss function design described above aims to enhance the model's ability to identify correct calls while reducing its tendency to recognize incorrect calls. This is achieved by adjusting the weights of λw and λl, and by using the sensitivity parameter β to modulate the model's sensitivity, thus enabling more accurate judgments in tool call tasks. This design is suitable for handling subtle differences, such as those found in the PTC dataset, capturing and amplifying the nuances in how the model handles correct and incorrect calls, thereby providing clearer guidance during the improvement process.
[0181] In the HiTEC-KTO framework, gradient formulas are used to improve model parameters during the fine-tuning phase. This ensures the model can effectively distinguish between correct and incorrect tool calls, reducing failures caused by parameter errors or formatting inconsistencies. Through gradient improvements using KTO, the model learns better parameter configurations, not only reducing call errors but also exhibiting greater adaptability and robustness to new tools or parameter configurations, resulting in significant improvements in tool call accuracy and overall performance. The gradient formula is as follows:
[0182] in, This represents the model's loss contribution on the correct call sequence yw. The negative sign indicates that the model's goal is to maximize the generation probability of yw under conditional input x. The weighting coefficient aw ensures that the gradient contribution of correct calls is amplified according to their importance, helping the model focus more on learning the pattern of correct calls; This represents the contribution of the erroneous call sequence yl to the loss. In contrast to the case of correct calls, the goal here is to reduce the probability of the model generating erroneous call sequences. By introducing a weighted coefficient al, the impact of erroneous calls on model learning can be emphasized or weakened, guiding the model to avoid the pitfalls of erroneous calls.
[0183] The above formula guides the model during fine-tuning by updating the parameter θ to maximize the probability of generating correct tool invocation sequences while minimizing the probability of generating incorrect sequences. This design helps the model learn and correct errors in tool invocation more efficiently, improving the model's accuracy and robustness in real-world invocation tasks.
[0184] To further explore how the KTO mechanism drives the probability towards a better outcome during training, this disclosure reorganizes and analyzes the derivative of the KTO loss function. In this mechanism, each time the gradient of the loss function is calculated, the instructions in each tool call sequence are meticulously evaluated. The gradient calculation here is not merely a simple evaluation of the model's performance under the current parameter settings, but rather a specially designed two-stage feedback mechanism to achieve precise adjustment of the model parameters:
[0185] When the sequence generated by the model matches the correct call: the mechanism gives a negative feedback signal (which can be understood as a negative gradient), meaning that during training, the model needs to adjust its parameters to reduce the contribution of this match to the total loss. Essentially, this is an incentive mechanism, encouraging the model to increase the probability of generating tokens that match the correct call.
[0186] A positive feedback signal (positive gradient) is received when the sequence generated by the model matches an incorrect call, or when the generated token does not come from the correct tool call sequence. The existence of this positive gradient is to penalize the model for generating undesirable solutions, prompting it to avoid such errors in future generation, i.e., reducing the probability of generating mismatched or incorrect tokens.
[0187] This mechanism ensures that, while minimizing the KTO loss function, the model is incentivized to increase the logits of correct tool call tokens, while simultaneously reducing the logits of incorrect or irrelevant tokens. This means that when faced with tool call tasks, the model will be more inclined to generate the correct parameter combinations, effectively addressing the issue that the DPO method's model may not be sufficiently inclined to generate good tool call sequences.
[0188] Through this fine-tuning, KTO not only guides the model to better understand and execute correct invocation instructions, but also enhances its performance in various invocation scenarios, reducing failures caused by incorrect parameters or improper invocation, thereby improving the overall efficiency of the model and the user experience. The core of this mechanism lies in its ability to systematically diagnose and correct deviations in the model's invocation generation process. By learning from both positive and negative examples, it continuously improves model parameters, enabling the model to make more accurate decisions when generating tool invocation sequences, especially in tasks with strict requirements for parameter selection and formatting.
[0189] The gradient calculation of the KTO loss function promotes better results for the model in tool-invocation tasks by adjusting the logit value. This approach not only helps improve the model's ability to select correct parameters but also avoids generating inaccurate calls by reducing the logit value of incorrect parameters. This dynamic adjustment of parameter selection probabilities is crucial for improving the accuracy and efficiency of large models interacting with external tools, demonstrating the unique advantage of the KTO method in improving model behavior and making it more adaptable to tool-invocation scenarios.
[0190] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.
[0191] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this disclosure is not limited to the described order of actions, because according to this disclosure, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this disclosure.
[0192] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, they can also be implemented by hardware. Based on this understanding, the technical solutions of this disclosure, in essence, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of this disclosure.
[0193] Figure 4 is a flowchart of a parameter generation method according to an embodiment of the present disclosure. As shown in Figure 4, the method may include the following steps:
[0194] Step S402: In response to the input command applied to the operation interface, display the input data of the target model and the global detection list on the operation interface.
[0195] The global checklist is used to represent a list of error corrections performed on the call parameters of any calling object.
[0196] When a user issues an input command on the user interface, the system responds and displays the target model's input data and a global checklist on the interface. The global checklist is a pre-designed error correction checklist that covers common error types that may exist in the parameters of any invoked object, such as misspelled parameter names, missing parameters, and mismatched parameter types. This checklist provides a reference for users, helping them understand the rules the model must follow when generating invocation parameters. It also provides a framework for the model to refer to the checklist when generating parameters to proactively avoid these common errors, thereby improving the accuracy of parameter generation and the success rate of invocation.
[0197] Step S404: In response to the processing command applied to the operation interface, display the target call parameters on the operation interface.
[0198] The target call parameters are obtained by inputting the initial call parameters and the local detection list into the target model and then using the target model to correct the errors in the initial call parameters. The local detection list is a list of local detections for at least one call object. The initial call parameters are generated by inputting the input data and the global detection list into the target model and then using the target model. The initial call parameters are used to represent the call parameters for calling at least one call object, and the local detection list is used to represent the list of call parameters for error correction for at least one call object.
[0199] When a user issues a processing command on the user interface, the system responds and displays the target call parameters on the interface. These target call parameters are a set of parameters generated by the target model based on the user input command and the global detection list, used to call a specific service or tool. By directly displaying these parameters on the user interface, users can intuitively see the details of the parameters generated by the model, and thus determine whether the parameters meet expectations and whether further adjustments are needed. This process increases user participation, allowing for checking and adjusting the parameters after they are generated by the model, ensuring the accuracy and effectiveness of the call.
[0200] Through the above steps, in response to input commands applied to the operation interface, the input data of the target model and the global detection list are displayed on the operation interface. The global detection list represents a list of error corrections performed on the call parameters of any called object. In response to processing commands applied to the operation interface, the target call parameters are displayed on the operation interface. The target call parameters are obtained by inputting the initial call parameters and the local detection list into the target model and then performing error corrections on the initial call parameters using the target model. The local detection list is a list of local detections for at least one called object. The initial call parameters are generated by inputting the input data and the global detection list into the target model and then using the target model. The initial call parameters represent the call parameters used to invoke at least one called object, and the local detection list represents a list of error corrections performed on the call parameters of at least one called object. This approach improves the accuracy of parameter generation in the model. Notably, by employing a combined global and local error checking and correction strategy, the target model can ensure the correctness of parameters from both a broad global perspective and a specific local perspective when generating call parameters. This directly enhances the quality of the call parameters. The target model avoids common errors at the global level while refining the parameter details of specific call objects at the local level, ensuring that checks and improvements are made based on a comprehensive error knowledge base during the call process. This comprehensive adjustment and progressively refined error correction strategy improves the quality of generated call parameters, fundamentally solving the technical problem of low call accuracy caused by insufficient error correction in previous models. This improves the accuracy of parameter calls and thus addresses the overall technical issue of low parameter call accuracy in models.
[0201] Figure 5 is a flowchart of a parameter generation method according to an embodiment of the present disclosure. As shown in Figure 5, the method may include the following steps:
[0202] Step S502: Obtain input data and global detection list by calling the first interface.
[0203] The first interface includes a first parameter, the value of which includes input data and a global detection list.
[0204] The aforementioned first interface can be an interface for data interaction between the cloud server and the client. Input data and the global detection list can be passed to the interface function as the first parameter of the interface function to achieve the purpose of uploading the input data and the global detection list to the cloud server.
[0205] Step S504: Input the input data and global detection list into the target model, and use the target model to generate the initial call parameters.
[0206] The initial call parameters are used to represent the call parameters for calling at least one call object, and the global check list is used to represent the list of call parameters for error correction of any call object.
[0207] Step S506: Retrieve at least one local detection list corresponding to the calling object.
[0208] The local detection list is used to represent a list of call parameters for error correction of at least one calling object.
[0209] Step S508: Input the initial call parameters and local detection list into the target model, and use the target model to perform error correction on the initial call parameters to obtain the target call parameters.
[0210] Step S510: Output the target call parameters by calling the second interface.
[0211] The second interface includes a second parameter, the value of which includes the target call parameter.
[0212] The aforementioned second interface can be an interface for data interaction between the cloud server and the client. The cloud server can pass the target call parameters into the interface function as the second parameter of the interface function, thereby achieving the purpose of sending the target call parameters to the client.
[0213] Through the above steps, input data and a global detection list are obtained by calling a first interface, wherein the first interface includes a first parameter, the value of which includes the input data and the global detection list; the input data and the global detection list are input into the target model, and the target model is used to generate initial call parameters, wherein the initial call parameters represent call parameters for calling at least one call object, and the global detection list represents a list for error correction of call parameters for any call object; the local detection list corresponding to at least one call object is retrieved, wherein the local detection list represents a list for error correction of call parameters for at least one call object; the initial call parameters and the local detection list are input into the target model, and the target model is used to perform error correction on the initial call parameters to obtain the target call parameters; the target call parameters are output by calling a second interface, wherein the second interface includes a second parameter, the second... The parameter values include the target call parameters, which improves the accuracy of the model's parameter call generation. It's noteworthy that, due to the combined global and local error checking and correction strategy, the target model can ensure the correctness of the parameters from both a broad global perspective and a specific local perspective when generating call parameters. This directly improves the quality of the call parameters. When generating call parameters, the target model can avoid common errors at the global level while refining the parameter details of specific call objects at the local level, ensuring that the call is checked and improved based on a comprehensive error knowledge base. This comprehensive adjustment and progressively refined error correction strategy improves the quality of generated call parameters, fundamentally solving the technical problem of low call accuracy caused by insufficient error correction in previous models during parameter call processes. This improves the model's parameter call accuracy and thus solves the technical problem of low model parameter call accuracy.
[0214] According to an embodiment of this disclosure, a parameter generation device configured to implement the above-described parameter generation method is also provided. FIG6 is a schematic diagram of a parameter generation device according to an embodiment of this disclosure. As shown in FIG6, the 600 includes: a generation module 602, a retrieval module 604, and a correction module 606.
[0215] The generation module is configured to, in response to receiving input data from the target model, input the input data and a global detection list into the target model, and use the target model to generate initial invocation parameters. The initial invocation parameters are configured to represent invocation parameters for at least one invocation object, and the global detection list is configured to represent a list for error correction of invocation parameters for any invocation object. The retrieval module is configured to retrieve the local detection list corresponding to at least one invocation object, where the local detection list represents a list for error correction of invocation parameters for at least one invocation object. The correction module is configured to input the initial invocation parameters and the local detection list into the target model, and use the target model to perform error correction on the initial invocation parameters to obtain the target invocation parameters.
[0216] It should be noted that the generation module 602, retrieval module 604, and correction module 606 correspond to steps S202 to S206 in the above embodiments. The three modules and their corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the above embodiments. It should be noted that the above modules or units can be hardware or software components stored in memory and processed by one or more processors. The above modules can also run as part of the device in the server 10 provided in the above embodiments.
[0217] In the above embodiments of this disclosure, the generation module is configured to perform intent recognition on the input data to obtain intent invocation information, wherein the intent invocation information is configured to indicate the invocation object type of the target model; and to guide the target model to generate initial invocation parameters based on the intent invocation information using a global detection list, wherein the guidance is configured to indicate that the target model avoids the error information listed in the global detection list during the generation of initial invocation parameters.
[0218] In the above embodiments of this disclosure, the correction module is configured to input initial call parameters and a local detection list into the target model, use the target model to identify errors in the initial call parameters, obtain error information in the initial call parameters, generate correction information corresponding to the error information based on the local detection list, and perform error correction on the error information in the initial call parameters based on the correction information to obtain the target call parameters.
[0219] In the embodiments described above, the device is further configured to: acquire a first call error type of the target model, wherein the first call error type is configured to represent the error type generated by the target model during the call to any call object; construct a structured description template corresponding to the first call error type, wherein the structured description template describes the call error type using a structured language; extract error prevention conditions corresponding to the first call error type, wherein the error prevention conditions are configured to represent the conditions that the target model needs to meet during parameter generation; and generate a global detection list based on the first call error type, the structured description template, and the error prevention conditions.
[0220] In the above embodiments of this disclosure, the device is further configured to acquire historical error data of the target model, wherein the historical error data is configured to represent error data generated during the historical call process of the target model; classify the historical error data to obtain an initial call error type and output the initial call error type; receive feedback information on the initial call error type, and adjust the initial call error type based on the feedback information to obtain a first call error type.
[0221] In the embodiments described above, the device is further configured to: acquire object attributes of at least one calling object and a second calling error type of the target model, wherein the second calling error type is configured to represent different error types generated during the process of the target model calling at least one calling object; construct an example sample library of the second calling error type, wherein the example sample library contains a first error example sample of the second calling error type and a first correct example sample of the second calling error type; generate an error description corresponding to the second calling error type and acquire error correction guidance for the second calling error type, wherein the error correction guidance is configured to guide the target model to correct errors in the calling parameters; and generate a local detection list based on object attributes, the second calling error type, the example sample library, the error description, and the error correction guidance.
[0222] In the embodiments disclosed above, the device is further configured to: acquire input samples; construct a second erroneous example sample and a second correct example sample corresponding to the input sample based on a global detection list and / or a local detection list; input the input sample into a target model and generate sample calling parameters using the target model; construct a loss function based on the sample calling parameters, the second erroneous example sample, and the second correct example sample; and update the model parameters of the target model using the loss function.
[0223] In the embodiments described above, the device is further configured to determine a first similarity between a sample calling parameter and a second incorrect example sample, and to determine a second similarity between the sample calling parameter and a second correct example sample; to determine a first product of the first similarity and a first weight, and to determine a second product of the second similarity and a second weight, wherein the first weight is less than the second weight; and to construct a loss function based on the first product, the second product, and the training objective of the target model.
[0224] It should be noted that the preferred embodiments involved in the above embodiments of this disclosure are the same as the solutions, application scenarios and implementation processes provided in the above embodiments, but are not limited to the solutions provided in the above embodiments.
[0225] According to an embodiment of this disclosure, a parameter generation device configured to implement the above-described parameter generation method is also provided. FIG7 is a schematic diagram of a parameter generation device according to an embodiment of this disclosure. As shown in FIG7, the 700 includes: a first display module 702 and a second display module 704.
[0226] The first display module is configured to respond to input commands on the operation interface and display the input data of the target model and the global detection list on the operation interface. The global detection list is configured to represent a list for error correction of the call parameters of any calling object. The second display module is configured to respond to processing commands on the operation interface and display the target call parameters on the operation interface. The target call parameters are obtained by inputting the initial call parameters and the local detection list into the target model and using the target model to perform error correction on the initial call parameters. The local detection list is a local detection list of at least one calling object. The initial call parameters are generated by inputting the input data and the global detection list into the target model and using the target model. The initial call parameters are configured to represent the call parameters for calling at least one calling object. The local detection list is configured to represent a list for error correction of the call parameters of at least one calling object.
[0227] It should be noted that the first display module 702 and the second display module 704 mentioned above correspond to steps S402 to S406 in the above embodiments. The two modules and the corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the above embodiments. It should be noted that the above modules or units can be hardware components or software components stored in memory and processed by one or more processors. The above modules can also be part of the device and run in the server 10 provided in the above embodiments.
[0228] According to an embodiment of this disclosure, a parameter generation device configured to implement the above-described parameter generation method is also provided. FIG8 is a schematic diagram of a parameter generation device according to an embodiment of this disclosure. As shown in FIG8, the 800 includes: a calling module 802, a generation module 804, a retrieval module 806, a correction module 808, and an output module 810.
[0229] The system comprises the following modules: a calling module, configured to obtain input data and a global detection list by calling a first interface, wherein the first interface includes a first parameter whose value includes the input data and the global detection list; a generation module, configured to input the input data and the global detection list into a target model and generate initial calling parameters using the target model, wherein the initial calling parameters are configured to represent calling parameters for at least one calling object, and the global detection list is configured to represent a list for error correction of calling parameters for any calling object; a retrieval module, configured to retrieve local detection lists corresponding to at least one calling object, wherein the local detection lists are configured to represent a list for error correction of calling parameters for at least one calling object; a correction module, configured to input the initial calling parameters and the local detection list into the target model and perform error correction on the initial calling parameters using the target model to obtain target calling parameters; and an output module, configured to output the target calling parameters by calling a second interface, wherein the second interface includes a second parameter whose value includes the target calling parameters.
[0230] It should be noted that the calling module 802, generating module 804, retrieving module 806, correcting module 808, and output module 810 correspond to steps S502 to S510 in the above embodiments. The five modules and their corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the above embodiments. It should also be noted that the above modules or units can be hardware or software components stored in memory and processed by one or more processors. These modules can also run as part of a device in the server 10 provided in the above embodiments.
[0231] Figure 9 is a flowchart of a parameter generation method according to an embodiment of the present disclosure. As shown in Figure 9, the method may include the following steps:
[0232] Step S902: In response to receiving input data from the target model, the input data is input into the target model, and the target calling parameters are generated using the target model.
[0233] The target model is trained based on a global detection list and a local detection list. The global detection list represents a list of error corrections performed on the call parameters of any calling object, while the local detection list represents a list of error corrections performed on the call parameters of at least one calling object.
[0234] Through the above steps, in response to receiving input data from the target model, the input data is fed into the target model, which then generates target invocation parameters. The target model is trained based on a global detection list and a local detection list. The global detection list represents a list for error correction of invocation parameters for any invocation object, while the local detection list represents a list for error correction of invocation parameters for at least one invocation object. This improves the accuracy of parameter generation. Notably, due to the combined global and local error checking and correction strategy, the target model can ensure the correctness of parameters from both a broad global perspective and a specific local perspective when generating invocation parameters. This directly improves the quality of the invocation parameters. The target model avoids common errors at the global level while refining the parameter details of specific invocation objects at the local level, ensuring that invocation is checked and improved based on a comprehensive error knowledge base. This comprehensive adjustment and progressively refined error correction strategy improves the quality of generated invocation parameters, fundamentally solving the technical problem of low invocation accuracy caused by insufficient error correction in previous models. This improves the accuracy of parameter invocation and thus addresses the overall technical issue of low invocation accuracy in models.
[0235] In the embodiments disclosed above, the method further includes: acquiring an input sample; constructing a second erroneous example sample and a second correct example sample corresponding to the input sample based on a global detection list and / or a local detection list; inputting the input sample into a target model and generating sample calling parameters using the target model; constructing a loss function based on the sample calling parameters, the second erroneous example sample, and the second correct example sample; and updating the model parameters of the target model using the loss function.
[0236] Embodiments of this disclosure may provide a computing device. FIG10 is a structural block diagram of a computing device according to an embodiment of the present disclosure. As shown in FIG10, the computing device 100 may include: one or more (only one is shown in the figure) processors 102, memory 104, memory controller, and peripheral interfaces.
[0237] The aforementioned computing device can be understood as an integrated smart terminal, including but not limited to servers, desktop computers, PCs (Personal Computers), all-in-one model machines, etc., and the computing device may have the model in the above embodiments of this disclosure pre-installed.
[0238] Specifically, this computing device can pre-install various types of models, including but not limited to models in natural language processing, visual processing, speech processing, code processing, and multimodal task processing, thus providing diverse model selection. In different product forms, this computing device can support one or more model usage methods, including but not limited to model training, model invocation, model fine-tuning, model deployment, model inference, and application. In some product forms, this computing device also supports model management, including but not limited to multi-type model management (supporting the management of discriminative, generative, and other types of models), model version control (supporting the control of different model versions), and model evaluation (evaluating model performance and effectiveness based on model evaluation tools). In other product forms, this computing device can also create applications based on models, providing API calling capabilities, allowing models to be called into created applications through API interfaces, and providing application management tools for application management and monitoring.
[0239] Furthermore, the computing device may also include data management (supporting the creation and management of model tuning datasets), a training center (providing abundant training resources to help users learn and master AI technology), and basic control capabilities (providing enterprise-level basic control capabilities to ensure the security and efficient operation of the system). Through the above functions, it provides a comprehensive and integrated device for AI development, training, deployment, and application.
[0240] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the methods and apparatus in the embodiments of this disclosure. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby implementing the methods in the above embodiments. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to terminal A via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.
[0241] The processor can invoke an executable program stored in memory via a transmission device to execute any of the methods described in the above embodiments.
[0242] Embodiments of this disclosure can provide an electronic device. FIG11 is a structural block diagram of an electronic device according to an embodiment of this disclosure. As shown in FIG11, the electronic device may include: an input / output device 112; a memory 114; and a processor 116, wherein the processor 116 is connected to the input / output device 112 and the memory 114 via a bus 118.
[0243] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the methods and apparatus in the embodiments of this disclosure. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby implementing the methods in the above embodiments. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to terminal A via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.
[0244] The processor can invoke an executable program stored in memory via a transmission device to execute any of the methods described in the above embodiments.
[0245] It will be understood by those skilled in the art that the structure shown in FIG11 is merely illustrative, and the computing device may also be a smartphone, tablet computer, handheld computer, mobile internet device (MID), PAD, or other terminal device. FIG11 does not limit the structure of the aforementioned computing device. For example, the computing device 100 may also include more or fewer components (such as network interfaces, display devices, etc.) than those shown in FIG11, or have a different configuration than that shown in FIG11.
[0246] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.
[0247] Embodiments of this disclosure also provide a computer-readable storage medium. Optionally, in this embodiment, the computer-readable storage medium can be used to store program code executed by the method provided in the above embodiments.
[0248] Optionally, in this embodiment, the storage medium may be located in a computing device.
[0249] Optionally, in this embodiment, the computer-readable storage medium is configured to store an executable program, which, when the executable program is running, controls the device where the computer-readable storage medium is located to execute any of the methods described in the above embodiments.
[0250] Embodiments of this disclosure also provide a computer program product. Optionally, in this embodiment, the computer program product may include a computer program that, when executed by a processor, implements the methods provided in the embodiments described above.
[0251] Embodiments of this disclosure also provide a computer program product. Optionally, the computer program product may include a non-volatile computer-readable storage medium, which can be used to store a computer program that, when executed by a processor, implements the methods provided in the embodiments described above.
[0252] Embodiments of this disclosure also provide a computer program. Optionally, in this embodiment, when the computer program is executed by a processor, it implements the method provided in the above embodiments.
[0253] In the above embodiments of this disclosure, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0254] In the several embodiments provided in this disclosure, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between units or modules may be electrical or other forms.
[0255] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0256] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0257] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.
[0258] The above description is only a preferred embodiment of this disclosure. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principles of this disclosure, and these improvements and modifications should also be considered within the scope of protection of this disclosure.
Claims
1. A parameter generation method, comprising: In response to receiving input data from the target model, the input data and a global detection list are input into the target model, and initial calling parameters are generated using the target model. The initial calling parameters are used to represent calling parameters for at least one calling object, and the global detection list is used to represent a list for error correction of calling parameters for any calling object. Retrieve the local detection list corresponding to the at least one calling object, wherein the local detection list is used to represent a list of error corrections performed on the calling parameters of the at least one calling object; The initial call parameters and the local detection list are input into the target model, and the target model is used to correct errors in the initial call parameters to obtain the target call parameters.
2. The parameter generation method of claim 1, wherein, The input data and global detection list are input into the target model, and the target model is used to generate initial calling parameters, including: The input data is subjected to intent recognition to obtain intent invocation information, wherein the intent invocation information is used to indicate the invocation object type of the target model; The target model is guided to generate the initial invocation parameters based on the intent invocation information using the global detection list, wherein the guidance is used to indicate that the target model avoids the error information listed in the global detection list during the generation of the initial invocation parameters.
3. The parameter generation method of claim 1, wherein, The initial invocation parameters and the local detection list are input into the target model, and the target model is used to perform error correction on the initial invocation parameters to obtain the target invocation parameters, including: The initial call parameters and the local detection list are input into the target model, and the target model is used to identify errors in the initial call parameters to obtain error information in the initial call parameters. Based on the local detection list, generate correction information corresponding to the error information; Based on the correction information, the error information in the initial call parameters is corrected to obtain the target call parameters.
4. The parameter generation method according to any one of claims 1 to 3, wherein The method further includes: Obtain the first call error type of the target model, wherein the first call error type is used to represent the error type generated by the target model during the call to the arbitrary call object; Construct a structured description template corresponding to the first call error type, wherein the structured description template describes the call error type using a structured language; Extract the error prevention conditions corresponding to the first call error type, wherein the error prevention conditions are used to represent the conditions that the target model needs to meet during the parameter generation process; The global detection list is generated based on the first call error type, the structured description template, and the error prevention conditions.
5. The parameter generation method of claim 4, wherein, The first call error type of the target model is obtained, including: Obtain historical error data of the target model, wherein the historical error data is used to represent error data generated during the historical invocation of the target model; The historical error data is classified to obtain the initial call error type, and the initial call error type is output. Receive feedback information on the initial call error type, and adjust the initial call error type based on the feedback information to obtain the first call error type.
6. The parameter generation method according to any one of claims 1 to 3, wherein The method further includes: Obtain the object attributes of the at least one calling object and the second calling error type of the target model, wherein the second calling error type is used to represent different error types generated during the process of the target model calling the at least one calling object; Construct an example sample library for the second call error type, wherein the example sample library contains a first error example sample and a first correct example sample for the second call error type; Generate an error description corresponding to the second call error type and obtain error correction guidance for the second call error type, wherein the error correction guidance is used to guide the target model to correct errors in the call parameters; The local detection list is generated based on the object attributes, the second call error type, the example sample library, the error description, and the error correction guidelines.
7. The parameter generation method according to any one of claims 1 to 3, wherein The method further includes: Obtain the input sample; Based on the global detection list and / or the local detection list, construct the second incorrect example sample and the second correct example sample corresponding to the input sample; The input sample is input into the target model, and the target model is used to generate sample calling parameters; Construct a loss function based on the sample call parameters, the second incorrect example sample, and the second correct example sample; The model parameters of the target model are updated using the loss function.
8. The parameter generation method of claim 7, wherein, Constructing a loss function based on the sample call parameters, the second incorrect example sample, and the second correct example sample includes: Determine the first similarity between the sample invocation parameters and the second incorrect example sample, and determine the second similarity between the sample invocation parameters and the second correct example sample; Determine the first product of the first similarity and the first weight, and determine the second product of the second similarity and the second weight, wherein the first weight is less than the second weight; The loss function is constructed based on the first product, the second product, and the training objective of the target model.
9. The parameter generation method of claim 3, wherein, The correction information includes guidance on correcting errors.
10. The parameter generation method of claim 5, wherein, Obtain historical error data of the target model, including: Obtain the first failure record caused by parameter generation error during the interaction process between the target model and external tools in the historical time period. Obtain the second failure record generated due to parameter generation error during the interaction between the target model and the application interface in the historical time period; Based on the first failure record and the second failure record, the historical error data is determined.
11. The parameter generation method of claim 5, wherein, The feedback information is a correction suggestion for the initial call error type, either through manual evaluation or automatic system analysis.
12. The parameter generation method of claim 6, wherein, The object attributes are the object characteristics and parameter requirements of the at least one calling object.
13. The parameter generation method of claim 1, wherein, The sample invocation parameters are parameters generated by the target model after receiving the input sample, used to invoke tools or services.
14. The parameter generation method of claim 1, wherein, The method further includes: The target application is invoked based on the target invocation parameters.
15. A parameter generation method, comprising: In response to receiving input data from the target model, the input data is input into the target model, and the target model is used to generate target calling parameters; The target model is trained based on a global detection list and a local detection list. The global detection list represents a list for error correction of the call parameters of any calling object, and the local detection list represents a list for error correction of the call parameters of at least one calling object.
16. A parameter generation method, comprising: In response to input commands applied to the operation interface, the input data of the target model and a global detection list are displayed on the operation interface, wherein the global detection list is used to represent a list of error corrections performed on the call parameters of any calling object; In response to a processing instruction applied to the operation interface, target call parameters are displayed on the operation interface. The target call parameters are obtained by inputting initial call parameters and a local detection list into the target model, and then using the target model to perform error correction on the initial call parameters. The local detection list is a local detection list for at least one call object. The initial call parameters are generated by inputting the input data and the global detection list into the target model, and then using the target model. The initial call parameters represent call parameters for calling at least one call object, and the local detection list represents a list for error correction of the call parameters for the at least one call object.
17. A parameter generation method, comprising: Input data and a global detection list are obtained by calling a first interface, wherein the first interface includes a first parameter, and the parameter value of the first parameter includes the input data and the global detection list; The input data and global detection list are input into the target model, and the target model is used to generate initial call parameters, wherein the initial call parameters are used to represent the call parameters for calling at least one call object, and the global detection list is used to represent the list of error corrections for the call parameters of any call object; Retrieve the local detection list corresponding to the at least one calling object, wherein the local detection list is used to represent a list of error corrections performed on the calling parameters of the at least one calling object; The initial invocation parameters and the local detection list are input into the target model, and the target model is used to perform error correction on the initial invocation parameters to obtain the target invocation parameters; The target call parameters are output by calling the second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter includes the target call parameters.
18. A computing device, comprising: Memory, which stores executable programs; A processor for running the program, wherein the program, when running, performs the method according to any one of claims 1 to 17.
19. An electronic device comprising: Memory, which stores executable programs; A processor, connected with the memory through a bus, for running the program, wherein the program, when running, performs the method of any one of claims 1 to 17.
20. A computer readable storage medium comprising a stored executable program, wherein, The computer readable storage medium is controlled to perform the method of any one of claims 1 to 17 when the executable program is running.
