A large model calling detection method, device, medium and equipment

Abstract

This specification discloses a large-scale model call detection method. This method obtains normal tool call information during the large-scale model's inference process, identifies positive samples, and adjusts the normally calling tools or their calling parameters based on these positive samples to construct negative samples. A detection large-scale model trained on these positive and negative samples learns how to infer which parameters are normal for calling which tools based on the large-scale model's input information. When this detection large-scale model is applied to a business process, it can detect whether there are any improper tool calls based on the tool call information of the business large-scale model. By learning the contextual dependencies and semantic rationality of tool calls, it avoids the problem of manually setting detection rules being difficult to adapt to complex dynamic scenarios, and eliminates the need to configure rules separately for different tools, thus improving detection accuracy and versatility.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular to a method, apparatus, medium and device for detecting large model calls. Background Technology

[0002] With the development of artificial intelligence (AI) technology, large language models (LLMs) have been widely applied in various fields. When performing tasks, LLMs can call various tools to complete complex tasks.

[0003] However, when LLM calls tools, problems such as incorrect tool selection, parameter mismatch, and abnormal calling logic often occur, leading to task execution failure. Among these, the phenomenon of unusable output results due to LLM's incorrect tool selection or use can be called "tool illusion".

[0004] In existing technologies, this type of problem is typically addressed by matching LLM tool invocation instructions with pre-defined detection rules based on expert experience to determine the rationality of the tool invocation. However, in complex contexts or dynamic task scenarios, tool invocation may depend on multi-dimensional information such as dialogue history, user implicit intent, and environmental state, making it difficult for detection rules to match. Furthermore, different tools often require customized detection strategies, resulting in poor universality and high costs for the detection methods.

[0005] Therefore, how to accurately detect the instructions generated by the large language model during tool invocation has become an urgent problem to be solved. Thus, this specification provides a method for detecting large model invocations. Summary of the Invention

[0006] This specification provides a large model call detection method, apparatus, storage medium, and electronic device to partially solve the problems existing in the prior art.

[0007] The embodiments in this specification adopt the following technical solutions: This specification provides a method for detecting large model calls, the method comprising: Obtain the calling tool dataset, wherein each calling record in the calling tool dataset contains the input information of the large model, a list of candidate tools, the calling tool, and the calling parameters; For each call record, the data contained in that call record is used as the training sample, and normal tool calls are used as labels to determine positive samples; Select at least a portion of the call records, adjust the calling tools or calling parameters in the selected call records, and determine the negative samples; Based on the positive and negative samples, a detection model is trained to detect tool calls in the business process.

[0008] This specification provides a large model call detection device, the device comprising: The record acquisition module is used to acquire the calling tool dataset. Each calling record in the calling tool dataset contains the input information of the large model, a list of candidate tools, the calling tool, and the calling parameters. The positive sample module is used to determine positive samples for each call record, using the data contained in that call record as training samples and normal tool calls as labels. The negative sample module is used to select at least a portion of the call records, adjust the calling tools or calling parameters in the selected call records, and determine negative samples. The training and detection module is used to train the detection model based on the positive and negative samples, so as to detect the tool calls of the business model in the business process through the trained detection model.

[0009] This specification provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned large model call detection method.

[0010] This specification provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the aforementioned large model call detection method.

[0011] The above-described at least one technical solution adopted in the embodiments of this specification can achieve the following beneficial effects: This specification discloses a large model call detection method. This method obtains normal tool call information during the large model's inference process, identifies positive samples, and adjusts the normally calling tools or their calling parameters based on these positive samples to construct negative samples. A detection large model trained on these positive and negative samples learns how to infer which parameters are normal for calling which model based on the large model's input information. When this detection large model is applied to a business process, it can detect whether there are any improper tool calls based on the business large model's tool call behavior. By learning the contextual dependencies and semantic rationality of tool calls, it avoids the problem of manually setting detection rules being difficult to adapt to complex dynamic scenarios, and eliminates the need to configure rules separately for different tools, thus improving detection accuracy and versatility. Attached Figure Description

[0012] The accompanying drawings, which are included to provide a further understanding of this specification and form part of this specification, illustrate exemplary embodiments and are used to explain this specification, but do not constitute an undue limitation thereof. In the drawings: Figure 1 A flowchart of a large model call detection process provided in the embodiments of this specification; Figure 2 A schematic diagram provided for the embodiments of this specification to supplement the implicit steps; Figure 3 This is a schematic diagram of a large model call detection device provided in the embodiments of this specification; Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this specification. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0014] Currently, for Large Language Models (LLMs), also known simply as large models, the problem of unusable output results due to incorrect tool calls during inference is generally addressed using two methods. One approach is to predefine a series of detection rules to perform syntax and format matching on the tool call instructions generated by the LLM, thereby determining whether the tool selection and parameter filling are reasonable.

[0015] However, as task scenarios become more complex and toolsets dynamically expand, detection rules have gradually revealed their shortcomings. First, detection rules struggle to handle complex contexts and semantic dependencies in dynamic task scenarios.

[0016] The appropriateness of LLM calling tools usually requires a comprehensive judgment based on multi-dimensional information such as dialogue history, user implicit intent, and environmental state. Static detection rules can only match surface features of text and cannot capture deep semantics, leading to situations where they cannot match or the detection results are incorrect.

[0017] For example, in a series of multi-turn conversations, users may omit some information from the context, requiring the LLM to infer the tool to be invoked based on historical information; or, the applicability of the same tool may vary drastically in different contexts, making it impossible to adjust detection rules based on business scenarios. Furthermore, when task objectives or available tools change frequently, detection rules need to be constantly updated, making it difficult to cover all edge cases and resulting in a significant drop in detection accuracy.

[0018] Secondly, different detection strategies are often required for different tools, resulting in poor versatility and high costs. Since each tool has different functional semantics, parameter constraints, and invocation scenarios, different detection rules need to be designed for each tool. This requires deep expert involvement, analyzing the characteristics of each tool and writing matching logic. As the number of tools increases, maintaining the detection rule base becomes difficult, consuming significant manpower and easily leading to misjudgments due to rule conflicts or omissions.

[0019] Furthermore, for implicit semantic misuse, the detection rules are unable to provide accurate results because they lack the ability to understand the consistency between the calling intent and the task requirements. For example, if the LLM selects a tool with the correct format but completely mismatched functionality, or if the parameter values ​​conform to type constraints but are unreasonable in business logic, the detection rules often fail to provide correct results.

[0020] Based on this, this specification provides a method for detecting whether a tool is being called correctly for a large model, which differs from rule matching. The technical solutions provided by the various embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0021] Figure 1 A flowchart for large model call detection provided in the embodiments of this specification specifically includes the following steps: S100: Obtain the calling tool dataset. Each calling record in the calling tool dataset contains the input information of the large model, a list of candidate tools, the calling tool, and the calling parameters.

[0022] In the embodiments described in this specification, the following are employed: Figure 1 The device used for large model call detection as illustrated can be any electronic device, such as a computer, server, or server cluster consisting of multiple servers. For ease of description, the following explanation uses a server as an example only.

[0023] In order to detect whether the tools are called correctly during the LLM inference process and reduce the possibility of model illusion caused by incorrect tool calls, the server needs to obtain the record information of large model tool calls in the business that has been executed normally in the past, so as to build samples for training and testing large models in the future.

[0024] Therefore, the server can obtain the historical records of the correct tool calls made by the business big model in actual business scenarios, i.e., the call records. Since the detection big model needs to learn which tool the business big model should call and with what parameters under what input conditions and when faced with which available tools, the call records should at least include the input information of the business big model, the list of candidate tools, the tool actually called, and the call parameters.

[0025] Input information refers to the original request content that triggers the business big data model to invoke the tool, such as a query submitted by the user in natural language. Furthermore, in multi-turn dialogue scenarios, input information can also include the history of previous user questions and the business big data model's responses. For example, if a user first asks "I want to go to Beijing tomorrow," and the business big data model replies by asking for the specific destination, and the user then asks "to Shanghai," the input information could include the complete text of both rounds of dialogue. Input information provides context for understanding user intent and is the foundational information for the model to determine the rationality of the tool invocation.

[0026] The candidate tool list is a collection of tools that the business large-scale model can access and select during the inference phase. This list can be determined by the server based on the business system's configuration file and can also be formatted in a structured and standardized way. Furthermore, each candidate tool in this list includes a unique identifier, a functional description, and parameter specifications. The candidate tool list defines the scope of tools that the business large-scale model can call during inference, and the large-scale model can use this list to evaluate the appropriateness of the tools selected.

[0027] The tool invoked refers to the tool that the large model actually selects and decides to invoke after processing the current input information. The invoked tool is recorded in the form of a unique identifier.

[0028] Similarly, invocation parameters refer to the specific parameters filled in by the business model according to the interface specification of the selected invocation tool. Generally, invocation parameters must strictly match the parameter names, data types, and constraints defined in the tool. Since the invocation parameters record the specific instructions for the business model to execute the tool invocation, the business model can learn and verify the rationality of the parameter values ​​and their consistency with the input information based on this.

[0029] S102: For each call record, use the data contained in that call record as training samples and normal tool calls as labels to determine positive samples.

[0030] In the embodiments described in this specification, after the server obtains the call records, it can construct positive samples for training the large-scale detection model. Positive samples refer to call records where the tool call behavior has been confirmed to meet business expectations and no hallucination phenomenon has occurred. The subsequent training process adopts a supervised learning method. The large-scale detection model needs to learn the correct mapping relationship between input information, candidate tool list, calling tool, and calling parameters from the positive samples, thereby forming a judgment benchmark for normal calls. Without accurate positive samples, the large-scale detection model will have difficulty correctly distinguishing between normal calls and hallucinations, resulting in a decrease in detection accuracy. Therefore, after obtaining the calling tool dataset, each record can be labeled as a positive sample.

[0031] Specifically, since the call record comes from the production environment, it is generally determined by the log system. The log system filters out records of tool call failures or exceptions, and retains only the calls that were actually executed successfully. Therefore, the server can directly use the call record as a training sample and mark it as a normal tool call, i.e. a positive sample.

[0032] For example, in a user's query "What's the weather like today?", the business big model selects an online search tool from the candidate tool list and fills in the query keyword "today's weather". The tool is then successfully invoked and returns the weather information. The log system records the complete call process, and the server marks the call record as a positive sample.

[0033] In addition, in the embodiments described in this specification, the server can also generate call records through a simulation test environment. For call records collected from the simulation test environment, the server can make judgments based on the simulation execution results. In the simulation test, each call record is associated with a test case and its expected result. When the tool call instructions generated by the large business model are completely consistent with expectations and the simulation environment executes successfully, this call record can be used to construct positive samples.

[0034] S104: Select at least some call records, adjust the calling tools or calling parameters in the selected call records, and determine the negative samples.

[0035] In the embodiments described in this specification, positive samples provide correct tool calls, but the detection model also needs to distinguish between normal calls and illusions; therefore, it needs to be trained using negative samples. However, tool illusions manifest in various ways in business scenarios, including incorrect tool selection, missing call parameters, mismatched call parameter types, call parameter values ​​exceeding reasonable ranges, and call parameters inconsistent with user intent. Relying solely on naturally occurring illusion samples obtained from the production environment would result in sparse negative samples, incomplete coverage, and long collection cycles. Therefore, the server can construct negative samples by specifically adjusting existing positive samples.

[0036] Specifically, the server can first select at least a portion of the call records as the adjustment targets. The selection method can be based on random sampling, or it can be targeted to specific tools or specific error types according to business needs; this specification does not impose any restrictions on this.

[0037] Secondly, for each selected call record, the server can modify the call tool or call parameters contained therein to generate one or more new call records, and set the label field of these new records to negative, indicating that they are negative samples.

[0038] In one or more embodiments of this specification, negative samples include tool selection errors, which include a large model selecting a tool from a list of candidate tools that does not match the user's current intent.

[0039] Specifically, the server first selects at least a portion of the call records as the processing targets. The selection method can be random sampling, or it can be targeted to specific tool types or specific scenarios based on business needs. For example, call records involving high-frequency tools can be prioritized to increase the negative sample density of key scenarios.

[0040] Secondly, for each selected call record, the server obtains the candidate tool list corresponding to the call record, and selects a candidate tool from the candidate tool list that is different from the call tool originally included in the call record.

[0041] The server's selection of a specific tool can be determined according to preset rules. For example, it can randomly select another tool, or choose a tool semantically unrelated to the original tool based on functional differences. When selecting randomly, the server can generate a random number index pointing to any tool identifier in the candidate tool list, excluding the tool originally used in the call record. In the semantic difference selection method, the server can use the tool's functional description text to select a candidate tool with different functions.

[0042] Finally, the server keeps the input information field of the call record unchanged, keeps the candidate tool list field unchanged, and replaces the call tool field with the newly selected candidate tool to construct negative samples. Of course, if the server selected multiple candidate tools in the previous step, multiple negative samples can be constructed accordingly.

[0043] It should be noted that the parameter fields can be adjusted according to the characteristics of the new tool, such as setting them to empty values ​​or filling them with default parameters that conform to the format of the new tool, but the original parameters can also be kept unchanged.

[0044] In one or more embodiments of this specification, due to insufficient reasoning ability or scenarios such as prompt word injection attacks, large models may generate tool invocation instructions for fictitious search tools because they lack the tools required for reasoning. To enable the detection model to learn that any tool invocation outside a given list of candidate tools constitutes an anomaly, the server, when determining candidate tools to replace the invoking tool, can also generate another tool that does not exist in the current call record's candidate tool list and construct a negative sample.

[0045] The server can generate other tools based on various rules. For example, the server can maintain a glossary of tool names that are known but not relevant to the current business scenario, and randomly select another tool from it. Alternatively, the server can generate new tools by modifying the strings of existing tool identifiers. Or, the server can generate entirely random, fictional tools that conform to tool naming conventions.

[0046] In one or more embodiments of this specification, negative samples include parameter errors. Parameter errors refer to situations where, although the large model has selected the correct tool, the generated call parameters do not conform to the tool's interface specifications. Examples include incorrect parameter types, abnormal parameter formats, parameter values ​​exceeding the allowed range, or missing necessary parameters. Such errors can also cause tool calls to fail or return incorrect results, representing another important aspect of tool illusion.

[0047] To train the large model to detect anomalies at the parameter level, the server can actively construct negative samples with incorrect parameters by adjusting the call parameters in the existing call information.

[0048] Specifically, the server can first select at least a portion of the call records as processing objects. The selection method can be referred to the previous description and will not be repeated here. For example, tool call records with complex parameter structures or prone to errors can be selected to increase the negative sample density in relevant scenarios.

[0049] Subsequently, for each selected call record, the server can retrieve the parameter type description corresponding to the calling tool in that call record. The parameter type description is pre-stored in the candidate tool list, detailing the name, data type, format constraints, value range, and whether the parameters required by each tool are mandatory. Based on the parameter type description, the server can perform different adjustments to the original call parameters to generate parameter values ​​that do not conform to the specifications.

[0050] The server can adjust the call parameters according to the parameter type description of the calling tool in the call record. That is, the type of the call parameters remains unchanged, but the parameter values ​​change to non-compliant values. The adjustment process includes: modifying the parameter value to a value outside the preset range, such as changing the month parameter to 13 or the star rating to 6. Alternatively, for string parameters with specific format requirements, deleting or modifying their format identifiers to generate non-compliant strings, such as removing the @ symbol from an email address or shortening the number of digits in a phone number. Alternatively, deleting one or more required parameters, resulting in missing parameters. Alternatively, replacing the parameter value with a reasonable value that clearly contradicts the semantics of the input information, such as changing the city parameter to Shanghai when a user queries Beijing weather.

[0051] In one or more embodiments of this specification, the server may also determine different types of call parameters according to the parameter type description of the calling tool in the call record. That is, it addresses situations where the parameter values ​​generated by the large model do not match the data types defined in the tool interface at all; for example, the large model provides string values ​​for parameters expecting integers, or arrays for parameters expecting boolean values. Such errors typically stem from a misunderstanding of the tool interface by the large model or from formatting errors in the generation of the format.

[0052] The server can then determine alternative data types based on the parameter type description, replacing the original parameter type with a value of the corresponding type. For example, for a parameter described as a string, the server can generate an integer, boolean, or JSON object as a replacement. For a parameter described as an integer, the server can generate a string literal or a floating-point number. For a parameter described as a boolean, the server can generate an array or an empty object. For a parameter described as an object, the server can generate a simple string. Furthermore, when generating replacement values, the server can randomly generate any value that conforms to the target type syntax.

[0053] Finally, after adjusting the parameters, the server generates a new call record based on the original call record. The input information field in the new call record remains unchanged, the candidate tool list field remains unchanged, the called tool field remains unchanged, and the call parameter field is replaced with the adjusted parameter values. The new call record is then labeled as a negative sample.

[0054] S106: Based on the positive samples and the negative samples, train the detection model to detect the tool calls of the business model in the business process.

[0055] In the embodiments of this specification, based on the positive and negative samples obtained in the aforementioned steps, the general large model is subjected to supervised fine-tuning to obtain the detection large model. The detection large model can, based on the context, i.e. the input information and the tools that the business large model can call, infer whether the tools called by the business large model and the calling parameters used during inference are abnormal.

[0056] Therefore, after training the large-scale detection model, the tool invocation behavior of the business model can be monitored in real time. During the reasoning process in a business scenario, although the business model possesses the ability to invoke tools, it may still experience tool illusions due to semantic understanding biases, insufficient context modeling, or reasoning errors, leading to tool invocations that do not match user intent or defective invocation parameters. This detection model can check the rationality of each tool invocation before it is executed and dynamically determine the direction of the business process based on the judgment result, thereby achieving quality assurance and risk control for tool invocations. Alternatively, after the business model outputs results, the detection model can check each tool invocation during the business model's reasoning process and determine the usability of the output result based on the detection results, similarly dynamically determining the direction of the business process.

[0057] Specifically, taking the server's real-time detection of the application's large-scale detection model as an example, the server first continuously listens for or receives tool call detection requests from the business process. This request is typically triggered after the large-scale business model infers and generates tool call instructions.

[0058] Subsequently, in response to the detection request, the server retrieves business process-related information, including the input information of the business model, the list of tools invoked by the business model, and the tool invocation records during the business model's inference process. The input information is generally natural language content, such as user questions or multi-turn dialogue history text. The tool invocation list is a pre-configured set of accessible tools by the business system for the corresponding business scenario or user permissions. Each tool in the list includes a tool identifier, a function description, and a parameter type description. The tool invocation records are the actual sequence of invocation instructions generated by the business model during inference.

[0059] Then, the server parses the actual tools selected by the business model and the populated call parameters from the acquired tool call records. The tools are extracted as tool identifiers, while the call parameters are generally parsed into structured key-value pairs. After information extraction, the server organizes the input information, the list of tools, the tools themselves, and the call parameters according to the expected input format of the detection model, and adds preset prompts to construct a sample to be detected. These prompts can be fixed templates and are used to prompt the detection model to determine whether the tool call is abnormal based on the input information, the list of tools, the tools, and the tool parameters.

[0060] Subsequently, the server inputs the sample to be detected into the pre-trained detection model for inference calculations. Based on the tool calls learned during the training phase, the detection model uses normal and abnormal modes to discriminate the current sample and outputs the detection result.

[0061] Finally, after obtaining the detection results, the server determines the execution flow of subsequent business processes based on preset business strategies. When the detection result is normal, the server allows the tool call to continue execution and forwards the call instruction to the external system to complete the actual operation. When the detection result is abnormal, the server can intercept the call and record the abnormal log, or trigger an alarm to notify the operations and maintenance personnel, or require the business big model to regenerate the call instruction, or transfer it to human customer service for handling. Through the above process, the server realizes intelligent detection and dynamic process control of tool calls to the business big model, effectively reducing the business risks caused by tool illusion.

[0062] For example, in a hotel booking scenario, the business model receives user input "Book me a hotel in Shanghai for tomorrow" and generates a tool invocation command during inference. Upon receiving the detection request, the server retrieves the input text, obtains a list of invoked tools (such as hotel booking tools, flight search tools, and weather search tools), and retrieves the tool invocation records. It parses the invoked tool as a hotel booking tool, with parameters including the city Shanghai, check-in date tomorrow, and room type not specified. After inputting this information into the detection model, the model outputs a normal detection result, and the server allows the invocation, allowing the hotel booking tool to execute normally.

[0063] In a weather query scenario, a user enters "Will it rain in Beijing today?", but the business model incorrectly generates a call to a flight query tool. When the server checks, the detection model identifies that the call to the tool does not match the input intent, outputs an anomaly detection result, and the server immediately intercepts the call and triggers a regeneration process, thus preventing the incorrect execution of the business logic.

[0064] based on Figure 1 The large-scale model call detection method described herein acquires normal tool call information during the large-scale model's inference process to identify positive samples. Based on these positive samples, it adjusts the tools or their call parameters to construct negative samples. The detection model, trained on both positive and negative samples, learns how to infer which parameters are normal for calling which model based on the large-scale model's input information. When this detection model is applied to a business process, it can detect improper tool calls based on the large-scale business model's tool call behavior. By learning the contextual dependencies and semantic rationality of tool calls, it avoids the problem of manually setting detection rules being difficult to adapt to complex and dynamic scenarios, and eliminates the need for separate rule configurations for different tools, thus improving detection accuracy and versatility.

[0065] In addition, in the embodiments of this specification, in step S100, when the server obtains the tool call dataset, it can collect data from multiple data sources. These data sources include, but are not limited to, the large model call logs recorded in the production environment and the tool call records generated in the simulation test platform. In the production environment, after the large model responds to a business request and generates a tool call instruction, the server can persistently store the complete information of that call in the log system. Each log record contains the input information of the large model, specifically referring to the original user question that triggered the tool call and the possible multi-turn dialogue history text, as well as a list of candidate tools available for this call, which is pre-configured by the business system according to the current scenario. It also records the actual calling tool selected by the large model and its calling parameters, where the calling parameters refer to data in the form of specific key-value pairs filled according to the tool interface specification. For the simulation test environment, the server simulates various business scenarios, drives the large model to execute tool calls, and collects and stores the input information, candidate tool list, calling tool, and calling parameters for each simulation.

[0066] Furthermore, after the server collects the dataset of the calling tools, data cleaning and format standardization can be performed. The cleaning process removes incomplete records or failed calls, ensuring that every record represents a successful call. Standardization converts data from different sources into a pre-defined storage format; for example, input information is uniformly encoded as text strings, the candidate tool list is arranged in order by tool identifier, and call parameters are serialized into structured objects. Of course, this manual does not restrict the specific methods used and can be configured as needed.

[0067] In addition, in one or more embodiments of this specification, in order to further improve the generalization ability and robustness of the detection model and enable it to adapt to the diversity of user questions and differences in expression in business scenarios, the server may perform data augmentation processing on the input information in the call record.

[0068] Therefore, in step S100, when performing data augmentation, the server first treats the input information fields in each call record as raw text and calls a third-party large model service for synonym rewriting. The server can configure multiple third-party large model interfaces and can choose to call one or more of them according to the settings. When sending a request to the third-party large model, the server constructs prompts, requesting the third-party large model to rewrite the given input information into several different expressions that retain the original meaning of natural language questions. For example, if the original input information is "How is the weather today?", the server can request to generate synonyms such as "How is the weather today?", "What is the temperature today?", and "Is today suitable for outdoor activities?".

[0069] Subsequently, in response to the rewritten results returned by the third-party large model, the server parses and cleans each generated synonym, removing invalid results that are exactly the same as the original sentence or deviate from its semantics.

[0070] Then, for each valid paraphrase, the server generates a new call record based on the original call record. The input information in the new call record is replaced with the paraphrased text, while the candidate tool list, calling tool, and call parameters remain unchanged. This effectively expands the training sample size and improves the adaptability of the large detection model to the diversity of natural language without increasing the cost of collecting real business data.

[0071] Additionally, in step S106 of the embodiments of this specification, the detection result may be represented as a binary identifier of normal or abnormal. Alternatively, it may include an abnormality type classification.

[0072] In other words, the large detection model not only determines whether the call is abnormal, but also, if the call is abnormal, further outputs the specific error type. The classification of error types can correspond to the error patterns when constructing negative samples, such as tool selection error, parameter missing error, parameter type error, parameter value range error, and tool non-existent error.

[0073] When classifying the types of anomalies in the output of the large-scale business model, the server can determine differentiated response strategies based on the type of error. For example, for a tool selection error, the server can determine that it is a misunderstanding of the model's intent, triggering the large-scale business model to reinterpret the user input and generate a new calling instruction. For a parameter missing error, the server can identify it as incomplete information, guiding the large-scale business model to supplement the missing parameters and try again. For a parameter type error, the server can determine that it is an interface misunderstanding error, returning a format requirement explanation to the large-scale business model and requesting a retry. For a tool not found error, the server can classify it as a serious anomaly, directly record an alarm, and transfer it to manual handling.

[0074] By employing this mechanism of handling different errors based on their type, the server can salvage correctable errors, enabling potentially failed calls to execute successfully. This reduces business process interruptions caused by interception, ensuring normal tool usage while improving the smoothness of the user experience. In other words, error correction is performed on the server side, preventing users from repeatedly outputting errors.

[0075] Figure 2 This is a schematic diagram illustrating the detection process for the tool usage provided in the embodiments of this specification. Figure 2Each step has been described in detail in the aforementioned specification and will not be repeated here. In response to a tool call detection request, the system obtains the input information of the business big model in the business process, the list of tools called by the business big model, and the tool call records during the business big model's inference process. Based on the obtained tool call records, the system parses and determines the actual calling tool selected by the business big model and the filled calling parameters. The input information, the list of calling tools, the calling tools, and the calling parameters are organized according to the input format preset by the detection big model to construct a sample to be detected, and input into the trained detection big model. The system obtains the detection results and determines the execution flow of the business process based on the detection results. If the detection result is normal, the tool call is allowed, and the call instruction is forwarded to the external system for execution. If the detection result is abnormal and contains an error type, the system executes the processing strategy according to the error type.

[0076] The above is a large model call detection method provided by the embodiments of this specification. Based on the same idea, this specification also provides corresponding devices, storage media and electronic devices.

[0077] Figure 3 This is a schematic diagram of a large model call detection device provided in an embodiment of this specification. The device includes: The record acquisition module 201 is used to acquire the calling tool dataset. Each calling record in the calling tool dataset includes the input information of the large model, a list of candidate tools, the calling tool, and the calling parameters. The positive sample module 202 is used to determine positive samples for each call record by using the data contained in the call record as training samples and normal tool calls as labels. The negative sample module 203 is used to select at least a portion of the call records, adjust the calling tools or calling parameters in the selected call records, and determine negative samples; The training and detection module 204 is used to train the detection model based on the positive samples and the negative samples, so as to detect the tool calls of the business model in the business process through the trained detection model.

[0078] Optionally, the negative sample module 203 is used to select at least a portion of the call records; for each selected call record, select a candidate tool from the candidate tool list that is different from the call tool contained in the call record; replace the call tool in the call record with the selected candidate tool to obtain a negative sample.

[0079] Optionally, the negative sample module 203 is used to select at least a portion of the call records; for each selected call record, generate other tools that do not exist in the candidate tool list; and replace the calling tool in the call record with the other tools to obtain a negative sample.

[0080] Optionally, the candidate tool list includes each candidate tool that can be called by the large model in the business process, and the candidate tool includes at least a tool function description and a parameter type description.

[0081] Optionally, the negative sample module 203 is used to select at least a portion of the call records; for each selected call record, adjust the call parameters according to the parameter type description of the calling tool in the call record; and replace the call parameters of the call record with the adjusted call parameters to obtain a negative sample.

[0082] Optionally, the acquisition module 201 is used to select at least a portion of the call records; for each selected call record, to determine different types of call parameters according to the parameter type description of the calling tool in the call record; and to replace the call parameters of the call record with different types of call parameters to obtain negative samples.

[0083] Optionally, the training and detection module 204 is configured to, in response to a tool call detection request, obtain input information of the business big model in the business process, a list of tools called by the business big model, and tool call records in the inference layer of the business big model; determine the calling tool and calling parameters based on the tool call records; input the input information, the list of calling tools, the calling tool, and the calling parameters into the trained detection big model, and determine the detection result output by the detection big model; and determine the execution flow of the business process based on the detection result.

[0084] This specification also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can be used to perform the large model call detection method provided above.

[0085] based on Figure 1 The large model call detection method shown in this specification also provides the embodiments. Figure 4 The diagram shows the structure of the electronic device. Figure 4 At the hardware level, the electronic device includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to implement the large model call detection method described above.

[0086] The above description is merely an embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification.

Claims

1. A method for detecting large model calls, the method comprising: Obtain the calling tool dataset, wherein each calling record in the calling tool dataset contains the input information of the large model, a list of candidate tools, the calling tool, and the calling parameters; For each call record, the data contained in that call record is used as the training sample, and normal tool calls are used as labels to determine positive samples; Select at least a portion of the call records, adjust the calling tools or calling parameters in the selected call records, and determine the negative samples; Based on the positive and negative samples, a detection model is trained to detect tool calls in the business process.

2. The method as described in claim 1, wherein at least a portion of the call records are selected, and the calling tools or calling parameters in the selected call records are adjusted to determine negative samples, specifically includes: Select at least some of the call records; For each selected call record, select a candidate tool from the candidate tool list that is different from the call tools contained in that call record; The calling tool in the calling record is replaced with the selected candidate tool to obtain a negative sample.

3. The method as described in claim 1, wherein at least a portion of the call records are selected, and the calling tools or calling parameters in the selected call records are adjusted to determine negative samples, specifically including: Select at least some of the call records; For each selected call record, generate other tools that are not in the candidate tool list; The calling tool in the calling record is replaced with the other tools mentioned above to obtain negative samples.

4. The method as described in claim 1, wherein the candidate tool list includes each candidate tool that can be called by the large model in the business process, and the candidate tool includes at least a tool function description and a parameter type description.

5. The method as described in claim 4, wherein at least a portion of the call records are selected, and the calling tools or calling parameters in the selected call records are adjusted to determine negative samples, specifically including: Select at least some of the call records; For each selected call record, the call parameters are adjusted according to the parameter type description of the calling tool in that call record; The negative sample is obtained by replacing the call parameters of the call record with the adjusted call parameters.

6. The method as described in claim 4, wherein at least a portion of the call records are selected, and the calling tools or calling parameters in the selected call records are adjusted to determine negative samples, specifically including: Select at least some of the call records; For each selected call record, determine the different types of call parameters according to the parameter type description of the calling tool in that call record; The negative sample is obtained by replacing the call parameters of the call record with different types of call parameters.

7. The method as described in claim 1, wherein the tool call status of the business large model in the business process is detected using the trained detection large model, specifically including: In response to a tool call detection request, obtain the input information of the business big model in the business process, the list of tools called by the business big model, and the tool call records in the inference layer of the business big model; Based on the tool call records, determine the calling tool and the calling parameters; The input information, the list of calling tools, the calling tools, and the calling parameters are input into the trained detection model to determine the detection result output by the detection model; and the execution flow of the business process is determined based on the detection result.

8. A large model call detection device, the device comprising: The record acquisition module is used to acquire the calling tool dataset. Each calling record in the calling tool dataset contains the input information of the large model, a list of candidate tools, the calling tool, and the calling parameters. The positive sample module is used to determine positive samples for each call record, using the data contained in that call record as training samples and normal tool calls as labels. The negative sample module is used to select at least a portion of the call records, adjust the calling tools or calling parameters in the selected call records, and determine negative samples. The training and detection module is used to train the detection model based on the positive and negative samples, so as to detect the tool calls of the business model in the business process through the trained detection model.

9. A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in any one of claims 1-7.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in any one of claims 1-7.

Images