A training external verification and reliable repair method for small language model function call

Abstract

The application discloses a training external verification and reliable repair method for small language model function calling, and belongs to the technical field of artificial intelligence and intelligent agent tool calling. In view of the problems of demand judgment error, tool selection deviation, parameter missing distortion and execution result being wrongly rewritten when a small language model calls external tools, the application provides an execution pre-verification, local repair and execution post-result checking scheme. The method first acquires user requests, tool descriptions and function modes and generates an initial call; then performs tool demand, tool matching, parameter mode and request basis verification to form an error set; then repairs error fields based on user requests and tool modes and executes tools; finally, according to the return value, the response consistency is checked, and the final call and response result are output. The application does not need to train or modify a language model, can reduce the risk of error calling and parameter execution, and improve the function calling reliability and auditability in the registered tool environment.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and natural language processing technology. More specifically, it belongs to the method for verifying and repairing small language model function calls in intelligent agent tool calls. Background Technology

[0002] The purpose of function calls is to enable language models to select and execute external tools based on user requests, thereby obtaining computational results, date results, database records, or retrieval information that are difficult to accurately generate using only the model's internal knowledge. A complete function call process typically includes tool requirement assessment, target tool selection, function parameter generation, tool execution, and generating a final response based on the tool's return value.

[0003] Small language models offer advantages such as low deployment costs, fast inference speeds, local device compatibility, and controllable data privacy, making them suitable for tool invocation tasks in resource-constrained scenarios. However, limited by model size and structured generation capabilities, small language models are prone to issues such as incorrect judgments about tool needs, selection of the wrong tool, omission or distortion of parameters, and incorrect rewriting of tool return values. Some of these errors, while conforming to function patterns and executing normally, produce results inconsistent with the user's request, representing semantic errors that are difficult to detect through execution status.

[0004] Existing native function call methods typically execute model-generated calls directly, lacking independent pre-execution validation. While JSON Schema can check parameter structure, it cannot determine the correctness of tool selection and parameter criteria. Self-checking or retrying upon execution failure requires additional model inference and may alter previously correct fields during the overall rewrite process. Methods relying on model fine-tuning or standard answers also suffer from high training costs and difficulty in adapting to new tools. Therefore, a function call method is needed that does not update small language model parameters, does not read test sample standard answers, and can perform pre-execution validation, local repair, real-tool execution, and post-execution result verification. Summary of the Invention

[0005] To address the problems in the prior art, this invention provides a method for out-of-training verification and reliability repair of function calls for small language models. This method constructs a pre-execution verification process independent of the small language model, jointly verifying tool requirements, tool matching, function patterns, and request criteria. Simultaneously, it employs a structured error-driven local repair strategy, repairing only erroneous fields while retaining correct ones. After tool execution, the final response is verified using the actual tool return value, thus forming a complete closed loop for function call reliability.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] S1: Obtain the user request, the tool name, tool description, and function pattern of the candidate tools, and generate the initial function call using the small language model;

[0008] S2, perform tool requirement judgment and tool matching verification on the initial function call to determine whether the user request needs to call a tool and whether the selected tool matches the user request;

[0009] S3, perform function pattern constraint verification and request basis verification on the function parameters in the initial function call to form a structured error set containing error types and error fields;

[0010] S4. Based on the user request, function pattern, and structured error set, perform partial repair on the call fields with errors, re-verify the repaired function call, and execute the corresponding tool to obtain the tool return value after the verification is passed.

[0011] S5. Based on the return value of the tool, perform result consistency verification on the response generated by the small language model. If the response does not faithfully contain or express the return value of the tool, use the return value of the tool to correct it, and output the final function call and response result.

[0012] Furthermore, in step S1, each candidate tool is represented as a structured tool object consisting of a tool name, a tool function description, and a function pattern. The function pattern includes parameter name, parameter description, parameter type, required parameter set, enumeration constraints, format constraints, and additional parameter constraints. The initial function call generated by the small language model is normalized into a call object consisting of a tool requirement identifier, a tool name, and parameter mapping.

[0013]

[0014] Where c0 represents the initial function call, z0 represents the tool requirement identifier, t0 represents the initially selected tool name, and A0 represents the initial parameter mapping; when z0 indicates that the tool does not need to be called, t0 and A0 are normalized to null values ​​to avoid residual tool names or parameters being executed incorrectly.

[0015] Furthermore, when the small language model output contains natural language descriptions, code fences, or incomplete call fields, a structured call object is extracted from the model output; unparseable fields are treated as null values. The generation, verification, and repair processes do not update the small language model parameters and do not use standard tool labels, standard parameters, or standard answers from the test samples.

[0016] Furthermore, the tool requirement determination in step S2 includes: detecting whether the user request contains explicit instructions to directly answer, prohibit calling tools, or not require external tools; when no explicit instructions are found, determining whether there is a target tool that meets preset conditions based on the correlation between candidate tools and the user request.

[0017] Furthermore, the relevance score of the candidate tool is determined based on the matching results between the user request and the tool name, tool description, parameter name, parameter description, enumerated values, and preset task mode. The calculation method is as follows:

[0018]

[0019] Where Ri(q) represents the relevance score of the i-th candidate tool relative to the user request q, Kq represents the lexical set of the user request, Ki represents the lexical set of the tool name and tool description of the i-th candidate tool, Pi represents the parameter field set of the i-th candidate tool, Ki,p represents the lexical set of the name and description of the parameter field p, Ei(q) represents the matching result of the enumerated value or preset task mode in the user request, and α represents the matching weight.

[0020] Furthermore, candidate tools are ranked from highest to lowest according to their relevance scores, and the candidate tool with the highest score is selected as the target tool. When the highest relevance score reaches the tool requirement threshold and the user request does not contain a direct answer instruction, it is determined that the tool needs to be invoked. When the initially selected tool does not exist, or the difference between its score and the target tool's score reaches a preset threshold, the initially selected tool is marked as a tool matching error.

[0021] Furthermore, the function pattern constraint verification in step S3 includes: detecting whether the required parameters exist, detecting whether the string, integer, number, boolean, array and object types are correct, detecting whether the parameter values ​​meet the enumeration constraints and date format constraints, and detecting undefined parameter fields when the function pattern prohibits additional parameters.

[0022] Furthermore, when the parameter type is an array, each element in the array is recursively verified according to the element function pattern; when the parameter type is an object, the required properties of the object, the object property type, and the object additional properties are checked, so that errors in nested parameters can be located to the corresponding array element or object property path.

[0023] Furthermore, the request basis verification extracts parameter basis mappings from the user request based on the function pattern of the target tool, and performs an equivalence comparison between the initial parameter values ​​and the corresponding parameter basis after numerical or string normalization; a basis judgment value is calculated for each parameter field:

[0024]

[0025] Where Gj represents the basis judgment value of the j-th parameter field, I represents the indicative function, aj represents the j-th parameter value in the initial function call, xj(q) represents the basis of the j-th parameter extracted from the user request q, Eq represents the equivalence judgment, and Support indicates whether the user request explicitly supports the corresponding parameter value; when Gj is 0, the corresponding parameter field is marked as request basis error.

[0026] Furthermore, for tools with registered parameter extraction adapters, numerical, date, enumeration, string, boolean, array, or object parameters are extracted from the user request according to the target tool adapter; for unregistered tools, only the content explicitly associated with the parameter name or parameter description is used as the parameter basis. When no explicit basis can be obtained, the meaning of the parameters is not inferred based on the order of their appearance, nor is the standard parameter completion call used.

[0027] Furthermore, in step S3, errors generated from tool requirement judgment, tool matching verification, function pattern constraint verification, and request basis verification are written into a structured error set according to error type, error field, and error reason. The structured error set is represented as follows:

[0028]

[0029] Where E(c0) represents the structured error set corresponding to the initial function call c0, Eneed represents the tool requirement error set, Etool represents the tool matching error set, Eschema represents the function pattern error set, and Eground represents the request basis error set.

[0030] Furthermore, step S4 employs a minimum local repair strategy, prioritizing the repair function call with the smallest distance from the initial function call field modification, provided the verification error set is empty. The repair target is expressed as:

[0031]

[0032] Where c* represents the repair function call, C represents the candidate repair call set, D represents the field modification distance of the repair call relative to the initial function call c0, and V represents the pre-execution verification function for user request q, function call c, and candidate tool set T; local repair only modifies the fields marked in the structured error set and retains the unmarked call fields.

[0033] Furthermore, when there is a tool requirement error, the tool requirement identifier is repaired based on the direct response instruction or the relevance of the target tool; when there is a tool matching error, the unknown tool or the low relevance tool is replaced with the target tool, parameters that do not correspond to the original tool function pattern are cleared, and the target tool parameter mapping is constructed based on the verifiable parameters in the user request.

[0034] Furthermore, when there is a function pattern error, undefined parameters are deleted, parameter values ​​that can be unambiguously converted are type-normalized, and necessary parameters are only added when there is a corresponding basis in the user request; when there is an error in the request basis, only conflicting parameters are replaced with parameter basis extracted from the user request, and fields without clear basis are left uncompleted.

[0035] Furthermore, the repair function call is re-verified a preset number of times. When the structured error set is empty, the target tool is executed, and the tool execution status, the actual tool return value, and error information are recorded. When errors still exist, the target tool does not exist, or the required parameters lack the basis for the request, the execution of the target tool is stopped and a verification failure status is output.

[0036] Furthermore, in step S5, when the target tool executes successfully and the tool's return value is not empty, a consistency check is performed on the response generated by the small language model based on the tool's return value. The consistency judgment value and the final response result are respectively expressed as:

[0037]

[0038]

[0039] Where 'a' represents the response generated by the small language model based on the tool's return value, 'r' represents the tool's return value obtained when the tool executes successfully, 'Contain' indicates that the response includes the normalized tool's return value, 'Faithful' indicates that the response expresses the tool's return value semantically consistently without changing the facts, 'C(a,r)' represents the consistency judgment value, 'Norm' indicates that the tool's return value is converted into a preset response format, and 'a*' represents the final response result. When the tool fails to execute or the tool's return value is empty, the tool's execution status and error information are retained, and the result is not replaced.

[0040] Furthermore, the present invention also provides a training-outside verification and reliability repair system for function calls of small language models, including a call generation module, a pre-execution verification module, a structured error generation module, a local repair module, a tool execution module, and a result consistency verification module; each module is used in sequence to generate initial function calls, detect errors, record error fields, repair marked fields, obtain real tool return values, and output final function call and response results.

[0041] Furthermore, the method and system can be deployed on a local computer, edge device, or server, and can be connected to a small language model that supports native tool calls, JSON structured output, or text function call output; candidate tools can be dynamically registered in function mode, adding new tools does not require retraining the small language model, and corresponding tool adapters can be added for tools with deterministic parameter extraction requirements.

[0042] Compared with existing technologies, the beneficial effects of this method are: (1) By jointly verifying tool requirements, tool matching, function patterns and request basis before the execution of real tools, it is possible to detect both structural errors and semantic errors of surface structure legality; (2) The validator is independent of the small language model and does not update model parameters, which reduces the cost of training data, computing resources and adaptation of new tools; (3) The verification and repair do not use the standard answer of the test sample, avoiding reliance on evaluation labels to obtain call accuracy.

[0043] Compared with existing technologies, this method also has the following beneficial effects: (4) By using a structured error set and a minimal local repair strategy, only the error fields are modified and the correct fields are retained, reducing the risk of introducing new errors through full retry; (5) A conservative request basis strategy is adopted for unregistered tools to reduce the risk of unfounded parameter filling and over-repair; (6) The return value of the real tool is used to perform consistency verification of the execution result, so that the function call forms a complete reliability closed loop from pre-execution verification, local repair, real execution to the final response. Attached Figure Description

[0044] Figure 1 This is a schematic diagram of the overall process of the training-outside verification and reliable repair method for function calls of small language models according to the present invention.

[0045] Figure 2 This is a schematic diagram of the system structure of the present invention, which includes call generation, pre-execution verification, local repair, tool execution, and result consistency verification.

[0046] Figure 3 This is a schematic diagram of the process of verifying the four dimensions of tool requirements, tool matching, function patterns, and request basis before execution in this invention.

[0047] Figure 4 This is a schematic diagram illustrating the process of structured error-driven minimal local repair and real tool execution in this invention.

[0048] Figure 5 This is a schematic diagram of the process in this invention for verifying the consistency of results based on the return value of the real tool and outputting the final response. Detailed Implementation

[0049] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings. It should be understood that the following embodiments are only for illustrating the technical solutions of this invention and are not intended to limit the scope of protection of this invention.

[0050] This embodiment is designed for a single-round, single-tool function call scenario, deploying a small language model, an external validator, and a tool runtime environment on a local computer. The small language model can be Qwen2.5-1.5B or Llama3.2-1B, or can be replaced with other models that support native tool calls, JSON structured output, or text function call output; the validator and repairer do not perform fine-tuning, backpropagation, or parameter updates on the small language model.

[0051] In this embodiment, the small language model is run via Ollam, with the temperature set to 0, the random seed set to 42, and the context length set to 4096. The above parameters are only one reproducible implementation; in practical applications, they can be adjusted according to computing resources and model interfaces without affecting the technical concepts of pre-execution verification, local repair, execution with real tools, and post-execution result verification of this invention.

[0052] like Figure 1 As shown, this embodiment executes steps S1 to S5 in sequence: generating an initial function call, verifying tool requirements and tool matching, verifying function patterns and request basis, performing minimal local repair based on structured errors and calling the real tool, and verifying and correcting the final response based on the return value of the real tool.

[0053] like Figure 2 As shown, the system in this embodiment includes a call generation module, a pre-execution verification module, a structured error generation module, a local repair module, a tool execution module, and a result consistency verification module. Each module transmits data through structured call objects, structured error sets, repair function calls, tool execution records, and final response records.

[0054] Step S1: Obtain the user request q and the candidate tool set T. Each candidate tool in the candidate tool set T includes at least a tool name, a tool function description, and a function pattern. The function pattern includes parameter name, parameter description, parameter type, required parameter set, enumeration constraints, format constraints, and additional parameter constraints.

[0055] Input the user request and candidate tool set into the small language model to obtain the initial output; extract the tool requirement identifier, tool name, and parameter mapping from the initial output, and normalize them into an initial function call:

[0056]

[0057] Where c0 represents the initial function call, z0 represents the tool requirement identifier, t0 represents the initially selected tool name, and A0 represents the initial parameter mapping. When the initial output contains natural language descriptions, code fences, or incomplete fields, the parsable structured objects are extracted; unparsable fields are treated as null values ​​and their parsing status is recorded, without using evaluation tags to complete the fields.

[0058] Step S2: Perform tool requirement judgment and tool matching verification. First, check whether the user request contains explicit instructions to directly answer, prohibit the use of tools, or not require external tools; if no such instructions are found, calculate the relevance score of each candidate tool relative to the user request, and determine whether there is a target tool that meets the tool requirement threshold.

[0059] The relevance score of the i-th candidate tool relative to the user request q can be expressed as:

[0060]

[0061] Where Ri(q) represents the relevance score of the i-th candidate tool, Kq represents the set of terms requested by the user, Ki represents the set of terms for the name and function description of the i-th candidate tool, Pi represents the set of parameter fields, Ki,p represents the set of terms for the name and description of parameter field p, Ei(q) represents the matching result of the enumerated value or the preset task mode, and α represents the matching weight.

[0062] The candidate tool with the highest relevance score is selected as the target tool. If the initial function call indicates that the tool is not needed and the target tool's score reaches the tool requirement threshold, a tool requirement error is recorded. If the initial tool does not exist in the candidate tool set, or the score difference between the initial tool and the target tool reaches a preset threshold, a tool matching error is recorded.

[0063] like Figure 3 As shown, the pre-execution verification consists of tool requirement verification, tool matching verification, function pattern verification, and request basis verification. The first two are used to determine whether the tool should be called and whether the selected tool is correct, while the latter two are used to determine whether the parameter structure meets the function pattern and whether the parameter values ​​can be supported by the user request.

[0064] Step S3: Perform function pattern constraint verification on the initial parameter mapping. This involves sequentially checking required parameters, strings, integers, numbers, booleans, array and object types, enumeration values, date formats, and additional parameters. For array elements and object properties, recursively verify according to the corresponding nested function patterns and pinpoint errors to specific parameter paths.

[0065] Furthermore, based on the parameter name, parameter description, and registered parameter extraction adapter of the target tool, parameter basis xj(q) is extracted from the user request, and the initial parameter value aj is compared with the parameter basis after numerical or string normalization for equivalence. The request basis determination value of the j-th parameter field is represented as:

[0066]

[0067] Where Gj represents the request basis judgment value of the j-th parameter field, I represents the indicative function, Eq represents the normalized equivalence judgment, and Support indicates whether the user request explicitly supports the parameter value; when Gj is 0, the corresponding parameter field is marked as request basis error.

[0068] This embodiment registers eight tools: calculate, shift_date, lookup_inventory, get_warranty, convert_temperature, text_statistics, schedule_event, and filter_products. These tools are used for numerical calculation, date offsetting, inventory lookup, warranty information lookup, temperature conversion, text statistics, scheduling, and product filtering, respectively. Each tool is configured with a function pattern and can be configured with a deterministic extraction adapter corresponding to its parameter semantics.

[0069] For tools with registered adapters, extract numeric values, dates, strings, boolean values, enumeration values, arrays, or objects from user requests as parameters; for unregistered tools, only use content explicitly associated with parameter names or descriptions, without inferring semantics based on the order of parameter appearance or completing parameters using standard parameters from test samples.

[0070] The four types of verification results are written into the structured error set:

[0071]

[0072] Where E(c0) represents the structured error set corresponding to the initial function call c0, Eneed represents the tool requirement error set, Etool represents the tool matching error set, Eschema represents the function pattern error set, and Eground represents the request basis error set. Each error record includes at least the error type, parameter path, observation, expected constraint, and error reason.

[0073] The k-th structured error record can be represented as:

[0074]

[0075] Where ek represents the k-th error record, typek represents the error type, pathk represents the error field or parameter path, observedk represents the observed call value, constraintk represents the function pattern or the expected constraints given by the request, and reasonk represents the error reason that can be used by the repair module.

[0076] Step S4: Perform minimal local repair based on the structured error set. The repair module only processes the tool requirement identifier, tool name, or parameter field pointed to by the error record, and retains the correct fields that are not marked; when there are multiple candidate repair calls, the call with the smallest field modification distance and which has been verified to be error-free is selected first.

[0077] The minimum local repair objective is expressed as:

[0078]

[0079] Where c* represents the repair function call, C represents the candidate repair call set, D represents the field modification distance of the candidate call relative to the initial function call c0, and V represents the pre-execution verification function for the user request q, function call c, and candidate tool set T.

[0080] Specifically, when there is an error in the tool requirement, the tool requirement identifier is repaired and the calling object is cleared or created accordingly; when there is an error in the tool matching, the target tool is replaced and fields unrelated to the target function pattern are removed; when there is an error in the function pattern, additional parameters are deleted, unambiguous type conversion is performed, and necessary parameters are only added when there is clear evidence in the user request; when there is an error in the request basis, only the conflicting parameter values ​​are replaced.

[0081] like Figure 4 As shown, after completing the local repair, the repair function call is resubmitted to the four-dimensional pre-execution verification. When the structured error set is empty, the tool execution module calls the real tool according to the repaired tool name and parameter mapping; if errors still exist, the target tool does not exist, or the necessary parameters lack evidence, the tool execution is stopped and a verification failure status is output.

[0082] The tool execution module records the execution status, tool return values, and error information. The tool return values ​​are obtained by the actual execution of calculation functions, date functions, query interfaces, conversion functions, statistical functions, or business interfaces, and are not generated internally by the small language model, nor are they read from the standard answers of the test samples.

[0083] Step S5: When the tool executes successfully and its return value is not empty, the actual tool return value is provided to the small language model to generate a natural language response. The result consistency verification module then determines whether the response faithfully contains or expresses the tool's return value. The consistency judgment value is expressed as:

[0084]

[0085] The final response result is expressed as follows:

[0086]

[0087] Where 'a' represents the response generated by the small language model based on the tool's return value, 'r' represents the actual tool's return value, 'Contain' indicates that the response includes the normalized tool's return value, 'Faithful' indicates that the response does not change the facts expressed by the tool's return value, 'C(a,r)' represents the consistency judgment value, 'Norm' indicates that the tool's return value is converted into a preset response format, and 'a*' represents the final response result.

[0088] like Figure 5 As shown, when the consistency judgment value is 1, the small language model response is retained; when the consistency judgment value is 0, the normalized expression of the real tool return value is used to correct the response; when the tool fails to execute or the tool return value is empty, only the execution status and error information are retained, and no result replacement is performed.

[0089] To verify the feasibility of this embodiment, comparative experiments were conducted using two small language models: Qwen2.5-1.5B and Llama3.2-1B. The experiments used the original four-tool main test set, the newly added four-tool extended set, the frozen template external challenge set, the external unseen tool hold-out set, the function pattern stress set, and a three-random seed stability test. All formal inference data and configurations were frozen; standard tool labels, standard parameters, and standard answers were only read by an independent evaluator.

[0090] The comparison methods include: the Native method, which directly executes the model's native calls and uses the model's response; the Native+Result method, which does not fix the function calls but adopts the same result consistency strategy as this invention; the LLM diagnostic retry method, which provides the model with the problems found by the validator and requires the model to regenerate the complete calls; and the VBR-Guard method of this invention.

[0091] In the original four-tool main test, 150 Qwen2.5-1.5B test samples were used; in the cross-model test, 100 Llama3.2-1B samples were used; the newly added four-tool extended test uses 60 samples for each model. End-to-end success requires that the tool requirements, target tool, parameter mapping, actual execution status, and final response all meet the evaluation criteria.

[0092] Table 1 shows the end-to-end success rate of the registration tool under fair outcome consistency control.

[0093]

[0094] As shown in Table 1, in the original four-tool Qwen2.5-1.5B main test, the end-to-end success rates of Native, Native+Result, and LLM diagnostic retry were 66.0%, 72.7%, and 66.7%, respectively, while the present invention achieved 98.7%. In the Llama3.2-1B cross-model test, the present invention achieved 88.0%. In the newly added four-tool test, the present invention achieved 100.0% and 81.7% on the two models, respectively.

[0095] In the above results, Native+Result uses the same real-tool result consistency strategy as the present invention, but does not perform function call repair. In the main test, this method improved from 66.0% to 72.7%, but is still lower than the 98.7% of the present invention, indicating that the performance improvement of the present invention cannot be explained solely by replacing the return value of the real tool; pre-execution verification and local repair have independent effects on overall reliability.

[0096] Further, a frozen template-based challenge set was constructed, containing 200 required tasks for each of the eight registered tools, with 25 tasks for each tool. This challenge set was frozen and its summary value was recorded before formal inference, and the generation of the final natural language answer was disabled. Only exact function calls were evaluated to reduce the impact of the answer generation strategy on the correctness of the calls.

[0097] Table 2 shows the exact function call success rate and false repair rate on the challenge set outside the frozen template.

[0098]

[0099] As shown in Table 2, the success rates of Native, normal retry, LLM diagnostic retry, and the present invention on Qwen2.5-1.5B were 55.0%, 55.5%, 70.5%, and 92.0%, respectively; while the corresponding success rates on Llama3.2-1B were 15.5%, 33.5%, 41.5%, and 77.5%, respectively. The present invention exhibited one false repair on Qwen2.5-1.5B, with a false repair rate of 0.5%, but no false repairs occurred on Llama3.2-1B.

[0100] Table 3 presents the module ablation results on the main test of the original four-tool Qwen2.5-1.5B.

[0101]

[0102] As shown in Table 3, removing request basis verification or partial repair both reduced the end-to-end success rate from 98.7% to 72.7%; removing result consistency verification or function pattern verification both reduced it to 84.0%. This result indicates that request basis verification, function pattern verification, minimal local repair, and result consistency verification together constitute a closed loop for function call reliability.

[0103] In the fairness test, paired significance tests were performed on the registration tool experiments, and global Holm correction was applied. In the original four-tool Qwen2.5-1.5B main test, the corrected p-values ​​of this invention relative to Native, Native+Result, and LLM diagnostic retry were all less than 0.001; in the newly added four-tool Qwen2.5-1.5B test, the corrected p-value of this invention relative to Native+Result was 0.0312.

[0104] In the blind label test, all evaluation labels were randomly replaced for 770 prediction records. The invariance rates of the inference input, validator question list, and final repair call were all 100.0%. This indicates that the verification and repair in this embodiment only use runtime input and tool definitions, and do not rely on standard tool labels, standard parameters, or standard answers.

[0105] In the function pattern stress test, the detection rate and accurate repair rate of registered function patterns were both 100.0%, with a false positive rate of 0.0%. For 100 unseen function patterns, the detection rate was 100.0%, the accurate repair rate was 13.0%, and the false positive rate was 0.0%. The 13.0% repair rate all came from deleting additional fields, indicating that this invention adopts a conservative strategy for unregistered tools, supplementing semantically missing parameters without justification.

[0106] On a holdout set of 80 unseen external tools, the success rates for Qwen2.5-1.5B (Native, Retry, and Invention) were 78.8%, 78.8%, and 77.5%, respectively, while those for Llama3.2-1B were 38.8%, 41.2%, and 43.8%, respectively, with no significant differences. These results limit the applicability of this invention to improving system-level reliability in registered tool environments, and should not be interpreted as an enhancement of the general inference capabilities of small language models or the zero-sample repair capability for arbitrary unseen interfaces.

[0107] In the three-random-seed stability test, the end-to-end success rate of this invention was 98.0% ± 0.0%. In the original four-tool main test, the average number of tokens, model call count, and latency of this invention increased by approximately 2.2%, 8.9%, and 8.7% respectively compared to Native, and the end-to-end success rate improved by 32.7 percentage points, indicating that the increased verification and repair overhead is within a controllable range.

[0108] In summary, this embodiment identifies structural and semantic errors through independent four-dimensional pre-execution verification, reduces field drift caused by full retries through minimal local repair driven by structured errors, prevents unverified calls from entering the real tool through re-verification, and controls the final response through consistency verification of the real tool's return value, thereby forming a function call reliability closed loop that does not update small language model parameters and does not read test standard answers.

[0109] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. All equivalent substitutions, simple modifications, or combinations made in accordance with the claims and description of the present invention should fall within the scope of protection of the present invention. Parts of the present invention not described in detail can be implemented using conventional techniques in the art.

Claims

1. A method for out-of-training validation and reliable repair of function calls in small language models, characterized in that, Includes the following steps: S1: Obtain the user request, the tool name, tool description, and function pattern of the candidate tools, and generate the initial function call using the small language model; S2, perform tool requirement judgment and tool matching verification on the initial function call to determine whether the user request needs to call a tool and whether the selected tool matches the user request; S3, perform function pattern constraint verification and request basis verification on the function parameters in the initial function call to form a structured error set containing error types and error fields; S4. Based on the user request, function pattern, and structured error set, perform partial repair on the call fields with errors, re-verify the repaired function call, and execute the corresponding tool to obtain the tool return value after the verification is passed. S5. Based on the return value of the tool, perform result consistency verification on the response generated by the small language model. If the response does not faithfully contain or express the return value of the tool, use the return value of the tool to correct it, and output the final function call and response result.

2. The method for external validation and reliable repair of function calls for small language models according to claim 1, characterized in that, Step S1 includes the following steps: S11, Each candidate tool is represented as a structured tool object consisting of a tool name, a tool function description, and a function pattern. The function pattern includes at least parameter name, parameter type, required parameter set, enumeration constraint, format constraint, and additional parameter constraint. S12, the initial function call is normalized into a call object consisting of tool requirement identifier, tool name and parameter mapping; S13, the function patterns of the user request and candidate tools are used as the basis for verification and repair. During the verification and repair process, the parameters of the small language model are not updated, and the standard function calls or standard answers of the test samples are not read.

3. The training-outside verification and reliability repair method for function calls of small language models according to claim 1, characterized in that, Steps S2 and S3 constitute a pre-execution verification process, which includes: Verify the tool requirement identifier based on the direct answer instructions in the user request and the relevance between the candidate tools and the user request; The candidate tools are sorted by relevance based on their tool name, tool description, and parameter description. The existence of the tool name in the initial function call is verified, and it is verified whether the tool name in the initial function call points to a candidate tool whose relevance meets the preset conditions. Verify parameter integrity, parameter type, enumeration values, parameter format, and additional parameters based on the function mode of the selected tool; Extract explicit evidence corresponding to the parameter fields from the user request, and perform an equivalence comparison between the explicit evidence and the parameter values ​​in the initial function call.

4. The training-outside verification and reliability repair method for function calls of small language models according to claim 3, characterized in that, The relevance score of the candidate tool is calculated as follows: Where Ri(q) represents the relevance score of the i-th candidate tool relative to the user request q, Kq represents the lexical set of the user request, Ki represents the lexical set of the tool name and tool description of the i-th candidate tool, Pi represents the parameter field set of the i-th candidate tool, Ki,p represents the lexical set of the name and description of parameter field p, Ei(q) represents the matching result of the enumerated value or preset task mode in the user request, and α represents the matching weight; the candidate tool with the highest score is selected as the target tool. When the maximum relevance score reaches the preset threshold and the user request does not contain a direct answer instruction, it is determined that the tool needs to be called.

5. The training-outside verification and reliability repair method for function calls of small language models according to claim 3, characterized in that, The function pattern constraint verification includes the following steps: S31, Check whether each parameter in the required parameter set is included in the parameter mapping; S32 checks whether the parameter value meets the constraints of string, integer, number, boolean, array or object type, and recursively verifies array elements and object properties; S33, check whether the parameter values ​​satisfy the enumeration constraints and date format constraints; S34, when the function pattern prohibits additional parameters, detect and mark undefined parameter fields in the function pattern.

6. The method for external validation and reliable repair of function calls for small language models according to claim 3, characterized in that, The request basis verification includes: extracting parameter basis mapping from the user request according to the function pattern of the target tool, and calculating a basis judgment value for each parameter field: Where Gj represents the basis judgment value of the j-th parameter field, I represents the indicative function, aj represents the j-th parameter value in the initial function call, xj(q) represents the basis of the j-th parameter extracted from the user request q, Eq represents the equivalence judgment after numerical normalization or string normalization, and Support indicates whether the user request explicitly supports the corresponding parameter value; when Gj is 0, the corresponding parameter field is marked as request basis error. For tools with registered parameter extraction adapters, the parameter basis is determined from the user request according to the adapter; for unregistered tools, only numeric, string, boolean, or enumerated values ​​explicitly associated with the parameter name or parameter description are used as the parameter basis.

7. The method for external validation and reliable repair of function calls for small language models according to claim 1, characterized in that, The structured error set and local repair satisfy: Where c0 represents the initial function call, E(c0) represents the structured error set, Eneed, Etool, Eschema and Eground represent tool requirement error, tool matching error, function pattern error and request basis error, respectively, C represents the candidate repair call set, D represents the field modification distance of the repair call relative to the initial function call, V represents the pre-execution verification function for user request q, function call c and candidate tool set T, and c* represents the repair function call with the smallest field modification distance under the constraint that the verification error set is empty.

8. The method for external validation and reliable repair of function calls for small language models according to claim 7, characterized in that, The local repair includes the following steps: S41, when there is a tool requirement error and the user request contains a direct answer instruction, clear the tool name and parameter mapping; when there is a tool requirement error and the relevant candidate tools meet the preset conditions, fix the tool requirement identifier to indicate that a tool needs to be called. S42, when there is a tool matching error, the tool name is replaced with the target tool whose relevance meets the preset conditions, and the parameter mapping corresponding to the target tool is reconstructed according to the user request; S43, when a function pattern error exists, delete undefined parameters, normalize the types of convertible parameter values, and supplement missing parameters only when the corresponding parameter reference exists in the user request; S44, when there is a request basis error, only replace the conflicting parameters with the parameters extracted from the user request, and retain the call fields that are not marked by the structured error set; S45 performs a preset number of re-verifications on the repaired function call. If errors still exist or parameter information cannot be obtained, the corresponding tool is stopped and a verification failure message is output.

9. The method for external validation and reliable repair of function calls for small language models according to claim 1, characterized in that, The consistency check of the result in step S5 satisfies: Where 'a' represents the response generated by the small language model based on the tool's return value, 'r' represents the tool's return value obtained when the tool executes successfully, 'Contain' indicates that the response includes the normalized tool's return value, 'Faithful' indicates that the response expresses the tool's return value semantically consistently without changing the facts, 'C(a,r)' represents the consistency judgment value, 'Norm' indicates that the tool's return value is converted into a preset response format, and 'a*' represents the final response result. When the tool fails to execute or the tool's return value is empty, the tool's execution status and error information are retained, and the result is not replaced.

10. A training-outside validation and reliability repair system for function calls of small language models, characterized in that, It includes a call generation module, a pre-execution verification module, a structured error generation module, a local repair module, a tool execution module, and a result consistency verification module; The generation module is used to obtain user requests, candidate tool descriptions, and function patterns, and then calls the small language model to generate initial function calls. The pre-execution verification module is used to determine tool requirements, verify tool matching, verify function pattern constraints, and verify the basis of requests. The structured error generation module is used to record error types and error fields; The local repair module is used to repair the marked call fields based on the user request, function pattern, and structured error set, while retaining the unmarked call fields; The tool execution module is used to execute verified function calls and obtain the tool's return value; The result consistency verification module is used to verify whether the response generated by the small language model faithfully contains or expresses the return value of the tool, and outputs the final function call and response results.

Images