Large model feedback optimization system and method based on process identification and tool calling

Abstract

The invention provides a large model feedback optimization system and method based on process identification and tool calling. The method comprises the following steps: S1, converting received input data into semantic vectors and inputting the semantic vectors into a large model; s2, enabling the large model to output an intention label, and carrying out semantic matching on the intention label and a preset business flow chart; s3, according to a semantic matching result and a task node sequence of the flow chart, gradually scheduling and executing a corresponding tool function; s4, recording the behavior of each time of tool calling, and forming a tool calling track chain; s5, evaluating the rationality and correctness of the tool calling track chain based on a reward model, and outputting an evaluation result; and S6, finishing fine adjustment of the main model according to the evaluation result. According to the method, the user intention can be automatically identified, the tool call chain can be called, the model is optimized in combination with the execution result and the evaluation result of the reward model, and illusion problems such as false call and assume execution of a large model are effectively reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a large model feedback optimization system and method based on process identification and tool calling, and especially to a large model general execution system based on process identification and tool calling and a feedback optimization method thereof. BACKGROUND

[0002] With the rapid development of large models such as GPT, Claude, and Wenxin Yanyan, they have shown great capabilities in text generation, image recognition, and language understanding. Currently, there are various large model integration solutions that can support users to drive tasks through natural language, such as Agent systems, plug-in systems, and tool functions (function calling). For example, OpenAI's Function Calling can structure the model output to map to API interface parameters, thereby realizing task execution. However, most of these systems are still in the "single round response" or "static task calling" stage, lacking recognition, arrangement, and feedback training mechanisms for complex task processes.

[0003] The current combination of large models and tool systems is mostly based on natural language single round triggering API, which cannot identify whether the user's intention belongs to a certain business process, nor can it dynamically arrange multi-step tasks. After the function calling is successful or fails, the large model usually cannot know whether the execution is correct, and also lacks a mechanism to retrain the execution results as rewards, so it is easy to produce "hallucinations" or inaccurate calls. The matching of complex business processes relies on hard coding or manual rules, lacking a general, efficient, and extensible intent-process matching mechanism, making it difficult to adapt to general task expansion needs.

[0004] Patent document CN119440539A discloses a multi-round reasoning and feedback optimization method and system for industrial software code, which introduces an automated verification mechanism to perform logical consistency checks and performance evaluations on generated and optimized code to ensure code correctness and execution efficiency. However, the technical problems addressed by this patent document are different from those addressed by the present application, and the technical means used are also different. SUMMARY

[0005] In view of the defects in the prior art, the purpose of the present application is to provide a large model feedback optimization system and method based on process identification and tool calling.

[0006] According to the large model feedback optimization system based on process identification and tool calling provided by the present application, it comprises: a multi-modal instruction analysis module, configured to receive input data, and convert the input data into a semantic vector through a multi-modal encoder and input the semantic vector into a large model; An intent recognition and process matching module enables the large model to output an intent label according to the semantic vector and perform semantic matching with a preset business process graph; A process execution and tool scheduling module schedules and executes corresponding tool functions step by step according to the semantic matching result and the task node sequence of the process graph; An execution record module is configured to record the behavior of each tool call to form a tool call track chain; A reward model evaluation module is configured to evaluate the rationality and correctness of the tool call track chain based on a reward model and output an evaluation result; A model optimization training module is configured to fine-tune the main model according to the evaluation result.

[0007] Preferably, the intent recognition and process matching module comprises: The intent label is matched with the preset business process graph, and if the process is hit, the process execution is triggered; if not, the large model is enabled to autonomously determine whether to call one or more tools to complete the task.

[0008] Preferably, the input data comprises text data, voice data and image data input by a user; and the execution record module records the context of the original input by the user, the name of the tool actually called, the parameters, the calling time and the return result, and the sequence and dependency relationship between multiple calls.

[0009] Preferably, the reward model adopts a method comprising contrast learning or reinforcement learning to judge the rationality and correctness of the entire tool call track chain; if the output evaluation result is correct, it is determined that the calling path is reasonable and the result meets the user's demand; if the output evaluation result is incorrect, it is determined that the calling sequence is chaotic, the parameters are incorrect, and the tool selection is incorrect.

[0010] Preferably, the model optimization training module takes the evaluation result of the reward model as a supervision signal, combines the original input and the calling track to construct a training sample for fine-tuning the main model.

[0011] According to the large model feedback optimization method based on process recognition and tool calling provided by the application, the input data received is converted into a semantic vector by a multi-modal encoder and input into a large model. Step S1: The received input data is converted into a semantic vector by a multi-modal encoder and input into a large model; Step S2: The large model outputs an intent label according to the semantic vector and performs semantic matching with a preset business process graph; Step S3: The tool functions are scheduled and executed step by step according to the semantic matching result and the task node sequence of the process graph; Step S4: The behavior of each tool call is recorded to form a tool call track chain; Step S5: evaluating the rationality and correctness of the tool calling trajectory chain based on the reward model, and outputting the evaluation result; Step S6: fine-tuning the main model according to the evaluation result.

[0012] Preferably, the step S2 comprises: The intention label is semantically matched with the preset business process diagram, if the process is hit, the process execution is triggered, if not, the large model is used to judge whether to call one or more tools to complete the task.

[0013] Preferably, the input data comprises text data, voice data and image data input by the user; and the content recorded in the step S4 further comprises the original input context of the user, the tool name actually called, the parameters, the calling time, the returned result and the sequence and dependency relationship between multiple calls.

[0014] Preferably, the reward model uses a method comprising contrast learning or reinforcement learning to judge the rationality and correctness of the entire tool calling trajectory chain; if the output evaluation result is correct, it is determined that the calling path is reasonable and the result meets the user's demand; if the output evaluation result is incorrect, it is determined that the calling sequence is chaotic, the parameters are incorrect, and the tool selection is incorrect.

[0015] Preferably, the step S6 comprises: The evaluation result of the reward model is used as a supervision signal, combined with the original input and the calling trajectory, to construct a training sample for fine-tuning the main model.

[0016] Compared with the prior art, the present application has the following beneficial effects: 1. The present application can automatically identify user intentions and mobilize tool calling chains by using a large model to automatically match user instructions and business process diagrams, and can optimize the model by combining the execution result and the evaluation result of the reward model, thereby effectively reducing the illusion problems of large model incorrect calling and arbitrary execution.

[0017] 2. The present application fine-tunes the main model by recording the tool calling trajectory and using the evaluation result of the reward model as a supervision signal, and further improves the accuracy of process recognition and tool calling; generates a reward signal according to the execution result, so that the large model has a rectification ability and can be continuously self-optimized; at the same time, it can support input forms such as voice, image and text, greatly improving the flexibility and intelligence of human-computer interaction.

[0018] 3. The process diagram in the present application can be configured, and the tool calling interface can be registered, which can be applied to multiple industries and multiple business scenarios, significantly improving the universality and practicality of the large model application system. BRIEF DESCRIPTION OF DRAWINGS

[0019] Other features, objects, and advantages of the application will become more apparent from the following detailed description when read in conjunction with the accompanying drawings: Figure 1 Figure 1 is a system architecture diagram of the present application.

[0020] Figure 2 Figure 2 is a flowchart of the method of the present application. DETAILED DESCRIPTION

[0021] The present application will be described in detail below with reference to specific embodiments. The following examples will help those skilled in the art to further understand the present application, but in no way limit the present application. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of changes and improvements can be made. These are within the scope of the present application.

[0022] Referring to Figure 1 As shown in the figure, a large model feedback optimization system based on process recognition and tool calling includes: A multi-modal instruction analysis module: receives user input in the form of text, voice, image, etc., which greatly improves the flexibility and intelligence of system human-computer interaction; a multi-modal encoder is used to convert it into a semantic vector, and input into a large model. The input data of different modalities is first processed by the corresponding feature extraction encoder, for example: 1) Text input obtains semantic vector representation vtv_tvt via a text encoder (such as a Transformer encoding layer); 2) Speech input obtains speech feature vector vsv_svs via an acoustic feature extraction model (such as Wav2Vec2.0 or Whisper Encoder); 3) Image input obtains image feature vector viv_ivi via a visual feature extraction network (such as CLIP-ViT module).

[0023] Then map the above different modal features to a unified semantic space, fuse them through a multi-modal alignment network (Multi-modal Alignment Network), and obtain a unified semantic vector VVV, the calculation process of which can be represented as: V=falign(Wt*Vt+Ws*Vs+Wi*Vi) Where Wt, Ws, Wi are mapping matrices for different modalities, and falign(·) is a nonlinear fusion function (such as a multi-layer perception or attention weighted fusion layer). The fused unified semantic vector VVV is input into the large model for subsequent intent recognition and process matching.

[0024] The intent recognition and process matching module: the large model outputs an intent label according to the semantic vector, and performs semantic matching with the preset business process graph. If the process is hit, the process is triggered for execution; if it is not hit, the large model needs to independently judge whether to call one or more function tools to complete the task, and form a tool calling chain.

[0025] After the large model receives the semantic vector V, it outputs a set of candidate intent label set {I1, I2, …, In} through the intent classification head, and each intent label corresponds to a business target. The system performs semantic similarity calculation on the candidate intent and the process label set {F1, F2, …, Fm} in the preset business process library: Sij=cos(Embed(Ii),Embed(Fj)) Where cos(·) represents the cosine similarity function, and Embed(·) is the semantic embedding of the process node or process title. If there is a process Fk that satisfies the similarity Sik>δ (threshold), it is determined that the process is hit, and the process execution phase is entered.

[0026] If it is not hit (i.e. all Sij≤δ), the large model enters an adaptive judgment mode and decides whether to call a tool (function) through the following steps: Task decomposability judgment: analyze whether the input semantic vector V contains executable task actions (such as “calculate”, “generate”, “analyze” and other verb triggers). Tool availability matching: search for the most relevant tool description embedding fk in the registered function library, and calculate: Rk=cos(V,fk) If the maximum similarity Rk> (the calling threshold), a tool calling chain is generated.

[0027] Decision basis: the model makes a comprehensive judgment according to the historical reward model feedback weight (reward confidence), i.e. if the success rate of function calling for the same type of task in history is high, function calling is preferred; otherwise, direct model generation is attempted. Through the above mechanism, the large model still has certain self-determination ability when the process is not hit, ensuring the continuous execution of the task.

[0028] The present application realizes the automatic matching of user instructions and business processes, supports multi-step task automatic execution, and has high process perception ability. Among them, the process graph is configurable, the function calling interface is registered, and it can be applied to multiple industries and multiple business scenarios, and has high scalability.

[0029] Process Execution and Tool Scheduling Module: After a process is hit, the system triggers the corresponding function tools step by step according to the task node sequence in the flowchart. The call parameters of each function are filled by the large model, and the context information before and after the call is recorded.

[0030] Execution logging module: Whether the function is called step-by-step after the process is hit, or the model makes the call autonomously, every tool call is recorded in detail. The records include the user's original input context; the actual function name, parameters, call time, and return result; and the order and dependencies between multiple calls. This logging process is automated by the backend logging system, forming a complete "tool call trajectory chain."

[0031] Reward Model Evaluation Module: The system inputs the call trajectory chain, the user's original question, and the context into the reward model. The reward model uses methods such as contrastive learning or reinforcement learning to judge the rationality and correctness of the entire tool call sequence and outputs the evaluation results. Correct: The call path is reasonable, and the result meets the user's needs; Errors: Incorrect call order, incorrect parameters, wrong tool selected.

[0032] The inputs to the reward model include: 1) User's original input semantic vector V; 2) The actual sequence of functions called [f1, f2, ..., fn]; 3) Input and output parameters of each function; 4) The degree to which the final output matches the user's needs.

[0033] The reward model outputs a score R∈[0,1], representing the rationality of the calling path. Its calculation method is as follows: R=σ(Wr*[Ev;Ef;Eo]) in: Ev represents the input semantic embedding; Ef is the temporal code of the function sequence (which can be extracted by Transformer Encoder); Eo is the semantic vector of the final output; σ(·) is the Sigmoid function; Judgment criteria are set based on the scoring results: When R ≥ 0.8: it is judged as "correct", indicating that the calling order and parameters are reasonable; When R < 0.8: it is judged as "error", indicating that there are problems such as incorrect tool selection, incorrect parameters, or disordered order.

[0034] The reward model can be optimized by contrast learning or reinforcement learning with human feedback (RLHF) to make the evaluation results closer to human judgment standards.

[0035] Model optimization training module: use the reward model evaluation results as a supervision signal, combine the original input and the call trajectory to build training samples for fine-tuning the main model, and achieve the following optimization goals: improve process recognition ability; enhance the accuracy of function call timing and parameter decision; reduce hallucination probability; improve multi-step task execution success rate.

[0036] After the main model obtains the evaluation signal of the reward model, it stores the context of each function call, the trajectory chain and the evaluation score to form a training sample set.

[0037] Each sample contains: Sample={V,[f1,…,fn],[p1,…,pn],[r1,…,rn],R} The system fine-tunes the training through the following optimization goals: L= R logPθ (f1,…,fn∣V) Where: L is the loss function; Pθ(·) is the function call probability under the main model parameter θ; R is the reward signal, guiding the model to optimize in the direction of higher reward.

[0038] The present application generates reward signals through function execution results, enabling the large model to obtain correction ability and continuously optimize itself; at the same time, it can combine process specifications, call result verification and training feedback to effectively reduce hallucination problems such as large model error call and conjectural execution, and has good practicality.

[0039] Referring to Figure 2 A large model feedback optimization method based on process identification and tool calling, comprising: Step S1: convert the received input data into a semantic vector through a multi-modal encoder, and input it into the large model; Step S2: make the large model output an intent label according to the semantic vector, and perform semantic matching with the pre-set business process diagram; Step S3: according to the semantic matching result and the task node sequence of the process diagram, gradually schedule and execute the corresponding tool function; Step S4: record the behavior of each tool invocation, form a tool invocation track chain; Step S5: evaluate the rationality and correctness of the tool invocation track chain based on the reward model, and output the evaluation result; Step S6: fine-tune the main model according to the evaluation result.

[0040] Those skilled in the art know that, in addition to implementing the system provided by the present application and each device, module, unit thereof in the form of pure computer readable program code, the system provided by the present application and each device, module, unit thereof can also be implemented in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers and embedded microcontrollers, etc. to achieve the same function by logically programming the method steps. Therefore, the system provided by the present application and each device, module, unit thereof can be considered as a hardware component, and the devices, modules, units included therein for implementing various functions can also be considered as structures within the hardware component; the devices, modules, units for implementing various functions can also be considered as both software modules implementing methods and structures within hardware components.

[0041] The specific embodiments of the present application are described above. It needs to be understood that the present application is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essential content of the present application. The embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily without conflict.

Claims

1. A large model feedback optimization system based on process identification and tool invocation, characterized in that, Comprise: A multi-modal instruction analysis module for receiving input data, converting it into a semantic vector through a multi-modal encoder, and inputting it into a large model; An intent recognition and process matching module that causes the large model to output an intent label based on the semantic vector and perform semantic matching with a pre-set business process diagram; A process execution and tool scheduling module that schedules and executes corresponding tool functions step by step based on the semantic matching results and the task node order of the process diagram; An execution record module for recording the behavior of each tool call to form a tool call trajectory chain; A reward model evaluation module that evaluates the rationality and correctness of the tool call trajectory chain based on a reward model and outputs an evaluation result; A model optimization training module that fine-tunes the main model based on the evaluation result.

2. The large model feedback optimization system based on process identification and tool invocation of claim 1, wherein, The intent recognition and process matching module comprises: Semantic matching of the intent label with the pre-set business process diagram, if the process is hit, trigger process execution; if not, let the large model autonomously determine whether to call one or more tools to complete the task.

3. The large model feedback optimization system based on process identification and tool invocation of claim 1, wherein, The input data includes user input text data, voice data, and image data; the content recorded by the execution record module includes user original input context, actual called tool name, parameters, call time and return result, and the order and dependency relationship between multiple calls.

4. The large model feedback optimization system based on process identification and tool invocation of claim 1, wherein, The reward model uses methods including contrast learning or reinforcement learning to judge the rationality and correctness of the entire tool call trajectory chain; if the output evaluation result is correct, it is determined that the call path is reasonable and the result meets the user's demand; if the output evaluation result is incorrect, it is determined that the call order is chaotic, the parameters are incorrect, and the tool selection is incorrect.

5. The large model feedback optimization system based on process identification and tool invocation of claim 1, wherein, The model optimization training module uses the evaluation result of the reward model as a supervision signal, combines the original input and the call trajectory to construct a training sample for fine-tuning the main model.

6. A large model feedback optimization method based on process identification and tool invocation, characterized in that, Comprise: Step S1: converting the received input data into a semantic vector through a multi-modal encoder and inputting it into a large model; Step S2: causing the large model to output an intent label based on the semantic vector and performing semantic matching with a pre-set business process diagram; Step S3: scheduling and executing corresponding tool functions step by step based on the semantic matching results and the task node order of the process diagram; Step S4: recording the behavior of each tool call to form a tool call trajectory chain; Step S5: evaluating the rationality and correctness of the tool call trajectory chain based on a reward model and outputting an evaluation result; Step S6: fine-tuning the main model based on the evaluation result.

7. The large model feedback optimization method based on process identification and tool invocation according to claim 6, characterized in that, The step S2 comprises: Semantic matching of the intent label with the pre-set business process diagram, if the process is hit, trigger process execution; if not, let the large model autonomously determine whether to call one or more tools to complete the task.

8. The large model feedback optimization method based on process identification and tool invocation according to claim 6, characterized in that, The input data includes user input text data, voice data, and image data; the content recorded by the step S4 also includes user original input context, actual called tool name, parameters, call time and return result, and the order and dependency relationship between multiple calls.

9. The large model feedback optimization method based on process identification and tool invocation according to claim 6, characterized in that, The reward model adopts a method including contrast learning or reinforcement learning to judge rationality and correctness of the entire tool calling track chain; if the output evaluation result is correct, it is determined that the calling path is reasonable and the result meets the user's demand; if the output evaluation result is incorrect, it is determined that the calling sequence is chaotic, the parameters are incorrect, and the tool selection is incorrect.

10. The large model feedback optimization method based on process identification and tool invocation of claim 6, wherein, The step S6 comprises: The evaluation result of the reward model is used as a supervision signal, combined with the original input and the calling track, to construct a training sample for fine-tuning the main model.

Images