Task processing method and electronic equipment

Abstract

The invention discloses a task processing method and electronic equipment, and relates to the technical field of artificial intelligence. The task processing method comprises the following steps: in response to an obtained target task, calling a first type of tool through an intelligent agent, and determining first knowledge meeting an association condition with the target task from a knowledge base; based on the target task and the first knowledge, reasoning through a processing model to generate first processing information corresponding to the target task; and generating second knowledge based on the target task and the first processing information, wherein the second knowledge can be used for the intelligent agent to update the knowledge base or update model parameters of the processing model.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a task processing method and an electronic device. Background Technology

[0002] Current general-purpose intelligent agents are generally static, meaning that the basic large model used by the intelligent agent and its internal knowledge base are statically configured in the early stages of development. This approach cannot adapt to dynamic external changes. When the external environment in which the intelligent agent interacts changes, it will affect the intelligent agent's ability to solve tasks, leading to an increase in task failure rate. Therefore, how to solve or mitigate this problem has become a research hotspot in this field. Summary of the Invention

[0003] Therefore, this application discloses the following technical solution:

[0004] A task processing method, comprising:

[0005] In response to obtaining the target task, the agent invokes a first type of tool to determine first knowledge from the knowledge base that satisfies the association conditions with the target task;

[0006] Based on the target task and the first knowledge, reasoning is performed through a processing model to generate first processing information corresponding to the target task;

[0007] Based on the target task and the first processing information, second knowledge is generated. The second knowledge can be used by the agent to update the knowledge base or update the model parameters of the processing model.

[0008] Optionally, the first processing information includes the processing result and the processing path;

[0009] Generate second knowledge based on the target task and the first processed information, including:

[0010] Based on the target task, the processing results, and the processing path, second knowledge is generated through the processing model.

[0011] Optionally, the task processing method further includes:

[0012] The agent invokes a second type of tool to update the knowledge base based on the second knowledge.

[0013] Optionally, generating second knowledge based on the target task and the first processing information includes:

[0014] A reward signal is generated based on the first processed information;

[0015] The model parameters of the processing model are adjusted based on the reward signal;

[0016] Based on the target task and the first processing information, second knowledge is generated by the processing model after the model parameters are adjusted.

[0017] Optionally, the task processing method further includes:

[0018] Based on the second knowledge, the processing model is updated by the intelligent agent, and the knowledge base is updated by calling the second type of tool.

[0019] Optionally, based on the second knowledge, the processing model is updated by the intelligent agent, and the knowledge base is updated by calling the second type of tool, including:

[0020] Based on the second knowledge, a set of relative strategies is used to optimize the trainer and perform target processing in an iterative manner to at least enhance the knowledge generation capability of the processing model; the target processing includes performing task reasoning on the target task through the processing model, updating the parameters of the processing model based on the task reasoning, and generating knowledge based on the updated processing model;

[0021] The knowledge base is updated based on the third knowledge used in the target round processing during the iteration process;

[0022] Wherein, the target round processing and the last round processing in the iteration process satisfy the proximity condition, and the third knowledge is the knowledge generated by the processing model based on the second knowledge during the iteration process.

[0023] Optionally, based on the second knowledge, the trainer is optimized using a set of relative policies to perform target processing iteratively, including:

[0024] Perform sampling processing to obtain the current sampling data; the sampling data includes the target task, as well as the latest knowledge generated during the iteration process of the second knowledge or the processing model based on the second knowledge;

[0025] The sampled data is input into the current processing model to obtain the second processing information of the target task generated by the processing model;

[0026] A reward signal is generated based on the second processed information, and the model parameters of the processing model are adjusted based on the generated reward signal;

[0027] Based on the target task and the second processing information, knowledge is generated using the current processing model with adjusted model parameters. The generated knowledge is then used to execute the sampling process until the termination condition is met, at which point the processing ends.

[0028] Optionally, the task processing information corresponding to the target task is the first processing information;

[0029] Alternatively, the task processing information corresponding to the target task may be the third processing information, which is the processing information of the target task generated by the processing model at the end of the iteration.

[0030] Optionally, the second knowledge is generated based on the target task and the first processing information, including at least one of the following:

[0031] The task requirement description in the target task is parameterized to obtain the parameterization result of the task requirement description.

[0032] Based on the target task and the first processing information, determine the tool usage information of the tools used in the task processing process;

[0033] Based on the target task and the first processing information, determine the processing steps that effectively solve the task during the task processing process;

[0034] Based on the target task and the first processing information, determine whether the target task was successfully executed, and record the target task and the first processing information as corresponding success or failure examples;

[0035] The second knowledge includes at least one of the parameterization results, the tool usage information, the processing steps, and the success or failure examples.

[0036] An electronic device, comprising:

[0037] Memory, used to store at least one set of computer instructions;

[0038] A processor is configured to perform the following processing by executing a set of instructions stored in the memory:

[0039] In response to obtaining the target task, the agent invokes a first type of tool to determine first knowledge from the knowledge base that satisfies the association conditions with the target task;

[0040] Based on the target task and the first knowledge, reasoning is performed through a processing model to generate first processing information corresponding to the target task;

[0041] Based on the target task and the first processing information, second knowledge is generated. The second knowledge can be used by the agent to update the knowledge base or update the model parameters of the processing model.

[0042] A storage medium carrying one or more computer instruction sets, which, when executed by an electronic device, enable the electronic device to perform any of the task processing methods provided above. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0044] Figure 1 This is a flowchart illustrating one of the task processing methods provided in this application;

[0045] Figure 2 This is another flowchart illustrating the task processing method provided in this application;

[0046] Figure 3 This is a schematic diagram illustrating how the intelligent agent provided in this application summarizes and reuses historical experience knowledge.

[0047] Figure 4 This is another flowchart illustrating the task processing method provided in this application;

[0048] Figure 5 This is a schematic diagram of the target processing performed iteratively using the GRPO trainer provided in this application;

[0049] Figure 6 This is a structural diagram of the task processing device provided in this application;

[0050] Figure 7 This is a structural diagram of the electronic device provided in this application. Detailed Implementation

[0051] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0052] This application provides a task processing method and an electronic device, wherein the provided method can be applied to electronic devices in a variety of general or special computing device environments or configurations, such as personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor devices, etc.

[0053] See Figure 1 The flowchart shown illustrates the task processing method provided in this application embodiment, which may include the following steps 101 to 103, which are described in detail below.

[0054] Step 101: In response to obtaining the target task, the agent invokes a first type of tool to determine first knowledge from the knowledge base that satisfies the association conditions with the target task.

[0055] Optionally, the target task is a task proposed by the user to be performed by the intelligent agent. It can be, but is not limited to, automated daily operation tasks, question-and-answer tasks, decision-making tasks, or related tasks for solving complex professional problems. There are no restrictions on this, depending on the actual application scenario.

[0056] The target task includes a task requirement description, which, optionally, is in natural language form.

[0057] Intelligent agents can be embodied intelligent agents with physical entities, such as intelligent robots, or non-embodied intelligent agents without physical entities, such as intelligent assistants and other software programs; there are no restrictions on either.

[0058] A knowledge base contains at least one piece of knowledge, which can also be referred to as "memory" or "experience memory." The core function of the knowledge base is to provide external knowledge support for model-based task processing by intelligent agents, overcoming the limitations of the model's own knowledge and providing rich contextual information for the model's task processing, thereby improving the quality and efficiency of the model's task processing.

[0059] In response to obtaining the target task, the agent can search the knowledge base based on the target task to determine the first knowledge that meets the association conditions with the target task. This facilitates the subsequent use of the first knowledge as contextual information of the target task, which is then input into the processing model along with the target task for task reasoning.

[0060] The association condition can be, but is not limited to, being set to a semantic similarity between the target task and the corresponding knowledge in the knowledge base reaching a predetermined threshold.

[0061] Current general-purpose intelligent agents are typically static, meaning their underlying basic model and internal knowledge base are statically configured during the initial development phase. This approach cannot meet dynamically changing application requirements; for example, the agent cannot adapt to changes in the interactive environment, external knowledge, or user requests. To address this issue, this application proposes a technical concept whereby an intelligent agent "summarizes, learns, and internalizes new information based on its interaction with the external environment, thereby achieving self-evolution." In this application, during the agent's self-evolution process, the agent can update its knowledge base based on the summarized or learned new knowledge, such as inserting new knowledge or modifying existing knowledge, to expand or optimize the knowledge base, thus achieving self-evolution of the knowledge within it.

[0062] To address the various operational needs of intelligent agents in accessing knowledge bases, such as searching and retrieving knowledge, inserting knowledge, and modifying knowledge, embodiments of this application design a series of operational tools that can be used by intelligent agents to operate on knowledge bases. These operational tools include a first type of tool and / or a second type of tool. The first type of tool can be simply referred to as a "reading" tool, which the intelligent agent can call to perform various operations such as searching / matching and retrieving knowledge from the knowledge base. The second type of tool can be simply referred to as a "writing" tool, which the intelligent agent can call to perform various operations such as inserting, modifying, and replacing knowledge from the knowledge base.

[0063] Based on this, in this step, in response to obtaining the target task, the agent can call the first type of tool to search the knowledge base, so as to determine and read the first knowledge that meets the association conditions with the target task from the knowledge base.

[0064] Optionally, each piece of knowledge in the knowledge base is stored in both natural language and vector form. That is, for each piece of knowledge, the knowledge base stores both its natural language data information and vector information. When determining the first piece of knowledge that satisfies the association condition with the target task, the first type of tool, such as the "reading" tool, first converts the target task into a vector to obtain a task vector. Then, it matches the task vector with the vectors of each piece of knowledge in the knowledge base. Knowledge whose vector similarity to the task vector reaches a threshold is identified as the first piece of knowledge that satisfies the association condition with the target task. The natural language data information of this first piece of knowledge is then read from the knowledge base so that it can be used as contextual information for the target task to assist the model's task reasoning process.

[0065] Vector similarity is one option for calculating semantic similarity. In natural language processing, semantic similarity measures the degree of semantic closeness between words or texts, while vector similarity indirectly assesses semantic closeness by representing semantics as vectors and calculating the similarity of these vectors.

[0066] Step 102: Based on the target task and the first knowledge, reason through the processing model to generate the first processing information corresponding to the target task.

[0067] After obtaining the first knowledge that satisfies the association conditions with the target task, the agent can call the processing model and input the first knowledge as context information of the target task along with the target task into the processing model. The processing model will then perform task reasoning based on the input target task and the first knowledge, and generate first processing information corresponding to the output target task based on the reasoning.

[0068] The processing model can be a large or small AI (Artificial Intelligence) model. The large model can be a general-purpose large model, such as a general-purpose large language model (LLM), a visual large model, a multimodal large model, etc., or it can be an expert large model obtained by fine-tuning a general-purpose large model based on requirements. This application does not limit this. The large model learns the features and patterns of the input information by training on a large amount of diverse data. It typically has hundreds of millions to hundreds of billions of model parameters (such as the weights corresponding to the model network layers, used to control model behavior), and can capture complex relationships and patterns in the input information for accurate and efficient reasoning / computation.

[0069] Furthermore, the processing model can be a local model built into the agent, or it can be an external model such as a cloud model; there are no restrictions on this.

[0070] When the processing model is a large model, prompt words for the processing model can be constructed first based on the target task and the first knowledge. Then, the prompt words are input into the processing model for model reasoning to generate the first processing information corresponding to the target task.

[0071] The constructed prompts include the target task and first knowledge. In addition, they may include, but are not limited to, instructions on related tasks for the large model, such as role instructions for the large model and instructions on what processing the large model should perform based on the indicated role.

[0072] The first processing information corresponding to the target task includes at least the processing result of the target task. For example, if the target task is a question-and-answer task, the first processing information includes at least the answer information to the user's question (target task). If the target task is a decision-making task, the first processing information includes at least the decision result corresponding to the task to be decided. If the target task is an operation task, the first processing information may include at least the operation information of the operation to be performed (such as operation type, operation location / time, etc.).

[0073] In addition, optionally, the first processing information corresponding to the target task may include, but is not limited to, a processing path. The processing path is used to explain the processing procedure on which the processing result of the target task is based, and may include, but is not limited to, the workflow for solving the task (such as breaking down the target task into multiple sub-tasks, and the processing steps for each sub-task and the processing tools used in each step), or the steps for solving / solving a problem to be solved, etc.

[0074] Step 103: Generate second knowledge based on the target task and the first processing information. The second knowledge can be used by the agent to update the knowledge base or update the model parameters of the processing model.

[0075] Generating second knowledge based on the target task and the first processed information can be implemented, but is not limited to, as including at least one of the following ad:

[0076] a. Parameterize the task requirement description in the target task to obtain the parameterized result of the task requirement description.

[0077] Optionally, parameterizable objects in the task requirement description of the target task can be identified first, and then the identified parameterizable objects can be converted into corresponding parameter information. The parameterizable objects may include, but are not limited to, entity objects, behaviors / operations, various variables and / or constants, etc.

[0078] For example, assuming the task requirement description of the target task is "Please process the data of file xxx2 into format xxx3 and output it at time xxx1, with the subject being xxx4", then we can identify the various parameterizable objects in the task requirement description, such as file, subject, output format, time, etc., and then parameterize these parameterizable objects and output the following parameterized results: file: xxx2; subject: xxx4; output format: xxx3; time: xxx1.

[0079] b. Based on the target task and the first processing information, determine the tool usage information of the tools used in the task processing process.

[0080] Optionally, tool usage information for the tools used in the task processing can be determined based on the target task and the processing results and processing paths contained in the first processing information.

[0081] The identified tool usage information may include, but is not limited to, one or more of the following: tool type, tool name, number of times used, time, task status before and after tool use, or changes in the status of data involved in the task, for different processing stages / steps / subtasks of the target task.

[0082] c. Based on the target task and the first processing information, determine the processing steps that effectively solve the task during the task processing process.

[0083] Optionally, based on the target task and the processing results and processing paths included in the first processing information, effective processing steps for resolving the task can be determined during the task processing process. For example, from the task-solving workflow included in the processing path (such as breaking down the target task into multiple sub-tasks, and the processing steps for each sub-task and the processing tools used in each step), steps that can effectively advance the task processing process of the target task and enable the successful acquisition of the target task's processing results can be selected as effective processing steps for resolving the task during the task processing process.

[0084] d. Based on the target task and the first processing information, determine whether the target task was successfully executed, and record the target task and the first processing information as corresponding success or failure examples.

[0085] Optionally, the success or failure of the target task can be determined based on the target task and the processing results and processing paths contained in the first processing information.

[0086] The second knowledge includes at least one of the parameterization results determined based on the above steps ad, the tool usage information, the processing steps, and the success or failure examples. Corresponding to the knowledge types included in the second knowledge, each piece of knowledge in the knowledge base can also include at least one of these four data types.

[0087] In this embodiment of the application, in addition to task reasoning ability and the ability to generate first processing information corresponding to the target task, the processing model can also have knowledge summarization and generation capabilities, and can summarize knowledge and generate second knowledge based on the target task and the first processing information.

[0088] In one possible implementation, second knowledge can be generated through a processing model based on the target task and the processing results and processing paths contained in the first processing information. That is, the target task and the processing results and processing paths are input into the processing model, so that the processing model can summarize knowledge based on the input information and generate the second knowledge.

[0089] In this implementation, when the processing model is a large model, a prompt word can be constructed that includes the target task, the processing result, the processing path, and related task instructions (such as instructing the processing model to perform knowledge summarization and generation). The constructed prompt word is then input into the processing model, so that the processing model can summarize the target task, the processing result, and the processing path based on the input task instructions and generate the second knowledge.

[0090] In other possible implementations, a reward signal can be generated based on the first processing information corresponding to the target task, and the model parameters of the processing model can be adjusted based on the generated reward signal. On this basis, second knowledge can be generated through the processing model with adjusted model parameters based on the target task and the first processing information.

[0091] Specifically, the processing effect of the processing results and / or processing path represented by the first processing information can be evaluated based on evaluation indicators, but is not limited to, to obtain the corresponding evaluation indicator values, and then a reward signal for the processing model can be generated based on the evaluation indicator values.

[0092] The reward signal of a model is a core concept in reinforcement learning frameworks. It represents the immediate feedback given by the environment or evaluation mechanism to the model's (agent's) behavior, indicating the degree of good or bad of that behavior. The reward signal is typically represented as a numerical value: positive numbers indicate encouragement, negative numbers indicate punishment, and zero indicates neutrality. The goal is to maximize the long-term cumulative reward through learning.

[0093] The evaluation metrics may include, but are not limited to, the reasoning time of the task, whether the task is successful or not, the accuracy of the task processing results and / or the accuracy and rationality of the task processing path, etc.

[0094] After adjusting the model parameters of the processing model based on the reward signal, this implementation further generates second knowledge based on the target task and the first processing information through the processing model with adjusted model parameters. For example, prompt words summarizing knowledge based on the indicator model constructed from the processing results and processing paths contained in the target task and the first processing information are input into the processing model with adjusted model parameters, thereby generating second knowledge based on the input information.

[0095] The process of generating second knowledge in this implementation differs from the process of generating second knowledge based on the target task and first processing information (including processing results and processing paths) through a processing model in the previous implementation. The only difference is the model parameters of the processing model used. Therefore, please refer to the relevant explanation of the implementation process of generating second knowledge based on the processing model in the previous implementation. It will not be repeated here.

[0096] After generating second knowledge based on the target task and the first processing information, the agent can update the knowledge base or update the model parameters of the processing model based on the second knowledge. This can be done by expanding and / or optimizing the knowledge base, or by enhancing the reasoning ability of the processing model by updating the model parameters. For example, it can enhance the knowledge generation ability or task reasoning ability of the processing model, so that the processing model can summarize and generate knowledge that is more valuable for subsequent tasks, or perform more accurate and efficient task reasoning.

[0097] In summary, the task processing method provided in this embodiment, in response to obtaining a target task, not only uses an agent to call a first type of tool to determine the first knowledge that satisfies the association conditions with the target task from the knowledge base, and generates the first processing information corresponding to the target task through reasoning by the processing model based on the target task and the first knowledge, but also generates second knowledge based on the target task and the first processing information. It can also support the agent to update the knowledge base or update the model parameters of the processing model based on the generated second knowledge. Therefore, the agent can summarize, learn, and internalize new information based on its own interaction with the external environment, thereby achieving self-evolution. Accordingly, the agent can adapt to external dynamic changes and improve the performance of the agent in solving complex problems and other tasks (such as improving the task reasoning speed or accuracy, reducing token overhead, etc.).

[0098] In an alternative embodiment, see Figure 2 The flowchart shown in this application illustrates that the task processing method provided may further include the following processes:

[0099] Step 104: The agent invokes the second type of tool to update the knowledge base based on the second knowledge.

[0100] When updating the knowledge base based on the second knowledge, it is possible, but not limited to, using a second type of tool through an agent to insert the second knowledge into the knowledge base, or to adjust / modify the corresponding existing knowledge in the knowledge base based on the second knowledge, or to replace the corresponding existing knowledge in the knowledge base with the second knowledge (such as replacing knowledge in the knowledge base that conflicts or contradicts the second knowledge with the second knowledge). There are no restrictions on this, and it can be determined according to the actual application.

[0101] Taking the second type of knowledge and the knowledge base as an example, which includes four types of data: parameterized results of task requirement descriptions, tool usage information of tools used in task processing, effective task-solving steps, and success or failure examples, the second type of tools can include four writing tools. Each writing tool is used to update one type of data from the four types of data included in the second knowledge to the knowledge base in the required manner (such as insertion, modification, replacement, etc.). Similarly, in this example, the first type of tools can include four reading tools. Each reading tool is used to search, match, and read the matched data from one type of data from the four types of data included in the knowledge base based on the target task, as context information for the target task. Thus, this example involves a total of eight reading and writing tools to meet the agent's needs for reading and writing these four types of data in the knowledge base.

[0102] Furthermore, when updating the knowledge base based on the second knowledge, the agent can invoke a second type of tool to update the knowledge base based on the natural language data and vectors of the second knowledge. For example, the agent can invoke the second type of tool to insert the natural language data and vectors of the second knowledge into the knowledge base, or adjust / modify the natural language data of the corresponding knowledge already existing in the knowledge base based on the natural language data of the second knowledge, and simultaneously adjust / modify the vectors of the corresponding knowledge based on the vectors of the second knowledge, or replace the natural language data of the corresponding knowledge already existing in the knowledge base with the natural language data of the second knowledge, and simultaneously replace the vectors of the corresponding knowledge with the vectors of the second knowledge.

[0103] In practical applications, refer to Figure 3 The diagram illustrates how an agent summarizes and reuses historical experience. Optionally, after receiving and processing a target task based on model reasoning, the agent can automatically trigger a learning instruction to generate second knowledge. By comparing the second knowledge with the knowledge in the knowledge base (e.g., comparing the second knowledge with the historical information currently read from the knowledge base, i.e., the first knowledge), the agent can autonomously decide how to update the knowledge base based on the second knowledge (e.g., insertion, modification, replacement). Then, it can call the corresponding tools to update various types of data (e.g., the four types of data mentioned above) in the second knowledge to the knowledge base in the required manner (e.g., insertion, modification, replacement), so that various types of data in the knowledge base can be reused in subsequent task processing.

[0104] Furthermore, the processing model can be either a closed-source model or an open-source model, depending on the specific application. In a closed-source model, the code or algorithm is non-public information with access and modification restrictions, while in an open-source model, the code or algorithm is public information with access and modification restrictions.

[0105] When processing a closed-source model, the implementation method of this embodiment can be used, which updates the knowledge base only based on the second knowledge without updating the model parameters of the processed model. However, it is not limited to this; the implementation method of this embodiment can also be applied to open-source models. That is, when processing an open-source model, the implementation method of this embodiment can also be used, which updates the knowledge base only based on the second knowledge.

[0106] It's easy to understand that in the second knowledge generation stage, if the processing model is a closed-source model, the second knowledge can be generated from the processing model whose parameters have not been updated. However, if the processing model is an open-source model, the second knowledge can be generated from either the processing model whose parameters have not been updated or the processing model whose parameters have been updated. Among these, generating the second knowledge from the processing model whose parameters have been updated is preferred.

[0107] In summary, in this embodiment, the agent can continuously summarize knowledge based on its interactions with the environment / user, and update the knowledge base based on the summarized knowledge, thereby achieving self-evolution to adapt to dynamically changing environments. Once the agent possesses self-evolution capabilities, it can summarize errors, learn solutions to those errors, and thus improve its performance in solving subsequent complex problems / tasks, thereby increasing task processing accuracy and efficiency. For example, if the agent lacks self-evolution capabilities, and an error occurs while using a certain tool to solve a complex problem, it will likely encounter the same error again when facing the same problem. However, in this application, the agent, possessing self-evolution capabilities, can summarize errors, learn solutions to those errors, and thus improve its performance in solving subsequent complex problems / tasks, increasing speed and accuracy while reducing token overhead.

[0108] In an alternative embodiment, see Figure 4 The flowchart shown in this application illustrates that the task processing method provided may further include the following processes:

[0109] Step 105: Based on the second knowledge, update the processing model through the intelligent agent and call the second type of tool to update the knowledge base.

[0110] In this embodiment, in addition to updating the knowledge base based on the second knowledge, the model parameters of the processing model are also updated based on the agent. This implementation method is suitable for cases where the processing model is an open-source model.

[0111] Optionally, based on the second knowledge, a reinforcement learning (RL) approach can be used to perform target processing in an iterative manner using a GRPO (Group Relative Policy Optimization) trainer, which can at least enhance the knowledge generation capability of the processing model, and the knowledge base can be updated based on the third knowledge used in the target round processing during the iteration process.

[0112] The target processing includes performing task reasoning on the target task through a processing model, updating the parameters of the processing model based on the task reasoning, and generating knowledge based on the updated processing model.

[0113] The target round processing satisfies the proximity condition with the last round processing in the above iteration process. This proximity condition can mean that the target round processing is the last round processing in the iteration process, or that the target round processing is not the last round processing in the iteration process, but the absolute value of the difference between their corresponding rounds is less than a threshold.

[0114] The third knowledge is the knowledge generated by the processing model directly or indirectly based on the second knowledge during the iteration process.

[0115] Based on the second knowledge, the target processing is performed iteratively using the GRPO trainer, which can be further implemented as the following steps "1-1"-"1-4":

[0116] 1-1: Perform sampling processing to obtain the current sampled data.

[0117] The sampled data includes the target task, as well as the latest knowledge generated during the iteration process of the second knowledge or processing model based on the second knowledge.

[0118] 1-2: Input the sampled data into the current processing model to obtain the second processing information of the target task generated by the processing model.

[0119] Optionally, the second processing information includes the processing results and processing path of the target task generated by the current processing model based on the sampled data.

[0120] 1-3: Generate a reward signal based on the second processing information, and adjust the model parameters of the processing model based on the generated reward signal.

[0121] Optionally, the processing results and / or processing paths contained in the second processing information can be evaluated based on evaluation metrics to obtain corresponding evaluation metric values, and then a reward signal for the processing model can be generated based on the evaluation metric values.

[0122] 1-4: Based on the target task and the second processing information, knowledge is generated using the current processing model after the model parameters are adjusted, and the generated knowledge is used to proceed to the sampling process until the termination condition is met, thus ending the processing.

[0123] The termination condition may be set as, but is not limited to, the number of iterations reaching a predetermined number, or the iteration duration reaching a predetermined duration, or the evaluation result (such as the evaluation index value) of the latest output of the processing model for the target task (such as the processing result, processing path) indicating that the processing model has achieved the expected effect in processing the target task.

[0124] In the above-mentioned process using the GRPO trainer, each sampling is essentially a round of experience memory evolution and reuse. The reward for each sampling can characterize the effect of the solution after reusing experience memory, so as to evaluate the role of experience memory summarized in the sampling, thereby strengthening the agent's processing model's ability to summarize experience memory from historical experience.

[0125] See Figure 5This provides a schematic diagram of using the GRPO trainer to iteratively execute target processing that at least enhances the knowledge generation capability of the processing model. Here, q represents the target task, Agent represents the intelligent agent, Self-edit AgenticMemory represents the relevant knowledge input to the processing model along with the target task q during iteration (e.g., the first knowledge initially, and the latest knowledge summarized by the model in subsequent iterations), Trajectory represents the processing information generated by the processing model for the target task during iteration (e.g., first processing information, second processing information), which can include the processing results and processing trajectory of the target task. Policy Model represents the knowledge summarization / generation part of the processing model, and RewardModel represents the reward mechanism, used to generate reward signals based on the task processing information output by the model, and to update the model parameters based on the reward signals. Δ represents gradient update. This indicates that model parameter optimization is achieved based on gradient updates; and These represent the model parameters before and after the update, respectively.

[0126] like Figure 5 As shown, during the iteration process, initially, in response to obtaining the target task q, the agent first inputs the target task q and relevant knowledge queried from the knowledge base into the processing model for task reasoning, obtaining processing information such as the processing result and processing trajectory of the target task (e.g., ...). Figure 4 The process begins with the Trajectory in the model. Then, a reward signal is generated based on the processing information output by the model, and the model parameters are updated based on the reward signal, such as updating the parameters of the Policy Model in the processing model. The updated processing model is then used to summarize and generate knowledge based on the target task and the latest Trajectory. This updated processing model is then used to generate processing information such as the processing result and trajectory of the target task in the current iteration, based on the target task and the latest generated knowledge. On this basis, a reward signal is generated based on the latest generated processing information, and the model parameters are updated based on the reward signal. The process continues iterating according to the above steps until the termination condition is met, at which point the iteration ends.

[0127] The task processing information corresponding to the target task is either the first processing information or the third processing information. The first processing information is the processing information of the target task generated by the processing model based on the target task and the first knowledge queried from the knowledge base. The third processing information is the processing information of the target task generated by the processing model at the end of the above iterative process.

[0128] For example, when the processing model is a closed-source model, the first processing information can be used as the task processing information corresponding to the target task, thereby outputting the first processing information, such as the processing results and processing trajectory included in the first processing information, for the user to view or use. When the processing model is an open-source model, the third processing information can be used as the task processing information corresponding to the target task, thereby outputting the third processing information, such as the processing results and processing trajectory included in the third processing information, for the user to view or use.

[0129] This embodiment continuously improves the model's knowledge summarization ability through reinforcement learning. It enables the model to continuously summarize experience memories that are more valuable for subsequent tasks during use to update the knowledge base, thereby summarizing, optimizing and reusing historical experience. By summarizing, optimizing and reusing historical experience, the task processing performance of the agent can be effectively improved, the agent's computing time and token overhead can be reduced, and the agent's environmental adaptability and task processing ability when facing complex tasks can be improved.

[0130] This application also provides a task processing device, the composition of which is as follows: Figure 6 As shown, it includes:

[0131] The determination module 601 is used to, in response to obtaining the target task, invoke a first type of tool through an intelligent agent to determine first knowledge from the knowledge base that satisfies the association conditions with the target task;

[0132] The first generation module 602 is used to generate first processing information corresponding to the target task by reasoning through a processing model based on the target task and the first knowledge.

[0133] The second generation module 603 is used to generate second knowledge based on the target task and the first processing information. The second knowledge can be used by the agent to update the knowledge base or update the model parameters of the processing model.

[0134] In one optional implementation, the first processing information includes the processing result and the processing path;

[0135] The second generation module 603 is specifically used to generate second knowledge based on the target task, the processing result, and the processing path through the processing model.

[0136] In an optional embodiment, the above-mentioned device further includes a first update module, which is used to call a second type of tool through the intelligent agent to update the knowledge base based on the second knowledge.

[0137] In an optional implementation, the second generation module 603 is specifically used for:

[0138] A reward signal is generated based on the first processed information;

[0139] The model parameters of the processing model are adjusted based on the reward signal;

[0140] Based on the target task and the first processing information, second knowledge is generated by the processing model after the model parameters are adjusted.

[0141] In an optional embodiment, the above apparatus further includes a second update module, configured to: update the processing model through the intelligent agent based on the second knowledge, and call a second type of tool to update the knowledge base.

[0142] In an optional implementation, the second update module, when updating the processing model through the intelligent agent and calling the second type of tool to update the knowledge base based on the second knowledge, is specifically used for:

[0143] Based on the second knowledge, a set of relative strategies is used to optimize the trainer and perform target processing in an iterative manner to at least enhance the knowledge generation capability of the processing model; the target processing includes performing task reasoning on the target task through the processing model, updating the parameters of the processing model based on the task reasoning, and generating knowledge based on the updated processing model;

[0144] The knowledge base is updated based on the third knowledge used in the target round processing during the iteration process;

[0145] Wherein, the target round processing and the last round processing in the iteration process satisfy the proximity condition, and the third knowledge is the knowledge generated by the processing model based on the second knowledge during the iteration process.

[0146] In an optional implementation, the second update module, when optimizing the trainer using a set of relative policies based on the second knowledge and performing target processing iteratively, is specifically used for:

[0147] Perform sampling processing to obtain the current sampling data; the sampling data includes the target task, as well as the latest knowledge generated during the iteration process of the second knowledge or the processing model based on the second knowledge;

[0148] The sampled data is input into the current processing model to obtain the second processing information of the target task generated by the processing model;

[0149] A reward signal is generated based on the second processed information, and the model parameters of the processing model are adjusted based on the generated reward signal;

[0150] Based on the target task and the second processing information, knowledge is generated using the current processing model with adjusted model parameters. The generated knowledge is then used to execute the sampling process until the termination condition is met, at which point the processing ends.

[0151] In one optional implementation, the task processing information corresponding to the target task is the first processing information;

[0152] Alternatively, the task processing information corresponding to the target task may be the third processing information, which is the processing information of the target task generated by the processing model at the end of the iteration.

[0153] In one alternative implementation, the second generation module 603 is specifically configured to perform at least one of the following:

[0154] The task requirement description in the target task is parameterized to obtain the parameterization result of the task requirement description.

[0155] Based on the target task and the first processing information, determine the tool usage information of the tools used in the task processing process;

[0156] Based on the target task and the first processing information, determine the processing steps that effectively solve the task during the task processing process;

[0157] Based on the target task and the first processing information, determine whether the target task was successfully executed, and record the target task and the first processing information as corresponding success or failure examples;

[0158] The second knowledge includes at least one of the parameterization results, the tool usage information, the processing steps, and the success or failure examples.

[0159] This application also discloses an electronic device, the composition and structure of which are as follows: Figure 7 As shown, it includes at least:

[0160] Memory 10 is used to store the computer instruction set;

[0161] Computer instruction sets can be implemented in the form of computer programs.

[0162] The processor 20 is configured to implement the task processing method provided in any of the above method embodiments by executing a set of computer instructions in the memory.

[0163] The processor 20 can be a central processing unit (CPU), a graphics processor, an application-specific integrated circuit (ASIC), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a neural network processor (NPU), a deep learning processor (DPU), or other programmable logic devices.

[0164] Optionally, the electronic device may also include a display device for displaying information.

[0165] Optionally, electronic devices may also include storage resources such as memory and cache.

[0166] Optionally, the electronic device may also include an image acquisition device.

[0167] In addition to these components, electronic devices may also include communication interfaces, communication buses, and other parts. Memory, processor, and communication interface communicate with each other through the communication bus.

[0168] Communication interfaces are used for communication between electronic devices and other devices. Communication buses can be Peripheral Component Interconnect (PCI) buses or Extended Industry Standard Architecture (EISA) buses, and can be categorized into address buses, data buses, control buses, etc.

[0169] This application also discloses a storage medium carrying one or more computer instruction sets, which, when executed by an electronic device, enable the electronic device to implement the task processing method provided in any of the above method embodiments.

[0170] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0171] For ease of description, the above systems or devices are described separately as various modules or units based on their functions. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware components.

[0172] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence or the part that makes a creative contribution, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0173] Finally, it should be noted that in this document, relational terms such as first, second, third, and fourth are used to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0174] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A task processing method, comprising: In response to obtaining the target task, the agent invokes a first type of tool to determine first knowledge from the knowledge base that satisfies the association conditions with the target task; Based on the target task and the first knowledge, reasoning is performed through a processing model to generate first processing information corresponding to the target task; Based on the target task and the first processing information, second knowledge is generated. The second knowledge can be used by the agent to update the knowledge base or update the model parameters of the processing model.

2. The task processing method according to claim 1, wherein the first processing information includes processing result and processing path; Generate second knowledge based on the target task and the first processed information, including: Based on the target task, the processing results, and the processing path, second knowledge is generated through the processing model.

3. The task processing method according to claim 2 further includes: The agent invokes a second type of tool to update the knowledge base based on the second knowledge.

4. The task processing method according to claim 1, wherein generating second knowledge based on the target task and the first processing information includes: A reward signal is generated based on the first processed information; The model parameters of the processing model are adjusted based on the reward signal; Based on the target task and the first processing information, second knowledge is generated by the processing model after the model parameters are adjusted.

5. The task processing method according to claim 1, further comprising: Based on the second knowledge, the processing model is updated by the intelligent agent, and the knowledge base is updated by calling the second type of tool.

6. The task processing method according to claim 5, based on the second knowledge, updating the processing model through the intelligent agent and calling the second type of tool to update the knowledge base, includes: Based on the second knowledge, the trainer is optimized using a set of relative strategies to perform target processing in an iterative manner, so as to at least enhance the knowledge generation capability of the processing model; The target processing includes performing task reasoning on the target task through the processing model, updating the parameters of the processing model based on the task reasoning, and generating knowledge based on the updated processing model. The knowledge base is updated based on the third knowledge used in the target round processing during the iteration process; Wherein, the target round processing and the last round processing in the iteration process satisfy the proximity condition, and the third knowledge is the knowledge generated by the processing model based on the second knowledge during the iteration process.

7. The task processing method according to claim 6, based on the second knowledge, utilizes a set of relative policy optimization trainers to perform target processing iteratively, comprising: Perform sampling processing to obtain the current sampled data; The sampled data includes the target task, as well as the knowledge generated latest during the iteration process of the second knowledge or the processing model based on the second knowledge; The sampled data is input into the current processing model to obtain the second processing information of the target task generated by the processing model. A reward signal is generated based on the second processed information, and the model parameters of the processing model are adjusted based on the generated reward signal. Based on the target task and the second processing information, knowledge is generated using the current processing model with adjusted model parameters. The generated knowledge is then used to execute the sampling process until the termination condition is met, at which point the processing ends.

8. The task processing method according to claim 3 or 7, wherein the task processing information corresponding to the target task is the first processing information; Alternatively, the task processing information corresponding to the target task may be the third processing information, which is the processing information of the target task generated by the processing model at the end of the iteration.

9. The task processing method according to claim 1, wherein generating second knowledge based on the target task and the first processing information includes at least one of the following: The task requirement description in the target task is parameterized to obtain the parameterization result of the task requirement description. Based on the target task and the first processing information, determine the tool usage information of the tools used in the task processing process; Based on the target task and the first processing information, determine the processing steps that effectively solve the task during the task processing process; Based on the target task and the first processing information, determine whether the target task was successfully executed, and record the target task and the first processing information as corresponding success or failure examples; The second knowledge includes at least one of the parameterization results, the tool usage information, the processing steps, and the success or failure examples.

10. An electronic device, comprising: Memory, used to store at least one set of computer instructions; A processor is configured to perform the following processing by executing a set of instructions stored in the memory: In response to obtaining the target task, the agent invokes a first type of tool to determine first knowledge from the knowledge base that satisfies the association conditions with the target task; Based on the target task and the first knowledge, reasoning is performed through a processing model to generate first processing information corresponding to the target task; Based on the target task and the first processing information, second knowledge is generated. The second knowledge can be used by the agent to update the knowledge base or update the model parameters of the processing model.

Images