Data processing method and device

Abstract

The invention provides a data processing method and device. The data processing method comprises the steps of receiving a to-be-processed task; the to-be-processed task is input into an intelligent processing unit, a task processing result output by the intelligent processing unit is obtained, and the task processing result is generated by executing all to-be-processed sub-tasks corresponding to the to-be-processed task based on a preset tool set. The to-be-processed sub-task is determined based on a reasoning model corresponding to the intelligent processing unit according to the to-be-processed task and historical sub-task information, and the historical sub-task information is extracted and generated from interaction historical information under the condition that an information extraction identifier is detected. Through the method, the intelligent processing unit can autonomously perform reasoning on the to-be-processed task, dynamically discover necessary tools in the process of processing the to-be-processed task, execute related operations, and manage own memory information through compression of interactive historical information.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular to data processing methods. This specification also relates to data processing apparatus, a computing device, a computer-readable storage medium, and computer program products. Background Technology

[0002] With the development of computer technology, large language models and intelligent agent technology have developed rapidly. An intelligent agent is an artificial intelligence system that can perceive the environment, make autonomous decisions and execute actions. It has core characteristics such as autonomy, adaptability and learning ability.

[0003] Intelligent agents are typically built upon large language models, leveraging their natural language processing capabilities to complete tasks. When performing a task, the agent analyzes the task using its corresponding large language model and determines whether external tools are needed. However, current intelligent agents operate according to predefined, fixed workflows. This approach applies the same processing flow to all tasks, lacking flexibility and making it difficult to adapt to complex tasks requiring dynamic decision-making. Furthermore, when invoking external tools, the agent can only access a limited set of pre-defined tools, failing to accommodate diverse tasks requiring a large-scale toolset. Summary of the Invention In view of this, embodiments of this specification provide a data processing method. This specification also relates to a data processing apparatus, a computing device, a computer-readable storage medium, and a computer program product, to address the aforementioned problems existing in the prior art.

[0004] According to a first aspect of the embodiments of this specification, a data processing method is provided, comprising: Receive pending tasks; The task to be processed is input into the intelligent processing unit, and the task processing result output by the intelligent processing unit is obtained. The task processing result is generated based on a preset tool set to execute each sub-task corresponding to the task to be processed. The sub-tasks to be processed are determined based on the reasoning model corresponding to the intelligent processing unit according to the task to be processed and historical sub-task information. When an information extraction identifier is detected, the historical sub-task information is extracted and generated from the interaction history information.

[0005] According to a second aspect of the embodiments of this specification, a data processing apparatus is provided, comprising: The receiving module is configured to receive tasks to be processed. The processing module is configured to input the task to be processed into the intelligent processing unit and obtain the task processing result output by the intelligent processing unit. The task processing result is generated based on a preset toolset to execute each sub-task corresponding to the task to be processed. The sub-tasks are determined based on the reasoning model corresponding to the intelligent processing unit according to the task to be processed and historical sub-task information. When an information extraction identifier is detected, the historical sub-task information is extracted and generated from the interaction history information.

[0006] According to a third aspect of the embodiments of this specification, a computing device is provided, comprising: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the above method.

[0007] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores a computer program / instructions that, when executed by a processor, implement the steps of the above-described method.

[0008] According to a fifth aspect of the embodiments of this specification, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described method.

[0009] The data processing method provided in this specification includes: receiving a task to be processed; inputting the task to be processed into an intelligent processing unit, and obtaining a task processing result output by the intelligent processing unit, wherein the task processing result is generated based on a preset toolset to execute each sub-task corresponding to the task to be processed, the sub-tasks to be processed are determined based on the reasoning model corresponding to the intelligent processing unit according to the task to be processed and historical sub-task information, and when an information extraction identifier is detected, the historical sub-task information is extracted and generated from the interaction history information.

[0010] The method provided in the embodiments of this specification allows an intelligent agent to autonomously reason about a task, dynamically discover necessary tools in the process of handling the task, execute related operations, and manage its own memory information by compressing interaction history information. This processing approach abandons the traditional, rigid agent workflow, enabling the intelligent agent to maintain a global perspective on the task at hand. Attached Figure Description

[0011] Figure 1 This is a flowchart of a data processing method provided in one embodiment of this specification; Figure 2This is a flowchart illustrating a data processing method for a multi-tool task invocation scenario provided in one embodiment of this specification. Figure 3 This is a schematic diagram of the structure of a data processing device provided in one embodiment of this specification; Figure 4 This is an architecture diagram of a data processing system provided in one embodiment of this specification; Figure 5 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0012] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0013] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0014] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0015] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this manual are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0016] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0017] LLM (Large Language Models) refers to deep learning models trained on large amounts of text data, enabling them to generate natural language text or understand the meaning of language text. These models can provide in-depth knowledge and language production on a wide range of topics by training on massive datasets. The core idea is to learn patterns and structures of natural language through large-scale unsupervised training, mimicking human language cognition and generation processes to some extent.

[0018] Intelligent Agent: An intelligent agent is an intelligent application or entity capable of acting autonomously, perceiving its environment, making decisions, and interacting with the environment. It can perceive its environment and take actions to achieve specific goals. Intelligent agents combine large language models to automatically think, plan, verify, and execute in order to accomplish specific objectives.

[0019] With the advancement of technology, large language models have made significant progress, and intelligent agents, as a specific application of large language models, have also achieved remarkable advancements. Most existing intelligent agents rely on predefined, fixed workflows. This approach lacks flexibility and struggles to adapt to complex tasks requiring dynamic decision-making and tool discovery. Furthermore, existing intelligent agents are typically limited to the use of fixed external tools, failing to adapt to diverse tasks requiring large-scale toolsets. For example, when an intelligent agent is handling a complex task, it cannot automatically find new tools matching the task's execution status. Additionally, intelligent agents engage in multiple rounds of interaction during task processing. When there is a lot of context, the lack of effective context management can lead to memory and decision-making problems when handling long-term tasks.

[0020] Based on this, a data processing method is provided in this specification. This specification also relates to a data processing apparatus, a computing device, a computer-readable storage medium, and a computer program product, which will be described in detail in the following embodiments.

[0021] Figure 1 A flowchart of a data processing method according to an embodiment of this specification is shown, which specifically includes the following steps: Step 102: Receive the tasks to be processed.

[0022] In the methods provided in the embodiments of this specification, the task to be processed can be understood as the task that needs to be processed in the methods provided in the embodiments of this specification. The task to be processed may be sent by the user or by other terminals. In the methods provided in the embodiments of this specification, the source of the task to be processed is not limited, and the actual application shall prevail.

[0023] In one or more specific implementations provided in the embodiments of this specification, the example of a task sent by a user is used for explanation. The terminal receives the task sent by the user. Specifically, the task may include a question raised by the user and corresponding instructions. In practical applications, the task can be presented in the form of text or voice. When the user sends the task to the terminal in text form, the task received by the terminal is text information. When the user sends the task to the terminal in voice form, the task received by the terminal is voice information. When the task is voice information, the terminal can first convert the voice information into text information and then process the text information, or it can directly process the voice information. The methods provided in the embodiments of this specification do not further limit this approach.

[0024] Step 104: Input the task to be processed into the intelligent processing unit and obtain the task processing result output by the intelligent processing unit. The task processing result is generated based on the execution of each sub-task corresponding to the task to be processed by a preset tool set. The sub-tasks to be processed are determined based on the reasoning model corresponding to the intelligent processing unit according to the task to be processed and the historical sub-task information. When an information extraction identifier is detected, the historical sub-task information is extracted and generated from the interaction history information.

[0025] The data processing method provided in the embodiments of this specification proposes a DeepAgent framework. Through unified process reasoning, an autonomous memory folding mechanism, and a reinforcement learning-based ToolPO training strategy, the agent can handle complex multi-tool, multi-step tasks, improving its performance and generalization ability in real-world tasks. The core of the DeepAgent framework lies in unifying the processes of thinking, tool discovery, and action execution within a coherent reasoning framework. Through memory management and reinforcement learning strategies, it enhances the agent's flexibility and efficiency in handling complex, multi-step tasks. Further explanation and illustration will be provided in one or more specific embodiments provided in this specification.

[0026] The methods provided in the embodiments of this specification are applied to an intelligent processing unit (also known as an intelligent agent). An intelligent processing unit is an intelligent processing unit that can take initiative, sense pipe diameter, make decisions and take actions. It has a large language model as its core brain and also includes abilities such as memory, planning, tool use, and reflection. It can autonomously and continuously pursue goals.

[0027] The intelligent processing unit can receive information from the environment, such as user natural language commands, sensor data, database query results, and web page content. The large language model corresponding to the intelligent agent then understands the input information received by the agent and transforms it into a semantic representation that it can process internally.

[0028] In the method provided in the embodiments of this specification, the task to be processed is input into an intelligent processing unit, and the task processing result output by the intelligent processing unit is obtained. Within the intelligent processing unit, the task to be processed is parsed based on the inference model corresponding to the intelligent processing unit to obtain the current sub-task to be processed. Then, the current sub-task to be processed is executed according to the historical sub-task information that has already been processed, until the final task processing result is obtained. During the task processing process, the task to be processed is also executed in conjunction with a preset toolset. Furthermore, if an information extraction identifier is detected during the execution of the task to be processed by the intelligent processing unit, historical sub-task information is extracted from the interaction history information, thereby simplifying the content of the historical sub-task information and improving the processing efficiency of the intelligent processing unit.

[0029] In a specific embodiment provided in this specification, the task to be processed is input into an intelligent processing unit, and the task processing result output by the intelligent processing unit is obtained, including S1042-S1046: S1042. Based on the inference model corresponding to the intelligent processing unit, the task to be processed is analyzed to obtain the current sub-task to be processed corresponding to the task to be processed, and the historical sub-task information corresponding to the current sub-task to be processed.

[0030] In the methods provided in the embodiments of this specification, applied in an intelligent processing unit (i.e., an intelligent agent), the inference model corresponding to the intelligent processing unit parses the task to be processed, deconstructing it into a continuous decision-making process. This continuous decision-making process includes multiple sub-task action information, through which the agent interacts with the environment until the final task objective is achieved. In the methods provided in the embodiments of this specification, the environment can provide an external toolkit of arbitrary scale and magnitude for the intelligent agent to use, thereby assisting the intelligent agent in processing the task to be processed.

[0031] Specifically, the inference model corresponding to the intelligent processing unit divides the task to be processed into multiple sub-tasks based on the actual task processing situation, and processes each sub-task. During the processing of each sub-task, sub-tasks with a specific order of priority are executed sequentially. Sub-tasks that can be processed simultaneously can be executed in parallel.

[0032] This explanation uses the example of an agent processing a specific subtask. The inference model corresponding to the intelligent processing unit obtains the current subtask based on the actual task processing status. The current subtask can be understood as the subtask the agent is about to process. The agent also obtains historical subtask information corresponding to the current subtask.

[0033] Historical subtask information can be understood as a set of information about subtasks that the agent has already completed in relation to the task to be processed. In the method provided in the embodiments of this specification, the agent determines the specific situation of the current subtask to be processed based on the information of the completed subtasks. By using historical subtask information to guide the current subtask to be processed, there is no need to process the task according to a pre-set processing flow, making the agent more flexible.

[0034] In a specific embodiment provided in this specification, the task T to be processed is divided into 5 sub-tasks to be processed for explanation. That is, the task T to be processed is divided into sub-tasks t1, t2, t3, t4, and t5. When the sub-task t3 is the current sub-task to be processed, the completed historical sub-tasks are t1 and t2. The obtained historical sub-task information is the historical sub-task action information and historical sub-task feedback information corresponding to t1 and t2, respectively.

[0035] S1044. Based on the historical subtask information, determine the subtask execution information of the current subtask to be processed, and execute the current subtask to be processed based on the subtask execution information to obtain the current subtask information.

[0036] In the method provided in the embodiments of this specification, when processing each currently pending subtask, the intelligent system determines the subtask execution information corresponding to the currently pending subtask based on historical subtask information. The currently pending subtask is then executed using the subtask execution information to obtain the current subtask information.

[0037] The subtask execution information can be understood as the operation information for the currently pending subtask. That is, what operations need to be performed to execute the currently pending subtask.

[0038] Once the subtask execution information is determined, corresponding operations can be performed on the currently pending subtask based on this information to obtain the current subtask information. The current subtask information can be understood as the subtask processing result obtained after performing corresponding operations on the currently pending subtask based on the subtask execution information.

[0039] In a specific embodiment of this specification, historical subtask information includes historical subtask action information and historical subtask feedback information; Based on the historical subtask information, the subtask execution information of the currently pending subtask is determined, including: Based on the task to be processed, the historical subtask action information, and the historical subtask feedback information, the current subtask action information of the current subtask to be processed is determined.

[0040] In this embodiment, the historical subtask information records the information corresponding to the completed subtasks. Specifically, it includes the actions and feedback corresponding to each subtask. That is, the historical subtask information includes historical subtask action information and historical subtask feedback information.

[0041] In this context, historical subtask action information and historical subtask feedback information appear in pairs; that is, each historical subtask action information corresponds to one historical subtask feedback information. Historical subtask action information can be understood as the execution action corresponding to the historical subtask, while historical subtask feedback information can be understood as the processing result after processing the historical subtask based on the executed action.

[0042] The current subtask action information can be understood as the action that needs to be performed for the current subtask to be processed. For example, the current subtask action information could be thinking, querying tools, calling tools, compressing information, etc.

[0043] In the method provided in the embodiments of this specification, the processing of the current subtask by the intelligent agent is based on the historical subtask action information and historical subtask feedback information that have already been processed. The intelligent agent will determine the current subtask action information of the current subtask based on the historical subtask action information and historical subtask feedback information.

[0044] In this embodiment, the agent determines the current subtask action information based on historical subtask action information and feedback information, combined with the tasks to be processed. The agent can confirm the operation action of the current subtask based on the processing status of completed subtasks and the total number of tasks to be processed, without needing to follow a pre-set fixed process. This allows the agent to handle tasks more flexibly based on the actual task execution situation.

[0045] In a specific embodiment provided in this specification, the currently pending subtask is executed based on the subtask execution information to obtain the current subtask information, including: Execute the current subtask to be processed based on the current subtask action information, and obtain the current subtask feedback information corresponding to the current subtask action information.

[0046] In this embodiment, after determining the subtask execution information, the currently pending task can be executed based on the subtask execution information. Specifically, in the above steps, the subtask execution information can be understood as the current subtask action information, that is, the operation information that needs to be performed on the currently pending subtask.

[0047] Once the current subtask action information is determined, the agent can execute the currently pending subtask based on this information and obtain the corresponding feedback information. This feedback information can be understood as the observation of the subtask processing result after the agent has completed executing the current pending task based on the action information. For example, if the current subtask action information involves calling an external tool, the feedback information can be set to either "call successful" or "call failed" depending on the actual call status.

[0048] In another specific embodiment provided in this specification, the currently pending subtask is executed based on the current subtask action information to obtain the current subtask feedback information corresponding to the current subtask action information, including: If the current subtask action information is a thinking action, analyze the current subtask to be processed and obtain the task analysis result corresponding to the current subtask to be processed. If the current subtask action information is a tool search, obtain the subtask requirement information of the current subtask to be processed, and search in a preset tool set based on the subtask requirement information to obtain the tool search results; If the current subtask action information is a tool call, generate tool call parameters for the target tool, and call the target tool based on the tool call parameters to obtain the tool call result; If the current subtask action information is information extraction, the interaction history information of the task to be processed is sent to the information extraction model to obtain the reference prompt information output by the information extraction model.

[0049] In the embodiments provided in this specification, the current subtask action information may specifically include thinking actions, tool search, tool invocation, and information extraction. Correspondingly, the current task feedback information corresponding to the current subtask action information will have different feedback depending on the specific action information.

[0050] Taking the current subtask action information as an example of a thinking action, this illustrates that the agent needs to perform a deep search and thinking process on the current task to analyze the problem or plan the next execution step. The corresponding feedback information for the current subtask is the task analysis result.

[0051] Taking the current subtask action information as an example, this illustrates that the agent needs to search for tools to perform the task from the toolset based on the already executed subtasks. Accordingly, the agent generates a corresponding data query statement, which is used to retrieve relevant tools from the preset toolset. Specifically, the agent encapsulates the generated data query statement into search tags, uses a tool retrieval tool to perform a similarity query between the data query statement and the tool description information of each tool, sorts the tools based on similarity, and obtains the top n tools. The corresponding feedback information for the current subtask is the tool retrieval results obtained through the search. The tool retrieval results can be a tool list, including the tool description information, tool invocation parameters, etc., for each tool.

[0052] Taking the current subtask action information as a tool invocation as an example, this illustrates that the agent needs to invoke the target tool from the tool list. In this case, the agent obtains the tool invocation parameters corresponding to the target tool. Here, the target tool can be understood as the tool the agent needs to invoke, and the tool invocation parameters can be understood as the parameters required to invoke the target tool. The agent can invoke the target tool using these tool invocation parameters, thereby obtaining the current subtask feedback information as the corresponding tool invocation result. The tool invocation result can include the execution result after a failed or successful invocation.

[0053] Taking the current subtask action information as an example, this illustrates that if the interaction history information is too extensive, the agent may not be able to quickly and accurately obtain relevant information from it. In other words, retrieving information from the interaction history of the task to be processed may result in omissions or errors. In this situation, the agent needs to extract crucial information from the interaction history of the task to be processed to aid subsequent processing.

[0054] Specifically, the agent can input the interaction history information of the task to be processed into the information extraction model to obtain reference prompt information output by the information extraction model. The reference prompt information can be understood as information extracted from the interaction history information of the task to be processed for subsequent processing. The corresponding feedback information of the current subtask is the reference prompt information.

[0055] In one specific embodiment provided in this specification, the method further includes: If an information extraction identifier is detected, the current subtask action information is determined to be information extraction.

[0056] In this embodiment, the agent detects the presence of an information extraction identifier during the execution of a task. If the identifier is detected, an information extraction task is triggered. The information extraction identifier can be understood as a specific identifier generated by the agent during operation to trigger the information extraction task. When the agent detects this identifier, it initiates a memory folding process. The agent acquires all interaction history information from the start of the task and sends this information to the information extraction model for processing.

[0057] By setting information extraction flags, the processing difficulty of intelligent agents can be simplified, so that the intelligent agents only need to detect whether the information extraction flag appears, without having to constantly pay attention to whether information extraction of interaction history is needed.

[0058] Furthermore, in one specific embodiment provided in this specification, the method further includes: If the information folding condition is met based on the historical subtask information, an information extraction identifier is generated.

[0059] In the specific embodiments provided in this specification, the agent can trigger the information extraction operation at any logical point in its inference process. For example, when the agent completes a subtask or detects an error in the processing path of the task to be processed, it can trigger the generation of an information extraction identifier. This allows the agent to perform information extraction based on the information extraction identifier.

[0060] Specifically, the interaction history information of the task to be processed is sent to the information extraction model to obtain reference prompt information output by the information extraction model, including: The interaction history information of the task to be processed is sent to the information extraction model to obtain the historical record information, work record information and tool call information output by the information extraction model.

[0061] In practical applications, the information extraction model processes the interaction history information corresponding to the task to be processed, and generates three structured compressed information: historical record information, work record information, and tool call information.

[0062] Historical records can be understood as logs of tasks to be processed, including key events, major decision points, and the completion status of each subtask. Historical records provide the agent with long-term background information on the overall task structure and its overall goals.

[0063] Work log information can be understood as recent information during the processing of pending tasks, such as current subtasks, problems encountered during task processing, and plans. Work log information is a core component ensuring the continuity of the agent's memory during memory folding.

[0064] Tool call information can be understood as information about the tools used during the processing of a task, including which tools were used, how they were called, and the effectiveness of each tool. Tool call information helps the agent learn from its tool call experience, thereby adjusting its strategy for selecting and using tools.

[0065] To ensure that the extracted historical data, work logs, and tool call information are stable and easily parsed by the intelligent agent, a specific embodiment provided in this specification employs a data pattern presented in a specific format that the intelligent agent can use, rather than unstructured natural language. This structured data presentation pattern maintains a controllable and predictable structure and mitigates the problem of potential loss of key information when summarizing long texts.

[0066] S1046. If it is determined that the task to be processed has not met the task termination condition, the current subtask information is added to the historical subtask information, and the operation of obtaining the current subtask to be processed corresponding to the task to be processed continues until the task to be processed meets the task termination condition and the task processing result is generated.

[0067] After completing the current subtask and receiving feedback information from it, the agent checks whether it has finished processing the task. Specifically, it checks whether the task has reached its termination condition. If the termination condition has not been met, the agent needs to continue processing the task.

[0068] Specifically, the current subtask information is added to the historical subtask information for use as historical information for subsequent pending subtasks. Then, the process continues to return to and execute the current pending subtask corresponding to the previously identified pending task, and its subsequent operations, until the pending task meets the task termination conditions.

[0069] In this embodiment, the task termination condition for the task to be processed includes obtaining the task processing result corresponding to the task to be processed, or reaching the maximum number of steps that the agent can process the task to be processed.

[0070] In other words, if an agent obtains the final result of a task while processing it, the task can be considered to have reached its termination condition. Alternatively, if the agent processes the task to the maximum number of steps, the task can also be considered to have reached its termination condition.

[0071] In one specific embodiment provided in this specification, adding the current subtask information to the historical subtask information includes: The current subtask action information and the current subtask feedback information are saved to the historical subtask information.

[0072] In this embodiment, the current subtask information is added to the historical subtask information. Specifically, this can be understood as saving the current subtask action information and the current subtask feedback information to the historical subtask information.

[0073] The method provided in the embodiments of this specification is applied to an intelligent agent, which determines the process of handling a task through a pre-trained inference model. Through this method, the intelligent agent can autonomously infer the task, dynamically discover necessary tools in the process of handling the task, execute related operations, and manage its own memory information by compressing interaction history information. This processing approach abandons the traditional, fixed agent workflow, enabling the intelligent agent to maintain a global perspective when handling tasks.

[0074] In one or more specific embodiments provided in this specification, the method further includes: Receive sample processing task and at least one sample subtask information corresponding to the sample processing task; The sample processing task is parsed to obtain the current sub-task to be processed for the sample corresponding to the sample processing task, and the sample historical sub-task information corresponding to the current sub-task to be processed for the sample. Based on the historical subtask information of the sample, the predicted subtask execution information of the current subtask to be processed of the sample is determined, and the current subtask to be processed of the sample is executed based on the predicted subtask execution information to obtain the predicted current subtask information; The model reward information is calculated based on the predicted current subtask information and the sample subtask information, and the intelligent processing unit is trained based on the model reward information.

[0075] In one specific embodiment provided in this specification, a method for training an agent is also provided. In the method provided in this specification, a tool strategy is used to train the agent in an end-to-end manner.

[0076] Specifically, the system receives sample training data for the intelligent agent. This data includes sample processing tasks and at least one corresponding sample subtask information. In practical applications, sample processing tasks include tool usage tasks, interaction tasks, deep analysis tasks, and mathematical reasoning tasks.

[0077] The sample processing task is input into the agent, which parses the sample processing task to obtain the current subtask to be processed corresponding to the sample processing task, as well as the sample historical subtask information corresponding to the current subtask to be processed. The predicted subtask execution information of the current subtask to be processed is determined by the sample historical subtask information, and the current subtask to be processed is executed based on the predicted subtask execution information to obtain the predicted current subtask information.

[0078] The training samples also include sample subtask information corresponding to each subtask. The agent is trained using the sample subtask information and the information for predicting the current subtask.

[0079] In the method provided in the embodiments of this specification, two reward information is defined. The first reward information is the reward information for the sample processing task as a whole, which reflects the quality of the final output result of the agent in processing the sample processing task. The second reward information represents the agent's use of tools during the sample processing task, which reflects the quality of the agent's intermediate behavior. A higher reward value will be set if the agent can correctly use tools and efficiently extract historical information (memory folding).

[0080] Based on the first and second reward information, their respective relative advantage information can be calculated, and the agent can be adjusted accordingly, specifically, the inference model corresponding to the agent can be adjusted. The first and second reward information provide targeted signals for learning correct and efficient tools.

[0081] In one or more specific embodiments provided in this specification, the focus is on training the agent to query and invoke tools, which involves assigning the agent a tool set consisting of multiple tools. However, due to issues such as network latency, tool instability, and high cost of invoking tools within the tool set, interacting with the actual tool set during the agent's training phase would result in high training costs and heavy resource burdens. Therefore, the method further includes: When the execution information of the predicted subtask is a tool call, tool call parameters for the target tool are generated, and tool description information of the target tool is obtained. The tool call parameters and the tool description information are input into the tool simulation call model to obtain the predicted tool call result returned by the tool simulation call model.

[0082] In the training method for intelligent agents provided in the embodiments of this specification, when the predicted subtask execution information is a tool call, a tool simulator will be used instead of a real tool call.

[0083] Specifically, the tool simulator, driven by a tool invocation simulation model, can simulate the response of a real tool. When the agent determines that the execution information of the predicted subtask is a tool invocation, it generates tool invocation parameters for the target tool and simultaneously obtains the tool description information. These parameters and description information are then input into the tool invocation simulation model to obtain the predicted tool invocation result returned by the model. By designing the tool simulator, a stable, efficient, and low-cost interactive environment is provided for the agent's training, enabling robust reinforcement learning training.

[0084] The data processing method provided in the embodiments of this specification determines the process of an intelligent agent handling a task through a pre-trained inference model. Through this method, the intelligent agent can autonomously infer the task, dynamically discover necessary tools in the process of handling the task, execute related operations, and manage its own memory information by compressing interaction history information. This processing approach abandons the traditional, fixed agent workflow, enabling the intelligent agent to maintain a global perspective when handling tasks.

[0085] During the agent's training phase, a reinforcement learning training method using a tool simulator was employed. This enabled the agent to learn how to use tools conveniently and efficiently, and to operate stably in complex environments. Furthermore, two reward strategies were used: one rewarded for correctly handling the task at hand, and another for correctly calling up tools and efficiently performing memory folding. This allowed the agent to use tools and perform memory folding more flexibly, efficiently, and accurately, thereby improving its processing capabilities and ultimately enhancing the user experience.

[0086] The following is in conjunction with the appendix Figure 2 Taking the data processing method provided in this specification as an example of its application in multi-tool task invocation, the data processing method will be further explained. Among other things, Figure 2 This specification illustrates a data processing method for a multi-tool task invocation scenario, according to an embodiment of this specification. The method includes the following steps: Step 202: Receive the pending tasks input by the user.

[0087] In the method provided in the embodiments of this specification, the user-input task includes: "I am organizing a film festival and need help finding the best films. Can you search for videos related to documentaries on a certain website? Also, find relevant people in the 'Film' category and invite them as speakers. Finally, please provide a streaming link for video A, whose ID is 'abc'."

[0088] Step 204: Input the task to be processed into the agent. The agent parses the task to be processed and obtains the current subtask to be processed and the historical subtask information corresponding to the current subtask to be processed.

[0089] In the method provided in the embodiments of this specification, the agent parses the task to be processed and obtains three sub-tasks: searching for documentaries on a certain website, finding movie-related tasks on the same website, and obtaining the streaming link of video A. Based on this, the agent determines that the current sub-task to be processed is a suitable tool for searching.

[0090] If the current subtask to be processed is the first subtask, there will be no historical subtask information. If the current subtask to be processed is a subsequent subtask, the corresponding historical subtask information can be obtained.

[0091] Step 206: The agent determines the subtask execution information of the current subtask to be processed based on the historical subtask information.

[0092] In the method provided in the embodiments of this specification, the agent searches within a preset set of tools. Specifically, it finds search tools that are closely related to the current subtask to be processed, based on keywords such as "video search on a certain website, for documentary purposes".

[0093] Step 208: After determining the search tool, the agent determines that the current subtask to be processed is to use the search tool to search for documentaries and obtain the query results.

[0094] In the method provided in the embodiments of this specification, the current subtask to be processed is changed to search, that is, search information is generated according to the search task, and a search is performed using a search tool based on the search information, thereby obtaining a list of popular documentary videos.

[0095] Step 210: The agent switches the current pending subtask to finding relevant people in the "Movies" category.

[0096] In the solution provided in the embodiments of this specification, the intelligent agent did not find any relevant tools to search for people. Therefore, it searched for videos tagged with "movie" using the aforementioned search tools and identified their corresponding creators. Users A and B were obtained through the search and were then selected as candidates for speaker positions.

[0097] Step 212: The agent switches the current pending subtask to finding the streaming connection for video A.

[0098] In the solution provided in the embodiments of this specification, a video ID verification tool and a streaming media link building tool are selected from the toolset. The video ID verification tool verifies whether the video ID 'abc' is valid. After successful verification, the link to video A is obtained, and the streaming media link corresponding to video A is built using the streaming media link building tool.

[0099] Step 214: The agent integrates the above content and outputs the final result.

[0100] In the solution provided in the embodiments of this specification, the intelligent agent outputs "1. A list of documentaries found on a certain website; 2. Speaker candidates are user A and user B; 3. The streaming media link is "http: / / *******".

[0101] Corresponding to the above method embodiments, this specification also provides data processing apparatus embodiments. Figure 3 A schematic diagram of the structure of a data processing apparatus provided in one embodiment of this specification is shown. For example... Figure 3 As shown, the device includes: The receiving module 302 is configured to receive tasks to be processed; The processing module 304 is configured to input the task to be processed into the intelligent processing unit and obtain the task processing result output by the intelligent processing unit. The task processing result is generated based on a preset toolset to execute each sub-task corresponding to the task to be processed. The sub-tasks to be processed are determined based on the reasoning model corresponding to the intelligent processing unit according to the task to be processed and historical sub-task information. When an information extraction identifier is detected, the historical sub-task information is extracted and generated from the interaction history information.

[0102] In one specific embodiment provided in this specification, the processing module 304 is further configured to: Based on the reasoning model corresponding to the intelligent processing unit, the task to be processed is analyzed to obtain the current sub-task to be processed corresponding to the task to be processed, and the historical sub-task information corresponding to the current sub-task to be processed. Based on the historical subtask information, determine the subtask execution information of the current subtask to be processed, and execute the current subtask to be processed based on the subtask execution information to obtain the current subtask information; If it is determined that the pending task has not met the task termination condition, the current subtask information is added to the historical subtask information, and the operation of obtaining the current pending subtask corresponding to the pending task continues until the pending task meets the task termination condition and the task processing result is generated.

[0103] In one specific embodiment provided in this specification, the historical subtask information includes historical subtask action information and historical subtask feedback information; The processing module 304 is further configured to: Based on the task to be processed, the historical subtask action information, and the historical subtask feedback information, the current subtask action information of the current subtask to be processed is determined.

[0104] In one specific embodiment provided in this specification, the processing module 304 is further configured to: Execute the current subtask to be processed based on the current subtask action information, and obtain the current subtask feedback information corresponding to the current subtask action information.

[0105] In one specific embodiment provided in this specification, the processing module 304 is further configured to: The current subtask action information and the current subtask feedback information are saved to the historical subtask information.

[0106] In one specific embodiment provided in this specification, the processing module 304 is further configured to: If the current subtask action information is a thinking action, analyze the current subtask to be processed and obtain the task analysis result corresponding to the current subtask to be processed. If the current subtask action information is a tool search, obtain the subtask requirement information of the current subtask to be processed, and search in a preset tool set based on the subtask requirement information to obtain the tool search results; If the current subtask action information is a tool call, generate tool call parameters for the target tool, and call the target tool based on the tool call parameters to obtain the tool call result; If the current subtask action information is information extraction, the interaction history information of the task to be processed is sent to the information extraction model to obtain the reference prompt information output by the information extraction model.

[0107] In one specific embodiment provided in this specification, the processing module 304 is further configured to: The interaction history information of the task to be processed is sent to the information extraction model to obtain the historical record information, work record information and tool call information output by the information extraction model.

[0108] In one specific embodiment provided in this specification, the processing module 304 is further configured to: If an information extraction identifier is detected, the current subtask action information is determined to be information extraction.

[0109] In one specific embodiment provided in this specification, the processing module 304 is further configured to: If the information folding condition is met based on the historical subtask information, an information extraction identifier is generated.

[0110] In one specific embodiment provided in this specification, the device further includes a training module configured to: Receive sample processing task and at least one sample subtask information corresponding to the sample processing task; The sample processing task is parsed to obtain the current sub-task to be processed for the sample corresponding to the sample processing task, and the sample historical sub-task information corresponding to the current sub-task to be processed for the sample. Based on the historical subtask information of the sample, the predicted subtask execution information of the current subtask to be processed of the sample is determined, and the current subtask to be processed of the sample is executed based on the predicted subtask execution information to obtain the predicted current subtask information; The model reward information is calculated based on the predicted current subtask information and the sample subtask information, and the intelligent processing unit is trained based on the model reward information.

[0111] In one specific embodiment provided in this specification, the training module is further configured as follows: When the execution information of the predicted subtask is a tool call, tool call parameters for the target tool are generated, and tool description information of the target tool is obtained. The tool call parameters and the tool description information are input into the tool simulation call model to obtain the predicted tool call result returned by the tool simulation call model.

[0112] The data processing apparatus provided in the embodiments of this specification determines the process of an intelligent agent handling a task through a pre-trained inference model. Through this method, the intelligent agent can autonomously infer the task, dynamically discover necessary tools in the process of handling the task, execute related operations, and manage its own memory information by compressing interaction history information. This processing method abandons the traditional, fixed agent workflow, enabling the intelligent agent to maintain a global perspective when handling tasks.

[0113] During the agent's training phase, a reinforcement learning training method using a tool simulator was employed. This enabled the agent to learn how to use tools conveniently and efficiently, and to operate stably in complex environments. Furthermore, two reward strategies were used: one rewarded for correctly handling the task at hand, and another for correctly calling up tools and efficiently performing memory folding. This allowed the agent to use tools and perform memory folding more flexibly, efficiently, and accurately, thereby improving its processing capabilities and ultimately enhancing the user experience.

[0114] The above is an illustrative scheme of a data processing apparatus according to this embodiment. It should be noted that the technical solution of this data processing apparatus and the technical solution of the data processing method described above belong to the same concept. For details not described in detail in the technical solution of the data processing apparatus, please refer to the description of the technical solution of the data processing method described above.

[0115] See Figure 4 , Figure 4 This specification illustrates an architecture diagram of a data processing system according to one embodiment of the present specification. The data processing system may include a client 100 and a server 200. Client 100 is used to send pending tasks to server 200; Server 200 is used to input the task to be processed into the intelligent processing unit, obtain the task processing result output by the intelligent processing unit, wherein the task processing result is generated based on a preset toolset to execute each sub-task corresponding to the task to be processed, the sub-tasks to be processed are determined based on the inference model corresponding to the intelligent processing unit according to the task to be processed and historical sub-task information, and when an information extraction identifier is detected, the historical sub-task information is extracted and generated from the interaction history information; and the task processing result is sent to client 100. Client 100 is also used to receive task processing results sent by server 200.

[0116] The data processing system may include multiple clients 100 and a server 200. Clients 100 can be referred to as edge devices, and server 200 can be referred to as cloud devices. Multiple clients 100 can establish communication connections through server 200. In the data processing scenario, server 200 is used to provide data processing services between multiple clients 100. Each client 100 can act as a sender or receiver, communicating through server 200.

[0117] Users can interact with server 200 through client 100 to receive data sent by other clients 100, or send data to other clients 100, etc. In data processing scenarios, users can publish data streams to server 200 through client 100, server 200 can generate task processing results based on the data stream, and push the task processing results to other clients that have established communication.

[0118] In this system, client 100 and server 200 establish a connection via a network. The network provides the medium for communication between client 100 and server 200. The network can include various connection types, such as wired or wireless communication links or fiber optic cables. Data transmitted by client 100 may need to undergo encoding, transcoding, compression, or other processing before being published to server 200.

[0119] Client 100 can be a browser, an app (application), a web application such as an H5 (HyperText Markup Language 5) application, a lightweight application (also known as a mini-program), or a cloud application. Client 100 can be developed based on the software development kit (SDK) of the corresponding service provided by server 200, such as a real-time communication (RTC) SDK. Client 100 can be deployed on a computing device and depends on the device or certain apps on the device to run. The computing device may have a display screen and support information browsing, such as a personal mobile terminal like a mobile phone, tablet, or personal computer. Various other types of applications can also be configured on the computing device, such as human-computer interaction applications, model training applications, text processing applications, web browser applications, shopping applications, search applications, instant messaging tools, email clients, and social media platform software.

[0120] Server 200 may include servers providing various services, such as servers providing communication services to multiple clients, servers supporting backend training of models used on clients, and servers processing data sent by clients. It should be noted that server 200 can be implemented as a distributed server cluster composed of multiple servers, or as a single server. The server can also be a server in a distributed system, or a server integrated with blockchain. The server can also be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology.

[0121] It is worth noting that the data processing methods provided in the embodiments of this specification are generally executed by the server. However, in other embodiments of this specification, the client may also have similar functions to the server, thereby executing the data processing methods provided in the embodiments of this specification. In other embodiments, the data processing methods provided in the embodiments of this specification may also be executed jointly by the client and the server.

[0122] Figure 5 A structural block diagram of a computing device 500 according to an embodiment of this specification is shown. The components of the computing device 500 include, but are not limited to, a memory 510 and a processor 520. The processor 520 is connected to the memory 510 via a bus 530, and a database 550 is used to store data.

[0123] The computing device 500 also includes an access device 540, which enables the computing device 500 to communicate via one or more networks 560. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 540 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and so on.

[0124] In one embodiment of this specification, the above-described components of the computing device 500 and Figure 5 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 5 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0125] The computing device 500 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 500 can also be a mobile or stationary server.

[0126] The processor 520 is used to execute the following computer program / instructions, which, when executed by the processor, implement the steps of the above-described data processing method.

[0127] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the data processing method described above belong to the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the data processing method described above.

[0128] An embodiment of this specification also provides a computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the above-described data processing method.

[0129] The above is an illustrative embodiment of a computer-readable storage medium. It should be noted that the technical solution of this storage medium and the technical solution of the data processing method described above belong to the same concept. Details not described in detail in the technical solution of the storage medium can be found in the description of the technical solution of the data processing method described above.

[0130] An embodiment of this specification also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described data processing method.

[0131] The above is an illustrative scheme of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product and the technical solution of the data processing method described above belong to the same concept. For details not described in detail in the technical solution of the computer program product, please refer to the description of the technical solution of the data processing method described above.

[0132] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0133] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0134] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this specification is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this specification. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this specification.

[0135] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0136] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. These embodiments have been selected and specifically described in this specification to better explain the principles and practical applications of this specification, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A data processing method, characterized by, The method comprises the following steps: receiving a to-be-processed task; inputting the to-be-processed task into an intelligent processing unit to obtain a task processing result output by the intelligent processing unit, wherein the task processing result is generated based on a preset tool set executing each to-be-processed subtask corresponding to the to-be-processed task, and the to-be-processed subtask is determined based on an inference model corresponding to the intelligent processing unit according to the to-be-processed task and historical subtask information, and the historical subtask information is extracted from interaction history information in the case of detecting an information extraction identifier.

2. The method of claim 1, wherein, inputting the to-be-processed task into an intelligent processing unit to obtain a task processing result output by the intelligent processing unit, comprising: analyzing the to-be-processed task based on an inference model corresponding to the intelligent processing unit to obtain a current to-be-processed subtask corresponding to the to-be-processed task and historical subtask information corresponding to the current to-be-processed subtask; determining subtask execution information of the current to-be-processed subtask based on the historical subtask information, and executing the current to-be-processed subtask based on the subtask execution information to obtain current subtask information; in the case of determining that the to-be-processed task does not reach a task termination condition, adding the current subtask information to the historical subtask information, and continuing to execute the operation of obtaining the current to-be-processed subtask corresponding to the to-be-processed task until the to-be-processed task meets the task termination condition to generate a task processing result.

3. The method of claim 2, wherein, The historical subtask information comprises historical subtask action information and historical subtask feedback information. determining the subtask execution information of the current to-be-processed subtask based on the historical subtask information, comprising: determining current subtask action information of the current to-be-processed subtask according to the to-be-processed task, the historical subtask action information and the historical subtask feedback information.

4. The method of claim 3, wherein, executing the current to-be-processed subtask based on the subtask execution information to obtain current subtask information, comprising: executing the current to-be-processed subtask based on the current subtask action information to obtain current subtask feedback information corresponding to the current subtask action information.

5. The method of claim 4, wherein, adding the current subtask information to the historical subtask information, comprising: correspondingly saving the current subtask action information and the current subtask feedback information to the historical subtask information.

6. The method of claim 4, wherein, executing the current to-be-processed subtask based on the current subtask action information to obtain current subtask feedback information corresponding to the current subtask action information, comprising: in the case of the current subtask action information being a thinking action, analyzing the current to-be-processed subtask to obtain a task analysis result corresponding to the current to-be-processed subtask; in the case of the current subtask action information being a tool search, obtaining subtask requirement information of the current to-be-processed subtask, and searching in a preset tool set based on the subtask requirement information to obtain a tool search result; in the case of the current subtask action information being a tool call, generating a tool call parameter for a target tool, and calling the target tool based on the tool call parameter to obtain a tool call result; In a case where the current subtask action information is information extraction, the interaction history information of the to-be-processed task is sent to an information extraction model to obtain reference prompt information output by the information extraction model.

7. The method of claim 6, wherein, The interaction history information of the to-be-processed task is sent to an information extraction model to obtain reference prompt information output by the information extraction model, including: The interaction history information of the to-be-processed task is sent to an information extraction model to obtain historical record information, work record information and tool calling information output by the information extraction model.

8. The method of claim 3, wherein, The method further includes: In a case where the information extraction identifier is detected, it is determined that the current subtask action information is information extraction.

9. The method of claim 8, wherein, The method further includes: In a case where the information folding condition is met according to the historical subtask information, an information extraction identifier is generated.

10. The method of any one of claims 1-9, wherein, The method further includes: A sample processing task and at least one sample subtask information corresponding to the sample processing task are received. The sample processing task is parsed to obtain a sample current to-be-processed subtask corresponding to the sample processing task and sample historical subtask information corresponding to the sample current to-be-processed subtask. Predicted subtask execution information of the sample current to-be-processed subtask is determined based on the sample historical subtask information, and the sample current to-be-processed subtask is executed based on the predicted subtask execution information to obtain predicted current subtask information. Model reward information is calculated according to the predicted current subtask information and a sample subtask information calculation model, and the intelligent processing unit is trained based on the model reward information.

11. The method of claim 10, wherein, The method further includes: In a case where the predicted subtask execution information is tool calling, tool calling parameters for a target tool are generated, and tool description information of the target tool is obtained, the tool calling parameters and the tool description information are input to a tool simulation calling model to obtain a predicted tool calling result returned by the tool simulation calling model.

12. A data processing apparatus, characterized by It includes: A receiving module configured to receive a to-be-processed task; A processing module configured to input the to-be-processed task into an intelligent processing unit to obtain a task processing result output by the intelligent processing unit, wherein the task processing result is generated based on each to-be-processed subtask corresponding to the to-be-processed task being executed by a preset tool set, and each to-be-processed subtask is determined based on an inference model corresponding to the intelligent processing unit according to the to-be-processed task and historical subtask information, and in a case where an information extraction identifier is detected, the historical subtask information is extracted from interaction history information to generate.

13. A computing device, comprising: It includes: A memory and a processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which realize the steps of the method in any one of claims 1 to 11 when executed by the processor.

14. A computer readable storage medium storing computer programs / instructions, characterized in that, The computer programs / instructions realize the steps of the method in any one of claims 1 to 11 when executed by the processor.

15. A computer program product comprising computer programs / instructions, characterized in that, The computer programs / instructions realize the steps of the method in any one of claims 1 to 11 when executed by the processor.

Images