Intelligent agent data processing method and related device

Abstract

The invention discloses an intelligent agent data processing method and a related device. Based on a target data type of target input data of the input target agent, determining a processing tool corresponding to the target data type from multi-modal processing tools of the target agent; determining a target state of the target agent through the target input data, a processing tool corresponding to the target data type and output data, namely historical processing data, of a previous execution step of a target execution step to which the target input data belongs; according to the method, a target action is determined from a plurality of candidate actions based on a target state according to an action determination strategy of a target agent for a plurality of candidate actions determined by a multi-modal processing tool and a plurality of processing steps of the multi-modal processing tool, so that an optimal processing tool and an optimal processing step are flexibly and adaptively selected; through the processing tool in the target action, the processing steps in the target action are executed on the target input data to obtain the target output data, and the data processing effect of the intelligent agent is improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a data processing method and related apparatus for an intelligent agent. Background Technology

[0002] With the rapid development of artificial intelligence technology, intelligent agents are widely used in many fields. The data to be processed for a task is input into the intelligent agent, which then uses processing tools to perform processing steps on the data, obtaining the processed data.

[0003] In related technologies, the data processing method of intelligent agents is as follows: for the input data of the task to be performed, the intelligent agent has fixed processing tools and processing steps, and the intelligent agent performs fixed processing steps on the input data through fixed processing tools to obtain output data.

[0004] However, in the above methods, the agent performs fixed processing steps on the input data using fixed processing tools to obtain the output data. This results in a problem that the agent cannot flexibly and adaptively select the optimal processing tools and processing steps, leading to poor data processing results. Summary of the Invention

[0005] To address the aforementioned technical problems, this application provides a data processing method and related apparatus for an intelligent agent. The intelligent agent automatically selects a processing tool that matches the data type of the input data. Based on the state formed by the input data, the selected processing tool, and the output data from the previous step, the intelligent agent flexibly and adaptively selects the optimal processing tool and processing steps from multiple processing steps of the multimodal processing tool, using an action determination strategy. The intelligent agent then executes the optimal processing steps on the input data using the optimal processing tool, improving the accuracy of the output data and thus enhancing the data processing effect of the intelligent agent.

[0006] The embodiments of this application disclose the following technical solutions:

[0007] On one hand, embodiments of this application provide a data processing method for an intelligent agent, the method comprising:

[0008] Based on the target data type of the target input data of the target intelligent agent, determine the processing tool corresponding to the target data type from the multimodal processing tools of the target intelligent agent;

[0009] The target input data, the processing tool corresponding to the target data type, and the historical processing data of the target execution step to which the target input data belongs are determined as the target state; the historical processing data is the output data of the previous execution step of the target execution step.

[0010] Based on the target state and the action determination strategy of the target agent, a target action is determined from multiple candidate actions; the multiple candidate actions are determined based on the multimodal processing tool and multiple processing steps of the multimodal processing tool.

[0011] The processing tools in the target action execute the processing steps in the target action on the target input data to obtain the target output data.

[0012] On the other hand, embodiments of this application provide a data processing device for an intelligent agent, the device comprising: a determining unit and an execution unit;

[0013] The determining unit is used to determine the processing tool corresponding to the target data type from the multimodal processing tools of the target intelligent agent based on the target data type of the target input data of the target intelligent agent.

[0014] The determining unit is further configured to determine the target input data, the processing tool corresponding to the target data type, and the historical processing data of the target execution step to which the target input data belongs as the target state; the historical processing data is the output data of the previous execution step of the target execution step;

[0015] The determining unit is further configured to determine a target action from multiple candidate actions based on the target state and the action determination strategy of the target agent; the multiple candidate actions are determined based on the multimodal processing tool and multiple processing steps of the multimodal processing tool;

[0016] The execution unit is used to perform the processing steps of the target action on the target input data through the processing tool in the target action to obtain the target output data.

[0017] On the other hand, embodiments of this application provide a computer device, the computer device including a processor and a memory:

[0018] The memory is used to store computer programs and to transfer the computer programs to the processor;

[0019] The processor is configured to execute the method described in any of the foregoing aspects according to instructions in the computer program.

[0020] On the other hand, embodiments of this application provide a computer-readable storage medium for storing a computer program that, when run on a computer device, causes the computer device to perform the methods described in any of the foregoing aspects.

[0021] On the other hand, embodiments of this application provide a computer program product, including a computer program that, when run on a computer device, causes the computer device to perform the method described in any of the foregoing aspects.

[0022] As can be seen from the above technical solution, based on the target data type of the target input data of the target intelligent agent, the corresponding processing tool for the target data type is determined from the multimodal processing tools of the target intelligent agent, realizing the automatic selection of the processing tool that matches the target data type of the target input data; through the target input data, the processing tool corresponding to the target data type, and the output data of the previous execution step of the target execution step to which the target input data belongs, i.e., historical processing data, the target state of the target intelligent agent is determined, so as to clarify the target state in which the target intelligent agent selects the optimal processing tool and processing step; based on the determination of multiple candidate actions by the multimodal processing tool and multiple processing steps of the multimodal processing tool, the multiple candidate actions cover the multiple processing steps of the multimodal processing tool and, based on the target state and according to the action determination strategy of the target intelligent agent, the target action is determined from the multiple candidate actions, realizing the selection of the optimal processing tool and processing step considering the target state to form the target action; through the processing tool in the target action, the processing steps in the target action are executed on the target input data, realizing the execution of the optimal processing steps on the target input data by the optimal processing tool, and obtaining accurate target output data.

[0023] Based on this, the method allows the agent to automatically select a processing tool that matches the data type of the input data. The agent, through the state formed by the input data, the selected processing tool, and the output data from the previous step, flexibly and adaptively selects the optimal processing tool and processing steps from the multimodal processing tools and their multiple processing steps, based on an action determination strategy. The agent then uses the optimal processing tool among the actions to execute the optimal processing steps on the input data, improving the accuracy of the output data and thus enhancing the agent's data processing performance. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 A system schematic diagram of a data processing method for an intelligent agent provided in an embodiment of this application;

[0026] Figure 2A flowchart illustrating a data processing method for an intelligent agent provided in an embodiment of this application;

[0027] Figure 3 A flowchart illustrating the steps for determining the processing tool corresponding to a target data type when providing a multimodal processing tool including a text processing tool, an image processing tool, and a speech processing tool in an embodiment of this application;

[0028] Figure 4 A flowchart illustrating steps for obtaining target output data when the output data of a processing tool in a target action is erroneous, as provided in an embodiment of this application.

[0029] Figure 5 A flowchart illustrating the steps of a target agent for obtaining value based on updating state-action pairs with sample rewards, as provided in this application embodiment;

[0030] Figure 6 A structural diagram of a data processing device for an intelligent agent provided in an embodiment of this application;

[0031] Figure 7 A structural diagram of a server provided in an embodiment of this application;

[0032] Figure 8 This is a structural diagram of a terminal provided in an embodiment of this application. Detailed Implementation

[0033] The embodiments of this application will now be described with reference to the accompanying drawings.

[0034] Currently, intelligent agents have fixed processing tools and steps for the input data of a task. After acquiring the input data, the agent performs these fixed processing steps using the fixed tools to obtain the output data. However, for tasks, especially complex ones, the agent cannot flexibly and adaptively select the optimal processing tool to perform the optimal processing steps on the input data, resulting in inaccurate output data and poor data processing results.

[0035] This application provides a data processing method for an intelligent agent. The intelligent agent automatically selects a processing tool that matches the data type of the input data. Based on the state formed by the input data, the selected processing tool, and the historical processing data from the previous step, the intelligent agent flexibly and adaptively selects the optimal processing tool and processing steps from multiple processing steps of the multimodal processing tool, according to an action determination strategy. The intelligent agent executes the optimal processing steps on the input data using the optimal processing tool, thereby improving the accuracy of the output data and enhancing the data processing effect of the intelligent agent.

[0036] The data processing method for intelligent agents provided in this application can be applied to text processing tasks, image processing tasks, speech processing tasks, multimodal processing tasks, etc.

[0037] (a) Text processing tasks

[0038] Intelligent agents for text processing tasks need to understand, generate, analyze, or transform text data. Text processing tasks can include text recognition, text analysis, text summarization, text translation, text generation, text classification, text annotation, etc.

[0039] (II) Image Processing Tasks

[0040] Intelligent agents need to analyze, generate, or modify image data for image processing tasks. Image processing tasks can include image recognition, image analysis, image generation, image classification, image segmentation, image inpainting, image enhancement, and image reconstruction, among others.

[0041] (III) Speech Processing Tasks

[0042] Intelligent agents need to recognize, synthesize, or convert speech data for speech processing tasks. Speech processing tasks can include speech recognition, speech analysis, speech generation, speech enhancement, speech conversion, speech separation, etc.

[0043] (iv) Multimodal processing tasks

[0044] Intelligent agents need to fuse multimodal data for multimodal processing tasks. Multimodal processing tasks can be text-image processing tasks, text-speech processing tasks, image-speech processing tasks, text-image-speech processing tasks, etc.

[0045] Next, the system architecture of the data processing method for intelligent agents will be introduced. (See also...) Figure 1 ,Should Figure 1 This is a system schematic diagram of a data processing method for an intelligent agent provided in an embodiment of this application. The system includes a terminal 101 and a server 102, which are used to execute the data processing method for the intelligent agent.

[0046] Terminal 101 can be a smartphone, tablet computer, laptop computer, desktop computer, intelligent voice interaction device, vehicle terminal, extended reality device or aircraft, etc., but is not limited to these.

[0047] Among them, server 102 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services, but it is not limited to these.

[0048] Furthermore, terminal 101 and server 102 can be connected directly or indirectly via wired or wireless communication, and this application does not impose any limitations on this. For example, terminal 101 and server 102 can be connected via a network, which can be a wired or wireless network.

[0049] Terminal 101 acquires the target input data of the target agent; terminal 101 sends the target input data of the target agent to server 102. Server 102 determines the processing tool corresponding to the target data type from the multimodal processing tools of the target agent based on the target data type of the target input data.

[0050] As an example, the intelligent agent is called an Agent. Terminal 101 obtains target input data from the target Agent, including target text data and target image data. Server 102, based on the target data types (text type and image type) of the target text data and target image data, determines the corresponding text type processing tool and the corresponding image type processing tool from the multimodal processing tools of the target Agent.

[0051] Server 102 determines the target state as the target input data, the processing tool corresponding to the target data type, and the historical processing data of the target execution step to which the target input data belongs; the historical processing data is the output data of the previous execution step of the target execution step.

[0052] As an example, based on the above example, server 102 determines that the target state of the target agent is target s by using the target text data and target image data, the processing tools corresponding to the text type and the processing tools corresponding to the image type, and the output data of the previous execution step of the target execution step to which the target text data and target image data belong, i.e., historical processing data.

[0053] Server 102 determines a strategy based on the target state and the target agent's actions, and determines the target action from multiple candidate actions; the multiple candidate actions are determined based on the multimodal processing tool and multiple processing steps of the multimodal processing tool.

[0054] As an example, based on the above example, server 102 determines that the target action is target a from the multiple candidate a based on the multimodal processing tool and the multiple processing steps of the multimodal processing tool, according to the target agent's action determination strategy.

[0055] Server 102 uses the processing tools in the target action to execute the processing steps in the target action on the target input data to obtain the target output data. Server 102 sends the target output data to terminal 101 so that terminal 101 can display the target output data.

[0056] As an example, based on the above example, server 102 uses the processing tools in target a to perform the processing steps in target a on the target text data and target image data to obtain target output data. Server 102 sends the target output data to terminal 101 so that terminal 101 can display the target output data.

[0057] In other words, based on the target data type of the target intelligent agent's target input data, the corresponding processing tool is determined from the multimodal processing tools of the target intelligent agent, realizing the automatic selection of a processing tool that matches the target data type of the target input data; the target state of the target intelligent agent is determined through the target input data, the processing tool corresponding to the target data type, and the output data of the previous execution step of the target execution step to which the target input data belongs, i.e., historical processing data, to clarify the target state in which the target intelligent agent selects the optimal processing tool and processing step; based on the determination of multiple candidate actions by the multimodal processing tool and its multiple processing steps, and with multiple candidate actions covering multiple processing steps of the multimodal processing tool and its multiple processing steps, the target action is determined from multiple candidate actions according to the target intelligent agent's action determination strategy based on the target state, realizing the selection of the optimal processing tool and processing step considering the target state to form the target action; through the processing tool in the target action, the processing steps in the target action are executed on the target input data, realizing the execution of the optimal processing steps on the target input data by the optimal processing tool, and obtaining accurate target output data.

[0058] It should be noted that, in the embodiments of this application, the execution device for the game character control method is a computer device, which can be a server or a terminal. The method provided in the embodiments of this application can be executed by the terminal or the server alone, or by the terminal and the server in cooperation. Specifically, when the method provided in the embodiments of this application is executed by the terminal or the server alone, its execution method is similar to... Figure 1 The corresponding embodiments are similar, mainly by merging the execution steps of terminal 101 and server 102 into the execution steps of the terminal, or merging the execution steps of terminal 101 and server 102 into the execution steps of the server.

[0059] The terminal can be a smartphone, tablet, laptop, desktop computer, intelligent voice interaction device, vehicle terminal, extended reality device, or aircraft, but is not limited to these. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services, but is not limited to these. The terminal and server can be connected directly or indirectly through wired or wireless communication, and this application does not impose any restrictions. For example, the terminal and server can be connected through a network, which can be wired or wireless.

[0060] Next, the data processing method of the intelligent agent provided in the embodiments of this application will be described in detail with reference to the accompanying drawings, taking the execution of the method provided in the embodiments of this application by a computer device as an example.

[0061] See Figure 2 ,Should Figure 2 A flowchart of a data processing method for an intelligent agent provided in an embodiment of this application is included. The method includes:

[0062] S201: Based on the target data type of the target input data of the target agent, determine the processing tool corresponding to the target data type from the multimodal processing tools of the target agent.

[0063] S202: The target state is determined by the target input data, the processing tool corresponding to the target data type, and the historical processing data of the target execution step to which the target input data belongs; the historical processing data is the output data of the previous execution step of the target execution step.

[0064] S203: Determine the strategy based on the target state and the target agent's actions, and determine the target action from multiple candidate actions; the multiple candidate actions are determined based on the multimodal processing tool and multiple processing steps of the multimodal processing tool.

[0065] In related technologies, intelligent agents have fixed processing tools and steps for the input data of a task to be performed. After acquiring the input data, the intelligent agent performs fixed processing steps on the input data using fixed tools to obtain output data. However, for the task to be performed, especially complex tasks, the intelligent agent cannot flexibly and adaptively select the optimal processing tool to perform the optimal processing steps on the input data, and cannot obtain accurate output data, resulting in poor data processing results for the intelligent agent.

[0066] In this embodiment of the application, in order to solve the above problems, considering that reinforcement learning allows an agent to learn how to choose the optimal action to obtain the maximum reward through interaction with the environment; based on the agent having a multimodal processing tool, the agent trained by reinforcement learning can take the input data, the processing tool selected by the multimodal processing tool based on the data type of the input data, and the output data of the previous execution step of the execution step to which the input data belongs, i.e., historical processing data, as the state. From the multiple candidate actions determined by the multimodal processing tool and multiple processing steps of the multimodal processing tool, based on the agent's action determination strategy, the agent flexibly and adaptively selects the optimal processing tool and processing step as the action, so that the agent can improve the accuracy of the output data when executing the action in the future.

[0067] Based on this, for the target intelligent agent receiving target input data, the processing tool corresponding to the target data type is determined from the multimodal processing tools of the target intelligent agent based on the target data type of the target input data. The target state of the target intelligent agent is determined by the target input data, the processing tool corresponding to the target data type, and the output data of the previous execution step of the target execution step to which the target input data belongs, i.e., the historical processing data. Based on the determination of multiple candidate actions by the multimodal processing tool and its multiple processing steps, the target action is determined from the multiple candidate actions according to the target intelligent agent's action determination strategy based on the target state.

[0068] Here, the target agent is the agent trained through reinforcement learning; the target input data is the input data of the target agent; the target data type is the data type of the target input data; the multimodal processing tool includes processing tools for different modalities, i.e., processing tools corresponding to different data types; the processing tool corresponding to the target data type is the processing tool in the multimodal processing tool that can process the input data of the target data type; multiple candidate actions are multiple actions formed through the multimodal processing tool and multiple processing steps of the multimodal processing tool; the action determination strategy is the strategy for selecting actions based on the state; and the target action is the candidate action selected from multiple candidate actions according to the action determination strategy in the target state.

[0069] Specifically, when the target execution step to which the target input data belongs is the first execution step, the historical processing data of the first execution step is empty; when the target execution step to which the target input data belongs is the second execution step, the historical processing data of the second execution step is the output data of the first execution step; when the target execution step to which the target input data belongs is the third execution step, the historical processing data of the third execution step is the output data of the second execution step, and so on, without further explanation.

[0070] Where the target input data is X, and the target data type of X is T, then the processing tool corresponding to the target data type is M=Route(X,T); Route(X,T) means that the processing tool corresponding to the target data type is determined from the multimodal processing tools of the target agent based on the target data type of the target input data.

[0071] S201-S203, based on the target data type of the target intelligent agent's target input data, determines the corresponding processing tool from the multimodal processing tools of the target intelligent agent, thus automatically selecting a processing tool that matches the target data type of the target input data. By using the target input data, the processing tool corresponding to the target data type, and the output data of the previous execution step (i.e., historical processing data) of the target execution step to which the target input data belongs, the target state of the target intelligent agent is determined. This clarifies the target state in which the target intelligent agent selects the optimal processing tool and processing step, laying the foundation for subsequent selection of the optimal processing tool and processing step. Based on the determination of multiple candidate actions by the multimodal processing tool and its multiple processing steps, and with these candidate actions covering the multiple processing steps of the multimodal processing tool, the target action is determined from the multiple candidate actions according to the target intelligent agent's action determination strategy, thus achieving the selection of the optimal processing tool and processing step considering the target state to form the target action.

[0072] As an example of S201-S203, the intelligent agent is called Agent, and the target input data includes target text data and target image data. The computer device, based on the target data types of the target text data and target image data (text type and image type), determines the corresponding text type processing tool and the corresponding image type processing tool from the multimodal processing tools of the target Agent. The computer device determines the target state of the target Agent as target s by using the target text data and target image data, the processing tools corresponding to the text type and the image type, and the output data of the previous execution step of the target execution step to which the target text data and target image data belong, i.e., historical processing data. Based on the determination of multiple candidate actions as multiple candidate a by the multimodal processing tools and multiple processing steps of the multimodal processing tools, the computer device determines the target action as target a from the multiple candidate a based on target s and the action determination strategy of the target Agent.

[0073] S204: By using the processing tools in the target action, the processing steps in the target action are executed on the target input data to obtain the target output data.

[0074] In this embodiment, after executing the above-described S201-S203 steps to determine the target input data, the processing tool corresponding to the target data type in the multimodal processing tool, and the target state formed by the historical processing data of the target intelligent agent, and determining the target action from multiple candidate actions determined by the multimodal processing tool and multiple processing steps of the multimodal processing tool based on the target state, since the processing tool and processing steps in the target action are the optimal processing tool and processing steps selected by the target intelligent agent for the target input data, the intelligent agent can obtain accurate target output data by executing the target action for the target input data.

[0075] Based on this, the processing tools in the target action execute the processing steps in the target action on the target input data to obtain the target output data.

[0076] Among them, the target output data is the output data obtained by the processing tool in the target action after the target input data is input into the target action and processed through the processing steps in the target action.

[0077] The above-mentioned S204 uses the processing tool in the target action to execute the processing steps in the target action on the target input data, so as to realize the optimal processing tool to execute the optimal processing steps on the target input data, improve the accuracy of the output data, and thus obtain accurate target output data, thereby improving the data processing effect of the target intelligent agent.

[0078] As an example of S204, based on the examples of S201-S203 above, the computer device performs the processing steps in target a on the target text data and target image data through the processing tool in target a, and obtains the target output data.

[0079] As can be seen from the above technical solution, the intelligent agent automatically selects processing tools that match the data type of the input data. Based on the state formed by the input data, the selected processing tool, and the output data of the previous step, the intelligent agent flexibly and adaptively selects the optimal processing tool and processing steps from the multimodal processing tools and their multiple processing steps, using an action determination strategy. The intelligent agent then executes the optimal processing steps on the input data using the optimal processing tool, improving the accuracy of the output data and thus enhancing the data processing effect of the intelligent agent.

[0080] In this embodiment of the application, the action determination strategy in S203 above is a strategy for selecting actions based on a state. It can be expressed as selecting actions based on the selection probability of actions in a state. That is, the action determination strategy is a strategy for determining the selection probability of actions in a state. For the target state, the selection probability of each candidate action in the target state is considered. The larger the selection probability, the more likely the candidate action is to be the optimal action, i.e., the target action. Thus, the target action can be selected from multiple candidate actions.

[0081] Based on this, when executing the above-described S203, which determines the specific implementation of the target action from multiple candidate actions based on the target state and the action determination strategy of the target agent, it can be as follows: when the action determination strategy is a strategy for determining the selection probability of actions in a state, the candidate action corresponding to the maximum selection probability is determined from multiple candidate actions based on the selection probability of each candidate action in the target state. Therefore, this application provides a possible implementation method where the action determination strategy is the selection probability of actions in a state, and the above-described S203 may include, for example, S203a (not shown in the figure): determining the target action from multiple candidate actions based on the selection probability of each candidate action in the target state; the selection probability of the target action is the maximum in the target state.

[0082] Here, the probability of choosing an action in a given state is the probability that the agent will choose an action in that state, representing the likelihood of the agent choosing an action in that state; the probability of choosing each candidate action in the target state is the probability that the target agent will choose each candidate action in the target state, representing the likelihood of the target agent choosing each candidate action in the target state; the maximum probability of choosing the target action in the target state means that the target agent has the maximum probability of choosing the target action in the target state, that is, the maximum likelihood of the target agent choosing the target action in the target state.

[0083] S203a addresses the situation where the action determination strategy is a strategy for determining the selection probability of actions in a state. That is, based on the state, the optimal action is selected from multiple actions according to the selection probability of actions in that state. On the basis that the maximum selection probability of an action in a state indicates that the action is optimal in that state, the selection probability of each candidate action in the target state can be used to determine that the candidate action corresponding to the maximum selection probability is the optimal action in the target state. The candidate action corresponding to the maximum selection probability is determined as the target action, thereby achieving the selection of the optimal action in the target state. This is achieved by considering the probability of the processing tool and processing step in selecting each candidate action in the target state, accurately selecting the processing tool and processing step corresponding to the maximum probability, and forming the target action as the optimal processing tool and processing step.

[0084] As an example of S203a, based on the examples of S201-S203 above, when the action determination strategy is the strategy for determining the selection probability of an action in a state, the computer device determines the candidate a with the highest selection probability from multiple candidates a based on the selection probability of each candidate a under the target s.

[0085] In this embodiment of the application, the action determination strategy in S203 above is a strategy for selecting actions based on the state. It can be expressed as selecting actions based on the state-action pair value for the state. That is, the action determination strategy is a strategy for determining the state-action pair value. For the target state, the state-action pair value of the target state and each candidate action is considered. The larger the state-action pair value, the more likely the candidate action is to be the optimal action, i.e., the target action. Thus, the target action can be selected from multiple candidate actions.

[0086] Based on this, when executing the above-described S203, which determines the specific implementation of the target action from multiple candidate actions based on the target state and the action determination strategy of the target agent, it can be as follows: when the action determination strategy is a strategy for determining the value of state-action pairs, based on the value of the state-action pair between the target state and each candidate action, the candidate action corresponding to the largest state-action pair value is determined from multiple candidate actions as the target action. Therefore, this application provides a possible implementation method where the action determination strategy is a strategy for determining the value of state-action pairs. The above-described S203 may include, for example, S203a (not shown in the figure): determining the target action from multiple candidate actions based on the value of the state-action pair between the target state and each candidate action; where the value of the state-action pair between the target state and the target action is the largest.

[0087] Among them, the state-action pair value is the expected reward of performing an action in a state; the state-action pair value of the target state and each candidate action is the expected reward of performing a candidate action in the target state; the maximum state-action pair value of the target state and the target action indicates that the expected reward of performing the target action in the target state is maximized.

[0088] S203b addresses the case where the action determination strategy is a state-action pair value determination strategy, i.e., selecting the optimal action from multiple actions based on the state and the state-action pair value. It establishes that the action is optimal in the state when the state-action pair value is maximized. By comparing the target state with the state-action pair value of each candidate action, it can determine that the candidate action corresponding to the maximum state-action pair value is the optimal action in the target state. This allows for the selection of the optimal action in the target state. Furthermore, it considers the expected return of the processing tools and steps in executing each candidate action in the target state, making it easier to accurately select the processing tools and steps corresponding to the maximum expected return, thus forming the target action as the optimal processing tools and steps.

[0089] As an example of S203b, based on the examples of S201-S203 above, the value of the state-action pair is the Q value; when the action determination strategy is a Q value determination strategy, the computer device determines the candidate a with the largest Q value from multiple candidates a based on the target s and the Q value of each candidate a.

[0090] In this embodiment, considering that the data type of the target intelligent agent's input data includes one or more of text, image, or speech types, the multimodal processing tool for the target intelligent agent needs to include a text processing tool, an image processing tool, and a speech processing tool. When the target data type of the target input data is a unimodal data type, the corresponding processing tool for the unimodal data type is determined directly from the text processing tool, image processing tool, and speech processing tool of the target intelligent agent based on the unimodal data type. Specifically, when the unimodal data type is text, the corresponding text processing tool is determined from the multimodal processing tools; when the unimodal data type is image, the corresponding image processing tool is determined from the multimodal processing tools; and when the unimodal data type is speech, the corresponding speech processing tool is determined from the multimodal processing tools.

[0091] When the target input data is a cross-modal data type, it is necessary to determine the corresponding processing tool from the text processing tools, image processing tools, and speech processing tools of the target agent based on the cross-modal data type. Specifically, when the cross-modal data type includes text and image types, the corresponding text processing tool and the corresponding image processing tool are determined from the multimodal processing tools; when the cross-modal data type includes text and speech types, the corresponding text processing tool and the corresponding speech processing tool are determined from the multimodal processing tools; when the cross-modal data type includes image and speech types, the corresponding image processing tool and the corresponding speech processing tool are determined from the multimodal processing tools; and when the cross-modal data type includes text, image, and speech types, the corresponding text processing tool, the corresponding image processing tool, and the corresponding speech processing tool are determined from the multimodal processing tools. Therefore, this application provides a possible implementation method in which the multimodal processing tool includes a text processing tool, an image processing tool, and a speech processing tool; if the target data type is a cross-modal data type, the above S201 may include, for example, the following S201a, S201b, S201c or S201d (not shown in the figure).

[0092] S201a: If the cross-modal data type includes text type and image type, determine the text processing tool and image processing tool from the multimodal processing tools.

[0093] S201b: If the cross-modal data type includes text and speech types, determine the text processing tool and speech processing tool from the multimodal processing tools.

[0094] S201c: If the cross-modal data type includes image type and speech type, determine the image processing tool and speech processing tool from the multimodal processing tools.

[0095] S201d: If the cross-modal data type includes text, image, and speech types, determine the text processing tool, image processing tool, and speech processing tool from the multimodal processing tools.

[0096] Among them, text processing tools are tools that can process text-type input data; image processing tools are tools that can process image-type input data; speech processing tools are tools that can process speech-type input data; cross-modal data types include different data types; text type is a data type that exists and is expressed in text form; image type is a data type that exists and is expressed in image form; speech type is a data type that exists and is expressed in speech form.

[0097] Furthermore, to ensure seamless collaboration among text processing, image processing, and speech processing tools in the multimodal processing framework, a unified interface design is adopted for these tools. This allows the target agent to dynamically select and invoke different processing tools. Specifically, each processing tool is independent, with clearly defined functions and input / output interfaces. This enables the target agent to flexibly select the appropriate processing tool based on the target data type of the input data. Different processing tools exchange data through standardized interfaces. The inputs and outputs of different processing tools are exchanged using a unified format to ensure compatibility, such as the following JSON format:

[0098] Inputmodule={"data": input_data, "tool_type": ool_identifier}

[0099] Outputmodule={"result": output_data, "status": status_code}.

[0100] Among them, tool_identifier represents the data type corresponding to the processing tool, input_data represents the input data of the processing tool, output_data represents the output data of the processing tool, and status_code represents the execution status or output data identifier of the processing tool, which is used to determine whether the output data of the processing tool is erroneous.

[0101] Among them, the text processing tool is M T The input data for the text processing tool is Text, and the output data is R. TThe formula for calculating R in text processing tools is R. T =M T (Text), the M T (Text) indicates that the input data of the text processing tool is processed to obtain the output data; the image processing tool is M. I The input data for the image processing tool is an Image, and the output data is an Image. I The calculation formula for image processing tools is R. I =M I (Image), the M I (Image) represents the input data of an image processing tool being processed to obtain the output data; the speech processing tool is M. S The input data for the speech processing tool is Speech, and the output data is R. S The calculation formula for the speech processing tool is R. S =M S (Speech), the M S (Speech) means processing the input data of a speech processing tool to obtain the output data.

[0102] S201a addresses the multimodal processing tools for the target agent, including text processing, image processing, and speech processing tools. It integrates these tools within the target agent. When the target input data is a cross-modal data type including both text and image types, it selects the appropriate text processing tool and speech processing tool from the target agent's available tools. This automatic selection of the appropriate processing tool for the target input data enables the target agent to process cross-modal data based on both text and image data. This facilitates accurate data processing by integrating text and image data, thereby improving the overall data processing results for cross-modal data based on both text and image data.

[0103] S201b addresses the multimodal processing tools for the target agent, including text processing, image processing, and speech processing tools. It integrates these tools within the target agent. When the target input data is a cross-modal data type including both text and speech, it selects the appropriate text processing tool and speech processing tool from the target agent's available tools. This automatic selection of the appropriate processing tool for the target input data enables the target agent to process cross-modal data based on both text and speech. This facilitates accurate data processing by integrating text and speech data, thereby improving the overall data processing results for cross-modal data based on both text and speech.

[0104] The S201c addresses the multimodal processing tools for the target agent, including text processing, image processing, and speech processing tools. It integrates these tools within the target agent. When the target input data is a cross-modal data type encompassing both image and speech types, it selects the appropriate image processing tool and speech processing tool from the target agent's text, image, and speech processing tools. This automatic selection of the appropriate image and speech processing tools for the target input data enables the target agent to process cross-modal data based on both image and speech data. This facilitates accurate data processing by the target agent in integrating image and speech data, thereby improving the overall data processing results for cross-modal data based on both image and speech data.

[0105] The S201d provides a multimodal processing tool for the target agent, including text processing, image processing, and speech processing tools. It integrates these tools within the target agent. When the target input data is a cross-modal data type including text, image, and speech, the system automatically selects the appropriate processing tool from the target agent's available tools. This allows the target agent to process cross-modal data based on text, image, and speech, facilitating accurate data processing and improving the overall data processing results.

[0106] Additionally, see Figure 3 ,Should Figure 3 This application provides a step diagram illustrating the process of determining the processing tool corresponding to a target data type when a multimodal processing tool includes a text processing tool, an image processing tool, and a speech processing tool. The steps include: acquiring target input data of a target agent; determining the target data type of the target input data; if the target data type is text, determining the text processing tool corresponding to the text type from the target agent's text processing tool, image processing tool, and speech processing tool; if the target data type is an image, determining the image processing tool corresponding to the image type from the target agent's text processing tool, image processing tool, and speech processing tool; and if the target data type is speech, determining the speech processing tool corresponding to the speech type from the target agent's text processing tool, image processing tool, and speech processing tool.

[0107] Furthermore, in this embodiment, considering that processing tools can be added to the multimodal processing tool to update the multimodal processing tool, the above-mentioned S201 determines the processing tool corresponding to the target data type from the updated multimodal processing tool of the target intelligent agent based on the target data type of the target input data of the target intelligent agent, and the multiple candidate actions in the above-mentioned S203 are determined by the updated modal processing tool and the multiple processing steps of the updated multimodal processing tool. Therefore, this application provides a possible implementation, and the method may further include, for example, S1 (not shown in the figure): adding tools to the multimodal processing tool to obtain an updated multimodal processing tool; correspondingly, the above-mentioned S201, which determines the processing tool corresponding to the target data type from the multimodal processing tool of the target intelligent agent, may include, for example, S201e (not shown in the figure): determining the processing tool corresponding to the target data type from the updated multimodal processing tool; the step of determining multiple candidate actions in the above-mentioned S203 may include, for example, S2 (not shown in the figure): determining multiple candidate actions based on the updated multimodal processing tool and the multiple processing steps of the updated multimodal processing tool.

[0108] Among them, the updated multimodal processing tool has a larger number of processing tools than the multimodal processing tool has a larger number of processing tools.

[0109] Among them, the multimodal processing tool is M multi The new processing tool is M new The updated multimodal processing tool M multi =Integrate(M new ), the Integrate (M new This indicates that adding the new processing tool to the multimodal processing tool results in an updated multimodal processing tool.

[0110] S1, S201e, and S2 further enhance the multimodal processing tool by adding processing tools to update it. This improves the multimodal processing tool's scalability and maintainability, and enriches its data processing methods. Based on the target data type of the target agent's input data, the updated multimodal processing tool for the target agent determines the corresponding processing tool for that data type, enabling more flexible and automatic selection of the appropriate processing tool. Furthermore, based on the updated multimodal processing tool and its multiple processing steps, multiple candidate actions are determined, ensuring that these actions cover all aspects of the updated multimodal processing tool and its processing steps, resulting in a more comprehensive range of candidate actions.

[0111] Furthermore, in the embodiments of this application, the output data of the processing tool in the target action may be erroneous data. The target agent can adjust the target action itself based on the erroneous data output by the processing tool in the target action, so that the target agent can execute the adjusted target action for the target input data and obtain the target output data.

[0112] Based on this, when the output data of the processing tool in the target action is erroneous, the target action is further adjusted to obtain the adjusted target action. Correspondingly, when executing the above-mentioned S204, which uses the processing tool in the target action to perform the processing steps in the target action on the target input data to obtain the target output data, it can be implemented as follows: using the processing tool in the adjusted target action to perform the processing steps in the adjusted target action on the target input data to obtain the target output data. Therefore, this application provides a possible implementation method, which may further include S3 (not shown in the figure): if the output data of the processing tool in the target action is erroneous, the target action is adjusted to obtain the adjusted target action; correspondingly, the above-mentioned S204 may include S204a (not shown in the figure): using the processing tool in the adjusted target action to perform the processing steps in the adjusted target action on the target input data to obtain the target output data.

[0113] Among them, the adjusted target action is the action obtained by adjusting the target action through error data; the target output data is the output data obtained by the processing tool in the adjusted target action after the target input data is input and processed through the processing steps in the adjusted target action.

[0114] Where the target input data is X, the processing tool in the target action is M, and the output data of the processing tool in the target action is R. M Then R M=M(X), where M(X) represents the output data obtained by performing the processing steps of the target action on the target input data through the processing tool in the target action. If the output data is erroneous data F fail Further adjustments to the target action yield the adjusted target action, i.e., Adjust(F) fail This indicates that the target action is adjusted based on the erroneous data to obtain the adjusted target action.

[0115] S3 and S204a address the situation where the output data of the processing tool in the target action is erroneous. This indicates that the processing tool or processing step in the target action is unsuitable and needs adjustment. Further adjustment of the target action yields an adjusted target action, making it an optimized action. By using the processing tool in the adjusted target action to execute the processing steps in the adjusted target action on the target input data, the accuracy of the output data is further improved, resulting in more accurate target output data and thus further enhancing the data processing effect of the intelligent agent.

[0116] As an example of S3 and S204a, based on the example of S204 above, when the output data of the processing tool in target a is erroneous data, the computer device further adjusts target a to obtain an adjusted target a; the computer device uses the processing tool in the adjusted target a to perform the processing steps in the adjusted target a on the target text data and the target image data to obtain the target output data.

[0117] See Figure 4 ,Should Figure 4 This application provides a step diagram illustrating the process of obtaining target output data when the output data of a processing tool in a target action is erroneous. The process involves determining whether the output data of the processing tool in the target action is erroneous; if so, adjusting the target action to obtain an adjusted target action; and then using the processing tool in the adjusted target action to perform the processing steps of the adjusted target action on the target input data to obtain the target output data.

[0118] In this embodiment, error data may indicate an error in the processing tool of the target action. In this case, the processing tool in the target action needs to be adjusted so that the adjusted target action includes the adjusted processing tool. Error data may also indicate an error in the processing steps of the target action. In this case, the processing steps in the target action need to be adjusted so that the adjusted target action includes the adjusted processing steps. Therefore, this application provides a possible implementation where error data indicates an error in the processing tool of the target action, and the adjusted target action includes the adjusted processing tool; or error data indicates an error in the processing steps of the target action, and the adjusted target action includes the adjusted processing steps.

[0119] The adjusted processing tool is obtained by adjusting the processing tool in the target action based on the error data; the adjusted processing steps are obtained by adjusting the processing steps in the target action based on the error data.

[0120] This method addresses situations where the output data of the processing tool in the target action indicates an error in the processing tool itself. If the processing tool in the target action is unsuitable, it needs to be adjusted. This adjustment further refines the processing tool to obtain an adjusted target action, ensuring that the adjusted target action includes the adjusted processing tool. This achieves optimization of the processing tool when it is unsuitable. Similarly, it addresses situations where the output data of the processing tool in the target action indicates an error in the processing steps within the target action. If the processing steps in the target action are unsuitable, they need to be adjusted. This adjustment further refines the processing steps to obtain an adjusted target step, ensuring that the adjusted target action includes the adjusted processing step. This achieves optimization of the processing step when it is unsuitable.

[0121] Furthermore, in this embodiment, the target agent is obtained by training an initial agent using reinforcement learning based on sample input data. That is, given that the initial agent possesses multimodal processing tools, it can use sample input data, the processing tool selected based on the sample data type in the multimodal processing tools, and the output data of the previous execution step (i.e., historical processing data) of the sample execution step to which the sample input data belongs as sample states. From the multiple candidate actions determined by the multimodal processing tools and their multiple processing steps, based on the initial agent's action determination strategy, it selects processing tools and processing steps as sample actions. Executing these sample actions on the sample input data yields a sample reward, thereby optimizing the aforementioned action determination strategy. This optimized strategy allows for flexible and adaptive selection of the optimal processing tools and processing steps as actions based on the state, thus completing the training and obtaining the target agent.

[0122] Based on this, the specific training process refers to the following: First, based on the sample data type of the initial agent's sample input data, the corresponding processing tool for the sample data type is determined from the multimodal processing tools of the initial agent; second, the sample state of the initial agent is determined through the sample input data, the processing tool corresponding to the sample data type, and the output data of the previous execution step of the sample execution step to which the sample input data belongs, i.e., historical processing data; next, based on the determination of multiple candidate actions by the multimodal processing tool and its multiple processing steps, the sample action is determined from the multiple candidate actions according to the initial agent's action determination strategy based on the sample state; then, the processing tool in the sample action is used to execute the processing steps in the sample action on the sample input data to obtain the sample reward of the sample action in the sample state; finally, the action determination strategy is optimized based on the sample reward to obtain the target agent corresponding to the initial agent. Therefore, this application provides a possible implementation method, which also includes the following S4-S8 (not shown in the figure).

[0123] S4: Based on the sample data type of the initial agent's sample input data, determine the processing tool corresponding to the sample data type from the initial agent's multimodal processing tools.

[0124] S5: The sample state is determined by the sample input data, the processing tool corresponding to the sample data type, and the historical processing data of the sample execution step to which the sample input data belongs; the historical processing data is the output data of the previous execution step of the sample execution step.

[0125] S6: Determine the strategy based on the sample state and the initial agent's actions, and determine the sample action from multiple candidate actions; the multiple candidate actions are determined based on the multimodal processing tool and multiple processing steps of the multimodal processing tool.

[0126] S7: Using the processing tools in the sample action, execute the processing steps in the sample action on the sample input data to obtain the sample reward of the sample action in the sample state.

[0127] S8: Optimize the action determination strategy based on the sample reward to obtain the target agent corresponding to the initial agent.

[0128] In this context, the initial agent is an agent that has not undergone reinforcement learning training; the sample input data is the input data of the initial agent; the sample data type is the data type of the sample input data; the multimodal processing tool includes processing tools for different modalities, i.e., processing tools corresponding to different data types; the processing tool corresponding to the sample data type is the processing tool in the multimodal processing tool that can process the input data of the sample data type; multiple candidate actions are multiple actions formed through the multimodal processing tool and multiple processing steps of the multimodal processing tool; the action determination strategy is the strategy for selecting actions based on the state; the sample action is the candidate action selected from multiple candidate actions according to the action determination strategy in the sample state; and the sample reward for the sample action in the sample state is the feedback obtained by performing the sample action in the sample state.

[0129] Specifically, when the sample execution step to which the sample input data belongs is the first execution step, the historical processing data of the first execution step is empty; when the sample execution step to which the sample input data belongs is the second execution step, the historical processing data of the second execution step is the output data of the first execution step; when the sample execution step to which the sample input data belongs is the third execution step, the historical processing data of the third execution step is the output data of the second execution step, and so on, without further explanation.

[0130] The above steps S4-S8, based on the sample data type of the initial agent's sample input data, determine the corresponding processing tool from the initial agent's multimodal processing tools, thereby automatically selecting the processing tool that matches the sample data type. By using the sample input data, the processing tool corresponding to the sample data type, and the output data of the previous execution step (i.e., historical processing data) of the sample execution step to which the sample input data belongs, the sample state of the initial agent is determined. This clarifies the sample state in which the initial agent selects the processing tool and processing step, laying the foundation for the subsequent selection of processing tools and processing steps by the initial agent.

[0131] Based on the determination of multiple candidate actions by the multimodal processing tool and its multiple processing steps, and with these candidate actions covering multiple processing steps, a sample action is determined from the multiple candidate actions according to the initial agent's action determination strategy based on the sample state. This realizes the selection of processing tools and processing steps considering the sample state to form the sample action. Through the processing tools in the sample action, the processing steps in the sample action are executed on the sample input data to obtain the sample reward of the sample action in the sample state, thus clarifying the feedback obtained by the initial agent when executing the sample action in the sample state. The action determination strategy is optimized through the sample reward, so that the optimized action determination strategy can flexibly and adaptively select the optimal processing tools and processing steps as actions according to the state. After training, the target agent is obtained, which enables the target agent to flexibly and adaptively select the optimal processing tools and processing steps according to the input data, thereby improving the data processing effect of the target agent.

[0132] In this embodiment, the action determination strategy in S6 above is a strategy for selecting actions based on the state. It can be expressed as selecting actions based on the selection probability of actions in the state. That is, the action determination strategy is a strategy for determining the selection probability of actions in the state. For the sample state, the selection probability of each candidate action in the sample state is considered. The larger the selection probability, the more likely the candidate action is to be the optimal action, i.e., the sample action. Thus, the sample action can be selected from multiple candidate actions. Correspondingly, the strategy optimization of the action determination strategy in S8 above is actually optimizing the selection probability of sample actions in the sample state.

[0133] Based on this, when executing S6 to determine the specific implementation of the sample action from multiple candidate actions according to the sample state and the action determination strategy of the initial agent, it can be: when the action determination strategy is a strategy for determining the selection probability of actions in the state, the candidate action corresponding to the maximum selection probability is determined from multiple candidate actions based on the selection probability of each candidate action in the sample state. Correspondingly, when executing S8 to optimize the action determination strategy according to the sample reward to obtain the specific implementation of the target agent corresponding to the initial agent, it can be: optimizing the selection probability of the sample action in the sample state based on the sample reward until convergence, to obtain the target agent corresponding to the initial agent. Therefore, this application provides a possible implementation method where the action determination strategy is a strategy for determining the selection probability of actions in the state. S6 may include, for example, S6a (not shown in the figure): determining the sample action from multiple candidate actions according to the selection probability of each candidate action in the sample state; the selection probability of the sample action in the sample state is the maximum; S8 may include, for example, S8a (not shown in the figure): optimizing and updating the selection probability of the sample action in the sample state according to the sample reward to obtain the target agent.

[0134] Here, the probability of choosing an action in a given state is the probability that the agent will choose an action in that state, representing the likelihood of the agent choosing an action in that state; the probability of choosing each candidate action in a sample state is the probability that the initial agent will choose each candidate action in the sample state, representing the likelihood of the initial agent choosing each candidate action in the sample state; the maximum probability of choosing a sample action in a sample state means that the initial agent has the maximum probability of choosing a sample action in the sample state, that is, the initial agent has the maximum likelihood of choosing a sample action in the sample state.

[0135] S6a addresses the case where the action determination strategy is based on the probability of action selection under a state. That is, it selects the optimal action from multiple actions according to the probability of action selection under a state. Based on the premise that the maximum probability of action selection under a state indicates that the action is optimal under that state, the candidate action corresponding to the maximum selection probability can be determined as the optimal action under the sample state by using the selection probability of each candidate action under the sample state. The candidate action corresponding to the maximum selection probability is determined as the sample action, thereby realizing the selection of the optimal action under the sample state. This achieves the consideration of the probability of selecting the processing tool and processing step in each candidate action under the sample state, and accurately selects the processing tool and processing step corresponding to the maximum probability as the optimal processing tool and processing step to form the sample action.

[0136] S8a addresses the situation where the action determination strategy is based on the probability of action selection in a state. That is, it selects the optimal action from multiple actions according to the probability of action selection in the state. It optimizes the probability of action selection in the sample state based on the sample reward until convergence. This allows the optimized probability of action selection in the state to flexibly and adaptively select the optimal processing tools and processing steps as actions for the state, thus completing the training to obtain the target agent. This enables the target agent to flexibly and adaptively select the optimal processing tools and processing steps for the input data, thereby improving the data processing effect of the target agent.

[0137] In this embodiment of the application, the action determination strategy in S203 above is a strategy for selecting actions based on the state. It can be expressed as selecting actions based on the state-action pair value for the state. That is, the action determination strategy is a strategy for determining the state-action pair value. For the sample state, the state-action pair value of the sample state and each candidate action is considered. The larger the state-action pair value, the more likely the candidate action is to be the optimal action, i.e., the sample action. Thus, the sample action can be selected from multiple candidate actions. Correspondingly, the strategy optimization of the action determination strategy in S8 above is actually optimizing the state-action pair value of the sample state and the sample action.

[0138] Based on this, when executing S203 to determine the specific implementation of a sample action from multiple candidate actions based on the sample state and the action determination strategy of the initial agent, it can be as follows: when the action determination strategy is a strategy for determining the value of state-action pairs, based on the sample state and the value of the state-action pairs of each candidate action, the candidate action corresponding to the maximum value of the state-action pairs is determined from multiple candidate actions as the sample action. Correspondingly, when executing S8 to optimize the action determination strategy based on the sample reward to obtain the specific implementation of the target agent corresponding to the initial agent, it can be as follows: based on the sample reward and the next state of the sample state, the value of the state-action pairs of the sample state and the sample action is optimized until convergence, thus obtaining the target agent corresponding to the initial agent. Therefore, this application provides a possible implementation method where the action determination strategy is a strategy for determining the value of state-action pairs. For example, S6 can include S6b (not shown in the figure): determining the sample action from multiple candidate actions based on the sample state and the value of the state-action pairs of each candidate action; S8 can include S8b (not shown in the figure): optimizing and updating the value of the state-action pairs of the sample state and the sample action based on the sample reward and the next state of the sample state, thus obtaining the target agent.

[0139] Among them, the state-action pair value is the expected reward of performing an action in a state; the state-action pair value of a sample state and each candidate action is the expected reward of performing a candidate action in the sample state; the maximum state-action pair value of a sample state and a sample action indicates that the expected reward of performing the sample action in the sample state is the maximum; the next state of a sample state is the new state determined after performing the sample action in the sample state and obtaining the sample reward.

[0140] S6b addresses the case where the action determination strategy is based on the value of state-action pairs. Specifically, it involves selecting the optimal action from multiple actions based on the state's state-action pair value. By considering the maximum state-action pair value (representing the optimal action in that state) and the state-action pair value of each candidate action, it can determine that the candidate action corresponding to the maximum state-action pair value is the optimal action in the sample state. This allows for the selection of the optimal action within the sample state. Furthermore, it considers the expected return of the processing tools and steps in executing each candidate action under the sample state, accurately selecting the processing tools and steps corresponding to the maximum expected return as the optimal processing tools and steps to form the sample action.

[0141] S8b addresses the scenario where the action determination strategy is based on the value of state-action pairs. Specifically, it selects the optimal action from multiple actions based on the state and the value of the state-action pair. Based on the sample reward and the next state of the sample state, it optimizes the state-action pair value of the sample state and sample action until convergence. This allows the optimized state-action pair value to flexibly and adaptively select the optimal processing tools and steps as actions for the state, completing the training to obtain the target agent. This enables the target agent to flexibly and adaptively select the optimal processing tools and steps for the input data, thereby improving the data processing performance of the target agent.

[0142] As an example, the optimized update formula for value based on state action is shown below:

[0143] Q(s t ,a t )←Q(s t ,a t ) +α[R(s t ,a t )+γmaxa t+1 Q(s t+1 ,a t+1 )−Q(s t ,a t )];

[0144] Where Q(s) t ,a t R(s) represents the value of a state-action pair, where α is the learning rate, controlling the step size for updating the state-action pair value; t ,a t ) represents the sample reward for the sample action in the sample state; γ is the discount factor, representing the degree of influence of future rewards; maxa t+1 Q(s t+1 ,a t+1 ) is the value of the next state of the sample state and its chosen optimal action.

[0145] See Figure 5 ,Should Figure 5This application provides a step diagram for obtaining a target agent by updating the state-action pair value based on sample reward in an embodiment of the present application. The steps include: determining the processing tool corresponding to the sample data type from the multimodal processing tools of the initial agent based on the sample input data of the initial agent; determining the sample state of the initial agent using the sample input data, the processing tool corresponding to the sample data type, and the output data of the previous execution step of the sample execution step to which the sample input data belongs (i.e., historical processing data); determining multiple candidate actions based on the multimodal processing tool and its multiple processing steps, and determining the candidate action corresponding to the maximum state-action pair value from the multiple candidate actions based on the sample state and the state-action pair value of each candidate action; executing the processing steps of the sample action on the sample input data using the processing tool in the sample action to obtain the sample reward of the sample action in the sample state; and optimizing the state-action pair value of the sample state and the sample action until convergence based on the sample reward and the next state of the sample state to obtain the target agent corresponding to the initial agent.

[0146] In this embodiment of the application, when the action determination strategy in S6b is the strategy for determining the value of state-action pairs, when determining a sample action from multiple candidate actions, considering the balance between selecting a sample action through exploration and selecting a sample action using the value of state-action pairs, a preset probability can be used to represent the probability of selecting a sample action through exploration. Combined with the sample state and the value of the state-action pairs of each candidate action, a sample action is selected from multiple candidate actions. Correspondingly, in the above-mentioned S8b, while optimizing and updating the value of the state-action pairs of the sample state and the sample action, it is also necessary to optimize and update the preset probability.

[0147] Based on this, when performing S6b to determine the specific implementation of the sample action from multiple candidate actions based on the state-action pair value of the sample state and each candidate action, it can be: based on the preset probability, the state-action pair value of the sample state and each candidate action, the candidate action corresponding to the maximum state-action pair value is determined as the sample action from multiple candidate actions. Correspondingly, when performing S8b to optimize and update the state-action pair value of the sample state and the sample action based on the sample reward and the next state of the sample state to obtain the specific implementation of the target agent, it can be: based on the sample reward and the next state of the sample state, the preset probability, the state-action pair value of the sample state and the sample action is optimized until convergence, to obtain the target agent corresponding to the initial agent. Therefore, this application provides a possible implementation method. S6b may include, for example, determining the sample action from multiple candidate actions based on the preset probability, the state-action pair value of the sample state and each candidate action; S8b may include, for example, optimizing and updating the preset probability, the state-action pair value of the sample state and the sample action based on the sample reward and the next state of the sample state to obtain the target agent.

[0148] The preset probability is a probability that has been set in advance.

[0149] This method addresses the scenario where the action determination strategy is based on the value of state-action pairs. Specifically, it selects the optimal action from multiple actions based on the state and the value of the state-action pair. Building upon the premise that the action with the maximum value of the state-action pair is optimal in that state, it further considers the balance between exploring different action selection methods and utilizing the value of state-action pairs. Based on preset probabilities, sample states, and the value of state-action pairs for each candidate action, it can determine with a high probability that the candidate action corresponding to the maximum value of the state-action pair is the optimal action in the sample state. This candidate action is then designated as the sample action. This approach achieves the goal of selecting the optimal action in the sample state by considering the value of the state-action pair with a high probability. Furthermore, it considers the expected return of the processing tools and steps in executing each candidate action in the sample state, selecting the processing tools and steps corresponding to the maximum expected return with a high probability, thus forming the sample action.

[0150] Based on the sample reward and the next state of the sample state, optimize the preset probability, the state-action pair value of the sample state and the sample action until convergence, so that the preset probability is reduced. This allows the optimized state-action pair value to flexibly and adaptively select the optimal processing tool and processing step as the action for the state under a high probability, and complete the training to obtain the target agent. This enables the target agent to flexibly and adaptively select the optimal processing tool and processing step for the input data, thereby improving the data processing effect of the target agent.

[0151] As an example, the preset probability is ε; in the sample state, the candidate action corresponding to the maximum state action pair value is determined from multiple candidate actions with a probability of 1-ε; as ε is optimized and updated, ε will gradually decrease.

[0152] In summary, the data processing method for intelligent agents provided in this application has the following advantages:

[0153] 1. Intelligent agents possess multimodal tools

[0154] Based on the data type of the input data, the intelligent agent determines the corresponding processing tool from the multimodal processing tools. With multimodal processing tools including text processing tools, image processing tools, and speech processing tools, the intelligent agent can not only process text data, but also single-modal data of different data types such as image data or speech data. This enables the intelligent agent to respond flexibly and process efficiently when faced with complex tasks, reducing task execution time and improving the success rate of calling processing tools.

[0155] 2. The agent is trained through reinforcement learning.

[0156] Through reinforcement learning training and autonomous learning via a reward mechanism, the intelligent agent can optimize its action-determining strategy and reasoning strategy, improving the accuracy of its processing tools and steps. This allows the agent to flexibly and adaptively select the optimal processing tool and steps from multiple processing steps within a multimodal processing tool, based on the state formed by the input data, the selected processing tool, and the output data from the previous step. By using the optimal processing tool among the actions, the agent executes the optimal processing steps on the input data to obtain accurate output data.

[0157] 3. Cross-modal data processing

[0158] Based on multimodal processing tools, including text processing tools, image processing tools, and speech processing tools, intelligent agents can simultaneously process cross-modal data of different data types, enabling them to quickly adapt and make reasonable data processing when faced with complex, multimodal tasks.

[0159] 4. Self-assessment and self-adjustment

[0160] The intelligent agent can detect in real time whether the output data of the processing tool is erroneous. When the erroneous data indicates that the processing tool is faulty, it adjusts the processing tool itself. When the erroneous data indicates that the processing steps are faulty, it adjusts the processing steps itself to avoid repeating the same mistakes. This makes the intelligent agent exhibit a certain degree of adjustment autonomy and adjustment intelligence.

[0161] It should be noted that, based on the implementation methods provided in the above aspects, this application can be further combined to provide more implementation methods.

[0162] The data processing method for intelligent agents provided in this application improves the accuracy of task execution. For example, in mathematical problem-solving tasks, the data processing method for intelligent agents provided in this application improves the execution accuracy to a certain extent compared with traditional methods.

[0163] The data processing method for intelligent agents provided in this application improves the efficiency of processing tool selection, reduces erroneous processing tool selection through reinforcement learning mechanisms, and increases the success rate of processing tool invocation.

[0164] The data processing method for intelligent agents provided in this application improves the ability to process cross-modal data. By processing cross-modal data through multimodal tools, it effectively integrates input data of different data types, thereby improving the comprehensiveness and accuracy of data processing.

[0165] The data processing method for intelligent agents provided in this application improves self-adjustment capabilities, avoids repeated errors through a self-adjustment mechanism, and enhances the stability and efficiency of task execution.

[0166] In summary, the data processing method for intelligent agents provided in this application has high practical value and broad application potential in multimodal tasks, especially complex multimodal tasks.

[0167] based on Figure 2 Corresponding to the data processing method for intelligent agents provided in the embodiments, this application also provides a data processing apparatus for intelligent agents, see [link to relevant documentation]. Figure 6 ,Should Figure 6 This is a structural diagram of a data processing device for an intelligent agent provided in an embodiment of the present application. The data processing device 600 for the intelligent agent includes: a determination unit 601 and an execution unit 602.

[0168] The determining unit 601 is used to determine the processing tool corresponding to the target data type from the multimodal processing tools of the target intelligent agent based on the target data type of the target input data of the target intelligent agent.

[0169] The determining unit 601 is also used to determine the target input data, the processing tool corresponding to the target data type, and the historical processing data of the target execution step to which the target input data belongs as the target state; the historical processing data is the output data of the previous execution step of the target execution step;

[0170] The determining unit 601 is also used to determine the target action from multiple candidate actions based on the target state and the action determination strategy of the target agent; the multiple candidate actions are determined based on the multimodal processing tool and multiple processing steps of the multimodal processing tool;

[0171] The execution unit 602 is used to perform the processing steps of the target action on the target input data through the processing tool in the target action to obtain the target output data.

[0172] In one possible implementation, the action determination strategy is a strategy for determining the probability of choosing an action in a given state. The determination unit 601 is specifically used for:

[0173] The target action is determined from multiple candidate actions based on the selection probability of each candidate action under the target state; the target action has the highest selection probability under the target state.

[0174] In one possible implementation, the action determination strategy is a strategy for determining the value of state-action pairs. Determination unit 601 is specifically used for:

[0175] The target action is determined from multiple candidate actions based on the value of the state-action pair between the target state and each candidate action; the state-action pair between the target state and the target action has the highest value.

[0176] In one possible implementation, the multimodal processing tools include text processing tools, image processing tools, and speech processing tools; if the target data type is a cross-modal data type, the determining unit 601 is specifically used for:

[0177] If the cross-modal data type includes both text and image types, determine the text processing tool and the image processing tool from the multimodal processing tools;

[0178] If the cross-modal data type includes both text and speech types, determine the text processing tool and speech processing tool from the multimodal processing tools;

[0179] If the cross-modal data type includes both image and speech types, determine the image processing tool and speech processing tool from the multimodal processing tools;

[0180] If the cross-modal data type includes text, image, and speech types, determine the text processing tool, image processing tool, and speech processing tool from the multimodal processing tools.

[0181] In one possible implementation, device 600 further includes: an addition unit;

[0182] An additional unit is added to the multimodal processing tool to obtain an updated multimodal processing tool.

[0183] Unit 601 is specifically used for:

[0184] Determine the processing tool corresponding to the target data type from the updated multimodal processing tools;

[0185] The determining unit 601 is also used for:

[0186] Based on the updated multimodal processing tool and its multiple processing steps, multiple candidate actions are identified.

[0187] In one possible implementation, the device 600 further includes: an adjustment unit;

[0188] The adjustment unit is used to adjust the target action if the output data of the processing tool in the target action is erroneous, so as to obtain the adjusted target action.

[0189] Execution unit 602 is specifically used for:

[0190] By using the processing tools in the adjusted target action, the processing steps in the adjusted target action are executed on the target input data to obtain the target output data.

[0191] In one possible implementation, error data indicates an error in the processing tool of the target action, and the adjusted target action includes the adjusted processing tool; error data also indicates an error in the processing step of the target action, and the adjusted target action includes the adjusted processing step.

[0192] In one possible implementation, device 600 further includes: a training unit;

[0193] Training units are used for:

[0194] Based on the sample data type of the initial agent's sample input data, determine the processing tool corresponding to the sample data type from the initial agent's multimodal processing tools;

[0195] The sample state is defined as the sample input data, the processing tool corresponding to the sample data type, and the historical processing data of the sample execution step to which the sample input data belongs; the historical processing data is the output data of the previous execution step of the sample execution step.

[0196] The strategy is determined based on the sample state and the initial agent's actions, and the sample action is determined from multiple candidate actions; the multiple candidate actions are determined based on the multimodal processing tool and multiple processing steps of the multimodal processing tool;

[0197] By using the processing tools in the sample action, the processing steps in the sample action are executed on the sample input data to obtain the sample reward of the sample action in the sample state.

[0198] Based on the sample rewards, the action determination strategy is optimized to obtain the target agent corresponding to the initial agent.

[0199] In one possible implementation, the action determination policy is a policy for determining the probability of choosing an action in a given state. The training unit is specifically used for:

[0200] Based on the selection probability of each candidate action in the sample state, the sample action is determined from multiple candidate actions; the sample action has the highest selection probability in the sample state.

[0201] The training unit is specifically used for:

[0202] The selection probability of sample actions in the sample state is optimized and updated based on the sample reward to obtain the target agent.

[0203] In one possible implementation, the action determination policy is a policy for determining the value of state-action pairs, and the training unit is specifically used for:

[0204] Based on the state-action pair value of the sample state and each candidate action, determine the sample action from multiple candidate actions;

[0205] The training unit is specifically used for:

[0206] Based on the sample reward and the next state of the sample state, the value of the state-action pair of the sample state and sample action is optimized and updated to obtain the target intelligent agent.

[0207] In one possible implementation, the training unit is specifically used for:

[0208] Based on the preset probability, sample state and the state-action pair value of each candidate action, the sample action is determined from multiple candidate actions;

[0209] The training unit is specifically used for:

[0210] Based on the sample reward and the next state of the sample state, the preset probability, sample state and the state-action pair value of the sample action are optimized and updated to obtain the target intelligent agent.

[0211] As can be seen from the above technical solution, the data processing device of the intelligent agent includes a determination unit and an execution unit. The process involves several key components: First, a determination unit identifies the corresponding processing tool from the multimodal processing tools of the target agent based on the target data type of the target input data. This automatic selection of the appropriate processing tool matches the target input data. Second, the determination unit uses the target input data, the corresponding processing tool, and the output data (historical processing data) of the previous execution step of the target execution step to determine the target state of the target agent. This clarifies the target state at which the target agent selects the optimal processing tool and processing steps. Third, based on multiple candidate actions determined by the multimodal processing tools and their processing steps (covering multiple processing steps), the determination unit selects the target action from these candidate actions according to the target agent's action determination strategy, considering the target state to form the optimal processing tool and processing steps. Finally, the execution unit uses the processing tool within the target action to execute the processing steps of the target action on the target input data, achieving the optimal processing steps for the target input data using the optimal processing tool, resulting in accurate target output data.

[0212] Based on this, the intelligent agent of the device automatically selects a processing tool that matches the data type of the input data. The intelligent agent, through the state formed by the input data, the selected processing tool, and the output data of the previous step, flexibly and adaptively selects the optimal processing tool and processing steps from the multimodal processing tool and its multiple processing steps based on the action determination strategy. The intelligent agent then uses the optimal processing tool among the actions to execute the optimal processing steps on the input data, thereby improving the accuracy of the output data and thus enhancing the data processing effect of the intelligent agent.

[0213] Furthermore, the agent's data processing system includes an input module, a tool module, a processing module, an output module, and a learning module. The input module receives and preprocesses the agent's input data; the tool module integrates multimodal tools, providing a unified interface for the agent to use; the processing module calls the processing tools to execute processing steps; the output module outputs the corresponding output data; and the learning module trains the agent based on reinforcement learning.

[0214] This application also provides a computer device, which may be a server, see [link to previous document]. Figure 7 ,Should Figure 7This application provides a structural diagram of a server 700. The server 700 can vary significantly due to different configurations or performance characteristics. It may include one or more processors, such as a central processing unit (CPU) 722, and a memory 732, as well as one or more storage media 730 (e.g., one or more mass storage devices) for storing application programs 742 or data 744. The memory 732 and storage media 730 can be temporary or persistent storage. The program stored in the storage media 730 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the server. Furthermore, the CPU 722 may be configured to communicate with the storage media 730 and execute the series of instruction operations stored in the storage media 730 on the server 700.

[0215] Server 700 may also include one or more power supplies 726, one or more wired or wireless network interfaces 750, one or more input / output interfaces 758, and / or one or more operating systems 741, such as Windows Server. TM Mac OS X TM Unix TM Linux TM FreeBSD TM etc.

[0216] In this embodiment, the central processing unit 722 in the server 700 can execute the methods provided in the various optional implementations of the above embodiments.

[0217] The computer device provided in this application embodiment can also be a terminal, see [link to relevant documentation]. Figure 8 ,Should Figure 8 This is a structural diagram of a terminal provided in an embodiment of this application. Taking a smartphone as an example, the smartphone includes components such as a radio frequency (RF) circuit 810, a memory 820, an input unit 830, a display unit 840, a sensor 850, an audio circuit 860, a wireless Fidelity (WiFi) module 870, a processor 880, and a power supply 890. The input unit 830 may include a touch panel 831 and other input devices 832, the display unit 840 may include a display panel 841, and the audio circuit 860 may include a speaker 861 and a microphone 862. Those skilled in the art will understand that... Figure 8 The smartphone structure shown does not constitute a limitation on smartphones and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0218] The memory 820 can be used to store software programs and modules. The processor 880 executes various functions and data processing of the smartphone by running the software programs and modules stored in the memory 820. The memory 820 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the smartphone (such as audio data, phonebook, etc.). In addition, the memory 820 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0219] The processor 880 is the control center of the smartphone, connecting various parts of the smartphone via various interfaces and lines. It performs various functions and processes data by running or executing software programs and / or modules stored in the memory 820, and by accessing data stored in the memory 820. Optionally, the processor 880 may include one or more processing units; preferably, the processor 880 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into the processor 880.

[0220] In this embodiment, the processor 880 in the smartphone can execute the methods provided in the various optional implementations of the above embodiments.

[0221] According to one aspect of this application, a computer-readable storage medium is provided for storing a computer program that, when run on a computer device, causes the computer device to perform the methods provided in various optional implementations of the above embodiments.

[0222] According to one aspect of this application, a computer program product is provided, comprising a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform the methods provided in various optional implementations of the above embodiments.

[0223] The descriptions of the processes or structures corresponding to the above figures each have their own emphasis. For parts of a process or structure that are not described in detail, please refer to the relevant descriptions of other processes or structures.

[0224] The terms "first," "second," etc., used in this application's specification and the foregoing drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0225] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0226] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0227] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0228] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing computer programs, such as USB flash drives, portable hard drives, read-only memory (ROM), RAM, magnetic disks, or optical disks.

[0229] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0230] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A data processing method for an intelligent agent, characterized in that, The method includes: Based on the target data type of the target input data of the target intelligent agent, determine the processing tool corresponding to the target data type from the multimodal processing tools of the target intelligent agent; The target input data, the processing tool corresponding to the target data type, and the historical processing data of the target execution step to which the target input data belongs are determined as the target state; the historical processing data is the output data of the previous execution step of the target execution step. Based on the target state and the action determination strategy of the target agent, a target action is determined from multiple candidate actions; the multiple candidate actions are determined based on the multimodal processing tool and multiple processing steps of the multimodal processing tool. The processing tools in the target action execute the processing steps in the target action on the target input data to obtain the target output data.

2. The method according to claim 1, characterized in that, The action determination strategy is a strategy for determining the probability of action selection in a given state. The step of determining the target action from multiple candidate actions based on the target state and the action determination strategy of the target agent includes: The target action is determined from the plurality of candidate actions based on the selection probability of each candidate action in the target state; the target action has the highest selection probability in the target state.

3. The method according to claim 1, characterized in that, The action determination strategy is a strategy for determining the value of state-action pairs. The step of determining the target action from multiple candidate actions based on the target state and the target agent's action determination strategy includes: The target action is determined from the plurality of candidate actions based on the state-action pair value between the target state and each candidate action; the state-action pair value between the target state and the target action is the largest.

4. The method according to claim 1, characterized in that, The multimodal processing tools include text processing tools, image processing tools, and speech processing tools; if the target data type is a cross-modal data type, determining the processing tool corresponding to the target data type from the multimodal processing tools of the target agent based on the target data type of the target input data of the target agent includes: If the cross-modal data type includes text type and image type, the text processing tool and the image processing tool are determined from the multimodal processing tools; If the cross-modal data type includes text type and speech type, the text processing tool and the speech processing tool are determined from the multimodal processing tools; If the cross-modal data type includes image type and speech type, the image processing tool and the speech processing tool are determined from the multimodal processing tools; If the cross-modal data type includes text, image, and speech types, then the text processing tool, the image processing tool, and the speech processing tool are determined from the multimodal processing tools.

5. The method according to claim 1, characterized in that, The method further includes: The multimodal processing tool is augmented to obtain an updated multimodal processing tool; The step of determining the processing tool corresponding to the target data type from the multimodal processing tools of the target agent includes: The processing tool corresponding to the target data type is determined from the updated multimodal processing tool; The steps for determining the plurality of candidate actions include: The multiple candidate actions are determined based on the updated multimodal processing tool and its multiple processing steps.

6. The method according to claim 1, characterized in that, The method further includes: If the output data of the processing tool in the target action is erroneous, the target action is adjusted to obtain the adjusted target action; The step of executing the processing steps of the target action on the target input data through the processing tool in the target action to obtain target output data includes: The processing tools in the adjusted target action are used to perform the processing steps in the adjusted target action on the target input data to obtain the target output data.

7. The method according to claim 6, characterized in that, The error data indicates that the processing tool in the target action is incorrect, and the adjusted target action includes the adjusted processing tool; the error data also indicates that the processing step in the target action is incorrect, and the adjusted target action includes the adjusted processing step.

8. The method according to claim 1, characterized in that, The training steps for the target agent include: Based on the sample data type of the initial agent's sample input data, determine the processing tool corresponding to the sample data type from the multimodal processing tools of the initial agent; The sample state is determined by the sample input data, the processing tool corresponding to the sample data type, and the historical processing data of the sample execution step to which the sample input data belongs; the historical processing data is the output data of the previous execution step of the sample execution step. Based on the sample state and the action determination strategy of the initial agent, a sample action is determined from multiple candidate actions; the multiple candidate actions are determined based on the multimodal processing tool and multiple processing steps of the multimodal processing tool. The processing tool in the sample action executes the processing steps in the sample action on the sample input data to obtain the sample reward of the sample action in the sample state. The action determination strategy is optimized based on the sample reward to obtain the target agent corresponding to the initial agent.

9. The method according to claim 8, characterized in that, The action determination strategy is a strategy for determining the probability of action selection in a given state. The step of determining a sample action from multiple candidate actions based on the sample state and the initial agent's action determination strategy includes: Based on the selection probability of each candidate action in the sample state, the sample action is determined from the plurality of candidate actions; the sample action has the highest selection probability in the sample state. The step of optimizing the action determination strategy based on the sample reward to obtain the target agent corresponding to the initial agent includes: The selection probability of the sample action in the sample state is optimized and updated based on the sample reward to obtain the target agent.

10. The method according to claim 8, characterized in that, The action determination strategy is a strategy for determining the value of state-action pairs. The step of determining a sample action from multiple candidate actions based on the sample state and the initial agent's action determination strategy includes: The sample action is determined from the plurality of candidate actions based on the state-action pair value of the sample state and each candidate action; The step of optimizing the action determination strategy based on the sample reward to obtain the target agent corresponding to the initial agent includes: Based on the sample reward and the next state of the sample state, the value of the state-action pair of the sample state and the sample action is optimized and updated to obtain the target intelligent agent.

11. The method according to claim 10, characterized in that, The step of determining the sample action from the plurality of candidate actions based on the state-action pair value of the sample state and each candidate action includes: The sample action is determined from the plurality of candidate actions based on the preset probability, the state of the sample and the state-action pair value of each candidate action; The step of optimizing and updating the value of the state-action pair of the sample state and the sample action based on the sample reward and the next state of the sample state to obtain the target agent includes: Based on the sample reward and the next state of the sample state, the preset probability, the value of the state-action pair of the sample state and the sample action are optimized and updated to obtain the target intelligent agent.

12. A data processing device for an intelligent agent, characterized in that, The device includes: a determining unit and an executing unit; The determining unit is used to determine the processing tool corresponding to the target data type from the multimodal processing tools of the target intelligent agent based on the target data type of the target input data of the target intelligent agent. The determining unit is further configured to determine the target input data, the processing tool corresponding to the target data type, and the historical processing data of the target execution step to which the target input data belongs as the target state; the historical processing data is the output data of the previous execution step of the target execution step; The determining unit is further configured to determine a target action from multiple candidate actions based on the target state and the action determination strategy of the target agent; the multiple candidate actions are determined based on the multimodal processing tool and multiple processing steps of the multimodal processing tool; The execution unit is used to perform the processing steps of the target action on the target input data through the processing tool in the target action to obtain the target output data.

13. A computer device, characterized in that, The computer device includes a processor and memory: The memory is used to store computer programs and to transfer the computer programs to the processor; The processor is configured to execute the method according to any one of claims 1-11 according to instructions in the computer program.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program that, when run on a computer device, causes the computer device to perform the method according to any one of claims 1-11.

15. A computer program product, comprising a computer program, characterized in that, When the computer program is run on a computer device, it causes the computer device to perform the method according to any one of claims 1-11.

Images