Task execution method, apparatus, device, medium, and product

Abstract

Embodiments of the present application provide a task execution method, device, equipment, medium and product, relating to the field of artificial intelligence. The method comprises: in response to receiving a task execution instruction input by a user, generating an execution behavior based on the task execution instruction using a tool calling module; executing the execution behavior using a pre-trained model to generate result data after the execution behavior, and generating a joint embedding vector based on the result data and state information before and after an execution device; performing multi-path search in a Monte Carlo tree based on the joint embedding vector using a pre-set multi-path nearest neighbor algorithm to determine whether there is a path that meets matching requirements; and in response to there being a path that meets the matching requirements, obtaining a reflection execution behavior according to the path that meets the matching requirements, and executing the reflection execution behavior using the pre-trained model. The method is used to improve the efficiency of the search method that gradually expands in a multi-step task, thereby improving the decision-making efficiency.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and more particularly to a task execution method, apparatus, device, medium, and product. Background Technology

[0002] In multi-step task processing, existing intelligent devices typically employ agents (which can be called agents) based on Monte Carlo tree search and driven by large language models to perform state perception, path planning, and action decision-making.

[0003] Among the aforementioned related technologies, the Monte Carlo tree search scheme evaluates the long-term benefits of different actions by progressively expanding the search tree in order to select the optimal decision path.

[0004] In this approach, the path search lacks parallelism, resulting in low efficiency of the incremental search method in multi-step tasks, which in turn leads to low decision-making efficiency. Summary of the Invention

[0005] This application provides a task execution method, apparatus, device, medium, and product to solve the technical problem that the search method is inefficient, which in turn leads to low decision-making efficiency.

[0006] In a first aspect, embodiments of this application provide a task execution method, including:

[0007] In response to receiving a task execution command from the user, the tool invokes the module to generate execution behavior based on the task execution command;

[0008] A pre-trained model is used to execute the execution behavior to generate the result data after the execution behavior, and a joint embedding vector is generated based on the result data and the state information of the execution device before and after execution.

[0009] A pre-defined multi-path nearest neighbor algorithm is used to perform multi-path search in the Monte Carlo tree based on joint embedding vectors to determine whether there is a path that meets the matching requirements;

[0010] If a path that meets the matching requirements exists, the reflective execution behavior is obtained based on the path that meets the matching requirements, and the reflective execution behavior is executed using a pre-trained model.

[0011] In one possible embodiment, the result data is text type, and the before-and-after state information of the execution device is image type; the before-and-after state information of the execution device includes the state information of the execution device before execution and the state information of the execution device after execution.

[0012] A joint embedding vector is generated based on the result data and the pre- and post-execution device state information, including:

[0013] The result data, the state information before execution and the state information after execution are encoded by a preset multimodal encoder to obtain the first encoded data, the second encoded data and the third encoded data respectively.

[0014] The first, second, and third encoded data are fused together to obtain a joint embedding vector.

[0015] In one possible embodiment, the Monte Carlo tree is a multimodal Monte Carlo tree; the Monte Carlo tree includes multiple task subtrees, and each task subtree includes multiple child nodes; the preset multi-path nearest neighbor algorithm is a confidence upper bound algorithm;

[0016] A pre-defined multi-path nearest neighbor algorithm is used to perform multi-path search in the Monte Carlo tree based on joint embedding vectors to determine whether a path exists that meets the matching requirements, including:

[0017] Search for multiple candidate paths in the task subtree that have a similarity greater than a preset similarity threshold with the joint embedding vector;

[0018] For any candidate path, calculate the matching score of the child nodes based on the historical access count of the child nodes in the candidate path;

[0019] The confidence upper bound algorithm is used to calculate the confidence upper bound score of the child node based on the matching score of the child node and the number of historical visits.

[0020] Multi-path search is performed based on the upper confidence bound scores of the child nodes in the candidate paths to determine whether there is a path that meets the matching requirements.

[0021] In one possible embodiment, a multi-path search is performed based on the upper confidence bound scores of the child nodes in the candidate paths to determine whether a path that meets the matching requirements exists, including:

[0022] If the upper confidence score of any child node in the candidate path is greater than the preset upper confidence score, then it is determined that there is a path that meets the matching requirements.

[0023] If any child node in any candidate path has a confidence upper bound score that is less than or equal to the preset confidence upper bound score, it is determined that there is no path that meets the matching requirements.

[0024] After determining that no path meets the matching requirements, the method also includes:

[0025] Create a new child node in the task subtree where the candidate path is located, and update the access count of the child node. The new child node is a sibling node of the target node, and the target node is a child node whose confidence upper bound score is less than or equal to the preset confidence upper bound score.

[0026] In one possible embodiment, after performing reflective execution using a pre-trained model, the process further includes:

[0027] Update the visit count of child nodes in the path that meets the matching requirements.

[0028] In one possible embodiment, it also includes:

[0029] In response to the existence of a path that meets the matching requirements, the reflection execution behavior corresponding to the path that meets the matching requirements is sent to the tool invocation module. When the tool invocation module receives a task execution instruction that is semantically consistent with the task execution instruction, it sends the reflection execution behavior to the pre-trained model and executes the reflection execution behavior.

[0030] If no matching path exists, the sub-paths in the path that does not meet the matching requirements are sent to the tool invocation module to invoke the pre-trained model to generate reflective execution behavior. The sub-paths include target sub-nodes in the path that does not meet the matching requirements. The target sub-nodes are sub-nodes whose confidence upper bound scores are greater than the preset confidence upper bound scores.

[0031] Secondly, embodiments of this application provide a task execution apparatus, comprising:

[0032] The first generation module is used to respond to a task execution instruction input by the user and generate an execution behavior based on the task execution instruction using a tool invocation module;

[0033] The execution module is used to execute actions using a pre-trained model to generate result data after the execution actions.

[0034] The second generation module is used to generate a joint embedding vector based on the result data and the state information of the execution device before and after execution.

[0035] The determination module is used to perform multi-path search in the Monte Carlo tree based on the joint embedding vector using a preset multi-path nearest neighbor algorithm to determine whether there is a path that meets the matching requirements;

[0036] The acquisition module is used to acquire the reflection execution behavior based on the path that meets the matching requirements if a path that meets the matching requirements exists.

[0037] The Reflection module is used to perform reflective execution behavior using a pre-trained model.

[0038] Thirdly, embodiments of this application provide an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0039] The memory stores instructions that the computer executes;

[0040] The processor executes computer execution instructions stored in memory to implement the methods provided above.

[0041] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the methods provided above.

[0042] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the method described above.

[0043] The task execution method, apparatus, device, medium, and product provided in this application generate execution behavior in response to user-input task execution instructions, execute the execution behavior using a pre-trained model, and generate a joint embedding vector by combining the result data after the execution behavior and the state information of the execution device before and after execution. This enables a unified characterization of the result data and state changes during task execution. By employing a preset multi-path nearest neighbor algorithm to perform multi-path search in a Monte Carlo tree based on the joint embedding vector, and obtaining the reflection execution behavior when a path that meets the matching requirements exists and executing it using a pre-trained model, the efficiency of path expansion and search can be effectively improved, thereby enhancing the decision-making efficiency in multi-step complex tasks. Attached Figure Description

[0044] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0045] Figure 1 This is an application scenario diagram of a task execution method provided in one embodiment of this application;

[0046] Figure 2 A flowchart illustrating a task execution method provided in an embodiment of this application;

[0047] Figure 3 A flowchart of a task execution method provided in another embodiment of this application;

[0048] Figure 4 This is a schematic diagram of the structure of a Monte Carlo tree provided in an embodiment of this application;

[0049] Figure 5 This is a schematic diagram of the structure of a task execution device provided in an embodiment of this application;

[0050] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0051] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0052] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0053] To clearly understand the technical solution of this application, the solutions of the prior art will be described in detail first.

[0054] Multi-step task execution technology for intelligent devices involves agent-based state perception, path planning, and action decision-making. It is applicable to scenarios such as smart home devices, mobile terminals, and industrial equipment that require continuous operation based on user instructions. Related systems typically include a user interface, a tool invocation module, an execution device, and a model module for processing the execution results.

[0055] In existing solutions, after receiving task instructions, the agent typically parses the instructions first, then calls tools or models to generate actions, performing step-by-step execution and search decisions. Some solutions use Monte Carlo tree search for path selection, while others rely on large language models combined with historical context to generate subsequent behaviors. The basic idea is to continue expanding the next action based on feedback after each step. However, this step-by-step search method often requires repeatedly traversing candidate paths in multi-step tasks, resulting in limited search efficiency; furthermore, selecting only one path for searching at a time leads to a lack of parallelism.

[0056] In view of this, improving decision-making efficiency and accuracy, increasing path expansion efficiency, and reducing computational overhead in complex multi-step tasks have become urgent technical problems. To address these issues, a task execution method is provided. This method, upon receiving a user-input task execution command, employs a pre-defined multi-path nearest neighbor algorithm to perform multi-path search in a Monte Carlo tree. This search leverages historical task experience to find paths matching the current task state, thus overcoming the efficiency limitations of incremental expansion along a single path. To support effective retrieval of the multi-path nearest neighbor algorithm within the tree, the execution results of the current task step and changes in device state must be uniformly represented as vectors capable of participating in the search. Therefore, this application further proposes generating joint embedding vectors based on result data and device state information before and after execution. The result data originates from the output of a pre-trained model after executing its execution behavior. Based on this, when a path satisfying the matching requirements is identified, a reflective execution behavior is obtained based on that path. This reflective execution behavior is then executed using the pre-trained model, directly reusing validated execution behaviors from historical paths. This effectively improves the efficiency of path expansion and search, thereby enhancing decision-making efficiency in complex multi-step tasks.

[0057] Figure 1 This is an application scenario diagram of a task execution method provided in one embodiment of this application, such as... Figure 1 As shown, the application scenario corresponding to the task execution method provided in this application may include: a user terminal 101, an execution device 102, and a task execution apparatus 103 that can be deployed in the execution device. Specifically, a client for the execution device may be installed on the user terminal. The user terminal inputs task execution instructions to the execution device 102 through the client. The execution device 102 uses a tool invocation module to generate execution behavior based on the task execution instructions; it uses a pre-trained model to execute the execution behavior to generate result data after the execution behavior, and generates a joint embedding vector based on the result data and the state information of the execution device before and after; it uses a preset multi-path nearest neighbor algorithm to perform multi-path search in the Monte Carlo tree based on the joint embedding vector to determine whether there is a path that meets the matching requirements; in response to the existence of a path that meets the matching requirements, it obtains the reflective execution behavior based on the path that meets the matching requirements, and executes the reflective execution behavior using the pre-trained model. The execution device may be a smart home device, a mobile terminal, an industrial device, or other forms of electronic equipment, and this embodiment does not limit this.

[0058] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0059] Figure 2A flowchart of a task execution method provided in an embodiment of this application is shown below. Figure 2 As shown, the execution entity in this embodiment is a task execution device. This task execution device can be implemented through a computer program, or through a medium storing the relevant computer program, such as a USB flash drive and / or optical disc, or through a physical device integrating or installing the relevant computer program, such as a chip or electronic device. The electronic device can be a server, server cluster, etc. The task execution method provided in this embodiment includes the following steps:

[0060] S201: In response to receiving a task execution instruction from the user, the tool calling module generates execution behavior based on the task execution instruction.

[0061] The task execution instructions input by the user can be any type of multimodal instruction, such as text instructions, image instructions, voice instructions, or any combination thereof.

[0062] The tool calling module (which can be called the LLM calling module) can be any method, program, or model that can generate execution behavior.

[0063] Among them, the execution behavior can be the action instructions generated by the tool calling module according to the task execution instructions, which are used to instruct the subsequent pre-trained model to perform the corresponding operations.

[0064] It should be noted that the execution behavior can return a predefined function format within the tool's calling module, allowing for direct execution of that function later. An execution behavior typically consists of one or more operation steps, each corresponding to a specific action on the smart device, such as opening an application or clicking on a UI element.

[0065] In this embodiment, the task execution device can be deployed on a server, edge computing node, smart terminal, or task processing device in the local controller of the execution device. User-inputted task execution instructions can originate from a touch interface, text input box, voice acquisition unit, graphical control panel, or structured instructions issued by an external business system, and are transmitted to the task processing device via a communication interface. The task execution instructions carry at least task target information; in the case of multi-step tasks, they can also carry object identifiers, constraints, execution order, time requirements, and device context information corresponding to the execution device. The tool invocation module can include an instruction parsing unit, a task splitting unit, a tool mapping unit, and a behavior orchestration unit. The instruction parsing unit identifies the task type, target object, and operation semantics; the task splitting unit divides a complex task into multiple subtasks; the tool mapping unit establishes a correspondence between subtasks and target tools in a preset tool set; and the behavior orchestration unit forms a sequence of execution behaviors with an execution order.

[0066] During this step, task execution instructions can be standardized. Upon receiving the instruction, natural language, speech-to-text, or structured interface data can be converted into a unified task description format. This format can include a subject-verb-object structure, time parameters, target state parameters, and device identification parameters. Then, the tool invocation module matches the corresponding tool according to the task type. For example, in a smart home scenario, it matches device control tools, status reading tools, and environmental perception tools; in a mobile terminal scenario, it matches application invocation tools, interface operation tools, and access control tools; and in an industrial equipment scenario, it matches mechanical control tools, process parameter setting tools, and visual inspection tools. After matching the tools, the module generates the execution behavior corresponding to the current task objective. This behavior can be a single action instruction or a set of actions arranged sequentially. If the task execution instruction contains multiple subtasks, the module constructs an execution graph based on dependencies and outputs the currently pending execution behavior based on the fulfillment of preconditions. The data structure for the execution behavior can be a structured message containing fields such as action type, target object, input parameters, calling tool identifier, expected return type, and security verification, so that the pre-trained model can directly receive and process it.

[0067] In one possible embodiment, the tool invocation module can also normalize task execution instructions by combining historical task templates, mapping semantically similar expressions to the same execution intent. For example, when a user inputs "turn on the living room lights and brighten them" and "increase the brightness of the main living room light to a higher level," both can be normalized to actions related to turning on and setting the brightness of living room lighting equipment. For open instructions that cannot be directly mapped, the tool invocation module can call the intent recognition model to generate intermediate task slots, and then form execution behaviors based on the content of the slots. Based on the above analysis, this step, through the tool invocation module, completes the conversion of task instructions into execution behaviors, enabling subsequent execution chains to process user input from multiple sources and with multiple expressions based on a unified format, and establishing a clear action starting point for multi-step continuous tasks.

[0068] It should be understood that the above examples are for illustrative purposes only and are not intended to be limiting.

[0069] S202: Use a pre-trained model to perform the execution behavior to generate the result data after the execution behavior, and generate a joint embedding vector based on the result data and the state information of the execution device before and after execution.

[0070] In this embodiment, the pre-trained model is used to execute the execution behavior generated by the tool invocation module and output the result data after the execution behavior. It is also used for subsequent execution reflection. The pre-trained model can be a model with multimodal understanding and action reasoning capabilities. Its inputs include the execution behavior, the current state of the execution device, and the necessary environmental context. Its outputs include the execution result, the state change description, and the feature information required for subsequent semantic representation. The result data is used to characterize the objective feedback after the execution behavior acts on the execution device, and may specifically include one or more of the following: result description text, log record description, or error code information description.

[0071] Among them, the device state information before and after execution corresponds to the device state before and after the execution behavior, respectively. The device state information before execution can be obtained by the state reading interface, device driver layer or cache state pool before the execution behavior is triggered, and the device state information after execution can be collected again after the execution behavior is completed.

[0072] For example, the pre-state information of the execution device may include: screenshots of the execution device's interface and interface layout before the execution action is performed. The post-state information of the execution device may include: screenshots of the execution device's interface and interface layout after the execution action is performed.

[0073] It should be noted that there are no restrictions on the specific data type of the result data; it can be text, image, etc. Similarly, there are no restrictions on the data type of the device's state information before and after execution; it can be image, text, etc.

[0074] Among them, the pre-trained model can be a commonly used large language model.

[0075] The joint embedding vector is used to generate a unified representation based on the result data and the state information of the execution device before and after execution, so as to support the subsequent multi-path search in the Monte Carlo tree.

[0076] In practice, after receiving the execution behavior, the pre-trained model calls the corresponding execution interface according to the action type and target object in the execution behavior, and monitors the return events during the execution process. Before the behavior execution begins, the system first collects the execution device's pre-state information; after the behavior execution is completed, it collects the execution device's post-state information and receives the result data output by the pre-trained model. Subsequently, feature representation processing is performed based on the result data and the pre- and post-state information of the execution device to generate a joint embedding vector that can represent the current task execution state.

[0077] For example, the joint embedding vector dimension can be set to 128, 256, or 512 dimensions to control the computational overhead of subsequent nearest neighbor retrieval and tree search while ensuring semantic distinguishability.

[0078] Based on the above analysis, this step incorporates the execution result and the state information of the execution device before and after execution into a unified semantic space. This not only preserves the direct feedback after the action is executed but also retains the relationship between scene changes before and after execution, enabling the current task state to participate in subsequent path matching as a single vector. It should be understood that the above example is merely illustrative and not limiting.

[0079] S203: A pre-defined multi-path nearest neighbor algorithm is used to perform multi-path search in the Monte Carlo tree based on the joint embedding vector to determine whether there is a path that meets the matching requirements.

[0080] In this embodiment, the Monte Carlo tree serves as a tree-structured decision-making structure for multi-path search and path reuse, used to store state information, action transition relationships, and path statistics generated during the execution of historical tasks. Each node in the tree corresponds to a historical task state representation, each edge corresponds to an execution action, indicating the operation required to transition from the parent node's state to the child node's state, and each path from the root node to the current node represents a set of historical continuous decision-making processes. Nodes can record state representations and path search-related statistics.

[0081] The multi-path nearest neighbor algorithm is used to perform multi-path search in the Monte Carlo tree based on the current joint embedding vector to filter historical paths similar to the current task state and determine whether there is a path that meets the matching requirements. A path that meets the matching requirements can refer to a path that meets preset matching conditions.

[0082] In practice, the current joint embedding vector is first used as the retrieval vector, and a nearest neighbor search is performed on the candidate node set of the Monte Carlo tree. The candidate node set can be the entire tree or obtained after range filtering based on task type, device category, or step stage. The multi-path nearest neighbor algorithm calculates the similarity between the retrieval vector and the historical joint embedding vectors of the candidate nodes. The similarity measure can be cosine similarity, the reciprocal of Euclidean distance, or the normalized value of Mahalanobis distance. To improve search efficiency, a vector index structure can be used to coarsely screen the nodes first, and then the coarsely screened results can be finely ranked. During fine ranking, not only the similarity between a single node and the current joint embedding vector is considered, but the continuous state correlation of the path to which the node belongs can also be further considered. For example, the local matching degree related to the current task stage is calculated for multiple key nodes in the candidate path, and the path-level matching result is determined accordingly. Through this calculation method, the correlation between the current state and the historical path can be incorporated into the matching judgment.

[0083] During the search process, candidate expansion directions can be ranked by incorporating relevant statistical information from the path search process. The multi-path nearest neighbor algorithm does not retain only a single path, but rather the top K candidate paths with the highest scores, where K can be set from 2 to 10 depending on system resources and task complexity. Then, matching requirements are applied to the top K candidate paths. These matching requirements include not only path matching degree but also device state compatibility verification, action executability verification, and time validity verification. For example, if the historical device type corresponding to a candidate path is inconsistent with the currently executing device, or if subsequent actions in the path depend on resources that do not currently exist, then that path is not considered a path that meets the matching requirements.

[0084] Based on the above analysis, this step performs multi-path nearest neighbor search in the Monte Carlo tree by jointly embedding vectors. It no longer relies on exhaustively searching all candidate branches step by step. Instead, it first uses a unified semantic representation to quickly locate multiple potentially related historical paths, and then combines statistical information from the tree to complete the matching determination. This balances search efficiency and path determination accuracy in a multi-step task. It should be understood that the above example is merely illustrative and not limiting.

[0085] S204: In response to the existence of a path that meets the matching requirements, the reflective execution behavior is obtained based on the path that meets the matching requirements, and the reflective execution behavior is executed using a pre-trained model.

[0086] In this embodiment, the reflective execution behavior is used to extract, modify, combine, and re-execute subsequent behaviors from historical paths when a path that meets the matching requirements exists, thereby continuing the current task. The reflective execution behavior does not simply repeat the original historical actions; instead, it filters reusable subsequent actions based on the historical execution context corresponding to the path that meets the matching requirements, and generates subsequent behaviors adapted to the current task stage by combining the current joint embedding vector, the current device state, and the task objective.

[0087] The path that meets the matching requirements can record multiple historical behavior nodes, the execution results of each node, and reward markers related to task success. The task execution device can then identify the behavior segment that is most relevant to the current state. In this step, the pre-trained model continues to act as the execution subject, reasoning and executing the reflective execution behavior, and outputting new result data for subsequent task iterations.

[0088] In practice, after determining that there is a path that meets the matching requirements, the execution behavior text stored in the child node corresponding to the path that meets the matching requirements is read. The task execution device determines the read execution behavior text as a reflective execution behavior and sends it to the pre-trained model for execution. The pre-trained model then completes the specific operation corresponding to the execution behavior text to execute the reflective execution behavior.

[0089] In one possible embodiment, after the pre-trained model performs reflective execution behavior, it continues to generate post-execution result data, and uses the new result data and the updated execution device state information before and after execution to construct joint embedding vectors again, thereby forming a closed-loop iterative process.

[0090] Based on the above analysis, this step, by extracting and executing reflective actions from historical paths that meet the matching requirements, enables the agent to reuse verified behavioral experience in the current task processing, reducing the repeated generation and reasoning of complete historical contexts, and ensuring that subsequent actions remain connected to the current state. It should be understood that the above example is merely illustrative and not limiting.

[0091] This embodiment provides a task execution method that receives a task execution command input by a user, uses a tool invocation module to generate an execution behavior based on the command, executes the execution behavior using a pre-trained model to generate result data, and generates a joint embedding vector based on the result data and the state information of the execution device before and after execution. A preset multi-path nearest neighbor algorithm is used to perform multi-path search in a Monte Carlo tree based on the joint embedding vector to determine if a path that meets the matching requirements exists. If a path that meets the matching requirements exists, a reflective execution behavior is obtained based on that path, and the reflective execution behavior is executed using the pre-trained model. This method effectively improves the efficiency of path expansion and search, thereby enhancing decision-making efficiency in complex multi-step tasks.

[0092] Based on the aforementioned embodiments, the result data is of text type, and the before and after state information of the execution device is of image type; the before and after state information of the execution device includes the state information of the execution device before execution and the state information of the execution device after execution.

[0093] Accordingly, when generating the joint embedding vector based on the result data and the before-and-after state information of the execution device, it includes:

[0094] The result data, the state information before execution and the state information after execution are encoded by a preset multimodal encoder to obtain the first encoded data, the second encoded data and the third encoded data respectively; the first encoded data, the second encoded data and the third encoded data are fused to obtain the joint embedding vector.

[0095] The result data may include one or more of the following: result description text, log information description text, or error code information description text.

[0096] The execution device's pre-execution and post-execution status information includes pre-execution status information and post-execution status information. This status information may include screenshots and layout details.

[0097] The preset multimodal encoder can be a pre-trained image-text model based on contrastive learning, or a visual language model with few-shot learning capabilities.

[0098] The first encoded data corresponds to the result data, the second encoded data corresponds to the pre-execution state information, and the third encoded data corresponds to the post-execution state information. In practical implementation, the input to the preset multimodal encoder is the result data after text segmentation, and the pre-execution and post-execution state information after image preprocessing. After receiving the aforementioned three types of input, the preset multimodal encoder obtains the first encoded data representing the semantics of the execution result, the second encoded data representing the pre-execution scene, and the third encoded data representing the post-execution scene. The first, second, and third encoded data are then fused. This fusion can be achieved by projecting features after concatenation through a fully connected layer, or by using a weighted fusion method with a self-attention mechanism, establishing a correlation between the three types of encoded data within the same semantic space, thereby outputting a joint embedding vector. This joint embedding vector simultaneously contains the result semantics, pre-execution state differences, and post-execution state change information, serving as a unified representation for subsequent path matching.

[0099] When fusing the first, second, and third encoded data, a self-attention mechanism is used to dynamically calculate the weights of each encoded data to enhance the expression of key modal information and weaken the interference of redundant or conflicting information, thereby obtaining a joint embedding vector.

[0100] The weights of the encoded data corresponding to the self-attention mechanism can be dynamically calculated by a preset multimodal encoder based on the correlation between the first, second, and third encoded data.

[0101] In this embodiment, a preset multimodal encoder is used to encode the result data, the pre-execution device state information, and the post-execution device state information respectively to obtain first encoded data, second encoded data, and third encoded data. Based on the first encoded data, second encoded data, and third encoded data, a fusion process is performed to obtain a joint embedding vector. Since encoding and fusion are completed within the same representation space, the joint embedding vector can simultaneously reflect the semantic output of the execution behavior and changes in the scene state. Subsequent judgments on path matching requirements can be based on more complete multimodal information, providing consistent feature input for multi-path nearest neighbor search and improving the applicability to complex task execution scenarios.

[0102] In one possible implementation, the Monte Carlo tree is a multimodal Monte Carlo tree; the Monte Carlo tree includes multiple task subtrees, and each task subtree includes multiple child nodes; the preset multi-path nearest neighbor algorithm is a confidence upper bound algorithm; when using the preset multi-path nearest neighbor algorithm to perform multi-path search in the Monte Carlo tree based on the joint embedding vector to determine whether there is a path that meets the matching requirements, it includes:

[0103] Search for multiple candidate paths in the task subtree whose similarity to the joint embedding vector is greater than a preset similarity threshold; for any candidate path, calculate the matching score of the child node based on the historical access count of the child node in the candidate path; use the confidence upper bound algorithm to calculate the confidence upper bound score of the child node based on the matching score and historical access count of the child node; perform multi-path search based on the confidence upper bound score of the child node in the candidate path to determine whether there is a path that meets the matching requirements.

[0104] In the specific implementation, after obtaining the joint embedding vector, the task subtree corresponding to the semantics of the current task is first located in the Monte Carlo tree. Then, multiple candidate paths are selected in the task subtree based on vector similarity. The similarity can be calculated using cosine similarity or Euclidean distance, and paths with a similarity greater than a preset threshold are retained as candidate paths.

[0105] For example, the candidate paths can be the top N paths whose similarity to the joint embedding vector is greater than a preset similarity threshold, which can be represented as Top-N.

[0106] Specifically, when calculating the matching score of a child node based on the historical access count of the child node in any candidate path, the following formula (1) can be used.

[0107] Formula (1)

[0108] in, For matching scores; This indicates the number of leaf nodes under the root node of the task subtree containing the candidate path; This indicates the number of times the child node has been accessed. Similarity; This represents the number of visits to the j-th child node under the root node of the task subtree containing the candidate path.

[0109] Formula (1) means that the reuse value of the candidate path in the current task step is evaluated by comprehensively considering the proportion of visits to the sub-nodes of the historical path and the semantic similarity between the current state and the historical state.

[0110] Optionally, the confidence upper bound score for a child node is calculated using the UCB1 algorithm based on the matching score and historical access count of the child node. When, it can be represented as shown in formula (2):

[0111] Formula (2)

[0112] in, This indicates the number of times the root node of this leaf node has been visited.

[0113] Formula (2) means that by balancing exploration and utilization, priority is given to child nodes with high matching scores and fewer visits, thereby expanding the underexplored path with limited search cost.

[0114] Optionally, the upper confidence bound algorithm (UCB1) is used to calculate the upper confidence bound score of the child node based on the matching score of the child node and the number of historical visits. When, it can be expressed as shown in formula (3):

[0115] Formula (3)

[0116] in, This indicates the number of times the root node of this leaf node has been visited; To explore the coefficients, the reinforcement learning model can dynamically adjust them based on the current task complexity and output them in real time.

[0117] In this embodiment, when performing multi-path search based on the upper confidence bound scores of the child nodes in the candidate path to determine whether there is a path that meets the matching requirements, if all child nodes in a certain candidate path are child nodes that meet the requirements, then the candidate path is regarded as a path that meets the matching requirements. If there are no child nodes in all candidate paths that meet the requirements, then it is determined that there is no path that meets the matching requirements among the candidate paths.

[0118] In this embodiment, when using a preset multi-path nearest neighbor algorithm to perform multi-path search in the Monte Carlo tree based on the joint embedding vector to determine whether there is a path that meets the matching requirements, multiple candidate paths with a similarity greater than a preset similarity threshold to the joint embedding vector are searched in the task subtree. For any candidate path, the matching score of the child node is calculated based on the historical access count of the child node in the candidate path. A confidence upper bound algorithm is used to calculate the confidence upper bound score of the child node based on the matching score and the historical access count. Multi-path search is performed based on the confidence upper bound score of the child node in the candidate path to determine whether there is a path that meets the matching requirements. After finding multiple candidate paths with a similarity greater than a preset similarity threshold to the joint embedding vector in the task subtree, for any candidate path, the matching score of the child node is calculated based on the historical access count of the child node in the candidate path. A confidence upper bound algorithm is used to calculate the confidence upper bound score of the child node based on the matching score and the historical access count. Multi-path search is performed based on the confidence upper bound score of the child node in the candidate path to determine whether there is a path that meets the matching requirements. This allows path selection within a task subtree to utilize both the current joint embedding vector and historical access statistics, thereby retaining more valuable paths with limited search costs and outputting matching results that better align with the semantics of the current task.

[0119] In one possible implementation, a multi-path search is performed based on the upper confidence score of the child nodes in the candidate path to determine whether a path that meets the matching requirements exists. This includes: if the upper confidence score of any child node in the candidate path is greater than a preset upper confidence score, then a path that meets the matching requirements exists; if the upper confidence score of any child node in each candidate path is less than or equal to the preset upper confidence score, then no path that meets the matching requirements exists. After determining that no path meets the matching requirements exists, the method further includes: creating a new child node in the task subtree where the candidate path is located and updating the access count of the child node. The new child node is a sibling node of the target node, and the target node is a child node whose upper confidence score is less than or equal to the preset upper confidence score.

[0120] The initial value of the pre-set upper bound score is set based on human experience, and the optimal value is obtained through subsequent learning and optimization.

[0121] In its implementation, after screening candidate paths, the task execution device first compares the upper confidence bound score of each child node on each candidate path with a preset upper confidence bound score. When all child nodes in a candidate path exceed the preset upper confidence bound score, the path is determined to meet the matching requirements, and the corresponding path is output for subsequent action execution. When any child node in a candidate path has a score less than or equal to the preset upper confidence bound score, the child node is determined as the target node, and the current candidate path is determined not to meet the matching requirements. If all candidate paths fail the judgment, it means that there is no reflexive execution behavior matching the task execution instruction in the task subtree. In this case, a new sibling node is created for the target node in the corresponding task subtree. The node identifier, parent node index, and hierarchy information of the new child node are consistent with the nodes at the same level, while its action candidate or state representation is reinitialized according to the current task context. At the same time, the access count of the new child node is initialized to a preset initial value, such as 0, and dynamically updated according to the actual access situation during subsequent execution.

[0122] In terms of working principle, candidate paths are first uniformly filtered based on the upper bound score of the child nodes. Then, a path that can continue execution is determined based on the preset upper bound score. When the current set of paths is insufficient to meet the matching requirements, a new child node at the same level as the target node is created in the task subtree to supplement the search branches that have not yet been fully explored, and their visit counts are updated to be included in the subsequent determination process. Thus, the search process can continue to expand the branches that have not been fully explored while retaining high-confidence paths, so that the path selection in the task subtree is more in line with the current matching status.

[0123] Therefore, in this embodiment, when performing multi-path search based on the upper confidence bound scores of child nodes in candidate paths to determine whether a path meets the matching requirements, if the upper confidence bound score of any child node in a candidate path is greater than a preset upper confidence bound score, then a path meeting the matching requirements is determined to exist; if the upper confidence bound score of any child node in any candidate path is less than or equal to the preset upper confidence bound score, then no path meeting the matching requirements is determined to exist. After determining that no path meeting the matching requirements exists, a new child node is created in the task subtree where the candidate path is located, and the access count of the child node is updated. The new child node is a sibling node of the target node, and the target node is a child node whose upper confidence bound score is less than or equal to the preset upper confidence bound score. This allows for rapid determination of the validity of candidate paths based on the upper confidence bound scores, and timely generation of new sibling nodes to expand the search space when requirements are not met, thereby making the multi-path search results more complete, the path updates more timely, and providing more stable candidate support for subsequent decisions during task execution.

[0124] In one possible implementation, after performing reflective execution using a pre-trained model, the process also includes updating the visit counts of child nodes in the path that meet the matching requirements.

[0125] Among them, the child nodes in the path that meet the matching requirements are those whose confidence upper bound score is greater than the preset confidence upper bound score.

[0126] In the specific implementation, after receiving the reflection execution behavior corresponding to the path that meets the matching requirements, the tool calling module sends the reflection execution behavior to the pre-trained model to complete the execution, and after obtaining the execution reflection execution behavior, it triggers the access count update instruction, and updates the access count of each child node according to the node index corresponding to the child node.

[0127] In this embodiment, after performing reflective execution using a pre-trained model, the visit counts of child nodes in paths that meet the matching requirements are updated. This allows the actual execution results of the reflective execution to be reflected in the path statistics, enabling subsequent multi-path nearest neighbor algorithms to obtain more accurate data when selecting candidate paths.

[0128] In one possible implementation, the reflective execution behavior is obtained based on the path that meets the matching requirements, including: obtaining the corresponding reflective execution sub-behavior from the child nodes of the path that meets the matching requirements; and merging the reflective execution sub-behaviors to obtain the reflective execution behavior.

[0129] In the specific implementation, once a path satisfying the matching requirements is identified, the behavior fragments stored in each child node under that path are read and extracted as reflexive execution sub-behaviors. The extracted sub-behaviors can be command fragments, control instructions, etc., and fragments from different levels within each child node can participate in the extraction. Subsequently, these reflexive execution sub-behaviors are fused. The fusion process can splice, merge, or reorganize multiple fragments based on semantic consistency, execution order relevance, and task continuity. The fusion process generates reflexive execution behaviors that can be directly invoked by the model. The fused reflexive execution behaviors maintain consistency with the current task context and are fed into the pre-trained model as input for subsequent execution.

[0130] In this embodiment, corresponding reflective execution sub-behaviors are obtained from the child nodes of the path that meet the matching requirements; these reflective execution sub-behaviors are then fused to obtain reflective execution behaviors. This approach integrates scattered local information within the path into coherent execution behaviors, enabling path matching results to be directly converted into executable content, thus improving the integrity and consistency of behavior connections in multi-step tasks.

[0131] Based on the foregoing embodiments, it further includes:

[0132] In response to the existence of a path that meets the matching requirements, the reflection execution behavior corresponding to the path that meets the matching requirements is sent to the tool invocation module. When the tool invocation module receives a task execution instruction that is semantically consistent with the task execution instruction, it sends the reflection execution behavior to the pre-trained model and executes the reflection execution behavior.

[0133] If no matching path exists, the sub-paths in the path that does not meet the matching requirements are sent to the tool invocation module to invoke the pre-trained model to generate reflective execution behavior. The sub-paths include target sub-nodes in the path that does not meet the matching requirements. The target sub-nodes are sub-nodes whose confidence upper bound scores are greater than the preset confidence upper bound scores.

[0134] It is understandable that all child nodes included in the sub-path are target child nodes.

[0135] Specifically, the task execution device first determines whether a path that meets the matching requirements exists. If it does, the reflective execution behavior corresponding to that path is directly reused, and the tool invocation module forwards the reflective execution behavior to the pre-trained model for execution. If it does not exist, the target sub-nodes that meet the threshold conditions in the paths that do not meet the matching requirements are extracted to form sub-paths, and then the tool invocation module calls the pre-trained model to generate new reflective execution behaviors.

[0136] Specifically, when the sub-paths in the path that does not meet the matching requirements are sent to the tool invocation module, and the tool invocation module calls the pre-trained model to generate reflective execution behavior, the sub-paths are used to provide the pre-trained model with the historical context reference closest to the current task state, so that the pre-trained model can generate new reflective execution behavior based on the historical state information, historical operation sequence and historical execution results in the sub-path, combined with the current task state.

[0137] In this embodiment, if a path that meets the matching requirements exists, the corresponding reflective execution behavior is sent to the tool invocation module. When the tool invocation module receives a task execution instruction semantically consistent with the task execution instruction, it sends the reflective execution behavior to the pre-trained model and executes it. If no path that meets the matching requirements exists, the sub-paths of the paths that do not meet the matching requirements are sent to the tool invocation module to invoke the pre-trained model to generate reflective execution behavior. This reduces the processing overhead of repeated searching and generation, improving the path reuse efficiency and behavior generation consistency in multi-step task execution.

[0138] Figure 3 A flowchart of a task execution method provided in another embodiment of this application is shown below. Figure 3 As shown, the task execution method provided in this embodiment includes the following steps:

[0139] S301: In response to receiving a task execution instruction from the user, the tool calling module generates execution behavior based on the task execution instruction.

[0140] S302: Use a pre-trained model to execute the action to generate the result data after the action is executed.

[0141] S303: The result data, the state information before execution and the state information after execution are encoded by a preset multimodal encoder to obtain the first encoded data, the second encoded data and the third encoded data respectively; the first encoded data, the second encoded data and the third encoded data are fused to obtain the joint embedding vector.

[0142] S304: Search for multiple candidate paths in the task subtree whose similarity to the joint embedding vector is greater than a preset similarity threshold; for any candidate path, calculate the matching score of the child node based on the historical access count of the child node in the candidate path; use the confidence upper bound algorithm to calculate the confidence upper bound score of the child node based on the matching score and historical access count of the child node.

[0143] S305: If the upper confidence score of any child node in the candidate path is greater than the preset upper confidence score, then it is determined that there is a path that meets the matching requirements, and S306 is executed; if the upper confidence score of any child node in each candidate path is less than or equal to the preset upper confidence score, it is determined that there is no path that meets the matching requirements, and S307 is executed.

[0144] S306: In response to the existence of a path that meets the matching requirements, the reflective execution behavior corresponding to the path that meets the matching requirements is sent to the tool invocation module, so that when the tool invocation module receives a task execution instruction that is semantically consistent with the task execution instruction, it sends the reflective execution behavior to the pre-trained model and executes the reflective execution behavior.

[0145] Execute S306 and then execute S309.

[0146] S307: In response to the absence of a path that meets the matching requirements, the sub-paths in the path that does not meet the matching requirements are sent to the tool invocation module to invoke the pre-trained model to generate reflective execution behavior. The sub-paths include target sub-nodes in the path that does not meet the matching requirements. The target sub-nodes are sub-nodes whose confidence upper bound scores are greater than the preset confidence upper bound scores.

[0147] S308: Create a new child node in the task subtree containing the candidate path and update the access count of that child node. The new child node is a sibling node of the target node, and the target node is a child node whose confidence upper bound score is less than or equal to the preset confidence upper bound score. Simultaneously, store the generated reflexive execution behavior in the newly created child node.

[0148] Optionally, the access count for a new node is initialized to 0.

[0149] After executing S308, execute S301.

[0150] S309: Update the access count of child nodes in the path that meets the matching requirements.

[0151] The following is based on Figure 4 The Monte Carlo tree example shown illustrates multi-path search based on joint embedding vectors to determine if a path exists that meets the matching requirements. Figure 4 As shown, root node 1 corresponds to the initial state of the task, containing a joint embedding vector obtained by fusing the first encoded data (tEncoding), the second encoded data (bEncoding), and the third encoded data (aEncoding), as well as the number of visits p to this node. Assuming that during the current task execution, the task execution device generates a joint embedding vector and calculates its similarity with the historical task information stored in each of the root node 1's direct child nodes (i.e., child nodes 2, 3, and 4). If the similarity between child nodes 2 and 3 is higher than a preset similarity threshold, and the similarity between child node 4 is lower than the preset similarity threshold, then the paths containing child nodes 2 and 3 are selected as candidate paths for further processing; the path containing child node 4 is eliminated and no longer participates in the subsequent search. Subsequently, the upper confidence scores of child nodes 2 and 3 are calculated respectively, and their respective upper confidence scores are compared with the preset upper confidence scores.

[0152] If the upper confidence bound score of child node 2 is less than or equal to the preset upper confidence bound score, the search for the path containing child node 2 terminates and does not continue downward; child node 3 is then evaluated. If the upper confidence bound scores of both child nodes 2 and 3 are greater than the preset upper confidence bound score, the path with the higher upper confidence bound score is selected to continue the search. If the upper confidence bound score of child node 2 is greater than the preset upper confidence bound score, and the upper confidence bound score of child node 3 is lower than the upper confidence bound score of child node 2, the search continues downward along child node 2. That is, in the direct child nodes of child node 2 (i.e., child nodes 5, 6, and 7), the similarity between the current joint embedding vector and the historical task information stored in child nodes 5, 6, and 7 is calculated. If the similarity between child nodes 5 and 6 is higher than the preset similarity threshold, and the similarity between child node 7 is lower than the preset similarity threshold, then child nodes 5 and 6 are considered as candidate child nodes at the current level, and child node 7 is eliminated. Subsequently, the upper confidence bound scores of child nodes 5 and 6 are calculated respectively. The child node with the higher upper confidence bound score is selected to continue searching downwards until a leaf node is reached, thus determining the complete path that meets the matching requirements. The task execution device reads the stored execution behavior text from this path that meets the matching requirements and executes it.

[0153] If neither the path containing child node 2 nor the path containing child node 3 meets the matching requirements, then it is determined that there is no path that meets the matching requirements. In this case, a new child node is created under the root node 1, and the access count of the new child node is initialized to 0.

[0154] In this embodiment, the implementation method and technical effect of steps 301-309 are similar to those of the corresponding solutions in the above embodiments, and will not be repeated here.

[0155] Figure 5 This is a schematic diagram of the structure of a task execution device provided in an embodiment of this application, as shown below. Figure 5 As shown, the task execution device provided in this embodiment includes: a first generation module 51, an execution module 52, a second generation module 53, a determination module 54, an acquisition module 55, and a reflection module 56.

[0156] The system comprises the following modules: a first generation module 51, which, in response to receiving a task execution instruction from a user, generates an execution behavior based on the task execution instruction using a tool invocation module; an execution module 52, which executes the execution behavior using a pre-trained model to generate result data after the execution behavior; a second generation module 53, which generates a joint embedding vector based on the result data and the state information of the execution device before and after execution; a determination module 54, which performs a multi-path search in a Monte Carlo tree based on the joint embedding vector using a preset multi-path nearest neighbor algorithm to determine whether there is a path that meets the matching requirements; an acquisition module 55, which, in response to the existence of a path that meets the matching requirements, acquires the reflective execution behavior based on the path that meets the matching requirements; and a reflection module 56, which executes the reflective execution behavior using a pre-trained model.

[0157] The task execution device provided in this embodiment can execute... Figure 2 The implementation principles and technical effects of the methods shown are similar, and will not be repeated here.

[0158] Optionally, in the second generation module 53, the result data is text type, and the pre- and post-execution state information of the execution device is image type; the pre- and post-execution state information of the execution device includes the pre-execution state information and the post-execution state information of the execution device. Accordingly, when generating the joint embedding vector based on the result data and the pre- and post-execution state information of the execution device, the module specifically performs the following: uses a preset multimodal encoder to encode the result data, the pre-execution state information, and the post-execution state information of the execution device respectively to obtain first encoded data, second encoded data, and third encoded data; and performs fusion processing on the first encoded data, second encoded data, and third encoded data to obtain the joint embedding vector.

[0159] Optionally, module 54 is defined as follows: the Monte Carlo tree is a multimodal Monte Carlo tree; the Monte Carlo tree includes multiple task subtrees, and each task subtree includes multiple child nodes; the preset multi-path nearest neighbor algorithm is a confidence upper bound algorithm; when using the preset multi-path nearest neighbor algorithm to perform multi-path search in the Monte Carlo tree based on the joint embedding vector to determine whether there is a path that meets the matching requirements, it is specifically used for: searching multiple candidate paths in the task subtree whose similarity to the joint embedding vector is greater than a preset similarity threshold; for any candidate path, calculating the matching score of the child node based on the historical access count of the child node in the candidate path; using the confidence upper bound algorithm to calculate the confidence upper bound score of the child node based on the matching score and the historical access count of the child node; and performing multi-path search based on the confidence upper bound score of the child node in the candidate path to determine whether there is a path that meets the matching requirements.

[0160] Optionally, the acquisition module 55, when acquiring the reflective execution behavior based on the path that meets the matching requirements, is specifically used to: acquire the corresponding reflective execution sub-behavior from the child nodes of the path that meets the matching requirements; and merge the reflective execution sub-behaviors to obtain the reflective execution behavior.

[0161] Optionally, the reflection module 56, when performing reflection execution behavior using a pre-trained model, is specifically used to: in response to the existence of a path that meets the matching requirements, send the reflection execution behavior corresponding to the path that meets the matching requirements to the tool invocation module, so that when the tool invocation module receives a task execution instruction that is semantically consistent with the task execution instruction, it sends the reflection execution behavior to the pre-trained model and executes the reflection execution behavior.

[0162] The technical effects of the task execution device provided in this embodiment are similar to those of the corresponding method solutions in the above embodiments, and will not be repeated here.

[0163] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application, as shown below. Figure 6 As shown, the electronic device 60 provided in this embodiment includes a processor 61 and a memory 62 that is communicatively connected to the processor.

[0164] The memory 62 stores computer-executable instructions; the processor 61 executes the computer-executable instructions stored in the memory 62 to implement the task execution method provided in any of the above embodiments. Related descriptions can be found in the accompanying drawings, which describe the steps and their corresponding effects; further elaboration is not provided here.

[0165] The program may include program code, which includes computer-executable instructions. Memory 62 may include high-speed RAM, and may also include non-volatile memory, such as at least one disk storage device.

[0166] In this embodiment, the memory 62 and the processor 61 are connected via a bus. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 6 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0167] This application also provides a computer-readable storage medium storing computer-executable instructions. When executed by a processor, these instructions are used to implement the task execution method provided in any of the above embodiments. For example, the computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, or optical data storage device.

[0168] This application also provides a computer program product, including a computer program, which, when executed by a processor, implements the task execution method provided in any of the above embodiments.

[0169] This application also provides a chip, which includes at least one processor and an interface circuit. The interface circuit and the at least one processor are connected, and the processor executes the above-described method by running program instructions.

[0170] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0171] It should be further noted that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0172] It should be understood that the above-described device embodiments are merely illustrative, and the device of this application can also be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units, modules, or components may be combined, or integrated into another system, or some features may be ignored or not executed.

[0173] Furthermore, unless otherwise specified, the functional units / modules in the various embodiments of this application can be integrated into one unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated together. The integrated units / modules described above can be implemented in hardware or as software program modules.

[0174] When integrated units / modules are implemented in hardware, the hardware can be digital circuits, analog circuits, etc. The physical implementation of the hardware structure includes, but is not limited to, transistors, memristors, etc. Unless otherwise specified, the processor can be any suitable hardware processor, such as a CPU, GPU, FPGA, DSP, and ASIC, etc. Unless otherwise specified, the storage unit can be any suitable magnetic or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), etc.

[0175] If the integrated unit / module is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0176] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0177] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0178] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A task execution method, characterized in that, include: In response to receiving a task execution instruction from the user, the tool invocation module generates an execution behavior based on the task execution instruction; The execution behavior is performed using a pre-trained model to generate result data after the execution behavior, and a joint embedding vector is generated based on the result data and the state information of the execution device before and after execution. A preset multi-path nearest neighbor algorithm is used to perform multi-path search in the Monte Carlo tree based on the joint embedding vector to determine whether there is a path that meets the matching requirements; In response to the existence of a path that meets the matching requirements, a reflective execution behavior is obtained based on the path that meets the matching requirements, and the reflective execution behavior is executed using the pre-trained model.

2. The method according to claim 1, characterized in that, The result data is text type, and the before-and-after state information of the execution device is image type; the before-and-after state information of the execution device includes the state information of the execution device before execution and the state information of the execution device after execution. The step of generating a joint embedding vector based on the result data and the before-and-after state information of the execution device includes: The result data, the pre-execution state information, and the post-execution state information are encoded using a preset multimodal encoder to obtain the first encoded data, the second encoded data, and the third encoded data respectively. The first encoded data, the second encoded data, and the third encoded data are fused together to obtain a joint embedding vector.

3. The method according to claim 2, characterized in that, The Monte Carlo tree is a multimodal Monte Carlo tree; the Monte Carlo tree includes multiple task subtrees, and each task subtree includes multiple child nodes; the preset multi-path nearest neighbor algorithm is a confidence upper bound algorithm; The step of employing a preset multi-path nearest neighbor algorithm to perform multi-path search in the Monte Carlo tree based on the joint embedding vector to determine whether a path meeting the matching requirements exists includes: Search within the task subtree for multiple candidate paths whose similarity to the joint embedding vector is greater than a preset similarity threshold; For any candidate path, calculate the matching score of the child node based on the historical access count of the child node in the candidate path; The confidence upper bound algorithm is used to calculate the confidence upper bound score of the child node based on the matching score of the child node and the number of historical visits. A multi-path search is performed based on the upper confidence bound scores of the child nodes in the candidate paths to determine whether there exists a path that meets the matching requirements.

4. The method according to claim 3, characterized in that, The step of performing a multi-path search based on the upper confidence bound scores of the child nodes in the candidate paths to determine whether a path exists that meets the matching requirements includes: If the upper confidence score of any child node in the candidate path is greater than the preset upper confidence score, then it is determined that there is a path that meets the matching requirements. If any child node in any candidate path has a confidence upper bound score that is less than or equal to the preset confidence upper bound score, it is determined that there is no path that meets the matching requirements. After determining that no path meets the matching requirements, the method further includes: Create a new child node in the task subtree where the candidate path is located, and update the access count of the child node. The new child node is a sibling node of the target node, and the target node is a child node whose confidence upper bound score is less than or equal to a preset confidence upper bound score.

5. The method according to claim 3, characterized in that, After executing the reflective execution behavior using the pre-trained model, the method further includes: Update the access count of child nodes in the path that meets the matching requirements.

6. The method according to claim 1, characterized in that, Also includes: In response to the existence of a path that meets the matching requirements, the reflective execution behavior corresponding to the path that meets the matching requirements is sent to the tool invocation module, so that when the tool invocation module receives a task execution instruction that is semantically consistent with the task execution instruction, it sends the reflective execution behavior to the pre-trained model and executes the reflective execution behavior. In response to the absence of a path that meets the matching requirements, the sub-paths in the path that does not meet the matching requirements are sent to the tool invocation module to invoke the pre-trained model to generate reflective execution behavior. The sub-paths include target sub-nodes in the path that does not meet the matching requirements, and the target sub-nodes are sub-nodes whose confidence upper bound scores are greater than the preset confidence upper bound scores.

7. A task execution device, characterized in that, include: The first generation module is used to respond to a task execution instruction input by the user and generate an execution behavior based on the task execution instruction using a tool invocation module; An execution module is used to execute the execution behavior using a pre-trained model to generate result data after the execution behavior; The second generation module is used to generate a joint embedding vector based on the result data and the before-and-after state information of the execution device. The determination module is used to perform multi-path search in the Monte Carlo tree based on the joint embedding vector using a preset multi-path nearest neighbor algorithm to determine whether there is a path that meets the matching requirements; The acquisition module is used to acquire the reflection execution behavior based on the path that meets the matching requirements if a path that meets the matching requirements exists. The reflection module is used to execute the reflection execution behavior using the pre-trained model.

8. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 7.

10. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1 to 7.

Images