Long-term owned task planning method and device based on large model, and robot

Abstract

The embodiment of the invention discloses a long-term ownership task planning method and device based on a large model, a robot and a storage medium. The method comprises the following steps: acquiring working environment information of a robot through multi-modal sensing equipment, and constructing a semantic scene graph; a natural language instruction is received, a scene graph is combined to input a task planning large language model to generate an initial task decision tree, nodes represent atomic actions, and branches correspond to different execution results; starting from the root node, matching branches based on node execution results in sequence and executing corresponding atomic actions; for any current tail end node in the decision-making tree, calling the large language model in parallel to infer different execution results and corresponding subsequent actions, and dynamically updating and expanding the decision-making tree; and after the execution of the current node is completed, matching the actual result of the current node with the corresponding branch in the decision tree: if the matching is successful, determining the next to-be-executed sub-node according to the matched branch, and pruning the decision tree to reserve the sub-tree corresponding to the branch.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robot control and artificial intelligence, in particular to a long-term embodied task planning method and device based on a large model, a robot and a storage medium. BACKGROUND

[0002] Long-term embodied task planning is the key to the autonomous completion of complex instructions by robots. The existing technology mainly uses a large language model (LLM) to decompose natural language instructions into action sequences. The specific process is as follows: the robot first perceives the environment, and then the LLM generates an action based on the current state. After the robot executes the action, the result is fed back to the LLM, which generates the next action, and the cycle is repeated in series. However, the LLM needs time to generate an action each time, which causes the robot to have to stop moving while thinking, resulting in overall execution lag and unsmoothness, and poor real-time performance. The model only plans the current step when triggered, and once the action fails (such as a failed grasp), the robot must stop and wait for the model to re-plan to respond to the new situation. This passive response mode is slow to respond to unexpected situations and has low recovery efficiency.

[0003] Therefore, how to design a long-term embodied task planning method that can efficiently decouple model reasoning and action execution and has the ability to prospectively plan for execution uncertainties has become a key technical problem to be solved to enable intelligent robots to reliably complete complex real-world tasks. SUMMARY

[0004] The purpose of the embodiments of the present application is to provide a long-term embodied task planning method and device based on a large model, a robot and a storage medium, to solve the problems of robot execution pause caused by serial reasoning of the large model and lack of prospective planning ability for execution uncertainties in the prior art.

[0005] To achieve the above-mentioned purpose, the first aspect of the present application provides a long-term embodied task planning method based on a large model, comprising: obtaining multi-modal perception data of a robot working environment, and constructing a semantic scene graph describing objects and their three-dimensional spatial relationships in the environment based on the multi-modal perception data; receiving a natural language task instruction, inputting the semantic scene graph and the task instruction into a task planning large language model for reasoning to generate a task decision tree of an initial state corresponding to the natural language task instruction, the task decision tree of the initial state at least including a root node, a node in the task decision tree representing a corresponding atomic action task, and child nodes of any node different branches representing subsequent atomic action tasks corresponding to different execution results of the atomic action task of the any node; based on the execution result of any node, sequentially matching branches from the root node and executing nodes in the task decision tree of the initial state; For any end node in the task decision tree of the current state, the task planning large language model is called in parallel to infer different execution results of the atomic action task corresponding to the any end node and the subsequent atomic action task under different execution results, to update the child node corresponding to the any end node in the task decision tree; In the case where the current node is executed and completed, the actual execution result of the current node is matched with the branch corresponding to the current node in the decision tree; In the case where the matching is successful, the subsequent to-be-executed child node is determined according to the matched branch, and the task decision tree is pruned to retain the sub-tree corresponding to the subsequent to-be-executed child node.

[0006] In the embodiment of the present application, a semantic scene graph describing objects and their three-dimensional spatial relationships in the environment is constructed based on multi-modal perception data, including: extracting key image frames from multi-modal perception data; identifying the class label of the object in the key image frame using a visual language model; determining the two-dimensional position and pixel mask of each object in the image based on an open vocabulary object detection and segmentation model according to the class label; mapping the two-dimensional position of each object to a three-dimensional coordinate system and spatially aligning with the point cloud data to determine the three-dimensional spatial position and bounding box of each object; generating a structured semantic scene graph according to the class, attribute and three-dimensional spatial position relationship of each object.

[0007] In the embodiment of the present application, the semantic scene graph and the task instruction are input into the task planning large language model for inference to generate a task decision tree of the initial state corresponding to the natural language task instruction, including: converting the semantic scene graph into a natural language description, and inputting the task instruction together into the task planning large language model as prompt information; guiding the task planning large language model to perform multi-step inference and decomposition on the task instruction according to the semantic scene graph; mapping each step obtained by the model inference and decomposition to the corresponding atomic action task in the predefined robot atomic action library; taking the first atomic action task as the root node, generating at least one branch and its corresponding child node atomic action task for the root node based on the preset execution result hypothesis, thereby constructing the task decision tree of the initial state; wherein the atomic action is the smallest and deterministic action unit that can be executed by the robot.

[0008] In this embodiment, the task planning large language model is invoked in parallel to infer different execution results of the atomic action task corresponding to any end node and subsequent atomic action tasks under different execution results, so as to update the child nodes corresponding to any end node in the task decision tree. This includes: for each end node in the task decision tree in the current state that has not yet undergone subsequent branch inference, an independent task planning large language model inference process is initiated; wherein, the task planning large language model inference process corresponding to any end node is used to perform: obtaining the input natural language task instruction, a node sequence consisting of executed nodes and a task chain of any end node, predicting the subsequent atomic action tasks of any end node in the task chain under at least two preset execution results; adding the subsequent atomic action tasks under each preset execution result as child nodes of any end task node to the decision tree to form a new branch.

[0009] In this embodiment, the task planning large language model is invoked in parallel to infer different execution results of the atomic action tasks corresponding to any terminal node and subsequent atomic action tasks under different execution results, so as to update the child nodes corresponding to any terminal node in the task decision tree. This includes: for each terminal node in the task decision tree in the current state that has not yet undergone subsequent branch inference, starting a first task planning large language model inference process and a second task planning large language model inference process; wherein, the first task planning large language model inference process is used to perform: obtaining the input natural language task instruction, a node sequence containing executed nodes and a task chain of any terminal node; predicting any terminal node in the task chain. The first task is to determine the subsequent atomic action task following a successful execution result; the predicted subsequent atomic action task is added as a child node of any terminal task node to the decision tree, forming a new branch; the second task is to plan the large language model inference process to execute: obtain the input natural language task instructions, a node sequence consisting of executed nodes and a task chain of any terminal node; predict at least one preset execution result of any terminal node in the task chain that fails, and the subsequent atomic action task under each preset execution result; the predicted subsequent atomic action task under each preset execution result is added as a child node of any terminal task node to the decision tree, forming a new branch.

[0010] In this embodiment of the application, pruning the task decision tree to retain the subtree corresponding to the subsequent child node to be executed includes: deleting all unmatched branches and subtrees containing branches in the decision tree, retaining the subtree corresponding to the subsequent child node to be executed, and updating the task decision tree; terminating the task planning large language model inference process associated with all nodes in the deleted subtree.

[0011] In this embodiment of the application, the method further includes: in the case of a matching failure, re-invoking the task planning large language model based on the actual execution result to generate a new branch and corresponding child node corresponding to the current node.

[0012] In this embodiment, in the event of a matching failure, the large language model is invoked again based on the actual execution result to generate a new branch and corresponding child node for the current node. This includes: inputting the natural language task instruction, the actual execution result, and the node sequence consisting of executed nodes in the task decision tree into the newly started task failure planning large language model inference process; the task failure planning large language model inference process analyzes the execution result and directly generates a new branch and corresponding child node for the current node; wherein, when a clear failure reason is identified, the task failure planning large language model is used to determine the failure reason and the subsequent corrective action task for the failure reason as the new branch and corresponding child node for the current node; when no clear reason is identified, the task containing the instruction requesting manual intervention is determined as the child node of the new branch for the current node.

[0013] In this embodiment of the application, the method further includes: during task execution, if the length of the historical context information exceeds a set threshold, calling the summary big language model to compress and summarize the historical information, and using the summary information for subsequent planning reasoning of the task planning big language model.

[0014] In this embodiment of the application, the summary big language model is invoked to compress and summarize historical information, including: triggering summary generation when the length of the context input to the task planning big language model exceeds a preset threshold; the summary big language model summarizes the historical dialogue and task status to generate summary text containing natural language task instructions, a node sequence consisting of executed nodes in the task decision tree, remaining targets, and the current execution status; and replacing part or all of the original historical messages with the summary text as context input for subsequent reasoning of the task planning big language model.

[0015] In this embodiment of the application, matching the actual execution result of the current node with the corresponding branch in the decision tree includes: after executing the atomic action corresponding to the current node, determining whether the atomic action achieves the expected execution result corresponding to each branch of the current node through sensor feedback and multimodal model analysis; if the expected execution result of any branch is achieved, determining that the actual execution result of the current node matches that branch successfully; otherwise, determining that the actual execution result of the current node fails to match.

[0016] A second aspect of this application provides a long-term embodied task planning apparatus based on a large model, the apparatus comprising: The memory is configured to store instructions; The processor is configured to retrieve instructions from memory and, when executing instructions, implement a long-term embodied task planning method based on a large model, which is capable of any of the above.

[0017] A third aspect of this application provides a robot that includes the aforementioned long-term embodied task planning device based on a large model.

[0018] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described long-term embodied task planning methods based on a large model.

[0019] The technical solution of this application asynchronously parallelizes robot action execution and large-model inference. While executing the current atomic action, it simultaneously invokes the large model to perform forward-looking inference of subsequent actions under multiple possible outcomes and constructs a decision tree. After execution, it matches the decision tree branches based on the actual results and dynamically prunes them. This method transforms the execution mode of robot long-term task planning from the traditional passive, sequential, and unforeseen planning paradigm to a novel mode that is proactively parallel, possesses multi-branch forward-looking capabilities, and dynamic decision optimization. This solution can fundamentally reduce robot action waiting time and improve response speed and robustness to execution uncertainties.

[0020] Other features and advantages of the embodiments of this application will be described in detail in the following detailed description section. Attached Figure Description

[0021] The accompanying drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the following detailed description to explain the embodiments of this application, but do not constitute a limitation on the embodiments of this application. In the drawings: Figure 1 The illustration shows a flowchart of a long-term embodied task planning method based on a large model according to an embodiment of this application; Figure 2 The illustration shows a schematic diagram of the interaction flow of the core modules of a long-term embodied task planning system based on a large model according to an embodiment of this application; Figure 3 This illustration schematically shows a scene graph generation process of an embodied perception model according to an embodiment of the present application; Figure 4 The diagram illustrates the internal structure of a computer device according to an embodiment of this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only for illustration and explanation of the embodiments of this application and are not intended to limit the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0023] It should be noted that the acquisition, transmission, storage, use, and processing of data in the technical solution of this application all comply with relevant laws and regulations. In the embodiments of this application, certain existing industry solutions such as software, components, and models may be mentioned. These should be considered exemplary, intended only to illustrate the feasibility of implementing the technical solution of this application, and do not imply that the applicant has already used or necessarily used such solutions.

[0024] Furthermore, if the embodiments of this application involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, features defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application.

[0025] Figure 1 The illustration shows a flowchart of a long-term embodied task planning method based on a large model according to an embodiment of this application. Figure 1 As shown, it includes: Step 101: Obtain multimodal perception data of the robot's working environment, and construct a semantic scene graph describing objects in the environment and their three-dimensional spatial relationships based on the multimodal perception data.

[0026] In one embodiment, the robot constructs a semantic scene graph describing objects in the environment and their three-dimensional spatial relationships based on multimodal perception data, including: extracting key image frames from the multimodal perception data; identifying the category labels of objects in the key image frames using a visual language model; determining the two-dimensional position and pixel mask of each object in the image based on the category labels using an open vocabulary object detection and segmentation model; mapping the two-dimensional position of each object to a three-dimensional coordinate system and spatially aligning it with point cloud data to determine the three-dimensional spatial position and bounding box of each object; and generating a structured semantic scene graph based on the category, attributes, and three-dimensional spatial position relationships of each object.

[0027] In one embodiment, the robot acquires a continuous RGB-D (color-depth) video stream as raw perceptual data using its onboard camera. However, continuous sampling generates a large amount of redundant information and suffers from motion blur and other problems. To improve processing efficiency, this application proposes a real-time keyframe extraction method: when the camera's translation or rotation change compared to the last selected keyframe exceeds a preset threshold, or when significant dynamic changes such as object movement are detected in a static scene, the current frame is considered a keyframe candidate. Subsequently, each candidate frame undergoes image quality evaluation, and only frames that simultaneously meet the motion, dynamic change, and quality conditions are ultimately selected as keyframes. Key image frames refer to discrete image frames selected from the continuous video stream that are rich in information and of high quality, used to represent the visual state of the scene at a specific moment.

[0028] In this embodiment, the robot inputs selected key image frames into a pre-trained Visual Language Model (VLM). The VLM receives images and automatically generates natural language descriptions or object label lists describing the image content without requiring specific training for particular object categories. The robot uses the object category labels identified by the VLM as text prompts, inputting them into an open-vocabulary object detection model. Based on the text prompts, this model locates and outlines the two-dimensional bounding boxes of all relevant objects in the key image frames. Open-vocabulary object detection refers to an object detection technique that can detect objects of any category based on input natural language descriptions without pre-defining a fixed category list. Next, the detected bounding box regions are input into a segmentation model to generate pixel-level accurate object segmentation masks. A pixel mask is a binary matrix of the same size as the image, where pixels with a value of 1 belong to the target object, and pixels with a value of 0 belong to the background.

[0029] In this embodiment, the robot, based on the depth information from RGB-D keyframes, projects the obtained two-dimensional pixel mask of each object into the robot coordinate system or world coordinate system through camera intrinsic parameters and pose transformation. Simultaneously, it utilizes point cloud data synchronously acquired by its onboard LiDAR for assistance and fusion. Point cloud data refers to a dataset of numerous spatial points obtained through LiDAR scanning, used to accurately describe the three-dimensional geometry of the environment. Each point contains three-dimensional coordinates and sometimes information such as reflection intensity. By associating and clustering the object masks segmented from the image with the three-dimensional points in the point cloud, the precise position of each object in three-dimensional space can be calculated, and its three-dimensional bounding box can be fitted. A three-dimensional bounding box is a cuboid describing the area occupied by an object in three-dimensional space, typically defined by its center point coordinates, dimensions (length, width, and height), and orientation.

[0030] In this embodiment, the robot utilizes the acquired categories, attributes, and 3D spatial positions and bounding boxes of all objects. Then, by analyzing the relative spatial relationships between objects and performing relational reasoning using Visual Modeling (VLM), a graph structure is ultimately constructed. In this graph structure, nodes represent objects in the scene, and their attributes include their category, geometric properties, etc.; edges represent relationships between objects, and their attributes describe specific spatial or semantic relationships. This graph structure is a semantic scene graph, a machine-readable, structured representation of the environment rich in semantic and spatial information.

[0031] Step 102: Receive natural language task instructions, input the semantic scene graph and task instructions into the task planning large language model for reasoning, so as to generate the task decision tree of the initial state corresponding to the natural language task instructions. The task decision tree of the initial state includes at least a root node. The nodes in the task decision tree represent the corresponding atomic action tasks. The child nodes of different branches of any node represent the subsequent atomic action tasks corresponding to different execution results of the atomic action tasks corresponding to that node.

[0032] A decision tree is a tree-like data structure used to represent all possible execution paths during task planning. A node in a decision tree represents an atomic action. An edge represents a logical connection from a specific execution result of an atomic action to its subsequent atomic actions. The root node typically represents the first atomic action in the initial task sequence. The path from the root node to any leaf node represents a complete task execution sequence that may occur under specific assumptions.

[0033] In one embodiment, the robot inputs a semantic scene graph and task instructions into a task planning large language model for reasoning, forming a task decision tree of the initial state corresponding to the natural language task instructions. This includes: converting the semantic scene graph into a natural language description, which, together with the task instructions, constitutes a prompt message input to the task planning large language model; guiding the task planning large language model to perform multi-step reasoning and decomposition of the task instructions based on the semantic scene graph; mapping each step obtained from the model's reasoning decomposition to a corresponding atomic action task in a predefined robot atomic action library (which must contain at least one atomic action task and form a root node); in the case of containing at least two atomic action tasks with an execution order, using the first atomic action task as the root node, and based on a preset execution result assumption, generating at least one branch and its corresponding child node atomic action tasks for the root node reasoning, thereby constructing the task decision tree of the initial state; wherein, an atomic action is the smallest, deterministic action unit that the robot can execute.

[0034] In one embodiment, the robot first converts the semantic scene graph into a structured natural language description. This description includes the categories, attributes, and spatial relationships of objects in the environment, and together with the received natural language task instructions, it constructs input prompts. Next, these prompts are input into a task planning language model, guiding the model to progressively decompose and logically reason about the task instructions based on scene understanding, outputting a step-by-step natural language plan. The robot then maps each step of this natural language plan to a corresponding atomic action in a predefined robot atomic action library using predefined semantic mapping rules or a lightweight parser. The atomic action library is a finite set of all executable basic operations, such as movement, grasping, and placement, predefined and implemented for the robot; an atomic action is the smallest, deterministic unit of action that the robot can execute. Finally, the robot uses the first atomic action as the root node and, based on preset execution result assumptions, infers and generates at least one branch and its corresponding child node atomic actions for the root node, thereby constructing the task decision tree of the initial state.

[0035] Step 103: Based on the execution result of any node, start from the root node, sequentially match branches, and execute the nodes in the task decision tree of the initial state.

[0036] Step 104: For any end node in the task decision tree of the current state, the task planning big language model is called in parallel to reason about the different execution results of the atomic action tasks corresponding to the end node and the subsequent atomic action tasks under different execution results, so as to update the child nodes in the task decision tree corresponding to the end node.

[0037] In one embodiment, the robot invokes a task planning large language model in parallel to infer different execution results of the atomic action task corresponding to any end node and subsequent atomic action tasks under different execution results, so as to update the child nodes corresponding to any end node in the task decision tree. This includes: for each end node in the task decision tree in the current state that has not yet undergone subsequent branch inference, an independent task planning large language model inference process is initiated; wherein, the task planning large language model inference process corresponding to any end node is used to perform: obtaining the input natural language task instruction, a node sequence consisting of executed nodes and a task chain of any end node, predicting the subsequent atomic action tasks of any end node in the task chain under at least two preset execution results; adding the subsequent atomic action tasks under each preset execution result as child nodes of any end task node to the decision tree to form a new branch.

[0038] In one embodiment, the robot first initiates an independent task planning large language model inference process for each current end task node in the decision tree. Then, the task chain containing historically executed atomic action sequences and the currently executed atomic action task—the complete path from the initial task to the end node—is provided as input context to the corresponding inference process. Based on this task chain, each inference process predicts the subsequent atomic action sequence for the current atomic action task under at least two preset execution outcomes, such as success, failure reason 1, failure reason 2, etc. Finally, the robot dynamically adds the starting action of the subsequent atomic action task or action sequence predicted by each inference process for different execution outcomes as a new child node of the end task node to the decision tree, thereby expanding the branching structure of the decision tree.

[0039] In one embodiment, at least two preset execution results are common result types derived from historical task execution data, such as successful grasping, grasping failure (object slides), and grasping failure (object not detected). The robot embeds these result types as part of the prompt information into the reasoning process of the large language model, guiding the model to perform conditional planning for each specific result.

[0040] In one embodiment, the system invokes the task planning large language model in parallel to infer the different execution results of the atomic action tasks corresponding to any terminal node and the subsequent atomic action tasks under different execution results, so as to update the child nodes in the task decision tree corresponding to any terminal node. This includes: for each terminal node in the task decision tree in the current state that has not yet undergone subsequent branch inference, starting a first task planning large language model inference process and a second task planning large language model inference process; wherein, the first task planning large language model inference process is used to perform: obtaining the input natural language task instructions, the node sequence consisting of executed nodes, and the task chain of any terminal node; predicting any terminal node in the task chain. The first task is to determine the subsequent atomic action task following a successful execution result; the predicted subsequent atomic action task is added as a child node of any terminal task node to the decision tree, forming a new branch; the second task is to plan the large language model inference process to execute: obtain the input natural language task instructions, a node sequence consisting of executed nodes and a task chain of any terminal node; predict at least one preset execution result of any terminal node in the task chain that fails, and the subsequent atomic action task under each preset execution result; the predicted subsequent atomic action task under each preset execution result is added as a child node of any terminal task node to the decision tree, forming a new branch.

[0041] In one embodiment, for each terminal node in the decision tree that has not yet undergone subsequent branch inference, the system simultaneously initiates a first and a second task planning large language model inference process. The first process is dedicated to inferring the optimal subsequent task after the successful execution of the terminal node's task. Its input includes the current task chain, the sequence of historical execution nodes, the natural language instructions for the terminal node, and scenario information. Its output is the subsequent atomic action to be added as a child node of the successful branch. The second process specifically handles at least one preset failure scenario that may occur at the terminal node. Its input includes the task chain and predefined typical failure categories such as grasp-slip. The model infers and generates corresponding restorative or alternative subsequent atomic actions for each type of failure, adding each result as an independent child node of the failure branch to the decision tree. The two processes execute in parallel based on the same context, with their respective outputs dynamically expanding the branch structure of the decision tree as parallel child nodes.

[0042] It should be noted that, in this embodiment, the task execution process based on the current task decision tree (step 103) and the dynamic update process of the task decision tree (such as expansion or pruning, including steps 104 and 106) are executed independently and in parallel in time. This means that while the robot executes atomic actions according to the existing decision tree structure, the system runs the inference process independently and asynchronously in the background to prospectively expand the decision tree. The two do not block each other and together constitute the core concurrency mechanism for achieving efficient and smooth planning in this method.

[0043] Step 105: If the current node has completed execution, match the actual execution result of the current node with the corresponding branch of the current node in the decision tree.

[0044] In one embodiment, the actual execution result of the current node is matched with the branch in the decision tree corresponding to the current node. This includes, after executing the atomic action corresponding to the current node, determining whether the atomic action achieves the expected execution result corresponding to each branch of the current node through sensor feedback and multimodal model analysis. If the expected execution result of the task corresponding to any branch is achieved, it is determined that the actual execution result of the current node matches that branch successfully; otherwise, it is determined that the actual execution result of the current node fails to match.

[0045] In one embodiment, the process by which the robot determines whether an atomic action has achieved its intended sub-goal through sensor feedback and multimodal model analysis includes: after the atomic action is completed, the robot immediately acquires environmental state feedback data through its onboard multimodal sensors, and then inputs this feedback data into a large multimodal model for comprehensive reasoning and analysis. This large multimodal model, based on the task state before the action, the executed atomic action instructions, and the real-time sensor data after execution, determines whether the atomic action successfully achieved its preset sub-goal. The determination result is formalized as a structured actual execution result, such as success or with a label indicating a specific reason for failure. This actual execution result is then directly used to match the corresponding branch in the decision tree.

[0046] Step 106: If a match is successful, determine the child node to be executed next based on the matched branch, and prune the task decision tree to retain the subtree corresponding to the child node to be executed next.

[0047] In one embodiment, pruning the task decision tree to preserve the subtree corresponding to the subsequent child node to be executed includes: deleting all unmatched branches and subtrees containing those branches from the decision tree, preserving the subtree corresponding to the subsequent child node to be executed, and updating the task decision tree; terminating the task planning large language model inference process associated with all nodes in the deleted subtree. By performing necessary pruning on the decision tree update process based on the actual execution results of the task, the pre-generated task plan is prevented from being meaningless for the actual execution process, avoiding waste of computational resources, thereby achieving a balance between improving the robot's task execution efficiency and saving system computational resources.

[0048] In one embodiment, if a match fails, the task planning language model is invoked again based on the actual execution result to generate a new branch and corresponding child node for the current node.

[0049] In this embodiment, in the event of a matching failure, the large language model is re-invoked based on the actual execution result to generate a new branch and corresponding child node for the current node. This includes: inputting the natural language task instruction, the actual execution result, and the node sequence consisting of executed nodes in the task decision tree into the newly started task failure planning large language model inference process; the task failure planning large language model inference process analyzes the execution result and directly generates a new branch and corresponding child node for the current node; wherein, when a clear failure reason is identified, the task failure planning large language model is used to determine the failure reason and the generated subsequent corrective action task for the failure reason as the new branch and corresponding child node for the current node; when no clear reason is identified, the task containing the instruction requesting manual intervention is determined as the child node of the new branch for the current node.

[0050] In one embodiment, after the current atomic action is completed, the robot matches the actual execution result of the atomic action with the branch of the corresponding node in the decision tree, and determines the subsequent atomic action to be executed based on the matched branch.

[0051] Specifically, upon successful matching, the robot locates the corresponding atomic action node in the decision tree and then designates the atomic action represented by that node as the next atomic action to be executed. Next, the robot performs a pruning operation on the decision tree: deleting all unmatched branches and their entire subtrees, and simultaneously terminating the large language model inference process associated with these deleted branches to free up computational resources. Finally, only the matched branches and their subsequent subtree structures are retained as the basis for subsequent task planning decisions.

[0052] In the event of a matching failure—that is, when the actual execution result does not appear in the pre-defined branch of the decision tree—the robot inputs the actual execution result along with contextual information about the complete task history from the start of the task to the current moment into a newly initiated large language model inference process. This inference process analyzes the specific reasons for the current execution result and directly generates a new sequence of subsequent atomic actions. If a clear reason for the failure can be identified, the generated sequence includes targeted corrective actions; if the exact reason cannot be analyzed, the generated sequence includes instructions requesting human intervention, such as pausing and waiting for human review.

[0053] In one embodiment, the method further includes: during task execution, if the length of the historical context information exceeds a set threshold, calling the summary big language model to compress and summarize the historical information, and using the summary information for subsequent planning reasoning of the task planning big language model.

[0054] To address the aforementioned issues, in one embodiment, the robot invokes a summarizing big language model to compress and summarize historical information, including: triggering summary generation when the length of the context input to the big language model for task planning exceeds a preset threshold; having the summarizing big language model summarize historical dialogues and task states to generate summary text containing natural language task instructions, a node sequence consisting of executed nodes in the task decision tree, remaining objectives, and the current execution state; and replacing part or all of the original historical messages with the summary text as context input for subsequent reasoning by the task planning big language model.

[0055] In one embodiment, as the number of task steps increases and historical interaction information accumulates, the context length continues to grow. Due to the inherent limitations of large language models in processing long sequences, they are prone to forgetting initial task objectives or losing early key information, leading to deviations from the predetermined trajectory or errors in subsequent planning. To address this issue, the robot continuously monitors the length of the context input to the large language model during task execution. When this length exceeds a preset threshold, a summary generation process is automatically triggered. In this process, the robot inputs the currently accumulated complete historical context, including multi-turn human-computer dialogues, executed atomic action sequences, environmental state changes, and task execution results, into the large language model and instructs the model to extract and summarize the historical information. Based on this information, the large language model generates a structured summary text, which keyly extracts which sub-tasks have been completed, what the remaining task objectives are, what execution state the robot is currently in, and any anomalies or constraints that may need attention. Subsequently, the robot uses this generated summary text to replace the original lengthy, partially or entirely original historical messages, forming a new, concise context window. This new context containing the summary will be used in all subsequent inference requests related to task planning in the large language model, thereby significantly reducing the length of the model input and computational load while effectively maintaining long-term task consistency and target memory, and avoiding problems such as distraction, target forgetting or decreased instruction compliance caused by excessively long context.

[0056] This application transforms the traditional long-term task execution mode based on large-model serial reasoning and passive reactive planning into a forward-looking execution mechanism based on asynchronous parallel planning and dynamic decision tree maintenance. This fundamentally solves the problems of robot action execution lag caused by model inference delays, slow response to emergencies due to a lack of prediction of execution uncertainties, and instruction forgetting and goal deviation caused by long contexts. It eliminates the inherent contradiction in traditional solutions where execution must wait for planning, and planning is only triggered after failure. The robot, based on a multi-branch look-ahead mechanism and dynamic pruning, achieves decoupling and parallelization of planning and execution. This allows the robot to prepare contingency plans for multiple possible outcomes while executing the current step, and achieves efficient path selection through real-time matching and pruning after execution. Simultaneously, by integrating context compression summarization and memory management mechanisms, it can maintain goal consistency and instruction compliance in long-cycle tasks, significantly reducing the model computational burden and planning error risk caused by the increase in task steps. This provides robotic support for achieving smooth, robust, and efficient long-term autonomous robot task execution.

[0057] Figure 2 The illustration schematically depicts the interaction flow of the core modules of a long-term embodied task planning system based on a large model, according to an embodiment of this application. For example... Figure 2As shown, the various modules of this application are described in detail below: (1) Multi-modal sensing: The robot uses a lidar on its body to collect 3D point cloud data of its working scene and a camera to collect RGB images of the working scene.

[0058] (2) Embodied perception model: This model utilizes data collected through multimodal perception and converts it into a scene graph. The specific process is as follows: Figure 3 As shown, it includes: Keyframe Detection: Robots acquire environmental information through continuous time-series data, such as RGB-D video frames. However, continuous sampling generates a large amount of repetitive data, and the quality of a single frame may be unstable due to motion blur. Real-time processing of each frame is both inefficient and often unnecessary. To improve processing efficiency, this application designs a real-time keyframe extraction method that filters out information-rich and high-quality frames from the observation sequence. When the camera's translation or rotation change compared to the last selected keyframe exceeds a threshold, or when significant dynamic changes, such as the movement of objects or people, are detected in a static or slowly moving scene, the frame is considered a keyframe candidate. Subsequently, each candidate frame undergoes image quality and content relevance evaluation: sharpness is determined using Laplacian variance to eliminate blurry frames, and the effectiveness of scene information in the frame is verified by combining object detection results. Only frames that simultaneously meet the conditions of motion, dynamic change, and quality are ultimately selected as keyframes.

[0059] Image label recognition: The Vision-Language Model (VLM) combines images with natural language descriptions, enabling it to automatically infer the categories of objects present in an image without special training. This method can not only identify single objects but also handle the coexistence of multiple objects in complex scenes, thereby achieving an understanding of the object labels and attributes contained in the image content.

[0060] Open-vocabulary object detection and segmentation: This application presents an object detection method combining visual and language models, capable of recognizing arbitrary categories not present during training, achieving zero-shot detection of new categories or custom objects. Grounding DINO combined with VLM-recognized item labels is used for object detection in images. Then, SAM (an image segmentation model) is used to accurately segment the identified targets and extract the object masks.

[0061] 3D geometric fusion: This integrates point cloud data and RGB-D results to achieve spatiotemporal semantic consistency across the entire scene. Specifically, the coordinates of objects in the RGB-D image are first mapped to the world coordinate system, enabling alignment of the point clouds corresponding to the objects in 3D reconstruction. The 3D bounding box along the Z-axis is then calculated. Finally, the mask, label, attributes, and other features of each object are marked in the 3D reconstructed image.

[0062] Scene graph generation: Based on VLM, reason about the 3D reconstructed image to identify the relative spatial relationships between objects, and then construct a scene graph.

[0063] (3) Memory management: Record and maintain historical information in dialogues or tasks to prevent the model from forgetting the previous text in long dialogues and improve the consistency and coherence of the answers.

[0064] (4) Memory compression: Memory compression can reduce the amount of historical information stored in the model while retaining core information, thereby reducing memory usage. It maintains the core value of information by extracting key information and discarding redundant content.

[0065] (5) Embodied planning model: Based on the environmental structure information provided by the scene diagram, the set of executable operations of atomic action instructions, and the goals and constraints of specific tasks, the high-level tasks are systematically decomposed into several operable sub-tasks or steps so that the subsequent planning and execution process can efficiently and accurately achieve the overall task goals.

[0066] (6) Atomic skills of embodied entities: Atomic actions of a robot refer to the smallest, indivisible, executable units of action. They are deterministic and repeatable, and form the basis for building complex behaviors and task sequences. By combining atomic actions, robots can accomplish higher-level task objectives.

[0067] (7) Task verification: Through comprehensive analysis of the force feedback signals of the multimodal large model and the robot gripper, the intelligent determination of the completion status of the sub-task is realized.

[0068] The specific process is as follows: First, the robot's LiDAR and depth camera collect raw data of the current working environment. Then, the embodied perception module processes the collected data to identify the categories, attributes, and relative spatial relationships of objects, thereby generating a structured scene map. Next, the embodied planning model understands the user's task based on the scene map, generates atomic actions to be executed in the ReAct mode based on the model, writes the atomic tasks into a buffer, and assumes that the current sub-task has been successfully completed, providing feedback to the model so that it can continue to generate subsequent sub-tasks. Simultaneously, the robot continuously retrieves atomic tasks from the buffer, calls the corresponding robot cerebellum control skills to execute the corresponding actions, and then the task verification module judges whether the sub-task is successful. If the result of the cerebellum task execution is consistent with the assumed result, the next action can be executed directly using the pre-generated plan. If the result of the cerebellum execution is inconsistent with the assumed result, the buffer is cleared, allowing the embodied planning model to regenerate the next plan based on the actual situation. By allowing the task planning of the embodied planning model and the cerebellum skills to be executed in parallel most of the time, the time required for the robot to complete the task is reduced, as is the waiting time for switching between different robot actions, making the robot's movements smoother.

[0069] This application also constructs a dynamic decision-making mechanism based on historical task data and real-time execution feedback. Specifically, it first builds a large-scale task planning model using historical failure information of each subtask. The input of this model is a completed task chain, and the output is the next subtask of all branches of the completed task chain. The completed task chain contains an array of executed subtasks arranged in execution order. The next subtask includes the next subtask of the last subtask in the completed task chain under different execution results. For example, the next subtask corresponding to the last subtask 'a' can include subtasks a1, a2, and a3, representing the next subtasks under the following conditions: successful execution, failure with reason 1, and failure with reason 2, respectively. Multiple of the above-mentioned large-scale task planning models can be launched in parallel to perform parallel reasoning for subsequent tasks on different completed task chains.

[0070] This application embodiment dynamically maintains the decision tree generated by the large model, specifically including the following process: During the execution of the current subtask, for all terminal task nodes in the decision tree that have not yet undergone backward reasoning, a new task planning large model process is created. This process can be a model that reasones for both success and failure simultaneously, or two independent models that handle success and failure separately, but the input and output formats must be consistent. The complete task chain corresponding to the terminal node is input into this process. The task chain includes historically completed subtasks, the currently executing subtask, and all subtasks from the current subtask to the terminal node. The large model process infers several possible next subtasks and adds them as child nodes to the decision tree, thereby expanding the depth and breadth of the decision tree. This forward reasoning process continues, and as long as the current subtask has not been completed, the decision tree will continue to grow backward, covering branches for all possible scenarios after the current task chain.

[0071] After the current subtask is completed, the robot obtains the actual execution result of that subtask. The robot then checks whether this result matches any sub-node task in the decision tree for the current subtask. If a matching sub-node task is found, the robot directly executes the subtask corresponding to that node. Simultaneously, the robot prunes the decision tree, deleting all unmatched branches and their associated large model inference processes, retaining only the matching branches and their subsequent subtrees as the basis for subsequent planning decisions.

[0072] If no matching sub-task is found, it indicates that the actual execution result has exceeded the range of the preset branch. At this point, the robot combines the task chain corresponding to the current sub-task with the actual execution result and inputs it into a newly launched task failure planning model, attempting to replan the next sub-task based on the actual anomaly. This process degenerates into a sequential "action-think-action" pattern. If the model still cannot analyze the cause of failure, the generated next sub-task is "stop and wait for manual inspection" to ensure the robot's safety when encountering unforeseen errors.

[0073] When performing long-term tasks, the robot engages in multi-turn dialogues with the embodied planning model. These dialogues are handled by a memory management module, which checks whether the number of input tokens exceeds a set threshold before feeding them into the model. If it does, the embodied model is used to compress historical messages, summarizing which subtasks have been completed, what tasks remain, and what the current action is. Here, a token is the smallest semantic unit after text segmentation. For example, when performing a multiple-screw picking task (picking 30 screws), the robot can compress historical messages to show that 20 screws have been picked, 10 remain, and the 21st screw is currently being picked. Finally, the summarized information replaces previous historical messages, compressing them to reduce the number of input tokens and prevent issues like target loss and decreased instruction compliance in extremely long context scenarios.

[0074] This application also provides a long-term embodied task planning apparatus based on a large model. The apparatus includes: a memory configured to store instructions; and a processor configured to retrieve instructions from the memory and, when executing the instructions, to implement the aforementioned long-term embodied task planning method based on a large model.

[0075] This application also provides a robot, which includes the above-described long-term embodied task planning device based on a large model.

[0076] This application also provides a machine-readable storage medium storing instructions for causing a machine to execute the above-described long-term embodied task planning method based on a large model.

[0077] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 4As shown, the computer device includes a processor A01, a network interface A02, memory (not shown), and a database (not shown) connected via a robot bus. The processor A01 provides computational and control capabilities. The memory includes internal memory A03 and a non-volatile storage medium A04. The non-volatile storage medium A04 stores an operating system B01, a computer program B02, and a database (not shown). The internal memory A03 provides an environment for the operation of the operating system B01 and the computer program B02 stored in the non-volatile storage medium A04. The database stores multimodal data, model parameters, historical task records, and operation logs required for long-term embodied task planning. The network interface A02 communicates with external terminals via a network connection. When the computer program B02 is executed by the processor A01, it implements a long-term embodied task planning method based on a large model.

[0078] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0079] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0080] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0081] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0082] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0083] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0084] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, like read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0085] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0086] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0087] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A long-term embodied task planning method based on a large model, characterized in that, The method includes: Acquire multimodal perception data of the robot's working environment, and construct a semantic scene graph describing objects in the environment and their three-dimensional spatial relationships based on the multimodal perception data; Receive natural language task instructions, input the semantic scene graph and the task instructions into the task planning large language model for reasoning, so as to generate the task decision tree of the initial state corresponding to the natural language task instructions. The task decision tree of the initial state includes at least a root node. The nodes in the task decision tree represent the corresponding atomic action tasks. The child nodes of different branches of any node represent the subsequent atomic action tasks corresponding to different execution results of the atomic action tasks corresponding to that node. Based on the execution result of any node, starting from the root node, branches are matched sequentially and nodes in the task decision tree of the initial state are executed. For any terminal node in the task decision tree of the current state, the task planning big language model is called in parallel to reason about the different execution results of the atomic action tasks corresponding to the terminal node and the subsequent atomic action tasks under different execution results, so as to update the child nodes in the task decision tree corresponding to the terminal node. If the current node has completed execution, the actual execution result of the current node is matched with the branch in the decision tree corresponding to the current node. If a match is successful, the subsequent child node to be executed is determined based on the matched branch, and the task decision tree is pruned to retain the subtree corresponding to the subsequent child node to be executed.

2. The long-term embodied task planning method according to claim 1, characterized in that, The construction of a semantic scene graph describing objects and their three-dimensional spatial relationships in the environment based on the multimodal perception data includes: Extract key image frames from the multimodal sensing data; The category labels of objects in the key image frames are identified using a visual language model; Based on the open vocabulary object detection and segmentation model, the two-dimensional position and pixel mask of each object in the image are determined according to the category label; The two-dimensional positions of each object are mapped to a three-dimensional coordinate system and spatially aligned with the point cloud data to determine the three-dimensional spatial position and bounding box of each object. Based on the category, attributes, and three-dimensional spatial positional relationships of each object, a structured semantic scene diagram is generated.

3. The long-term embodied task planning method according to claim 1, characterized in that, The step of inputting the semantic scene graph and the task instruction into the task planning large language model for reasoning to generate a task decision tree for the initial state corresponding to the natural language task instruction includes: The semantic scene graph is converted into a natural language description, which, together with the task instructions, constitutes a prompt message that is input into the task planning large language model. The task planning language model is guided to perform multi-step reasoning and decomposition of the task instructions based on the semantic scene graph. Each step obtained from the model reasoning decomposition is mapped to the corresponding atomic action task in the predefined robot atomic action library; With the first atomic action task as the root node, based on the preset execution result over assumption, at least one branch and its corresponding child node atomic action tasks are generated for the root node inference, thereby constructing the task decision tree of the initial state; The atomic action is the smallest deterministic action unit that the robot can execute.

4. The long-term embodied task planning method according to claim 1, characterized in that, The parallel invocation of the task planning large language model to infer different execution results of the atomic action tasks corresponding to any end node and subsequent atomic action tasks under different execution results, in order to update the child nodes in the task decision tree corresponding to any end node, includes: For each terminal node in the current task decision tree that has not yet undergone subsequent branch inference, an independent task planning large language model inference process is initiated; wherein, the task planning large language model inference process corresponding to any terminal node is used for execution: Obtain the input natural language task instructions, a node sequence consisting of executed nodes, and a task chain with any end node; Predict the subsequent atomic action tasks of any end node in the task chain under at least two preset execution results; The subsequent atomic action tasks under each predicted preset execution result are added to the decision tree as child nodes of any end task node, forming a new branch.

5. The long-term embodied task planning method according to claim 1, characterized in that, The parallel invocation of the task planning large language model to infer different execution results of the atomic action tasks corresponding to any end node and subsequent atomic action tasks under different execution results, in order to update the child nodes in the task decision tree corresponding to any end node, includes: For each terminal node in the current task decision tree that has not yet undergone subsequent branch inference, initiate the first task planning big language model inference process and the second task planning big language model inference process. The first task planning large language model inference process is used to perform: obtaining the input natural language task instruction, a task chain consisting of a node sequence composed of executed nodes and any end node; predicting the subsequent atomic action task of any end node in the task chain under the execution result of successful execution; adding the predicted subsequent atomic action task as a child node of any end task node to the decision tree to form a new branch. The second task, planning the large language model inference process, is used for execution: The system obtains the input natural language task instruction, a node sequence consisting of executed nodes, and a task chain containing any end node; predicts at least one preset execution result of the execution failure of any end node in the task chain, and the subsequent atomic action task under each preset execution result; and adds the predicted subsequent atomic action task under each preset execution result as a child node of the end task node to the decision tree to form a new branch.

6. The long-term embodied task planning method according to claim 4 or 5, characterized in that, The step of pruning the task decision tree to preserve the subtree corresponding to the subsequent child node to be executed includes: Delete all unmatched branches and subtrees containing the branches in the decision tree, and retain the subtree corresponding to the subsequent child node to be executed, so as to update the task decision tree; Terminate the task planning large language model inference process associated with all nodes in the deleted subtree.

7. The long-term embodied task planning method according to claim 1, characterized in that, The method further includes: If a match fails, the task planning language model is invoked again based on the actual execution result to generate a new branch and corresponding child node for the current node.

8. The long-term embodied task planning method according to claim 7, characterized in that, In the event of a matching failure, the process of re-invoking the large language model based on the actual execution result to generate a new branch and corresponding child nodes for the current node includes: The natural language task instructions, the actual execution results, and the node sequence consisting of the executed nodes in the task decision tree are input into the newly started task failure planning large language model inference process. The execution results are analyzed by planning the large language model inference process based on the task failure, and a new branch and corresponding child node are directly generated for the current node. Specifically, the task failure planning big language model is used to analyze and identify a clear cause of failure, and then determine the cause of failure and the generated subsequent corrective action task for the cause of failure as the new branch and the corresponding child node of the current node; if no clear cause of failure can be identified, the task containing the instruction requesting manual intervention is determined as the child node of the new branch corresponding to the current node.

9. The long-term embodied task planning method according to claim 1, characterized in that, The method further includes: During task execution, if the length of the historical context information exceeds a set threshold, the summary language model is invoked to compress and summarize the historical information, and the summarized information is used for subsequent planning reasoning by the task planning language model.

10. The method according to claim 9, characterized in that, The step of calling the summary big language model to compress and summarize historical information includes: If the length of the context input to the task planning large language model exceeds a preset threshold, summary generation is triggered; The summary language model summarizes the historical dialogue and task status to generate a summary text that includes the natural language task instructions, the node sequence consisting of executed nodes in the task decision tree, the remaining objectives, and the current execution status. The summary text replaces part or all of the original historical messages and serves as the context input for subsequent reasoning in the task planning large language model.

11. The method according to claim 1, characterized in that, The step of matching the actual execution result of the current node with the branch in the decision tree corresponding to the current node includes: After executing the atomic action corresponding to the current node, the system uses sensor feedback and multimodal model analysis to determine whether the atomic action achieves the expected execution result for each branch corresponding to the current node. If the expected execution result of the task corresponding to any branch is achieved, it is determined that the actual execution result of the current node matches that branch successfully; otherwise, it is determined that the actual execution result of the current node fails to match.

12. A long-term embodied task planning device based on a large model, characterized in that, The device includes: The memory is configured to store instructions; A processor is configured to retrieve the instructions from the memory and, when executing the instructions, to implement the long-term embodied task planning method based on a large model according to any one of claims 1 to 11.

13. A robot, characterized in that, The robot includes the long-term embodied task planning device based on a large model as described in claim 12.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the long-term embodied task planning method based on a large model as described in any one of claims 1 to 11.

Images