Dynamic feedback-based embodied robot task processing method and robot system

Abstract

The application relates to a dynamic feedback-based embodied robot task processing method and a robot system. The method comprises the following steps: determining a current subtask to be executed from a current subtask sequence, verifying the current subtask to obtain a task verification result; in the case that the task verification result does not satisfy a rearrangement condition, executing the current subtask to obtain a task execution result; in the case that the task verification result satisfies the rearrangement condition or the task execution result satisfies the rearrangement condition, rearranging subtasks of a target task based on the task execution result and execution context information of the current subtask to obtain a reconstructed subtask sequence; updating the current subtask sequence by using the reconstructed subtask sequence, and again entering the step of determining a current subtask to be executed from the current subtask sequence until the target task is executed. The method can improve the task completion rate of the embodied robot.

Description

Technical Field

[0001] This application relates to the field of embodied robot technology, and in particular to an embodied robot task processing method and robot system based on dynamic feedback. Background Technology

[0002] With the development of technology in the field of intelligence, the application of embodied robots is becoming more and more widespread, and the functions of embodied robots are becoming more and more powerful. For example, embodied robots can perform tasks according to the user's instructions.

[0003] In traditional technologies, embodied robot task execution solutions mainly rely on two approaches: end-to-end deep learning models and hierarchical task planning and hard-coded execution.

[0004] However, in the two traditional approaches, since the embodied robot executes step by step according to a preset static task sequence, it is easy to cause task rigidity and failure, which affects the task completion rate of the embodied robot. Summary of the Invention

[0005] Therefore, it is necessary to provide a dynamic feedback-based embodied robot task processing method and embodied robot system that can improve the task completion rate in response to the above-mentioned technical problems.

[0006] On the one hand, this application provides a dynamic feedback-based embodied robot task processing method, including: determining the current subtask to be executed from the current subtask sequence, verifying the current subtask to obtain a task verification result; the current subtask sequence is obtained by decomposing the target task to be executed; if the task verification result does not meet the rearrangement conditions, executing the current subtask to obtain a task execution result; if the task verification result meets the rearrangement conditions, or the task execution result meets the rearrangement conditions, rearranging the target task into subtasks based on the task execution result and the execution context information of the current subtask to obtain a reconstructed subtask sequence; updating the current subtask sequence using the reconstructed subtask sequence, and re-entering the step of determining the current subtask to be executed from the current subtask sequence, until the target task is completed.

[0007] On the other hand, this application also provides an embodied robot system, which is used to: determine the current subtask to be executed from the current subtask sequence, verify the current subtask to obtain a task verification result; the current subtask sequence is obtained by decomposing the target task to be executed; if the task verification result does not meet the rearrangement condition, execute the current subtask to obtain a task execution result; if the task verification result meets the rearrangement condition, or the task execution result meets the rearrangement condition, rearrange the target task into subtasks based on the task execution result and the execution context information of the current subtask to obtain a reconstructed subtask sequence; update the current subtask sequence using the reconstructed subtask sequence, and re-enter the step of determining the current subtask to be executed from the current subtask sequence, until the target task is completed.

[0008] On the other hand, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described embodied robot task processing method based on dynamic feedback.

[0009] On the other hand, this application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps in the above-described embodied robot task processing method based on dynamic feedback.

[0010] On the other hand, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps in the above-described embodied robot task processing method based on dynamic feedback.

[0011] The aforementioned embodied robot task processing method, embodied robot system, computer equipment, computer-readable storage medium, and computer program product based on dynamic feedback determine the current subtask to be executed from the current subtask sequence, verify the current subtask to obtain a task verification result, whereby the current subtask sequence is obtained by decomposing the target task to be executed. If the task verification result does not meet the reordering conditions, the current subtask is executed to obtain the task execution result. Thus, by verifying the current subtask, it is possible to detect early whether reordering is needed. If the task verification result or the task execution result meets the reordering conditions, based on the task execution result and the execution context information of the current subtask, the target task is reordered into subtasks to obtain a reconstructed subtask sequence. The current subtask sequence is updated using the reconstructed subtask sequence, and the process of determining the current subtask to be executed from the current subtask sequence is repeated until the target task is completed. Therefore, by reordering the target task into subtasks, the current subtask sequence can be dynamically changed, which helps to improve adaptability to the environment and thus improve the task completion rate of the embodied robot. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a diagram illustrating the application environment of the embodied robot task processing method based on dynamic feedback in some embodiments.

[0014] Figure 2 This is a schematic diagram of a task processing method for an embodied robot based on dynamic feedback in one embodiment;

[0015] Figure 3 This is a schematic diagram of the overall process of the dynamic feedback closed loop of the embodied robot in one embodiment;

[0016] Figure 4 This is a schematic diagram of the dynamic feedback logic during the execution phase of a subtask in one embodiment;

[0017] Figure 5 This is a schematic diagram of the semantic-driven execution flow of the replanning module in one embodiment;

[0018] Figure 6 This is an internal structural diagram of a computer device in one embodiment;

[0019] Figure 7 This is a diagram of the internal structure of a computer device in another embodiment. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0021] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0022] The embodied robot task processing method based on dynamic feedback provided in this application embodiment can be applied to, for example... Figure 1 The application scenario shown includes an embodied robot 102 and a target environment, with the embodied robot 102 situated within the target environment. The embodied robot possesses sensing, movement, and interaction capabilities, and can interact with its environment in real time. It can capture information about its surroundings through sensory organs such as cameras, LiDAR, or tactile LiDAR installed on the embodied robot. The embodied robot can be, but is not limited to, cleaning robots, companion robots, and humanoid robots. Cleaning robots can be, but are not limited to, sweeping robots, mopping robots, and combined sweeping and mopping robots. The target environment can be any environment suitable for the application of the embodied robot, including, but is not limited to, home environments, industrial manufacturing environments, medical environments, warehousing and logistics environments, agricultural production environments, public service environments, educational and research environments, or entertainment performance environments.

[0023] Traditional embodied robot task execution solutions mainly rely on two approaches: end-to-end deep learning models and hierarchical task planning and hard-coded execution. These approaches have significant drawbacks: 1. Poor environmental adaptability: The system cannot understand and respond to environmental changes outside the preset task sequence (such as changes in the target object's position or the appearance of new obstacles), leading to task stagnation and failure. For example, if the instruction is "put the clothes in the washing machine" but the clothes are not in the preset position, the system will stall due to the failure of the "grab the clothes" step. 2. Lack of task-level semantic feedback loops: Existing feedback is mostly limited to the low-level control layer (such as "insufficient grasping force" or "reaching the target point"), lacking semantic-level verification related to the task objective (such as visual judgment of "whether the clothes have been safely placed in the washing machine"). This prevents the robot from truly understanding whether the task has been accomplished and from explaining the root cause of failure. 3. Lack of online dynamic replanning capability: When a subtask fails, the system can usually only retry a limited number of times or directly report the failure. It cannot generate alternative solutions (such as "opening the door" before "taking") based on a semantic understanding of the cause of failure (e.g., "the goal is unattainable" because "the door is closed"). This results in low task continuity and success rate. 4. Insufficient human-computer interaction and explainability: Due to the lack of semantic-level state representation and decision-making processes, the system struggles to clearly explain to users "what it is doing," "why it failed," and "how it will adjust." This leads to low trust and collaboration efficiency in complex scenarios requiring human-computer collaboration.

[0024] This application aims to address the aforementioned deficiencies in traditional technologies. Specifically, the core technical problems to be solved include: how to overcome the problems of poor environmental adaptability, weak task continuity, and insufficient interpretability caused by the lack of semantic feedback and online replanning capabilities in existing technologies, thereby improving the autonomous task completion capabilities of embodied robots in open, dynamic, and complex environments.

[0025] Based on this, in an exemplary embodiment, such as Figure 2 As shown, a task processing method for embodied robots based on dynamic feedback is provided, which is executed by the embodied robot (hereinafter referred to as the robot), and includes the following steps:

[0026] Step 202: Determine the current subtask to be executed from the current subtask sequence, and verify the current subtask to obtain the task verification result; the current subtask sequence is obtained by decomposing the target task to be executed.

[0027] The target task can be a task triggered by user instructions or a pre-defined task. For example, the target task could be "put the clothes in the washing machine to wash".

[0028] The current subtask sequence contains subtasks corresponding to each step required to achieve the target task. The subtasks in the current subtask sequence can be arranged in execution order, with earlier subtasks executed before later ones. It can also be an ordered set of subtasks formed by breaking down the target task according to execution logic; for example, the current subtask sequence could be S={s1,s2,…,s…}. n}, where s i Let be a subtask, where 1 ≤ i ≤ n. The current subtask sequence is executed sequentially. The current subtask is the subtask that needs to be executed in the current subtask sequence; for example, the first current subtask is s1.

[0029] The current subtask sequence can be generated by a task planner (command_plan_prompt) deployed in the robot. The task planner can also generate task description information for the target task, which describes the target task. The task planner can generate the current subtask sequence using a task generation model. The task generation model can be an artificial intelligence model, such as a Vision Language Model (VLM). This application does not specifically limit the task generation model; any combination of multimodal large models or expert models capable of achieving similar functions as visual question answering, scene description, and sequence generation can be used to implement the task generation model. For example, the task generation model can be a locally deployed lightweight model to improve real-time performance, or it can be a combination of multiple specialized models, such as one VLM for scene description and another LLM for planning, to achieve the same effect.

[0030] Verification of the current subtask can be performed by verifying at least one of the necessity and feasibility of the current subtask. The necessity and feasibility can be verified sequentially. For example, the necessity can be verified first, and the feasibility can be verified after the necessity verification passes. If the necessity verification fails, the feasibility verification is skipped.

[0031] For example, the robot receives user instructions, performs semantic analysis on the instructions, and, if it determines that the user instructions indicate the execution of a target task, decomposes the target task to generate a current sequence of subtasks. The user instructions can also be referred to as natural language instructions, such as "put the clothes in the washing machine to wash."

[0032] For example, the robot can verify the current subtask based on at least two of the following: task description information of the target task, current environmental perception data, and historical execution states of the target task. The current environmental perception data may include at least one of color images, depth images, and semantic summaries. The historical execution states may include at least one of recently executed actions and reasons for failure.

[0033] For example, the robot can use a task verification model to verify the current subtask. This task verification model can be an artificial intelligence model, such as a visual language model. For instance, the task verification model can be used to perform semantic comparison and multi-level logical judgments to obtain the task verification result. These multi-level logical judgments include, but are not limited to, judgments on necessity and executability.

[0034] Step 204: If the task verification result does not meet the rearrangement conditions, execute the current subtask to obtain the task execution result.

[0035] The task verification result can indicate whether the current subtask is both necessary and feasible (represented by "Yes"), meaning the task objective has not yet been achieved but the operational conditions are met; it can also indicate whether the subtask is not necessary (represented by "No"), meaning the task objective has been met / achieved, for example, if the task objective is to pick up a target object, and the target object is already in the robot's hand; it can also indicate whether the subtask is necessary but not feasible; and it can indicate uncertainty (represented by "Uncertain"). This result can be generated when visual ambiguity or semantic confidence is low, making a reliable judgment impossible, for example, when the confidence level of the task verification result is less than a threshold. It is automatically generated or marked. The task execution result can be completed, failed, or uncertain.

[0036] For example, if the task verification result indicates that the current subtask is both necessary and executable, then the task verification result does not meet the reordering condition.

[0037] For example, the robot can generate a sequence of task actions for the current subtask, control the robot to execute the sequence of task actions, and if the sequence of task actions is executed successfully, generate a task execution result to indicate that the current subtask was executed successfully; if the sequence of task actions fails, generate a task execution result to indicate that the current subtask failed.

[0038] For example, during the execution of the current subtask, a task action sequence for the current subtask can be generated by calling a skill library interface, such as a pre-trained skill or a hard-coded skill. The robot is then controlled to execute this task action sequence to obtain the task execution result. The choice between calling a pre-trained skill or a hard-coded skill depends on whether the current subtask has pre-trained annotations. If the current subtask has pre-trained annotations, the task action sequence is generated using pre-trained skills; otherwise, the task action sequence is generated using hard-coded skills. Pre-trained annotations indicate that the current subtask supports pre-trained skills. This embodiment employs a dual-track collaborative architecture of "hard-coded skills + pre-trained skills," achieving a dynamic balance between stability and scalability in the execution layer. This ensures that even when the perception or learning model is unstable, critical tasks can still be completed using hard-coded branches, thereby guaranteeing structured fault tolerance and execution safety.

[0039] Of course, in addition to the dual-track architecture of "hard-coded skills and pre-trained skills" mentioned above, a unified skill learning framework can also be adopted, such as using skill models trained entirely based on imitation learning or reinforcement learning. This application is inclusive of the specific implementation methods of the underlying skills.

[0040] Step 206: If the task verification result meets the reordering condition, or the task execution result meets the reordering condition, the target task is reordered into subtasks based on the task execution result and the execution context information of the current subtask, to obtain the reconstructed subtask sequence.

[0041] The execution context information includes, but is not limited to, recent action sequences, failure reason descriptions, system error codes, and exception types. Recent action sequences include, but are not limited to, execution time, skill type, and success / failure labels. Failure reason descriptions include, for example, "target does not exist," "grab unsuccessfully," or "timeout."

[0042] For example, a completion judgment is performed when the task execution result is successful. The completion judgment stage is a post-execution verification mechanism based on the visual language model, used to perform semantic-level effect judgment and dynamic feedback after the task execution is completed. The completion judgment process includes: acquiring input data, which includes environmental perception data collected after the completion of the current subtask, such as environmental images, task target description (i.e., the target of the current subtask), and execution logs; inputting the input data into the visual language model to perform semantic evaluation of the execution result by calling the visual language model and obtaining the output result of the visual language model; if the output result is "completed" (i.e., the target state corresponding to the current subtask has been achieved), the current subtask is marked as successful and proceeds to the next subtask; if the output result is "failed", the failure record is written to the execution log, and if the number of consecutive failures exceeds the target number N_fail, the replanning module is triggered to regenerate the subtask sequence; if the output result is "uncertain", a secondary image acquisition or lightweight semantic clarification process is triggered, and if the output result after the secondary judgment is still "uncertain" or "failed", the replanning module is triggered to regenerate the subtask sequence. In this embodiment, the semantic-level verification and closed-loop control of the task completion status are realized through the linkage mechanism of "execution result - semantic judgment - replanning". It has interpretability, traceability and task continuity guarantee, and improves the success rate and stability of the embodied robot in complex environments.

[0043] For example, "re-arranging the target task into sub-tasks to obtain a reconstructed sub-task sequence" can be achieved through a replanning module in the robot. The replanning module can generate replanning prompts based on the task execution results and the execution context information of the current sub-tasks. These prompts can be input into the task orchestration model to generate the reconstructed sub-task sequence. The replanning prompts can be standardized prompts generated based on prompt templates, used to drive the task orchestration model to output a new executable task sequence. The prompt templates can be, but are not limited to, those from Qwen-VL (A Versatile Vision-Language Model) or GPT (Generative Pre-trained Transformer) series models.

[0044] Step 208: Update the current subtask sequence using the reconstructed subtask sequence, and then proceed to the step of determining the current subtask to be executed from the current subtask sequence again, until the target task is completed.

[0045] For example, the reconstructed subtask sequence can be automatically verified, and after successful verification, the reconstructed subtask sequence can be used as the current subtask sequence. Alternatively, the reconstructed subtask sequence can be presented to the user for manual verification. Alternatively, the reconstructed subtask sequence can be automatically verified first, and if the confidence level of the verification result is lower than a confidence threshold, the user can be requested to intervene and make a judgment.

[0046] For example, such as Figure 3 As shown, a flowchart of the dynamic feedback closed loop for embodied robots is provided. Dynamic feedback includes pre-execution feedback regarding executability and necessity ("start-up judgment") and post-execution result feedback ("completion judgment"). The dynamic feedback mechanism forms a core closed loop in sequence: "start-up judgment—task execution—completion judgment," and logically achieves task-level dynamic repair and semantic consistency maintenance through a key "replanning module." The start-up judgment phase is used for necessity analysis and environmental condition verification before action execution; the task execution phase uses layered skill invocation and anomaly monitoring mechanisms to complete specific operations; and the completion judgment phase is used for semantic-level effect verification and replanning trigger judgment after action execution. Specifically, firstly, the task scheduler generates an initial subtask sequence, i.e., the current subtask sequence. Then, from the current subtask sequence, the current subtask to be executed, such as subtask 1, is determined. For subtask 1, a "startup check" is performed. This "startup check" means "verifying the current subtask to obtain a task verification result." If the check result (i.e., the task verification result) is Yes, then subtask 1 is executed. After subtask 1 is completed, a "completion check" is performed on its execution result. This "completion check" determines whether the goal of subtask 1 has been achieved. If the goal of subtask 1 has been achieved, the processing flow for the next subtask proceeds. If the goal of subtask 1 has not been achieved, the subtask can be executed repeatedly. 1. If the number of consecutive failures reaches the target number, a replanning is triggered. If it is uncertain whether the goal of subtask 1 will be achieved, a second image acquisition or lightweight semantic clarification can be triggered. If the second judgment is still "uncertain" or "failed" (meaning the goal has not been achieved), the replanning module is triggered. If the "start judgment" result is No, subtask 1 is skipped, marked as "not required to execute", and the current subtask is updated to subtask 2. If the "start judgment" result is Uncertain, replanning is triggered to update the current subtask sequence, and then the step of "determining the current subtask to be executed from the current subtask sequence" is returned. This process is repeated until the target task is completed. Through the linkage mechanism of "execution result - semantic judgment - replanning", semantic-level verification and closed-loop control of task completion status are realized, which has interpretability, traceability and task continuity guarantee, and improves the success rate and stability of tasks in complex environments.

[0047] For example, when the robot performs the target task "put the clothes in the washing machine," if the visual language model determines that the target does not exist (i.e., there are no clothes on the sofa) in the step "retrieving clothes from the sofa," the replanning module is triggered to re-arrange the target task into subtasks. To re-arrange the target task into subtasks, the replanning module generates replanning prompts. These prompts can include at least one of the following: the original task target (e.g., "put in the washing machine"), execution history (e.g., "grabbing failed, target not found"), or current perception (e.g., "clothes are piled on the floor"). The replanning module inputs the replanning prompts into the generation model, which then calls the task orchestration model to generate a new task sequence based on the replanning prompts, i.e., a reconstructed subtask sequence. For example: step 1: pick up clothes from the floor; step 2: navigate to the washing machine; step 3: open the washing machine door; step 4: put the clothes in the washing machine. Then, the reconstructed subtask sequence, such as S′={s1′,s2′,…,s…}, is used to generate the new task sequence. m The module replaces the existing current subtask sequence, re-enters the "startup judgment" stage, and continues execution. This demonstrates that the replanning module maintains task continuity, semantic consistency, and state traceability even when tasks deviate or the environment changes, thus improving the success rate and fault tolerance of the embodied robot in open environments.

[0048] In the aforementioned dynamic feedback-based embodied robot task processing method, the current subtask to be executed is determined from the current subtask sequence. The current subtask is then verified to obtain a task verification result. The current subtask sequence is obtained by decomposing the target task to be executed. If the task verification result does not meet the reordering conditions, the current subtask is executed to obtain the task execution result. Thus, by verifying the current subtask, it is possible to detect early whether reordering is necessary. If the task verification result or the task execution result meets the reordering conditions, the target task is reordered based on the task execution result and the execution context information of the current subtask, resulting in a reconstructed subtask sequence. The current subtask sequence is updated using the reconstructed subtask sequence, and the process of determining the current subtask to be executed from the current subtask sequence is repeated until the target task is completed. Therefore, by reordering the target task into subtasks, the current subtask sequence can be dynamically changed, which helps improve adaptability to the environment and thus improves the task completion rate of the embodied robot.

[0049] In an exemplary embodiment, verifying the current subtask to obtain a task verification result includes: verifying the execution necessity of the current subtask based on current environmental perception data, the target task, and the historical execution status of the target task, to obtain a necessity verification result; if the first confidence level of the necessity verification result is greater than or equal to a first threshold, and the necessity verification result indicates that the current subtask has execution necessity, verifying the executability of the current subtask based on the current environmental perception data, to obtain an executability verification result; the first confidence level is used to reflect the accuracy of the necessity verification result; and a task verification result for the current subtask is generated based on the executability verification result and the first confidence level.

[0050] Among them, the necessity verification result is either necessary to execute, not necessary to execute, or uncertain; the first threshold can be a preset confidence threshold value used to determine whether the necessity verification result is credible; the executability verification result is either executability is satisfied, executability is not satisfied, or uncertain.

[0051] For example, the executability of the current sub-task is verified based on the current environmental perception data to obtain an executability verification result, including: determining the operation object targeted by the current sub-task; identifying the state information of the operation object from the current environmental perception data to obtain an object identification result; if the object identification result indicates that the operation object does not exist or is inoperable, then generating an executability verification result indicating that executability is not met; if the object identification result indicates that the operation object exists and is operable, then analyzing the environmental factors in the current environmental perception data to obtain an environmental factor analysis result; if the environmental factor analysis result indicates that the current environment does not meet the conditions for executing the current sub-task for the operation object, then generating an executability verification result indicating that executability is not met; environmental factors include, but are not limited to, at least one of lighting, obstacles, and spatial location; if the environmental factor analysis result indicates that the current environment meets the conditions for executing the current sub-task for the operation object, then generating an executability verification result indicating that executability is met; if the confidence level of the object identification result is less than a second threshold, or the confidence level of the environmental factor analysis result is less than the second threshold, then generating an executability verification result indicating uncertainty.

[0052] For example, the robot can input current environmental perception data, task description information of the target task, and historical execution status of the target task into the task verification model. Using the task verification model, the robot can determine the current environmental state based on the current environmental perception data, determine the task target based on the task description information, perform semantic comparison between the current environmental state and the task target, determine the degree of matching between the current environmental state and the task target, determine the action triggering conditions, and then generate the task verification result.

[0053] In this embodiment, the necessity and executability of the current subtask are verified sequentially. Executability verification is only performed when the current subtask is necessary to execute, thus eliminating redundant or unnecessary verifications and ensuring sufficient verification of the current subtask.

[0054] In an exemplary embodiment, generating a task verification result for the current subtask based on the executability verification result and a first confidence level includes: obtaining a second confidence level for the executability verification result; the second confidence level is used to reflect the accuracy of the executability verification result; if the second confidence level of the executability verification result is greater than or equal to a second threshold, and the executability verification result indicates that the current subtask is executable, then generating a task verification result to indicate that the rearrangement conditions are not met; if the second confidence level is less than the second threshold, or the first confidence level is less than the first threshold, then re-verifying the current subtask to obtain a task verification result.

[0055] The second confidence level can be the confidence level of the object identification result or the confidence level of the environmental factor analysis result. The second threshold can be a confidence level critical value used to quantify the accuracy and reliability of the executability verification result.

[0056] In this embodiment, since the current environmental perception data is collected by the robot, if the quality of the current environmental perception data is poor, it is easy to lead to a low confidence level during the verification process. Therefore, if the second confidence level is less than the second threshold, or the first confidence level is less than the first threshold, the current subtask is re-verified to obtain the task verification result. The environmental perception data can be re-obtained for verification, which can reduce the situation where verification fails due to poor quality of the collected environmental perception data.

[0057] In an exemplary embodiment, if the second confidence level is less than the second threshold, or the first confidence level is less than the first threshold, the current subtask is re-verified to obtain a task verification result, including: re-collecting the current environmental perception data to obtain collected environmental perception data; if the first confidence level is less than the first threshold, re-verifying the execution necessity of the current subtask based on the collected environmental perception data and the target task; if the re-verification determines that the current subtask is not necessary to execute, generating a task verification result to indicate that the re-arrangement conditions are met; if the second confidence level is less than the second threshold, re-verifying the executability of the current subtask based on the collected environmental perception data; if the re-verification determines that the current subtask is not executable, generating a task verification result to indicate that the re-arrangement conditions are met.

[0058] For example, the robot can re-verify the necessity of executing the current sub-task based on collected environmental perception data, task description information of the target task, and the historical execution status of the target task.

[0059] like Figure 4 The diagram shows the processing flow of the current subtask. "Receive Subtask" refers to determining the current subtask, and "Action Necessity" refers to the necessity of execution. The "Action Necessity" judgment is used to determine whether the task objective has been met, i.e., whether the target task's objective has been achieved. If it is determined that there is no "Action Necessity" and the first confidence level is greater than or equal to the first threshold, the subtask ends, and the processing flow of the next subtask begins. If it is determined that there is "Action Necessity" and the first confidence level is greater than or equal to the first threshold, the executability of the current subtask is verified. If the first confidence level is less than the first threshold, the execution necessity of the current subtask is verified again.

[0060] During the verification of the executability of the current subtask, the "object state" is analyzed first. The "object state" analysis refers to the object recognition result obtained by identifying the state information of the operation object from the current environmental perception data. If the "object state" analysis result indicates that the operation object exists and is operable, then the "environmental factors" analysis is performed to determine whether the environmental factors (including but not limited to at least one of lighting, obstacles, and spatial location) meet the conditions for executing the current subtask for the operation object. The environmental factor analysis result is obtained. If the environmental factor analysis result indicates that the conditions are not met, a task verification result is generated to indicate that the re-arrangement conditions are met. If the environmental factor analysis result indicates that the conditions are met, a task verification result is generated to indicate that the re-arrangement conditions are not met, and the current subtask is executed.

[0061] During the execution of the current subtask, the execution result can be judged as "executed in place". The "executed in place" judgment is used to determine whether the goal of the current subtask has been achieved. If it has been achieved, the current subtask ends. If it has not been achieved, the current subtask can be executed again. If the goal of the current subtask is still not achieved when the number of executions of the current subtask is greater than n, the current subtask will not be repeated, and the execution result of the current subtask will be determined to meet the reordering conditions.

[0062] In this embodiment, when the confidence level is low, the situation where the confidence level is low due to poor clarity of the collected environmental perception data can be reduced by collecting environmental perception data again and verifying it.

[0063] In an exemplary embodiment, if the task verification result does not meet the re-arrangement conditions, executing the current sub-task to obtain the task execution result includes: obtaining the task execution annotation of the current sub-task, where the task execution annotation is determined based on the task feature information of the current sub-task; if the task execution annotation indicates that the current sub-task supports a pre-trained skill, then invoking the pre-trained skill to determine the task action sequence of the current sub-task; if the task execution annotation indicates that the current sub-task supports a hard-coded skill, then invoking the hard-coded skill to determine the task action sequence of the current sub-task; and controlling the embodied robot to execute the task action sequence to obtain the task execution result.

[0064] Among them, the task execution annotation can be annotated by the task planner in the robot for the current sub-task. The task planner can generate task execution annotations for each sub-task in the current sub-task sequence. For example, task execution annotations can be generated for each sub-task during or after the generation of the current sub-task sequence.

[0065] Task feature information is used to indicate whether the pre-trained model has been trained to perform the current subtask, and to indicate whether the subtask needs to be adaptively expanded according to the environment.

[0066] For example, for each subtask in the current subtask sequence, the task planner can determine the task feature information of the subtask. If the task feature information indicates that the pre-trained model has been trained to execute the current subtask, or that the subtask needs to be adaptively expanded according to the environment, then task execution labels indicating that the subtask supports pre-trained skills are generated, namely the pre-training labels mentioned above. If the task feature information indicates that the pre-trained model has not been trained to execute the current subtask, and that the subtask does not need to be adaptively expanded according to the environment, then task execution labels indicating that the subtask supports hard-coded skills are generated.

[0067] For example, the robot system can uniformly abstract the calling interface of "skill name + parameters" at the task scheduling layer to facilitate subsequent skill expansion and replacement.

[0068] In this embodiment, a dual-track collaborative architecture of "hard-coded skills + pre-trained skills" is adopted, which can achieve a dynamic balance between stability and scalability in the execution layer. This allows the hard-coded branches to complete critical tasks even when the perception or learning model is unstable, thereby ensuring structured fault tolerance and execution safety.

[0069] In an exemplary embodiment, the method further includes: during the execution of a subtask in the current subtask sequence, acquiring task execution events of the current subtask sequence in real time, and collecting environmental feedback events in real time; storing the task execution events and environmental feedback events in an execution log related to the target task; when there is an abnormal event in the execution log that meets the abnormal conditions, determining the event type of the abnormal event, and processing the abnormal event based on the event type.

[0070] The task execution events include, but are not limited to, at least one of the following: timeout count, error code, and consecutive failure count. The timeout count can be, but is not limited to, a count performed when the action execution duration or the task execution duration has been exceeded. Error codes can be, but are not limited to, underlying runtime errors such as motor malfunctions. The consecutive failure count refers to the number of times a subtask has failed to execute consecutively.

[0071] Environmental feedback events can be generated by analyzing real-time collected environmental perception data through a visual language model. For example, a visual language model can be used to identify the operation object corresponding to the current sub-task from the environmental perception data and generate visual alignment information. The visual alignment information is used to indicate whether the operation object corresponding to the current sub-task exists in the environment. For example, if the current sub-task is "take the water cup from the table", then the visual alignment information is used to indicate "whether the water cup on the table exists".

[0072] Abnormal conditions may include, but are not limited to, the number of consecutive failures reaching a threshold, execution time exceeding the limit, or error code being non-zero (i.e., ErrorCode ≠ 0); an abnormal event is an event that meets an abnormal condition. For example, if N_fail represents the threshold, and the number of consecutive failures is ≥ N_fail, then the abnormal event "number of consecutive failures ≥ N_fail" is triggered.

[0073] Event types include, but are not limited to, recoverable and unrecoverable types. Unrecoverable types include, for example, communication interruptions, mechanical blockages, or motor failures. Recoverable types can have, but are not limited to, consecutive failures ≥ N_fail and ErrorCode ≠ 0.

[0074] For example, during the execution of the current subtask, if the current subtask supports pre-trained skills, the task action sequence of the current subtask is generated by calling the pre-trained skills and the task action sequence is executed. If an exception occurs during the execution process where the number of consecutive failures is ≥ N_fail or ErrorCode ≠ 0, the updated task action sequence of the current subtask is generated by calling the hard-coded skills and the updated task action sequence is executed. Thus, through the rollback mechanism, the stability lower limit and self-recovery capability of the execution layer are achieved, ensuring that the task lower limit is controllable.

[0075] For example, if the event type is recoverable, the exception information can be passed to the replanning module in a standardized log format for subsequent Replan Prompt input.

[0076] For example, if the event type is unrecoverable, an error log is recorded and the execution of the target task is paused, awaiting further instructions from the upper-level system or from a human to prevent the exception from escalating. The exception information includes, but is not limited to, at least one of the following: task ID, action category, failure reason, timestamp, or environment snapshot summary.

[0077] In this embodiment, by automatically detecting and handling abnormal events, developers do not need to pre-write processing branches for all possible abnormal situations. The "intelligence" of the system is reflected in its ability to dynamically solve problems, which reduces the complexity of system development and maintenance.

[0078] In an exemplary embodiment, the method for determining whether the task execution result meets the reordering condition further includes: if the task execution result indicates that the current subtask has failed, then the current subtask is executed repeatedly; if the number of times the current subtask has failed is greater than or equal to the target number, the task execution result is determined to meet the reordering condition; if the task execution result is a system error code returned when executing the current subtask, then the execution of the current subtask is paused, an exception prompt message is generated based on the system error code, the exception prompt message is reported, and if no exception handling instruction is received within the target time period after reporting the exception prompt message, the task execution result is determined to meet the reordering condition.

[0079] The target number of attempts can be set according to actual needs, such as 3 or 5, and this application does not impose a specific limitation. The system returns an error code, such as the error code mentioned above, i.e., ErrorCode. The exception message can include the system-returned error code. The duration of the target time period can be set according to actual needs. The exception handling instruction is the instruction to correct errors in response to the system-returned error code.

[0080] For example, if the number of execution failures of the current subtask is less than the target number, and an exception handling instruction is received within the target time period after the exception prompt information is reported, then the exception handling instruction is executed, and the current subtask is either re-executed or continues to be executed after the exception handling instruction is executed.

[0081] For example, the task execution occurs at the execution layer. If the task execution result is the detection of an unrecoverable abnormal event, such as communication interruption or mechanical blockage, an error log is generated based on the detected unrecoverable abnormal event, and the system waits for the exception handling instructions from the upper-level system of the execution layer.

[0082] In this embodiment, the anomaly reporting-replanning linkage mechanism enables the task chain to be automatically reconstructed without interruption when an anomaly occurs, thereby improving the robot's execution reliability, task continuity, and behavior interpretability in dynamic environments.

[0083] In an exemplary embodiment, based on the task execution result and the execution context information of the current subtask, the target task is rearranged into subtasks to obtain a reconstructed subtask sequence. This includes: packaging the task execution result, execution context information, and the task objective of the target task to obtain structured text information; generating rearranged prompts based on the structured text information and prompt word templates; and calling the task orchestration model to rearrange the target task into subtasks based on the rearranged prompts to obtain a reconstructed subtask sequence.

[0084] The structured text information can be automatically summarized by the task execution layer and the perception layer, ensuring consistency of the context state. The structured text information may also include at least one of the following: user instructions (i.e., user natural language instructions), current task subsequence, task information of the current subtask, or current perception state information.

[0085] The current perceived state information can include environmental state description information, which can be extracted from real-time acquired images using a visual language model. For example, color or multi-view images can be acquired, and semantic summarization can be extracted from the acquired images using a visual language model to generate environmental state description information such as "washing machine door is closed, there are obstacles on the ground". The current perceived state information can also include spatial state description information, which can come from the navigation or depth module. Spatial state description information can be used to describe spatial positional relationships, such as "the distance between the sofa and the washing machine is 1.2m". The task information of the current subtask includes at least one of the following: the current subtask number, context, and running state. For example, structured text information could be:

[0086] [Current Task Objective]: Put the clothes in the washing machine to wash (i.e., the objective of the target task);

[0087] [Current Step]: Step 3: Take clothes from the sofa;

[0088] [Execution Result]: Status: Failed; Reason for failure: Target does not exist;

[0089] [Environmental Description]: There are no clothes on the sofa, and the clothes are piled on the ground; the distance from the ground to the washing machine is 1.0m (i.e., current perception status information).

[0090] The task orchestration model can be a generative model or a visual language model with semantic understanding and task decomposition capabilities, and can be, but is not limited to, models such as Qwen-VL or GPT. The task orchestration model can combine the text and visual information in the re-arranged prompts to output a structured task re-arrangement result, i.e., a new sequence of subtasks.

[0091] A reprogramming prompt could be something like: "You are a holographic robot task programming assistant. The current task objective remains unchanged, but a failure or environmental change has occurred during execution. Please regenerate a new task plan based on the following information, ensuring that the actions are executable and logically coherent."

[0092] [Task Objective]: {Objective Description};

[0093] [Executed Steps]: {Historical Actions and Results};

[0094] [Reason for failure]: {Failure summary};

[0095] [Current Environment]: {Visual Semantic Description};

[0096] [Constraints]: Maintain safety, minimize repetitive actions, and avoid ineffective operations.

[0097] Please output:

[0098] New subtask sequence (adjusting only necessary steps);

[0099] The execution objective of each subtask;

[0100] If you need to add preparation steps (such as "get closer to the object" or "adjust the viewing angle"), please add them.

[0101] Output format:

[0102] Step 1:…;

[0103] Step 2:…

[0104] For example, the robot can input rearrangement prompts into the task orchestration model, thereby calling the task orchestration model to rearrange the target task into subtasks and generate a reconstructed subtask sequence.

[0105] In this embodiment, structured prompt technology is applied to the robot's task planning layer. When the system detects a failure or unexecutable state, it can automatically integrate the task context, execution history, and current environmental semantics to generate a "replanning prompt". This drives the model to generate a validated alternative task sequence that fits the current situation online, thus achieving dynamic self-repair at the task level.

[0106] In an exemplary embodiment, updating the current subtask sequence using the reconstructed subtask sequence includes: performing comprehensive verification on the reconstructed subtask sequence; and updating the current subtask sequence using the reconstructed subtask sequence if the comprehensive verification passes. The comprehensive verification includes, but is not limited to, target consistency verification, repeatability verification, logical verification, and executability verification. Target consistency verification can be used to verify whether the task target of the reconstructed subtask sequence is consistent with the task target of the target task. Repeatability verification can be used to verify whether there are duplicate subtasks in the reconstructed subtask sequence. Executability verification can be used to verify the executability of the subtasks in the reconstructed subtask sequence. Logical verification can be used to verify whether the execution logic of the reconstructed subtask sequence is reasonable.

[0107] Among these methods, goal consistency verification ensures that the new sequence does not deviate from the original task objective. Repeated subtasks can be multiple subtasks with the same objective in the reconstructed subtask sequence, such as multiple actions of taking clothes; or, repeated subtasks can refer to subtasks with the same objective as subtasks that have already been successfully executed, i.e., subtasks that have already been executed successfully, such as taking clothes, but now there are no more clothes to take. If there are still unexecuted subtasks of taking clothes in the reconstructed subtask sequence, then the subtask of taking clothes is a repeated subtask. Logical verification verifies the order and dependencies of subtasks, such as "open the door first, then put the clothes away."

[0108] For example, if the comprehensive verification passes, the reconstructed subtask sequence is taken as the current subtask sequence, and the current subtask sequence is updated to the reconstructed subtask sequence.

[0109] For example, if the comprehensive verification fails, the process of generating rearrangement prompts, generating a sequence of refactoring subtasks based on the rearrangement prompts, and performing comprehensive verification on the sequence of refactoring subtasks is repeated until a sequence of refactoring subtasks that passes comprehensive verification is obtained.

[0110] For example, such as Figure 5As shown, a semantically driven execution flowchart of the replanning module is provided. Based on execution feedback and environmental perception results, the replanning module generates failure analysis information that triggers replanning prompts. While maintaining the task objective, it dynamically rearranges the current subtask or local task sequence and generates alternative actions, forming a local self-healing closed loop of "replanning—re-execution—re-verification". Re-execution means re-entering the "start-up judgment" stage, thus forming a global feedback closed loop. Through the above process, the system forms a self-consistent "perception—semantic decision—execution—verification—feedback" link, ensuring that if a deviation occurs at any stage of task execution, a semantically interpretable response and action-level adaptive repair can be obtained. Specifically, the process includes: detecting triggering conditions, including: marking the current subtask as "unexecutable" in the "start-up judgment" stage; the result of the "completion judgment" stage being "failure" and the number of consecutive failures reaching the target number; the result of the "completion judgment" stage being "uncertain" and still being "uncertain" or "failure" after a second judgment; and the system returning an error code ErrorCode ≠ 0.

[0111] Upon detecting any triggering condition, the replanning module is activated. This module packages the input information into a contextual framework to obtain structured text information, including execution history, task objective description (description of the target task), and perception state. Next, a replanning prompt is constructed based on the structured text information and the prompt word template. A sequence of reconstructed subtasks is then generated based on the replanning prompt, and the reconstructed subtask sequence undergoes comprehensive verification, such as verification of objective consistency, repeatability, and logical and physical feasibility, to achieve action-level dynamic repair without altering the overall task objective.

[0112] If verification fails, the process returns to the step of reconstructing prompts based on structured text information and prompt word templates. If verification succeeds, it returns to the "startup judgment phase" for re-execution and feedback. By re-entering the execution loop, subtasks are executed step by step and monitored in real time. Execution results and logs are sent back to the re-planning module, forming an adaptive learning loop. This process can be implemented using a local inference engine or a large language model API (such as offline deployed Qwen-VL or GPT series models) to meet real-time response requirements. After execution, the system feeds back the status to the task scheduler and re-enters the "startup judgment" phase, forming a global feedback loop.

[0113] In this embodiment, the reconstructed subtask sequence is comprehensively verified. If the comprehensive verification passes, the current subtask sequence is updated using the reconstructed subtask sequence. This ensures that the reconstructed subtask sequence is executable, logically sound, and free of duplicate subtasks, thereby reducing the probability of task failure and minimizing the processing of redundant subtasks.

[0114] This application introduces a dynamic semantic feedback and replanning mechanism driven by a visual language model that spans the entire task lifecycle, enabling robots to "understand" the current situation, "analyze" the causes, and "think" out new feasible solutions when task execution is hindered, just like humans, thereby achieving intelligent self-repair.

[0115] This application constructs a semantically driven dynamic feedback and adaptive replanning closed-loop system, specifically embodied in: 1. Full-process semantic judgment nodes: At two key nodes, before task initiation (initiation judgment) and after execution (completion judgment), a visual language model is introduced to perform semantic-level necessity verification and effect evaluation, replacing the traditional single sensor signal judgment. That is, the underlying sensor judges based on the execution result after execution, providing an interpretable decision basis for the system. 2. Prompt-driven online task replanning module: Structured prompt word technology is applied to the robot's task planning layer. When the system detects a failure or unexecutable state, it can automatically integrate the task context, execution history, and current environmental semantics to generate "replanning prompt words," driving the large model to generate a validated alternative task sequence that conforms to the current context online, realizing dynamic self-repair at the task level. 3. "Global-Local" dual closed-loop control structure: A global main closed loop consisting of "initiation judgment - execution - completion judgment" is designed, as well as a local self-repair closed loop triggered by the "replanning module." The two closed loops are tightly coupled through semantic states, forming an adaptive control system that unifies the rigor of task logic with the flexibility of execution process.

[0116] The embodied robot task processing method based on dynamic feedback provided in this application, through a dynamic replanning mechanism, enables the robot to effectively cope with sudden environmental changes and initial plan failures, increasing the probability of completing tasks in unstructured environments and improving task success rate and robustness. It does not rely on a fixed, unchanging pre-set script, but possesses the ability to adjust and optimize online based on real-time semantic understanding, truly adapting to dynamically changing complex environments and enhancing the system's adaptability in open environments. All decisions (whether to execute, whether to succeed, how to replan) can be based on semantic justification provided by a visual language model, making the robot's behavior more human-friendly. In terms of class, it is transparent, understandable, and predictable, enhancing human-machine mutual trust and collaboration efficiency, and improving the intelligence and explainability of the task execution process. Developers do not need to pre-write processing branches for all possible exceptions. The "intelligence" of the system is reflected in its ability to dynamically solve problems, reducing the complexity of system development and maintenance, and reducing the dependence on the completeness and accuracy of preset scripts. The "global main closed loop" ensures the rigor and traceability of task logic, while the "local self-healing closed loop" provides flexibility to cope with dynamic environments, enabling the system to have both high reliability and strong adaptability, and achieving a unity of task execution reliability and flexibility.

[0117] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0118] Based on the same inventive concept, this application also provides a dynamic feedback-based android system for implementing the aforementioned dynamic feedback-based android task processing method. The solution provided by this system is similar to the implementation described in the above method; therefore, the specific limitations in one or more android system embodiments provided below can be found in the limitations of the dynamic feedback-based android task processing method described above, and will not be repeated here.

[0119] In an exemplary embodiment, an embodied robot system is provided. The embodied robot system is used to: determine the current subtask to be executed from the current subtask sequence; verify the current subtask to obtain a task verification result; the current subtask sequence is obtained by decomposing the target task to be executed; if the task verification result does not meet the rearrangement conditions, execute the current subtask to obtain a task execution result; if the task verification result meets the rearrangement conditions, or if the task execution result meets the rearrangement conditions, rearrange the target task into subtasks based on the task execution result and the execution context information of the current subtask to obtain a reconstructed subtask sequence; update the current subtask sequence using the reconstructed subtask sequence, and re-enter the step of determining the current subtask to be executed from the current subtask sequence, until the target task is completed.

[0120] In one exemplary embodiment, a computer device is provided, the internal structure of which can be as shown in the figure. Figure 6As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores the current subtask sequence. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a dynamic feedback-based embodied robot task processing method.

[0121] In one exemplary embodiment, a computer device is provided, the internal structure of which can be as shown in the figure. Figure 7 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a dynamic feedback-based embodied robot task processing method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0122] Those skilled in the art will understand that Figure 6 and Figure 7The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0123] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0124] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0125] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0126] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0127] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0128] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0129] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for processing embodied robot tasks based on dynamic feedback, characterized in that, The method includes: The current subtask to be executed is determined from the current subtask sequence, and the current subtask is verified to obtain the task verification result; the current subtask sequence is obtained by decomposing the target task to be executed. If the task verification result does not meet the rearrangement conditions, the current subtask is executed to obtain the task execution result; If the task execution result satisfies the rearrangement condition, the task execution result, the execution context information of the current subtask, and the task objective of the target task are packaged to obtain structured text information. Based on the structured text information and the prompt word template, a rearrangement prompt is generated. Based on the rearrangement prompt, the target task is rearranged into subtasks to obtain a reconstructed subtask sequence. The current subtask sequence is updated using the reconstructed subtask sequence, and the step of determining the current subtask to be executed from the current subtask sequence is repeated until the target task is completed.

2. The method according to claim 1, characterized in that, The step of rearranging the target task into subtasks based on the rearrangement prompt to obtain a reconstructed subtask sequence includes: The task orchestration model is invoked, and based on the re-orchestration prompt, the target task is re-orchestrated into subtasks to obtain a reconstructed subtask sequence.

3. The method according to claim 1, characterized in that, The step of updating the current subtask sequence using the reconstructed subtask sequence includes: The reconstructed subtask sequence is comprehensively verified, and if the comprehensive verification passes, the current subtask sequence is updated using the reconstructed subtask sequence. The comprehensive verification includes one or more of the following: target consistency verification, repeatability verification, logical verification, and executability verification. The target consistency verification is used to verify whether the task objective of the reconstructed subtask sequence is consistent with the task objective of the target task. The repeatability verification is used to verify whether there are duplicate subtasks in the reconstructed subtask sequence. The executability verification is used to verify the executability of the subtasks in the reconstructed subtask sequence. The logical verification is used to verify whether the execution logic of the reconstructed subtask sequence is reasonable.

4. The method according to claim 1, characterized in that, The step of executing the current subtask to obtain the task execution result when the task verification result does not meet the rearrangement conditions includes: If the task verification result does not meet the reordering conditions, the task execution label of the current subtask is obtained, and the task execution label is determined based on the task feature information of the current subtask; If the task execution annotation indicates that the current subtask supports a pre-trained skill, then the pre-trained skill is invoked to determine the task action sequence of the current subtask; If the task execution annotation indicates that the current subtask supports a hard-coded skill, then the hard-coded skill is invoked to determine the task action sequence of the current subtask; The robot is controlled to execute a sequence of task actions to obtain the task execution result.

5. The method according to any one of claims 1-4, characterized in that, The step of verifying the current subtask to obtain the task verification result includes: Based on the current environmental perception data, the target task, and the historical execution status of the target task, the necessity of executing the current sub-task is verified, and the necessity verification result is obtained. If the first confidence level of the necessity verification result is greater than or equal to the first threshold, and the necessity verification result indicates that the current subtask is necessary to execute, the executability of the current subtask is verified based on the current environment perception data to obtain an executability verification result; the first confidence level is used to reflect the accuracy of the necessity verification result. Based on the executability verification results and the first confidence level, generate a task verification result for the current subtask.

6. The method according to claim 5, characterized in that, The step of generating a task verification result for the current subtask based on the executability verification result and the first confidence level includes: Obtain a second confidence level for the executability verification result; the second confidence level is used to reflect the accuracy of the executability verification result. If the second confidence level of the executability verification result is greater than or equal to the second threshold, and the executability verification result indicates that the current subtask is executable, then a task verification result is generated to indicate that the rearrangement conditions are not met. If the second confidence level is less than the second threshold, or the first confidence level is less than the first threshold, then the current subtask is re-verified to obtain the task verification result.

7. The method according to claim 6, characterized in that, The step of re-verifying the current subtask to obtain a task verification result when the second confidence level is less than the second threshold, or the first confidence level is less than the first threshold, includes: Collect the current environmental perception data again to obtain the collected environmental perception data; If the first confidence level is less than the first threshold, the necessity of executing the current sub-task is re-verified based on the collected environmental awareness data and the target task. If the re-verification determines that the current sub-task is not necessary to execute, a task verification result is generated to indicate that the rearrangement conditions are met. If the second confidence level is less than the second threshold, the executability of the current subtask is re-verified based on the collected environmental awareness data. If the re-verification determines that the current subtask is not executable, a task verification result is generated to indicate that the rearrangement conditions are met.

8. The method according to any one of claims 1-4, characterized in that, The method further includes: If the task execution result indicates that the current subtask has failed, then the current subtask is executed repeatedly. If the number of times the current subtask has failed is greater than or equal to the target number, then the task execution result is determined to satisfy the rearrangement condition. If the task execution result is a system error code returned when executing the current subtask, then the execution of the current subtask is paused, an exception prompt message is generated based on the system error code, the exception prompt message is reported, and if no exception handling instruction is received within the target time period after reporting the exception prompt message, it is determined that the task execution result meets the reordering condition.

9. The method according to any one of claims 1-4, characterized in that, The method further includes: During the execution of subtasks in the current subtask sequence, the task execution events of the current subtask sequence are acquired in real time, and environmental feedback events are collected in real time. The task execution events and the environmental feedback events are stored in the execution log related to the target task; When an abnormal event that meets the abnormal conditions exists in the execution log, the event type of the abnormal event is determined, and the abnormal event is processed based on the event type.

10. A hymenoidae robot system, characterized in that, The embodied robotic system is used for: The current subtask to be executed is determined from the current subtask sequence, and the current subtask is verified to obtain the task verification result; the current subtask sequence is obtained by decomposing the target task to be executed. If the task verification result does not meet the rearrangement conditions, the current subtask is executed to obtain the task execution result; If the task execution result satisfies the rearrangement condition, the task execution result, the execution context information of the current subtask, and the task objective of the target task are packaged to obtain structured text information. Based on the structured text information and the prompt word template, a rearrangement prompt is generated. Based on the rearrangement prompt, the target task is rearranged into subtasks to obtain a reconstructed subtask sequence. The current subtask sequence is updated using the reconstructed subtask sequence, and the step of determining the current subtask to be executed from the current subtask sequence is repeated until the target task is completed.

Images