A long-term behavior generation method for embodied intelligent robots based on thought chain strategy decomposition

Abstract

The application belongs to the technical field of embodied intelligent robots, and particularly relates to an embodied intelligent robot long-term behavior generation method based on thought chain strategy decomposition; the method comprises the following steps: constructing a primitive action unit library of a robot operation scene; establishing a coordinate system model based on three-dimensional atomic action semantics; performing hierarchical task planning based on a thought chain for a target task; generating a subtask sequence with a unified interface; converting action instructions according to a corresponding strategy generation module according to a subtask type; performing real-time state feedback and dynamic adjustment in an execution process; and continuously optimizing the model by using online reinforcement learning. The application realizes hierarchical decomposition of long-term tasks through thought chain reasoning, decouples high-level strategy planning and low-level action execution, retains the advantages of large model reasoning while avoiding the uncertainty of directly intervening in low-level control, and realizes adaptive, safe and reliable robot operation in a complex scene.

Description

Technical Field

[0001] This invention belongs to the field of embodied intelligent robot technology and relates to a method for generating long-term behavior of embodied intelligent robots based on thought chain strategy decomposition. It is particularly suitable for autonomous operation control of robots in complex operation scenarios such as electricity meter replacement. Background Technology

[0002] In the power industry, distribution network operation and maintenance (O&M) is a crucial component of ensuring the safe and stable operation of the power grid. With the deepening of smart grid construction, the demand for automation and intelligence in distribution network O&M is becoming increasingly urgent. Traditional distribution network O&M mainly relies on manual labor, which presents problems such as high frequency of operations, high labor intensity, high safety risks, and strong time constraints. With the development of robotics technology, "embodied intelligence + autonomous execution" has become an important approach to solving these problems.

[0003] Current robot control systems primarily target pre-programmed motion sequences at fixed workstations, controlling robots to complete specific tasks. However, in real-world applications such as meter replacement, robots need to operate in complex and dynamic environments, with numerous uncertainties in the workflow. Traditional control methods based on pre-programmed motion sequences struggle to handle these complex scenarios, primarily due to the following problems: (1) Complex environment: The environment at the meter installation site is varied, with problems such as narrow space, many interferences and obstacles. The robot needs to be flexibly adjusted according to the actual environment.

[0004] (2) Uncertain process: Various abnormal situations may occur during the replacement of electricity meters, such as equipment damage, position displacement, connection difficulties, etc. The robot needs to have the ability to detect and respond to abnormalities in real time.

[0005] (3) Long-term tasks: Meter replacement is a long-term and complex task involving multiple steps and actions. The robot needs to perform long-term planning and reasoning.

[0006] (4) Limited data: Traditional offline training data is difficult to cover all actual working conditions, and the robot's generalization ability is limited when facing new scenarios.

[0007] Therefore, practical tasks require transforming tasks into multiple behaviors, i.e., "behavior generation". "Behavior generation" can be defined as: under the constraints of multimodal semantic understanding and state awareness, and oriented towards long-term task objectives, the decision-making and execution process of unified planning, phased organization and dynamic scheduling of wheeled movement behavior and dual-arm operation behavior.

[0008] From a formal perspective, the long-term behavior generation problem can be abstracted into finding a set of optimal strategies in the state space that maximize the overall job benefit of the system while satisfying safety constraints throughout the entire task cycle. Its conceptual expression is shown in the following formula.

[0009]

[0010] in, Given the current state and mission objectives Under the given conditions, the optimal strategy that the system expects to adopt; Comprehensive characterization of the robot at time The multimodal state includes environmental perception results (such as key objects such as boxes and terminals identified by vision and their spatial relationships), robot's own state (pose, joint angles, gripper opening and closing and force feedback, etc.), and stage progress information. This indicates the current long-term task objective or high-level sub-task objective, such as "reach the designated meter box," "open the meter box door," or "complete meter insertion and reset." Action Indicates at time The selected control decision can correspond to the navigation control variable of the mobile chassis or the operation control variable of the robotic arm. (Function) Indicates at time Execute action The immediate gains (or the opposite of the costs) obtained afterward can be understood as a "comprehensive evaluation of the progress of the task and the risks and costs." For example, completing a phase goal will bring positive gains, while encountering collision risks, exceeding force thresholds, or unsafe postures will bring negative gains. (Symbol) This indicates an evaluation of uncertainties during the execution process (such as sensor noise, environmental changes, assembly tolerances, etc.) in the expected sense. Indicates the end time of the task or the end point of the planned vision.

[0011] Large language models excel in long-range reasoning, but they may not be stable and controllable in individual tasks. Directly using large language models for low-level control may introduce unpredictable risks. Therefore, how to fully utilize the reasoning advantages of large models while avoiding the uncertainties of direct intervention in low-level control has become a key issue in the control of embodied intelligent robots. Summary of the Invention

[0012] To address the shortcomings of the existing technologies, the present invention aims to provide a method for generating long-term behavior of embodied intelligent robots based on thought chain strategy decomposition. The present invention achieves hierarchical decomposition of long-term tasks through thought chain reasoning, decoupling high-level strategy planning from low-level action execution. In other words, it uses sub-tasks as a unified interface to abstract tasks into two categories: movement and operation. While retaining the advantages of large model reasoning, it avoids the uncertainty of directly intervening in low-level control, thus realizing adaptive, safe and reliable robot operation in complex scenarios.

[0013] The technical solution of the present invention is described in detail below.

[0014] This invention provides a method for generating long-term behavior of embodied intelligent robots based on thought chain strategy decomposition, comprising the following steps: Step 1: Based on the characteristics of the robot's task, define a set of primitive action units that include basic actions such as approach, detection, grasping, insertion, rotation, pulling, and release. Each primitive action unit is labeled with its representation parameters in the action coordinate system. Step 2: Construct a three-dimensional coordinate system model of atomic actions based on primitive actions and with "subject-predicate-object" as the semantic structure. Determine the executability of atomic actions in different work scenarios based on the feasibility constraints, execution conditions and target states of the actions. Step 3: Receive the description information of the target task, use the reasoning ability of the large language model to decompose the task hierarchically, and generate a sequence of sub-tasks with clear stage progression logic. Each sub-task includes the target object or target area, the expected completion status, and key safety constraints. Step 4: Based on the subtask type, call the corresponding strategy generation module to convert action instructions; during the execution of the subtask, monitor the environment status and execution feedback in real time. When the execution feedback indicates that the current subtask cannot meet the completion conditions, trigger the high-level strategy generation module to regenerate subsequent subtasks based on the latest status or trigger a local rollback strategy. Step 5: Based on the feedback from the execution results, optimize the parameters of the vision-language-action model through online reinforcement learning to improve the model's generalization ability and robustness in complex scenarios.

[0015] In this invention, in step three, the types of subtasks include movement-type subtasks and operation-type subtasks. The subtask outputs in the movement phase explicitly focus on "arrival conditions" and "safety thresholds for entering operations," while the subtask outputs in the operation phase explicitly focus on "object correctness, sequence dependencies, and exception handling," so that the generated subtask sequence reflects a clear stage progression logic.

[0016] In this invention, in step four, the strategy generation module includes a navigation strategy generation module and a vision-language-action model; the mobile subtask calls the navigation strategy generation module to generate movement commands through GNSS loopback SLAM and layered maps and nonlinear models; the operation subtask calls the vision-language-action model to generate action representations by combining environmental perception information, and converts them into action commands through an action decoder.

[0017] In this invention, the navigation strategy generation module includes: The GNSS loopback SLAM localization module is used to determine the robot's global position and orientation. The layered map building module divides the environment into map layers of different granularities; Nonlinear motion models are used to predict the motion trajectory of robots. The path planning algorithm module generates the optimal path based on map information and motion models.

[0018] In this invention, the visual-language-action model includes: A multi-view RGBD sensor input layer is used to acquire visual information about the environment; A language instruction encoder is used to convert task language instructions into vector representations; A visual feature extractor is used to extract environmental features from sensor input; An action representation generator combines language instructions and visual features to generate action representations. The motion decoder converts motion representations into specific robot control instructions.

[0019] In this invention, step four, the method for real-time monitoring of environmental status and execution feedback, includes: Environmental semantic uncertainty detection: When environmental semantics are uncertain, it triggers re-perception and task planning. Position accessibility detection: When the robot's position does not meet the operational accessibility requirements, position adjustment is triggered. Force anomaly detection: When the force sensor detects an abnormal force, it triggers a safety protection mechanism. Subtask completion condition check: If the subtask completion condition is not met, a retry or rollback strategy is triggered.

[0020] In this invention, step five, the online reinforcement learning optimization method includes: Design the reward function for execution results, based on task completion, security, and efficiency metrics; The model parameter update mechanism uses either policy gradient or actor-critic algorithms to update the model parameters; An experience replay buffer stores the state-action-reward-new state quadruple data during the execution process; Exploration – Utilizing a balancing strategy, maintaining a balance between implementing known strategies and exploring new strategies.

[0021] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) Decoupling between high-level and low-level: This invention achieves hierarchical decomposition of long-term tasks through thought chain reasoning, decoupling high-level strategy planning from low-level action execution, retaining the reasoning advantages of large models while avoiding the uncertainty of intervening in the low level.

[0022] (2) Reduce trajectory error: The separate navigation strategy generation module reduces trajectory error during movement, improving the robot's positioning accuracy and path planning accuracy.

[0023] (3) Possesses corrective capabilities: Allows repeated judgments, adjustments and executions, enabling the model to correct under failure conditions, thereby improving the robustness and reliability of the system.

[0024] (4) Adapting to complex scenarios: Through real-time status feedback and dynamic adjustment mechanisms, the system can adapt to complex and ever-changing working environments and respond to various abnormal situations in a timely manner.

[0025] (5) Continuous optimization: Through the online reinforcement learning mechanism, the system can continuously learn and improve from the execution process, thereby enhancing the system's generalization ability and long-term performance. Attached Figure Description

[0026] Figure 1 This is a flowchart of task planning and subtask generation based on thought chain disclosed in an embodiment of the present invention.

[0027] Figure 2 This is a schematic diagram of the navigation algorithm disclosed in an embodiment of the present invention.

[0028] Figure 3 This is a schematic diagram of the navigation algorithm disclosed in an embodiment of the present invention.

[0029] Figure 4 This is a schematic diagram of the action generation control strategy disclosed in an embodiment of the present invention.

[0030] Figure 5 This is a schematic diagram of the online reinforcement learning optimization mechanism disclosed in an embodiment of the present invention. Detailed Implementation

[0031] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0032] This invention provides a method for generating long-term behavior of embodied intelligent robots based on thought chain strategy decomposition, comprising the following steps: S10: Primitive Action Unit Library for Building Robot Operation Scenarios Based on the characteristics of the robot's electric meter tillage operation, a set of primitive action units is defined, which includes basic actions such as approach, detection, grasping, insertion, rotation, pulling, and release. Each primitive action unit is labeled with its representation parameters in the action coordinate system. S20: Establish a coordinate system model based on three-dimensional atomic action semantics A three-dimensional coordinate system model of atomic actions is constructed, based on primitive actions as the basic element and "subject-predicate-object" as the semantic structure. According to the feasibility constraints, execution conditions and target states of the actions, the executability of atomic actions in different work scenarios is determined, including meter replacement work order issuance, smart lock and box door opening, old meter value acquisition, meter reading camera and recognition, old meter disassembly based on uninterruptible power supply base plate, new meter installation based on uninterruptible power supply base plate, new meter value acquisition, meter reading camera and recognition, box closing and reset, etc.

[0033] S30: Perform hierarchical task planning for the target task based on the thought chain. When a meter replacement task is received, based on the description of the target task, the task is hierarchically decomposed using the reasoning ability of a large language model, generating a sequence of subtasks with clear phase progression logic. The subtask outputs in the movement phase explicitly focus on "arrival conditions" and "safety thresholds for entering the operation," while the subtask outputs in the operation phase explicitly focus on "object correctness, sequence dependencies, and exception handling." A specific implementation example discloses a flowchart of task planning and subtask generation based on the reasoning model, as shown below. Figure 1 As shown.

[0034] S40: Generate a sequence of subtasks with a unified interface. Each subtask includes a target object or target region, expected completion status, and key safety constraints. It employs a unified data structure interface to ensure the transitivity and executableness of the subtasks. This ensures that subtasks can be transferred and executed between different modules (high-level language model and lower-level navigation strategy generation and operation generation modules, etc.), providing a foundation for decoupling high-level strategy planning from low-level action execution. The unified interface data structure for subtasks includes: task identifier, target object / region description, expected completion status parameters, key safety constraints, prerequisite dependencies, and execution priority.

[0035] S50: Invoke the corresponding strategy generation module to convert action instructions based on the subtask type. Depending on the type of subtask, the corresponding strategy generation module is invoked to convert action commands. The strategy generation module includes a navigation strategy generation module and a vision-language-action model. For mobile subtasks, the navigation strategy generation module is invoked to generate movement commands through GNSS (Global Navigation Satellite System) loopback SLAM (Simultaneous Positioning and Mapping) and layered maps and nonlinear models. For operational subtasks, the vision-language-action model is invoked to generate action representations by combining environmental perception information, and then converted into action commands through an action decoder.

[0036] The navigation strategy generation module includes: a GNSS loopback SLAM localization module, used to determine the robot's global position and attitude; The invention comprises a layered map construction module that divides the environment into map layers of different granularities; a nonlinear motion model for predicting the robot's trajectory; and a path planning algorithm module that generates the optimal path based on map information and the motion model. This invention uses GNSS information as a global consistency constraint signal, triggering a loopback update only when the GNSS spatial position change trend aligns with visual / laser perception. It also proposes a two-layer map structure: a global layer describing the overall topology and a local layer focusing on real-time environmental changes. The layered map is combined with the nonlinear motion model for autonomous navigation behavior strategies. The navigation strategy generation module takes as input a movement subtask (target position, arrival conditions, safety constraints) and outputs movement commands (velocity, direction, acceleration). GNSS information is used as a global consistency constraint signal, triggering a loopback update only when the GNSS spatial position change trend aligns with visual / laser perception. Furthermore, it proposes a two-layer map structure: a global layer describing the overall topology and a local layer focusing on real-time environmental changes. The layered map is combined with the nonlinear motion model for autonomous navigation behavior strategies. The navigation strategy generation module's processing flow is as follows: Step 1: GNSS loopback SLAM positioning Multi-sensor fusion positioning (vision, laser, IMU, etc.); loop closure detection and optimization; GNSS as a global consistency constraint signal (loop closure update is triggered only when the GNSS spatial position change trend is consistent with visual / laser perception); Step 2: Layered Map Construction Global layer: describes the environmental topology and traversable areas (grid map or topology map); Local layer: focuses on real-time environmental changes around the robot (dynamic obstacles, temporary occlusions, etc.); updated in real time based on sensor data. Step 3: Nonlinear Motion Model Predicting robot trajectories; striking a balance between efficiency and safety; considering robot kinematic constraints. Step 4: Path Planning RRT algorithm: generates a feasible path from the current location to the target location; TOPP algorithm: performs time-optimized parameterization on the path and generates a velocity curve; generates the optimal path based on map information and motion model.

[0037] In a specific embodiment, the navigation algorithm is illustrated as follows: Figure 2 and Figure 3 As shown.

[0038] The vision-language-action model includes: a multi-view RGBD sensor input layer for acquiring visual information about the environment; a language instruction encoder for converting task language instructions into vector representations; a visual feature extractor for extracting environmental features from the sensor input; an action representation generator for generating action representations by combining language instructions and visual features; and an action decoder for converting action representations into specific robot control instructions. The inputs to the vision-language-action model are: multi-view RGBD sensor data: RGBD images from three views (global / left arm / right arm), each view including an RGB image and a depth map; language instructions: structured task descriptions (e.g., "open the meter box door," "remove the old meter," etc.); robot current state: 7-dimensional joint states, pose, end effector state, etc.; and output: robot control instructions (7-dimensional action vectors). The workflow of the vision-language-action model includes: Step 1: Visual Feature Extraction Environmental features are extracted from RGBD images from three perspectives; key objects (meter boxes, terminals, meters, etc.) and their spatial relationships are identified; and scene semantic information is extracted. Step 2: Language Encoding Convert task language instructions into vector representations; extract task semantic information and intent; generate task context vectors; Step 3: Action Representation Generation Generate motion representation by combining visual features and language commands; output a 7-dimensional motion vector (6-dimensional pose + 1-dimensional gripper opening and closing); Consider the current state and task constraints; Step 4: Motion Decoding Physical constraint mapping: Mapping 7-dimensional motion vectors to the robot control space; Safety boundary check: Checking whether the motion exceeds the safety range (collision detection, torque limit, etc.); Controller saturation limit: Ensuring that the control quantity is within the legal range.

[0039] A specific example of how the vision-language-action model generates action control strategies is illustrated below. Figure 4 As shown.

[0040] In this invention, in step S50, the corresponding strategy generation module is called according to the subtask type to realize the classification and processing of tasks. The movement subtask and the operation subtask are processed by the navigation strategy generation module and the vision-language-action model, respectively, which improves the modularity and maintainability of the system.

[0041] S60: Real-time status feedback and dynamic adjustment during execution. During the execution of subtasks, the environment status and execution feedback are monitored in real time. When the execution feedback indicates that the current subtask cannot meet the completion conditions, the high-level strategy generation module is triggered to regenerate subsequent subtasks based on the latest status or to trigger a local rollback strategy.

[0042] Real-time status feedback and dynamic adjustment include: Environmental semantic uncertainty detection: When environmental semantics are uncertain, it triggers re-perception and task planning. Position accessibility detection: When the robot's position does not meet the operational accessibility requirements, position adjustment is triggered. Force anomaly detection: When the force sensor detects an abnormal force, it triggers a safety protection mechanism. Subtask completion condition check: If the subtask completion condition is not met, a retry or rollback strategy is triggered.

[0043] In this invention, the real-time status feedback and dynamic adjustment mechanism in step S60 ensures the adaptability and robustness of the system during execution, enabling it to respond promptly to various abnormal situations.

[0044] S70: Continuously optimize the model using online reinforcement learning. By utilizing the Online VLA-specific RL module, the parameters of the vision-language-action model are updated based on the execution results, improving the model's generalization ability and robustness in complex scenarios. Online reinforcement learning optimizations include: The reward function for execution results is designed based on indicators such as task completion, security, and efficiency. The model parameter update mechanism uses algorithms such as policy gradient or actor-critic to update the model parameters; An experience replay buffer stores the state-action-reward-new state quadruple data during the execution process; Exploration – Utilizing a balancing strategy, maintaining a balance between implementing known strategies and exploring new strategies.

[0045] A specific example of an online reinforcement learning optimization mechanism is shown below. Figure 5 As shown.

[0046] In this invention, in step S70, the online reinforcement learning optimization mechanism enables the system to continuously learn and improve from the execution process, thereby enhancing the system's generalization ability and long-term performance.

[0047] Through the above steps, this invention realizes the generation of long-term behavior of embodied intelligent robots based on thought chain strategy decomposition, which can autonomously, safely and reliably complete complex tasks such as meter replacement in complex and ever-changing working environments.

[0048] This invention is applicable to various embodied intelligent robot application scenarios, especially those requiring long-duration tasks in complex and variable environments, such as power grid maintenance, industrial assembly, medical surgery, and home services. Through the method of this invention, robots can fully utilize the reasoning advantages of large language models while avoiding the uncertainties of direct intervention in low-level control, achieving adaptive, safe, and reliable operation.

[0049] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for generating long-term behavior of embodied intelligent robots based on thought chain strategy decomposition, characterized in that, Includes the following steps: Step 1: Based on the characteristics of the robot's task, define a set of primitive action units that include basic actions such as approach, detection, grasping, insertion, rotation, pulling, and release. Each primitive action unit is labeled with its representation parameters in the action coordinate system. Step 2: Construct a three-dimensional coordinate system model of atomic actions with primitive actions as the basic element and "subject-predicate-object" as the semantic structure. Determine the executability of atomic actions in different work scenarios based on the feasibility constraints, execution conditions and target states of the actions. Step 3: Receive the description information of the target task, use the reasoning ability of the large language model to decompose the task hierarchically, and generate a sequence of sub-tasks with clear stage progression logic. Each sub-task includes the target object or target area, the expected completion status, and key safety constraints. Step 4: Based on the subtask type, call the corresponding strategy generation module to convert action instructions; during the execution of the subtask, monitor the environment status and execution feedback in real time. When the execution feedback indicates that the current subtask cannot meet the completion conditions, trigger the high-level strategy generation module to regenerate subsequent subtasks based on the latest status or trigger a local rollback strategy. Step 5: Based on the feedback from the execution results, optimize the parameters of the vision-language-action model through online reinforcement learning to improve the model's generalization ability and robustness in complex scenarios.

2. The method for generating long-term behavior of an embodied intelligent robot according to claim 1, characterized in that, In step three, the types of subtasks include movement subtasks and operation subtasks. The output of subtasks in the movement phase explicitly focuses on "arrival conditions" and "safety thresholds for entering operations", while the output of subtasks in the operation phase explicitly focuses on "object correctness, sequence dependencies, and exception handling", so that the generated subtask sequence reflects a clear stage progression logic.

3. The method for generating long-term behavior of an embodied intelligent robot according to claim 2, characterized in that, In step four, the strategy generation module includes a navigation strategy generation module and a vision-language-action model; the movement subtask calls the navigation strategy generation module to generate movement commands through GNSS loopback SLAM and layered maps and nonlinear models; the operation subtask calls the vision-language-action model to generate action representations by combining environmental perception information, and converts them into action commands through an action decoder.

4. The method for generating long-term behavior of an embodied intelligent robot according to claim 3, characterized in that, The navigation strategy generation module includes: The GNSS loopback SLAM localization module is used to determine the robot's global position and orientation. The layered map building module divides the environment into map layers of different granularities; Nonlinear motion models are used to predict the motion trajectory of robots. The path planning algorithm module generates the optimal path based on map information and motion models.

5. The method for generating long-term behavior of an embodied intelligent robot according to claim 3, characterized in that, The visual-language-action model includes: A multi-view RGBD sensor input layer is used to acquire visual information about the environment; A language instruction encoder is used to convert task language instructions into vector representations; A visual feature extractor is used to extract environmental features from sensor input; An action representation generator combines language instructions and visual features to generate action representations. The motion decoder converts motion representations into specific robot control instructions.

6. The method for generating long-term behavior of an embodied intelligent robot according to claim 1, characterized in that, Step four includes methods for real-time monitoring of environmental status and execution feedback: Environmental semantic uncertainty detection: When environmental semantics are uncertain, it triggers re-perception and task planning. Position accessibility detection: When the robot's position does not meet the operational accessibility requirements, position adjustment is triggered. Force anomaly detection: When the force sensor detects an abnormal force, it triggers a safety protection mechanism. Subtask completion condition check: If the subtask completion condition is not met, a retry or rollback strategy is triggered.

7. The method for generating long-term behavior of an embodied intelligent robot according to claim 1, characterized in that, In step five, the methods for optimizing online reinforcement learning include: Design the reward function for execution results, based on task completion, security, and efficiency metrics; The model parameter update mechanism uses either policy gradient or actor-critic algorithm to update the model parameters; An experience replay buffer stores the state-action-reward-new state quadruple data during the execution process; Exploration – Utilizing a balancing strategy, maintaining a balance between implementing known strategies and exploring new strategies.

Images