An embodied robot control system and method integrating learning control and model control

Abstract

This invention discloses an embodied robot control system and method integrating learning control and model control, including a task parsing and recipe module, a mode routing switch, an LBC controller, an MBC controller, a state monitoring and feedback module, and an online mode adapter. This invention combines the advantages of LBC and MBC, employing precise MBC for the modelable stage and flexible LBC for the complex operation stage, avoiding the limitations of a single control mode. Furthermore, it decomposes long-cycle tasks into multiple short-cycle sub-tasks, each controlled independently, preventing error propagation across stages and significantly improving the success rate of long-cycle tasks.

Description

Technical Field

[0001] This invention relates to the field of robot control technology, specifically to an embodied robot control system and method that integrates learning control and model control. Background Technology

[0002] With the development of artificial intelligence technology, embodied intelligent robots are increasingly adopting learning-based control methods. LBC learns from human teaching data and uses models such as neural networks to fit control strategies in complex environments, making it particularly suitable for scenarios that are difficult to model explicitly, such as fine manipulation and decision-making in unstructured environments.

[0003] However, LBC faces several challenges in practical applications: training data is often sparse, making it difficult to cover all possible scenarios; and in long-cycle tasks, the accumulation of model prediction errors can easily lead to task failure. In contrast, model-based control (MBC) relies on accurate modeling of system dynamics, making it suitable for modelable scenarios such as motion control. It has high control accuracy and stability, but struggles to cope with complex and variable environments and unpredictable task phases.

[0004] In existing technologies, robot control systems typically employ either LBC or MBC alone, failing to fully utilize the advantages of both. For long-cycle tasks involving multiple phases with different characteristics (such as composite tasks of "movement-operation-movement-operation"), if LBC is used throughout, the accumulated error in the movement phase will severely affect subsequent operations; if MBC is used throughout, the operation phase is difficult to model, resulting in poor control performance.

[0005] Therefore, there is an urgent need for a fusion control system that can flexibly switch control modes according to the characteristics of the task stage. Summary of the Invention

[0006] The purpose of this invention is to provide an embodied robot control system and method that integrates learning control and model control, so as to solve the problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] An embodied robot control system integrating learning control and model control, including

[0009] The task parsing and recipe module receives long-cycle task instructions from external input, decomposes the task into multiple consecutive sub-task stages according to the execution logic, and configures a corresponding control mode identifier for each sub-task stage based on a preset recipe.

[0010] Furthermore, the pre-set recipes are manually set by technicians based on prior knowledge of the task, or automatically generated through offline simulation and iterative optimization. The recipes not only define the control modes corresponding to each sub-task stage, but also clearly set the stage switching conditions, including one or more combinations of time thresholds, position thresholds, visual sensor trigger signals, force sensor trigger signals, and end effector positioning signals.

[0011] The mode routing switch is connected between the task parsing and recipe module and the underlying controller. Based on the control mode identifier of the current subtask stage, the control is dynamically switched to the LBC controller or the MBC controller.

[0012] Furthermore, when the mode routing switch performs a control mode switch, it sequentially performs the following operations: pausing the current controller output, caching the robot's current pose state data, loading the initial parameters of the target controller, and restoring the control output. At the same time, it adopts transition control strategies such as linear interpolation and low-pass filtering to reduce the impact of mode switching on the robot's movement and ensure continuous and stable action.

[0013] The LBC controller is built based on a learning model and is used to handle sub-tasks such as grasping, assembling, and placing that are difficult to model explicitly through mathematical means. It takes robot state sensor data, environmental vision sensor data, and task target information as inputs and outputs corresponding motion control commands.

[0014] Furthermore, the LBC controller employs a deep neural network structure, using imitation learning or reinforcement learning to train control strategies based on human teaching data, enabling it to adapt to unstructured environments and flexible operation requirements.

[0015] The MBC controller is built based on the robot's physical and kinematic models. It is used to handle the movement, positioning, and trajectory tracking sub-tasks that can be accurately modeled. It takes robot state estimation information and preset reference trajectories as inputs and outputs high-precision motion control commands.

[0016] Furthermore, the MBC controller employs Model Predictive Control (MPC), Proportional-Integral-Derivative (PID) control, or inverse kinematics algorithm to achieve precise and stable motion control based on the robot's kinematic and dynamic models.

[0017] The status monitoring and feedback module collects robot operating status and environmental information in real time. When the phase switching conditions are met, it sends a mode switching signal to the mode routing switch to ensure a smooth transition of control modes.

[0018] The system also includes an online mode adapter. During task execution, if environmental changes exceed the preset range, unknown obstacles exist, or the control effect of the current control mode decreases, the online mode adapter can dynamically adjust the control mode of subsequent sub-task stages to improve the system's ability to cope with disturbances.

[0019] Corresponding to the above-mentioned control system, the present invention also provides a method for controlling an embodied robot that integrates learning control and model control, comprising the following steps:

[0020] Step S1: Receive long-cycle task instructions, decompose them into multiple consecutive sub-task stages based on the preset recipe, and configure LBC or MBC control mode for each stage.

[0021] Step S2: Initialize the controller corresponding to the first stage, and switch the mode routing switch to transfer control to this controller;

[0022] Step S3: The robot performs the task of the current stage, and the status monitoring and feedback module detects the end conditions of the stage in real time.

[0023] Step S4: When the phase end condition is detected, the mode routing switch smoothly switches to the corresponding controller according to the control mode identifier of the next phase.

[0024] Step S4: Repeat steps S3 to S4 until all sub-task stages have been completed.

[0025] Step S6: If environmental changes exceed expectations during task execution, the control mode of subsequent sub-task stages is dynamically adjusted through the online mode adapter.

[0026] Compared with the prior art, the beneficial effects of the present invention are:

[0027] 1. This invention combines the advantages of LBC and MBC, using precise MBC for the modelable stage and flexible LBC for the complex operation stage, avoiding the limitations of a single control mode, and decomposing long-cycle tasks into multiple short-cycle sub-tasks, each of which is controlled independently, blocking the transmission of errors across stages, and significantly improving the success rate of long-cycle tasks.

[0028] 2. Through pre-set recipes and dynamic routing, the system can adapt to different task scenarios. The online mode adapter further enhances the ability to cope with environmental changes and is modularly designed. It does not require modification of the internal structure of the original LBC and MBC controllers. It can be switched externally only through the routing switch, making it easy to integrate into existing robot systems. Attached Figure Description

[0029] Figure 1 This is a structural block diagram of the control system of the present invention.

[0030] Figure 2 This is a schematic diagram of the control method of the present invention. Detailed Implementation

[0031] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] Please see Figure 1 The present invention provides a technical solution:

[0033] An embodied robot control system integrating learning control and model control, including

[0034] The task parsing and recipe module is used to decompose tasks and configure control mode identifiers;

[0035] In this embodiment, the task parsing and recipe module receives the composite task instructions from external input, decomposes the overall task into a movement subtask, a grabbing subtask, a secondary movement subtask, and a placement subtask according to the action execution logic, and assigns a control mode to each stage according to the preset recipe. The movement stage and the secondary movement stage adopt the model control (MBC) mode, while the grabbing stage and the placement stage adopt the learning control (LBC) mode.

[0036] Based on the above embodiments, the pre-set recipe is automatically generated through offline simulation optimization. At the same time, stage switching conditions are set. When the robot moves to the target location area, a position threshold signal is triggered. When the grasping is completed, the end effector clamping force sensor signal is triggered. When the robot is placed in place, the vision sensor position recognition signal is triggered. The above signals serve as the basis for the end and switching of each sub-task stage.

[0037] A mode routing switch, which is used to dynamically switch control.

[0038] Furthermore, after receiving the switching signal, the mode routing switch first pauses the output of the currently running controller, caches the robot's current pose, joint velocity, and end effector torque status data, loads the initial parameters of the controller corresponding to the next stage, and resumes the control output after the parameter loading is completed, so that the robot can smoothly enter the next sub-task stage.

[0039] As a preferred option, the mode routing switch incorporates a linear interpolation transition strategy during mode switching, with a transition duration set to 0.5s. During the transition period, the output command of the current controller and the initial output command of the target controller are linearly weighted according to the time ratio to generate a transition control command output to the robot actuator, thereby avoiding motion jitter or motion interruption caused by control command jumps.

[0040] LBC controller, which is used to handle subtasks that are difficult to model explicitly.

[0041] In this embodiment, the LBC controller constructs a deep control model that integrates convolutional neural networks and recurrent neural networks. The convolutional neural network is used to extract the shape, position, and posture features of the target object acquired by the RGB-D vision camera, while the recurrent neural network is used to model the temporal dependencies of the robot's continuous actions. The model input consists of visual image data, robot joint position data, and end effector force sensor data, and the output consists of three-dimensional pose increment commands and gripping force control commands from the robot's end effector. The training process consists of two steps: the first step uses human-taught grasping and placement trajectory data for offline imitation learning to initialize the model parameters; the second step uses the Proximal Policy Optimization (PPO) reinforcement learning algorithm in the Gazebo simulation environment for iterative optimization, updating the model parameters with grasping success rate and placement accuracy as the core reward functions, and finally obtaining an operation control strategy that can adapt to different object shapes, materials, and placement postures.

[0042] The MBC controller is used to process modelable subtasks.

[0043] Furthermore, the MBC controller employs an algorithm combining model predictive control and inverse kinematics: first, a link kinematic model is established based on the robot's DH parameters, and a dynamic model is established based on the mass, moment of inertia, and center of mass position parameters of each link; the inverse kinematics module solves for the target angle sequence of each joint of the robot according to the preset Cartesian space reference trajectory; the model predictive control module uses the current joint angle and angular velocity as initial values, optimizes and solves for the torque control quantity of each joint within a finite time domain of 0.2s, and outputs it to the robot joint actuator to achieve high-precision trajectory tracking and positioning during the movement phase, with a positioning accuracy of ±2cm.

[0044] The status monitoring and feedback module is used to monitor the status and send switching signals to achieve a smooth transition.

[0045] In this embodiment, the state monitoring and feedback module collects robot state information in real time. The sensors used include an RGB-D vision camera, a joint position encoder, an end effector six-dimensional force / torque sensor, and an inertial measurement unit (IMU). These sensors are all conventional components in the field of robot control and are used to acquire environmental images and target object information, robot joint angle information, end effector contact force and torque information, and robot posture and acceleration information, respectively.

[0046] In this embodiment, the status monitoring and feedback module continuously determines whether the stage switching conditions are met. When the switching conditions are detected to be met, a mode switching command is immediately sent to the mode routing switch.

[0047] This embodiment takes the execution of a complex long-cycle task involving movement, grasping, secondary movement, and placement by an embodied intelligent robot as an example to provide a detailed description of the control system and method that integrates learning control and model control. Those skilled in the art can directly implement this embodiment without creative effort.

[0048] As a preferred option, the system enables an online mode adapter. The adapter collects obstacle detection results from the vision sensor and the deviation data of the robot's actual trajectory in real time, setting the trajectory deviation threshold to ±5cm. When a sudden obstacle is detected in the environment and the actual trajectory deviation exceeds the threshold, the online mode adapter sends a temporary switching command to the mode routing switch, switching the local road segment currently controlled by MBC to LBC mode, and the LBC controller outputs obstacle avoidance action commands. When the obstacle disappears and the trajectory deviation returns to within the threshold, it automatically switches back to MBC mode to continue executing the original reference trajectory, improving the robot's obstacle avoidance and dynamic environment adaptability.

[0049] Please see Figure 2 Based on the above embodiments, the control method is executed according to the following preset steps:

[0050] First, receive the composite task instruction and complete the subtask division and control mode allocation.

[0051] The first-stage MBC controller is then initialized to execute the movement task. Upon reaching the target grasping position, it smoothly switches to the LBC controller to execute the grasping task.

[0052] After the capture is completed, switch back to the MBC controller to perform a second movement task. After reaching the target placement location, switch back to the LBC controller to complete the placement task.

[0053] The entire task ends after all stages are completed in sequence.

[0054] Furthermore, during each stage of execution, the status monitoring and feedback module continuously collects information from vision, position, and force sensors to monitor the robot's operating status and task execution progress in real time, ensuring timely stage switching and normal operation of the control mode;

[0055] Furthermore, the online mode adapter is on standby in real time, providing dynamic mode adjustment in case of abnormal operating conditions to ensure reliable task completion.

[0056] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A control system for an embodied robot that integrates learning control and model control, characterized in that, include The task parsing and recipe module is used to decompose tasks and configure control mode identifiers; A mode routing switch, which is used to dynamically switch control rights; LBC controller, which is used to handle subtasks that are difficult to model explicitly; MBC controller, which is used to process modelable subtasks; The status monitoring and feedback module is used to monitor the status and send switching signals to achieve a smooth transition.

2. The system according to claim 1, characterized in that: The control mode identifier is determined based on a preset recipe, which defines whether each sub-task stage adopts either the Learning Control (LBC) mode or the Model Control (MBC) mode.

3. The system according to claim 2, characterized in that: The preset recipe is generated by manual pre-setting or offline simulation optimization, and also includes at least one stage switching condition among time threshold, position threshold, and sensor trigger signal.

4. The system according to claim 1, characterized in that: The LBC controller handles subtasks that are difficult to model explicitly, with sensor data and task objectives as inputs and action commands as outputs; the MBC controller handles modelable subtasks, with state estimates and reference trajectories as inputs and precise control commands as outputs.

5. The system according to claim 1, characterized in that: When the mode routing switch is switched, the output is paused, the buffer state is buffered, parameters are loaded, and the output is resumed. Linear interpolation or filtering transition strategies are used to reduce the impact.

6. The system according to claim 1, characterized in that: The LBC controller is obtained by using a deep neural network and training through imitation learning or reinforcement learning; the MBC controller uses MPC, PID or inverse kinematics solving algorithms.

7. The system according to claim 1, characterized in that: It also includes an online mode adapter for dynamically adjusting subsequent control modes in case of environmental anomalies.

8. A control method for an embodied robot that integrates learning control and model control, based on the system described in any one of claims 1-7, characterized in that, Includes the following steps: Step S1: Receive long-cycle task instructions, decompose them into sub-task stages based on the preset recipe, and configure LBC or MBC control mode. Step S2: Initialize the first-stage controller; the mode routing switch will transfer control to the corresponding controller. Step S3: Execute the task for the current stage and detect the stage termination conditions in real time; Step S4: When the termination condition is met, smoothly switch the controller according to the next stage control mode indicator. Step S5: Repeat steps S3-S4 until all sub-task stages have been completed.

9. The method according to claim 8, characterized in that: In step S4, the smooth switching includes pausing output, buffering state, loading parameters, and resuming output, and a transition control strategy is adopted to reduce the switching impact.

10. The method according to claim 8, characterized in that: It also includes step S6, online adaptation, where the subsequent control mode is adjusted through the online mode adapter when the environment exceeds expectations during task execution.

Images