Chain type hybrid deep reinforcement learning quadruped robot control system and control method
The quadruped robot control system, which utilizes chain-based hybrid deep reinforcement learning, solves the problems of low training efficiency, insufficient generalization ability, and poor system compatibility, achieving efficient and stable quadruped robot motion control that adapts to complex terrain and environmental changes.
Patent Information
- Application Number
- CN202511989959.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-26
- Publication Date
- 2026-04-10
AI Technical Summary
Existing quadruped robots suffer from low training efficiency, insufficient generalization ability, poor system compatibility, and poor high-biomimetic motion control in complex terrain environments, making it difficult to meet practical application requirements.
The control system employing chained hybrid deep reinforcement learning includes a perception module, a data processing module, a chained hybrid reinforcement learning model module, a control output module, and a feedback module. It combines high-precision motion capture equipment, lidar, IMU inertial measurement unit, and plantar pressure sensor. Through attention mechanism spatiotemporal graph neural network and adversarial imitation learning, it constructs a multi-scenario simulation environment. It adopts an improved PPO algorithm and meta-reinforcement learning, and integrates specialized expert models to achieve unified and lightweight control.
The training cycle is shortened by 50%, the control accuracy is maintained at ≥90%, the gait is continuous and natural, the joint torque fluctuation is reduced by 30%, and it is suitable for deployment needs in complex scenarios such as industrial inspection and emergency rescue.
Smart Images

Figure CN121832583A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application belongs to the technical field of robots, and particularly relates to a chain type hybrid deep reinforcement learning quadruped robot control system and a control method. BACKGROUND
[0002] At present, with the continuous expansion of special operation scenes such as industrial inspection, emergency rescue and polar exploration, quadruped robots have become the core equipment for replacing manual execution of complex tasks due to their strong mobility and terrain adaptability. At present, the complex terrain (such as soft ground, obstacle dense area and dynamic rugged terrain) environment puts forward strict requirements for the motion stability, gait naturalness and environmental self-adaptability of the robot, and the traditional control method has been difficult to meet the actual application requirements. Specifically, the existing quadruped robot motion control technology based on deep reinforcement learning has shown potential in autonomous decision-making, but still has three major core bottlenecks: firstly, the training efficiency is low, the pure reinforcement learning method lacks motion prior knowledge, needs a large number of interactive samples to converge, and the training process is easy to fall into local optimum, resulting in gait deformity and low motion efficiency; secondly, the generalization ability is insufficient, the model trained in a single simulation environment has a "simulation-reality gap" in the real scene, and the control accuracy will be greatly reduced when facing the fluctuation of dynamic parameters (such as load change and joint wear) and complex terrain switching; thirdly, the system compatibility is poor, different motion modes such as walking, running, climbing and obstacle crossing need to train multiple expert models independently, which increases the system complexity and engineering deployment difficulty, and the gait is easy to break and unstable when multiple models are switched.
[0003] At the same time, the existing technology has obvious short boards in the aspects of bionic gait restoration degree, real-time response speed and energy consumption optimization, for example: the end-to-end control method is difficult to accurately reproduce the joint coordination motion mechanism of real animals, resulting in gait stiffness; the dynamic characteristics of the environment (such as sudden obstacles and sudden changes in ground friction coefficient) are not considered when migrating across environments, and the adaptability is limited; the multi-model architecture increases the hardware computing burden, and it is difficult to meet the computing power demand of real-time control of the robot. Therefore, an efficient training, strong generalization ability, unified control and high bionics motion control scheme is needed to break through the existing technical bottlenecks and promote the large-scale application of quadruped robots in complex actual scenes. SUMMARY
[0004] The purpose of the application is to overcome the technical defects of low training efficiency, insufficient generalization ability, poor system compatibility and poor high bionics motion control of the current quadruped robot, and provide a chain type hybrid deep reinforcement learning quadruped robot control system and a control method.
[0005] The application aims to realize the above-mentioned purpose by the following technical scheme: a chain mixed deep reinforcement learning quadruped robot control system, comprising: a perception module, used for collecting bionic motion data, robot state data and environment data; a data processing module, used for carrying out denoising, normalization and feature extraction standardization processing on the data collected by the perception module; a chain mixed reinforcement learning model module, used for deploying a chain mixed deep reinforcement learning model, receiving the standardized data of the data processing module, and generating joint control instructions; a control output module, used for converting the joint control instructions into driving signals of a robot executor; and a feedback module, used for collecting motion execution result data of the robot executor, and feeding back to the data processing module and the chain mixed reinforcement learning model module, so as to realize closed-loop control and online optimization of the model.
[0006] Further, the perception module at least comprises a high-precision motion capture device, a laser radar, an IMU inertial measurement unit and a plantar pressure sensor, and the collection accuracy of the high-precision motion capture device satisfies the joint angle error ≤0.5°, and the ground reaction force measurement error of the plantar pressure sensor ≤5%.
[0007] The application further provides a control method of the chain mixed deep reinforcement learning quadruped robot control system, comprising the following steps:
[0008] S1, collecting bionic motion data, robot state data and environment data, training a reference gait model containing motion prior based on an attention mechanism spatio-temporal graph neural network and adversarial imitation learning;
[0009] S2, constructing a three-level parallel simulation environment, setting progressive course objectives and training the same by using an improved PPO algorithm, and optimizing gait stability, motion efficiency and environmental adaptability;
[0010] S3, building a differentiated simulation environment with multiple dynamics parameters and multiple ground types, combining meta-reinforcement learning and domain adversarial neural networks, narrowing the simulation reality gap, and improving the robustness of the model;
[0011] S4, training special expert models for five core motion modalities, and fusing the same into a unified lightweight control model by using a distillation algorithm and a gait transition loss function;
[0012] S5, based on multi-sensor data fusion, using a unified model to output joint control instructions, combining model predictive control and online incremental learning, and realizing closed-loop adaptive control.
[0013] Further, in step S1, the bionic motion data, robot state data and environment data are collected by using a motion capture system with a sampling frequency ≥240Hz, and a multi-scenario bionic motion data set is constructed.
[0014] As a preferred mode, the reference gait model in step S1 also adds a joint angle constraint and a torque smoothness regularization term.
[0015] The "three-level parallel simulation environment" in step S2 includes a basic static environment, a complex static environment and a dynamic disturbance environment, and the progressive course objectives at least include stability, efficiency and adaptability.
[0016] Further, the "improved PPO algorithm" in step S2 is a PPO-Lag algorithm.
[0017] The number of "differentiated simulation environments" in step S3 is ≥10, the meta-reinforcement learning is a MAML algorithm, and the domain adversarial neural network is a DANN network.
[0018] The "five core motion modalities" in step S4 are walking, fast walking, running, climbing and obstacle crossing.
[0019] The distillation algorithm is a KD-MoE distillation algorithm, and the parameter amount of the control model is ≤5 million.
[0020] Compared with the prior art, the present application has the following advantages and beneficial effects:
[0021] (1) The present application relies on high-precision data acquisition with a sampling frequency ≥240Hz and adversarial imitation learning, introduces motion prior knowledge, cooperates with joint angle constraints and torque smoothness regularization, shortens the training period by more than 50%, avoids abnormal gait, and realizes controllable training process.
[0022] (2) The present application is trained in ≥10 differentiated simulation environments, combines MAML meta-reinforcement learning and DANN domain adversarial network, can effectively reduce the simulation and reality gap, and when facing terrain switching and hardware parameter fluctuation, the control accuracy retention rate is ≥90%, and the robustness is outstanding.
[0023] (3) The present application adopts KD-MoE distillation algorithm to fuse five core motion modalities such as walking and fast walking, forms a lightweight unified model with a parameter amount ≤5 million, does not need multiple model switching, cooperates with gait transition loss function, and the gait is continuous and natural, and the joint torque fluctuation is reduced by 30%.
[0024] (4) The present application is based on multi-sensor data fusion (joint angle error ≤0.5°, pressure measurement error ≤5%) and online incremental learning, combined with closed-loop feedback control, the control instruction reasoning delay is ≤10ms, and it is suitable for deployment requirements of complex actual scenes such as industrial inspection and emergency rescue. BRIEF DESCRIPTION OF DRAWINGS
[0025] Figure 1This is a schematic diagram of the overall structure of the present invention;
[0026] Figure 2 This is a schematic diagram of the overall process of the present invention.
[0027] The reference numerals in the above figures are: 1-Perception module, 2-Data processing module, 3-Chained hybrid reinforcement learning model module, 4-Feedback module, 5-Control output module, 6-Robot actuator. Detailed Implementation
[0028] The present invention will be further described in detail below with reference to embodiments, but the implementation of the present invention is not limited thereto.
[0029] Example 1
[0030] like Figure 1 As shown, the quadruped robot control system of chained hybrid deep reinforcement learning in this embodiment consists of six parts: perception module 1, data processing module 2, chained hybrid reinforcement learning model module 3, feedback module 4, control output module 5, and robot actuator 6.
[0031] The sensing module 1 is used to collect biomimetic motion data, robot state data and environmental data. The sensing module 1 includes at least a high-precision motion capture device, a lidar, an IMU inertial measurement unit and a plantar pressure sensor.
[0032] The biomimetic motion data refers to the robot motion prior generated by processing the quadrupedal animal's motion trajectory acquired through high-precision motion capture equipment using a Spatio-Temporal Motion Retargeting (STMR) unit. This unit utilizes high-precision motion capture equipment (sampling frequency 240Hz) to extract key skeletal points of the animal and maps the animal's anatomical structure to the robot's twelve-DOF configuration space using the STMR spatial mapping operator and an inverse kinematics (IK) constraint solver. This approach maximizes the mapping of animal trajectory features to quadrupedal robot joint references while ensuring that movements remain within the robot's physical constraints. The data includes: retargeted joint space parameters (corresponding to the robot's joint angles, angular velocities, and angular accelerations); spatio-temporal cooperative features (body center of mass trajectory and foot swing / support phase relationship); and semantic adaptation features (extracting the animal's center of gravity shift strategy and ground contact phase distribution under different slopes and steps), ensuring that the original biological data is converted into a reference benchmark with robot physical feasibility.
[0033] The robot state data refers to the real-time collected four-legged robot running state data, which includes: joint state, such as the actual angle, torque, rotating speed and working current of the servo motor of the 12 joints of the four-legged robot; body posture, i.e. the pitch angle, roll angle, yaw angle and angular velocity, acceleration of the body collected by the IMU inertial measurement unit; motion state, i.e. the moving speed, direction, center of gravity position and contact state of the foot bottom with the ground of the four-legged robot; and robot executor state, such as the transmission efficiency of the joint harmonic reducer and the equipment running parameters such as motor temperature.
[0034] The environmental data refers to the external environmental feature data of the robot collected by the sensor, which includes: terrain features, such as the terrain type (flat ground, slope, step, soft ground) collected by the laser radar, slope (0°-30°), step height (0-15cm) and obstacle position and size; environmental physical parameters, including but not limited to ground friction coefficient, hardness and environmental temperature, humidity; contact feedback, i.e. the ground reaction force (measurement error ≤5%) collected by the foot bottom pressure sensor, used to judge the ground support stability.
[0035] The data processing module 2 is connected with the perception module 1 and interacts with the data, and the data processing module 2 is used for the denoising, normalization and feature extraction standardization processing of the data collected by the perception module 1. Among them, the data denoising refers to eliminating the interference signal, which is used to protect the data authenticity; the data normalization refers to unifying the dimension scale and adapting the model training, so as to eliminate the dimension difference of different types of data (such as joint angle unit °, pressure unit N, speed unit m / s), map the data to a unified interval, and avoid the influence of numerical range on model training; the feature extraction refers to mining key information and simplifying model input, from the standardized original data, extracting high-dimensional features which have core influence on motion control, and reducing the model calculation complexity.
[0036] The chain mixed reinforcement learning model module 3 is connected with the data processing module 2, the feedback module 4 and the control output module 5 respectively, and is used for deploying the chain mixed deep reinforcement learning model, receiving the standardized data of the data processing module 2, and generating joint control instructions.
[0037] In this embodiment, the chain mixed reinforcement learning model module 3 at least integrates a bionic imitation learning sub-module, a hierarchical curriculum reinforcement learning sub-module, a cross-domain adaptive learning sub-module, a multi-expert model distillation sub-module, a real-time decision and online optimization sub-module and a hardware adaptation layer.
[0038] The bionic imitation learning sub-module is deployed with an attention mechanism space-time graph neural network (ST-GNN), an adversarial imitation learning (AIL) framework, and a multi-scenario bionic motion dataset (including joint angle constraint and torque smoothness regularization modules). The hierarchical curriculum reinforcement learning sub-module integrates an improved PPO algorithm (PPO-Lag algorithm), a distributed training engine (Ray RLlib), and an interface for a "basic static - complex static - dynamic interference" three-level parallel simulation environment.
[0039] The cross-domain adaptive learning sub-module is embedded with a MAML meta-reinforcement learning algorithm and a DANN domain adversarial neural network, supports joint training of ≥10 differentiated simulation environments, and includes random dynamics disturbance and uncertainty quantization units. The multi-expert model distillation sub-module includes walking, fast walking, running, climbing, and obstacle-clearing five motion modal expert models, deploys a KD-MoE distillation algorithm and a gait transition loss function, and outputs a unified lightweight model with a parameter quantity ≤5 million.
[0040] The real-time decision and online optimization sub-module integrates a multi-sensor data fusion interface, a model predictive control (MPC) unit, and an incremental learning engine, and supports real-time inference of control instructions.
[0041] The hardware adaptation layer adapts to embedded hardware such as NVIDIA Jetson AGX Orin through quantitative perception training (QAT) and pruning technology, supports multi-protocol data interaction such as CAN bus and Ethernet, and the like.
[0042] The control output module 5 is connected with the robot executor 6, and is used to convert the obtained joint control instruction into a driving signal of the robot executor 6.
[0043] The feedback module 4 is connected with the robot executor 6, and is used to collect motion execution result data of the robot executor 6, and feed the data back to the data processing module 2 and the chain mixed reinforcement learning model module 3, so as to realize closed-loop control and online optimization of the model.
[0044] Embodiment 2
[0045] This embodiment is an implementation method of the control system based on embodiment 1, and the overall process is as shown in Figure 2 The embodiment includes five steps S1-S5. Specifically:
[0046] S1, collect bionic motion data, robot state data and environment data, and train a reference gait model containing motion priors based on an attention mechanism space-time graph neural network and adversarial imitation learning.
[0047] This step is the initialization step of the biomimetic imitation learning, wherein the biomimetic motion data, robot state data and environment data are collected by using a high-precision motion capture device with a sampling frequency of ≥240Hz. The details of the data collection have been described in Embodiment 1 and will not be repeated here. After collecting the above data, a multi-scene biomimetic motion dataset needs to be constructed. The multi-scene described herein includes: basic static scenes, such as flat hard ground, for collecting basic gait data of standard walking, fast walking and running; complex static scenes, such as slopes within 30°, steps within 15cm, soft ground, for collecting terrain adaptability gait data; and light dynamic scenes, such as slight ground vibration and small obstacle distribution area, for collecting gait adjustment data under simple interference.
[0048] The construction process of the multi-scene biomimetic motion dataset includes in turn: data screening, for removing abnormal data and retaining valid motion samples; time-space synchronization, for aligning the timestamps of the biomimetic motion data, robot state data and environment data based on the clock of the motion capture system, to ensure that each frame of data corresponds to a scene and a motion state; data labeling, for labeling each sample with scene type, motion mode (walking / fast walking / running / climbing / obstacle crossing), environment parameters (such as slope 30°, friction coefficient 0.6), etc.; dataset division, for dividing the dataset into training set (for model training), validation set (for parameter tuning) and test set (for model performance verification) in the ratio of 7:2:1, to finally form a multi-scene biomimetic motion dataset of valid samples, providing high-quality input for subsequent imitation learning.
[0049] The attention mechanism spatio-temporal graph neural network (ST-GNN) is used to solve the problem of modeling the spatio-temporal coordination relationship of the four limbs of a quadruped robot, i.e., taking the 12 joints of the quadruped robot as network nodes and the mechanical connection relationship between the joints as edges, and dynamically adjusting the edge weights according to the motion correlation. The attention mechanism refers to the introduction of a spatial attention module and a temporal attention module, wherein the spatial attention module is used to capture the importance of the motion of different joints, and the temporal attention module is used to focus on key timing nodes within a gait cycle.
[0050] The adversarial imitation learning (AIL) integrates a generator, a discriminator and an adversarial training logic. The generator is the above-mentioned spatio-temporal graph neural network (ST-GNN) network, responsible for learning biomimetic motion features and generating a robot reference gait; the discriminator is implemented by using a fully connected neural network, for distinguishing between "real animal motion data" and "gait data generated by the model"; and the adversarial training logic forces the generator to generate a natural gait closer to the real animal through the game between the generator and the discriminator, to avoid abnormal motion.
[0051] The reference gait model for motion prior refers to a set of biomimetic gait templates and adaptive rules trained based on the movement patterns of real quadrupeds (motion prior). Its data source is high-precision motion capture data (quadrupeds such as dogs) with a sampling frequency of ≥240Hz, covering multiple scenarios such as flat ground, slopes, and stairs; its basic network is a spatiotemporal graph neural network with attention mechanism (ST-GNN), using the robot's 12 joints as nodes and mechanical connections as edges, dynamically allocating joint motion weights; its training framework is a fusion of adversarial imitation learning (AIL) and behavior cloning (BC), using generator-discriminator game to ensure that the gait closely resembles the movement patterns of real animals.
[0052] To ensure practical effectiveness, joint angle constraints and torque smoothness regularization terms are also incorporated into the reference gait model. The joint angle constraints set physical motion limits for the 12 joints of the quadruped robot (4 each for hip, knee, and ankle), serving as a rigid constraint on the reference gait model to prevent joints from exceeding the mechanical structure's capacity. The torque smoothness regularization term constrains the rate of change of joint torque through regularization methods, preventing sudden spikes or drops in torque, ensuring smooth transitions in joint movement, and improving gait naturalness and mechanical lifespan. Its mathematical expression is as follows: ,in, Let λ be the joint torque at time t, λ be the penalty weight, and T be the number of frames in the gait cycle.
[0053] S2. Construct a three-level parallel simulation environment, set progressive course objectives, and train it using an improved PPO algorithm to optimize gait stability, movement efficiency, and environmental adaptability.
[0054] This step is a tiered curriculum reinforcement learning step, in which the three-level parallel simulation environment includes a basic static environment, a complex static environment, and a dynamic disturbance environment. The term "parallel" here refers to the use of the Ray RLlib distributed framework, which allows the three environments to train simultaneously and share data in real time, avoiding the limitations of training in a single environment.
[0055] The applicable scenario of the basic static environment is a flat hard ground without obstacles and external interference. At this time, the dynamics parameters of the quadruped robot are fixed, and the twelve-degree-of-freedom structure is adapted to verify the basic stability of the reference gait. The complex static environment is suitable for multiple terrain combinations, such as 0°-30° slope, 5-15 cm step, soft ground, etc., and is accompanied by fixed obstacles with a spacing of 0.5-1 m. At this time, the dynamics parameters of the quadruped robot are introduced to a small range of fluctuations (load mass 4-6 kg, joint damping coefficient 0.08-0.12), which is used to train the model to adapt to static complex terrain and break through the control limitations of a single terrain. The dynamic interference environment is suitable for adding dynamic interference on the basis of the complex static environment. At this time, the dynamics parameters of the quadruped robot change dynamically (simulate joint wear, damping coefficient 0.05-0.15; load mass 3-7 kg), which is used to train the model to resist interference in dynamic environments and hardware parameter fluctuations, and to close to real application scenarios.
[0056] The progressive course objectives at least include stability, efficiency and adaptability, which correspond to the three-level environment one by one, and the objectives are upgraded gradually. Specifically, in the first stage, the stability objective corresponds to the basic static environment, the training focus is to optimize joint collaborative control and correct minor deviations of the reference gait to avoid gait deformity and robot body collapse. In this stage, the robot body pitch / roll angle fluctuation is required to be ≤3°, the foot bottom landing success rate is required to be ≥98%, and there is no joint over-limit displacement (≤±90°). In the second stage, the efficiency objective corresponds to the complex static environment, the training focus is to introduce an energy consumption penalty term (add motor power weight in the loss function) to optimize the gait cycle (step length / step frequency match terrain) and improve the motion economy. In this stage, the motion speed on flat ground is required to be 0.5-1.5 m / s, the walking speed on a 30° slope is required to be ≥0.3 m / s, and the energy consumption is required to be reduced by 20% compared with the initial reference gait. In the third stage, the adaptability objective corresponds to the dynamic interference environment, the training focus is to strengthen the model's ability to quickly identify environmental features and optimize the joint fine-tuning strategy. In this stage, the gait is required to have no pause / turbulence when switching terrains, the control accuracy retention rate is required to be ≥90% under dynamic interference, and the obstacle avoidance response time is required to be ≤0.2 s.
[0057] The improved PPO algorithm in this step is the PPO-Lag algorithm. The core improvements of this algorithm are adaptive learning rate, multi-environment reward fusion and adaptive clipping of advantage function. Among them, for the adaptive learning rate, the basic environment learning rate is required to be 1e-4 and the dynamic interference environment is required to be 1e-3, which is dynamically adjusted according to the convergence speed to avoid local optimum. For multi-environment reward fusion, it sets differentiated reward weights for the three-level environment, specifically 40% for stability, 30% for efficiency and 30% for adaptability, which fits the progressive objectives. For adaptive clipping of advantage function, the clipping threshold is dynamically adjusted to balance the exploration and utilization efficiency of the model.
[0058] S3, build a multi-dynamics parameter and multi-ground type differentiated simulation environment, combine meta-reinforcement learning and domain adversarial neural network, narrow the simulation reality gap, and improve the model robustness.
[0059] This step is the virtual-real migration content in the quadruped robot gait training process, and the core goal is to solve the simulation-to-reality gap problem through environment diversification construction and algorithm collaborative optimization, so that the gait strategy trained in simulation can be stably deployed to the real quadruped robot, and at the same time has strong kinetic robustness and environmental adaptability.
[0060] In this step, the core of the meta-reinforcement learning is to enable the model to quickly adapt in different dynamics or terrain tasks, rather than single-task specialized training. The core of the meta-reinforcement learning is the MAML algorithm, which is combined with PPO (Proximal Policy Optimization) to ensure the speed and performance retention rate of cross-task adaptation. The domain adversarial neural network is a DANN network, and this step integrates differentiated simulation, meta-reinforcement learning and DANN network together to form a complete Sim2Real migration solution. Among them, the number of differentiated simulation environments is ≥10, which is used to provide "stress test" scenarios to expose the vulnerability of the strategy under different dynamics / terrain, and provide diversified training data for the DANN network. The meta-reinforcement learning is used to train the rapid adaptation ability of the strategy, so that the domain-invariant features learned by the DANN network can quickly adapt to the new environment; the DANN network eliminates the domain difference from the feature level, so that the meta-strategy can maximize the performance in the real environment, and achieve the goal of stable, efficient and adaptive gait training.
[0061] S4, train special expert models for five core motion modalities, and fuse them into a unified lightweight control model through distillation algorithm and gait transition loss function.
[0062] Among them, the five core motion modalities are walking, fast walking, running, climbing and obstacle crossing. The core advantage of the expert model is to focus on a single mode and avoid multi-task conflicts, so the training converges faster (30-50% faster than general models).
[0063] The knowledge distillation (KD) is a "teacher-student" learning framework, which uses a large and strong teacher model (multi-expert integration) to guide a small and fast student model (lightweight unified model), so that the student inherits the generalization ability and decision logic of the teacher. In this embodiment, the distillation algorithm is implemented by using a KD-MoE distillation algorithm. The KD-MoE distillation algorithm is a fusion algorithm of Knowledge Distillation (KD) and Mixture of Experts (MoE), which is a method of constructing a high-performance teacher model by MoE and then migrating multi-expert knowledge to a lightweight student model by KD. It not only retains the professionalism of each motion modality, but also realizes the unification and lightweight of the model.
[0064] The KD-MoE distillation algorithm includes the following three steps: first, the pre-training stage, the five special expert models are independently trained, and then the gating network is trained to form a complete MoE teacher model; second, the distillation stage, that is, fixing the MoE teacher model parameters, training the student model with a large amount of multi-modal scene data, and transferring knowledge through the above loss function; finally, the fine-tuning stage, that is, adding "gait transition loss" to optimize the connection effect between modalities and ensure that there is no jitter in the switching of walking → trotting and trotting → jumping.
[0065] In this step, the parameter amount of the student model is required to be ≤5 million, which is only 1 / 10 of the MoE teacher model, the reasoning delay is ≤10 ms, which meets the real-time control requirement, and the performance difference between each modality and the special expert model is ≤5%.
[0066] S5, based on multi-sensor data fusion, using a unified model to output joint control instructions, combining model predictive control and online incremental learning to realize closed-loop adaptive control.
[0067] This step is the execution process of real-time closed-loop control of the quadruped robot, and the logic is to provide comprehensive perception input through multi-sensor data fusion, rely on a unified lightweight model and model predictive control (MPC) to output accurate instructions, and then dynamically optimize the model through online incremental learning to form an adaptive closed loop of "perception-decision-execution-optimization", ensuring stable and efficient movement of the quadruped robot in complex dynamic scenarios.
[0068] Among them, multi-sensor data fusion is the perception basis of the whole process, and the purpose is to eliminate the noise, blind area and information redundancy of single sensor, and output the state data with time and space synchronization and complete characteristics. The data collected by the multi-sensor and the like are described in Embodiment 1, and the data fusion is realized in the following three steps: first step, data preprocessing: the data of each sensor is denoised, and the time and space synchronization of all data is completed based on the laser radar clock (synchronization error ≤1 ms); second step, feature layer fusion: through the attention mechanism space-time graph neural network (ST-GNN), the multi-source data of “joint state-body posture-environment feature” is fused into a unified feature vector of 512 dimensions, and the key associated information of joint cooperation and terrain adaptation is captured; third step, decision layer fusion: based on the fusion feature, the current motion scene and the robot state are judged, and the unified model is provided with clear input labels.
[0069] The “unified model” in this step refers to a lightweight control model obtained by the KD-MoE distillation algorithm. The input of the model is the feature vector after multi-sensor fusion and the target motion instruction, and the output is the target angle, torque upper limit and action timing parameter of 12 joints, as well as the current optimal motion mode. The instruction reasoning delay is ≤10 ms, so as to meet the real-time control requirement.
[0070] As described above, the application can be well implemented.
Claims
1. A quadruped robot control system that chains hybrid deep reinforcement learning, characterized by, Comprise: A perception module (1) for collecting bionic motion data, robot state data and environment data; A data processing module (2) for denoising, normalizing and standardizing the feature extraction of the data collected by the perception module (1); A chain mixed reinforcement learning model module (3) for deploying a chain mixed deep reinforcement learning model, receiving the standardized data of the data processing module (2), and generating joint control instructions; A control output module (5) for converting the joint control instructions into driving signals of the robot executor (6); A feedback module (4) for collecting motion execution result data of the robot executor (6), feeding back to the data processing module (2) and the chain mixed reinforcement learning model module (3), realizing closed-loop control and online optimization of the model.
2. The chain hybrid deep reinforcement learning quadruped robot control system according to claim 1, wherein, The perception module (1) at least includes high-precision motion capture equipment, laser radar, IMU inertial measurement unit and plantar pressure sensor, and the collection accuracy of the above-mentioned high-precision motion capture equipment meets the joint angle error ≤0.5°, and the ground reaction force measurement error of the plantar pressure sensor is ≤5%.
3. The control method of the chain hybrid deep reinforcement learning quadruped robot control system according to claim 1 or 2, characterized in that, Comprise the following steps: S1, collect bionic motion data, robot state data and environment data, based on attention mechanism spatio-temporal graph neural network and adversarial imitation learning, train to obtain reference gait model containing motion prior; S2, construct a three-level parallel simulation environment, set progressive course objectives and train them using an improved PPO algorithm to optimize gait stability, motion efficiency and environmental adaptability; S3, build a differentiated simulation environment with multiple dynamics parameters and multiple ground types, combine meta-reinforcement learning with domain adversarial neural network to narrow the simulation-reality gap and improve model robustness; S4, train special expert models for five core motion modalities, and fuse them into a unified lightweight control model through distillation algorithm and gait transition loss function; S5, based on multi-sensor data fusion, use the unified model to output joint control instructions, combine model predictive control with online incremental learning to realize closed-loop adaptive control.
4. The control method of the chain hybrid deep reinforcement learning quadruped robot control system according to claim 3, wherein, In step S1, the bionic motion data, robot state data and environment data are collected by a motion capture system with a sampling frequency of ≥240Hz, and a multi-scene bionic motion data set is constructed.
5. The control method of the chain hybrid deep reinforcement learning quadruped robot control system according to claim 4, wherein, In the reference gait model of step S1, joint angle constraints and torque smoothness regularization terms are also added.
6. The control method of the chain hybrid deep reinforcement learning quadruped robot control system according to claim 4, wherein, In step S2, the "three-level parallel simulation environment" includes a basic static environment, a complex static environment and a dynamic disturbance environment, and the progressive course objectives at least include stability, efficiency and adaptability.
7. The control method of the chain hybrid deep reinforcement learning quadruped robot control system according to claim 6, wherein, The "improved PPO algorithm" in step S2 is PPO-Lag algorithm.
8. The control method of the chain hybrid deep reinforcement learning quadruped robot control system according to claim 4, wherein, The number of "differentiated simulation environments" in step S3 is ≥10, the meta-reinforcement learning is MAML algorithm, and the domain adversarial neural network is DANN network.
9. The control method of the chain hybrid deep reinforcement learning quadruped robot control system according to claim 4, wherein, The "five core motion modalities" in step S4 are walking, fast walking, running, climbing and obstacle crossing.
10. The control method of the chain hybrid deep reinforcement learning quadruped robot control system according to claim 9, wherein, The distillation algorithm is KD-MoE distillation algorithm, and the parameter quantity of the control model is ≤5 million.
Citation Information
Cited By
Quadruped robot end side deployment method and system
CN122110748A
