Quadruped robot single motor fault tolerance control method based on reinforcement learning

CN119065240APending Publication Date: 2024-12-03HUAZHONG UNIV OF SCI & TECH
View PDF 0 Cites 6 Cited by

Patent Information

Application Number
CN202411048913.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-01
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

Four-legged robots face the risk of motor failure in complex unstructured environments. The existing fault-tolerant control methods are difficult to cope with unknown faults and incomplete controllability of legs, resulting in motion instability and hardware damage, and lack of adaptability and high reliability. Fault-tolerant control method.

Method used

Adopt the fault tolerance control method of single motor fault of four-legged robot based on reinforcement learning. By building a four-legged robot model in a simulation environment, designing a teacher-student network framework and reliability reward mechanism, training teacher network and student network, realizing the autonomy of fault tolerance control strategies Explore and transfer, and use the curriculum learning mechanism to improve ground passivity and environmental adaptability.

Benefits of technology

The four-legged robot is realized in the event of a rapid adaptive adjustment and stable and reliable movement of a four-legged robot in unknown motor failure scenarios, reducing the risk of secondary damage to the faulty legs and improving the reliability and robustness of movement in an unstructured environment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119065240A_ABST
    Figure CN119065240A_ABST
Patent Text Reader

Abstract

The invention belongs to the technical field of robot control, and discloses a quadruped robot single motor fault tolerance control method based on reinforcement learning. According to the method, reinforcement learning is introduced into an unknown single-leg motor fault condition of the quadruped robot, and a teacher-student network, a reliability reward and a course learning mechanism are ingeniously fused, so that the robot can quickly adapt to a joint failure condition without any fault priori, the possible falling risk is actively avoided, and the reliability of the robot is improved. The gait is flexibly adjusted to maintain stable and reliable movement, the environment adaptability, reliability and deployment convenience which are far better than those of traditional active fault-tolerant control and self-adaptive fault-tolerant control are shown, and key technical support can be provided for long-time autonomous operation of the quadruped robot in the unstructured environment.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of robot control, and more specifically, relates to a single motor fault-tolerant control method of a quadruped robot based on reinforcement learning. Background Art

[0002] Quadruped robots are increasingly used in complex unstructured environments due to their excellent terrain adaptability. However, complex working conditions also expose robots to a higher risk of motor failures, which can seriously affect the robot's performance and may cause its motion to become unstable or even damage its hardware. Given that immediate human intervention is not possible in many operating scenarios, it is critical for robots to automatically adapt to unknown motor failures and maintain stable motion. The unpredictability of motor failures and the incomplete controllability of leg motion caused by failures pose significant challenges to the reliable fault-tolerant control of quadruped robots.

[0003] Existing fault-tolerant control methods for quadruped robots mainly include model-based active fault-tolerant control and reinforcement learning-based adaptive fault-tolerant control. The former requires accurate modeling of dynamics and estimation of fault states, making it difficult to deal with unknown faults; the latter uses data-driven methods to autonomously explore adaptive control strategies through reinforcement learning, but rarely considers the impact of incomplete leg controllability caused by motor failure on robot motion, making it difficult to provide a reliable fault-tolerant control strategy. Therefore, a new method for fault-tolerant control of unknown motor faults in quadruped robots with strong adaptability and high reliability is urgently needed. Summary of the invention

[0004] The present invention is aimed at the scenario of unknown single motor failure of a quadruped robot. In response to the above-mentioned defects or improvement needs of the prior art, a single motor fault-tolerant control method for a quadruped robot based on reinforcement learning is provided. The purpose of the present invention is to achieve rapid adaptive adjustment of the quadruped robot in any scenario of unknown motor failure and maintain stable and reliable movement.

[0005] To achieve the above object, according to one aspect of the present invention, a single motor fault-tolerant control method for a quadruped robot based on reinforcement learning is provided, comprising the following steps:

[0006] Offline training:

[0007] S1: Build a quadruped robot model in a simulation environment and set up a variety of terrain environments of different types and difficulty levels;

[0008] Design a single motor fault simulation mechanism for a quadruped robot: Based on the single motor corresponding to each degree of freedom of motion of the quadruped robot's legs, the output torque of the single motor is randomly set to zero, and training is performed in the simulation environment, simulating the fault condition of complete failure of the single motor in the training round;

[0009] S2: Based on the simulated single motor complete failure fault condition, a reinforcement learning algorithm is designed to construct a teacher-student network framework, which includes a teacher network and a student network;

[0010] The input of the teacher network includes privileged information representing motor failure, proprioceptive information of the quadruped robot, and desired speed command; the input of the student network is the historical sequence of proprioceptive information of the quadruped robot and the desired speed command; the output of the teacher network and the student network are both position control instructions of the quadruped robot joints;

[0011] S3: training the teacher network through a reinforcement learning algorithm; introducing a reliability reward mechanism into the reinforcement learning algorithm and designing a reward function: the reliability reward mechanism is set such that the hanging height of the leg foot end corresponding to the faulty motor shall never be lower than a preset height;

[0012] Design a curriculum learning mechanism to enable the quadruped robot to interact with terrains of different types and difficulty through large-scale parallel sampling, in order to improve the ground passability of the quadruped robot;

[0013] Based on the reliability reward mechanism and curriculum learning mechanism, the fault-tolerant control strategy of the teacher network is obtained through the reinforcement learning algorithm;

[0014] S4: Training a quadruped robot;

[0015] A supervised learning method is used to train a student network to approximate the fault-tolerant control strategy of a teacher network. The trained student network is a fault-tolerant control network. The fault-tolerant control network is used to migrate to a physical quadruped robot to control the movement of the physical quadruped robot.

[0016] Online Control:

[0017] The historical sequence of proprioceptive information and expected speed commands of the physical quadruped robot are collected and input into the fault-tolerant control network, thereby outputting position control instructions for the joints of the quadruped robot and realizing single-motor fault-tolerant control of the quadruped robot.

[0018] Furthermore, in step S1, the simulation environment is built in Isaac Gym, and Unitree A1 is selected as the quadruped robot model, which has a total of 12 degrees of freedom of movement, corresponding to the hip joints, knee joints and ankle joints of the four legs, front, back, left and right.

[0019] Furthermore, in step S1, at a random moment after the start of each training round, one motor is uniformly sampled from the 12 motors and its output torque is set to zero until the end of the round to simulate complete failure. The calculation formula is:

[0020]

[0021] In the formula is the actual torque acting on the joint, ⊙ is the Hadamard product operator, indicating the multiplication of the corresponding position elements in the two vectors, fm t is the motor fault indicator, τ t is the force applied to the joint.

[0022] Furthermore, in step S2, the teacher network and the student network both adopt an MLP structure with three hidden layers, and the number of neurons in the hidden layers are 512, 256, and 128, respectively.

[0023] Furthermore, in step S3, the reward function is set as follows:

[0024]

[0025] In the formula, r t For the total reward, They are reliability bonus item, regular item, task tracking item, attitude stability item and motion smoothness item. The preset leg-lifting height is for the faulty leg, i is the leg number, h is t,i To correspond to the actual height of the foot, is the fault indicator of the corresponding leg, its value is 1 in fault state and 0 in normal state, and σ is the scaling factor.

[0026] Furthermore, in step S3, the terrain of the course learning mechanism includes flat land, undulating terrain and terrain with discrete obstacles, and the course learning mechanism is used to progressively perform training on terrains of different difficulty levels, wherein the quadruped robot starts training from a simple flat terrain, and the terrain difficulty gradually increases. Only if the target point is successfully reached within a certain round, will the terrain difficulty be increased in the next round of training.

[0027] Furthermore, in step S4, the teacher network adopts the proximal policy optimization PPO algorithm to realize deep reinforcement learning, and the student network adopts the data aggregation DAgger method to realize supervised learning.

[0028] The present invention also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is run on a computer, the computer executes the single-motor fault-tolerant control method of a quadruped robot based on reinforcement learning.

[0029] The present invention also provides an electronic device, comprising a processor and a memory, wherein the memory stores a computer program, and the processor calls the computer program stored in the memory to execute the single-motor fault-tolerant control method of a quadruped robot based on reinforcement learning.

[0030] In general, the single motor fault-tolerant control method of a quadruped robot based on reinforcement learning provided by the present invention has the following beneficial effects compared with the prior art:

[0031] 1. The teacher-student network framework in the reinforcement learning algorithm is applied to the fault-tolerant control of unknown single motor faults in quadruped robots, so that the robot can autonomously extract fault features from proprioceptive perception without having to master the fault priors, which greatly improves the fault adaptability and reduces the difficulty of actual deployment;

[0032] 2. Incorporate the reliability reward item into the reward function of the reinforcement learning algorithm to guide the robot to learn to actively lift the faulty leg and maintain the foot-ground contact so that the faulty leg does not participate in the movement process, so as to avoid the risk of secondary damage to the faulty motor and the risk of incomplete controllability of the leg caused by the failure of a single motor on the robot's movement, thereby improving its movement reliability in unstructured environments;

[0033] 3. A course learning mechanism is used to progressively conduct training on terrains of different difficulty levels, exploring the ground passability of the quadruped robot under unknown single motor failure conditions, so that the control strategy is continuously strengthened in large-scale parallel interaction, and its robustness and environmental adaptability are greatly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A schematic diagram of the principle of a single motor fault-tolerant control method for a quadruped robot based on reinforcement learning provided by the present invention;

[0035] Figure 2 A schematic diagram of a digital simulation environment used in an embodiment of a method provided by the present invention;

[0036] Figure 3 A motion sequence comparison diagram of an embodiment of the method provided by the present invention and a benchmark method without a reliability reward mechanism;

[0037] Figure 4 A comparison chart of step pass rates of an embodiment of the method provided by the present invention and a benchmark method without a reliability reward mechanism. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0039] See also Figure 1The present invention proposes a single motor fault-tolerant control method for a quadruped robot based on reinforcement learning. The method introduces a teacher-student network architecture and a reliability reward mechanism. Through deep reinforcement learning, a control strategy is obtained that can not only autonomously adapt to unknown motor failures, but also maintain stable and reliable walking under the condition that the corresponding leg of the faulty motor is not fully controllable. The method includes the following steps:

[0040] Offline training:

[0041] S1: Build a quadruped robot model in the simulation environment Isaac Gym and set up a variety of terrain environments of different types and difficulties; select Unitree A1 as the quadruped robot model, which has a total of 12 degrees of freedom, corresponding to the hip joints, knee joints and ankle joints of the four legs;

[0042] A single motor failure simulation mechanism for a quadruped robot is designed. Based on the single motor corresponding to each degree of freedom of motion of the quadruped robot's legs, the output torque of the single motor is randomly set to zero, and training is performed in a simulation environment. The failure condition of a single motor complete failure is simulated in the training round. The motor failure occurs at a random joint position at the initialization of each training round, and the single motor failure of the quadruped robot is randomly simulated. At a random moment after the start of each training round, one motor is uniformly sampled from the 12 motors, and its output torque is set to zero until the end of the round to simulate complete failure. The calculation formula is:

[0043]

[0044] In the formula is the actual torque acting on the joint, ⊙ is the Hadamard product operator, indicating the multiplication of the corresponding position elements in the two vectors, fm t is the motor fault indicator, τ t is the force applied to the joint;

[0045] S2: Based on the simulated single motor complete failure fault condition, a reinforcement learning algorithm is designed to construct a teacher-student network framework, which includes a teacher network and a student network;

[0046] Teacher network: using the fault code vector representing the motor working status as privileged information Input, x t Contains motor fault identification and the corresponding leg fault indicator Together with the robot's proprioceptive information and the desired speed command Together, they constitute the state s t, after an MLP structure with 3 hidden layers (number of neurons: 512, 256, 128), the position control instructions of 12 joints are output; among them, the fault identification value 0 indicates fault and 1 indicates normal; the robot proprioceptive information includes the body linear velocity, angular velocity, gravity unit vector projection, joint relative position, velocity, foot-ground contact identification and the last action; the expected velocity command includes the expected xy linear velocity and z-direction deflection angular velocity; the network weights are optimized through proximal strategy optimization PPO algorithm training to obtain a fault-tolerant control strategy under unknown motor fault scenarios;

[0047] Student network: The structure is the same as that of the teacher network, but the input only receives the historical sequence h of the robot's proprioceptive information t and the desired speed command Does not contain any motor fault priors; outputs position control commands for 12 joints;

[0048] Among them, the action space settings of the teacher network and the student network are: action space Represents the 12 joints relative to the initial joint angle q init The position offset of the joint, and the final force τ applied to the joint t Calculated by the PD controller:

[0049]

[0050] Where K p is the PD controller proportional coefficient, K d is the differential gain coefficient of the PD controller, q t is the joint position, is the joint velocity, and the initial joint position is calculated from the standing state with a fuselage height of 0.3 m.

[0051] S3: Train the teacher network through reinforcement learning algorithm; introduce the reliability reward mechanism in the reinforcement learning algorithm and design the reward function: the reliability reward mechanism is set to ensure that the leg foot end suspension height corresponding to the faulty motor must not be lower than the preset height; the reward function consists of conventional reward items such as task reward, motion stability reward, smoothness reward, and reliability reward items specially set for single motor fault conditions, so as to comprehensively weigh the various performance indicators of the controller. The reward function is defined as:

[0052]

[0053] In the formula, r t For the total reward, They are reliability bonus item, regular item, task tracking item, attitude stability item and motion smoothness item. The preset leg-lifting height is for the faulty leg, i is the leg number, h is t,i To correspond to the actual height of the foot, is the fault indicator of the corresponding leg, its value is 1 in the fault state and 0 in the normal state, and σ is the scaling factor. This reward item guides the robot to actively raise the leg with motor failure to This can avoid the risk of the faulty leg not being fully controllable as much as possible, thereby ensuring the stability and reliability of the robot's overall movement in the event of a single-leg motor failure.

[0054] In order to promote learning and achieve adaptation to challenging environments, a curriculum learning mechanism is designed to enable the quadruped robot to interact with terrains of different types and difficulties through large-scale parallel sampling to improve the ground passability of the quadruped robot. The terrains of the curriculum learning mechanism include flat land, undulating terrain, and terrain with discrete obstacles; among them, the noise of undulating terrain increases from ±1cm to ±3cm, and the height of terrain with discrete obstacles changes from ±1cm to ±5cm. The quadruped robot follows speed commands sampled from the range of (-0.5,1.5), (-0.5,0.5), and (-1.0,1.0), corresponding to forward, lateral, and yaw speeds, respectively. On flat land, commands are randomly sampled, while on complex terrain, the robot's goal is to reach a target point according to its direction, and the yaw speed is dynamically adjusted. The quadruped robot starts training from a simple flat terrain, and the terrain difficulty gradually increases. Only when the target point is successfully reached in a certain round will the terrain difficulty be increased in the next round of training.

[0055] Based on the reliability reward mechanism and curriculum learning mechanism, the fault-tolerant control strategy of the teacher network is obtained through the reinforcement learning algorithm.

[0056] S4: supervised learning is used to train the student network to approach the fault-tolerant control strategy of the teacher network. The trained student network is the fault-tolerant control network, which is used to migrate to the physical quadruped robot to control the movement of the physical quadruped robot.

[0057] Specifically, Isaac Gym was used to conduct large-scale parallel reinforcement learning in a simulation environment. Each training round simulated a 30-second motion process, and the time step was set to 0.02 seconds, so 1,500 state-action transition data were sampled in a single round. 8,192 environment instances were simulated in parallel for interactive training at the same time. When the robot's torso was detected to collide with the ground, it was considered a fall, and the current round was terminated in advance and the environment state was reset to improve training efficiency.

[0058] The teacher network uses the proximal policy optimization PPO algorithm to update the policy network weights, while the student network supervises the trajectory data of the teacher strategy based on the data set aggregation DAgger. The corresponding loss function is:

[0059]

[0060] In the formula, is the output action of the trained teacher strategy, is the output of the student network being trained. The entire reinforcement learning process runs on a single NVIDIA GeForce RTX 4090 graphics card, and the training totals 1e7 time steps.

[0061] Online Control:

[0062] The historical sequence of proprioceptive information and expected speed commands of the physical quadruped robot are collected and input into the fault-tolerant control network, thereby outputting position control instructions for the joints of the quadruped robot and realizing single-motor fault-tolerant control of the quadruped robot.

[0063] In the simulated environment, when the quadruped robot faces a scenario where any joint motor suddenly fails during normal walking, the above-mentioned embodiment can quickly and adaptively control the robot to maintain its movement ability, and in a complex interactive environment, it effectively avoids the instability factors that may be caused by the participation of the faulty joint in the movement, showing high reliability. When directly migrated and deployed on the actual robot, it was also successful in the simulated mission scenario range.

[0064] A comparison of the motion sequences of the quadruped robot in the method provided by the present invention and the benchmark method without a reliability reward mechanism is shown in FIG. Figure 3 As shown, the red color of the leg connecting rod in the figure indicates that the corresponding joint motor fails, and the green ball at the foot end indicates that the foot is in contact with the ground. Starting from the 4th frame slice in the figure, the side swing joint of the robot's right front leg suddenly fails and loses power. It can be concluded that the reliability reward-driven fault-tolerant control method of the quadruped robot of the present invention shows a significantly different timing motion pattern when facing a single-leg motor failure compared with the benchmark method that does not adopt this mechanism. That is, from the 6th frame, it can be seen that the faulty leg under the benchmark method is still involved in the movement and in contact with the ground, while the faulty leg under the method proposed by the present invention maintains the lifted state. After detecting the fault, the robot under the benchmark method continues to use the four legs to move forward alternately due to the lack of effective evasive action. It is difficult to avoid the unknown collision that may be caused by the incomplete control of the faulty leg, and it may also cause secondary damage to the faulty motor. The quadruped robot of the present invention can quickly lift the faulty leg to a safe height and keep it lifted at the first time of the fault, and flexibly adjust the gait to maintain stability, fully reflecting a stronger risk response capability.

[0065] Figure 4The ground passing success rates of the two methods at different heights and different forward speeds were further quantitatively compared. It can be seen that the success rate of the present invention under all working condition combinations is significantly higher than that of the benchmark method, especially at higher steps and faster speeds. It proves the important value of the reliability reward mechanism in improving the environmental adaptability and motion reliability of the controller in complex terrain. The present invention also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is run on a computer, the computer executes the above-mentioned single-motor fault-tolerant control method of a quadruped robot based on reinforcement learning.

[0066] The present invention also provides an electronic device, comprising a processor and a memory, wherein the memory stores a computer program, and the processor is used to execute the above-mentioned single-motor fault-tolerant control method of a quadruped robot based on reinforcement learning by calling the computer program stored in the memory.

[0067] The present invention introduces reinforcement learning into the unknown single-leg motor failure situation of the quadruped robot. By cleverly integrating the teacher-student network, reliability reward, curriculum learning and other mechanisms, the robot can quickly adapt to joint failure conditions without any prior knowledge of faults, actively avoid possible risks of falling, and flexibly adjust the gait to maintain stable and reliable movement. Therefore, it demonstrates environmental adaptability, reliability and deployment convenience that are far superior to traditional active fault-tolerant control and adaptive fault-tolerant control, and can provide key technical support for the long-term autonomous operation of quadruped robots in unstructured environments.

[0068] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A single motor fault-tolerant control method for a quadruped robot based on reinforcement learning. It is characterized in that The following steps are involved: Offline training: S1: Build a quadruped robot model in a simulation environment and set up a variety of terrain environments of different types and difficulty levels; Design a single motor fault simulation mechanism for a quadruped robot, based on a single motor corresponding to each degree of freedom of motion of the legs of the quadruped robot, randomly set the output torque of the single motor to zero, perform training in the simulation environment, and simulate a fault condition in which the single motor completely fails in the training round; S2: Based on the simulated single motor complete failure fault condition, a reinforcement learning algorithm is designed to construct a teacher-student network framework, which includes a teacher network and a student network; The input of the teacher network includes privileged information representing motor failure, proprioceptive information of the quadruped robot, and desired speed command; the input of the student network is the historical sequence of proprioceptive information of the quadruped robot and the desired speed command; the output of the teacher network and the student network are both position control instructions of the quadruped robot joints; S3: training the teacher network through a reinforcement learning algorithm; introducing a reliability reward mechanism into the reinforcement learning algorithm and designing a reward function: the reliability reward mechanism is set such that the hanging height of the leg foot end corresponding to the faulty motor shall never be lower than a preset height; Design a curriculum learning mechanism to enable the quadruped robot to interact with terrains of different types and difficulty through large-scale parallel sampling, in order to improve the ground passability of the quadruped robot; Based on the reliability reward mechanism and curriculum learning mechanism, the fault-tolerant control strategy of the teacher network is obtained through the reinforcement learning algorithm; S4: training the quadruped robot, using supervised learning to train the student network to approximate the fault-tolerant control strategy of the teacher network, the trained student network is the fault-tolerant control network, and the fault-tolerant control network is used to migrate to the physical quadruped robot to control the movement of the physical quadruped robot; Online Control: The historical sequence of proprioceptive information and expected speed commands of the physical quadruped robot are collected and input into the fault-tolerant control network, thereby outputting position control instructions for the joints of the quadruped robot and realizing single-motor fault-tolerant control of the quadruped robot.

2. The control method according to claim 1, It is characterized in that In step S1, the simulation environment is built in IsaacGym, and Unitree A1 is selected as the quadruped robot model, which has a total of 12 degrees of freedom of movement, corresponding to the hip joints, knee joints and ankle joints of the four legs, front, back, left and right.

3. The control method according to claim 1, It is characterized in that In step S1, at a random moment after the start of each training round, one of the 12 joint motors of the quadruped robot's legs is uniformly sampled and selected, and its output torque is set to zero until the end of the round to simulate complete failure; the calculation formula is: In the formula is the actual torque acting on the joint, ⊙ is the Hadamard product operator, indicating the multiplication of the corresponding position elements in the two vectors, fm t is the motor fault indicator, τ t is the force applied to the joint.

4. The control method according to claim 1, It is characterized in that In step S2, the teacher network and the student network both adopt an MLP structure with three hidden layers, and the number of neurons in the hidden layers are 512, 256, and 128 respectively.

5. The control method according to claim 1, It is characterized in that In step S3, the reward function is set as follows: In the formula, r t For the total reward, They are reliability bonus item, regular item, task tracking item, attitude stability item and motion smoothness item. The preset leg-lifting height is for the faulty leg, i is the leg number, h is t,i To correspond to the actual height of the foot, is the fault indicator of the corresponding leg, its value is 1 in fault state and 0 in normal state, and σ is the scaling factor.

6. The control method according to claim 1, It is characterized in that In step S3, the terrain of the course learning mechanism includes flat land, undulating terrain and terrain with discrete obstacles. The course learning mechanism is used to progressively perform training on terrains of different difficulty levels. The quadruped robot starts training from a simple flat terrain, and the terrain difficulty gradually increases. Only if the target point is successfully reached within a certain round, the terrain difficulty will be increased in the next round of training.

7. The control method according to claim 1, It is characterized in that In step S4, the teacher network uses the proximal policy optimization PPO algorithm to achieve deep reinforcement learning, and the student network uses the data aggregation DAgger method to achieve supervised learning.

8. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed on a computer, the computer is enabled to execute the method according to any one of claims 1 to 7.

9. An electronic device, It is characterized in that The electronic device comprises a processor and a memory, wherein a computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 7 by calling the computer program stored in the memory.

Citation Information

Cited By

  • Deep reinforcement learning quadruped robot motion control method and system based on constraint reward

    CN119512184A

  • Robot motion control model training method and device based on deep reinforcement learning

    CN120065751A

  • Motion control method for three-foot walking gait of quadruped robot

    CN120363183A

  • A quadruped robot three-legged walking gait motion control method

    CN120363183B

  • Quadruped robot reinforcement learning control method for migration from simulation to reality

    CN122172830A