A quadruped robot motion control method based on terrain map reinforcement learning

Abstract

This invention discloses a quadruped robot motion control method based on terrain map reinforcement learning, comprising the following steps: S1, building a quadruped robot model; S2, constructing a terrain map confidence estimation module; S3, building a state estimation model; S4, constructing an actor network and a critic network to train the quadruped robot model; and S5, utilizing sensor fusion technology to acquire LiDAR and inertial measurement unit data, generate a height map of the quadruped robot's surroundings in real time, and update the surrounding terrain information. By introducing a terrain map confidence estimation module and combining terrain map generation with terrain uncertainty estimation, this invention enables real-time adjustment of the quadruped robot's motion strategy in complex terrain, significantly improving the robot's adaptability to irregular terrain, obstacles, and terrain changes, and reducing the risk of control errors and falls caused by terrain changes or sensor noise.

Description

Technical Field

[0001] The present invention belongs to the technical field of quadruped robots, and in particular relates to a quadruped robot motion control method based on terrain map reinforcement learning. Background Art

[0002] Existing deep reinforcement learning-based quadruped robot motion control methods generally use elevation map-based terrain perception methods, which primarily rely on fixed elevation map inputs. While these methods can provide effective terrain information for the robot, the use of elevation maps can lead to reduced accuracy in complex terrain or when sensor data is incomplete. This is especially true in the event of sensor failure or occlusion, as environmental information cannot be fully utilized. They also perform poorly in handling terrain uncertainty and dramatic elevation changes, impacting the robot's motion performance. In particular, in complex and irregular terrain, the robot struggles to effectively plan based on the real-time terrain, resulting in reduced accuracy and efficiency in motion control.

[0003] To address this issue, recent research has proposed multimodal data fusion and uncertainty estimation solutions. However, existing methods often suffer from two drawbacks: inaccurate modeling of terrain information when dealing with complex terrain, and poor robustness to sensor noise or occlusion. Consequently, existing quadruped robot motion control methods based on perceptual reinforcement learning cannot effectively achieve stable motion in environments with unstable generated terrain maps, especially when the terrain changes dramatically or sensor errors are present. Summary of the Invention

[0004] In order to overcome the shortcomings of the existing technology, the present invention provides a quadruped robot motion control method based on terrain map reinforcement learning, which improves the robustness and obstacle surmounting ability of the quadruped robot in different terrains by combining terrain map with proprioception information.

[0005] The technical solution adopted by the present invention to solve the technical problem is: a quadruped robot motion control method based on terrain map reinforcement learning, comprising the following steps:

[0006] S1, building a quadruped robot model on the simulation platform;

[0007] S2, build the topographic map confidence estimation module, set the noise model in the simulation to obtain the height map h with noise t Simulating multiple real-world environments, using a quadruped robot with noise to perceive historical information and height map h t As input, it outputs terrain confidence weights and reconstructed terrain features;

[0008] S3, build a state estimation model based on the historical state observations perceived by the quadruped robot body and the latent vector l obtained by encoding the topographic map confidence estimation module in step S2 t As input, output is the estimated speed v of the quadruped robot at the current moment t , the next state of the quadruped robot is o t+1 , which serves as the basis for motion control decision-making of quadruped robots in complex terrain;

[0009] S4, builds an actor network and a critic network to train the quadruped robot model;

[0010] S5, uses sensor fusion technology to obtain lidar and inertial measurement unit data, generate a height map around the quadruped robot in real time, and update the environmental terrain information.

[0011] Furthermore, the step S2 includes the following sub-steps:

[0012] S21, add random noise to the height map and build a noise model Where z is the noise parameter;

[0013] S22, setting mapping conditions and configuring different noise parameters z for different conditions to simulate different terrain qualities and error sources;

[0014] S23, the height map H with noise nosie The historical state observations of the quadruped robot's proprioception Input noise model.

[0015] Furthermore, in step S22, the noise model includes at least one of the following noise types:

[0016] Normal noise, assuming that the map quality is good and suitable for routine operations, the map measurement error is simulated by Gaussian noise. The standard deviation of the Gaussian noise under the normal noise condition is σ∈[0.03m,0.08m];

[0017] Large offset noise: simulates map offset caused by pose estimation drift or terrain deformation through higher noise, and simulates pose estimation drift through spatial offset noise; the mean spatial offset of the large offset noise condition is μ∈[0.1m,0.2m];

[0018] Large noise amplitude applies larger noise to each scan point to simulate the complete loss of terrain information due to occlusion or sensor failure, and simulates sensor failure scenarios through random occlusion noise; the large noise amplitude condition applies random zero-value noise to at least 20% of the scan points in the height map.

[0019] Furthermore, the step S23 includes the following sub-steps:

[0020] S231, height map H with noise nosie and historical state observations After passing through a network M1, M2 for feature compression, the vector h is obtained t and l t ;

[0021] S232, h t and l t After splicing, it is fed into the fully connected network M3 to obtain the height map confidence vector b t ,b t Determines the proprioceptive information of the quadruped robot and height map H nosie The calculation process of how much weight is generated for the final height map is expressed as:

[0022]

[0023] b t =M1(cat(h t , l t ))

[0024] out t =b t +b t ⊙l t

[0025] Where ⊙ is element-wise multiplication, resulting in vector out t , during the training process, out t As input, it is fed into the decoder network to reconstruct the height map. The calculation formula is as follows:

[0026] H e =M4(out t )+M4(out t )⊙H true

[0027] Among them, H e Is the estimated height map, H true A true height map.

[0028] Furthermore, the height map H e and the real height map H true The error is calculated using the mse loss function, and the loss function is

[0029] Furthermore, the step S3 includes the following sub-steps:

[0030] S31, the state estimation model uses a deep network architecture, and the input includes the historical state observations of the quadruped robot proprioception and the latent vector l from the topographic map confidence estimation module t , the input together encodes the current state of the quadruped robot, the specific observation value o t is a 45-dimensional vector defined as follows;

[0031]

[0032] w t Indicates the robot's body angular velocity, g t represents the gravity vector of the body, c t Indicates the robot linear velocity and angular velocity instructions, θ t represents the joint angular velocity, represents the joint angular acceleration, a t-1 Indicates the offset of the joint position output by the strategy at the previous moment relative to the joint position at the initial moment;

[0033] S32, the state estimation model uses the structure of the twin network and the variational autoencoder framework to extract deep features and generate the current quadruped robot speed and latent vector in The potential representation extracted by the variational autoencoder structure can effectively capture the complex characteristics of the environment and robot state; the state estimation module outputs the estimated speed at the current moment Represents the motion state of the robot; latent vector Contains implicit information about the current environment and the quadruped robot;

[0034] S33, using quadruped robot speed and latent vector Jointly predict the next moment of the quadruped robot state o t+1 , the decoder network will and Combine and output the state prediction o at the next moment t+1 .

[0035] Furthermore, in step 3, the loss function in the variational autoencoder is used to train the network by minimizing the reconstruction error and the KL divergence of the latent variables. The loss function of the state estimation is: Used to evaluate the accuracy of the model in predicting the state at the next moment, the KL divergence of the latent variable is: The total loss is composed of the weighted reconstruction loss and KL divergence

[0036] Furthermore, in step 4, a reinforcement learning algorithm is used for training, including constructing an actor network and a critic network based on the PPO reinforcement learning algorithm to train the quadruped robot model, and the strategy input includes the quadruped robot proprioception information o t , the estimated speed of the output of the state estimation module and features extracted by variational autoencoder And the output of the topographic map confidence estimation module out t , represents the confidence of the current map, which is used to weight the quadruped robot's proprioceptive information and the noisy height map information.

[0037] Furthermore, in step 4, the actor network uses the policy input information to generate the action of the quadruped robot at the current moment to maximize the cumulative reward; the critic network accepts the same policy input information, evaluates the value of the current state by calculating the state value function, and provides the expected return of the action.

[0038] Furthermore, in step 5, the lidar and inertial measurement unit data are fused through an extended Kalman filter, and the odometer output frequency reaches 200 Hz.

[0039] By generating and updating terrain map information in real time and estimating the uncertainty of the terrain map, and combining terrain map generation with terrain uncertainty estimation, it can help quadrupeds make more accurate and stable motion decisions under different environmental conditions, thereby improving the adaptability and execution efficiency of quadruped robots in complex terrain.

[0040] The beneficial effects of the present invention are as follows: by introducing a terrain map confidence estimation module and combining terrain map generation with terrain uncertainty estimation, the motion strategy of a quadruped robot in complex terrain can be adjusted in real time, significantly improving the robot's adaptability to irregular terrain, obstacles, and terrain changes, and reducing the risk of control errors and falls caused by terrain changes or sensor noise; by adding noise and processing height map data through a noise model, sensor noise and terrain uncertainty are effectively estimated and compensated, thereby improving the robustness of the quadruped robot to terrain information. In particular, in the case of sensor failure or occlusion, the noise compensation and confidence assessment mechanism can ensure that the robot can still perform stable motion; by modeling terrain data at different noise levels and combining historical data with a confidence estimation module for weighted fusion, more accurate terrain perception and estimation are achieved; by combining proprioceptive information (such as joint angular velocity, gravity vector, etc.) with environmental information (such as terrain confidence, estimated speed, etc.), and using a reinforcement learning strategy to optimize robot motion, the multimodal information fusion method significantly improves the robot's understanding and decision-making ability of the current environment, especially showing stronger stability and autonomy in a changing environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 Schematic diagram of the topographic map confidence estimation module in the present invention.

[0042] Figure 2 Schematic diagram of the state estimation model in the present invention.

[0043] Figure 3 This is a schematic diagram of the actual machine deployment framework in the present invention.

[0044] Figure 4 Schematic diagram of the training framework under simulation in the present invention. DETAILED DESCRIPTION

[0045] In order to enable those skilled in the art to better understand the solutions of the present invention, the following will provide a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0046] A quadruped robot motion control method based on terrain map reinforcement learning includes the following steps:

[0047] S1, build a quadruped robot model on the simulation platform, including the overall structure of the robot and the motion model of each leg;

[0048] The quadruped robot model includes kinematic and dynamic models of four independent robot legs and their joints. The quadruped robot utilizes a high-degree-of-freedom mechanical structure, with each leg driven by multiple joints (such as the hip, knee, and ankle) and motors. The model's dimensions, mass, and center of gravity match those of an actual robot, ensuring realism and accuracy in the simulation environment.

[0049] Specifically, the simulation platform can be Isaac Gym; in the Isaac Gym platform, a physical engine is used to simulate and control the robot, and different terrains of the environment are set, such as flat ground, stairs, slopes, irregular ground, etc., to ensure that the robot can be trained in various complex environments.

[0050] S2, constructed as Figure 1 The terrain map confidence estimation module shown in the figure sets the noise model in the simulation to obtain the height map h with noise tSimulate various real-world environments, including scenarios such as sensor failure and terrain changes. Noise types include lateral scan point offset and height perturbation. Lateral scan point offset applies lateral displacement noise to each scan point, while height perturbation applies perturbation noise to the height value of each scan point. The noise amplitude is adjusted using the noise parameter z. The noise offset in each simulation environment is initially set to zero and updated at each training step. The noise amplitude is a random value generated based on a Gaussian distribution.

[0051] A historical information graph of quadruped robot proprioception with noise and height map h t As input, through the encoding and decoding networks, the terrain confidence weights and reconstructed terrain features are output. Specifically, the estimated height map information can be output to reduce the impact of terrain uncertainty on the strategy.

[0052] Specifically, step S2 includes the following sub-steps:

[0053] S21, add random noise to the height map during training to build a noise model Where z is the noise parameter, which determines the variance of different noises;

[0054] S22 sets the mapping conditions and configures different noise parameters z for different conditions to simulate different terrain qualities and error sources. The noise parameter z is learned as part of the training process and increases linearly over time, which helps the robot gradually adapt to more complex and uncertain terrain environments.

[0055] Specifically, the noise model sets three mapping conditions, including normal noise, large offset noise, and large noise amplitude. Normal noise assumes good map quality and is suitable for routine operations. Map measurement errors are simulated using Gaussian noise, and the Gaussian noise standard deviation of the normal noise condition is σ∈[0.03m,0.08m].

[0056] Large offset noise: simulates map offset caused by pose estimation drift or terrain deformation through higher noise, and simulates pose estimation drift through spatial offset noise; the mean spatial offset of the large offset noise condition is μ∈[0.1m,0.2m];

[0057] Large noise amplitude applies larger noise to each scan point to simulate the complete loss of terrain information due to occlusion or sensor failure, and simulates sensor failure scenarios through random occlusion noise; the large noise amplitude condition applies random zero-value noise to at least 20% of the scan points in the height map.

[0058] The noise type is randomly selected using a probability value. Different noise intensities are added according to the following rules, with the standard deviation of the noise added being selected within the above range: Normal noise (60% probability): This noise simulates normal sensor errors, with a standard deviation of 0.03 meters. Large drift noise (30% probability): This noise simulates large drifts caused by sensor drift or terrain changes (such as ground deformation and slippage). The standard deviation of the noise is increased to 0.12 meters. Specifically, the noise amplitude gradually increases with training time. It is calculated as: Noise standard deviation = 0.12 + 3.0 × (min(number of training steps / 36000), 1.0). The more training steps, the greater the noise amplitude. High noise amplitude (10% probability): This noise simulates severe sensor failures or occlusions, which can cause distortion or loss of height data. The standard deviation of the noise is set to 0.6 meters and gradually increases as training progresses. Each training environment is divided into multiple small cells (e.g., a 0.5 m x 0.5 m area), each representing a terrain patch. Within these small cells, the system adds a random offset to each cell to simulate the irregularity and local changes of the terrain. The size of the noise is achieved by randomly generating the offset, and its range is set to [-0.12, 0.12] m. The height map H used in this example is nosie The size is 1.6mX1.0m, the resolution is 0.1m, and the dimension of the height map is 187.

[0059] S23, setting mapping conditions and configuring different noise parameters z for different conditions to simulate different terrain qualities and error sources.

[0060] Specifically, step S23 includes the following sub-steps:

[0061] S231, the above-mentioned height map H with noise nosie and the historical state observations of the quadruped robot proprioception enter Figure 1 The topographic map confidence estimation module shown, where Using 5 frames of historical information, the dimension is 225, first the height map H with noise nosie and historical state observations After a fully connected network M1 and M2, feature compression is performed to obtain the vector h t and l t , where the hidden layer dimensions of M1M2 are set to {128, 64}, {256, 128}, h t and l t The dimensions are all 64;

[0062] S232, vector h t and l tAfter splicing, it is fed into the fully connected network M3 to obtain the height map confidence vector b t , b t The dimension of b is 64, where the hidden layer dimension of M3 is set to {128, 64}, t Determines the proprioceptive information of the quadruped robot and height map H nosie The calculation process of how much weight is generated for the final height map is expressed as:

[0063]

[0064] b t =M1(cat(h t , l t ))

[0065] out t =b t +b t ⊙l t

[0066] Where ⊙ is element-wise multiplication, resulting in vector out t , during the training process, out t As input, it is fed into the decoder network to reconstruct the height map. The calculation formula is as follows:

[0067] H e =M4(out t )+M4(out t )⊙H true

[0068] Among them, H e Is the estimated height map, H true is the real height map. In order to train this module, it is necessary to calculate the estimated loss of the height map. The estimated height map H e and the real height map H true The error is calculated using the mse loss function for supervised learning. The loss function can be defined as follows:

[0069] S3, build a state estimation model, such as Figure 2 As shown, the historical state observation value of the quadruped robot proprioception is and the latent vector l obtained by encoding the topographic map confidence estimation module in step S2 t As input, output is the estimated speed v of the quadruped robot at the current moment t , the next state of the quadruped robot is o t+1 ,This is used as the basis for the motion control decision of the quadruped robot in complex terrain;

[0070] Specifically, step S3 includes the following sub-steps:

[0071] S31, the state estimation model uses a deep network architecture, and the input includes the historical state observations of the quadruped robot proprioception and the latent vector l from the topographic map confidence estimation module t , the input together encodes the current state of the quadruped robot, the specific observation value o t is a 45-dimensional vector defined as follows;

[0072]

[0073] w t Indicates the robot's body angular velocity, g t represents the gravity vector of the body, c t Indicates the robot linear velocity and angular velocity instructions, θ t represents the joint angular velocity, represents the joint angular acceleration, a t-1 Indicates the offset of the joint position output by the strategy at the previous moment relative to the joint position at the initial moment. In this example, w t is the body angular velocity of the three axes of roll, pitch, and yaw, g t The dimension is 3, which is the gravity vector of the quadruped robot, c t is the linear velocity instruction in the x and y directions, in m / s, and the yaw angle velocity instruction; θ t are the angles and angular accelerations of the 12 joints, a t-1 The value output by the policy network at the previous moment.

[0074] S32, the state estimation model uses the structure of the twin network and the variational autoencoder (VAE) framework to extract deep features, input 5 frames of historical data, and generate the current quadruped robot speed and latent vector in The potential representation extracted by the variational autoencoder (VAE) structure can effectively capture the complex characteristics of the environment and robot state; the state estimation module outputs the estimated speed at the current moment Represents the motion state of the robot; latent vector Contains implicit information about the current environment and the quadruped robot;

[0075] S33, using quadruped robot speed and latent vector Jointly predict the next moment of the quadruped robot state o t+1 , this process is completed by the decoder part, the decoder network will and Combine and output the state prediction o at the next moment t+1 In this example, The dimension is 3, is 19, o t+1 is 45.

[0076] In order to optimize the state estimation module, the loss function in the variational autoencoder (VAE) is used to train the network by minimizing the reconstruction error and the KL divergence of the latent variables. Specifically, the loss function of the state estimation is: Used to evaluate the accuracy of the model in predicting the state at the next moment, the KL divergence of the latent variable is: The total loss is composed of the weighted reconstruction loss and KL divergence

[0077] S4, build an actor network and a critic network to train the quadruped robot model;

[0078] Specifically, a reinforcement learning algorithm is used for training. Based on the PPO reinforcement learning algorithm, an actor network and a critic network are constructed to train the quadruped robot model. The strategy input includes 45-dimensional proprioception information. t , the estimated speed of the output of the state estimation module and features extracted by variational autoencoder And the output of the topographic map confidence estimation module out t , represents the confidence of the current map, which is used to weight the quadruped robot's proprioceptive information and the noisy height map information.

[0079] The actor network uses policy input to generate the quadruped's current action, specifically the difference in joint angles relative to the initial default joint angles, to maximize cumulative reward. The critic network accepts the same policy inputs, such as proprioception information, estimated velocity, latent vectors, and confidence vectors, but its goal is to evaluate the value of the current policy and provide an expected reward for an action. The critic network evaluates the value of the current state by calculating a state-value function and provides the expected reward for the action. During training, the policy and value networks are updated using the PPO algorithm, using the advantage function to adjust network parameters.

[0080] In this example, the input of the actor network is 45-dimensional proprioceptive information o t , the output in step 3 and out output in step 2 t, the actor network hidden layer has a dimension of {256, 256, 256}, and outputs 12-dimensional joint actions. The critic network inputs the proprioceptive information o t , 187-dimensional height map true value H true , PPO is used for training optimization.

[0081] S5 utilizes sensor fusion technologies, such as Fast-LIO2, to acquire LiDAR odometry and IMU data, generating a real-time height map of the quadruped robot's surroundings and updating the surrounding terrain. Fast-LIO2 outputs point cloud and IMU data at a frequency of 10Hz. In practical applications, using a 10Hz LiDAR odometry can result in an error of approximately 10cm. This is especially true when the robot is performing jumping tasks, as timing errors can affect the accuracy of task execution. To improve accuracy, the LiDAR odometry and IMU data are further fused using an extended Kalman filter (EKF), achieving a high-frequency odometry output of 200Hz.

[0082] The above specific embodiments are used to illustrate the present invention rather than to limit the present invention. Any modifications and changes made to the present invention within the spirit of the present invention and the protection scope of the claims shall fall within the protection scope of the present invention.

Claims

1. A quadruped robot motion control method based on terrain map reinforcement learning, characterized in that: The following steps are involved: S1, building a quadruped robot model on the simulation platform; S2, build the topographic map confidence estimation module, set the noise model in the simulation to obtain the height map h with noise t Simulating multiple real-world environments, using a quadruped robot with noise to perceive historical information and height map h t As input, it outputs the topographic map confidence weight and the reconstructed topographic features; S3, build a state estimation model based on the historical state observations perceived by the quadruped robot body and the latent vector l obtained by encoding the topographic map confidence estimation module in step S2 t As input, output is the estimated speed v of the quadruped robot at the current moment t , the next state of the quadruped robot is o t+1 , which serves as the basis for motion control decision-making of quadruped robots in complex terrain; S4, builds an actor network and a critic network to train the quadruped robot model; S5, uses sensor fusion technology to obtain lidar and inertial measurement unit data, generate a height map around the quadruped robot in real time, and update the environmental terrain information.

2. The quadruped robot motion control method according to claim 1, characterized in that: The step S2 includes the following sub-steps: S21, add random noise to the height map and build a noise model Where z is the noise parameter; S22, setting mapping conditions and configuring different noise parameters z for different conditions to simulate different terrain qualities and error sources; S23, the height map H with noise nosie The historical state observations of the quadruped robot's proprioception Input noise model.

3. The quadruped robot motion control method according to claim 2, wherein: In step S22, the noise model includes at least one of the following noise types: Normal noise, assuming that the map quality is good and suitable for routine operations, the map measurement error is simulated by Gaussian noise. The standard deviation of the Gaussian noise under the normal noise condition is σ∈[0.03m,0.08m]; Large offset noise: simulates map offset caused by pose estimation drift or terrain deformation through higher noise, and simulates pose estimation drift through spatial offset noise; the mean spatial offset of the large offset noise condition is μ∈[0.1m,0.2m]; Large noise amplitude applies larger noise to each scan point to simulate the complete loss of terrain information due to occlusion or sensor failure, and simulates sensor failure scenarios through random occlusion noise; the large noise amplitude condition applies random zero-value noise to at least 20% of the scan points in the height map.

4. The quadruped robot motion control method according to claim 2, wherein: The step S23 includes the following sub-steps: S231, height map H with noise nosie and historical state observations After passing through a network M1, M2 for feature compression, the vector h is obtained t and l t ; S232, h t and l t After splicing, it is fed into the fully connected network M3 to obtain the height map confidence vector b t ,b t Determines the proprioceptive information of the quadruped robot and height map H nosie The calculation process of how much weight is generated for the final height map is expressed as: b t =M1(cat(h t ,l t )) out t =b t +b t ☉l t Where ⊙ is element-wise multiplication, resulting in vector out t , during the training process, out t As input, it is fed into the decoder network to reconstruct the height map. The calculation formula is as follows: H e =M4(out t )+M4(out t )☉H true Among them, H e Is the estimated height map, H true A true height map.

5. The quadruped robot motion control method according to claim 4, characterized in that: The height map H e and the real height map H true The error is calculated using the mse loss function, and the loss function is 6. The quadruped robot motion control method according to claim 1, wherein: The step S3 includes the following sub-steps: S31, the state estimation model uses a deep network architecture, and the input includes the historical state observations of the quadruped robot proprioception and the latent vector l from the topographic map confidence estimation module t , the input together encodes the current state of the quadruped robot, the specific observation value o t is a 45-dimensional vector defined as follows; w t Indicates the robot's body angular velocity, g t represents the gravity vector of the body, c t Indicates the robot linear velocity and angular velocity instructions, θ t represents the joint angular velocity, represents the joint angular acceleration, a t-1 Indicates the offset of the joint position output by the strategy at the previous moment relative to the joint position at the initial moment; S32, the state estimation model uses the structure of the twin network and the variational autoencoder framework to extract deep features and generate the current quadruped robot speed and latent vector in The potential representation extracted by the variational autoencoder structure can effectively capture the complex characteristics of the environment and robot state; the state estimation module outputs the estimated speed at the current moment Represents the motion state of the robot; latent vector Contains implicit information about the current environment and the quadruped robot; S33, using quadruped robot speed and latent vector Jointly predict the next moment of the quadruped robot state o t+1 , the decoder network will and Combine and output the state prediction o at the next moment t+1 .

7. The quadruped robot motion control method according to claim 6, characterized in that: In step 3, the loss function in the variational autoencoder is used to train the network by minimizing the reconstruction error and the KL divergence of the latent variables. The loss function of the state estimation is: Used to evaluate the accuracy of the model in predicting the state at the next moment, the KL divergence of the latent variable is: The total loss is composed of the weighted reconstruction loss and KL divergence 8. The quadruped robot motion control method according to claim 1, wherein: In step 4, a reinforcement learning algorithm is used for training, including constructing an actor network and a critic network based on the PPO reinforcement learning algorithm to train the quadruped robot model. The strategy input includes the quadruped robot proprioception information o t , the estimated speed of the output of the state estimation module and features extracted by variational autoencoder And the output of the topographic map confidence estimation module out t , represents the confidence of the current map, which is used to weight the quadruped robot's proprioceptive information and the noisy height map information.

9. The quadruped robot motion control method according to claim 8, characterized in that: In step 4, the actor network uses the policy input information to generate the action of the quadruped robot at the current moment to maximize the cumulative reward; the critic network accepts the same policy input information, evaluates the value of the current state by calculating the state value function, and provides the expected return of the action.

10. The quadruped robot motion control method according to claim 1, wherein: In step 5, the lidar and inertial measurement unit data are fused through an extended Kalman filter, and the odometer output frequency reaches 200 Hz.

Images