A method and device for controlling the movement of a quadruped robot

Abstract

The application provides a quadruped robot motion control method and device, wherein the method comprises: collecting the body perception observation vector of the current control period of the quadruped robot and constructing a two-dimensional body perception history sequence matrix based on the body perception observation vector; inputting the two-dimensional body perception history sequence matrix into the time branch and the space branch of the space-time decoupling body perception device STPE to obtain a time characteristic vector and a space characteristic vector respectively, splicing and fusing the two characteristic vectors, and obtaining an original vector through linear projection; estimating the base linear velocity, the quadruped contact force and the constrained implicit terrain representation based on the original vector; constructing a two-stage reinforcement learning training network and training the two-stage reinforcement learning training network to obtain a target network model; inputting the constrained implicit terrain representation and the body perception observation vector into the target network model after splicing to obtain an output expected joint angle offset. Through the method, the control accuracy and robustness of the robot in blind motion on complex terrain can be improved.

Description

Technical Field

[0001] This invention relates to the field of smart hardware technology, specifically to a method and device for controlling the motion of a quadruped robot. Background Technology

[0002] Quadruped robots have broad application prospects in unstructured scenarios such as field exploration, disaster relief, industrial inspection, and remote mountainous operations. These environments are characterized by complex and varied terrain, including gravel roads, steep slopes, discrete obstacles, and slippery surfaces, placing extremely high demands on the robustness of motion controllers. In recent years, motion control methods based on deep reinforcement learning have made significant progress in the field of quadruped robots, resulting in a large number of end-to-end learning motion strategies.

[0003] However, existing high-performance strategies generally rely on external sensors such as visual cameras, LiDAR, or depth cameras to acquire terrain geometry information in advance. In actual outdoor deployments, factors such as drastic changes in lighting, dust and vegetation obstruction, motion blur caused by high-frequency oscillations of the robot body, and onboard computing delays can easily lead to the failure of external sensors or severe distortion of perceived data. In such cases, the robot must rely solely on the proprioceptive signals provided by the inertial measurement unit (IMU) and joint encoders for so-called "blind" motion control.

[0004] Current mainstream blind motion control methods mostly adopt a teacher-student two-stage distillation framework: in the first stage, the teacher's policy is trained in a simulation environment with the help of privileged information to obtain an ideal motion policy; in the second stage, the student's policy relies solely on the ontological perception history sequence and completes deployment by imitating the teacher's actions. Although this paradigm has promoted the development of blind motion control to some extent, two key technical bottlenecks remain unresolved:

[0005] (1) Existing ontology-aware encoders often simply flatten historical sequences or use a single temporal model for mixed processing, lacking explicit decoupling modeling of single-joint temporal dynamics and multi-joint spatial coordination. This makes it difficult for the network to distinguish between short-term foot-ground impact transients, such as the abrupt joint response after the foot hits an obstacle, and long-term gait periodic features, such as the phase coordination of the diagonal leg, thus limiting the feature representation ability and policy generalization ability on highly unstructured terrain.

[0006] (2) Teacher strategies are based on deterministic spatial geometric information, such as heightmap encoding of the environment, while student strategies can only infer probabilistic temporal response patterns from ontological perception history. These are essentially different perceptual modalities. Existing distillation methods impose fixed-weight alignment constraints on the teacher and student latent spaces throughout the training process. This leads to a severe imbalance in gradient scales between multi-task regressions such as velocity estimation and contact force estimation. Furthermore, it over-constrains student strategies, forcing them to reconstruct geometric information not present in the ontological perception input, hindering the development of proprietary dynamic representations adapted to their own sensory input. Ultimately, this results in insufficient robustness of the strategies in complex terrains, and the success rate of Sim-to-Real transfer from simulation to physical objects is difficult to guarantee.

[0007] Overall, there is a lack of robust motion control solutions in the current technology that can provide high stability and high precision in complex unstructured terrain where external sensors fail. Summary of the Invention

[0008] This invention addresses the problems of severe coupling of ontological perception features, over-constraint of cross-modal teacher-student distillation, and multi-task gradient imbalance in existing blind motion control schemes for quadruped robots. It proposes a quadruped robot motion control method and device that can achieve robust motion control with high stability and high precision in complex unstructured terrain where external sensors fail.

[0009] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0010] This invention provides a motion control method for a quadruped robot, wherein the method includes:

[0011] Collect the body perception observation vector of the quadruped robot in the current control cycle;

[0012] The ontology perception observation vector is segmented and valued according to a preset sliding window to construct a two-dimensional ontology perception history sequence matrix.

[0013] The two-dimensional ontology perception history sequence matrix is ​​input into the time branch of the spatiotemporally decoupled ontology perceptron STPE to obtain the time feature vector.

[0014] The two-dimensional ontology perception history sequence matrix is ​​input into the spatial branch of the spatiotemporally decoupled ontology perceptron STPE to obtain the spatial feature vector.

[0015] The time feature vector and the spatial feature vector are concatenated and fused, and then linearly projected to obtain the original vector;

[0016] Based on the original vector, the base linear velocity, quadrupedal contact force, and constrained implicit terrain representation are estimated.

[0017] A two-stage reinforcement learning training network is constructed, and the two-stage reinforcement learning training network is trained based on the base linear velocity and the four-leg contact force to obtain the target network model;

[0018] The constrained implicit terrain representation and the ontology perception observation vector are concatenated and input into the target network model to obtain the desired joint angle offset. The desired joint angle offset is converted into joint torque by a proportional-derivative controller to drive the quadruped robot's movement.

[0019] Optionally, the step of acquiring the body perception observation vector of the quadruped robot in the current control cycle includes:

[0020] Preset parameter information is acquired by the inertial measurement unit and the joint encoder in each control cycle. The preset parameter information includes: base angular velocity, projected gravity, command velocity, joint position error in the first dimension, joint velocity in the first dimension, and joint movement in the previous moment in the first dimension.

[0021] Generate the ontology perception observation vector for the current control cycle based on the preset parameter information obtained for the current control cycle.

[0022] Optionally, the step of inputting the two-dimensional ontology perception history sequence matrix into the time branch of the spatiotemporally decoupled ontology perceptron STPE to obtain the time feature vector includes:

[0023] Input the two-dimensional ontology perception history sequence matrix into the time branch of STPE;

[0024] The STPE uses a Long Short-Term Memory (LSTM) network to process each sequence in the two-dimensional ontology-aware historical sequence matrix along the time axis. For each sequence, the hidden state at the end of the sequence is extracted and obtained by linear projection and ELU activation function to obtain a time feature vector.

[0025] Optionally, the step of inputting the two-dimensional ontology perception history sequence matrix into the spatial branch of the spatiotemporally decoupled ontology perceptron (STPE) to obtain spatial feature vectors includes:

[0026] The two-dimensional ontology-aware historical sequence matrix is ​​input into the spatial branch of STPE, and the mean and standard deviation of each channel are calculated along the time axis.

[0027] The calculated mean and standard deviation of each channel are concatenated into an enhancement vector;

[0028] The enhancement vector is encoded by a multilayer perceptron to obtain a spatial feature vector.

[0029] Optionally, the step of constructing a two-stage reinforcement learning training network and training the two-stage reinforcement learning training network based on the base linear velocity and the four-legged contact force to obtain the target network model includes:

[0030] A two-stage reinforcement learning training network is constructed; wherein, the two-stage reinforcement learning training network includes a teacher strategy network and a student strategy network;

[0031] The teacher encoder is used to map multidimensional privileged information to generate teacher latent variables;

[0032] The vector formed by concatenating the teacher's latent variables and ontology perception vectors is input into the teacher policy network, and the teacher policy network is trained based on a dense reward mechanism to obtain the trained teacher policy network.

[0033] Freeze the teacher policy network and the teacher encoder, and use the spatiotemporal location encoder STPE to estimate the multidimensional privileged information to obtain the student latent variables;

[0034] The vector formed by concatenating the student latent variables and the ontology-aware vector is input into the student policy network. The STPE and the student policy network are jointly optimized by an adaptive moment estimation optimizer to obtain the trained student policy network.

[0035] Optionally, the multidimensional privileged information includes: true base linear velocity, quadruped contact state, friction coefficient, mass parameters, proportional-derivative gain offset, and terrain height scan information.

[0036] Optionally, the rewards introduced in the dense reward mechanism include: linear velocity tracking, angular velocity tracking, base posture, vertical velocity, joint torque, joint acceleration, rate of change of motion, foot airtime, collision penalty, and joint limit.

[0037] This invention also provides a motion control device for a quadruped robot, wherein the device includes:

[0038] The acquisition module is used to acquire the body perception observation vector of the quadruped robot in the current control cycle;

[0039] The construction module is used to segment and extract values ​​from the ontology perception observation vector according to a preset sliding window, and construct a two-dimensional ontology perception history sequence matrix.

[0040] The first prediction module is used to input the two-dimensional ontology perception history sequence matrix into the time branch of the spatiotemporally decoupled ontology perceptron STPE to obtain a time feature vector.

[0041] The second prediction module is used to input the two-dimensional ontology perception history sequence matrix into the spatial branch of the spatiotemporally decoupled ontology perceptron STPE to obtain a spatial feature vector.

[0042] The splicing module is used to splice and fuse the time feature vector and the spatial feature vector and obtain the original vector through linear projection;

[0043] An estimation module is used to estimate the base linear velocity, quadrupedal contact force, and constrained implicit terrain representation based on the original vector;

[0044] The model building module is used to build a two-stage reinforcement learning training network and train the two-stage reinforcement learning training network based on the base linear velocity and the four-leg contact force to obtain the target network model.

[0045] The offset prediction module is used to concatenate the constrained implicit terrain representation and the ontology perception observation vector and input them into the target network model to obtain the output expected joint angle offset. The expected joint angle offset is converted into joint torque by a proportional-derivative controller to drive the quadruped robot to move.

[0046] Optionally, the acquisition module includes:

[0047] The first submodule is used to acquire preset parameter information in each control cycle through the inertial measurement unit and the joint encoder. The preset parameter information includes: base angular velocity, projected gravity, command velocity, joint position error in the first dimension, joint velocity in the first dimension, and joint movement in the first dimension at the previous moment.

[0048] The second submodule is used to generate the ontology perception observation vector for the current control cycle based on the preset parameter information of the current control cycle.

[0049] Optionally, the first prediction module is specifically used for:

[0050] Input the two-dimensional ontology perception history sequence matrix into the time branch of STPE;

[0051] The STPE uses a Long Short-Term Memory (LSTM) network to process each sequence in the two-dimensional ontology-aware historical sequence matrix along the time axis. For each sequence, the hidden state at the end of the sequence is extracted and obtained by linear projection and ELU activation function to obtain a time feature vector.

[0052] Optionally, the second prediction module is specifically used for:

[0053] The two-dimensional ontology-aware historical sequence matrix is ​​input into the spatial branch of STPE, and the mean and standard deviation of each channel are calculated along the time axis.

[0054] The calculated mean and standard deviation of each channel are concatenated into an enhancement vector;

[0055] The enhancement vector is encoded by a multilayer perceptron to obtain a spatial feature vector.

[0056] Optionally, the model building module includes:

[0057] The third submodule is used to construct a two-stage reinforcement learning training network; wherein, the two-stage reinforcement learning training network includes a teacher strategy network and a student strategy network;

[0058] The fourth submodule is used to map multidimensional privileged information to generate teacher latent variables using a teacher encoder;

[0059] The fifth submodule is used to input the vector formed by concatenating the teacher's latent variables and ontology perception vector into the teacher policy network, and to train the teacher policy network based on the dense reward mechanism to obtain the trained teacher policy network.

[0060] The sixth submodule is used to freeze the teacher policy network and the teacher encoder, and to estimate the student latent variables by using the spatiotemporal location encoder STPE to estimate the multidimensional privileged information.

[0061] The seventh submodule is used to input the vector formed by concatenating the student latent variables and the ontology perception vector into the student policy network. The STPE and the student policy network are jointly optimized by an adaptive moment estimation optimizer to obtain the trained student policy network.

[0062] Optionally, the multidimensional privileged information includes: true base linear velocity, quadruped contact state, friction coefficient, mass parameters, proportional-derivative gain offset, and terrain height scan information.

[0063] Optionally, the rewards introduced in the dense reward mechanism include: linear velocity tracking, angular velocity tracking, base posture, vertical velocity, joint torque, joint acceleration, rate of change of motion, foot airtime, collision penalty, and joint limit.

[0064] This invention also provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; the memory is used to store computer programs; and the processor, when executing the program stored in the memory, implements any of the above-described quadruped robot motion control method flows.

[0065] The quadruped robot motion control scheme provided in this application uses a spatiotemporally decoupled ontology perception estimator (STPE) to extract temporal dynamics and spatial joint coordination features from the ontology perception history sequence, thereby suppressing redundant information and enhancing key terrain interaction features. It employs a progressive relaxation distillation strategy to optimize the latent space alignment and multi-task regression of the student strategy, dynamically adjusts constraint weights and adaptively balances gradient scales during cross-modal knowledge transfer, thereby improving the control accuracy and robustness of blind motion in complex terrain. Attached Figure Description

[0066] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0067] Figure 1 This is a flowchart illustrating the steps of a quadruped robot motion control method according to an embodiment of this application;

[0068] Figure 2 This is a diagram of the dual-branch architecture of the STPE encoder according to an embodiment of this application;

[0069] Figure 3 This is a diagram illustrating the progressive relaxation distillation loss architecture of an embodiment of this application.

[0070] Figure 4 This is a convergence curve of the gradual relaxation distillation loss in an embodiment of this application;

[0071] Figure 5 This is a graph showing the convergence curves of teacher and student strategy training in an embodiment of this application.

[0072] Figure 6 This is a structural block diagram of a quadruped robot motion control device according to an embodiment of this application. Detailed Implementation

[0073] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. Based on the embodiments of this application, every other embodiment obtained by those skilled in the art without inventive effort falls within the scope of protection of this application.

[0074] This invention discloses a motion control scheme for a quadruped robot. The scheme first acquires the quadruped robot's body perception observation vectors and constructs a sliding history sequence matrix. Then, through a spatiotemporally decoupled body perception estimator (STPE) with a two-branch architecture, the temporal branch uses a Long Short-Term Memory (LSTM) network to extract gait rhythm and foot-ground contact dynamic features, while the spatial branch calculates the mean and standard deviation of each channel along the time axis and encodes inter-joint spatial collaborative features using a Multilayer Perceptron (MLP). Next, the features from the two branches are fused and structured channel allocation is applied to output the base linear velocity, quadruped contact force, and implicit terrain representation. Finally, a teacher-student two-stage reinforcement learning framework is adopted. In the distillation stage, a progressive relaxation distillation strategy is introduced, dynamically adjusting the latent space alignment constraints through cosine annealing scheduling and balancing the multi-task loss gradient using an adaptive uncertainty weighting mechanism to output joint control commands. This scheme effectively solves the problems of severe coupling of body perception features, over-constraint in cross-modal teacher-student distillation, and multi-task gradient imbalance in existing blind motion control schemes. It can significantly improve the motion stability of quadruped robots in complex outdoor environments such as rugged terrain, slopes, and slippery surfaces, as well as the ability to transfer zero-shot simulations to real-world applications.

[0075] The quadruped robot motion control method provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0076] As attached Figure 1 As shown, the quadruped robot motion control method of this application embodiment includes the following steps:

[0077] Step 101: Collect the body perception observation vector of the quadruped robot in the current control cycle.

[0078] In one optional embodiment, the body perception observation vector of the quadruped robot in the current control cycle can be acquired as follows: the preset parameter information is obtained by the inertial measurement unit and the joint encoder in each control cycle, and the body perception observation vector of the current control cycle is generated based on the preset parameter information of the current control cycle.

[0079] The preset parameter information may include, but is not limited to: base angular velocity, projected gravity, command velocity, joint position error in the first dimension, joint velocity in the first dimension, and joint motion in the previous moment in the first dimension.

[0080] Step 102: Segment and extract values ​​from the ontology perception observation vector according to the preset sliding window to construct a two-dimensional ontology perception historical sequence matrix.

[0081]

[0082] The two-dimensional ontology perception history sequence matrix can be represented as follows: , Let t be the proprioception observation vector at time t. The proprioception observation vector contains the base angular velocity, projected gravity, command velocity, joint position error, joint velocity, and joint motion at the previous time step.

[0083] The specific length of the sliding window can be flexibly set by those skilled in the art, and no specific limitation is made in this embodiment. For example: the sliding window length T=50 corresponds to a 1.0 second motion trajectory, covering 1.25~1.67 complete jogging cycles; the dimension of the proprioception observation vector d=45, and the proprioception observation vector includes the base angular velocity, projected gravity, command velocity, 12-dimensional joint position error, 12-dimensional joint velocity, and 12-dimensional previous action.

[0084] Step 103: Input the two-dimensional ontology perception history sequence matrix into the time branch of the spatiotemporally decoupled ontology perceptron STPE to obtain the time feature vector.

[0085] In one optional embodiment, the two-dimensional ontology perception history sequence matrix is ​​input into the time branch of the spatiotemporally decoupled ontology perceptron (STPE) to obtain the time feature vector, which can be done as follows:

[0086] First, input the two-dimensional ontology perception history sequence matrix into the time branch of STPE;

[0087] Secondly, STPE uses a long short-term memory network (LSTM) to process each sequence in the two-dimensional ontology perception historical sequence matrix along the time axis. For each sequence, the hidden state at the end of the sequence is extracted and obtained by linear projection and ELU activation function to obtain the time feature vector.

[0088] Specifically, when using LSTM to process each sequence in the two-dimensional ontology perception history sequence matrix along the time axis, the information flow can be adjusted through forget gates, input gates, and output gates to extract the hidden states at the end of the sequence. The temporal feature vector is obtained by linear projection and ELU activation function. :

[0089]

[0090] Wherein, projection matrix , For bias, The time feature dimension.

[0091] Furthermore, the calculation formula for the LSTM gating mechanism of time branching can be expressed as:

[0092]

[0093]

[0094]

[0095]

[0096] in It is the sigmoid activation function. This represents element-wise multiplication (Hadamard product). Let t' be the input vector for frame t. for Hidden state vector ( ), Let be the cell state vector. This indicates a vector concatenation operation. , The outputs of the forget gate and the input gate are respectively. This is the corresponding weight matrix. The bias vector. Forget gate. The input gate determines which outdated motion state information to discard. Determine which signals from the current frame will be written into the cell state. What timing characteristics does the output gate control expose downstream?

[0097] Step 104: Input the two-dimensional ontology perception history sequence matrix into the spatial branch of the spatiotemporally decoupled ontology perceptron STPE to obtain the spatial feature vector.

[0098] In an optional embodiment, the method of inputting the two-dimensional ontology perception history sequence matrix into the spatial branch of the spatiotemporally decoupled ontology perceptron STPE to obtain spatial feature vectors may include the following sub-steps:

[0099] Sub-step 1: Input the two-dimensional ontology perception history sequence matrix into the spatial branch of STPE, and calculate the mean and standard deviation of each channel along the time axis; Sub-step 2: Concatenate the calculated mean and standard deviation of each channel into an enhancement vector; Sub-step 3: Encode the enhancement vector through a multilayer perceptron to obtain a spatial feature vector.

[0100] Historical sequence matrix Input the spatial branch of the STPE module and calculate the mean of each channel along the time axis. with standard deviation Concatenate into an enhanced vector The spatial feature vector is then obtained through two layers of multilayer perceptron (MLP) encoding. .

[0101]

[0102] The formula for calculating and encoding the statistical characteristics of spatial branches is as follows:

[0103]

[0104]

[0105]

[0106]

[0107] in, and These are the d-dimensional channel mean vector and standard deviation vector, respectively. For 2d-dimensional augmentation vectors, It is a multilayer perceptron containing two fully connected layers (90→64→32), with each layer followed by an ELU activation function.

[0108] Step 105: Concatenate and fuse the temporal feature vector and spatial feature vector, and obtain the original vector through linear projection.

[0109] Time feature vector and spatial eigenvectors The vectors are then spliced ​​and merged, and the original vector is output via linear projection. The specific expression can be as follows:

[0110]

[0111]

[0112] in, for 3D fusion vector, This is the weight matrix. For bias, for 3D output vector.

[0113] Step 106: Estimate the base linear velocity, quadruped contact force, and constrained implicit terrain representation based on the original vectors.

[0114] The temporal and spatial feature vectors are concatenated and fused, and then linearly projected to obtain the original vector. Then, structured channel allocation is performed through hard index slicing to output the estimated base linear velocity. , quadruped contact force and the implicit terrain representation z constrained by tanh:

[0115]

[0116]

[0117]

[0118] In a preferred embodiment, structured channel allocation will It is divided into three physically defined subspaces, among which For 3D base linear velocity estimation, For the 4D contact force estimation corresponding to the four legs (FL / FR / RL / RR), z is a 16-dimensional implicit terrain representation constrained to the interval [-1,1] by the tanh function, used to encode environmental dynamic attributes such as ground compliance and irregularity that are difficult to model explicitly.

[0119] Steps 101 to 106 use the spatiotemporally decoupled ontology perception estimator STPE to extract temporal dynamics and spatial joint collaboration features of the ontology perception historical sequence, thereby suppressing redundant information and enhancing key terrain interaction features.

[0120] Step 107: Construct a two-stage reinforcement learning training network and train the two-stage reinforcement learning training network based on the base linear velocity and the contact force of the four legs to obtain the target network model.

[0121] In one optional embodiment, constructing a two-stage reinforcement learning training network and training the two-stage reinforcement learning training network based on the base linear velocity and the four-legged contact force to obtain the target network model may include the following sub-steps:

[0122] Sub-step 1: Construct a two-stage reinforcement learning training network.

[0123] The two-stage reinforcement learning training network includes a teacher strategy network and a student strategy network.

[0124] Sub-step 2: Use a teacher encoder to map multidimensional privileged information to the generated teacher latent variables;

[0125] The multidimensional privileged information includes: true base linear velocity, four-legged contact state, friction coefficient, mass parameters, proportional-derivative gain offset, and terrain height scan information.

[0126] Sub-step 3: Input the vector formed by concatenating the teacher's latent variables and ontology perception vector into the teacher policy network, and train the teacher policy network based on the dense reward mechanism to obtain the trained teacher policy network.

[0127] The rewards introduced in the dense reward mechanism include: linear velocity tracking, angular velocity tracking, base posture, vertical velocity, joint torque, joint acceleration, rate of change of motion, foot time in the air, collision penalty, and joint limit.

[0128] Sub-step 4: Freeze the teacher policy network and teacher encoder, and use the spatiotemporal location encoder STPE to estimate the multidimensional privileged information to obtain the student latent variables;

[0129] Sub-step 5: Input the vector formed by concatenating the student latent variables and the ontology perception vector into the student policy network. STPE and the student policy network are jointly optimized by the adaptive moment estimation optimizer to obtain the trained student policy network.

[0130] This step constructs a two-stage teacher-student reinforcement learning training framework. The first stage utilizes the teacher's policy, which includes privileged information such as terrain elevation maps and friction coefficients, and trains the student's policy using the PPO algorithm. The second stage freezes the teacher network and trains the student's policy using a progressive relaxation distillation strategy. The total loss function is... Loss due to action cloning Adaptive uncertainty-weighted regression loss and Dynamic latent space alignment loss Four components:

[0131]

[0132] An example of calculating the adaptive uncertainty-weighted regression loss is as follows:

[0133]

[0134]

[0135] in, and Provides realistic contact force and base velocity labels for the simulation environment. and For a learnable homoscedastic uncertainty scalar, its logarithmic form and As free parameters, they are jointly optimized with network weights using the Adam optimizer; The term serves as an adaptive weight to dynamically adjust the gradient scale. item prevention Divergent.

[0136] An instance of dynamic latent space alignment loss and dynamic latent space alignment loss weights Gradual relaxation is performed using cosine annealing scheduling, and the calculation formula is as follows:

[0137]

[0138]

[0139] in, The implicit terrain representation output by the STPE module for students. The teacher network is represented by the privileged information encoding; t is the current training iteration number. and These are the initial and final weights of the alignment loss, respectively. This is the decay period; it continues with training. Depend on Smooth decay to Gradually reduce the geometric constraints on students' latent space representations.

[0140] In this embodiment, a progressive relaxation distillation strategy is used to optimize the latent space alignment and multi-task regression of the student strategy. During the cross-modal knowledge transfer process, the constraint weights are dynamically adjusted and the gradient scale is adaptively balanced, thereby improving the control accuracy and robustness of blind motion in complex terrain.

[0141] Step 108: After concatenating the constrained implicit terrain representation and the ontology-aware observation vector, input the concatenation into the target network model to obtain the output expected joint angle offset.

[0142] The desired joint angle offset is converted into joint torque by a proportional-derivative controller (PD controller) to drive the quadruped robot's movement.

[0143] The quadruped robot motion control method provided in this application extracts temporal dynamics and spatial joint coordination features from the historical sequence of the ontology perception through a spatiotemporally decoupled ontology perception estimator STPE, thereby suppressing redundant information and enhancing key terrain interaction features. It also employs a progressive relaxation distillation strategy to optimize the latent space alignment and multi-task regression of the student strategy, dynamically adjusts constraint weights and adaptively balances gradient scales during cross-modal knowledge transfer, thereby improving the control accuracy and robustness of blind motion in complex terrain.

[0144] The following example illustrates the quadruped robot motion control method provided in this application.

[0145] This specific example presents a robust motion control method for a quadruped robot based on spatiotemporally decoupled ontology perception estimation. It extracts spatiotemporally decoupled features from the ontology perception history sequence using an STPE encoder and employs a progressive relaxation distillation strategy to optimize cross-modal knowledge transfer. The dual-branch architecture diagram of the TPE encoder is shown below. Figure 2 As shown, the progressive relaxation distillation loss architecture Figure 3 As shown in the figure. This embodiment of the application applies the method to the Unitree Go2 quadruped robot platform, trains it in the NVIDIA IsaacGym parallel simulation environment (PhysX physics engine), and verifies it on a physical robot using zero-shot Sim-to-Real transfer learning. The specific steps are as follows:

[0146] Step 1: Acquire the body perception observation vector of the quadruped robot in the current control cycle. This vector is obtained by the IMU and joint encoder in each control cycle and includes the base angular velocity, projected gravity, command velocity, 12-dimensional joint position error, 12-dimensional joint velocity, and 12-dimensional previous action. Maintain a sliding history window of length T=50 to construct a two-dimensional body perception history sequence matrix. This corresponds to a 1.0-second motion trajectory. At a control frequency of 50 Hz, this window covers approximately 1.25–1.67 complete gait cycles, including complete gait phase information (4–6 independent foot-ground contact events) and short-duration impact transients on a scale of 0.05–0.1 seconds. During environment reset, the history buffer is cleared and initialized with the current observations.

[0147] Step 2: The input is a time branch of the STPE encoder. A single-layer LSTM is used to process the sequence along the time axis. At each time step... LSTM regulates the information flow through forget gates, input gates, and output gates. The gating mechanism and the formula for calculating the projection of time features are as follows:

[0148]

[0149]

[0150]

[0151]

[0152]

[0153] The sequence consists of a forget gate (to discard outdated contact information when the robot moves to new terrain), an input gate (to write new foot contact cues, such as abrupt deceleration after hitting an obstacle or rhythmic mechanical patterns when climbing stairs), and an output gate (to encode multi-scale dynamic information into the hidden state). The hidden state at the end of the sequence is extracted, passed through a linear layer (64→32), and projected using the ELU activation function to obtain a 32-dimensional temporal feature vector. ELU was chosen instead of ReLU because it can preserve the negative value information that frequently appears in the body perception signal.

[0154] Step 3: Put Input the spatial branch of the STPE module. Calculate the mean for each of the 45 channels along the time axis. with standard deviation And concatenate them into a 90-dimensional augmented vector. :

[0155]

[0156]

[0157]

[0158]

[0159] mean Reflects the average joint posture within the window; for example, when a robot walks on a ramp, the hip joint will form a forward tilting posture bias that differs from that on flat ground; standard deviation. Reflecting the state of joint fluctuations—the variance of the standing leg is lower during the support phase, while the variance increases significantly during the swing phase or when walking on wet surfaces. The contrast of the diagonal leg pairs on h_std further implicitly encodes the trotting phase. A 32-dimensional spatial feature vector is obtained by encoding through two layers of multilayer perceptron (90→64→32, with each layer followed by an ELU activation function). .

[0160] Step 4: and Concatenate into a 64-dimensional fused vector The original 23-dimensional vector y_raw is output through linear projection, and then structured channel allocation is performed through hard index slicing:

[0161]

[0162]

[0163]

[0164]

[0165]

[0166] The first three dimensions estimate the base linear velocity, and dimensions 4-7 estimate the contact force of the four legs (FL / FR / RL / RR). Dimensions 8-23 are constrained to the [-1,1] interval using the tanh function, resulting in a 16-dimensional implicit terrain representation z. The first two channels provide interpretable physical quantities and are explicitly supervised during the distillation stage. The third channel encodes dynamic environmental properties that are difficult to model explicitly, such as ground compliance and surface irregularities, providing a basis for adjusting the foot landing point before contact occurs. The network structure parameters of the STPE encoder are shown in Table 1.

[0167] Table 1 STPE Network Architecture Parameters

[0168]

[0169] Step 5: Construct a two-stage reinforcement learning training framework for teachers and students.

[0170] Phase 1 Teacher Strategy Training: Teacher strategies incorporate 223-dimensional privileged information, including real base linear velocity, quadrupedal contact state, friction coefficient, mass parameters, PD gain offset, and terrain height scan. The teacher encoder is a three-layer MLP that maps the 223-dimensional privileged information to 16-dimensional teacher latent variables. Teacher strategy observation is based on ontology-aware vectors. and The input is assembled into a 61-dimensional structure. The PPO algorithm is used for training combined with terrain learning. The robot starts training on flat terrain and gradually progresses to more challenging terrain as its performance improves. The training employs dense rewards, including ten reward items: linear velocity tracking, angular velocity tracking, base posture, vertical velocity, joint torque, joint acceleration, rate of change of motion, foot airtime, collision penalty, and joint limit. The function definitions and weights of the reward items are detailed in Table 2.

[0171] Table 2. Reward Function Definition and Corresponding Weights

[0172]

[0173] Second-stage student policy distillation: Freezing all parameters of the teacher policy and teacher encoder. The student policy is estimated using the STPE encoder. ; ; z], and the hidden variables Compared with current observations The input and teacher policy networks are concatenated. The STPE encoder and student policy network are jointly optimized using the Adam optimizer, and the total loss function is:

[0174]

[0175] Among them: Action Cloning Loss weight =1.0 is fixed throughout, serving as the baseline scale; the contact force and velocity regression loss employs adaptive uncertainty weighting, with learnable parameters... and Joint optimization with network weights, The term serves as an adaptive weight to dynamically adjust the gradient scale. item prevention Divergence; Latent Space Alignment Loss weight Following cosine annealing scheduling, the value smoothly decays from an initial value of 0.5 to a final value of 0.01. The changes in various parameters during training are as follows: Figure 4 The convergence curve of the gradual relaxation distillation loss is shown in the figure. Figure 4 'a' represents the training graph for action cloning loss, with the horizontal axis representing the number of iterations and the vertical axis representing the action cloning loss. Figure 4b is the training graph of the learnable parameters, where the horizontal axis represents the number of iterations and the vertical axis represents the number of learnable parameters. and The value; Figure 4 c is the adaptive weighted training graph, where the horizontal axis represents the number of iterations and the vertical axis represents the weights of the contact force and velocity loss terms. Figure 4 d represents the cosine annealing training graph, with the horizontal axis representing the number of iterations and the vertical axis representing the latent space alignment loss. weight The loss configuration during the distillation stage is shown in Table 3.

[0176] Table 3 Summary of Distillation Loss Configurations

[0177]

[0178] Step 6: Compare the implicit terrain representation z with the current observations The vectors are concatenated into a 61-dimensional input to the student policy network, outputting a 12-dimensional desired joint angle offset. The offset is then scaled by a factor α = 0.25 and processed by the underlying PD controller (…). =20 N·m / rad, =0.5 N·m·s / rad) is converted into joint torque to drive the robot's joints to perform movements. For example Figure 5 The convergence curves of teacher strategy and student strategy training are shown in the graph. Figure 5 'a' represents the teacher strategy training diagram, where the horizontal axis represents the number of iterations and the vertical axis represents the reward value. Figure 5 b represents the training graph of the student strategy, with the horizontal axis representing the number of iterations and the vertical axis representing the reward value. The training reward curves of both the teacher's and student's strategies converge stably within 10,000 iterations, and the final performance of the student strategy approximates that of the teacher's strategy.

[0179] Implementation Verification: Training was conducted on the NVIDIA Isaac Gym parallel simulation platform using 4096 environments. Test terrains included rugged gravel, rough slopes, discrete obstacles, stairs, and slippery surfaces (friction coefficient μ=0.3). Experimental results showed that the method of this invention achieved an average success rate of over 95% on five terrain types, with a velocity tracking error as low as 0.066~0.125 m / s and a significantly lower number of collisions than the baseline method. Zero-shot Sim-to-Real migration tests were performed on the Unitree Go2 physical robot, achieving a 90%~100% success rate in eight types of complex outdoor terrain, including muddy gravel, steep slopes, and slippery stairs. Ablation experiments confirmed that removing the temporal or spatial branches resulted in a 10% and 25% decrease in success rate, respectively. Fixing the distillation weights or the normalization coefficients both caused gradient imbalance or over-constraint, verifying the necessity and synergistic effectiveness of the various modules in this invention.

[0180] The quadruped robot motion control method provided in this application, in its first aspect, employs a spatiotemporally decoupled body perception estimator (STPE) dual-branch architecture as a feature extraction module, explicitly separating the temporal dynamics and spatial joint coordination information in the body perception signal. The temporal branch effectively captures multi-scale dynamic features of short-term foot impact and long-term gait cycles through an LSTM gating mechanism; the spatial branch reflects the average joint posture bias and support / swing phase fluctuations through channel mean and standard deviation statistics, and implicitly encodes diagonal leg phase coordination. This decoupled design significantly enhances the feature representation accuracy and robustness under unstructured terrain, especially showing obvious advantages in highly complex working conditions such as rugged terrain and discrete obstacles.

[0181] Secondly, this method employs a progressive relaxation distillation strategy, dynamically reducing the weight of the teacher-student latent space alignment constraint through cosine annealing scheduling. This mechanism allows student strategies to receive reasonable guidance from the teacher's geometric representation in the early stages of training for rapid convergence, and gradually removes strong constraints in the later stages of training, autonomously developing probabilistic dynamic representations adapted to ontology-aware input. This effectively bridges the teacher-student cross-modal gap, significantly improving the strategy's generalization ability to unknown terrains and the success rate of zero-shot simulation to real-world transfer.

[0182] Thirdly, this method introduces an adaptive uncertainty weighting mechanism, modeling the homoscedastic uncertainty of the contact force and velocity regression tasks as learnable parameters to dynamically balance the gradient scale of the multi-task loss. This design eliminates the need for manual tuning of normalization coefficients, automatically solves the gradient imbalance problem between the contact force impact signal and the smoothed velocity signal, and significantly improves the convergence stability and final control accuracy of the distillation stage.

[0183] Figure 6 The structural block diagram of a quadruped robot motion control device according to an embodiment of this application is shown.

[0184] The quadruped robot motion control device provided in this application includes the following functional modules:

[0185] The acquisition module 601 is used to acquire the body perception observation vector of the quadruped robot in the current control cycle;

[0186] The construction module 602 is used to segment and extract values ​​from the ontology perception observation vector according to a preset sliding window to construct a two-dimensional ontology perception history sequence matrix.

[0187] The first prediction module 603 is used to input the two-dimensional ontology perception history sequence matrix into the time branch of the spatiotemporally decoupled ontology perceptron STPE to obtain a time feature vector.

[0188] The second prediction module 604 is used to input the two-dimensional ontology perception history sequence matrix into the spatial branch of the spatiotemporally decoupled ontology perceptron STPE to obtain a spatial feature vector.

[0189] The splicing module 605 is used to splice and fuse the time feature vector and the spatial feature vector and obtain the original vector through linear projection;

[0190] Estimation module 606 is used to estimate the base linear velocity, quadruped contact force, and constrained implicit terrain representation based on the original vector;

[0191] The model building module 607 is used to build a two-stage reinforcement learning training network and train the two-stage reinforcement learning training network based on the base linear velocity and the four-leg contact force to obtain the target network model.

[0192] The offset prediction module 608 is used to input the constrained implicit terrain representation and the ontology perception observation vector into the target network model to obtain the output expected joint angle offset, wherein the expected joint angle offset is converted into joint torque by a proportional-derivative controller to drive the quadruped robot to move.

[0193] Optionally, the acquisition module includes:

[0194] The first submodule is used to acquire preset parameter information in each control cycle through the inertial measurement unit and the joint encoder. The preset parameter information includes: base angular velocity, projected gravity, command velocity, joint position error in the first dimension, joint velocity in the first dimension, and joint movement in the first dimension at the previous moment.

[0195] The second submodule is used to generate the ontology perception observation vector for the current control cycle based on the preset parameter information of the current control cycle.

[0196] Optionally, the first prediction module is specifically used for:

[0197] Input the two-dimensional ontology perception history sequence matrix into the time branch of STPE;

[0198] The STPE uses LSTM to process each sequence in the two-dimensional ontology-aware historical sequence matrix along the time axis, extracts the hidden state at the end of each sequence, and obtains the time feature vector through linear projection and ELU activation function.

[0199] Optionally, the second prediction module is specifically used for:

[0200] The two-dimensional ontology-aware historical sequence matrix is ​​input into the spatial branch of STPE, and the mean and standard deviation of each channel are calculated along the time axis.

[0201] The calculated mean and standard deviation of each channel are concatenated into an enhancement vector;

[0202] The enhancement vector is encoded by a multilayer perceptron to obtain a spatial feature vector.

[0203] Optionally, the model building module includes:

[0204] The third submodule is used to construct a two-stage reinforcement learning training network; wherein, the two-stage reinforcement learning training network includes a teacher strategy network and a student strategy network;

[0205] The fourth submodule is used to map multidimensional privileged information to generate teacher latent variables using a teacher encoder;

[0206] The fifth submodule is used to input the vector formed by concatenating the teacher's latent variables and ontology perception vector into the teacher policy network, and to train the teacher policy network based on the dense reward mechanism to obtain the trained teacher policy network.

[0207] The sixth submodule is used to freeze the teacher policy network and the teacher encoder, and to estimate the student latent variables by using the spatiotemporal location encoder STPE to estimate the multidimensional privileged information.

[0208] The seventh submodule is used to input the vector formed by concatenating the student latent variables and the ontology perception vector into the student policy network. The STPE and the student policy network are jointly optimized by an adaptive moment estimation optimizer to obtain the trained student policy network.

[0209] Optionally, the multidimensional privileged information includes: true base linear velocity, quadruped contact state, friction coefficient, mass parameters, proportional-derivative gain offset, and terrain height scan information.

[0210] Optionally, the rewards introduced in the dense reward mechanism include: linear velocity tracking, angular velocity tracking, base posture, vertical velocity, joint torque, joint acceleration, rate of change of motion, foot airtime, collision penalty, and joint limit.

[0211] The embodiments provided in this application Figure 6 The shown quadruped robot motion control device can achieve Figure 1 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0212] The quadruped robot motion control device provided in this application uses a spatiotemporally decoupled body perception estimator (STPE) to extract temporal dynamics and spatial joint coordination features from the body perception history sequence, thereby suppressing redundant information and enhancing key terrain interaction features. It employs a progressive relaxation distillation strategy to optimize the latent space alignment and multi-task regression of the student strategy, dynamically adjusts constraint weights and adaptively balances gradient scales during cross-modal knowledge transfer, thereby improving the control accuracy and robustness of blind motion in complex terrain.

[0213] This invention also provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus.

[0214] Memory, used to store computer programs;

[0215] When the processor executes the program stored in the memory, it implements the quadruped robot motion control method shown in the above method embodiments.

[0216] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc.

[0217] The communication interface is used for communication between the aforementioned terminal and other devices.

[0218] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0219] In another embodiment of the present invention, a computer-readable storage medium is also provided, which stores instructions that, when executed on an electronic device, cause the electronic device to implement any of the quadruped robot motion control methods described in the above embodiments.

[0220] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on an electronic device, enables the electronic device to implement any of the quadruped robot motion control methods described in the above embodiments.

[0221] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0222] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A motion control method for a quadruped robot, characterized in that, The method includes: Collect the body perception observation vector of the quadruped robot in the current control cycle; The ontology perception observation vector is segmented and valued according to a preset sliding window to construct a two-dimensional ontology perception history sequence matrix. The two-dimensional ontology perception history sequence matrix is ​​input into the time branch of the spatiotemporally decoupled ontology perceptron STPE to obtain the time feature vector. The two-dimensional ontology perception history sequence matrix is ​​input into the spatial branch of the spatiotemporally decoupled ontology perceptron STPE to obtain the spatial feature vector. The time feature vector and the spatial feature vector are concatenated and fused, and then linearly projected to obtain the original vector; Based on the original vector, the base linear velocity, quadrupedal contact force, and constrained implicit terrain representation are estimated. A two-stage reinforcement learning training network is constructed, and the two-stage reinforcement learning training network is trained based on the base linear velocity and the four-leg contact force to obtain the target network model; The constrained implicit terrain representation and the ontology perception observation vector are concatenated and input into the target network model to obtain the desired joint angle offset. The desired joint angle offset is converted into joint torque by a proportional-derivative controller to drive the quadruped robot's movement.

2. The method according to claim 1, characterized in that, The steps for acquiring the body perception observation vectors of a quadruped robot during its current control cycle include: Preset parameter information is acquired by the inertial measurement unit and the joint encoder in each control cycle. The preset parameter information includes: base angular velocity, projected gravity, command velocity, joint position error in the first dimension, joint velocity in the first dimension, and joint movement in the previous moment in the first dimension. Generate the ontology perception observation vector for the current control cycle based on the preset parameter information obtained for the current control cycle.

3. The method according to claim 1, characterized in that, The step of inputting the two-dimensional ontology perception history sequence matrix into the time branch of the spatiotemporally decoupled ontology perceptron STPE to obtain the time feature vector includes: Input the two-dimensional ontology perception history sequence matrix into the time branch of STPE; The STPE uses a Long Short-Term Memory (LSTM) network to process each sequence in the two-dimensional ontology-aware historical sequence matrix along the time axis. For each sequence, the hidden state at the end of the sequence is extracted and obtained by linear projection and ELU activation function to obtain a time feature vector.

4. The method according to claim 1, characterized in that, The step of inputting the two-dimensional ontology perception history sequence matrix into the spatial branch of the spatiotemporally decoupled ontology perceptron STPE to obtain spatial feature vectors includes: The two-dimensional ontology-aware historical sequence matrix is ​​input into the spatial branch of STPE, and the mean and standard deviation of each channel are calculated along the time axis. The calculated mean and standard deviation of each channel are concatenated into an enhancement vector; The enhancement vector is encoded by a multilayer perceptron to obtain a spatial feature vector.

5. The method according to claim 1, characterized in that, The steps of constructing a two-stage reinforcement learning training network and training the two-stage reinforcement learning training network based on the base linear velocity and the four-legged contact force to obtain the target network model include: A two-stage reinforcement learning training network is constructed; wherein, the two-stage reinforcement learning training network includes a teacher strategy network and a student strategy network; The teacher encoder is used to map multidimensional privileged information to generate teacher latent variables; The vector formed by concatenating the teacher's latent variables and ontology perception vectors is input into the teacher policy network, and the teacher policy network is trained based on a dense reward mechanism to obtain the trained teacher policy network. Freeze the teacher policy network and the teacher encoder, and use the spatiotemporal location encoder STPE to estimate the multidimensional privileged information to obtain the student latent variables; The vector formed by concatenating the student latent variables and the ontology-aware vector is input into the student policy network. The STPE and the student policy network are jointly optimized by an adaptive moment estimation optimizer to obtain the trained student policy network.

6. The method according to claim 5, characterized in that, The multidimensional privileged information includes: true base linear velocity, four-legged contact state, friction coefficient, mass parameters, proportional-derivative gain offset, and terrain height scan information.

7. The method according to claim 1, characterized in that, The rewards introduced in the dense reward mechanism include: linear velocity tracking, angular velocity tracking, base posture, vertical velocity, joint torque, joint acceleration, rate of change of motion, foot airtime, collision penalty, and joint limit.

8. A motion control device for a quadruped robot, characterized in that, The device includes: The acquisition module is used to acquire the body perception observation vector of the quadruped robot in the current control cycle; The construction module is used to segment and extract values ​​from the ontology perception observation vector according to a preset sliding window, and construct a two-dimensional ontology perception history sequence matrix. The first prediction module is used to input the two-dimensional ontology perception history sequence matrix into the time branch of the spatiotemporally decoupled ontology perceptron STPE to obtain a time feature vector. The second prediction module is used to input the two-dimensional ontology perception history sequence matrix into the spatial branch of the spatiotemporally decoupled ontology perceptron STPE to obtain a spatial feature vector. The splicing module is used to splice and fuse the time feature vector and the spatial feature vector and obtain the original vector through linear projection; An estimation module is used to estimate the base linear velocity, quadrupedal contact force, and constrained implicit terrain representation based on the original vector; The model building module is used to build a two-stage reinforcement learning training network and train the two-stage reinforcement learning training network based on the base linear velocity and the four-leg contact force to obtain the target network model. The offset prediction module is used to concatenate the constrained implicit terrain representation and the ontology perception observation vector and input them into the target network model to obtain the output expected joint angle offset. The expected joint angle offset is converted into joint torque by a proportional-derivative controller to drive the quadruped robot to move.

9. The apparatus according to claim 8, characterized in that, The acquisition module includes: The first submodule is used to acquire preset parameter information in each control cycle through the inertial measurement unit and the joint encoder. The preset parameter information includes: base angular velocity, projected gravity, command velocity, joint position error in the first dimension, joint velocity in the first dimension, and joint movement in the first dimension at the previous moment. The second submodule is used to generate the ontology perception observation vector for the current control cycle based on the preset parameter information of the current control cycle.

10. The apparatus according to claim 8, characterized in that, The first prediction module is specifically used for: Input the two-dimensional ontology perception history sequence matrix into the time branch of STPE; The STPE uses a Long Short-Term Memory (LSTM) network to process each sequence in the two-dimensional ontology-aware historical sequence matrix along the time axis. For each sequence, the hidden state at the end of the sequence is extracted and obtained by linear projection and ELU activation function to obtain a time feature vector.

Images