A quadruped robot motion control method based on multi-modal online estimation

Abstract

The application discloses a kind of based on multimodal online estimation four-legged robot motion control method, system, electronic equipment and storable medium, the control method steps are as follows: input multimodal online estimator to comprehensive body perception information, parallel decoupling output explicit base linear velocity estimation value, explicit foot end contact state estimation value and implicit terrain representation, and constitute hybrid physical feature;Mixed physical feature and current time body perception information are input into strategy network, to generate joint position instruction.In the training phase, the estimator is supervised and constrained using the first privileged observation information, achieving single-stage joint convergence of state estimation and policy optimization.The application avoids the distribution shift problem caused by two-stage distillation by structurally decoupling dynamics and environment representation, enhances the environment state recognition accuracy in unstructured terrain, and improves the control robustness and migration performance in visual perception limited working conditions.

Description

Technical Field

[0001] This invention belongs to the field of legged robot technology, specifically relating to a method, system, electronic device, and storage medium for motion control of a quadruped robot based on multimodal online estimation. Background Technology

[0002] In unstructured terrain, quadruped robots demonstrate superior mobility compared to wheeled and tracked robots. However, in practical deployments, especially in complex scenarios such as field exploration, disaster relief, or underground mines, external sensing sensors like visual cameras and lidar are highly susceptible to environmental and weather interference, leading to their failure. Therefore, robust blind motion control relying solely on proprioceptive information such as joint encoders and inertial measurement units is a key technology for ensuring the high reliability of legged robots in all weather conditions.

[0003] Current mainstream technologies typically employ a two-stage distillation architecture of "teacher-student". While this two-stage distillation method achieves some degree of transfer from simulation to reality, it still has significant drawbacks. First, this architecture is essentially an offline supervised regression process, lacking an online correction mechanism, and the accumulation of inference errors over time leads to distribution shift problems. Second, the two-stage architecture requires training the privileged "teacher" policy and the "student" policy sequentially, resulting in a lengthy overall training cycle and low development efficiency; moreover, the motion performance of the "student" policy is strictly limited by the performance ceiling of the "teacher" policy, making it difficult to achieve further breakthroughs in dynamic response capabilities under extreme terrain. Furthermore, the implicit representations generated by existing distillation frameworks often mix and entangle robot dynamic states with environmental features, lacking physical interpretability. This makes it impossible to distinguish the fault source when motion fails, limiting the system's debugging efficiency and robust optimization space. Therefore, there is an urgent need for a robot motion control scheme that can overcome the distribution shift problem caused by two-stage distillation, achieve decoupled estimation of dynamic states and environmental features, and significantly improve training efficiency and motion performance. Summary of the Invention

[0004] The purpose of this invention is to overcome the problems of distribution offset, internal and external state coupling, and low training efficiency in existing quadruped robot motion control technology based on the "teacher-student" two-stage distillation architecture. It provides a quadruped robot motion control method based on multimodal online estimation. Through a single-stage end-to-end optimization framework, it achieves decoupling between explicit estimation of dynamic state and implicit representation of terrain. While reducing training complexity, it significantly improves the motion robustness and environmental adaptability of quadruped robots in unstructured terrain.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] In a first aspect, the present invention provides a motion control method for a quadruped robot based on multimodal online estimation, the control method comprising the following steps:

[0007] S1. Obtain the environmental state S at the current time t. t The integrated ontological perception information of the quadruped robot, including the environmental state S t This includes the ontology perception information at the current time t, the first privileged observation information, and the second privileged observation information;

[0008] S2. Input the comprehensive ontology perception information into the preliminary multimodal online estimator for feature encoding and decoupling to obtain hybrid physical features;

[0009] S3. Perform feature extraction and threshold determination on the first privileged observation information at the current time t to obtain the target physical characteristics;

[0010] S4. Input the hybrid physical features and the body perception information at the current time t into the preliminary strategy network, and output the joint position deviation a. t Input the first privileged observation information and the comprehensive ontology perception information at the current time t into the value network, and output the state value V at the current time t. t ;

[0011] S5, the quadruped robot based on joint position deviation a t After performing the action, obtain the environment state S at time t+1. t+1 Based on environmental state S t+1 Calculate the single-step compound reward function ;

[0012] S6, based on environmental conditions S t+1 Update the comprehensive ontology perception information, repeat steps S1 to S4, and obtain the state value V at time t+1. t+1 Based on the single-step composite reward function State value V t and state value V t+1 Thus, the reinforcement learning loss function is obtained;

[0013] The initial policy network is trained by the reinforcement learning loss function to obtain the trained target policy network. Based on the deviation between the mixed physical features and the target physical features, a multi-task estimation loss function is obtained. The initial multimodal online estimator is trained by the multi-task estimation loss function to obtain the trained target multimodal online estimator.

[0014] S7. In real-world scenarios, acquire comprehensive ontological perception information of the quadruped robot in real time and input it into the target multimodal online estimator to obtain hybrid physical features. Input the hybrid physical features and ontological perception information at the current time t into the target policy network to obtain joint control commands.

[0015] Furthermore, step S1 is performed as follows:

[0016] S11. Obtain the body perception information of the quadruped robot at the current time t. t This includes the user's velocity command for the quadruped robot, the base angular velocity, the gravity vector, the position error of the 12 joints relative to their initial positions, the velocities of the 12 joints, and the motion vectors of the 12 joints from the previous moment; specifically, the 12 joints are the hip, thigh, and lower leg joints of the left and right front legs, left and right hind legs of the quadruped robot. The aforementioned proprioceptive information... t The configuration constitutes the minimum observable set for the robot's underlying control. The gravity vector and base angular velocity provide direct references for the robot's posture; the information from the 12 joints contains the robot's underlying actuator dynamics. Acquiring this information aims to provide the underlying data for characterizing the current transient physical response of subsequent networks, ensuring the physical feasibility of motion control and the continuous smoothness of actions.

[0017] S12, Obtain the quadruped robot's body perception information at the current time t. t The ontological perception information of the past H historical moments tH,…,t-1 t-H , … , o t-1 By splicing, we obtain the comprehensive ontology perception information {o t-H , … , o t Since real-world sensors inevitably suffer from observation noise and communication delays, comprehensive ontological perception information, including historical observation data, is introduced. t-H , … , o t This enables neural networks to accurately infer the higher-order kinematics and dynamics evolution of the robot (i.e., explicit base linear velocity estimates). Explicit foot contact state estimation value ) and the geometric shape of the external environment (i.e., preliminary implicit topographic representation) This greatly improves the robustness and anti-interference ability of the algorithm when it is transferred from the simulation environment to the real physical world.

[0018] S13. Obtain the first privileged observation information at the current time t, including the base linear velocity at the current time t. Foot contact force f t Foot swing height d tAnd the local terrain elevation map around the quadruped robot e t The system acquires the second privileged observation information at the current time t, including the base height, the horizontal position of each of the four legs relative to the base, and the horizontal velocity of each of the four legs relative to the base. Specifically, the four legs are the left foreleg, right foreleg, left hindleg, and right hindleg of the quadruped robot. The first and second privileged observation information are typically acquired during the simulation training phase as absolute ground truth values ​​characterizing the quadruped robot's interaction with the physical environment. The first privileged observation information contains the key external environment interaction states that determine the robot's dynamic stability in complex terrain, providing a low-variance, high-precision state evaluation benchmark for the value network. The second privileged observation information further supplements the true kinematic values ​​of the body and legs in the absolute spatial coordinate system. The comprehensive introduction of this privileged information provides a powerful guiding signal for the subsequent feature decoupling and network supervision of the preliminary multimodal online estimator, effectively accelerating the training convergence speed of the preliminary multimodal online estimator and the preliminary policy network in complex unstructured terrain, and significantly improving the obstacle-crossing performance upper limit of the final motion control method.

[0019] Furthermore, step S2 is as follows:

[0020] S21, Integrate ontological perception information {o t-H , … , o t The flattening operation is used to transform the temporal ontology perception feature vector into a comprehensive ontology perception information containing multi-time quadruped robot ontology perception data. t-H , … , o t The data is transformed into a linear representation suitable for parallel computing of neural networks. While preserving the temporal evolution characteristics, it ensures the structured alignment of the data in the feature space, providing a standardized data entry point for subsequent high-dimensional nonlinear feature extraction.

[0021] S22. Input the temporal ontology-aware feature vector into the preliminary multimodal online estimator. The preliminary multimodal online estimator is a multilayer perceptron network consisting of an input layer, a shared hidden layer, and a multi-head output layer connected in sequence.

[0022] S23. High-dimensional spatiotemporal features are extracted from the temporal ontology-aware feature vectors through a shared hidden layer to obtain shared latent features. This step utilizes the shared hidden layer as the underlying feature extraction entity and mines the intrinsic mechanical relationships between different observation modalities through shared network hierarchical mapping. Its working principle lies in: compressing and extracting spatiotemporal representations with multi-task commonalities from the shared latent features through deep nonlinear fusion of temporal ontology-aware features. This design leverages the complementary information characteristics between different physical quantities, improving the compactness of feature representations while providing a consistent and high-confidence physical context reference for subsequent decoupled branch tasks, ensuring the consistency of the underlying physical logic in subsequent estimation tasks.

[0023] S24. The shared latent features are input to three independent fully connected branches of the multi-head output layer for feature decoupling mapping, and the hybrid physical features are output in parallel, including: explicit base linear velocity estimates. Explicit foot contact state estimation value and preliminary implicit terrain representation By introducing shared latent features into independent branches for feature decoupling, the aim is to isolate gradient interference from different physical property features during training, thereby improving the algorithm's network structure. This is due to the explicit base linear velocity estimation... This falls under the category of kinematics, specifically the estimation of explicit foot contact states. This falls under the category of dynamics, and represents a preliminary implicit terrain representation. Belonging to the category of geometric topography, this design effectively avoids negative transfer in multi-task learning through parallel mapping of independent fully connected branches. This design ensures the decoupling and independence of hybrid physical features at the representation level, significantly enhancing the physical interpretability of the estimation results and laying an accurate feature foundation for the perceptual robustness of quadruped robots in unstructured terrain.

[0024] Furthermore, step S3 is as follows:

[0025] S31, From the local terrain elevation point map e t Feature extraction is performed to obtain the target's implicit terrain representation. This step reduces and abstracts the original geometric data of the local terrain through feature encoding, transforming the high-dimensional original topological point cloud into a low-dimensional implicit semantic embedding. The technical principle is to extract key geometric features such as slope and undulation of the terrain through nonlinear mapping. The technical effect is that it effectively filters out random drift noise in terrain observations, providing a compact and physically salient learning objective for the subsequent preliminary multimodal online estimator, thereby significantly improving the network's ability to represent complex and unstructured environments.

[0026] S32, Foot contact force f in the first privileged observation information at the current time t tand foot swing height d t Threshold determination is performed to obtain the target foot contact state. The threshold determination formula is: ,

[0027] In the formula, i represents the leg index of the quadruped robot, i = 1, 2, 3, 4; The target foot contact state of the i-th leg is represented by a value of 1 indicating contact and a value of 0 indicating non-contact; I( ) is an indicator function, which is applied only if " The value is 1 if the condition is true, and 0 otherwise. This represents the contact force at the foot of the i-th leg. f represents the swing height of the i-th leg. th d is the preset force threshold. th The threshold value is set as a preset height threshold. The threshold determination mechanism employs dual verification logic based on both mechanical and geometric constraints, aiming to solve the misjudgment problem caused by ground impact noise or sensor temperature drift in traditional solutions using a single pressure sensor. Its working principle is that only when the force applied to the foot reaches the preset force threshold f... th And the foot swing height d t At the preset height threshold d th When the target is within the specified range, it is considered a valid ground contact. This design, by introducing logical connections between multi-dimensional physical quantities, provides a supervision label with strong physical consistency for model training, effectively avoiding interference from obstacle collisions during the swing phase on ground contact determination.

[0028] S33. Obtain the base linear velocity from the first privileged observation information at the current time t. , base linear velocity Implicit terrain representation of targets Contact state with the target's foot As a target physical characteristic, explicit observations with clear physical meaning (base linear velocity) will be used. Target foot contact state ) and highly abstract implicit features (target implicit terrain representation) The design concept of this approach is to construct a supervised truth system for multi-task parallel optimization. This step utilizes privileged perception capabilities in the simulation environment to establish a joint mapping benchmark of kinematics, dynamics, and environmental geometry. Through complementary constraints of multi-dimensional features, the convergence stability of the initial multimodal online estimator during the training phase is significantly enhanced, and a reliable benchmark is provided for robust feature inference of the quadruped robot control system during the actual deployment phase.

[0029] Furthermore, step S4 is as follows:

[0030] S41. Concatenate the hybrid physical features with the ontology perception information at the current time t, input the concatenation into the preliminary policy network for feature mapping, and output the joint position deviation a. t The design concept of this step is to construct an asymmetric input architecture, so that the initial policy network relies solely on the mixed physical features obtained from the initial multimodal online estimator and the ontology perception information at the current time t during the decision-making stage. This step utilizes a deep neural network to fit a mapping function from the "estimated feature space" to the "control action space," ensuring the hardware feasibility of the motion control method in a real-world deployment environment. This allows the robot, possessing only ontology perception, to achieve adaptive decision-making on complex terrains based on the physical priors provided by the initial multimodal online estimator, effectively improving the control robustness in the "blind motion" mode.

[0031] S42. Concatenate the first privileged observation information at the current time t with the comprehensive ontology perception information, input the concatenation into the value network for feature mapping, and output the scalar state value V. t The working principle of this design is to use the first privileged observation information (i.e., the absolute truth value representing the physical properties of the environment) provided by the simulation environment to provide a global state reference for the value network. Through feature extraction of high-dimensional privileged information by the value network, accurate evaluation of the current preliminary policy network exploration quality can be achieved. Its technical effects are: significantly improving the stability of the preliminary policy network update by utilizing the first privileged observation information; clarifying the optimization direction of the preliminary policy network in complex environments; and solving the problem of slow convergence and oscillation of the preliminary policy network in complex terrain control tasks due to the partial observability of the environment, thereby greatly improving the convergence efficiency and evaluation accuracy of the model during the training phase.

[0032] Both the initial strategy network and the value network are multilayer perceptron networks.

[0033] Furthermore, step S5 is as follows:

[0034] S51, Adjust joint position deviation a t The target positions q of the 12 joints are obtained by superimposing them onto the initial positions of the 12 joints respectively. targ It calculates the joint torque τ, drives the robot to generate physical displacement, and obtains the environmental state S at time t+1. t+1 This step achieves scale mapping from the initial policy network output space to the physical actuator space. Its working principle is based on position servo control logic, transforming the dimensionless deviation values ​​output by the algorithm into physically meaningful joint expectation angles, and further mapping them into joint torques τ. Through interaction with the physical environment, iterative updates of the environment state are completed. The technical effect is to ensure high-frequency response performance of control commands at the execution level, providing a physical execution foundation for closed-loop feedback in robot kinematics.

[0035] S52, From the environmental state S at time t+1 t+1 In the process, the body perception information of the quadruped robot at time t+1 is obtained. t+1 The first and second privileged observation information at time t+1; this step synchronously collects multimodal state observation information after the robot interacts with the environment, providing real-time state feedback for the next decision cycle, and contributing to the single-step composite reward function of the current cycle. The calculation provides the necessary truth reference and observation basis, thereby establishing a complete perception-decision-feedback closed-loop architecture.

[0036] S53. Calculate the single-step composite reward function : ,

[0037] Among them, task tracking rewards The calculation formula is as follows:

[0038] ,

[0039] Geometric Constraint Rewards The calculation formula is as follows:

[0040] ,

[0041] Regulate penalty items The calculation formula is as follows:

[0042] ,

[0043] In the formula, and These represent the ontology perception information extracted from time t+1, respectively. t+1 The horizontal linear velocity command and yaw rate command are included in the user speed commands; v t+1 ω represents the base linear velocity extracted from the first privileged observation information at time t+1. t+1 This represents the ontology perception information extracted from time t+1. t+1 The base angular velocity; and τ represents the horizontal position and horizontal velocity of the j-th foot relative to the base, respectively, extracted from the second privileged observation information at time t+1, where j represents the foot index of the quadruped robot, j = 1, 2, 3, 4; k This represents the k-th component of the joint torque τ, where k = 1, 2, … , 12 are the joint indices; This represents the ontology perception information extracted from time t+1. t+1 The velocity of the k-th joint; A represents the foot swing height at the j-th foot end extracted from the first privileged observation information at time t+1; t-1 and A t These represent the ontological perception information o from time t. t and the ontological perception information o at time t+1 t+1 The motion vectors of the 12 joints obtained from the previous moment; and Let exp( be the preset target width constant and target height constant for the j-th foot end); ) to represent " "An exponential function with the natural constant e as its base; || ||2 indicates that " The 2-norm operation; | | indicates " "Absolute value operation;" Indicates to " The summation operation of λ; n is the weight index, and n = 1, 2, … , 7; n σ is the nth weighting coefficient, used to adjust the contribution ratio of each weighted sub-item in the single-step composite reward function; m is the sensitivity index, and m = 1, 2, 3, 4; m is the m-th sensitivity coefficient, used to adjust the smoothness of the corresponding exponential function term as the physical quantity deviates.

[0044] For single-step compound reward function The design specifications are as follows:

[0045] 1. Task tracking rewards It utilizes an exponential kernel function to smooth and constrain velocity tracking errors. Its working principle is to guide the initial strategy network to adjust the output joint position deviation 'a' by penalizing the deviations in the actual base linear velocity and base angular velocity relative to the user's velocity command. t Iterative optimization is performed. The technical effect is that it ensures the robot has a high degree of confidence in executing commands, significantly improves the robot's tracking accuracy of user speed commands, and enhances the consistency between the actual motion path and the expected commands.

[0046] 2. Geometric Constraint Reward The design aims to maintain a robot's steady-state gait through spatial topological constraints. The first constraint controls the horizontal width of the feet to prevent interference between the four legs or excessively narrow strides; the second constraint maintains the dynamic balance of the robot's center of mass through symmetry constraints. This design addresses the abnormal posture problem that easily arises under unconstrained conditions in reinforcement learning strategies, effectively improving the robot's static and dynamic stability during movement.

[0047] 3. Regulate penalty items A multi-physical quantity joint constraint mechanism is adopted. The energy term (the absolute value of the product of joint torque and joint velocity) is used to quantify instantaneous power loss, aiming to improve the system's endurance efficiency; the motion smoothing term (the L2 norm of the difference between joint motion vectors at previous and subsequent moments) is used to suppress high-frequency oscillations in the control signal, reducing physical wear on the mechanical actuators while improving the smoothness of the control sequence; and the foot height penalty term (a weighted average of foot height deviation and horizontal velocity) is used to optimize the leg-lifting trajectory during the swing phase. These single-step composite reward functions significantly reduce the mechanical load and energy consumption of the quadruped robot's motion control while ensuring task performance, and avoid the risk of ground collisions caused by insufficient foot height.

[0048] Furthermore, step S6 is as follows:

[0049] S61. Based on the single-step composite reward function State value V t and state value V t+1 Through formula = r t +γV t+1 -V t Calculate action advantage value In the formula, γ is a preset discount factor; this step is used to quantify the gain of the joint position deviation of the initial strategy network output relative to the average expected level, by introducing a single-step composite reward function. A dominance evaluation index is constructed using the temporal difference between state values ​​and their values. This step utilizes the difference between state values ​​at adjacent time points to offset the impact of random environmental noise on action evaluation results. This effectively suppresses fluctuations in the initial policy network update and provides a high-confidence guiding direction for the subsequent accurate update of the initial policy network.

[0050] S62, Based on the aforementioned action advantage value Environmental State S t Joint position deviation a t Single-step composite reward function State value V t and state value V t+1 Obtain the reinforcement learning loss function L RL :

[0051] ,

[0052] In the formula, π θ (a t |S t ) and π θold (a t |S tThe numbers θ and θ represent the initial policy network under the current parameters to be updated, and the previous version's update parameters θ before this update, respectively. old Under environmental state S t Lower output joint position deviation a t The probability of; c1 is the preset clipping ratio; c1 is the preset value loss coefficient; min(a, b) represents the operation of selecting the minimum value from "a" and "b"; clip( , 1- , 1+ ) indicates " Perform a clipping operation to restrict its value to the interval [1- , 1+ ]Inside; Indicates to " "The calculation of the expected value under the current batch of samples; this step achieves nonlinear limitation on the update amplitude of the control network by constraining the ratio of the action distribution probability under the new and old parameters, and utilizes the clipping operation (clip)." , 1- , 1+ The update step size of the initial policy network is limited to a preset range [1- , 1+ Within, and by minimizing the state value V t With time-difference objective r t + γV t+1 The second-order norm error between the two is used to simultaneously optimize policy improvement and value assessment. This step effectively suppresses abrupt changes and oscillations in control commands during network updates, ensures the temporal continuity of the initial policy network output distribution, solves the policy collapse problem that easily occurs in deep reinforcement learning under complex dynamic environments, and guarantees the monotonic convergence of the initial policy network during the update process.

[0053] S63. Based on the deviation between the hybrid physical features and the target physical features, construct a multi-task estimation loss function L. est :

[0054] ,

[0055] In the formula, w p For the pre-defined learnable uncertainty parameter p, L p Let p be the loss function, where p = 1, 2, 3;

[0056] The formula for calculating the first loss function L1 is: ,in, It is the linear velocity of the base. This is an estimate of the explicit base linear velocity. ||2 indicates that " The 2-norm operation;

[0057] The formula for calculating the second loss function L2 is: ,in, This represents the target foot contact state of the i-th leg, extracted from the target foot contact state. , This represents the explicit foot contact state estimate of the i-th leg, extracted from the explicit foot contact state estimate. ;

[0058] The formula for calculating the third loss function L3 is: ,in, It is an implicit terrain representation of the target. This is a preliminary implicit terrain representation; the multi-task estimation loss function L est To achieve parallel learning and collaborative alignment of multidimensional physical properties, a learnable uncertainty weight parameter w is introduced. p The contribution ratio of the three heterogeneous tasks—linear velocity estimation, contact state discrimination, and terrain representation extraction—to the total loss is dynamically adjusted. Specifically, the first loss function L1 aims to minimize the bias in base linear velocity estimation and improve the kinematic fidelity of ontology perception; the second loss function L2 optimizes foot contact logic discrimination based on the cross-entropy criterion, enhancing the system's perception accuracy of support phase switching; and the third loss function L3 calculates preliminary implicit terrain representation. Implicit terrain representation of the target The vector space similarity between the parameters ensures that the estimated environmental features have physical saliency. These joint constraints significantly improve the nonlinear decoupling capability and state estimation accuracy of the preliminary multimodal online estimator for complex features in unstructured environments, effectively enhancing the robot's environmental state perception capability in unknown environments.

[0059] S64. Utilizing the reinforcement learning loss function L RL Update the initial policy network and estimate the loss function L using a multi-task approach. est The initial multimodal online estimator is updated, and the above update process is iteratively executed until the network converges, resulting in the target policy network and the target multimodal online estimator. This step constructs a joint optimization mechanism for perception and control, based on the reinforcement learning loss function L. RL With multi-task estimation loss function L est The joint network update realizes the hybrid physical features output by the preliminary multimodal online estimator and the joint position deviation α obtained by the preliminary policy network. tDeep decoupling and semantic alignment in the feature space. This step ensures that the physical priors extracted by the initial multimodal online estimator can be directly mapped to effective control commands, enabling the final target policy network and target multimodal online estimator to have excellent terrain adaptability and dynamic robustness, effectively solving the problem of robot deployment in unstructured environments.

[0060] Secondly, the present invention provides a quadruped robot motion control system based on multimodal online estimation, for executing the aforementioned quadruped robot motion control method based on multimodal online estimation, the control system comprising:

[0061] The status and information acquisition module is used to acquire the environmental state S at the current time t. t The integrated ontological perception information of the quadruped robot, including the environmental state S t This includes the ontology perception information at the current time t, the first privileged observation information, and the second privileged observation information;

[0062] The hybrid physical feature extraction module is used to input the integrated ontology perception information into the preliminary multimodal online estimator for feature encoding and decoupling to obtain hybrid physical features;

[0063] The target physical feature extraction module is used to extract features and determine thresholds from the first privileged observation information at the current time t to obtain the target physical features.

[0064] The state value extraction module is used to input the hybrid physical features and the ontology perception information at the current time t into the preliminary strategy network, and output the joint position deviation a. t Input the first privileged observation information and the comprehensive ontology perception information at the current time t into the value network, and output the state value V at the current time t. t ;

[0065] The single-step composite reward function calculation module is used by the quadruped robot to calculate the joint position deviation 'a'. t After performing the action, obtain the environment state S at time t+1. t+1 Based on environmental state S t+1 Calculate the single-step compound reward function ;

[0066] The policy network and estimator training module is used to train the policy network based on the environment state S. t+1 Update the integrated ontology perception information, repeatedly call the state and information acquisition module, the hybrid physical feature extraction module, the target physical feature extraction module, and the state value extraction module to obtain the state value V at time t+1. t+1 Based on the single-step composite reward function State value V t and state value Vt+1 Thus, the reinforcement learning loss function is obtained;

[0067] The initial policy network is trained by the reinforcement learning loss function to obtain the trained target policy network. Based on the deviation between the mixed physical features and the target physical features, a multi-task estimation loss function is obtained. The initial multimodal online estimator is trained by the multi-task estimation loss function to obtain the trained target multimodal online estimator.

[0068] The quadruped robot joint control module is used to acquire the comprehensive body perception information of the quadruped robot in real time in real-world scenarios, and input it into the target multimodal online estimator to obtain hybrid physical features. The hybrid physical features and the body perception information at the current time t are then input into the target policy network to obtain joint control commands.

[0069] Thirdly, the present invention provides an electronic device, including a processor and a memory for storing a processor-executable program, wherein when the processor executes the program stored in the memory, it implements the above-mentioned quadruped robot motion control method based on multimodal online estimation.

[0070] Fourthly, the present invention provides a storage medium storing a program, which, when executed by a processor, implements the above-mentioned method for motion control of a quadruped robot based on multimodal online estimation.

[0071] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0072] This invention proposes a motion control method for quadruped robots based on multimodal online estimation. Unlike the traditional two-stage framework in existing technologies, which involves pre-training a privileged teacher policy followed by offline distillation to a student policy, this invention constructs a multimodal online estimator containing a shared hidden layer and a multi-head output layer. It also employs a specific input concatenation mechanism between the policy network and the value network, and introduces a structured multi-task estimation loss function and a composite reward function within the same reinforcement learning framework, achieving single-stage joint training of state estimation and policy optimization. This invention not only significantly reduces the complexity of the training process but also significantly improves the perceptual robustness and motion stability of quadruped robots in visually impaired and unstructured terrain.

[0073] Specifically: (1) Based on the online architecture of asymmetric feature splicing and single-stage joint optimization, the training efficiency and policy robustness are significantly improved: When facing complex terrain, the existing two-stage distillation scheme is prone to serious distribution shift due to the inconsistency of the state distribution of the "teacher-student" policy. In the control step (see step S4), the present invention innovatively adopts an asymmetric feature splicing input mechanism: the mixed physical features and the current moment's ontology perception information are spliced ​​into the initial policy network, and the first privileged observation information and the comprehensive ontology perception information are spliced ​​into the value network. In terms of technical principle, this asymmetric structure ensures that the policy network relies only on the ontology perception information it can obtain at the current moment to make blind gait decisions in the actual deployment stage, and enables the value network to use the first privileged observation information to provide low variance and high accuracy state value evaluation in the training stage, thereby greatly stabilizing the policy gradient update direction of the policy network. Furthermore, in the network update step (see step S6), the present invention uses the reinforcement learning loss function based on action advantage value and the multi-task estimation loss function for synchronous iteration. In terms of technical principle, this method completely abandons the traditional offline behavior cloning paradigm. Through single-stage joint training, the joint position deviations output by the policy network and the resulting real-time environmental state sequence directly serve as the data-driven basis for updating the multimodal online estimator network. Conversely, the updated multimodal online estimator provides the policy network with physical priors (i.e., hybrid physical features) that are highly aligned with the current environmental state and joint position deviations. This online co-adaptive closed-loop mechanism of "state representation-action exploration" avoids the covariate shift phenomenon caused by the solidification of traditional pre-trained teacher strategies from the perspective of algorithm optimization paradigm. In terms of final technical effect, the above mechanism cuts off the chain of inference error accumulation caused by offline distillation and breaks the strong dependence of the control framework on external perfect teacher demonstration data. This invention not only significantly eliminates cumbersome pre-training steps and shortens the algorithm development cycle, but also fundamentally improves the dynamic fault tolerance and task success rate of quadruped robots after migration and deployment from simulation to reality, in the absence of extreme unstructured terrain.

[0074] (2) Based on the structural decoupling and learnable loss metric of the shared hidden layer combined with multi-head parallel branches, the physical interpretability and environmental adaptability of the control scheme are greatly enhanced: In view of the technical pain point of the mixed entanglement of robot kinematics, dynamic state and external environment features in the existing blind motion control, which makes it difficult to trace the source of the fault, the present invention configures a shared hidden layer in the multimodal online estimator (see step S2) to perform high-dimensional spatiotemporal fusion of multi-dimensional temporal ontology perception feature vectors, and then connects an independent feature mapping branch. In terms of technical principle, the architecture not only uses the shared hidden layer as an information bottleneck to extract low-dimensional shared latent features containing global dynamic context, realizing information complementarity between different perception modalities; but also effectively isolates the gradient conflict and negative migration interference of feature components with different physical properties (i.e., the base linear velocity and foot contact state reflecting the ontology kinematics and dynamic state, and the implicit terrain representation reflecting the geometric structure of the external environment) in the network update process through parallel independent output branches. Forced decoupling implemented at the network's underlying topology not only avoids state estimation distortion caused by feature entanglement but also provides high-confidence and mutually independent physical priors for subsequent policy networks. Furthermore, in constructing the multi-task estimation loss function (see step S63), this invention introduces learnable uncertainty parameters for dynamic weight adjustment of different physical quantities. Technically, this parameter mechanism adaptively optimizes the gradient weights of corresponding loss terms during training based on the inherent noise distribution and numerical scale of each physical estimation task, thereby completely avoiding the network optimization imbalance problem caused by different physical quantities having different dimensions and convergence speeds. Simultaneously, this invention employs a differentiated measurement mechanism adapted to each physical feature: the first loss function L1 uses a L2 norm constraint on the base linear velocity, and the second loss function L2 uses cross-entropy constraints on the foot contact state, ensuring the absolute physical accuracy of the dynamic representation; while innovatively, cosine similarity measurement is used for implicit terrain representation. This cosine metric mechanism enables the network to focus on extracting the evolution trend of the terrain geometry and topology around the quadruped robot rather than absolute height values. This allows the robot to still be keenly aware of terrain slope and unstructured undulations even when vision is impaired or sensor observation drift occurs, greatly enhancing its adaptive traversal capability in blind gait.

[0075] (3) Design of a composite reward function based on multidimensional physical feature association constraints to ensure dynamic balance and operational safety in complex terrain: This invention breaks through the limitations of traditional single reward functions that cannot take into account both flexibility and safety. It constructs a single-step composite reward function (see step S53) that includes task tracking reward, geometric constraint reward and regularization penalty term, and implements multidimensional physical prior constraints on the joint position deviation output by the policy network. In terms of technical principle: the exponential task tracking term provides high-sensitivity feedback on speed command deviation, effectively suppressing the speed overshoot phenomenon of the robot in the dynamic tracking process; the geometric constraint term explicitly introduces the absolute value deviation constraint between the foot horizontal position and the target width, forcing the robot to maintain the dynamic symmetry and smooth transition of the center of gravity trajectory in complex terrain, solving the problem of easy instability and rollover in blind movement; at the same time, the regularization penalty term is combined to jointly penalize the overall power loss of the system (product of joint torque and joint velocity), the amplitude of sudden movement, and the coupling relationship between foot height and horizontal velocity. The synergistic effect of the aforementioned rewards enabled the policy network to adaptively learn a high-swing obstacle-crossing gait and a low-impact ground-contact strategy based on terrain conditions. This significantly reduced the high-frequency oscillations of the drive joints and improved the mechanical lifespan and operational safety of the quadruped robot under heavy loads and extreme environments. Attached Figure Description

[0076] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0077] Figure 1 This is a flowchart of the quadruped robot motion control method based on multimodal online estimation proposed in this invention;

[0078] Figure 2 This is a schematic diagram of the joint degrees of freedom configuration of a single leg (taking the left hind leg as an example) of the quadruped robot in an embodiment of the present invention;

[0079] Figure 3 This is a schematic diagram of the network architecture of the quadruped robot motion control method based on multimodal online estimation proposed in this invention;

[0080] Figure 4 This is a schematic diagram of the movement posture of the quadruped robot in a simulation environment as it traverses a 70° ramp, according to an embodiment of the present invention.

[0081] Figure 5 This is a schematic diagram of the movement posture of the quadruped robot in a simulation environment as it moves up a continuous staircase with a step height of 30 cm in an embodiment of the present invention.

[0082] Figure 6 This is a real-world operational diagram of the quadruped robot successfully traversing a continuous staircase with a height of 16 cm in a real-world scenario, as shown in this embodiment of the invention.

[0083] Figure 7 This is a real-life operational diagram of the quadruped robot successfully climbing a 32° irregular steep grassy slope in a real-world scenario, as shown in this embodiment of the invention.

[0084] Figure 8 This is a schematic diagram of the motion posture of the quadruped robot successfully climbing a 40 cm single-step staircase in a simulated environment in an embodiment of the present invention;

[0085] Figure 9 This is a schematic diagram of the movement posture of the quadruped robot on a 3-meter-high platform in a simulated environment, as shown in this embodiment of the invention.

[0086] Figure 10 This is a real-world operational schematic diagram of the quadruped robot successfully crossing discrete platforms in a real-world scenario, as shown in this embodiment of the invention.

[0087] Figure 11 This is a real-world operational schematic diagram of the quadruped robot successfully traversing dense rigid vegetation in a real-world scenario, as shown in this embodiment of the invention.

[0088] Figure 12 This is a real-world operational diagram of the quadruped robot performing extreme heavy loads in a real-world scenario, as described in this embodiment of the invention.

[0089] Figure 13 This is a structural block diagram of the quadruped robot motion control system based on multimodal online estimation in Embodiment 3 of the present invention;

[0090] Figure 14 This is a structural block diagram of the electronic device in Embodiment 4 of the present invention. Detailed Implementation

[0091] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative effort are within the scope of protection of the present application.

[0092] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application can be combined with other embodiments.

[0093] In all embodiments of this invention, the Unitree Go1 quadruped robot is used as the physical carrier, and a large-scale parallel reinforcement learning training environment is constructed in the Isaac Gym simulation physics engine. During training, to improve the model's generalization ability and the success rate of transferring from simulation to reality, this embodiment preferably employs a domain randomization mechanism. Specifically, at the beginning of each training round, a random function is used to perturb and sample the simulation physical parameters. The sampling range includes: ground friction coefficient (0.2-1.2), load mass (0 kg-10 kg), center of mass offset (±5 cm), and random thrust applied to the base (±10 N).

[0094] Example 1

[0095] In this embodiment, the robustness of the proposed method under complex terrain is verified using a simulation physics engine. The simulated terrain specifically includes slopes ranging from 10° to 70° and continuous stairs ranging from 5 cm to 30 cm in height. Real-world experiments are conducted to verify the success rate of the proposed method on terrain involving 16 cm continuous stairs and a 32° sloping grassy slope.

[0096] Step S1: Obtain the environmental state S at the current time t. t Based on the integrated ontological perception information, the environmental state S at the current time t t This includes the ontology perception information at the current time t, the first privileged observation information at the current time t, and the second privileged observation information at the current time t. The process is as follows:

[0097] Step S11: Obtain the quadruped robot's body perception information at the current time t. t In this embodiment, the mechanical structure of the quadruped robot is as shown in the attached figure. Figure 2 As shown, it includes a base 5 and four legs mounted on the base: a left foreleg, a right foreleg, a left hind leg, and a right hind leg. Taking the left hind leg as an example, it includes three joints with independent degrees of freedom and a foot 4 at the end; the three joints, from the base to the foot, are defined as the hip joint 1, the thigh joint 2, and the lower leg joint 3, respectively; the joints and foot configurations of the other three legs are geometrically symmetrical with those of the left hind leg. The proprioceptive information o at the current time t. tThis includes the user's velocity command for the quadruped robot at the current moment t, the base angular velocity, the quadruped robot's gravity vector, the position error of the 12 joints relative to their initial positions, the velocities of the 12 joints, and the motion vectors of the 12 joints from the previous moment. The initial positions of the 12 joints refer to the joint angles of the robot in a preset reference posture. In this embodiment, the specific configuration is as follows: the hip joints of the left front leg and left hind leg are both 0.1 rad, and the hip joints of the right front leg and right hind leg are both -0.1 rad; the thigh joints of the left front leg and right front leg are both 0.8 rad, and the thigh joints of the left hind leg and right hind leg are both 1.0 rad; and the lower leg joints of all four legs are uniformly configured as -1.5 rad.

[0098] Step S12: Obtain the ontology perception information o at the current time t. t By concatenating the ontology perception information with the past 5 historical moments, we obtain the comprehensive ontology perception information {o t - 5 , … , o t};

[0099] Step S13: Obtain the first privileged observation information at the current time t, including the base linear velocity at the current time t. Foot contact force f t Foot swing height d t and local terrain elevation point map e t Obtain the second privileged observation information at the current time t, including the base height at the current time t, the horizontal position of the four feet relative to the base, and the horizontal velocity of the four feet relative to the base.

[0100] Step S2: Input the comprehensive ontology perception information into the preliminary multimodal online estimator for feature encoding and decoupling to obtain hybrid physical features. The process is as follows:

[0101] Step S21: Integrate the ontology perception information {o t - 5 , … , o t The flattened form is a temporal ontology-aware feature vector. In this embodiment, due to the ontology-aware information o at a single current time t... t The total dimension is 45. After flattening the integrated ontology perception information, a temporal ontology perception feature vector with a dimension of 270 is obtained.

[0102] Step S22: Input the temporal ontology-aware feature vector with dimension 270 into the preliminary multimodal online estimator. The preliminary multimodal online estimator adopts a multilayer perceptron (MLP) network, whose network structure includes, in sequence: an input layer with dimension 270, a shared hidden layer consisting of two layers with the number of nodes configured as 128 and 48 respectively, each layer being followed by an ELU activation function, and a multi-head output layer connected to the output of the shared hidden layer. The multi-head output layer consists of three parallel independent feature mapping branches, specifically: a velocity estimation branch consisting of an independent fully connected layer with an input dimension of 48 and an output dimension of 3, serving as the explicit base linear velocity; a contact prediction branch consisting of an independent fully connected layer with an input dimension of 48 and an output dimension of 4, serving as the explicit foot contact state; and a terrain representation branch consisting of an independent fully connected layer with an input dimension of 48 and an output dimension of 16, serving as the implicit terrain representation.

[0103] Step S23: The 270-dimensional temporal ontology-aware feature vector is input into the first shared fully connected layer for linear mapping and bias term superposition, resulting in a 128-dimensional first-level feature vector. The smooth nonlinear characteristics of the corresponding ELU activation function are used to process the first-level feature vector, effectively transmitting and smoothing the perception features containing negative physical states while preserving positive dynamic activation signals and suppressing redundant noise. Subsequently, the first-level feature vector undergoes linear and nonlinear mapping operations from 128 to 48 dimensions through the subsequent second shared fully connected layer. During this process, the shared hidden layer achieves deep fusion of the physical components in the temporal ontology-aware feature vector in the temporal dimension through the feature extraction of weights, ultimately compressing the original high-dimensional sensor observation data into 48-dimensional shared latent features containing dynamic context information.

[0104] Step S24: Input the 48-dimensional shared latent features into the three independent fully connected branches of the multi-head output layer for feature decoupling mapping, and output the hybrid physical features in parallel. (See attached image) Figure 3 As shown in the upper left portion, the multi-head output layer receives the 48-dimensional shared latent features from the shared hidden layer and inputs them in parallel to three independent feature mapping branches to perform decoupling operations: The velocity estimation branch utilizes its independent fully connected layer to perform linear mapping, mapping the 48-dimensional shared latent features into a 3-dimensional vector, obtaining explicit base linear velocity estimates in the three-axis directions. = [1.86, 0.13, -0.30] T The contact prediction branch performs a linear mapping using its independent fully connected layers, transforming the 48-dimensional shared latent features into a 4-dimensional vector to obtain an explicit contact state estimate representing the foot-ground contact state of the quadruped robot. = [0, 0, 0.05, 0.43]T The terrain representation branch uses its independent fully connected layers to perform linear mapping, transforming the 48-dimensional shared latent features into a 16-dimensional vector, thus obtaining a preliminary implicit terrain representation reflecting the terrain's geometric features.

[0105] = [-0.38, -0.44, 0.57, -0.02, -0.29, 0.37, 0.18, -0.59, 0.35, 0.04, 1.22, -1.06, 0.08, 0.23, 0.14, 0.95] T .

[0106] The physical estimation features generated in parallel by the three branches are collectively referred to as hybrid physical features.

[0107] Step S3: Perform feature extraction and threshold determination on the first privileged observation information at the current time t to obtain the target physical features. The process is as follows:

[0108] Step S31: Obtain local terrain elevation point map e t In this embodiment, the relative height information of 17×11 grid points (a total of 187 height sampling points) around the quadruped robot base is collected as a local terrain height point map e. t The input is a pre-trained static feature extractor for feature extraction. In this embodiment, the static feature extractor adopts a convolutional neural network architecture, including: a first convolutional block (using a 3 × 3 convolutional kernel, stride of 2, padding of 1, and 32 output channels, and processed by batch normalization and ReLU activation); a second convolutional block (using a 3 × 3 convolutional kernel, stride of 2, padding of 1, and 64 output channels, and processed by batch normalization and ReLU activation); a global average pooling layer, a flattening layer, and a fully connected layer. The fully connected layer outputs a 16-dimensional implicit terrain representation of the target:

[0109] = [-0.15, -0.21, 0.22, 0.01, -0.16, 0.18, 0.09, -0.17, 0.2, 0.08, 0.43, -0.3, 0, 0.07, 0.04, 0.37] T .

[0110] Step S32: Analyze the foot contact force f from the first privileged observation information at the current time t. t and foot swing height d t Threshold determination is performed to obtain the target foot contact state. Specifically, as shown in the attached document. Figure 3 As shown in the lower left part, the contact force f at the foot end t =[ffl , f fr ,f rl , f rr ] T and foot swing height d t = [d fl , d fr ,d rl , d rr ] T This includes the physical quantities corresponding to the left front leg, right front leg, left hind leg, and right hind leg, respectively. In this embodiment, the force threshold f is... th Configured to 0.6 N, threshold d th With a configuration of 0.03 m, the foot contact force f was obtained from the simulation environment. t = [0, 0, 45.28, 145.45] T (Unit: N), foot swing height d t =[0.05, 0.12, 0.02, 0.01] T (Unit: m). Based on the threshold determination formula... Obtain the target foot contact state of the i-th leg. The leg is considered to be in contact only if the normal contact force of the i-th leg is greater than 0.6 N and the foot's height above the ground is less than 0.03 m. = 1; otherwise, it is determined to be a non-contact state, i.e. = 0. Based on the above determination, the 4-dimensional target foot contact state is obtained. = [0, 0, 1, 1] T .

[0111] Step S33: Directly extract the base linear velocity from the first privileged observation information at the current time t. =[1.126, 0.085, -0.023] T Base linear velocity Implicit terrain representation of targets Contact state with the target's foot These are collectively referred to as target physical characteristics.

[0112] Step S4: Input the hybrid physical features and the ontology perception information at the current time t into the preliminary strategy network, and output the joint position deviation a. t The first privileged observation information and the comprehensive ontology perception information at the current time t are input into the value network, and the state value V is output. t In this embodiment, both the initial policy network and the value network are configured with the same multilayer perceptron network, and the process is as follows:

[0113] Step S41: Concatenate the hybrid physical features with the ontology perception information at the current time t, perform feature mapping through the preliminary policy network, and output the joint position deviation a. t In this embodiment, the initial policy network uses a multilayer perceptron network with [512, 256, 128] hidden layer nodes, and each hidden layer uses the ELU activation function. (See attached diagram.) Figure 3 As shown, its input layer receives ontology-aware information consisting of 23-dimensional hybrid physical features and 45-dimensional current time t. t The concatenated 68-dimensional feature vector, with an output layer dimension of 12, is processed by an action scaling factor of 0.25 to obtain the joint position deviation 'a' representing the target position of the 12 joints relative to their initial positions. t = [-0.6, -0.24, -0.41, 0.43, -0.33, -0.23, -0.55, 0.17, -0.25, 0.54, -0.12, 0.13] T (Unit: rad) In a specific embodiment of the present invention, the elements of all multidimensional feature vectors or matrices involving 12 joints are arranged in a fixed order as follows: corresponding to the hip joint, thigh joint and calf joint of the robot's left front leg, right front leg, left hind leg and right hind leg in sequence.

[0114] Step S42: Concatenate the first privileged observation information at the current time t with the integrated ontology perception information, perform feature mapping through the value network, and output the scalar form state value V. t In this embodiment, the hidden layer configuration of the value network is consistent with that of the initial policy network (i.e., the number of hidden layer nodes is also configured as [512, 256, 128] and the ELU activation function is used). The input layer receives a feature vector composed of 198-dimensional first privileged observation information and 270-dimensional comprehensive ontology perception information at the current time t (i.e., the input layer dimension is configured as 468). The output layer dimension is configured as 1, which is used to output the scalar state value V representing the quality of the current environmental state. t = 5.19.

[0115] Step S5: The robot adjusts the joint position deviation a. t After performing the action, obtain the environment state S at time t+1. t+1 Based on the environmental state S at time t+1 t+1 Calculate the single-step compound reward function The process is as follows:

[0116] Step S51: Adjust the joint position deviation a t Add the initial positions of the 12 joints element-wise to obtain the target positions of the 12 joints:

[0117] q targ = [-0.5, 0.56, -1.09, 0.33, 0.47, -1.73, -0.45, 1.17, -1.75, 0.44, 0.88, -1.37] T (Unit: rad). Extract the ontology perception information o at the current time t. t The positional error of the 12 joints at their current positions relative to their initial positions is: q erro = [-0.25, -0.54, -0.12, 0.26, -0.17, -0.47, -0.31, -0.14, -0.34, 0.3, -0.3, -0.16] T (Unit: rad), add element-wise the initial positions of the 12 joints to obtain the current positions q of the 12 joints. t = [-0.15, 0.26, -1.62, 0.16, 0.63, -1.97, -0.21, 0.86, -1.84, 0.2, 0.7, -1.66] T (Unit: rad), and simultaneously extract the ontology perception information o at the current time t. t The 12 joint velocities = [-0.82, -2.56, 15.53, -0.44, -9.7, 0.49, -0.47, 7.61, -3.5, 0.68, 7.53, -5] T (Unit: rad / s). In this embodiment, the proportional gain coefficient K is set. p = 20 Nm / rad, differential gain coefficient K d = 0.5 Nms / rad, calculate the joint torque τ using the PD controller. t The calculation formula is: τ t = K p ×(q targ - q t ) - K d × The calculated joint torque τ t = [-6.59, 7.28, 2.84, 3.62, 1.65, 4.56, -4.57, 2.4, 3.55, 4.46, -0.17, 8.3] T (Unit: N·m) The data is sent to the motor controllers of the robot's 12 joints to drive the robot to generate physical displacement and obtain the environmental state S at time t+1. t+1 .

[0118] Step S52, from the environmental state S at time t+1 t+1 In the process, obtain the ontology perception information at time t+1. t+1 The first privileged observation information at time t+1 and the second privileged observation information at time t+1; among which, the proprioceptive information at time t+1. t+1 The first privileged observation information and the second privileged observation information at time t+1, together with the ontological perception information at the current time t. t The components of the first privileged observation information at current time t and the second privileged observation information at current time t are the same.

[0119] Step S53: Calculate the single-step composite reward function In this embodiment, the weighting coefficients are set as follows: λ1 = 1.0, λ2 = 0.5, λ3 = 0.1, λ4 = 0.1, λ5 = 0.00002, λ6 = 0.02, λ7 = 0.02; the sensitivity coefficients are set as follows: σ1 = σ2 = 0.25, σ3 = 0.5, σ4 = 0.8; and the target width is preset and set. = 0.145 m, target altitude constant = 0.09 m, j ∈ {1,2, 3, 4}. In the current embodiment, = [1.0, 0] T (Unit: m / s), v t+1 =[1.26, 0.04] T (Unit: m / s) = 0 rad / s, ω t+1 = -0.14 rad / s, according to the formula Calculate the task tracking reward = 1.22; = 0.08 m, = -0.09 m, = 0.07 m, = -0.07 m, according to the formula Calculate the geometric constraint reward = 0.16; A t = [-0.6, -0.24, -0.41, 0.43, -0.33, -0.23, -0.55, 0.17, -0.25, 0.54, -0.12, 0.13] T (Unit: rad), A t-1= [-0.53, -0.21, 0.17, 0.45, -0.31, -0.47, -0.55, 0.07, -0.06, 0.59, -0.06, 0.15] T (Unit: rad), d t+1 = [0.03, 0.1, 0.02, 0.01] T (Unit: m) = [-1.22, -0.04] T m / s = [2.12, -0.07] T m / s = [-1.18, -0.02] T m / s = [-1.21, -0.03] T m / s, = [0.10, 6.92, -3.67, -0.54, -8.93, 8.10, -0.38, 7.73, -2.55, 0.43, 7.63, -3.41] T (Unit: rad / s), based on the joint torque τ t , The regularization penalty term was calculated. = -0.01. According to the formula Calculations yielded = 1.37.

[0120] Step S6: Based on the environmental state S at time t+1 t+1 Update the comprehensive ontology perception information, repeat steps S1 to S4, and obtain the state value V at time t+1. t+1 = 5.09, based on a single-step composite reward function State value V t and the state value V at time t+1 t+1 The reinforcement learning loss function is obtained to train the initial policy network and value network, resulting in the trained target policy network. Based on the deviation between the mixed physical features and the target physical features, a multi-task estimation loss function is obtained to train the initial multimodal online estimator, resulting in the trained target multimodal online estimator. The process is as follows:

[0121] Step S61: Based on the single-step composite reward function State value V t and the state value V at time t+1 t+1 Set the discount factor γ = 0.998, and use the formula... = r t+ γV t+1 - V t Calculate the action advantage value = 1.26;

[0122] Step S62: In this embodiment, the cutting ratio is set to... = 0.2, value loss coefficient c1 = 0.5. Based on the environmental state S at the current time t. t Joint position deviation a t get = 1.25, further, based on action advantage value Single-step composite reward function State value V t and the state value V at time t+1 t+1 According to the formula:

[0123] The reinforcement learning loss function L is calculated. RL = 0.72.

[0124] Step S63: In this embodiment, the learnable uncertainty parameters w1 = 0.85, w2 = 1.05, and w3 = 1.1. Based on the hybrid physical characteristics and the target physical characteristics, according to the formula... The first loss function L1 is calculated to be 0.62; according to the formula... The calculated second loss function L2 = 3.84; root formula The third loss function L3 is calculated to be 0.02; finally, according to the formula: Calculate the multi-task estimation loss function L est = 4.58.

[0125] Step S64: Utilize the reinforcement learning loss function L RL The initial policy network is updated by estimating the loss function L using a multi-task approach. est Update the initial multimodal online estimator and perform 3000 iterations or until the network loss function converges to obtain the trained target policy network and the target multimodal online estimator.

[0126] Step S7: In the deployment and control phase of the quadruped robot in the real-world scenario, the comprehensive ontological perception information of the quadruped robot is acquired and input into the trained target multimodal online estimator to obtain hybrid physical features. Subsequently, the hybrid physical features and the ontological perception information at the current time t are input into the trained target policy network to output the joint position deviation at the current time. After the joint position deviation is superimposed on the initial joint position to obtain the target joint position, it is converted into 12 joint torque commands by the PD controller and finally sent to the underlying motor of the quadruped robot to drive the robot to perform the corresponding movements.

[0127] To further verify the effectiveness and advancement of the quadruped robot motion control method based on multimodal online estimation proposed in this invention, comparative experiments and performance evaluations were conducted in both simulation environments and real-world physical scenarios.

[0128] 1. Environmental Comparison Experiment and Analysis: In this embodiment, the robustness of the proposed method under complex terrain is verified using the simulation physics engine Isaac Gym. To ensure the objectivity of the evaluation, 100 independent tests were conducted for each terrain configuration, and the proposed method was compared with the benchmark algorithms LQL and PA-LOCO, which use a two-stage distillation strategy framework based on the "teacher-student" model. The simulated terrain specifically included: slopes with a height of 1 m and a gradient ranging from 10° to 70°, and 10 consecutive steps with a single step height ranging from 5 cm to 30 cm. The statistical data on the success rate of each algorithm under different terrain difficulties are shown in Table 1.

[0129] Table 1. Comparison of Success Rates of Various Algorithms in Complex Terrain under Real-World Conditions

[0130]

[0131] As shown in Table 1, the present invention exhibits significantly better motion robustness and environmental adaptability under extreme conditions than the traditional benchmark algorithms LQL and PA-LOCO, which are based on a two-stage "teacher-student" distillation process. In sloping terrain, the traditional algorithms suffer from significant performance degradation at inclination angles exceeding 30° due to the inherent accumulation of inference errors and distribution shifts in offline distillation, and completely fail at a 70° slope (0% success rate). In contrast, the present invention achieves real-time alignment of the estimated and control distributions through single-stage synchronous optimization of a multimodal online estimator and a policy network, maintaining a 68% success rate even at a 70° extreme inclination angle. Meanwhile, in continuous staircase terrain, traditional algorithms suffer from poor perception robustness and are prone to blind collisions due to the entanglement of intrinsic dynamic states and external environmental features (e.g., PA-LOCO has a success rate of only 2% at a step height of 25 cm). In contrast, this invention, through structural decoupling at the representation level, accurately and independently estimates implicit terrain representations, foot contact states, and base linear velocities, effectively supporting the robot's adaptive leg-lifting obstacle avoidance and predictive landing point adjustment, enabling it to maintain an extremely high pass rate of 92% even at a step height of 25 cm. (See attached diagram) Figure 4 and attached Figure 5 The diagram shows the motion posture of a quadruped robot driven by the control strategy of this invention to steadily traverse a 70° ramp and a continuous staircase with a step height of 30 cm in a simulation environment.

[0132] 2. Real-world Deployment Comparison Experiment: To verify the "zero-shot" transfer capability of this invention, this embodiment directly deploys the simulation-trained converged model onto a real quadruped robot. The robot does not rely on external vision sensors at all, but walks blindly solely using its proprioceptive sensors. Table 2 shows the success rate comparison between the method of this invention and the aforementioned two-stage distillation benchmark algorithm in real-world challenge scenarios (20 independent tests each).

[0133] Table 2. Comparison of Success Rates of Various Algorithms in Real-World Physical Scenarios

[0134]

[0135] As shown in Table 2, when faced with a 16 cm real-world staircase with geometric overhangs, the comparative algorithm frequently tripped due to its inability to adapt to physical collisions (e.g., a 0% LQL success rate). In contrast, the present invention can adaptively adjust the swing leg trajectory through multimodal physical feature feedback, achieving 100% error-free passage. In a 32° irregular low-friction outdoor grassy slope environment, the comparative algorithm experienced irreversible and severe slippage, while the present invention, through real-time fine-tuning of the base posture and gait cycle, ensured foot adhesion, also achieving a 100% success rate. This experiment demonstrates that the present invention exhibits strong disturbance resistance and minimal simulation-to-real-world migration differences during hardware deployment. (See attached table.) Figure 6 and attached Figure 7The image shown is a real-world demonstration of the invention successfully traversing stairs and climbing steep grassy slopes in a real-world setting.

[0136] Example 2

[0137] In this embodiment, the same experimental steps as in Embodiment 1 are used, but the simulated and real-world experimental scenarios are changed. The impact robustness of this method under discrete elevation differences is verified through a simulation physics engine. The simulated terrain specifically includes: a single-level ascending step with a height range of 20 cm to 40 cm, and a single-level descending step with a drop range of 1 m to 3 m. In real-world experiments, the success rate of this method in traversing terrain with a 25 cm ascending step, a 50 cm descending step, and dense rigid vegetation is verified, as well as its ultimate load capacity under a static load of 9.5 kg and a dynamic drag of 10 kg.

[0138] Step S1: Obtain the environmental state S at the current time t. t Based on the integrated ontological perception information, the environmental state S at the current time t t This includes the ontology perception information at the current time t, the first privileged observation information at the current time t, and the second privileged observation information at the current time t. The process is as follows:

[0139] Step S11: Obtain the quadruped robot's body perception information at the current time t. t In this embodiment, the mechanical structure of the quadruped robot is as shown in the attached figure. Figure 2 As shown, it includes a base 5 and four legs mounted on the base, namely the left front leg, right front leg, left hind leg, and right hind leg. Taking the left hind leg as an example, it includes three joints with independent degrees of freedom and a foot 4 at the end; the three joints are defined sequentially from the base to the foot as the hip joint 1, thigh joint 2, and lower leg joint 3; the joints and foot configurations of the other three legs are geometrically symmetrical with those of the left hind leg. The ontology perception information o at the current time t. t This includes the user's velocity command for the quadruped robot at the current moment t, the base angular velocity, the quadruped robot's gravity vector, the position error of the 12 joints relative to their initial positions, the velocities of the 12 joints, and the motion vectors of the 12 joints from the previous moment. The initial positions of the 12 joints refer to the joint angles of the robot in a preset reference posture. In this embodiment, the specific configuration is as follows: the hip joints of the left front leg and left hind leg are both 0.2 rad, and the hip joints of the right front leg and right hind leg are both 0 rad; the thigh joints of the left front leg and right front leg are both 0.9 rad, and the thigh joints of the left hind leg and right hind leg are both 1.1 rad; and the lower leg joints of all four legs are uniformly configured as -1.4 rad.

[0140] Step S12: Obtain the ontology perception information o at the current time t. tBy concatenating the ontology-aware information with the past 6 historical moments, we obtain the comprehensive ontology-aware information {o t - 6 , o t};

[0141] Step S13: Obtain the first privileged observation information at the current time t, including the base linear velocity at the current time t. Foot contact force f t Foot swing height d t and local terrain elevation point map e t Obtain the second privileged observation information at the current time t, including the base height at the current time t, the horizontal position of the four feet relative to the base, and the horizontal velocity of the four feet relative to the base.

[0142] Step S2: Input the comprehensive ontology perception information into the preliminary multimodal online estimator for feature encoding and decoupling to obtain hybrid physical features. The process is as follows:

[0143] Step S21: Integrate the ontology perception information {o t - 6 , … , o t The flattened form is a temporal ontology-aware feature vector. In this embodiment, due to the ontology-aware information o at a single current time t... t The total dimension is 45. After flattening the integrated ontology perception information, a temporal ontology perception feature vector with a dimension of 315 is obtained.

[0144] Step S22: Input the temporal ontology-aware feature vector of dimension 315 into the preliminary multimodal online estimator. The preliminary multimodal online estimator adopts a multilayer perceptron (MLP) network, whose network structure includes: an input layer of dimension 315, a shared hidden layer consisting of three layers with the number of nodes configured as 256, 128 and 64 respectively, each layer being followed by an ELU activation function, and a multi-head output layer connected to the output of the shared hidden layer. The multi-head output layer consists of three parallel independent feature mapping branches, specifically: a velocity estimation branch consisting of an independent fully connected layer with an input dimension of 64 and an output dimension of 3, serving as the explicit base linear velocity; a contact prediction branch consisting of an independent fully connected layer with an input dimension of 64 and an output dimension of 4, serving as the explicit foot contact state; and a terrain representation branch consisting of an independent fully connected layer with an input dimension of 64 and an output dimension of 16, serving as the implicit terrain representation.

[0145] Step S23: The 315-dimensional temporal ontology-aware feature vector is input into the first shared fully connected layer for linear mapping and bias term superposition, resulting in a 256-dimensional first-level feature vector. The smoothing nonlinearity of the corresponding ELU activation function is used to process the first-level feature vector, effectively transmitting and smoothing the perception features containing negative physical states while preserving positive dynamic activation signals and suppressing redundant noise. Subsequently, the first-level feature vector is passed through the second and third shared fully connected layers, performing linear and nonlinear mapping operations from 256 to 128 dimensions and from 128 to 64 dimensions, respectively. During this process, the shared hidden layer, through the feature extraction of weights, achieves deep fusion of the physical components in the temporal ontology-aware feature vector along the temporal dimension, ultimately compressing the original high-dimensional sensor observation data into a 64-dimensional shared latent feature containing dynamic context information.

[0146] Step S24: Input the 64-dimensional shared latent features into the three independent fully connected branches of the multi-head output layer for feature decoupling mapping, and output the hybrid physical features in parallel. (See attached image) Figure 3 As shown in the upper left portion, the multi-head output layer receives the 64-dimensional shared latent features from the shared hidden layer and inputs them in parallel to three independent feature mapping branches to perform decoupling operations: The velocity estimation branch utilizes its independent fully connected layer to perform linear mapping, mapping the 64-dimensional shared latent features into a 3-dimensional vector, obtaining explicit base linear velocity estimates in the three-axis directions. = [1.75, -0.1, -0.2] T The contact prediction branch performs a linear mapping using its independent fully connected layers, transforming the 64-dimensional shared latent features into a 4-dimensional vector to obtain an explicit contact state estimate representing the foot-ground contact state of the quadruped robot. = [0.72, 0, 0.5, 0.96] T The terrain representation branch uses its independent fully connected layers to perform linear mapping, transforming the 64-dimensional shared latent features into 16-dimensional vectors, thus obtaining a preliminary implicit terrain representation reflecting the terrain's geometric features. = [-0.52, -0.9, 0.43, -0.11, -0.28, 0.72, 0.32, -0.7, 0.38, 0.06, 1.77, -1.4, -0.11, 0, 0.35, 1.44] T The physical estimation features generated in parallel by the three branches are collectively referred to as hybrid physical features.

[0147] Step S3: Perform feature extraction and threshold determination on the first privileged observation information at the current time t to obtain the target physical features. The process is as follows:

[0148] Step S31: Obtain local terrain elevation point map e t In this embodiment, the relative height information of 19×13 grid points (a total of 247 height sampling points) around the quadruped robot base is collected as a local terrain height point map e. t The input is a pre-trained static feature extractor for feature extraction. In this embodiment, the static feature extractor adopts a convolutional neural network architecture, including: a first convolutional block (using a 5 × 5 convolutional kernel, stride of 2, padding of 2, and 64 output channels, and processed by batch normalization and ELU activation); a second convolutional block (using a 3 × 3 convolutional kernel, stride of 1, padding of 1, and 128 output channels, and processed by batch normalization and ELU activation); a global average pooling layer, a flattening layer, and a fully connected layer. A 16-dimensional implicit terrain representation of the target is output through the fully connected layer.

[0149] = [-0.13, -0.17, 0.13, 0.02, -0.09, 0.22, 0.05, -0.12, 0.02, 0.09, 0.45, -0.33, -0.02, 0.07, 0.05, 0.32] T .

[0150] Step S32: Analyze the foot contact force f from the first privileged observation information at the current time t. t and foot swing height d t Threshold determination is performed to obtain the target foot contact state. Specifically, as shown in the attached document. Figure 3 As shown in the lower left part, the contact force f at the foot end t =[f fl , f fr ,f rl , f rr ] T and the swing height d t = [d fl , d fr ,d rl , d rr ] T This includes the physical quantities corresponding to the left front leg, right front leg, left hind leg, and right hind leg, respectively. In this embodiment, the force threshold f is... th Configured to 0.7 N, threshold d th With a configuration of 0.04 m, the foot contact force f was obtained from the simulation environment. t = [103.75, 0, 61.24, 46.78] T (Unit: N), foot swing height d t= [0.01, 0.11, 0.02, 0.02] T (Unit: m). Based on the threshold determination formula... Obtain the target foot contact state of the i-th leg. The leg is considered to be in contact only if the normal contact force of the i-th leg is greater than 0.7 N and the foot's height above the ground is less than 0.04 m. =1; otherwise, it is determined to be a non-contact state, i.e. = 0. Based on the above determination, the 4-dimensional target foot contact state is obtained. = [1, 0, 1, 1] T .

[0151] Step S33: Directly extract the base linear velocity from the first privileged observation information at the current time t. = [1.08, 0.05, -0.03] T Base linear velocity Implicit terrain representation of targets Contact state with the target's foot These are collectively referred to as target physical characteristics.

[0152] Step S4: Input the hybrid physical features and the ontology perception information at the current time t into the preliminary strategy network, and output the joint position deviation a. t The first privileged observation information and the comprehensive ontology perception information at the current time t are input into the value network, and the state value V is output. t In this embodiment, both the initial policy network and the value network are configured with the same multilayer perceptron network, and the process is as follows:

[0153] Step S41: Concatenate the hybrid physical features with the ontology perception information at the current time t, perform feature mapping through the preliminary policy network, and output the joint position deviation a. t In this embodiment, the initial policy network uses a multilayer perceptron network with [1024, 512, 256] hidden layer nodes, and each hidden layer uses the ELU activation function. (See attached diagram.) Figure 3 As shown, its input layer receives ontology-aware information consisting of 23-dimensional hybrid physical features and 45-dimensional current time t. t The concatenated 68-dimensional feature vector, with an output layer dimension of 12, is processed by an action scaling factor of 0.25 to obtain the joint position deviation 'a' representing the target position of the 12 joints relative to their initial positions. t = [-0.45, 0.09, 0.28, 0.51, -0.46, 0.07, -0.65, 0.18, -0.15, 0.42, -0.12, 0.26] T(Unit: rad) In a specific embodiment of the present invention, the elements of all multidimensional feature vectors or matrices involving 12 joints are arranged in a fixed order as follows: corresponding to the hip joint, thigh joint and calf joint of the robot's left front leg, right front leg, left hind leg and right hind leg in sequence.

[0154] Step S42: Concatenate the first privileged observation information at the current time t with the comprehensive ontology perception information, perform feature mapping through the value network, and output the scalar form of the state value V. t In this embodiment, the hidden layer configuration of the value network is consistent with that of the initial policy network (i.e., the number of hidden layer nodes is also configured as [1024, 512, 256] and the ELU activation function is used). The input layer receives a feature vector composed of 198-dimensional first privileged observation information at the current time t and 315-dimensional comprehensive ontology perception information (i.e., the input layer dimension is configured as 513). The output layer dimension is configured as 1, which is used to output the scalar state value V representing the quality of the current environmental state. t = 5.3.

[0155] Step S5: The robot adjusts the joint position deviation a. t After performing the action, obtain the environment state S at time t+1. t+1 Based on the environmental state S at time t+1 t+1 Calculate the single-step compound reward function The process is as follows:

[0156] Step S51: Adjust the joint position deviation a t The target positions of the 12 joints are obtained by adding them element-wise to their initial positions:

[0157] q targ = [-0.25, 0.99, -1.12, 0.51, 0.44, -1.33, -0.45, 1.28, -1.55, 0.42, 0.98, -1.14] T (Unit: rad). Extract the ontology perception information o at the current time t. t The positional error of the 12 joints at their current positions relative to their initial positions:

[0158] q erro = [-0.24, -0.42, 0.03, 0.17, -0.55, -0.14, -0.3, 0.1, -0.31, 0.31, -0.05, -0.14] T (Unit: rad), add element-wise the initial positions of the 12 joints to obtain the current positions q of the 12 joints. t= [-0.04, 0.48, -1.37, 0.17, 0.35, -1.54, -0.1, 1.2, -1.71, 0.31, 1.05, -1.54] T (Unit: rad), and simultaneously extract the ontology perception information o at the current time t. t The 12 joint velocities = [-1.3,4.95, -2.19, -2.99, -6.9, 21.4, 0.09, 4.51, 0.21, 0.29, 4.93, -0.46] T (Unit: rad / s). In this embodiment, the proportional gain coefficient K is set. p = 30 Nm / rad, differential gain coefficient K d = 0.7 Nms / rad The joint torque τ is calculated using a PD controller. t The calculation formula is: τ t = K p × (q targ - q t ) - K d × The calculated joint torque τ t = [-5.39, 11.84, 9.03, 12.29, 7.53, -8.68, -10.56, -0.76, 4.65, 3.09, -5.55, 12.32] T (Unit: N·m) The data is sent to the motor controllers of the robot's 12 joints to drive the robot to generate physical displacement and obtain the environmental state S at time t+1. t+1 .

[0159] Step S52, from the environmental state S at time t+1 t+1 In the process, obtain the ontology perception information at time t+1. t+1 The first privileged observation information at time t+1 and the second privileged observation information at time t+1; among which, the proprioceptive information at time t+1. t+1 The first privileged observation information and the second privileged observation information at time t+1, together with the ontological perception information at the current time t. t The components of the first privileged observation information at current time t and the second privileged observation information at current time t are the same.

[0160] Step S53: Calculate the single-step composite reward function In this embodiment, the weighting coefficients are set as follows: λ1 = 1.5, λ2 = 0.6, λ3 = 0.15, λ4 = 0.15, λ5 = 0.00003, λ6 = 0.03, λ7 = 0.05; the sensitivity coefficients are set as follows: σ1 = σ2 = 0.27, σ3 = 0.6, σ4 = 0.7; and the target width is preset and set. = 0.13 m, target height constant = 0.1 m, j ∈ {1,2, 3, 4}. In the current embodiment, = [1.1, 0.1] T (Unit: m / s), v t+1 =[1.15, 0.09] T (Unit: m / s) = 0.1 rad / s, ω t+1 = 0.16 rad / s, according to the formula Calculate the task tracking reward = 2.08; = 0.08 m, = -0.11 m, = 0.08 m, = -0.06 m, according to the formula Calculate the geometric constraint reward = 0.25; A t = [-0.45, 0.09, 0.28, 0.51, -0.46, 0.07, -0.65, 0.18, -0.15, 0.42, -0.12, 0.26] T (Unit: rad), A t-1 = [-0.91, -0.56, 0.52, 0.27, -0.32, 0.12, -0.17, -0.18, -0.2, 0.65, 0.07, 0.14] T (Unit: rad), d t+1 = [0.02, 0.08, 0.03, 0.02] T (Unit: m) = [-1.07, -0.02] T m / s = [-0.75, 0.81] T m / s = [-1.04, -0.05] T m / s = [-1.03, -0.06]T m / s, = [0.13, 4.28, -1.89, 2.52, 2.14, 3.89, 1.51, 2.86, 1.72, 1.36, 2.87, 1.83] T (Unit: rad / s), based on joint torque τ t ,according to The regularization penalty term was calculated. = -0.04. According to the formula Calculations yielded = 2.29.

[0161] Step S6: Based on the environmental state S at time t+1 t+1 Update the comprehensive ontology perception information, repeat steps S1 to S4, and obtain the state value V at time t+1. t+1 = 5.32, based on a single-step composite reward function State value V t and the state value V at time t+1 t+1 The reinforcement learning loss function is obtained to train the initial policy network and the value network, resulting in a trained target policy network. Based on the deviation between the mixed physical features and the target physical features, a multi-task estimation loss function is obtained to train the initial multimodal online estimator, resulting in a trained target multimodal online estimator. The process is as follows:

[0162] Step S61: Based on the single-step composite reward function State value V t and the state value V at time t+1 t+1 Set the discount factor γ = 0.92, and use the formula... = r t + γV t+1 - V t Calculate the action advantage value = 1.88;

[0163] Step S62: In this embodiment, the cutting ratio is set to... = 0.1, value loss coefficient c1 = 0.5. Based on the environmental state S at the current time t. t Joint position deviation a t get =1.05, further, based on action advantage value Single-step composite reward function State value V t and the state value V at time t+1 t+1 According to the formula:

[0164] The reinforcement learning loss function L is calculated. RL = 0.21.

[0165] Step S63: In this embodiment, the learnable uncertainty parameters w1 = 1.5, w2 = 0.8, and w3 = 25. Based on the hybrid physical characteristics and the target physical characteristics, according to the formula... The first loss function L1 can be calculated to be 0.5; according to the formula... The second loss function L2 can be calculated to be 1.06; according to the formula... The third loss function L3 is calculated to be 0.02; finally, according to the formula: Calculate the multi-task estimation loss function L est = 2.10.

[0166] Step S64: Utilize the reinforcement learning loss function L RL Update the initial policy network and estimate the loss function L using a multi-task approach. est Update the initial multimodal online estimator and perform 3000 iterations or until the network loss function converges to obtain the trained target policy network and the target multimodal online estimator.

[0167] Step S7: In the deployment and control phase of the quadruped robot in the real-world scenario, the robot's comprehensive ontological perception information is acquired and input into the trained target multimodal online estimator to obtain hybrid physical features. Subsequently, the hybrid physical features and the ontological perception information at the current time t are input into the trained target policy network to output the joint position deviation at the current time. After the joint position deviation is superimposed on the initial joint position to obtain the target joint position, it is converted into 12 joint torque commands by the PD controller and finally sent to the underlying motors of the quadruped robot to drive the robot to perform the corresponding movements.

[0168] To further verify the effectiveness and advancement of the quadruped robot motion control method based on multimodal online estimation proposed in this invention when facing drastic terrain changes and extreme physical disturbances, this embodiment conducted targeted comparative experiments and performance evaluations in both simulation environments and real physical scenarios.

[0169] 1. Simulation Environment Comparison Experiment and Analysis: In this embodiment, the impact robustness of the proposed method under discrete terrain with varying elevation differences is verified using the simulation physics engine Isaac Gym. To ensure the objectivity of the evaluation, 100 independent tests were conducted for each terrain configuration, and the proposed method was compared with the benchmark algorithms LQL and PA-LOCO of the "teacher-student" two-stage distillation strategy framework. The simulated terrain specifically included: a single step height ranging from 20 cm to 40 cm, and a single step drop (fall) ranging from 1 m to 3 m. The statistical data on the success rate of each algorithm under different terrain difficulties are shown in Table 3.

[0170] Table 3. Comparison of Success Rates of Various Algorithms in Complex Terrain under Simulation Environment

[0171]

[0172] As shown in Table 3, in discrete geometric jump environments, the performance advantage of this invention is more significant than that of traditional benchmark algorithms. In single-stage stair climbing tasks, traditional two-stage algorithms, lacking accurate prediction of sudden changes in local terrain, are prone to climbing failure due to blind collisions (e.g., LQL's success rate is 0% at 25 cm, and PA-LOCO's is 0% at 30 cm). In contrast, this invention implicitly constructs terrain features through a multimodal online estimator, enabling the robot to predictively adjust its foot placement, maintaining a 67% success rate even at a 30 cm step height. In single-stage stair descending tasks, facing the enormous transient impact of a fall, traditional algorithms struggle to maintain balance due to the delayed decoupling of state estimation and control strategy (e.g., PA-LOCO fails at a 3 m drop). This invention, benefiting from the real-time estimation of physical contact states by the online estimator, guides the policy network to rapidly recover base stability, achieving a 19% survival recovery rate even with a 3 m extreme drop. (See attached table.) Figure 8 and attached Figure 9 The diagram shows the motion posture of the quadruped robot driven by the control strategy of this invention to successfully climb a 40 cm extreme step and jump down a 3 m high platform in a simulation environment.

[0173] 2. Real-world Deployment Comparison Experiment: To verify the robustness against disturbances and load adaptability of the present invention under extreme physical disturbances in real-world scenarios, this embodiment directly deploys the converged simulation training model onto a real quadruped robot. The robot performs blind walking solely based on its proprioceptive sensors without any external visual input. The comparison of various indicators between the present invention's method and the two-stage distillation benchmark algorithm under real-world challenge scenarios (20 independent tests each) is shown in Table 4:

[0174] Table 4. Comparison of Success Rates of Various Algorithms in Real-World Physical Scenarios

[0175]

[0176] As shown in Table 4, this invention demonstrates overwhelming technical advantages in real-world scenarios with discrete drops and complex interference. When facing 25 cm ascending steps and 50 cm descending steps, the strategy of this invention can estimate the foot contact state through a multimodal online estimator, achieving a 100% success rate. When traversing dense rigid vegetation below 50 cm, the online estimator of this invention successfully decouples the dynamic disturbance of the vegetation from the implicit terrain representation of the underlying layer, avoiding confusion in environmental state inference, and achieving a success rate far exceeding that of the compared algorithms. Furthermore, under extreme physical load conditions, this invention successfully achieved a static load of up to 9.5 kg (approximately 80% of the robot's own weight) and a dynamic drag of 10 kg. Although the large load caused significant deformation of the base's passive height and generated a strong yaw drag torque, this invention can still adaptively correct the trajectory through real-time estimation of mixed physical features using a multimodal online estimator, ensuring the stability of the movement. (See attached table) Figure 10 Appendix Figure 11 and attached Figure 12 The image shown is a real-world operational schematic diagram of the invention crossing high platforms, traversing dense vegetation, and carrying extremely heavy loads in a real-world scenario.

[0177] Example 3

[0178] Reference Figure 13 This embodiment provides a quadruped robot motion control system based on multimodal online estimation, used to execute a quadruped robot motion control method based on multimodal online estimation disclosed in the above embodiment. The quadruped robot motion control system based on multimodal online estimation includes, in sequence, a state and information acquisition module 1301, a hybrid physical feature extraction module 1302, a target physical feature extraction module 1303, a state value extraction module 1304, a single-step composite reward function calculation module 1305, a policy network and estimator training module 1306, and a quadruped robot joint control module 1307, wherein:

[0179] The status and information acquisition module 1301 is used to acquire the environmental status S at the current time t. t The integrated ontological perception information of the quadruped robot, including the environmental state S t This includes the ontology perception information at the current time t, the first privileged observation information, and the second privileged observation information;

[0180] The hybrid physical feature extraction module 1302 is used to input the integrated ontology perception information into the preliminary multimodal online estimator for feature encoding and decoupling to obtain hybrid physical features;

[0181] The target physical feature extraction module 1303 is used to extract features and determine thresholds from the first privileged observation information at the current time t to obtain the target physical features.

[0182] The state value extraction module 1304 is used to input the hybrid physical features and the ontology perception information at the current time t into the preliminary strategy network, and output the joint position deviation a. t Input the first privileged observation information and the comprehensive ontology perception information at the current time t into the value network, and output the state value V at the current time t. t ;

[0183] The single-step composite reward function calculation module 1305 is used by the quadruped robot to calculate the joint position deviation 'a'. t After performing the action, obtain the environment state S at time t+1. t+1 Based on environmental state S t+1 Calculate the single-step compound reward function ;

[0184] Policy network and estimator training module 1306, used to train the policy network and estimator according to the environment state S t+1 Update the integrated ontology perception information, repeatedly call the state and information acquisition module, the hybrid physical feature extraction module, the target physical feature extraction module, and the state value extraction module to obtain the state value V at time t+1. t+1 Based on the single-step composite reward function State value V t and state value V t+1 Thus, the reinforcement learning loss function is obtained;

[0185] The initial policy network is trained by the reinforcement learning loss function to obtain the trained target policy network. Based on the deviation between the mixed physical features and the target physical features, a multi-task estimation loss function is obtained. The initial multimodal online estimator is trained by the multi-task estimation loss function to obtain the trained target multimodal online estimator.

[0186] The quadruped robot joint control module 1307 is used to acquire the comprehensive body perception information of the quadruped robot in real time in a real scene, and input it into the target multimodal online estimator to obtain hybrid physical features. The hybrid physical features and the body perception information at the current time t are input into the target policy network to obtain joint control commands.

[0187] Example 4

[0188] This embodiment provides an electronic device, which can be a computer, such as... Figure 14As shown, the system bus 1401 connects a processor 1402, a memory, an input device 1403, a display 1404, and a network interface 1405. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium 1406 and internal memory 1407. The non-volatile storage medium 1406 stores the operating system, computer programs, and a database. The internal memory 1407 provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. When the processor 1402 executes the computer programs stored in the memory, it implements the quadruped robot motion control method based on multimodal online estimation proposed in Embodiment 1. The quadruped robot motion control method includes the following steps:

[0189] S1. Obtain the environmental state S at the current time t. t The integrated ontological perception information of the quadruped robot, including the environmental state S t This includes the ontology perception information at the current time t, the first privileged observation information, and the second privileged observation information;

[0190] S2. Input the comprehensive ontology perception information into the preliminary multimodal online estimator for feature encoding and decoupling to obtain hybrid physical features;

[0191] S3. Perform feature extraction and threshold determination on the first privileged observation information at the current time t to obtain the target physical characteristics;

[0192] S4. Input the hybrid physical features and the body perception information at the current time t into the preliminary strategy network, and output the joint position deviation a. t Input the first privileged observation information and the comprehensive ontology perception information at the current time t into the value network, and output the state value V at the current time t. t ;

[0193] S5, the quadruped robot based on joint position deviation a t After performing the action, obtain the environment state S at time t+1. t+1 Based on environmental state S t+1 Calculate the single-step compound reward function ;

[0194] S6, based on environmental conditions S t+1 Update the comprehensive ontology perception information, repeat steps S1 to S4, and obtain the state value V at time t+1. t+1 Based on the single-step composite reward function State value V t and state value V t+1 Thus, the reinforcement learning loss function is obtained;

[0195] The initial policy network is trained by the reinforcement learning loss function to obtain the trained target policy network. Based on the deviation between the mixed physical features and the target physical features, a multi-task estimation loss function is obtained. The initial multimodal online estimator is trained by the multi-task estimation loss function to obtain the trained target multimodal online estimator.

[0196] S7. In real-time, acquire the comprehensive ontological perception information of the quadruped robot in the real-world scenario, and input it into the target multimodal online estimator to obtain hybrid physical features. Input the hybrid physical features and the ontological perception information at the current time t into the target policy network to obtain joint control commands.

[0197] Example 5

[0198] This embodiment provides a storable medium, which is a computer-readable and storable medium, storing a computer program. When the computer program is executed by a processor, it implements the quadruped robot motion control method based on multimodal online estimation proposed in Embodiment 1 above. The quadruped robot motion control method includes the following steps:

[0199] S1. Obtain the environmental state S at the current time t. t The integrated ontological perception information of the quadruped robot, including the environmental state S t This includes the ontology perception information at the current time t, the first privileged observation information, and the second privileged observation information;

[0200] S2. Input the comprehensive ontology perception information into the preliminary multimodal online estimator for feature encoding and decoupling to obtain hybrid physical features;

[0201] S3. Perform feature extraction and threshold determination on the first privileged observation information at the current time t to obtain the target physical characteristics;

[0202] S4. Input the hybrid physical features and the body perception information at the current time t into the preliminary strategy network, and output the joint position deviation a. t Input the first privileged observation information and the comprehensive ontology perception information at the current time t into the value network, and output the state value V at the current time t. t ;

[0203] S5, the quadruped robot based on joint position deviation a t After performing the action, obtain the environment state S at time t+1. t+1 Based on environmental state S t+1 Calculate the single-step compound reward function ;

[0204] S6, based on environmental conditions S t+1Update the comprehensive ontology perception information, repeat steps S1 to S4, and obtain the state value V at time t+1. t+1 Based on the single-step composite reward function State value V t and state value V t+1 Thus, the reinforcement learning loss function is obtained;

[0205] The initial policy network is trained by the reinforcement learning loss function to obtain the trained target policy network. Based on the deviation between the mixed physical features and the target physical features, a multi-task estimation loss function is obtained. The initial multimodal online estimator is trained by the multi-task estimation loss function to obtain the trained target multimodal online estimator.

[0206] S7. In real-time, acquire the comprehensive ontological perception information of the quadruped robot in the real-world scenario, and input it into the target multimodal online estimator to obtain hybrid physical features. Input the hybrid physical features and the ontological perception information at the current time t into the target policy network to obtain joint control commands.

[0207] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0208] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A motion control method for a quadruped robot based on multimodal online estimation, characterized in that, The control method includes the following steps: S1. Obtain the environmental state S at the current time t. t The integrated ontological perception information of the quadruped robot, including the environmental state S t This includes the ontology perception information at the current time t, the first privileged observation information, and the second privileged observation information; S2. Input the comprehensive ontology perception information into the preliminary multimodal online estimator for feature encoding and decoupling to obtain hybrid physical features; S3. Perform feature extraction and threshold determination on the first privileged observation information at the current time t to obtain the target physical characteristics; S4. Input the hybrid physical features and the body perception information at the current time t into the preliminary strategy network, and output the joint position deviation a. t Input the first privileged observation information and the comprehensive ontology perception information at the current time t into the value network, and output the state value V at the current time t. t ; S5, the quadruped robot based on joint position deviation a t After performing the action, obtain the environment state S at time t+1. t+1 Based on environmental state S t+1 Calculate the single-step compound reward function ; S6, based on environmental conditions S t+1 Update the comprehensive ontology perception information, repeat steps S1 to S4, and obtain the state value V at time t+1. t+1 Based on the single-step composite reward function State value V t and state value V t+1 Thus, the reinforcement learning loss function is obtained; The initial policy network is trained by the reinforcement learning loss function to obtain the trained target policy network. Based on the deviation between the mixed physical features and the target physical features, a multi-task estimation loss function is obtained. The initial multimodal online estimator is trained by the multi-task estimation loss function to obtain the trained target multimodal online estimator. S7. In real-time, acquire the comprehensive ontological perception information of the quadruped robot in the real-world scenario, and input it into the target multimodal online estimator to obtain hybrid physical features. Input the hybrid physical features and the ontological perception information at the current time t into the target policy network to obtain joint control commands.

2. The quadruped robot motion control method based on multimodal online estimation according to claim 1, characterized in that, The process of step S1 is as follows: S11. Obtain the body perception information of the quadruped robot at the current time t. t This includes the user's speed command for the quadruped robot, the base angular velocity, the gravity vector, the position error of the current position of the 12 joints relative to the initial position of the 12 joints, the velocities of the 12 joints, and the motion vectors of the 12 joints at the previous moment; among them, the 12 joints are specifically the hip joint, thigh joint, and calf joint of the left front leg, right front leg, left hind leg, and right hind leg of the quadruped robot. S12, Obtain the quadruped robot's body perception information at the current time t. t The ontological perception information o from the past H historical moments tH,…, t-1 t-H , … , o t-1 By splicing, we obtain the comprehensive ontology perception information {o t-H , … , o t }; S13. Obtain the first privileged observation information at the current time t, including the base linear velocity at the current time t. Foot contact force f t Foot swing height d t And the local terrain elevation map around the quadruped robot e t The system obtains the second privileged observation information at the current time t, including the base height at the current time t, the horizontal position of the four feet relative to the base, and the horizontal velocity of the four feet relative to the base. Specifically, the four feet are the left forefoot, right forefoot, left hindfoot, and right hindfoot of the quadruped robot.

3. The quadruped robot motion control method based on multimodal online estimation according to claim 1, characterized in that, The process of step S2 is as follows: S21, Integrate ontological perception information {o t-H , … , o t Flattened into temporal ontology-aware feature vectors; S22. Input the temporal ontology-aware feature vector into the preliminary multimodal online estimator. The preliminary multimodal online estimator is a multilayer perceptron network consisting of an input layer, a shared hidden layer, and a multi-head output layer connected in sequence. S23. High-dimensional spatiotemporal feature extraction is performed on the temporal ontology perception feature vector through a shared hidden layer to obtain shared latent features; S24. The shared latent features are input to three independent fully connected branches of the multi-head output layer for feature decoupling mapping, and the hybrid physical features are output in parallel, including: explicit base linear velocity estimates. Explicit foot contact state estimation value and preliminary implicit terrain representation .

4. The quadruped robot motion control method based on multimodal online estimation according to claim 2, characterized in that, The process of step S3 is as follows: S31, From the local terrain elevation point map e t Feature extraction is performed to obtain the target's implicit terrain representation. ; S32, Foot contact force f in the first privileged observation information at the current time t t and foot swing height d t Threshold determination is performed to obtain the target foot contact state. The threshold determination formula is: , In the formula, i represents the leg index of the quadruped robot, i = 1, 2, 3, 4; The target foot contact state of the i-th leg is represented by a value of 1 indicating contact and a value of 0 indicating non-contact; I( ) is an indicator function, which is applied only if " The value is 1 if the condition is true, and 0 otherwise. Represents the contact force at the foot end of the i-th leg. The swing height of the i-th leg, f th d is the preset force threshold. th The preset height threshold; S33. Obtain the base linear velocity from the first privileged observation information at the current time t. , base linear velocity Implicit terrain representation of targets Contact state with the target's foot As the target physical characteristics.

5. The quadruped robot motion control method based on multimodal online estimation according to claim 1, characterized in that, The process of step S4 is as follows: S41. Concatenate the hybrid physical features with the ontology perception information at the current time t, input the concatenation into the preliminary policy network for feature mapping, and output the joint position deviation a. t ; S42. Concatenate the first privileged observation information at the current time t with the comprehensive ontology perception information, input the concatenation into the value network for feature mapping, and output the scalar state value V. t ; Both the initial strategy network and the value network are multilayer perceptron networks.

6. The quadruped robot motion control method based on multimodal online estimation according to claim 1, characterized in that, The process of step S5 is as follows: S51, Adjust joint position deviation a t The target positions q of the 12 joints are obtained by superimposing them onto the initial positions of the 12 joints respectively. targ It calculates the joint torque τ, drives the robot to generate physical displacement, and obtains the environmental state S at time t+1. t+1 ; S52, From the environmental state S at time t+1 t+1 In the process, the body perception information of the quadruped robot at time t+1 is obtained. t+1 First-privileged observation information and second-privileged observation information at time t+1; S53. Calculate the single-step composite reward function : , Among them, task tracking rewards The calculation formula is as follows: , Geometric Constraint Rewards The calculation formula is as follows: , Regulate penalty items The calculation formula is as follows: , In the formula, and These represent the ontology perception information extracted from time t+1, respectively. t+1 The horizontal linear velocity command and yaw rate command are included in the user speed commands; v t+1 ω represents the base linear velocity extracted from the first privileged observation information at time t+1. t+1 This represents the ontology perception information extracted from time t+1. t+1 The base angular velocity; and τ represents the horizontal position and horizontal velocity of the j-th foot relative to the base, respectively, extracted from the second privileged observation information at time t+1, where j represents the foot index of the quadruped robot, j = 1, 2, 3, 4; k This represents the k-th component of the joint torque τ, where k = 1, 2, …, 12 are the joint indices; This represents the ontology perception information extracted from time t+1. t+1 The velocity of the k-th joint; A represents the foot swing height at the j-th foot end extracted from the first privileged observation information at time t+1; t-1 and A t These represent the ontological perception information o from time t. t and the ontological perception information o at time t+1 t+1 The motion vectors of the 12 joints obtained from the previous moment; and Let exp( be the preset target width constant and target height constant for the j-th foot end); ) to represent " "An exponential function with the natural constant e as its base; || ||2 indicates that " The 2-norm operation; | | indicates " "Absolute value operation;" Indicates to " The summation operation of λ; n is the weight index, and n = 1, 2, … , 7; n σ is the nth weighting coefficient, used to adjust the contribution ratio of each weighted sub-item in the single-step composite reward function; m is the sensitivity index, and m = 1, 2, 3, 4; m is the m-th sensitivity coefficient, used to adjust the smoothness of the corresponding exponential function term as the physical quantity deviates.

7. The quadruped robot motion control method based on multimodal online estimation according to claim 1, characterized in that, The process of step S6 is as follows: S61. Based on the single-step composite reward function State value V t and state value V t+1 Through formula = r t + γV t+1 -V t Calculate action advantage value In the formula, γ is the preset discount factor; S62, Based on the aforementioned action advantage value Environmental State S t Joint position deviation a t Single-step composite reward function State value V t and state value V t+1 Obtain the reinforcement learning loss function L RL : ， In the formula, π θ (a t |S t ) and π θold (a t |S t The numbers θ and θ represent the initial policy network under the current parameters to be updated, and the previous version's update parameters θ before this update, respectively. old Under environmental state S t Lower output joint position deviation a t The probability of; c1 is the preset clipping ratio; c1 is the preset value loss coefficient; min(a, b) represents the operation of selecting the minimum value from "a" and "b"; clip( , 1- , 1+ ) indicates " Perform a clipping operation to restrict its value to the interval [1- , 1+ ]Inside; Indicates to " "Calculation of the expected value under the current batch of samples;" S63. Based on the deviation between the hybrid physical features and the target physical features, construct a multi-task estimation loss function L. est : ， In the formula, w p For the pre-defined learnable uncertainty parameter p, L p Let p be the loss function, where p = 1, 2, 3; The formula for calculating the first loss function L1 is: , in, It is the linear velocity of the base. This is an estimate of the explicit base linear velocity. ||2 indicates that " The 2-norm operation; The formula for calculating the second loss function L2 is: ,in, This represents the target foot contact state of the i-th leg, extracted from the target foot contact state. , This represents the explicit foot contact state estimate of the i-th leg, extracted from the explicit foot contact state estimate. ; The formula for calculating the third loss function L3 is: ,in, It is an implicit terrain representation of the target. This is a preliminary implicit topographic representation; S64. Utilizing the reinforcement learning loss function L RL Update the initial policy network and estimate the loss function L using a multi-task approach. est Update the initial multimodal online estimator, iteratively execute the above update process until the network converges, and obtain the target policy network and the target multimodal online estimator.

8. A quadruped robot motion control system based on multimodal online estimation, used to execute the quadruped robot motion control method based on multimodal online estimation as described in any one of claims 1 to 7, characterized in that, The control system includes: The status and information acquisition module is used to acquire the environmental state S at the current time t. t The integrated ontological perception information of the quadruped robot, including the environmental state S t This includes the ontology perception information at the current time t, the first privileged observation information, and the second privileged observation information; The hybrid physical feature extraction module is used to input the integrated ontology perception information into the preliminary multimodal online estimator for feature encoding and decoupling to obtain hybrid physical features; The target physical feature extraction module is used to extract features and determine thresholds from the first privileged observation information at the current time t to obtain the target physical features. The state value extraction module is used to input the hybrid physical features and the ontology perception information at the current time t into the preliminary strategy network, and output the joint position deviation a. t Input the first privileged observation information and the comprehensive ontology perception information at the current time t into the value network, and output the state value V at the current time t. t ; The single-step composite reward function calculation module is used by the quadruped robot to calculate the joint position deviation 'a'. t After performing the action, obtain the environment state S at time t+1. t+1 Based on environmental state S t+1 Calculate the single-step compound reward function ; The policy network and estimator training module is used to train the policy network based on the environment state S. t+1 Update the integrated ontology perception information, repeatedly call the state and information acquisition module, the hybrid physical feature extraction module, the target physical feature extraction module, and the state value extraction module to obtain the state value V at time t+1. t+1 Based on the single-step composite reward function State value V t and state value V t+1 Thus, the reinforcement learning loss function is obtained; The initial policy network is trained by the reinforcement learning loss function to obtain the trained target policy network. Based on the deviation between the mixed physical features and the target physical features, a multi-task estimation loss function is obtained. The initial multimodal online estimator is trained by the multi-task estimation loss function to obtain the trained target multimodal online estimator. The quadruped robot joint control module is used to acquire the comprehensive body perception information of the quadruped robot in real time in real-world scenarios, and input it into the target multimodal online estimator to obtain hybrid physical features. The hybrid physical features and the body perception information at the current time t are then input into the target policy network to obtain joint control commands.

9. An electronic device comprising a processor and a memory for storing a processor-executable program, characterized in that, When the processor executes the program stored in the memory, it implements the quadruped robot motion control method based on multimodal online estimation as described in any one of claims 1 to 7.

10. A storable medium storing a program, characterized in that, When the program is executed by the processor, it implements the quadruped robot motion control method based on multimodal online estimation as described in any one of claims 1 to 7.

Images