Humanoid robot multi-terrain gait control method and system fused with visual perception

Abstract

The invention belongs to the field of robot motion control, and particularly relates to a humanoid robot multi-terrain gait control method and system fused with visual perception. The method comprises the following steps: acquiring multi-modal sensing data of the humanoid robot, and preprocessing the multi-modal sensing data; the preprocessed multi-modal sensing data are input into a pre-trained world model, and the world model updates and outputs a potential state at the current moment based on a historical recursive state, a historical random posteriori state and an action sequence in a historical updating interval; inputting the potential state at the current moment into a pre-trained strategy network, and outputting an action at the current moment; and based on the action at the current moment, the driving torque of each joint is calculated through a PD controller, and the humanoid robot is driven to move. According to the method, a world model structure is introduced, so that the robot can realize more stable and more efficient gait control and terrain adaptation of the humanoid robot under the condition that the robot only depends on perception information which can be acquired by the robot.

Description

Technical Field

[0001] This invention belongs to the field of robot motion control, specifically relating to a multi-terrain gait control method and system for humanoid robots that integrates visual perception. Background Technology

[0002] With the rapid development of humanoid robot technology, the application of robots in various environments is increasing. The ability to move stably on various terrains has become a problem that urgently needs to be solved. This not only requires robots to have the ability to accurately perceive their own state and the surrounding environment, but also requires them to have the ability to adaptively adjust their movement strategies according to changes in terrain.

[0003] Traditional control methods struggle to achieve satisfactory results in addressing this challenge. In recent years, reinforcement learning, as a data-driven control method, has begun to show its potential in the field of robot control due to its powerful learning and adaptive capabilities. However, existing methods typically employ a "teacher-student" architecture, where the student first learns a policy from a teacher with privileged information, and then learns a policy based on visual input to mimic the teacher's behavior. While this approach has made some progress, the performance of the student policy often falls short of the teacher policy's optimal level due to differences in the input information.

[0004] Therefore, there is an urgent need for a simpler and more efficient method that can accurately predict motion trajectories in the real world by constructing high-fidelity simulated environment models, thereby providing more informative feedback signals to the policy controller. Motion strategies learned based on this can significantly improve the motion adaptability and stability of humanoid robots in complex terrains, further expanding their application potential in real-world scenarios. Summary of the Invention

[0005] The purpose of this invention is to address the problems of insufficient adaptability of reinforcement learning in complex terrain, strong dependence on privileged information, low learning efficiency, and loss of policy performance in the prior art, and to propose a multi-terrain gait control method and system for humanoid robots that integrates visual perception.

[0006] To achieve the above objectives, the technical solution provided by this invention is as follows:

[0007] Firstly, a multi-terrain gait control method for humanoid robots that integrates visual perception is provided, including:

[0008] Collect multimodal perception data from the humanoid robot and perform preprocessing;

[0009] The preprocessed multimodal sensing data is input into the pre-trained world model. The world model updates and outputs the potential state at the current moment based on the historical recursive state, the historical random posterior state, and the action sequence within the historical update interval.

[0010] Input the current potential state into the pre-trained policy network and output the current action;

[0011] Based on the current action, the PD controller calculates the driving torque of each joint to drive the humanoid robot to perform motion.

[0012] Furthermore, the collection and preprocessing of multimodal perception data from the humanoid robot includes:

[0013] Humanoid robot for collecting Real-time body observation and depth images of the terrain ahead; body observation includes joint position, joint velocity, body attitude, linear velocity and angular velocity.

[0014] The perceptual vector is obtained by aligning the timestamps and spatially calibrating the ontological observations and depth images.

[0015] Zero-mean normalization is performed on all continuous variables in the perceptual vector, the depth image is center-cropped and resolution compressed, and features are extracted by a convolutional encoder to obtain a fixed-dimensional latent vector representation.

[0016] Furthermore, the world model includes a recurrent unit model, an encoder, a dynamic predictor, and a decoder; the world model uses an interval of... The update mechanism for each time step; the updating and output of the potential state at the current moment includes:

[0017] Circular unit model based on Recursive state at any moment The random posterior state at time t and Time's up The action sequence at each time step is used to calculate the current time step. A deterministic recursive state;

[0018] The encoder fuses the preprocessed multimodal sensing data at the current time with the deterministic recursive state at the current time, calculates the distribution parameters of the random posterior state at the current time, and samples the random posterior state at the current time based on the distribution parameters.

[0019] Based on the deterministic recursive state at the current moment and the random posterior state at the current moment, the potential state at the current moment is obtained.

[0020] Furthermore, the training process of the world model includes:

[0021] The reconstruction loss is calculated by reconstructing the observed data based on the latent state using a decoder.

[0022] The dynamic predictor predicts the prior distribution at the current time based on the deterministic recursive state at the current time and calculates the relative entropy divergence between the prior distribution and the random posterior state.

[0023] The parameters of the world model are optimized by minimizing a joint loss function that includes reconstruction loss and relative entropy divergence.

[0024] Furthermore, the training of the policy network employs an Actor-Critic framework based on an asymmetric structure, including:

[0025] Actor Network: It only receives the robot's observable potential states and sensory inputs, and outputs the probability distribution of actions;

[0026] Critic network: takes the potential state and privileged information provided by the simulation environment as input, and outputs a state value estimate;

[0027] The privileged information includes terrain geometry, foot contact force, and the complete dynamic state without noise, and this privileged information is used only during the training phase.

[0028] Furthermore, during the training of the policy network, a proximal policy optimization algorithm is used, and the cumulative expected reward is calculated based on the reward function; the reward function includes linear velocity tracking terms, angular velocity tracking terms, gait symmetry terms, and contact state constraint terms.

[0029] Furthermore, the calculation of the driving torque of each joint based on the current action using the PD controller includes:

[0030] To obtain the default joint positions of the humanoid robot when it is standing, the target joint positions are obtained by summing the positions of the moving parts and the default joint positions when it is standing. The formula is as follows:

[0031] ;

[0032] in, Indicates the position of the target joint. This is the default joint position when standing. express The actions output by the time-based strategy network;

[0033] The driving torque of each joint is calculated by the PD controller, and expressed by the following formula:

[0034] ;

[0035] in, and These represent the current position and current velocity of the joint, respectively. and Indicates the target joint position and target joint velocity. and These are the parameters of the PD controller.

[0036] Furthermore, before collecting multimodal sensing data, a multi-terrain training environment is constructed in the simulation platform, with the following steps:

[0037] Multiple terrain models are constructed in the physics simulation engine, and the terrain parameters are randomized.

[0038] When collecting multimodal sensing data, sensor noise, delay modeling, and gravity vector perturbation are introduced.

[0039] Secondly, a multi-terrain gait control system for humanoid robots integrating visual perception is provided, including a perception module, a world model module, a policy network, an execution module, and a simulation and deployment module, wherein:

[0040] The perception module is used to collect multimodal perception data from the humanoid robot and perform preprocessing.

[0041] World Model Module: This module is used to input pre-processed multimodal perception data into a pre-trained world model. The world model updates and outputs the potential state at the current time based on historical recursive states, historical random posterior states, and action sequences within historical update intervals.

[0042] The policy network is used to input the potential state at the current time into the pre-trained policy network and output the action at the current time.

[0043] The execution module is used to calculate the driving torque of each joint based on the current action and drive the humanoid robot to perform motion.

[0044] The simulation and deployment module is used to build multi-terrain training environments within the simulation platform.

[0045] Compared with the prior art, the significant advantages of this invention are:

[0046] 1. Deprivatization training: No longer relying on teacher strategies or artificially designed privileged information, reducing human intervention and making the learning process more natural and universal.

[0047] 2. Strong information fusion and compression capabilities: The world model can fuse high-dimensional visual information with low-dimensional ontology perception information into a compact latent representation, significantly improving the efficiency and stability of policy learning.

[0048] 3. Predictive decision-making capability: Compared with traditional reactive control, the world model of this invention has the ability to perceive and predict the future, enabling the robot to plan its movement behavior in advance.

[0049] 4. Strong robustness and generalization: After training with randomized physical parameters and delay compensation, the robot can maintain stable walking in unknown terrain and noisy environments, showing good transfer performance.

[0050] 5. High computational efficiency and low deployment cost: The world model runs with low-frequency updates, reducing real-time computation pressure and can be directly deployed on lightweight embedded computing platforms (such as NVIDIA Jetson NX).

[0051] 6. Wide range of applications: This method is not only applicable to humanoid robots, but can also be extended to quadrupedal, bipedal or wheeled robots, and has general motion control and perception decision-making capabilities. Attached Figure Description

[0052] Figure 1 This is a schematic diagram of the overall structure of a multi-terrain gait control system for a humanoid robot that integrates visual perception, according to the present invention.

[0053] Figure 2 This is a flowchart illustrating the steps of a multi-terrain gait control method for a humanoid robot that integrates visual perception, as described in this invention.

[0054] Figure 3 This is a schematic diagram of the structural principle of the world model of the present invention. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0056] Example 1.

[0057] This invention provides a multi-terrain gait control method for humanoid robots that integrates visual perception. This method combines the ideas of deep reinforcement learning and world modeling, enabling humanoid robots to achieve perception prediction, autonomous decision-making, and robust control in unknown or complex terrains.

[0058] This method learns a predictable latent world model, maps high-dimensional visual input and low-dimensional ontology perception data into a compact latent state representation, and then trains a control policy based on this latent state, thus achieving end-to-end learning and decision-making from perception to control.

[0059] This invention can be applied to, for example Figure 1 The system shown includes a perception module, a world model module, a policy network, an execution module, and a simulation and deployment module.

[0060] like Figure 2 As shown, the present invention specifically includes the following steps:

[0061] S1. Perception Data Acquisition and Environment Construction: This step is used to build the robot's perception system and simulation training environment, collect and integrate multimodal perception data, and provide an input foundation for subsequent world model training and policy learning.

[0062] S11. The perception module primarily includes: a body sensor unit, a vision perception unit, and a synchronization and preprocessing unit. The body sensor unit is used to collect the robot's own state in real time, including joint position sensors (encoders), gyroscopes (IMUs), and torque sensors. The output data includes:

[0063] ;

[0064] in: for Ontological observation at any moment for The joint position at any given moment. for Joint velocity at any given moment for The body's attitude at a given time (which can be represented as a rotation matrix or quaternion). for linear velocity at time t, for Angular velocity at time t.

[0065] The visual perception unit is mounted on the robot's head and acquires visual perception information (depth image) of the terrain ahead using a depth camera (e.g., RealSense D435i) or an RGB-D camera. The visual perception information includes:

[0066] ;

[0067] in, express Depth image at any given time, This represents the depth estimation function. These are two-dimensional coordinates on the depth image plane. and These represent the height and width of the depth image, respectively. The overall representation is given by coordinates... The pixel, whose depth value is obtained through the depth estimation function. The calculation yields a scalar real number, which represents the distance or parallax from the optical center of the robot's camera to the corresponding object point in the scene.

[0068] The synchronization and preprocessing unit performs timestamp alignment and spatial calibration on the multi-source data, fusing visual and ontological perception information into a unified input format. The final observation data output by the perception module is as follows:

[0069] ;

[0070] in, Representing the perception vector, for the robot in The global perception state at any given moment.

[0071] S12. Environment Construction and Simulation Platform: To achieve effective acquisition and training of sensor data, the simulation and deployment module constructs a multi-terrain training environment within the simulation platform. Preferably, the simulation platform uses the IsaacGym and Mujoco engines, which support GPU parallel computing and physically consistent dynamic simulations. Various typical terrain models are constructed, including flat land, slopes, steps, gaps, and randomized rugged terrain. During training, parameters such as terrain height field, friction coefficient, and slope are randomized to improve the model's generalization and robustness. The robot model is defined based on the real structure, including parameters such as body inertia, joint damping, and contact friction, ensuring that the simulated motion is consistent with reality. The control frequency is set to 50Hz, and the physical simulation step size is set to 0.001 seconds. To reduce the gap between simulation and reality, this example introduces sensor noise and delay modeling in the simulation: the visual delay is approximately 100ms, and Gaussian noise is introduced at joint positions and joint angular velocities. (The former is noise at the joint position, and the latter is noise at the joint angular velocity), as well as random offsets within ±1° of gravity vector perturbation, which can effectively reduce performance degradation in real-world deployments.

[0072] S13. Perceptual Data Formatting and Normalization: The collected perceptual data is formatted in a unified manner to facilitate subsequent input into the world model module training. The processing steps include: zero-mean normalization of all continuous variables (position, velocity, pose, IMU value) in the perceptual vectors; center cropping and resolution compression (e.g., 64×64) of the depth image; feature extraction from the pre-processed depth image using a convolutional encoder to form a fixed-dimensional latent vector representation; and processing the perceptual vectors... Stored as a time-series dataset for training a recursive state-space model (RSSM).

[0073] To maintain system real-time performance, this invention establishes a perception-control closed-loop mechanism with a control cycle of 20ms. The perception module completes one data acquisition cycle within each control cycle; the perception vector, after formatting and normalization in step S13, is transmitted to the world model module; the world model outputs the predicted state and potential state, which are then passed to the policy network; the control module generates actions and feeds back the execution results, then proceeds to the next control cycle. This structure achieves time synchronization and real-time online updates of visual and ontological data, ensuring the system's response speed to dynamic environmental changes.

[0074] In the actual deployment phase, this invention applies the same perception and acquisition process to a real robot platform (such as the Unitree G1). The vision sensor acquires images of the foreground environment via a Realsense D435i depth camera, while the body sensor samples in real time via a built-in IMU and joint encoders. The sampling frequency is kept consistent with the simulation to ensure consistent model input distribution, thereby achieving deployment from simulation to reality.

[0075] Through the above step S1, the present invention realizes a multimodal perception data acquisition and environment modeling system, which can accurately perceive the dynamic information of the robot and the environment; generate a large number of high-quality training samples through physical consistency simulation; and provide a consistent, rich and scalable input data foundation for the dynamic prediction and policy learning of the world model.

[0076] S2. World Model Construction and Learning: This step is used to construct a world model that can predict environmental dynamics based on the sensory data collected by the robot, realizing the dynamic mapping from high-dimensional sensory data to low-dimensional potential states, and providing a predictable potential space for subsequent policy learning.

[0077] S21. Overall Structure of the World Model: The world model module of this invention includes an encoder, a recurrent unit model, a dynamic predictor, and a decoder. This overall structure is built upon a recursive state-space model (RSSM), which can simultaneously model deterministic hidden states and random latent variables, thereby capturing both predictable dynamics and random perturbations. Considering the computational cost of acquiring depth images in a simulator and the time cost of running on a development board, this invention improves the model by updating the world model at a frequency lower than the policy. Figure 3 As shown, the world model every Update the recursive state once every time step. Formally, the RSSM in this method consists of four components containing φ, where φ represents the set of learnable parameters for each component of the model. These parameters are parameterized by training to learn the dynamic characteristics and visual features of the environment together.

[0078] Circular unit model: ;

[0079] Encoder: ;

[0080] Dynamic predictor: ;

[0081] Decoder: ;

[0082] More specifically, recursive state-space model In low-dimensional potential space Running in the middle, based on the previous Recursive state at each time step Action sequence and the former The random posterior states at each time step To predict the recursive state at the current time step without random sampling. Also known as deterministic recursive state Among them, the former Recursive state at each time step For RSSM in time A deterministic hidden vector is calculated and stored at each step; this value is read from the model's internal cache or memory buffer during runtime as an initial condition for the next step; action sequence. For the system in the time interval The sequence of actions already executed and recorded within the time window originates from the output of the policy module and is used as input to the recursive model during training and inference to reflect the control effects within that time window; [Previous] The random posterior states at each time step In time At any moment, the encoder Based on the perceptual vectors at that time With deterministic recursive state The calculated posterior random variable; the posterior random variable is sampled (or its mean is used) and stored within the model as conditional input for subsequent steps.

[0083] When a deterministic recursive state is obtained Next, RSSM calculates two distributions covering the random states: the prior distribution and the posterior distribution (for the random posterior states). Posterior distribution via encoder Integrated perceptual vectors Information, prior state Aimed at not relying on perceptual vectors Predicting the random posterior state in the following circumstances This enables recursive state-space models to predict future dynamics even in the absence of real observations. Specifically, the encoder... The current perception vector (including depth images) With ontology observation ) and deterministic recursive state After fusion, the distribution parameters of the random posterior state are output, and the random posterior state is obtained by sampling or taking the expectation based on this distribution. The random posterior state injects observed information into the latent representation, thus giving the latent state both historical memory and current observation information. The prior distribution, on the other hand, relies on a dynamic predictor based on deterministic states. Calculate the prior distribution parameters to obtain the perception vector independent of the current observations. The role of the predicted distribution prior state is to perform open-loop prediction when real observations are lacking, so that RSSM has short-term imaginative ability. This ensures that the strategy can predict future dynamics based on historical information and action sequences in the event of lost or delayed observations, which is of great significance for real robots (e.g., camera delay, data loss, or short-term occlusion scenarios).

[0084] The decoder is responsible for generating the estimated perceptual vector. This enables the reconstruction of high-dimensional observations. The recursive state-space model is implemented using a gated recurrent unit (GRU) network, where the encoder and decoder each employ a convolutional neural network (CNN) structure to process the depth image. Multilayer perceptron (MLP) is used to process the ontological observations. .

[0085] S22. Model Training and Optimization: The world model is trained by maximizing the joint likelihood function, and the four components are jointly optimized by minimizing the loss on a trajectory of length L. The optimization objective is:

[0086] ;

[0087] in, It is a non-negative integer. These are hyperparameters. The first term in the formula is the reconstruction loss, used to improve the model's prediction accuracy; it influences the posterior distribution. Includes about The second term, KL divergence, approximates the prior and posterior distributions by regularizing them, thus enabling the approximation of the current information. Open-loop predictions of future observations are made based on future actions, balancing model complexity with potential distribution stability. Training data comes from multimodal sensing sequences collected in step S1, and is iteratively optimized through random mini-batch sampling. The training process can be executed in parallel on a GPU platform to accelerate model convergence.

[0088] S23. Latent State Reasoning and Update: During the model runtime phase, RSSM updates the latent state in real time based on the action sequence, recursive state, and random state. Potential state The update cycle is consistent with the control cycle and serves as the input for the next stage of the policy network (step S3).

[0089] The world model obtained through training using the aforementioned structure can predict the robot's future observations, velocity, and posture changes without privileged information, thereby modeling environmental dynamics in the latent space. Experiments show that this model can effectively improve policy learning speed and sample efficiency, and significantly enhance walking stability on unknown terrain.

[0090] Through the world model construction process in step S2, this invention realizes a high-fidelity prediction model capable of learning environmental dynamics in the latent space, providing stable and generalizable input features for subsequent policy learning. This model possesses high efficiency, transferability, and real-time inference capabilities, which is one of the key innovations of this invention.

[0091] S3. Strategy Learning and Decision Control: This step is used to train a humanoid robot control strategy that can achieve autonomous balance and gait adjustment based on the latent state information output by the world model module under the reinforcement learning framework, thereby achieving stable motion control in complex terrain.

[0092] S31. Overall Structure of the Policy Learning Framework: This step adopts a reinforcement learning framework based on the World Model (WMP), including the following modules: Policy Network (Actor), Value Evaluation Network (Critic), Reward Evaluation Module (RewardModule), and Policy Optimizer (PPOOptimizer). Through these modules, the Policy Network learns from the latent states provided by the World Model. As input, output the robot's joint movements. And by optimizing the cumulative expected reward, a stable control strategy can be obtained.

[0093] S32. Definition of State and Action Space: During policy training, the system state is composed of the latent state of the world model and ontology observations. The output action strategy is .in, The parameterized strategy function (i.e., the Actor network) outputs the target position or velocity command for each joint. The motion is transmitted to the execution module (PD controller) through the control interface to generate torque control signals.

[0094] In training design, ontology observation This includes base angular velocity, gravity projection, commands, joint position and velocity, and the motion at the previous time step. Privileged information This includes the scanning point, foot contact force, and randomized physical parameters. It is an egocentric depth image with a 58°×58° field of view. (Motion-based) Specify target joint location: ,in, This is the default joint position when standing. Torque signal. (i.e., torque output) is calculated by the PD controller:

[0095] ;

[0096] in, and These represent the current position and current velocity of the joint, respectively. and Indicates the target joint position and target joint velocity, target joint velocity Set to 0, and These are the parameters of the PD controller.

[0097] S33. Reinforcement Learning Training Process: This invention uses the Proximal Policy Optimization (PPO) algorithm for training. Its optimization objective is:

[0098] ;

[0099] ;

[0100] in, Represents the policy probability ratio, used to measure the updated policy. Compared to the old strategy In the same state Next action The magnitude of change in selection tendency; The advantage function is used to measure the quality of an action. To reduce the pruning threshold and prevent excessive policy updates, the following approach is adopted. Limit the range of the strategy probability ratio to [ ]between.

[0101] During each training cycle, the policy network samples actions based on the latent state, the world model predicts the next latent state and generates a reward, and the value network estimates the state value. Then, the policy parameters are updated using the above objective function, and the sampling and optimization are repeated until the policy converges.

[0102] S34. Reward Function Design: To guide the strategy to learn natural and stable walking behavior, this invention designs multiple combined reward functions. The reward functions of this invention guide the robot to follow specific speed commands and maintain flexible contact with the ground. To achieve stable walking in the bipedal robot, this embodiment uses the reward functions shown in Table 1. Through a weighted combination of multiple rewards, the strategy can achieve natural and stable walking performance while maintaining energy efficiency.

[0103] Table 1

[0104] Reward function Weight Linear velocity tracking 3.0 Angular velocity tracking 1.2 Z-axis speed -0.8 Y-axis velocity -0.3 XY plane angular velocity -0.2 Aircraft posture -0.5 fuselage base height 1.0 Rate of change of motion -0.008 Foot airtime 1.6 Foot height 0.4 Torque limit -3e-4 Joint acceleration -2.5e-7 Joint speed -1.0e-3 Collision Contact 0.8 Static contact -0.2 Joint posture restrictions 0.8 Gait symmetry 1.6 Alternating gait 1.4 Leg coordination 1.5

[0105] S35. Actor-Critic Architecture Design: As a preferred embodiment of the present invention, policy learning employs an asymmetric Actor-Critic framework. The Actor-Critic framework includes an Actor network and a Critic network. The Actor network receives only the robot's observable latent states and sensory inputs (no privileged information) for direct use in real-world deployments. The Critic network, during simulation training, accesses privileged information (such as ground height and complete dynamic state) to provide accurate value assessment signals. The inputs to the two networks are as follows:

[0106] ;

[0107] ;

[0108] in, This indicates privileged information used only during the simulation phase. This structure can improve training stability and sample utilization while maintaining realistic deployment feasibility.

[0109] S36. Policy Control Mechanism: After training, the policy network and the world model module operate collaboratively: the world model predicts future potential states. Policy network based on Output Action The execution module is based on The system calculates torque and drives the robot; the perception module collects feedback and updates the world model input; thus forming a real-time closed-loop control. This mechanism enables the robot to maintain balance and gait coordination on different terrains (such as slopes, stairs, gaps, etc.) and automatically regain balance when disturbed.

[0110] Verification has shown that the strategy learning scheme of this invention performs excellently in various typical terrain tasks, with an average walking success rate exceeding 90%, a gait symmetry index improvement of 30%, and an energy consumption reduction of approximately 25%. Through step S3, this invention establishes a reinforcement learning control mechanism based on a world model, which can achieve highly stable and efficient strategy training under conditions without privileged information, enabling natural walking and balance control in complex terrain.

[0111] S4. Model Transfer and Real Deployment: The execution module of this invention mainly includes: an action parsing unit, a PD control unit, a motor drive unit, and a status feedback unit. These units are connected via bus communication to form a complete execution feedback loop.

[0112] S41. Action Analysis: The control action output by the policy network is:

[0113] ;

[0114] in, The number of joints in a humanoid robot (e.g., 12 degrees of freedom).

[0115] To obtain the default joint positions of the humanoid robot when it is standing, the target joint positions are obtained by summing the positions of the moving parts and the default joint positions when it is standing. The formula is as follows:

[0116] ;

[0117] in, Indicates the position of the target joint. This is the default joint position when standing. express The action output by the time-based strategy network.

[0118] S42. PD Control Law and Torque Generation: The PD control unit is used to calculate the output torque signal for each joint, as shown in step S32: .

[0119] To improve control stability, this embodiment preferably sets the parameter range as follows: Furthermore, through a terrain-adaptive adjustment mechanism, the gain is dynamically adjusted when the robot walks on different terrains.

[0120] S43. Motor Drive and Execution: The motor drive unit receives the torque signal output from the PD control unit and drives the robot joints to perform actions via servo motors. The experimental platform for this method, the Yushu Robot G1, uses a high-response servo driver with a control frequency reaching 1kHz to ensure dynamic response performance. The execution signals of each joint are transmitted via CAN bus or EtherCAT communication protocol, with a control delay of less than 2ms.

[0121] S44. Feedback Signal Acquisition and Closed-Loop Update: As a preferred embodiment of the present invention, in the actual deployment phase, the operating parameters of the execution and closed-loop control system are as follows: Control host: NVIDIA Jetson NX; Control frequency: 50Hz for policy update, 1kHz for low-level control; Data latency: less than 100ms; Communication protocol: EtherCAT / CAN bus; Control software framework: IsaacSDK real-time thread scheduling. This configuration ensures the synchronization of policy inference and physical execution and meets real-time control requirements.

[0122] S45. System Real-Time Control Flow (Actual Deployment): The closed-loop control cycle of this invention is 20ms, including the following stages: the perception module collects data and updates the potential state of the world model; the policy network outputs action signals. The execution module generates torque through the PD control law and drives the servo motor; the state feedback unit collects the execution results and returns them to the perception module; the world model updates its internal state based on the feedback and enters the next control cycle.

[0123] This process ensures that the robot can respond in real time to terrain changes and external disturbances in complex environments, achieving continuous and smooth motion control. Real-world testing showed that the closed-loop control mechanism enabled the robot to regain balance within 0.4 seconds under random external force disturbances. During a continuous 10-minute test, the system exhibited no cumulative attitude drift, indicating high control accuracy and stability. On unstructured terrain (15° slope, 12cm steps, and obstacles with a 5cm height difference), the robot maintained a stable state, verifying the robustness and practicality of the closed-loop control system of this invention.

[0124] Through the execution and closed-loop control of step S4, this invention establishes a control execution system with high real-time performance and high robustness, which can seamlessly connect reinforcement learning strategies with physical control systems to achieve autonomous and stable movement of robots on complex terrain.

[0125] This step, together with the aforementioned steps S1 to S3, forms a complete closed loop, realizing full-link control from perception to decision-making to execution. It has good versatility and scalability and can be widely applied to bipedal, quadrupedal, and humanoid robot platforms.

[0126] Example 2.

[0127] This embodiment provides a multi-terrain gait control system for a humanoid robot that integrates visual perception, including a perception module, a world model module, a policy network, an execution module, and a simulation and deployment module. Wherein:

[0128] The perception module is used to collect multimodal perception data from the humanoid robot and perform preprocessing.

[0129] World Model Module: This module is used to input pre-processed multimodal sensing data into a pre-trained world model. The world model updates and outputs the potential state at the current time based on historical recursive states, historical random posterior states, and action sequences within historical update intervals.

[0130] The policy network is used to input the potential state at the current time into the pre-trained policy network and output the action at the current time.

[0131] The execution module is used to calculate the driving torque of each joint based on the current action and drive the humanoid robot to perform movements.

[0132] The simulation and deployment module is used to build multi-terrain training environments within the simulation platform.

[0133] For specific limitations regarding the multi-terrain gait control system for humanoid robots that integrates visual perception, please refer to the limitations of the multi-terrain gait control method for humanoid robots that integrates visual perception mentioned above, which will not be repeated here.

[0134] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A multi-terrain gait control method for humanoid robots with fusion visual perception, characterized in that, The fusion visual perception humanoid robot multi-terrain gait control method comprises the following steps of: Collecting multi-modal perception data of the humanoid robot and performing preprocessing; Inputting the preprocessed multi-modal perception data into a pre-trained world model, updating and outputting the latent state at the current time based on the historical recursive state, the historical stochastic posterior state and the action sequence within the historical update interval; Inputting the latent state at the current time into the pre-trained strategy network and outputting the action at the current time; Based on the action at the current time, the driving torque of each joint is calculated through a PD controller to drive the humanoid robot to perform movement.

2. The fused vision-perception humanoid robot multi-terrain gait control method according to claim 1, wherein, The step of collecting multi-modal perception data of the humanoid robot and performing preprocessing comprises the following steps of: Anthropomorphic robot a body observation including joint positions, joint velocities, body pose, linear velocity, and angular velocity, and a depth image of a front terrain Timestamp alignment and spatial calibration are performed on the body observation and depth image to obtain a perception vector; All continuous variables in the perception vector are zero-mean normalized, the depth image is center cropped and resolution compressed, and features are extracted through a convolutional encoder to obtain a latent vector representation with fixed dimensions.

3. The fused vision-perception humanoid robot multi-terrain gait control method of claim 1, wherein, The world model comprises a recurrent cell model, an encoder, a dynamic predictor and a decoder; the world model adopts an update mechanism with an interval of a time step; the update and output of a latent state at a current time point comprise: Circular unit model based on Recursive state at any moment The random posterior state at time t and Time's up The action sequence at each time step is used to calculate the current time step. A deterministic recursive state; The encoder fuses the preprocessed multi-modal perception data at the current time and the deterministic recursive state at the current time to calculate the distribution parameters of the stochastic posterior state at the current time, and samples the stochastic posterior state at the current time based on the distribution parameters. Based on the deterministic recursive state at the current time and the stochastic posterior state at the current time, the latent state at the current time is obtained.

4. The fused vision-perception humanoid robot multi-terrain gait control method of claim 1, wherein, The training process of the world model comprises the following steps of: Using the decoder to reconstruct the observation data based on the latent state to calculate the reconstruction loss; The dynamic predictor predicts the prior distribution at the current time based on the deterministic recursive state at the current time, and calculates the relative entropy divergence between the prior distribution and the stochastic posterior state; The parameters of the world model are optimized by minimizing the joint loss function containing the reconstruction loss and the relative entropy divergence.

5. The fused vision-perception humanoid robot multi-terrain gait control method of claim 1, wherein, The training of the strategy network adopts an Actor-Critic framework based on an asymmetric structure, which comprises the following steps of: The Actor network only receives the observable latent state and perception input of the robot, and outputs the action probability distribution; The Critic network takes the latent state and privileged information provided by the simulation environment as input, and outputs the state value estimate; The privileged information includes terrain geometry information, foot contact force and complete dynamics state without noise, and the privileged information is only used in the training stage.

6. The fused vision-perception humanoid robot multi-terrain gait control method of claim 1, wherein, In the training process of the strategy network, the proximal policy optimization algorithm is used, and the cumulative expected reward is calculated based on the reward function; the reward function comprises a linear velocity tracking term, an angular velocity tracking term, a gait symmetry term and a contact state constraint term.

7. The fused vision-perception humanoid robot multi-terrain gait control method of claim 1, wherein, Based on the action at the current time, the driving torque of each joint is calculated through a PD controller, which comprises the following steps of: The default joint position of the humanoid robot when standing is obtained, the action is superimposed and summed with the default joint position when standing to obtain the target joint position, which is represented by the following formula: ； wherein, represents a target joint position, is a default joint position at standing, represents an action output by the moment policy network; The driving torque of each joint is calculated through a PD controller, which is represented by the following formula: ； wherein, and denote the current joint position and the current joint velocity, and denote the target joint position and the target joint velocity, and are parameters of the PD controller.

8. The fused vision-perception humanoid robot multi-terrain gait control method of claim 1, wherein, Before collecting the multi-modal perception data, a multi-terrain training environment is constructed in a simulation platform, and the steps are as follows: A plurality of terrain models are constructed in a physical simulation engine, and the terrain parameters are randomized; When collecting multi-modal perception data, sensor noise, delay modeling and gravity vector disturbance are introduced.

9. A multi-terrain gait control system for humanoid robots with fusion of visual perception, characterized by, The method comprises a perception module, a world model module, a strategy network, an execution module and a simulation and deployment module, wherein: The perception module is used for collecting multi-modal perception data of the humanoid robot and performing preprocessing; The world model module is used for inputting the preprocessed multi-modal perception data into a pre-trained world model, and the world model updates and outputs a latent state at a current time based on a historical recursive state, a historical random posterior state and a sequence of actions within a historical update interval; The strategy network is used for inputting the latent state at the current time into a pre-trained strategy network and outputting an action at the current time; The execution module is used for calculating driving torque of each joint through a PD controller based on the action at the current time, and driving the humanoid robot to perform movement; The simulation and deployment module is used for constructing a multi-terrain training environment in a simulation platform.

Images