A linear running method of humanoid robot based on AMP gait and hierarchical control

Abstract

The application relates to the field of humanoid robot motion control, and particularly discloses a linear running method for a humanoid robot based on an AMP gait and hierarchical control. The method is used for solving the problems of lateral deviation accumulation, slow drift of a heading angle and lateral sinusoidal oscillation when a humanoid robot G1 performs high-speed linear running on a narrow runway. A hierarchical control structure of high-level slow-loop fuzzy control, a command residual neural network and low-level fast-loop joint perturbation is constructed on the basis of pre-training of an AMP gait strategy. The left and right leg propulsion differences are fine-tuned without destroying the overall gait style, and the heading and lateral deviations are finely corrected. The application can significantly reduce the root mean square of the lateral deviation and the heading deviation during linear running on the G1 humanoid robot without changing the AMP gait network structure and the training process, and improves the stability and controllability of high-speed linear running in the narrow runway environment.

Description

Technical Field

[0001] This invention relates to the field of humanoid robot motion control, specifically a method for humanoid robots to run in a straight line based on AMP gait and hierarchical control. Background Technology

[0002] With the rapid development of artificial intelligence and robotics, the mobility of humanoid robots in complex environments has become a crucial indicator of their intelligence level and application value. Humanoid robots possess highly nonlinear, strongly coupled, and strongly contact-dependent dynamic characteristics, making it difficult for traditional control methods to simultaneously achieve both naturalness and stability in motion. In recent years, gait generation methods based on deep reinforcement learning and imitation learning have made groundbreaking progress, especially adversarial motion prior techniques. These techniques enable robots to learn natural, rich, and stylized walking and running movements in simulated environments, providing a powerful black-box underlying strategy for humanoid robots.

[0003] However, existing motion control technologies based on AMP have significant limitations in specific task scenarios: On the one hand, AMP strategies mainly focus on imitating movement styles. When the task is upgraded to long-term high-speed straight running on a narrow track, due to model uncertainty, contact disturbances, and sensor noise, relying solely on the underlying AMP gait can easily cause the robot's root position to gradually deviate from the track centerline. On the other hand, existing high-level navigation and low-level gait often lack a fine coupling mechanism. By controlling the robot with simple speed commands, it is very easy for the robot to experience lateral deviation accumulation, slow drift of the heading angle, or even obvious lateral sinusoidal oscillations.

[0004] Therefore, researching a control technology that can maintain the natural gait style generated by AMP and achieve high-precision trajectory tracking through a hierarchical mechanism is of great significance for improving the practicality and stability of humanoid robots running in straight lines. Summary of the Invention

[0005] This invention aims to solve the problems of lateral offset accumulation, heading angle drift and lateral sinusoidal oscillation in existing learning-based gait control technology in high-speed running scenarios on narrow runways, and provides a method for humanoid robots to run in a straight line based on AMP gait and hierarchical control.

[0006] This invention solves the above-mentioned technical problems through the following technical solution: a method for a humanoid robot to run in a straight line based on AMP gait and hierarchical control, comprising the following steps:

[0007] S1. Acquire running teaching data of the humanoid robot in the simulation environment, and establish error states including lateral deviation, heading deviation and speed error based on the straight running task;

[0008] S2. Train the AMP gait strategy based on the running teaching data, so that the AMP gait strategy has a command interface for receiving forward velocity commands and yaw angular velocity commands and outputting joint reference movements.

[0009] S3. During the straight-line running process, the high-level fuzzy controller generates the forward velocity initial command and the heading angular velocity initial command according to the error state.

[0010] S4. Generate a command compensation amount based on the error state, the forward velocity initial command, and the heading angular velocity initial command using a command residual neural network, and superimpose the command compensation amount onto the corresponding initial command to obtain the corrected forward velocity command and the corrected heading angular velocity command.

[0011] S5. Input the corrected forward velocity command and the corrected heading angular velocity command into the AMP gait strategy, and the AMP gait strategy outputs the joint reference trajectory;

[0012] S6. Generate amplitude-limited joint perturbations acting on the degrees of freedom of the steering-related leg joints based on the lateral deviation and the heading deviation, and superimpose the amplitude-limited joint perturbations onto the joint reference trajectory to obtain the desired joint trajectory.

[0013] And based on the expected trajectory of the joints, perform joint servo control on the humanoid robot so that the humanoid robot runs in a straight line while maintaining the AMP gait style.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0015] The high-level layer of this invention provides stable empirical control through fuzzy logic, the residual network compensates for nonlinear errors through data-driven processing, and the low-level perturbation enables fine adjustment at the joint level. Experiments show that this method can significantly reduce the root mean square of lateral deviation and heading deviation under various target velocities.

[0016] Unlike traditional forced correction, this invention adjusts the heading by tiny joint asymmetric perturbations while maintaining the natural and anthropomorphic running style generated by AMP, thus avoiding stiff or unnatural movements in the robot. The introduced residual neural network can learn the tiny gap between the command and the actual movement, effectively dealing with model uncertainty and environmental noise. At the same time, since the high-level and low-level layers are coupled only through speed commands, the framework has good interface versatility and is easy to migrate to different humanoid robot platforms. Attached Figure Description

[0017] Figure 1 This is a flowchart of the method of the present invention.

[0018] Figure 2This is a graph showing the lateral deviation membership function of the fuzzy controller of the present invention.

[0019] Figure 3 This is a diagram of the heading deviation membership function of the fuzzy controller of the present invention.

[0020] Figure 4 This is a graph showing the membership function of the speed error of the fuzzy controller of the present invention.

[0021] Figure 5 This is a schematic diagram of the joint mechanism of the humanoid robot in the present invention running in a straight line without perturbations.

[0022] Figure 6 This is a schematic diagram of the joint mechanism of the humanoid robot in this invention when a perturbation is applied and the robot corrects its deviation to the left.

[0023] Figure 7 This is a diagram illustrating the definition of the runway and errors in this invention.

[0024] Figure 8 This is a comparison chart showing the effects of the present invention and traditional methods in a simulation scenario.

[0025] In the diagram: 1. Left hip abduction joint; 2. Right hip abduction joint; 3. Pelvis; 4. Track center line. Detailed Implementation

[0026] The present invention will be further illustrated by way of embodiments below, but the present invention is not limited to the scope of the embodiments.

[0027] Example 1:

[0028] See Figures 1 to 8 A method for linear running of a humanoid robot based on AMP gait and hierarchical control, comprising the following steps:

[0029] S1. Acquire running teaching data of the humanoid robot in the simulation environment, and establish error states including lateral deviation, heading deviation and speed error based on the straight running task;

[0030] AMP teaching data acquisition and straight running task modeling: Focusing on the running task of humanoid robot G1 on a narrow straight track, teaching motion data including joint states, root pose and command variables are collected in the simulation environment, and a straight running error model is established in the track coordinate system.

[0031] S2. Train the AMP gait strategy based on the running teaching data, so that the AMP gait strategy has a command interface for receiving forward velocity commands and yaw angular velocity commands and outputting joint reference movements.

[0032] AMP Gait Policy Training and Command Interface Construction: Based on the teaching data, a G1 gait policy that can stably operate under given forward velocity and heading angular velocity commands is trained using the AMP framework, and the policy explicitly defines [v ref ,ω ref [This refers to the input task command interface;]

[0033] S3. During the straight-line running process, the high-level fuzzy controller generates the forward velocity initial command and the heading angular velocity initial command according to the error state.

[0034] High-level fuzzy controller design and command generation: During the straight-line running task execution phase, using lateral deviation, heading deviation, and velocity error as inputs, a Mamdani-type fuzzy controller is designed to output the forward velocity command v. cmd F (t) and heading angular velocity command ω cmd F (t), the fuzzy controller uses a first sampling period T s The system operates, forming a high-level slow-loop external control loop.

[0035] S4. Generate a command compensation amount based on the error state, the forward velocity initial command, and the heading angular velocity initial command using a command residual neural network, and superimpose the command compensation amount onto the corresponding initial command to obtain the corrected forward velocity command and the corrected heading angular velocity command.

[0036] Command Residual Neural Network Construction and Training: A small command residual neural network is constructed, taking the error state and fuzzy command as inputs, and outputting command compensation Δv. cmd (t), Δω cmd (t), after training, the correction command v is made cmd (t), ω cmd (t) To further reduce lateral and heading errors, the command residual neural network and the fuzzy controller use the same sampling period T. s The processes together constitute a high-level slow loop;

[0037] S5. Input the corrected forward velocity command and the corrected heading angular velocity command into the AMP gait strategy, and the AMP gait strategy outputs the joint reference trajectory;

[0038] Based on the lateral deviation and the heading deviation, amplitude-limited joint perturbations are generated that act on the degrees of freedom of the steering-related leg joints, and the amplitude-limited joint perturbations are superimposed on the joint reference trajectory to obtain the joint desired trajectory.

[0039] And based on the expected trajectory of the joints, perform joint servo control on the humanoid robot so that the humanoid robot runs in a straight line while maintaining the AMP gait style.

[0040] Low-level AMP gait execution and joint perturbation control: A pre-trained AMP gait policy is used as the low-level controller, which receives correction commands [v cmd (t), ω cmd Output the joint reference trajectory q under the condition of (t)]. amp (t), and a few joint perturbations with limited amplitude Δq(t) are superimposed on a few joints related to steering, and G1 is driven by joint PD control to achieve stable straight running, wherein the AMP gait execution and joint PD control are performed with a second sampling period T. f Run, T f For T s It is 1 / 5 to 1 / 20 of the total, thus forming a closed loop in the lower-level fast loop relative to the higher-level slow loop.

[0041] Specifically, the steps are: S1, AMP teaching data acquisition and straight running task modeling. This step mainly completes the construction of the simulation environment, the preprocessing of reference motion data, and the mathematical definition of the straight running evaluation index.

[0042] S11. Simulation Environment Setup and Coordinate System Definition: NVIDIA IsaacGym or MuJoCo is selected as the physical simulation platform, and a high-fidelity URDF model of the humanoid robot G1 is loaded. A straight track with a width of 1.25m and an infinite length is constructed in the simulation environment. A global coordinate system and a local coordinate system are established: the direction of the track centerline extension is set as the positive X-axis of the world coordinate system, the direction perpendicular to the track is the Y-axis, and the direction perpendicular to the ground upwards is the Z-axis. The position and attitude quaternions of the robot's root center in the world coordinate system are obtained in real time through the simulation interface. The attitude is converted into Euler angles to extract the yaw angle ψ(t). The position (x(t), y(t)) and orientation angle ψ(t) of the humanoid robot G1 root on the ground plane are obtained, where y(t) represents the lateral position relative to the track centerline, and ψ(t) represents the orientation angle relative to the track direction.

[0043] S12. Teaching Data Preprocessing: Collect motion capture data including standard straight-line running, starting, stopping, and small turning. Resample the raw data to 60Hz and perform motion redirection, mapping the source skeleton motion to the joint space of the G1 robot. Construct the teaching dataset D. demo Each sample contains s demo =[q,dq,v root ,ω root ], representing joint angle, joint angular velocity, root linear velocity, and angular velocity, respectively;

[0044] Based on the target's forward velocity v ref ,definition:

[0045] Lateral deviation: ,in The distance of the robot's root Y-axis coordinate from the centerline of the runway;

[0046] Heading deviation: This is the robot's actual facing angle. To characterize the nose deviation angle;

[0047] Speed ​​error: ,in This represents the robot's actual forward speed. To set a target running speed.

[0048] S13. In a discrete-time system with a sampling period of Δt, collect error samples at N time points and calculate the root mean square of the lateral deviation: ;

[0049] Root mean square of heading deviation: ;

[0050] Root mean square of speed error: ;

[0051] Where N is the total number of sampling points, and the root mean square is used to quantify the magnitude of deviation throughout the entire process. The smaller the value, the higher the accuracy of straight-line running, which is the performance evaluation index for straight-line running.

[0052] Linear running error model construction: To quantify the control effect, the following three core state variables are defined as the system feedback inputs:

[0053] Lateral deviation e y (t): The Y-axis component at the root position, e y (t) = y(t). This value reflects the distance the robot deviates from the center of the runway.

[0054] Heading deviation e ψ (t): Calculates the difference between the current orientation angle and the runway tangent, e ψ (t) = ψ(t) - 0. The result is restricted to the interval (-π, π].

[0055] Speed ​​error e v (t): Calculates the difference between the actual forward velocity at the root and the target velocity given by the user, e v (t)=v x (t)-v ref .

[0056] S14. Define the robot state vector and action vector for AMP training. The state vector s(t) includes:

[0057] ;

[0058] Where q(t) and dq(t) are the joint angle and joint angular velocity, respectively, and p root (t), R root (t) represents the root position and orientation, v root (t), ω root (t) represents the root linear velocity and angular velocity, c ref (t) is the reference command vector [v ref ,0] T or [v] ref ,ω ref ] T The motion vector a(t) represents the desired position or desired torque of each joint in space.

[0059] S15: In a physical simulation environment, generate multiple segments of straight-line running and basic gait teaching trajectories based on manually designed controllers or keyframe interpolation. Discretize the continuous time trajectory into equally spaced time sequences, and perform resampling, temporal alignment, posture normalization, and noise filtering to obtain the teaching dataset for AMP training. ,in, For discrete sampling times, i represents the i-th sampling point. for The robot's state vector at any given moment. for Always refer to the standard procedures.

[0060] S2 and AMP gait strategy training and command interface construction.

[0061] This step utilizes an adversarial motion prior framework to train the underlying gait policy, enabling it to respond to speed commands and generate natural running motions.

[0062] S21. State and Action Space Design:

[0063] Using teaching dataset D demo To construct a reference function for the imitation reward, the imitation reward r is... imit (t) includes sub-items such as joint angle, joint angular velocity, end position, and root attitude:

[0064] ;

[0065] Where, q ref (t), dq ref (t), p end,ref (t), p root,ref (t), R root,ref (t) represents the reference value in the teaching trajectory, w q wv w e w root k q k v k e k root These are the weights and sensitivity coefficients;

[0066] State space S: Designed as a 48-dimensional vector, containing the joint positions q(t), joint velocities dq(t), root height above ground, projection of the gravity vector onto the body coordinate system, and task commands for the robot's 23 degrees of freedom.

[0067] Motion space A: Defined as the target position offset of each joint, with the output range limited to [-0.5, 0.5] radians.

[0068] S22. Design of Composite Reward Function:

[0069] Constructing style reward r using adversarial motion priors style (t), where the discriminator D(x) takes the state-action sequence x as input and outputs the probability that the sequence comes from the teaching data;

[0070] The discriminator loss is:

[0071] ,in To determine the network, input an action sequence x and output the probability from the human-taught data;

[0072] Style reward is defined as: , The larger the size, the more human-like and natural the AMP's running movements will be;

[0073] To balance the naturalness of the movements with the ability to follow instructions, a total reward function r is constructed that includes task reward, imitation reward, and style reward. total :

[0074] Task reward r task Encourage the robot to maintain the target speed. Use a Gaussian kernel function.

[0075] ;

[0076] in As a smoothing factor, the closer the speed is to the target value, the higher the task reward;

[0077] Imitation reward r imit Calculate the Euclidean distance between the current action and the nearest neighbor teaching frame in terms of joint angle and end effector position to ensure that the basic motion characteristics conform to biological laws.

[0078] Style reward r style: A discriminator network D is introduced. The discriminator receives a sequence of actions and determines whether it comes from the teaching set D. demo It is still generated data from the policy network.

[0079] S23, Strategy Training:

[0080] Imitation reward r imit (t), Style Reward r style (t) and task reward r task (t) The weighted combination yields the total reward:

[0081] ;

[0082] The Proximal Policy Optimization (PPO) algorithm is used to optimize r. total (t) Update AMP gait policy π AMP (a|s,c), where c represents the command vector [v ref ,ω ref ] T The strategy update objective is: ;

[0083] Where A is the advantage function estimate, ε is the pruning coefficient, and r is the reward function, so that the obtained AMP gait policy can run stably under different commands.

[0084] End-to-end training was performed using a near-end strategy optimization algorithm. The robot was trained iteratively 10,000 times in 4096 parallel environments until it could run stably without falling within a speed range of 0.5 m / s to 3.0 m / s.

[0085] S3, Design and command generation of high-level fuzzy controller.

[0086] This step designs a rule-based expert system to translate physical deviations into preliminary speed and steering commands. In this embodiment, the high-level fuzzy controller uses T... s It runs on a period of 20ms.

[0087] S31. Blurring:

[0088] Fuzzy partitioning of input variables:

[0089] Lateral deviation e y The partition is divided into {NB, NS, Z, PS, PB}, and a triangular membership function is used.

[0090] like Figure 2As shown, when the robot deviates from the center line of the track, the fuzzy controller does not directly output a correction command using a fixed formula, but instead makes a graded judgment based on the degree of deviation, thereby generating a smoother control response.

[0091] Heading deviation e ψ The region is divided into {NB, NS, Z, PS, PB}, and the triangular membership function is used.

[0092] like Figure 3 As shown, this invention not only focuses on whether the robot deviates from the centerline, but also on whether the robot's orientation is deviating, thereby suppressing heading drift in advance and avoiding the continuous accumulation of lateral errors.

[0093] Speed ​​error e v : Divide into {NL,NS,Z,PS,PL}.

[0094] like Figure 4 As shown, the high-level control of this invention not only performs lateral correction but also takes into account the target running speed, ensuring that the robot does not significantly sacrifice its straight-line running speed during the correction process. Figures 2-4 The consistent results indicate that the normalized method is consistent.

[0095] S32, Fuzzy Rule Reasoning:

[0096] Define forward velocity command v cmd F The linguistic values ​​for (t) are "significant deceleration", "minor deceleration", "maintain", "minor acceleration", and "significant acceleration", defining the heading angular velocity command ω. cmd F The linguistic values ​​of (t) are "strong left turn", "medium left turn", "weak left turn", "hold", "weak right turn", "medium right turn", and "strong right turn". A corresponding membership function is designed for each linguistic value in the output space. A Mamdani-type fuzzy inference engine is designed, with the rule base following the principle of "position priority, heading assistance". A total of 25 fuzzy control rules are established, covering various deviation and correction scenarios.

[0097] S33, Deblurring:

[0098] A fuzzy rule base is constructed based on the principle of prioritizing position deviation, followed by heading deviation, and then speed error compensation. For example, when e y For PB and e ψ When ω approaches 0, cmd F Take the larger right turn value, when e y Close to 0 and e ψ When ω is PB, cmd F Take the moderate right turn value, when |e y |and|e ψ|Smaller and e v When it is NL, v cmd F Taking the significant acceleration value, the maximum-minimum composition rule and the centroid defuzzification method are used to infer all activation rules, resulting in continuous v. cmd F (t), ω cmd F (t).

[0099] The final output is calculated using the center of gravity method. The output variable is the forward velocity command v. cmd F (t) and heading angular velocity command ω cmd F (t), which serves as the baseline input for subsequent modules.

[0100] S4. Construction and training of command residual neural network.

[0101] This step constructs a lightweight neural network to compensate for the coarseness of fuzzy control and learn the nonlinear error of the system.

[0102] S41. Network Architecture Design:

[0103] Construct a fully connected residual network.

[0104] Construct the input vector for the residual network:

[0105] ;

[0106] Network output vector: ;

[0107] The final high-level order is:

[0108] ;

[0109] ;

[0110] in, Output coarse instructions to the fuzzy controller. / Output compensation amount for the residual network;

[0111] Hidden layers: two layers, 64 neurons per layer, GELU activation function, command residual neural network adopts two fully connected feedforward structure;

[0112] Apply amplitude constraints to the output: |Δv cmd (t)|≤β v ·|v cmd F(t)|、|Δω cmd (t)|≤β ω ·|ωcmd F(t)|, where 0 < β v ,β ω <1;

[0113] Output layer: 2D vector [Δv] cmd (t),Δω cmd The output is connected to the Tanh activation function and multiplied by the scaling factor lambda to limit the compensation range and prevent over-correction.

[0114] S42. Self-supervised data acquisition and training:

[0115] Perform multiple straight-line running experiments on the robot using only a fuzzy controller. Record the command c at each moment. F(t) The robot's actual motion state s at the next moment real (t+1).

[0116] By updating the network weights through backpropagation, the network learns to predict how much additional compensation is needed to make the robot's actual trajectory closer to the theoretical track, given the current error.

[0117] With residual compensation disabled, multiple sets of straight-line running tests were completed using only the fuzzy controller, and the values ​​of (e(t), c) were recorded. F (t),v x (t),ω real (t)), construct the error vector g(t) between the command and the actual motion = [v x (t)-v ref ,ω real (t)-0] T loss function Train the residual network parameters so that the output command compensation Δc(t) can be reduced by J under the same error state, thereby making the actual speed more consistent with the target speed and the actual heading more consistent with the runway direction.

[0118] S5, Low-level AMP gait execution and joint perturbation control.

[0119] like Figure 5 As shown, the underlying AMP strategy can generate natural running motions, but in narrow straight-line tasks, without joint-level fine-tuning, the robot may gradually deviate from the centerline due to contact perturbations, model errors, or heading drift. Figure 6 As shown, when the robot needs to correct its course to the left, how can asymmetric perturbations with limited amplitude be applied to a few steering-related leg joints, such as by adjusting the difference in the abduction / adduction directions of the left and right hip joints, so that the landing position or support reaction force generates a restoring torque pointing towards the center line of the runway?

[0120] During the runtime phase, the AMP gait strategy πAMP Receive current observation o(t) and higher-level command c(t) = [v cmd (t),ω cmd (t)] T Output the joint reference angle vector q AMP (t), the observed o(t) includes the current joint angle q(t), joint angular velocity dq(t), root position and attitude, root linear velocity and angular velocity, and foot contact state; this step is the last loop of the control loop, converting high-level commands into joint torques and introducing perturbations to eliminate heading drift. Low-level joint perturbation control is based on T f With a control cycle of 2ms, it operates much faster than the frequency of higher-level command updates, thus forming a fast loop.

[0121] S51, AMP strategy execution:

[0122] The trained AMP policy network π AMP Receive the current robot state o(t) and the command c(t) after residual correction, and output the reference joint target position q. AMP (t).

[0123] S52, Error-based joint perturbation:

[0124] To address the issue of lateral sinusoidal oscillations in humanoid robots during high-frequency running, perturbation signals are superimposed on the hip and ankle joints.

[0125] Construct a joint perturbation vector Δq(t) that applies only to a few leg joint degrees of freedom closely related to steering, including hip abduction / adduction, hip internal / external rotation, and ankle eversion / inversion. Construct correction coefficients based on the heading deviation eψ(t) and lateral deviation ey(t).

[0126] ;

[0127] in, and To correct the gain, when κ(t)>0, an antisymmetric perturbation is applied to the abduction angles of the left and right hip joints during the support phase, increasing the abduction angle of the right leg and decreasing the abduction angle of the left leg. This results in a restoring torque generated by the landing position and support reaction force, pushing the robot back towards the center line of the track. When κ(t)<0, an antisymmetric perturbation is applied to the abduction angles of the left and right hip joints in opposite directions, ensuring that the direction of the restoring torque always points towards the center line of the track.

[0128] Apply amplitude constraints to all perturbed joint degrees of freedom. , This represents the upper limit of the maximum fine-tuning for a single joint;

[0129] Left hip abduction joint:

[0130] ;

[0131] Right hip abduction joint:

[0132] ;

[0133] in This is for perturbation gain. This antisymmetric perturbation does not change the overall gait rhythm, but it changes the lateral stride of the legs when they land, thus generating a fine corrective torque.

[0134] S53, PD closed-loop control:

[0135] Calculate the final joint target:

[0136] ;

[0137] in, Output the raw reference joint angles for the AMP network. To superimpose a small correction disturbance, it is fed into the joint PD controller to calculate the torque. :

[0138] ;

[0139] Where τ(t) is the joint torque, K p ,K d Let dq be the stiffness and damping coefficients. des (t) can be derived from q des (t) numerical differentiation is obtained or set to 0, by adjusting K. p K d With Δq max This enables the humanoid robot G1 to stably track a straight running trajectory while maintaining the naturalness of the AMP gait. This torque directly drives the motors of the simulated or real robot, achieving stable and high-precision straight-line running.

[0140] like Figure 5 As shown, the four sub-figures illustrate the lateral trajectories of straight-line running under AMP only, AMP + proportional, AMP + fuzzy, and the complete hierarchical control of this invention. The comparison reveals that with AMP only control, the deviation continuously accumulates and deviates from the track; even with proportional or fuzzy control, the trajectory still approaches or crosses the boundary. The complete hierarchical control of this invention ensures that the trajectory consistently fluctuates slightly near the centerline, with a maximum |y| of approximately 0.11m and an average |y| of approximately 0.05m, demonstrating its ability to suppress lateral drift and improve straight-line running stability.

[0141] Example 2:

[0142] In another possible implementation, this method can be deployed in a humanoid robot control system. The robot control system includes a state estimation module, a high-level control module, an AMP gait strategy inference module, a joint perturbation module, and a joint servo module. The state estimation module estimates the root posture, root velocity, joint state, and foot contact state based on feedback information from the inertial measurement unit, joint encoders, foot contact sensors, or motors. The high-level control module calculates lateral deviation, heading deviation, and velocity error based on the runway centerline information and the state estimation results, and outputs commands to correct the forward velocity and heading angular velocity. The AMP gait strategy inference module outputs the joint reference trajectory based on the current robot observations and correction commands. The joint perturbation module generates amplitude-limited joint perturbations based on the lateral and heading deviations. The joint servo module generates joint actuation quantities based on the desired joint trajectory and the actual joint state.

[0143] During physical deployment, input observations can be normalized to align with the observation distribution during the AMP gait strategy training phase. For sensor noise, low-pass filtering or state estimation filtering can be employed. For control commands, safety boundaries can be set, such as limiting maximum forward velocity, maximum directional angular velocity, maximum desired joint angle, maximum desired joint angular velocity, and maximum joint torque. When the robot's lateral deviation, directional deviation, torso tilt angle, or foot slippage exceeds preset thresholds, the target forward velocity can be reduced or a safe stopping strategy can be switched to.

[0144] In the above implementation, when the robot performs straight-line running, the AMP gait strategy provides a natural and coordinated running baseline motion, the high-level fuzzy controller provides interpretable straight-line tracking control, the command residual neural network provides small-amplitude command compensation, and the joint perturbation module provides local correction capability. These modules work together to enable the robot to maintain good straight-line running ability even in the presence of model errors, changes in ground friction, sensor noise, and drive delays.

[0145] The beneficial effects of this embodiment are as follows: by decomposing the straight-line tracking task into command-level correction and joint-level perturbation correction, it is possible to avoid directly and significantly modifying the AMP output trajectory at the joint level; by setting amplitude limiting, filtering and smoothing processing, the safety from simulation to physical deployment can be improved; by preserving the teaching style of the AMP gait strategy, the robot's running movements can be made closer to the natural gait in the teaching data, reducing stiff or uncoordinated joint movements.

[0146] Example 3:

[0147] The difference in this embodiment is that a safe command fusion mechanism is set between the high-level fuzzy controller and the command residual neural network. This mechanism is used to determine whether the forward velocity correction command and the heading angular velocity correction command are within the range that the adversarial motion prior gait strategy can stably respond to, and to decide whether to use all compensation, partial compensation, or turn off compensation based on the compensation confidence of the command residual neural network.

[0148] In the above implementation, the range of commands that can be stably executed is first recorded when training the adversarial motion prior gait strategy. This command range includes the minimum and maximum allowable values ​​for forward velocity commands, the minimum and maximum allowable values ​​for yaw angular velocity commands, and the maximum allowable amplitude of command variation within two adjacent control cycles. This command range is not simply determined according to the robot's mechanical limits, but is statistically obtained based on command samples that allow the gait strategy to maintain a stable running posture, normal foot contact patterns, continuous joint movements, and no significant trunk tilting during training. This forms the policy input safety domain.

[0149] Specifically, during the robot's straight-line running process, the high-level fuzzy controller first outputs initial commands for forward velocity and directional angular velocity based on lateral deviation, heading deviation, and velocity error. Subsequently, the command residual neural network outputs forward velocity compensation, directional angular velocity compensation, and compensation confidence based on the lateral deviation, heading deviation, velocity error, and the two initial commands. The compensation confidence indicates whether the current state is close to the state range covered during the training of the command residual neural network. If the current lateral deviation is small, the heading deviation is small, the velocity error is within a common range, and the initial commands are also within the commonly used control range, the compensation confidence is high; if the robot experiences a large lateral deviation, a large attitude yaw, or a sudden change in velocity error, the compensation confidence decreases.

[0150] In the above implementation, when the compensation confidence level is in the high confidence range, the system uses all the compensation amounts output by the command residual neural network and superimposes these compensation amounts onto the forward velocity initial command and the heading angular velocity initial command, respectively, to obtain the corrected forward velocity command and the corrected heading angular velocity command. When the compensation confidence level is in the medium confidence range, the system does not directly use all the compensation amounts, but instead reduces the compensation amounts according to the confidence level; the closer the confidence level is to the high confidence range, the more compensation amounts are retained, and the closer the confidence level is to the low confidence range, the less compensation amounts are retained. When the compensation confidence level is in the low confidence range, the system disables the compensation output of the command residual neural network and only uses the forward velocity initial command and the heading angular velocity initial command output by the high-level fuzzy controller.

[0151] Furthermore, upon receiving the correction command, the system compares it with the policy input safety domain. If the corrected forward velocity command exceeds the upper speed limit for a stable response by the adversarial motion prior gait policy, it is adjusted to within the upper speed limit; if it is below the lower speed limit, it is adjusted to within the lower speed limit. Similarly, if the corrected heading angular velocity command exceeds the range of stable angular velocities, boundary constraints are applied in the same manner. Simultaneously, the system compares the magnitude of change between the correction command of the current control cycle and the correction command of the previous control cycle; if the magnitude of change exceeds the allowable range, the current command is smoothed and limited according to the allowable magnitude of change to prevent abrupt changes in the input gait policy command.

[0152] In the above implementation, the safety command fusion mechanism does not replace the high-level fuzzy controller or the command residual neural network. Instead, it is located after the outputs of both and before the input of the adversarial motion prior gait strategy. Its function is to unify and coordinate the rule-based commands generated by the high-level controller, the compensatory commands generated by the neural network, and the command range that the gait strategy can withstand. When the robot is in a normal straight-line running state, the command residual neural network can fully exert its compensation function to improve straight-line tracking accuracy. When the robot is in an abnormal deviation or uncertain state, the system can gradually weaken or even shut down the neural network compensation to avoid the neural network outputting excessive compensation outside the training distribution, thereby protecting the stability of the underlying gait.

[0153] This embodiment prevents adversarial motion prior gait strategies from receiving untrained extreme speed or steering commands by limiting the policy input safety domain, thus reducing the risk of abnormal gait, joint oscillations, and instability. By adjusting the participation of the command residual neural network through compensation confidence, the interpretability and safety of the system can be improved while preserving the nonlinear compensation capability of the neural network. By limiting the magnitude of command changes, the impact of high-level command mutations on the joint reference trajectory can be reduced, enabling the robot to move more smoothly along a straight line while maintaining its running style.

[0154] Example 4:

[0155] This embodiment incorporates a gait phase prediction mechanism within the joint perturbation module and applies asymmetric joint perturbations to the left and right legs based on different gait phases. This embodiment addresses the issues of response lag, insufficient foot placement correction, and easy accumulation of lateral deviations when relying solely on speed and yaw rate commands for correction during high-speed straight-line running.

[0156] In the above implementation, the gait phase prediction mechanism determines whether the robot is currently in the initial support phase, middle support phase, takeoff preparation phase, swing phase, or landing preparation phase based on the robot's current foot contact state, the movement state of the left and right hip and knee joints, the vertical movement trend of the root, the foot height, and the gait phase of the previous control cycle. The initial support phase refers to the stage where the foot has just established contact with the ground and begins to bear the robot's weight; the middle support phase refers to the stage where the supporting leg mainly bears the support and propulsion functions of the body; the takeoff preparation phase refers to the stage where the supporting leg is about to leave the ground; the swing phase refers to the stage where the swing leg swings forward but has not yet approached the landing point; and the landing preparation phase refers to the stage where the swing leg is about to touch the ground and determines the next landing position.

[0157] After determining the gait phase, the system generates a correction level based on lateral and heading deviations. The calculation process is as follows: first, determine which side the robot's root projection position deviates from relative to the runway centerline; then, determine which side the robot's root orientation deviates from relative to the runway centerline. When the lateral and heading deviations point in the same direction, it indicates that the robot is not only deviating in position but also in orientation, and the correction level is increased. When the lateral and heading deviations are in opposite directions, it indicates that the robot is trending towards returning to the centerline, and the correction level is decreased. When the lateral deviation is close to zero but the heading deviation still exists, the main focus is on orientation correction. When the heading deviation is close to zero but the lateral deviation still exists, the main focus is on lateral position regression correction.

[0158] Specifically, in the initial support phase, joint perturbations preferentially affect the inversion and eversion degrees of freedom of the supporting leg's ankle joint. If the robot deviates to one side of the runway, a slight correction is made through the supporting leg's ankle joint in the opposite direction of the deviation, so that the lateral component of the foot support reaction force helps the robot return to the runway centerline. The perturbation amplitude in this stage is small and needs to be smoothed to avoid impact when the foot first contacts the ground.

[0159] Furthermore, during the mid-stance phase, joint perturbations preferentially apply to the abduction and adduction degrees of freedom of the supporting leg's hip joint. At this time, the supporting leg has a relatively strong ability to adjust the lateral position of the torso. By applying appropriate perturbations to the supporting hip joint, the lateral positional relationship between the torso and the supporting foot can be changed, causing the robot to gradually tend to return to its normal position. If the robot deviates to the left, perturbations in opposite directions are applied to the corresponding joints of the left and right legs, causing the robot to tend to return to the right; if the robot deviates to the right, perturbations in the opposite direction are used.

[0160] Furthermore, during the mid-swing and landing preparation phases, joint perturbations preferentially affect the internal and external rotation degrees of freedom of the swing leg's hip joint, and can also assist in affecting the flexion and extension degrees of freedom of the swing leg's hip and knee joints. The goal of this stage is not to directly change the current support force, but to adjust the next landing position. The system determines whether the next foot should be slightly closer to the runway centerline or slightly further away from the current deviation side based on lateral and directional deviations. When the robot deviates significantly and still tends to deviate further, the swing leg landing position is significantly corrected in the direction conducive to returning to center; when the robot deviates slightly or has already begun to return to center, the swing leg landing position is only slightly corrected to avoid over-correction causing lateral swaying.

[0161] In the above implementation, the asymmetrical perturbations of the left and right legs do not act simultaneously on all joints in the same way, but are distributed according to the principle that the supporting leg adjusts the direction of force and the swinging leg adjusts the next landing position. The leg in the support phase mainly adjusts the direction of the support reaction force and the lateral posture of the torso through the ankle and hip joints; the leg in the swinging phase mainly adjusts the landing point through internal and external rotation of the hip joint and necessary flexion and extension perturbations. The perturbation directions of the corresponding joints of the left and right legs are opposite, which can form an asymmetrical correction effect while maintaining the overall running rhythm.

[0162] To prevent joint perturbations from disrupting the running style output by the anti-motor prior gait strategy, the system sets upper limits on single-joint amplitude, rate of change, and smoothing filtering for each perturbated joint.

[0163] Specifically, if the perturbation calculated in a certain control cycle exceeds the maximum perturbation amplitude allowed for the corresponding joint, only the maximum value within the allowable range is used; if the perturbation changes too rapidly between two adjacent control cycles, it is gradually transitioned according to the allowable rate of change; if the perturbation direction changes frequently, filtering is used to make the perturbation continuous and smooth. After the above processing, the joint perturbation is superimposed on the joint reference trajectory output by the adversarial motion prior gait strategy to form the final desired joint trajectory.

[0164] The beneficial effects of this embodiment are as follows: by predicting gait phase, the most suitable joint degrees of freedom can be selected for correction at different running stages, avoiding the application of perturbations at inappropriate times; by using asymmetrical perturbations of the left and right legs, the current support force direction and the next foot placement can be corrected simultaneously, improving the lateral correction capability in high-speed straight running; by limiting amplitude, limiting rate of change, and smoothing, the perturbations can be kept within a range that does not disrupt the original running gait style, thereby achieving stable, natural, and highly linear running control for humanoid robots.

[0165] This invention is not limited to the above-described embodiments. Any changes in shape or structure shall fall within the protection scope of this invention. The protection scope of this invention is defined by the appended claims. Those skilled in the art may make various changes or modifications to these embodiments without departing from the principles and essence of this invention, but all such changes and modifications shall fall within the protection scope of this invention.

Claims

1. A method for linear running of a humanoid robot based on AMP gait and hierarchical control, characterized in that, Includes the following steps: S1. Acquire running teaching data of the humanoid robot in the simulation environment, and establish error states including lateral deviation, heading deviation and speed error based on the straight running task; S2. Train the AMP gait strategy based on the running teaching data, so that the AMP gait strategy has a command interface for receiving forward velocity commands and yaw angular velocity commands and outputting joint reference movements. S3. During the straight-line running process, the high-level fuzzy controller generates the forward velocity initial command and the heading angular velocity initial command according to the error state. S4. Generate a command compensation amount based on the error state, the forward velocity initial command, and the heading angular velocity initial command using a command residual neural network, and superimpose the command compensation amount onto the corresponding initial command to obtain the corrected forward velocity command and the corrected heading angular velocity command. S5. Input the corrected forward velocity command and the corrected heading angular velocity command into the AMP gait strategy, and the AMP gait strategy outputs the joint reference trajectory; S6. Generate amplitude-limited joint perturbations acting on the degrees of freedom of the steering-related leg joints based on the lateral deviation and the heading deviation, and superimpose the amplitude-limited joint perturbations onto the joint reference trajectory to obtain the desired joint trajectory. And based on the expected trajectory of the joints, perform joint servo control on the humanoid robot so that the humanoid robot runs in a straight line while maintaining the AMP gait style.

2. The method for linear running of a humanoid robot based on AMP gait and hierarchical control as described in claim 1, characterized in that, The high-level fuzzy controller and the command residual neural network operate at a first sampling period, while the AMP gait strategy and the joint servo control operate at a second sampling period that is less than the first sampling period.

3. The method for linear running of a humanoid robot based on AMP gait and hierarchical control as described in claim 1, characterized in that, In S1, the error state is obtained by determining the root projection position, root orientation angle, and root forward velocity of the humanoid robot in the runway coordinate system; The runway coordinate system has the direction of the runway centerline as the longitudinal axis and the direction perpendicular to the runway centerline as the transverse axis. The lateral deviation is the lateral distance of the root projection position relative to the centerline of the runway. The heading deviation is the angular deviation of the root heading angle relative to the direction of the runway centerline. The velocity error is the difference between the forward velocity at the root and the forward velocity of the target.

4. The method for linear running of a humanoid robot based on AMP gait and hierarchical control according to claim 1, characterized in that, In S2, when training the AMP gait policy, a total reward function is constructed, which includes task reward, imitation reward and style reward. The imitation reward is used to constrain the consistency of joint state, foot movement, and root posture with the running teaching data; The style reward is output by the adversarial discriminator based on the state-action sequence; The task reward is used to constrain forward velocity tracking, attitude stabilization, and straight-line running constraints.

5. The method for linear running of a humanoid robot based on AMP gait and hierarchical control according to claim 4, characterized in that, The AMP gait strategy takes the current robot observations and command vectors as inputs. The command vectors include forward velocity commands and yaw angular velocity commands. The current robot observations include joint states, root motion states, and foot contact states. The AMP gait strategy outputs the reference angle, reference angular velocity, desired torque, or a combination thereof for each joint.

6. The method for linear running of a humanoid robot based on AMP gait and hierarchical control according to claim 1, characterized in that, In S3, the high-level fuzzy controller is a Mamdani-type fuzzy controller. The high-level fuzzy controller fuzzifies the lateral deviation, heading deviation and speed error respectively, and generates the forward speed initial command and the heading angular velocity initial command according to the rule of prioritizing lateral deviation, followed by heading deviation, and then speed error compensation.

7. The method for linear running of a humanoid robot based on AMP gait and hierarchical control according to claim 6, characterized in that, The high-level fuzzy controller divides the lateral deviation and heading deviation into five linguistic variables: negative large, negative small, zero, positive small, and positive large, respectively, and the velocity error into five linguistic variables: negative large, negative small, zero, positive small, and positive large. It outputs continuous forward velocity initial commands and heading angular velocity initial commands through the maximum-minimum synthesis rule and the centroid defuzzification method.

8. The method for linear running of a humanoid robot based on AMP gait and hierarchical control according to claim 1, characterized in that, In S4, the input vector of the command residual neural network includes lateral deviation, heading deviation, velocity error, forward velocity initial command, and heading angular velocity initial command. The output vector of the command residual neural network includes forward velocity compensation and heading angular velocity compensation, and amplitude constraints are applied to the forward velocity compensation and the heading angular velocity compensation respectively.

9. The method for linear running of a humanoid robot based on AMP gait and hierarchical control according to claim 8, characterized in that, The command residual neural network is a feedforward neural network containing at least one fully connected hidden layer. The hidden layer of the feedforward neural network uses a nonlinear activation function. The magnitude constraint includes limiting the forward velocity compensation amount to a preset ratio range of the initial forward velocity command and limiting the heading angular velocity compensation amount to a preset ratio range of the initial heading angular velocity command.

10. The method for linear running of a humanoid robot based on AMP gait and hierarchical control according to claim 1, characterized in that, Steps S3 and S4 run with a first sampling period to form a high-level slow loop, and steps S5, S6 and the joint servo control run with a second sampling period to form a low-level fast loop. The second sampling period is shorter than the first sampling period. Between two adjacent high-level slow loop updates, the forward velocity correction command and the heading angular velocity correction command are maintained or interpolated before being input into the AMP gait strategy.

11. The method for linear running of a humanoid robot based on AMP gait and hierarchical control according to claim 1, characterized in that, In S6, the steering-related leg joint degrees of freedom include at least one of hip joint abduction and adduction degrees of freedom, hip joint internal and external rotation degrees of freedom, and ankle joint inversion and eversion degrees of freedom. The amplitude-limited joint perturbation is generated based on the correction coefficient obtained by the linear combination of lateral deviation and heading deviation, and asymmetric perturbations in opposite directions are applied to the corresponding joint degrees of freedom of the left and right legs.

Images