A method and system for motion control of humanoid robots based on stability constraints

Abstract

This invention provides a method and system for motion control of a humanoid robot based on stability constraints, relating to the field of robot motion control. The method includes: acquiring proprioceptive observation data of the humanoid robot and constructing an augmented confidence state; inputting the augmented confidence state into a policy network to generate joint control commands, and simultaneously inputting it into a neural Lyapunov evaluation network to calculate the Lyapunov energy value of the current state-action pair; calculating the difference between the expected energy and the current energy at the next moment based on exponential stability conditions, and anchoring the current energy to a constant using a gradient separation strategy to construct a constraint violation quantity unaffected by parameter updates; generating a stability penalty term based on the constraint violation quantity to construct stability constraints, jointly constructing a total optimization objective to update the policy network parameters, and generating the optimal joint control commands. This invention integrates neural Lyapunov stability control theory and representation learning to achieve stable motion control of a humanoid robot.

Description

Technical Field

[0001] This invention relates to the field of robot motion control, and specifically to a motion control method and system for humanoid robots based on stability constraints. Background Technology

[0002] Humanoid robots, with their anthropomorphic structure and flexible whole-body dynamics, exhibit unique mobility potential in unstructured environments designed by humans, such as multi-level industrial facilities and disaster relief scenarios. Unlike wheeled platforms limited by flat ground, humanoid robots need to traverse vertically discontinuous terrain areas using discrete support feet. This poses a significant challenge to control strategies in ensuring dynamic stability under high-dimensional underactuated dynamics and unpredictable ground contact.

[0003] In existing technologies, model-based control methods (such as model predictive control and whole-body control) can provide theoretical stability guarantees, but they often rely on simplified dynamic models or rigid contact assumptions. In high-dynamic scenarios involving complex terrain geometry or flexible ground, tracking errors are prone to exceeding the stability margin. While deep reinforcement learning methods can directly learn agile motion behavior from interactive data, standard algorithms optimize by maximizing cumulative reward, lacking formal stability guarantees. They tend to converge to "high-gain" control strategies, causing the system state to operate near the physical stability boundary and often exhibiting divergent oscillations when faced with external disturbances.

[0004] Therefore, existing technologies lack formal guarantees for the dynamic stability of high-dimensional humanoid robots, especially in observable scenarios where proprioception alone cannot capture environmental dynamics. This leads to divergent oscillations or even motion failures in complex discrete terrains. Summary of the Invention

[0005] To address the aforementioned problems, this invention proposes a motion control method and system for humanoid robots based on stability constraints. This method integrates neural Lyapunov stability control theory with representation learning to achieve stable motion control of humanoid robots.

[0006] According to some embodiments, the present invention adopts the following technical solution:

[0007] A motion control method for humanoid robots based on stability constraints, comprising:

[0008] We acquire proprioceptive observation data of a humanoid robot and extract explicit velocity estimates and implicit environmental representations through a hybrid internal model to construct an augmented confidence state.

[0009] The augmented confidence state is input into the policy network to generate joint control commands, and simultaneously input into the neural Lyapunov evaluation network to calculate the Lyapunov energy value of the current state-action pair.

[0010] Based on the exponential stability condition, the difference between the expected energy at the next time step and the current energy is calculated, and the current energy is anchored to a constant through a gradient separation strategy to construct a constraint violation quantity that is not disturbed by parameter updates.

[0011] A stability penalty term is generated based on the constraint violation amount to construct stability constraints. Combined with the PPO pruning objective and Lyapunov fitting error, a total optimization objective is jointly constructed to update the policy network parameters and generate the optimal joint control command.

[0012] According to some embodiments, the present invention adopts the following technical solution:

[0013] A motion control system for a humanoid robot based on stability constraints, comprising:

[0014] The acquisition module is configured to: acquire proprioceptive observation data of the humanoid robot, and extract explicit velocity estimates and implicit environmental representations through a hybrid internal model to construct an augmented confidence state;

[0015] The evaluation module is configured to: input the augmented confidence state into the policy network to generate joint control commands, and simultaneously input the neural Lyapunov evaluation network to calculate the Lyapunov energy value of the current state-action pair;

[0016] The construction module is configured to: calculate the difference between the expected energy at the next time step and the current energy based on the exponential stability condition, and anchor the current energy as a constant through a gradient separation strategy, so as to construct a constraint violation quantity that is not disturbed by parameter updates;

[0017] The generation module is configured to: generate a stability penalty term based on the constraint violation amount to construct stability constraints; combine the PPO pruning objective and the Lyapunov fitting error to jointly construct the overall optimization objective to update the policy network parameters and generate the optimal joint control command.

[0018] According to some embodiments, the present invention adopts the following technical solution:

[0019] A computer program product includes a computer program that, when executed by a processor, implements the aforementioned motion control method for a humanoid robot based on stability constraints.

[0020] According to some embodiments, the present invention adopts the following technical solution:

[0021] A non-transitory computer-readable storage medium is provided for storing computer instructions, which, when executed by a processor, implement the aforementioned motion control method for a humanoid robot based on stability constraints.

[0022] According to some embodiments, the present invention adopts the following technical solution:

[0023] An electronic device includes a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to implement the aforementioned humanoid robot motion control method based on stability constraints.

[0024] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0025] 1. This invention proposes the Lyapunov Enhanced Adaptive Policy (LEAP) framework, which integrates robust representation learning with stability constraints of control theory; by associating the neural Lyapunov evaluation network with the implicit embedding features extracted by the hybrid internal model (HIM), formal stability analysis is extended to high-dimensional humanoid robots in some observable scenarios.

[0026] 2. A gradient separation formula is designed for stability constraint calculation. This technological innovation solves the gradient conflict problem in constraint optimization based on the actor-critic architecture, avoids the evaluation network from falsifying stability estimation results, and ensures that policy updates can truly reduce system energy. Attached Figure Description

[0027] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0028] Figure 1 This is a framework diagram of the Lyapunov Enhanced Adaptive Strategy (LEAP) for Example 1.

[0029] Figure 2 This is a comparison chart of the average reward per round during the training iteration process.

[0030] Figure 3 This is a comparison chart showing the average stability cost per step during the training iteration process. Detailed Implementation

[0031] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0032] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0033] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0034] Example 1

[0035] One embodiment of the present invention provides a motion control method for a humanoid robot based on stability constraints, comprising:

[0036] Step 1: Acquire proprioceptive observation data of the humanoid robot, and extract explicit velocity estimates and implicit environmental representations through a hybrid internal model to construct an augmented confidence state;

[0037] Step 2: Input the augmented confidence state into the policy network to generate joint control commands, and simultaneously input it into the neural Lyapunov evaluation network to calculate the Lyapunov energy value of the current state-action pair;

[0038] Step 3: Based on the exponential stability condition, calculate the difference between the expected energy at the next time step and the current energy, and anchor the current energy as a constant through a gradient separation strategy to construct a constraint violation quantity that is not disturbed by parameter updates;

[0039] Step 4: Generate a stability penalty term based on the constraint violation amount to construct stability constraints. Combine the PPO pruning objective and Lyapunov fitting error to jointly construct the overall optimization objective, update the policy network parameters, and generate the optimal joint control command.

[0040] As an example, deep reinforcement learning (DRL) still faces significant challenges in ensuring the formal stability of high-dimensional humanoid robots moving in complex terrain, especially in partially observable scenarios where proprioception alone cannot capture environmental dynamics. This example proposes Lyapunov Enhancement Adaptive Strategy (LEAP), a novel framework designed specifically for robust vision-free motion.

[0041] This framework integrates control theory and representation learning, utilizing a hybrid internal model (HIM) to extract robust implicit environmental representations. Based on this confidence state, a neural Lyapunov evaluation network is explicitly trained to approximate the attraction domain, and an adaptive Lagrangian mechanism is used to strengthen the exponential stability constraint. Crucially, this embodiment introduces a gradient separation strategy to resolve gradient conflict issues in constrained optimization, ensuring the effectiveness of policy updates. The proposed method effectively suppresses divergent oscillations caused by terrain impacts, expanding the robot's operational range in complex discrete terrains. Furthermore, zero-shot simulation and deployment in real-world environments validate the framework's strong generalization ability in various unmodeled environments. A detailed explanation follows:

[0042] I. Problem Modeling

[0043] The motion of the humanoid robot is modeled as a constrained Markov decision process (CMDP) under partially observable conditions, defined by the tuple (S,O,A,P,r,c,γ), where S∈ The state space contains unobservable terrain parameters, where n represents the dimension of the state space. The agent can only acquire partial observation data. ∈O (proprioception); Action space A Let m represent the target joint position of each actuator's PD controller, where m represents the dimension of the motion space. In this embodiment, m is defined as the joint control command that achieves the target joint position. The transfer dynamics are represented by P: S×A→Δ(S), where Δ(S) is the given current state. and actions At that time, the next state The probability distribution; the reward function r(s,a): S×A→R encodes the task objective (such as tracking the desired speed command); γ∈[0,1) is the discount factor.

[0044] Crucially, to incorporate stability into the learning process, a non-negative stability cost function c(s,a): S×A→C≥0 is introduced. This function quantifies the degree of instability of the system or its deviation from the ideal gait manifold. When the cost c(s,a) approaches 0, the state is considered "stable." In humanoid robot motion scenarios, c(s,a) typically penalizes large tracking errors, excessive torso tilting, and high-frequency joint oscillations—all precursors to robot tipping. The agent's goal is to find a strategy. The goal is to maximize the expected cumulative return while satisfying the stability constraints imposed by the Lyapunov function.

[0045] II. Lyapunov stability conditions

[0046] Lyapunov's concept of stability provides sufficient conditions for the stability of dynamical systems. Based on nonlinear control theory, consider discrete-time autonomous systems. (in If there exists a scalar function L(s): S→R≥0 (called the Lyapunov function) and a positive constant... If for all states s∈S the following two conditions are satisfied, then the equilibrium point (or stable limit cycle) is said to be exponentially stable in the mean-square sense:

[0047] (1)

[0048] (2)

[0049] Condition (1) guarantees that the Lyapunov function L(s) is positive definite and bounded by the stability cost c(s), which means that L(s) can be used as a generalized "energy" measure of the system, and that the energy measure is 0 only when the system is completely stable (c(s)=0).

[0050] Condition (2) is the core energy reduction condition, requiring that the expected energy L(s′) of the system at the next moment must be strictly less than the current energy L(s), and the difference must be proportional to the current instability c(s). This condition ensures that the system state will converge exponentially to a stable equilibrium point or invariant set over time. In motion scenarios with discrete terrains such as stairs, ground impacts will generate state disturbances s→s+δ. Lyapunov constraints ensure that even if such disturbances inject energy into the system, the strategy can still choose an appropriate action a to dissipate the energy, allowing the system to return to a stable manifold.

[0051] III. Lyapunov Enhanced Adaptive Strategy (LEAP)

[0052] like Figure 1 As shown, LEAP is a stability-constrained control framework that maps high-dimensional proprioceptive history information into precise joint control commands. The framework uses augmented confidence states. Among them, the original observation The explicit velocity estimation was supplemented by a hybrid internal model (HIM). and implicit environmental representation .

[0053] (1) Architecture components

[0054] The LEAP core contains three dedicated network branches, each consisting of parameters θ, ψ, and ψ. Parameterization, in shared confidence states Parallel execution:

[0055] Action Network ( The target joint position is output using a multilayer perceptron (MLP) with hidden layers of [512, 256, 128] and an activation function of ELU. ;

[0056] Value Network An MLP with hidden layers of [512,256,128] predicts task-related state value functions, providing support for the updating of the dominant actor-critic;

[0057] Lyapunov Evaluation Networks: Specialized Modules, Through Neural Lyapunov Functions Evaluate the energy dissipation characteristics of the system, i.e., the energy value.

[0058] (2) Reward function design

[0059] The reward function is composed of the weighted sum of the reward items and their corresponding weights in Table 1. The design aims to maximize the agility and gait efficiency of robot movement; following the mainstream design principles of legged robot movement, it employs an exponential kernel function to handle tracking tasks, providing smooth gradient information.

[0060] This embodiment uses the following symbols: subscripts xy represent the horizontal component, z represents the vertical / yaw component; superscripts / subscripts cmd, prev, lim, def, and target represent the command value, historical value, limit value, default value, and target value, respectively; symbols These represent the base linear velocity, angular velocity, height, attitude angle, joint torque, position, velocity, and acceleration, respectively. Time for the feet to lift off the ground Here, σ is the indicator function, σ is the kernel bandwidth, body_collision represents the robot's unexpected torso collision event, and contact represents the effective contact event between the foot and the ground.

[0061] Table 1 Reward Functions and Corresponding Weights

[0062]

[0063] (3) Stability cost design

[0064] reward function While it can incentivize the robot to achieve its speed target, it cannot fundamentally guarantee that it will dissipate impact energy on irregular surfaces; therefore, a stability cost c( ), as a substitute indicator for the dynamic instability of the system; according to the definition of the Lyapunov stability condition, c( The penalty for a state deviating from a stable equilibrium manifold is expressed by the following formula:

[0065] (3)

[0066] in, This represents the current horizontal velocity of the base. The horizontal linear velocity represents the desired (target) velocity. This represents the current yaw rate (Z-axis angular velocity) of the base. The yaw rate represents the desired (target) angular velocity. For the roll angle and pitch angle of the torso, For mechanical power, For joint torque, For joint velocity, These represent fixed weighting coefficients for velocity tracking error, yaw rate tracking error, attitude (torso tilt), and mechanical power penalty terms, respectively. This composite cost function provides a multi-dimensional measure of motion safety.

[0067] 1) For the tracking error part: and The quadratic penalty can strongly constrain the loss of control (a common precursor to robot tipping over);

[0068] 2) For the posture adjustment section: For high-degree-of-freedom humanoid robots, posture is the foundation of balance. Excessive trunk tilt is penalized to prevent the center of mass from exceeding the supporting polygon;

[0069] 3) For oscillation suppression: mechanical power term Suppress high-frequency jitter and "brute-force" balancing behavior to ensure that the learned stability domain is suitable for hardware implementation.

[0070] This dual-objective design ensures that the strategy achieves terrain traversal not only through reward-driven agility but also through cost-driven proactive energy management, laying the foundation for the subsequent Lyapunov analysis.

[0071] The following explanation details the implementation of LEAP, based on an overview of the LEAP framework:

[0072] 3.1 Constructing Augmented Confidence States

[0073] To address the partial observability issue in visually unobservable motion, a hybrid internal model (HIM) is employed to synthesize robust confidence states. HIM processes short-term proprioceptive windows (H=5) to capture high-frequency contact dynamics and terrain-induced disturbances.

[0074] The HIM architecture maps historical observations of time series into two complementary branches:

[0075] 1) Explicit branch: Regression base linear velocity This provides a reliable reference for command tracking and compensates for drift in the inertial measurement unit (IMU);

[0076] 2) Implicit Branching: Extracting implicit environmental representation vectors through contrastive learning. Implicitly encodes unobservable environmental properties such as friction coefficient and step height.

[0077] HIM is optimized in a simulation environment using multi-objective loss:

[0078] (4)

[0079] in, These are the weighting coefficients of the mean squared error loss term in velocity regression. It is an estimate of the base linear velocity predicted by the model. To obtain the actual speed of the base from the simulator, A prototype-based Sinkhorn clustering mechanism is used to cluster trajectories with similar dynamic characteristics; where K is the preset total number of cluster prototypes. The predicted probability of an implicit environment representation vector belonging to the k-th cluster prototype is calculated using the Softmax function: In this formula Let the cluster center be the k-th cluster prototype. These are hyperparameters used to adjust the smoothness of the predicted probability distribution. Assign a target probability (or pseudo-label) to the current implicit environment representation to belong to the k-th cluster prototype. This target assignment probability is used to indicate the current implicit environment representation. The ideal cluster category to which it should belong; this ensures the augmented state. It can provide the necessary contextual information for Lyapunov evaluation networks, enabling them to distinguish between external disturbances and internal control errors, thereby achieving effective stability management.

[0080] 3.2 Lyapunov-enhanced policy optimization

[0081] LEAP's core contribution is a constrained optimization framework that strengthens stability constraints based on control theory. It utilizes the confidence states output by HIM. We construct neural Lyapunov functions to explicitly regulate the energy dissipation process of the system.

[0082] (1) Lyapunov's evaluation network

[0083] In traditional Lyapunov deterministic conditions, L(s) is a function that depends only on the state. However, in order to enable the Lyapunov evaluator to not only be used for stability analysis but also to directly guide the learning and updating of the policy network (action network), this embodiment designs it as a non-negative function that depends on both the state s and the action a. Therefore, L(s) and The following expected relationship exists between them:

[0084]

[0085] Lyapunov function The parameterization is performed as a dedicated network branch. To avoid feature interference caused by maximizing task rewards, a decoupled architecture is adopted: the Lyapunov evaluation network is an independent MLP with hidden layers of [256, 256]. To strictly satisfy the nonnegativity condition (L>0), the output layer uses squaring to calculate the Lyapunov energy value of the state-action pair, expressed by the formula:

[0086] (5)

[0087] in, This is the feature vector of the penultimate layer. The network approximates the cumulative stability cost of infinite time through temporal difference (TD) learning, with the fitting error (denoted as...)... ) is defined as:

[0088] (6)

[0089] Among them, the target value Utilize the current strategy to bootstrap and ensure It can evaluate the stability of the manifold of the current effective policy. In the formula, is the aforementioned discount factor; d is the round termination indicator variable (d=1 when the termination state is reached, otherwise d=0); s' is the state at the next moment; The action command output by the action network in the next time step, state s'. The parameters of the target network in the Lyapunov evaluation network; in formula (6) This represents the experience replay pool (or sample data distribution) used for training.

[0090] (2) Overall optimization objective

[0091] Unlike standard unconstrained reinforcement learning, LEAP directly incorporates stability requirements into the training objective; following the PPO algorithm, the pruned alternative objective is denoted as... The regression loss of the value function is denoted as To incorporate the Lyapunov stability conditions, we first define a theoretical stability constraint violation quantity Δ, which represents the energy drift of the system:

[0092] (7)

[0093] in, To regulate the energy decay rate. Therefore, the overall optimization objective is... It is constructed as a joint minimization problem:

[0094] (8)

[0095] in, For the PPO item, in this item As a substitute target for PPO tailoring, The regression loss is the value function. As a stability penalty item, For the positive part of the constraint violation (i.e., when When >0, the value is taken as ,when When the value is 0), For adaptive Lagrange multipliers, are the weights of the stability penalty term, which is used to suppress policy updates that violate the energy diminishing condition. Let be the fitting error term, representing the temporal difference fitting error of the Lyapunov evaluation network. However, directly optimizing equation (8) in high-dimensional space will cause gradient conflict. That is, when extending the Lyapunov constraint to a high-dimensional humanoid robot system, the Lyapunov value inflation problem will be encountered. In order to solve this instability problem of high-dimensional evaluation networks, a stability penalty term is considered for the evaluation network parameters. gradient:

[0096] (9)

[0097] Without intervention, the optimizer will artificially raise the baseline energy. Rather than actually reducing future energy To minimize the objective function. Although the pruning mechanism of PPO does not necessarily lead to numerical divergence, it introduces a significant estimation bias—this bias decouples the estimated “energy” from the actual physical stability of the system, causing the policy to converge to a suboptimal equilibrium point, which manifests as gait conservatism and higher steady-state cost.

[0098] To address this problem theoretically, the current energy L(s) is treated as a fixed anchor point during policy updates. A gradient separation strategy is introduced, and the constraint violation amount used in training is redefined. That is, the constraint violation amount that is not affected by parameter updates:

[0099] (10)

[0100] Among them, sg( ) indicates stopping the gradient operator.

[0101] In practical implementation, using Replacing Δ in the overall objective (8) ensures Only depend on This forces the strategy to actively seek out state transition processes that dissipate energy, thereby ensuring the existence of the effective field of attraction (ROA).

[0102] The adaptive Lagrange multiplier λ is used as a dynamic penalty weight and updated via dual gradient ascent:

[0103] (11)

[0104] in, and and represent the adaptive Lagrange multipliers at the k-th and k+1-th iterations, respectively, serving as dynamic penalty weights for stability constraints. It is the update step size (learning rate) of the dual gradient ascent algorithm. It is the training constraint violation quantity The expected value of the mathematical value, It is the maximum allowed value for Lagrange multipliers. This is a numerical pruning function used to ensure that the updated Lagrange multipliers are always constrained to the specified range. Within the non-negative valid interval.

[0105] This allows λ to adaptively increase during dangerous transitions such as those at the edge of stairs (Δ>0), while relaxing during stable limit cycle walking, thus effectively balancing the robot's agility and formal stability.

[0106] The performance of the Noetix N2 humanoid robot was verified based on the Noetix N2 humanoid robot, which is 118cm tall, weighs 30kg, and has 18 active degrees of freedom. The simulation was run at 500Hz to capture high-frequency contact dynamics, and the control strategy was run at 50Hz. The reward items and their corresponding weights are detailed in Table 1.

[0107] To verify the LEAP framework and analyze the contributions of techniques such as state estimation and gradient separation stability constraints, this embodiment designs the following four algorithm configurations for comparison:

[0108] Proprio-MLP: A naive baseline algorithm that uses a simple MLP architecture to process stacked ontological receptive observations. It does not have explicit representation learning or stability guarantees and serves as a performance lower bound.

[0109] Unconstrained HIM: The current best baseline algorithm, which uses the original HIM estimator. It is an unconstrained and highly agile strategy that only aims to maximize task rewards. It has no explicit stability control and is prone to high-energy oscillations.

[0110] LEAP w / o Detach: An ablation version of LEAP that removes the gradient separation strategy in stability constraint calculations, directly verifying the theoretical assumptions of Lyapunov value inflation and optimization stability;

[0111] LEAP (the method in this embodiment): a complete framework that integrates HIM-based confidence states, adaptive Lyapunov stability constraints, and gradient separation strategies.

[0112] All configurations were trained and evaluated using the same terrain course to ensure fairness in the comparison; the experiment recorded the training process of task rewards and stability costs over 10,000 iterations, yielding results such as... Figure 2 and Figure 3 The comparison results shown reveal three key conclusions regarding the interaction between perception, stability, and optimization dynamics:

[0113] (1) Perception determines the upper limit of performance: As shown in Figure 2, there is a significant performance gap between the HIM-based method and the naive baseline algorithm. The reward value of Proprio-MLP enters the plateau period early (about 40) and has a high stability cost (about 0.65). This proves that for the visual-free motion of high-dimensional humanoid robots, the original proprioceptive information is insufficient to capture environmental dynamics. The implicit confidence state provided by HIM is a prerequisite for learning agile motion behavior.

[0114] (2) Stability constraints ensure motion safety: Both Unconstrained HIM and LEAP can achieve high reward values ​​(about 60), but LEAP has a significantly lower stability cost (about 0.43 vs 0.50); ​​this proves that Lyapunov constraints can effectively eliminate dangerous behaviors and guide the strategy to converge to a control manifold that is equally agile but physically more stable.

[0115] (3) Gradient separation is the key to optimization: Ablation experiments on the gradient separation strategy verified the theoretical assumption about ill-conditioned value inflation. LEAP w / o Detach could not converge to the optimal performance, as shown in Figure 3. Its stability cost was always high (about 0.50), which was comparable to the unconstrained baseline algorithm, and the reward value was suboptimal (about 55). The reason is that if the gradient of L(s) is not separated, the optimizer can easily satisfy the constraint by artificially increasing the current Lyapunov value, rather than reducing the future energy by improving the strategy. This ill-conditioned optimization path hinders the action network from learning the truly stable behavior, proving that gradient separation is crucial to the effectiveness of reinforcement learning based on Lyapunov constraints.

[0116] In summary, the LEAP framework proposed in this embodiment integrates HIM-based representation learning with neural Lyapunov stability to achieve stable motion control of humanoid robots. By introducing a gradient separation strategy and an adaptive Lagrangian relaxation method, it successfully achieves stable optimization of high-dimensional constraint strategies and effectively suppresses divergent oscillations. Extensive experimental results show that LEAP significantly expands the robot's operational range—enabling visual crossing of 14cm high stairs and resisting extreme external impacts through active energy dissipation. Zero-shot deployment on the N2 robot verifies the framework's physical rationality and robust generalization ability in various unstructured terrains.

[0117] Example 2

[0118] One embodiment of the present invention provides a humanoid robot motion control system based on stability constraints, comprising:

[0119] The acquisition module is configured to: acquire proprioceptive observation data of the humanoid robot, and extract explicit velocity estimates and implicit environmental representations through a hybrid internal model to construct an augmented confidence state;

[0120] The evaluation module is configured to: input the augmented confidence state into the policy network to generate joint control commands, and simultaneously input the neural Lyapunov evaluation network to calculate the Lyapunov energy value of the current state-action pair;

[0121] The construction module is configured to: calculate the difference between the expected energy at the next time step and the current energy based on the exponential stability condition, and anchor the current energy as a constant through a gradient separation strategy, so as to construct a constraint violation quantity that is not disturbed by parameter updates;

[0122] The generation module is configured to: generate a stability penalty term based on the constraint violation amount to construct stability constraints; combine the PPO pruning objective and the Lyapunov fitting error to jointly construct the overall optimization objective to update the policy network parameters and generate the optimal joint control command.

[0123] Example 3

[0124] One embodiment of the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the aforementioned motion control method for a humanoid robot based on stability constraints.

[0125] Example 4

[0126] In one embodiment of the present invention, a non-transitory computer-readable storage medium is provided for storing computer instructions. When the computer instructions are executed by a processor, they implement the aforementioned motion control method for a humanoid robot based on stability constraints.

[0127] Example 5

[0128] One embodiment of the present invention provides an electronic device, including: a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to implement the aforementioned humanoid robot motion control method based on stability constraints.

[0129] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0130] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0131] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A motion control method for a humanoid robot based on stability constraints, characterized in that, include: We acquire proprioceptive observation data of a humanoid robot and extract explicit velocity estimates and implicit environmental representations through a hybrid internal model to construct an augmented confidence state. The augmented confidence state is input into the policy network to generate joint control commands, and simultaneously input into the neural Lyapunov evaluation network to calculate the Lyapunov energy value of the current state-action pair. Based on the exponential stability condition, the difference between the expected energy at the next time step and the current energy is calculated, and the current energy is anchored to a constant through a gradient separation strategy to construct a constraint violation quantity that is not disturbed by parameter updates. A stability penalty term is generated based on the constraint violation amount to construct stability constraints. Combined with the PPO pruning objective and Lyapunov fitting error, a total optimization objective is jointly constructed to update the policy network parameters and generate the optimal joint control command. The overall optimization objective is expressed by the formula: in, As a substitute target for PPO tailoring, For the value function regression loss, For adaptive Lagrange multipliers, To constrain the positive part of the violation function, The temporal difference fitting error of the Lyapunov evaluation network is used.

2. The motion control method for a humanoid robot based on stability constraints as described in claim 1, characterized in that, The hybrid internal model comprises two complementary branches: an explicit branch for regressing the base linear velocity and an implicit branch for extracting implicit environmental representation vectors through contrastive learning. The model is optimized using a multi-objective loss function that includes a mean squared error term for velocity regression and a contrastive learning term based on prototype Sinkhorn clustering.

3. The motion control method for a humanoid robot based on stability constraints as described in claim 1, characterized in that, The neural Lyapunov evaluation network adopts a decoupled architecture. Its output layer ensures non-negativity through squaring operations and approximates the cumulative stability cost in the infinite time domain through temporal difference.

4. The motion control method for a humanoid robot based on stability constraints as described in claim 1, characterized in that, The exponential stability condition requires that the difference between the expected energy at the next moment and the current energy be proportional to the current instability cost, and that the proportionality coefficient be a positive constant.

5. The motion control method for a humanoid robot based on stability constraints as described in claim 1, characterized in that, The gradient separation strategy applies a stopping gradient operator to the current energy term when calculating the constraint violation, so that the gradient of the stability penalty term depends only on the expected energy at the next time step, forcing the policy network to learn a control manifold with energy dissipation characteristics.

6. A motion control system for a humanoid robot based on stability constraints, characterized in that, The method for motion control of a humanoid robot based on stability constraints as described in any one of claims 1-5 includes: The acquisition module is configured to: acquire proprioceptive observation data of the humanoid robot, and extract explicit velocity estimates and implicit environmental representations through a hybrid internal model to construct an augmented confidence state; The evaluation module is configured to: input the augmented confidence state into the policy network to generate joint control commands, and simultaneously input the neural Lyapunov evaluation network to calculate the Lyapunov energy value of the current state-action pair; The construction module is configured to: calculate the difference between the expected energy at the next time step and the current energy based on the exponential stability condition, and anchor the current energy as a constant through a gradient separation strategy, so as to construct a constraint violation quantity that is not disturbed by parameter updates; The generation module is configured to: generate a stability penalty term based on the constraint violation amount to construct stability constraints; combine the PPO pruning objective and the Lyapunov fitting error to jointly construct the overall optimization objective to update the policy network parameters and generate the optimal joint control command.

7. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the humanoid robot motion control method based on stability constraints as described in any one of claims 1-5.

8. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium is used to store computer instructions, which, when executed by a processor, implement a humanoid robot motion control method based on stability constraints as described in any one of claims 1-5.

9. An electronic device, characterized in that, include: The device includes a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to implement a humanoid robot motion control method based on stability constraints as described in any one of claims 1-5.

Images