A method for optimizing motion style weights in humanoid robot gait training

By calculating speed tracking error and motion style similarity, and dynamically adjusting training weights, the problems of poor adaptability and imbalance of optimization objectives caused by fixed weights in AMP training are solved, thus achieving adaptive optimization and accuracy improvement in robot gait training.

CN123103642APending Publication Date: 2026-11-17LEJUTONGYAN (BEIJING) ROBOT TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611296771.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-25
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

In existing AMP training methods, fixed weights cannot be adaptively adjusted, resulting in poor adaptability of robot gait training, imbalance of optimization objectives, high parameter tuning costs, and inability to dynamically adapt to the training state.

Method used

By calculating speed tracking error and motion style similarity based on historical data, performing inverse transformation and normalization, and combining logarithmic weighted summation and performance scoring optimization, the training weights are dynamically adjusted to achieve adaptive optimization.

Benefits of technology

This study achieves adaptive optimization of weights during robot gait training, improving training adaptability, balancing speed tracking accuracy and motion style similarity, and reducing parameter tuning costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123103642A_ABST
    Figure CN123103642A_ABST
Patent Text Reader

Abstract

This application discloses a method for optimizing motion style weights in gait training for humanoid robots. The method includes: determining the current speed tracking error based on historical speed time-series data and historical speed command time-series data of the target humanoid robot, performing an inverse transformation and normalization to obtain a speed tracking quality score; determining the current motion style similarity to a standard motion reference trajectory based on historical motion feature time-series data of the target humanoid robot, performing normalization to obtain a motion style similarity quality score; performing a logarithmic weighted sum of the speed tracking quality score and the motion style similarity quality score based on the current training weights to obtain a current gait performance score; optimizing the current training weights based on the current gait performance score and the historical performance baseline of the target humanoid robot to obtain target training weights, and using the target training weights for gait training. This scheme achieves adaptive optimization of gait training weights.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of robotics, and more particularly to the field of humanoid robot motion control, specifically to a method for optimizing motion style weights in humanoid robot gait training. Background Technology

[0002] Adversarial Motion Prior (AMP) algorithm is widely used in humanoid robot anthropomorphic style reinforcement learning training. This method can quickly learn highly humanoid gait and also has strong task tracking capabilities.

[0003] Current AMP training generally uses manually set fixed weights, which lacks an adaptive adjustment mechanism and has many technical defects. For example, manually tuned fixed weights are only suitable for specific robots, reference actions and motion conditions. After changing training data, speed ranges or task configurations, the fixed weights cannot adapt to new training scenarios. When the fixed weight values ​​are too large, the model overfits the action style, and the speed tracking accuracy decreases. When the values ​​are too small, the gait style constraints are insufficient, and the motion quality deteriorates. Existing technologies do not have automatic balancing adjustment methods. Summary of the Invention

[0004] This application provides a method for optimizing motion style weights in humanoid robot gait training to achieve adaptive weight optimization during the humanoid robot gait training process.

[0005] According to one aspect of this application, a method for optimizing motion style weights in humanoid robot gait training is provided, the method comprising: Based on the historical speed time series data and historical speed command time series data of the target humanoid robot during the historical gait training period, the current speed tracking error of the target humanoid robot is determined, and the current speed tracking error is subjected to inverse transformation and exponential normalization to obtain the speed tracking quality score of the target humanoid robot. Based on the historical motion feature time series data of the target humanoid robot during the historical gait training period, the current motion style similarity between the target humanoid robot and the standard motion reference trajectory is determined, and the current motion style similarity is subjected to nonlinear normalization processing to obtain the motion style similarity quality score of the target humanoid robot. Based on the current training weights of the target humanoid robot, the speed tracking quality score and the motion style similarity quality score are logarithmically weighted and summed to obtain the current gait performance score of the target humanoid robot. Based on the current gait performance score and the historical performance baseline of the target humanoid robot in the previous training iteration cycle, the weight of the motion style similarity quality score in the current training weight is optimized to obtain the target training weight of the motion style similarity quality score, and the target training weight is used to train the gait of the target humanoid robot.

[0006] This application determines the current speed tracking error of the target humanoid robot based on historical speed time-series data and historical speed command time-series data during the historical gait training period. After inverse transformation and exponential normalization, the current speed tracking error is obtained to achieve a speed tracking quality score. Based on historical motion characteristic time-series data during the historical gait training period, the current motion style similarity between the target humanoid robot and the standard motion reference trajectory is determined. This similarity is then non-linearly normalized to obtain a motion style similarity quality score. Based on the current training weights, the speed tracking quality score and the motion style similarity quality score are logarithmically weighted and summed to obtain the current gait performance score. Based on the current gait performance score and the historical performance baseline of the target humanoid robot in the previous training iteration, the weights of the motion style similarity quality score in the current training weights are optimized to obtain target training weights for the motion style similarity quality score. These target training weights are then used to train the gait of the target humanoid robot. The above technical solution relies on the speed command tracking performance and motion style similarity index collected in real time during the training process to adaptively balance the robot's speed tracking accuracy and the similarity of the reference motion style. This achieves adaptive optimization of weights during the humanoid robot gait training process, solving the technical problems of poor training adaptability, imbalance of optimization targets, high parameter tuning costs, and inability to dynamically adapt to the training state caused by fixed weights. Attached Figure Description

[0007] Figure 1 This is a flowchart of a motion style weight optimization method for humanoid robot gait training according to Embodiment 1 of this application; Figure 2 This is a flowchart of a motion style weight optimization method for humanoid robot gait training according to Embodiment 2 of this application; Detailed Implementation

[0008] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0009] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0010] Furthermore, it should be noted that the collection, storage, use, processing, transmission, provision, and disclosure of historical gait training time periods and historical speed time series data involved in the technical solution of this application all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0011] Example 1 Figure 1 This is a flowchart of a motion style weight optimization method for humanoid robot gait training according to Embodiment 1 of this application. This embodiment is applicable to optimizing the training weights of a humanoid robot during gait training. It can be executed by a motion style weight optimization device for humanoid robot gait training. This device can be implemented in hardware and / or software and can be configured in a computer device, such as a server. Figure 1 As shown, the method includes: S110. Based on the historical speed time series data and historical speed command time series data of the target humanoid robot during the historical gait training period, determine the current speed tracking error of the target humanoid robot, and perform exponential normalization on the current speed tracking error after inverse transformation to obtain the speed tracking quality score of the target humanoid robot.

[0012] The target humanoid robot refers to the bipedal robot currently undergoing gait training. The historical gait training period refers to the complete time window during which the humanoid robot performed gait training and optimization tasks before the current weight optimization moment; it serves as the basis for extracting historical speed-related data. Historical speed time-series data refers to the sequence of actual motion speeds collected by the robot's sensors (such as joint encoders and inertial measurement units) in chronological order within the historical gait training period, including components such as the robot's actual output linear velocity and angular velocity. Historical speed command time-series data refers to the sequence of target speed commands issued by the robot's upper-level motion planning module in chronological order within the historical gait training period; it represents the expected input value for the robot's speed tracking task. The current speed tracking error refers to the deviation between the actual output speed (taken from the latest segment of historical speed time-series data) and the speed command (taken from the latest segment of historical speed command time-series data) at the corresponding moment in the latest gait execution process; it characterizes the degree of deviation in the robot's following of speed commands under the current gait. The speed tracking quality score is a quantitative indicator obtained after the entire process of speed tracking error calculation, inverse transformation, and exponential normalization. It is used to characterize the quality of speed tracking task completion of the target humanoid robot in its current gait. The higher the score, the better the speed tracking accuracy.

[0013] Specifically, based on the historical velocity timing data and historical velocity command timing data of the target humanoid robot during the historical gait training period, the current linear velocity tracking error and the current angular velocity tracking error of the target humanoid robot are determined. The current linear velocity tracking error and the current angular velocity tracking error are then weighted and summed to obtain the current velocity tracking error. After inverse transformation and exponential normalization, the velocity tracking quality score of the target humanoid robot is obtained.

[0014] For example, the current linear velocity tracking error can be determined by the following formula: ; in, This refers to the current linear velocity tracking error. E[] refers to the mathematical expectation operator. Because the velocity tracking error is subject to randomness due to sensor noise, ground friction fluctuations, gait random disturbances, etc., this operator is used to characterize the long-term / statistical average tracking deviation, rather than the instantaneous error of a single sampling. This refers to the value of the linear velocity at the corresponding moment in the historical speed command timing data. This refers to the value of linear velocity at the corresponding moment in historical velocity time series data. It refers to the calibration normalization coefficient for linear velocity tracking, which is an inherent parameter obtained in advance through robot calibration experiments.

[0015] For example, the current angular velocity tracking error can be determined by the following formula: ; in, This refers to the current angular velocity tracking error. E[] refers to the mathematical expectation operator. It refers to the value of the angular velocity at the corresponding moment in the historical speed command timing data. It refers to the value of angular velocity at the corresponding moment in historical velocity time series data. It refers to the calibration normalization coefficient for angular velocity tracking, which is an inherent parameter obtained in advance through robot calibration experiments.

[0016] For example, the speed tracking quality score can be determined by the following formula: ; in, This refers to the speed tracking quality score. It refers to the exponential function, which is used to map non-negative error values ​​to a fixed interval of [0,1]. This refers to the current speed tracking error.

[0017] S120. Based on the historical motion characteristics time series data of the target humanoid robot during the historical gait training period, determine the current motion style similarity between the target humanoid robot and the standard motion reference trajectory, and perform nonlinear normalization on the current motion style similarity to obtain the motion style similarity quality score of the target humanoid robot.

[0018] Historical motion feature time-series data refers to a sequence of multi-dimensional motion parameters with timestamps extracted from historical gait training periods at a fixed sampling rate. This typically includes parameters describing the robot's motion state, such as joint angles, joint angular velocities, center of mass displacement / velocity / acceleration, foot contact force, gait phase, and zero-moment point. The standard motion reference trajectory refers to a pre-calibrated / acquired ideal motion benchmark. Current motion style similarity refers to the degree of feature matching between the real-time motion features of the robot under test and the standard motion reference trajectory. It is the raw similarity index without scale adjustment; a higher value indicates that the current motion style is closer to the standard reference. The motion style similarity quality score is the final quantitative evaluation result output after nonlinear normalization processing.

[0019] For example, the historical motion feature time series data of the target humanoid robot during the historical gait training period is input into the AMP discriminator, which outputs the motion style similarity of the target humanoid robot at each time point during the historical gait training period, and calculates the average of these values ​​to obtain the current motion style similarity. The sigmoid function is then used to perform nonlinear normalization on the current motion style similarity to obtain the motion style similarity quality score of the target humanoid robot.

[0020] S130. Based on the current training weights of the target humanoid robot, perform a logarithmic weighted summation of the speed tracking quality score and the motion style similarity quality score to obtain the current gait performance score of the target humanoid robot.

[0021] Here, the current training weights refer to the non-negative real-valued weight coefficients of the two evaluation dimensions (velocity tracking quality and motion style similarity) dynamically assigned to the target humanoid robot in the current training iteration, reflecting the relative priority of velocity accuracy and style matching degree under the current training strategy. The current gait performance score is a quantitative indicator used to evaluate the overall performance of the target robot in the current training iteration.

[0022] For example, the current gait performance score can be determined by the following formula: ; in, This refers to the current gait performance score. This refers to the weight of velocity tracking quality in the current training weights. This refers to the speed tracking quality score. This refers to the weight of sports style similarity in the current training weights. This refers to the quality score for the similarity of sports styles. This refers to a preset minimum value used to prevent logarithmic singularity.

[0023] S140. Based on the current gait performance score and the historical performance baseline of the target humanoid robot, optimize the weight of the motion style similarity quality score in the current training weights to obtain the target training weight of the motion style similarity quality score, and use the target training weight to train the gait of the target humanoid robot.

[0024] The historical performance baseline refers to a dynamic baseline that is continuously updated iteratively during training using an exponential moving average rule. It is obtained by smoothing the gait performance scores of all previous training iterations of the robot. The target training weight refers to a new weight coefficient obtained by combining the current gait performance with the historical performance baseline. It is used for priority allocation in the next stage of gait training, aiming to specifically improve the weaker dimensions and drive gait performance to converge towards the baseline target.

[0025] In one alternative implementation, after gait training of the target humanoid robot using target training weights, the current gait performance score and historical performance baseline are redefined, and the target training weights are optimized until the redefined current gait performance score does not show significant improvement in multiple consecutive training iterations, and the speed tracking error and motion style similarity both meet the design thresholds, at which point training is terminated.

[0026] This application embodiment determines the current speed tracking error of the target humanoid robot based on historical speed time-series data and historical speed command time-series data during the historical gait training period. The current speed tracking error is then subjected to inverse transformation and exponential normalization to obtain a speed tracking quality score for the target humanoid robot. Based on historical motion characteristic time-series data during the historical gait training period, the current motion style similarity between the target humanoid robot and the standard motion reference trajectory is determined. This current motion style similarity is then subjected to nonlinear normalization to obtain a motion style similarity quality score for the target humanoid robot. Based on the current training weights of the target humanoid robot, a logarithmic weighted sum of the speed tracking quality score and the motion style similarity quality score is performed to obtain the current gait performance score for the target humanoid robot. Based on the current gait performance score and the historical performance baseline of the target humanoid robot in the previous training iteration cycle, the weight of the motion style similarity quality score in the current training weights is optimized to obtain a target training weight for the motion style similarity quality score. This target training weight is then used to train the gait of the target humanoid robot. The above technical solution relies on the speed command tracking performance and motion style similarity index collected in real time during the training process to adaptively balance the robot's speed tracking accuracy and the similarity of the reference motion style. This achieves adaptive optimization of weights during the humanoid robot gait training process, solving the technical problems of poor training adaptability, imbalance of optimization targets, high parameter tuning costs, and inability to dynamically adapt to the training state caused by fixed weights.

[0027] Example 2 Figure 2This is a flowchart of a motion style weight optimization method for humanoid robot gait training according to Embodiment 2 of this application. Based on the technical solutions of the above embodiments, this embodiment refines the process of "optimizing the weight of motion style similarity quality score in the current training weight according to the current gait performance score and the historical performance baseline of the target humanoid robot in the previous training iteration cycle to obtain the target training weight of motion style similarity quality score" into "determining the current performance baseline of the target humanoid robot according to the historical performance baseline of the target humanoid robot in the previous training iteration cycle and the current gait performance score; mapping the weight of motion style similarity quality score in the current training weight to a logarithmic variable without lower bound constraints to obtain candidate training weight variables, and performing a binary perturbation operation on the candidate training weight variables to obtain perturbed training weight variables; the perturbed training weight variables include forward perturbed training weight variables and backward training weight variables; inversely converting the perturbed training weight variables into training weights to obtain the perturbed training weight of motion style similarity quality score, and determining the target training weight of motion style similarity quality score according to the perturbed training weight, the current performance baseline and the candidate training weight variables". It should be noted that for parts not described in detail in the embodiments of this application, please refer to the relevant descriptions in other embodiments. For example... Figure 2 As shown, the method includes: S210. Based on the historical speed time series data and historical speed command time series data of the target humanoid robot during the historical gait training period, determine the current speed tracking error of the target humanoid robot, and perform exponential normalization processing on the current speed tracking error after inverse transformation to obtain the speed tracking quality score of the target humanoid robot.

[0028] S220. Based on the historical motion characteristics time series data of the target humanoid robot during the historical gait training period, determine the current motion style similarity between the target humanoid robot and the standard motion reference trajectory, and perform nonlinear normalization on the current motion style similarity to obtain the motion style similarity quality score of the target humanoid robot.

[0029] S230. Based on the current training weights of the target humanoid robot, perform a logarithmic weighted summation of the speed tracking quality score and the motion style similarity quality score to obtain the current gait performance score of the target humanoid robot.

[0030] S240. Based on the historical performance baseline of the target humanoid robot in the previous training iteration cycle and the current gait performance score, determine the current performance baseline of the target humanoid robot.

[0031] The current performance baseline refers to the comprehensive performance smoothing reference value updated after the end of this training iteration cycle. It is calculated by the historical performance baseline of the previous cycle and the current gait performance score according to the preset smoothing rules, replacing the old baseline of the previous cycle, and reflecting the latest long-term stable trend of the robot's gait performance after the end of this cycle.

[0032] S250. Convert the weight mapping of the motion style similarity quality score in the current training weights into a logarithmic variable without lower bound constraints to obtain candidate training weight variables. Perform a binary perturbation operation on the candidate training weight variables to obtain perturbed training weight variables. The perturbed training weight variables include positive perturbation training weight variables and negative training weight variables.

[0033] Here, candidate training weight variables refer to the weight variables corresponding to the current training weights obtained by performing an inverse exponential transformation on the logarithmic variable without lower bound constraints. Forward perturbation training weight variables refer to the perturbation weight variables obtained by increasing the corresponding weight dimension by a preset step size after a binary perturbation operation. Backward training weight variables refer to the perturbation weight variables obtained by decreasing the corresponding weight dimension by a preset step size after a binary perturbation operation.

[0034] For example, the weight of motion style similarity in the current training weights is used as the weight to be optimized. The weight of motion style similarity in the current training weights is mapped into a logarithmic variable without lower bound constraints. A binary perturbation operation is applied to the logarithmic variable to obtain the logarithmic variables after applying the positive perturbation and the negative perturbation, respectively.

[0035] S260. Convert the perturbation training weight variables into training weights in reverse to obtain the perturbation training weights of the motion style similarity quality score. Based on the perturbation training weights, the current performance baseline, and the candidate training weight variables, determine the target training weights of the motion style similarity quality score. Use the target training weights to train the gait of the target humanoid robot.

[0036] Optionally, the perturbation training weights include positive perturbation training weights and negative perturbation training weights. Correspondingly, based on the positive perturbation training weights, a short-window training is performed on the target humanoid robot to obtain the target humanoid robot's positive perturbation velocity time-series data, positive perturbation velocity command time-series data, and positive perturbation motion characteristic time-series data. Based on these data, the positive perturbation gait performance score of the target humanoid robot is determined. Based on the negative perturbation training weights, the target humanoid robot is trained... A humanoid robot is trained within a short window to obtain time-series data of its reverse perturbation velocity, reverse perturbation velocity command, and reverse perturbation motion characteristics. Based on these data, the reverse perturbation gait performance score of the humanoid robot is determined. The target training weights for the motion style similarity quality score are determined based on the forward perturbation gait performance score, the reverse perturbation gait performance score, the current performance baseline, and candidate training weight variables.

[0037] Among them, the positive perturbation training weights refer to the weights of the positive perturbation training weight variables after inverse logarithmic transformation. The negative perturbation training weights refer to the weights of the negative perturbation training weight variables after inverse logarithmic transformation. The positive perturbation velocity time series data refers to the time series data of the robot's actual motion velocity collected during short-term training, including core motion indicators such as linear velocity and angular velocity. The positive perturbation velocity command time series data refers to the time series of the ideal velocity that the robot is expected to achieve before short-term training. The positive perturbation motion feature time series data refers to the set of time series data of all dimensions of gait features except velocity collected during short-term training, including body posture (pitch / roll / yaw) fluctuation sequences, foot contact force time series, zero-moment point trajectory deviation, step frequency / stride length time series, energy consumption time series, etc. The positive perturbation gait performance score refers to a dimensionless scalar that quantifies the comprehensive gait performance under positive perturbation weights. Reverse perturbation velocity time-series data refers to the time-series data of the robot's actual motion velocity collected during short-term training, including core motion indicators such as linear velocity and angular velocity. Reverse perturbation velocity command time-series data refers to the time-series data of the ideal velocity that the robot is expected to achieve, preset before short-term training. Reverse perturbation motion feature time-series data refers to the collection of time-series data of all dimensions of gait features except velocity, collected during short-term training, including body attitude (pitch / roll / yaw) fluctuation sequences, foot contact force time-series, zero-moment point trajectory deviation, stride frequency / stride length time-series, energy consumption time-series, etc. Reverse perturbation gait performance score refers to a dimensionless scalar that quantifies the comprehensive gait performance under reverse perturbation weights.

[0038] Furthermore, based on the positive perturbation gait performance score and the current performance baseline, the positive perturbation performance baseline of the target humanoid robot is determined, and supergradient estimation is performed on the positive perturbation performance baseline and the current performance baseline to obtain the positive perturbation supergradient estimate of the positive perturbation performance baseline; based on the negative perturbation gait performance score and the current performance baseline, the negative perturbation performance baseline of the target humanoid robot is determined, and supergradient estimation is performed on the negative perturbation performance baseline and the current performance baseline to obtain the negative perturbation supergradient estimate of the negative perturbation performance baseline; based on the positive perturbation supergradient estimate and the negative perturbation supergradient estimate, the target perturbation supergradient estimate of the target humanoid robot is determined; based on the target perturbation supergradient estimate and the candidate training weight variables, the target training weights for the motion style similarity quality score are determined.

[0039] The positive perturbation performance baseline refers to the stable value obtained by processing the positive perturbation gait performance score obtained from the short-term training with the current positive perturbation using an exponential moving average. The positive perturbation supergradient estimate is a specific value calculated using the supergradient estimation method, representing the estimated performance improvement per unit step size if the weights are perturbed in the positive direction while maintaining the current baseline level. The negative perturbation performance baseline refers to the stable value obtained by processing the negative perturbation gait performance score obtained from the short-term training with the current negative perturbation using an exponential moving average. The negative perturbation supergradient estimate is a specific value calculated using the supergradient estimation method, representing the estimated performance improvement per unit step size if the weights are perturbed in the negative direction while maintaining the current baseline level. The target perturbation supergradient estimate is the supergradient estimate corresponding to the perturbation direction that improves the overall performance, selected from the positive and negative perturbation supergradient estimates.

[0040] For example, based on the performance baselines of the forward / reverse perturbations obtained before and after the forward / reverse perturbations, the corresponding hyperparameter gradient estimates are calculated. Based on the hyperparameter gradient estimates, the perturbation directions that can improve the overall performance in the forward and reverse perturbations are determined, and the gradient estimates corresponding to the perturbation directions are determined as the target perturbation hypergradient estimates.

[0041] Furthermore, based on the target perturbation hypergradient estimate, the candidate training weight variables are updated to obtain the target training weight variables for the motion style similarity quality score; the target training weight variables are then converted into training weights to obtain the target training weights for the motion style similarity quality score.

[0042] Among them, the target training weight variable refers to the intermediate weight form obtained after the update operation is completed. It is still in the variable parameter space and is the update result that has not yet been converted into actual usable weights.

[0043] For example, the target training weight variables can be determined by the following formula: ; in, This refers to the target training weight variables; This refers to candidate training weight variables; This refers to the stability gating factor, which is determined by the current iteration step and the current training state. This refers to the preset learning rate. This refers to the hypergradient estimate corresponding to the direction of the target perturbation. This refers to the preset gradient threshold; This refers to the lower bound of the physical feasibility of the pre-defined training weight variables; This refers to the upper bound of the physical feasibility of the preset training weight variables.

[0044] In one optional implementation, to avoid training crashes caused by a single drastic change in weights, constraints are set for the target training weight variable and the target training weight during each weight update process in each training iteration: the absolute difference between the target training weight variable and the candidate training weight variable is less than or equal to a preset threshold, and the ratio of the target training weight to the current training weight is within a preset ratio range; if both the target training weight variable and the target training weight meet the constraints, the target training weight is output normally; if the target training weight variable and / or the target training weight does not meet the constraints, the current training weight is not updated, and gait iteration training is performed again.

[0045] In another optional implementation, this application also sets a stability gating judgment mechanism; if the training state is stable (no abnormal values, small index variance, discriminator convergence), the stability gating factor is increased, approaching 1; if the training state is abnormal (a large number of abnormal values ​​exist, gait training terminates, discriminator loss fluctuates violently), the stability gating factor is decreased, approaching 0; if the current iteration step is less than the first threshold and / or the stability gating factor is 0, weight updates are prohibited, and weight adaptive updates are only performed periodically when the stability gating factor is greater than 0.

[0046] In another alternative implementation, the iterative training control mode can be automatically switched according to the current total number of training iterations, constraining the weight update intensity throughout the process.

[0047] Specifically, if the number of training iterations is less than the first threshold, a warm-up freeze is performed, the stability gating factor is set to 0, and perturbation hypergradient estimation is not performed. If the number of training iterations is greater than or equal to the first threshold but less than the second threshold, weight updates are enabled, the preset update learning rate and preset ratio range are lowered, the single adjustment range of weights is limited, and the optimal weight range is slowly explored. If the number of training iterations is greater than or equal to the second threshold but less than the third threshold, the preset update learning rate and preset ratio range are restored, the perturbation hypergradient estimation algorithm is fully executed, and the speed tracking accuracy and motion style are autonomously balanced. If the number of training iterations is greater than the third threshold, the performance baseline is monitored to be less than the baseline threshold for multiple consecutive rounds, the stability gating factor is set to 0, and adaptive weight updates are stopped. Among these, the first threshold is less than the second threshold, and the second threshold is less than the third threshold.

[0048] This application embodiment determines the current speed tracking error of the target humanoid robot based on historical speed time-series data and historical speed command time-series data during the historical gait training period. The current speed tracking error is then subjected to inverse transformation and exponential normalization to obtain a speed tracking quality score for the target humanoid robot. Based on historical motion characteristic time-series data of the target humanoid robot during the historical gait training period, the current motion style similarity between the target humanoid robot and the standard motion reference trajectory is determined. This current motion style similarity is then subjected to nonlinear normalization to obtain a motion style similarity quality score for the target humanoid robot. Finally, based on the current training weights of the target humanoid robot, a logarithmic weighted sum of the speed tracking quality score and the motion style similarity quality score is performed to obtain the current gait performance of the target humanoid robot. Scoring: Based on the historical performance baseline of the target humanoid robot in the previous training iteration cycle and the current gait performance score, determine the current performance baseline of the target humanoid robot; convert the weights of the motion style similarity quality score in the current training weights into logarithmic variables without lower bound constraints to obtain candidate training weight variables, and perform a binary perturbation operation on the candidate training weight variables to obtain perturbed training weight variables; the perturbed training weight variables include forward perturbed training weight variables and backward training weight variables; inversely convert the perturbed training weight variables into training weights to obtain the perturbed training weights of the motion style similarity quality score, and determine the target training weights of the motion style similarity quality score based on the perturbed training weights, the current performance baseline, and the candidate training weight variables, and use the target training weights to perform gait training on the target humanoid robot. The above technical solution relies on the speed command tracking performance and motion style similarity index collected in real time during the training process to adaptively balance the robot's speed tracking accuracy and the similarity of the reference motion style. This achieves adaptive optimization of weights during the humanoid robot gait training process, solving the technical problems of poor training adaptability, imbalance of optimization targets, high parameter tuning costs, and inability to dynamically adapt to the training state caused by fixed weights.

Claims

1. A method for optimizing motion style weights in gait training of a humanoid robot, characterized in that, include: Based on the historical speed time series data and historical speed command time series data of the target humanoid robot during the historical gait training period, the current speed tracking error of the target humanoid robot is determined, and the current speed tracking error is subjected to inverse transformation and exponential normalization to obtain the speed tracking quality score of the target humanoid robot. Based on the historical motion feature time series data of the target humanoid robot during the historical gait training period, the current motion style similarity between the target humanoid robot and the standard motion reference trajectory is determined, and the current motion style similarity is subjected to nonlinear normalization processing to obtain the motion style similarity quality score of the target humanoid robot. Based on the current training weights of the target humanoid robot, the speed tracking quality score and the motion style similarity quality score are logarithmically weighted and summed to obtain the current gait performance score of the target humanoid robot. Based on the current gait performance score and the historical performance baseline of the target humanoid robot in the previous training iteration cycle, the weight of the motion style similarity quality score in the current training weight is optimized to obtain the target training weight of the motion style similarity quality score, and the target training weight is used to train the gait of the target humanoid robot.

2. The method of claim 1, wherein, Based on the current gait performance score and the historical performance baseline of the target humanoid robot in the previous training iteration, the weights of the motion style similarity quality score in the current training weights are optimized to obtain the target training weights for the motion style similarity quality score, including: The current performance baseline of the target humanoid robot is determined based on the historical performance baseline of the target humanoid robot in the previous training iteration cycle and the current gait performance score. The weight mapping of the motion style similarity quality score in the current training weights is converted into a logarithmic variable without lower bound constraints to obtain candidate training weight variables. The candidate training weight variables are then subjected to a binary perturbation operation to obtain perturbed training weight variables. The perturbed training weight variables include positive perturbation training weight variables and negative perturbation training weight variables. The perturbation training weight variables are inversely converted into training weights to obtain the perturbation training weights of the motion style similarity quality score. Based on the perturbation training weights, the current performance baseline, and the candidate training weight variables, the target training weights of the motion style similarity quality score are determined.

3. The method of claim 2, wherein, The perturbation training weights include positive perturbation training weights and negative perturbation training weights; correspondingly, based on the perturbation training weights, the current performance baseline, and the candidate training weight variables, the target training weights for the motion style similarity quality score are determined, including: Based on the positive perturbation training weights, the target humanoid robot is trained in a short window to obtain the positive perturbation velocity time-series data, positive perturbation velocity command time-series data, and positive perturbation motion feature time-series data of the target humanoid robot. Based on the positive perturbation velocity time-series data, the positive perturbation velocity command time-series data, and the positive perturbation motion feature time-series data, the positive perturbation gait performance score of the target humanoid robot is determined. Based on the reverse perturbation training weights, the target humanoid robot is trained in a short window to obtain the reverse perturbation velocity time-series data, reverse perturbation velocity command time-series data, and reverse perturbation motion feature time-series data of the target humanoid robot. Based on the reverse perturbation velocity time-series data, the reverse perturbation velocity command time-series data, and the reverse perturbation motion feature time-series data, the reverse perturbation gait performance score of the target humanoid robot is determined. The target training weights for the motion style similarity quality score are determined based on the positive perturbation gait performance score, the negative perturbation gait performance score, the current performance baseline, and the candidate training weight variables.

4. The method of claim 3, wherein, The step of determining the target training weights for the movement style similarity quality score based on the positive perturbation gait performance score, the negative perturbation gait performance score, the current performance baseline, and the candidate training weight variables includes: Based on the positive perturbation gait performance score and the current performance baseline, the positive perturbation performance baseline of the target humanoid robot is determined, and supergradient estimation is performed on the positive perturbation performance baseline and the current performance baseline to obtain the positive perturbation supergradient estimate of the positive perturbation performance baseline. Based on the reverse perturbation gait performance score and the current performance baseline, the reverse perturbation performance baseline of the target humanoid robot is determined, and super gradient estimation is performed on the reverse perturbation performance baseline and the current performance baseline to obtain the reverse perturbation super gradient estimate of the reverse perturbation performance baseline. Based on the positive perturbation supergradient estimate and the negative perturbation supergradient estimate, the target perturbation supergradient estimate of the target humanoid robot is determined; The target training weights for the motion style similarity quality score are determined based on the target perturbation hypergradient estimate and the candidate training weight variables.

5. The method of claim 4, wherein, The step of determining the target training weights for the motion style similarity quality score based on the target perturbation hypergradient estimate and the candidate training weight variables includes: Based on the target perturbation hypergradient estimate, the candidate training weight variables are updated to obtain the target training weight variables for the motion style similarity quality score. The target training weight variable is converted into training weight in reverse to obtain the target training weight of the motion style similarity quality score.

6. The method of claim 5, wherein, The target training weight variables can be determined using the following formula: ; wherein, denotes a target training weight variable; denotes a candidate training weight variable; denotes a stability gating factor determined by a current iteration step number and a current training state; denotes a preset update learning rate; denotes a hyper-gradient estimation value corresponding to a target perturbation direction; denotes a preset gradient threshold value; denotes a preset lower bound of physical feasibility of the training weight variable; denotes a preset upper bound of physical feasibility of the training weight variable.