A humanoid robot safe foot landing training method based on future foot end contact imagination

Abstract

The application discloses a kind of humanoid robot safety foot landing training methods based on future foot end contact imagination.The method sets future foot end contact imagination model in training stage, with the privilege state and current action of humanoid robot as input, predicts the distribution of future contact position of foot end;With the foot bottom coverage area corresponding to the predicted contact position as the object, the support deficiency degree is calculated in combination with local terrain height information, so as to convert the expected support quality evaluation under the future contact distribution into the prospective landing guide signal for training the motion control strategy of humanoid robot.Through the above mode, the present application can forward the sparse safety feedback generated only when landing to the pre-contact guide in swing stage, make the robot learn to correct swing trajectory and select the expected landing position with sufficient support before foot end landing, obtain the motion control strategy with higher foot bottom support sufficiency and complex terrain safety landing ability, reduce the risk of edge stepping, sliding and falling.

Description

Technical Field

[0001] This invention relates to the field of motion control technology for humanoid robots, and in particular to a training method for safe footing training of humanoid robots based on the imagination of future foot contact. Background Technology

[0002] With the continuous improvement of the hardware capabilities and intelligence level of humanoid robots, they have broad application prospects in complex task scenarios such as industrial inspection, daily life services, disaster relief, and scientific research exploration. Compared with wheeled and tracked robots, humanoid robots have a bipedal structure similar to humans, enabling them to move on typical terrains in human environments, such as steps, gaps, high platforms, and discrete support surfaces. However, humanoid robots themselves have characteristics such as limited support area, high center of gravity, and frequent switching of contact states. When moving in complex terrain, they not only need to maintain body posture stability but also need to accurately select the foot placement position to ensure sufficient support between the soles of the feet and the terrain.

[0003] To achieve stable walking in humanoid robots, traditional methods typically start from the robot's kinematics and dynamics models, generating executable walking movements by designing gait periods, center-of-mass trajectories, foot trajectories, and stability constraints. For example, Zhang Guoliang et al. (CN103770111A) proposed a gait planning and synthesis method for humanoid robots. This method uses polynomial functions to represent the trajectories of the robot's hip joint and swing leg, and combines geometric constraints, maximum swing leg stride height, gait periodicity, collision effects, and zero-point moment criteria to achieve gait planning and synthesis. This type of method has good interpretability and is suitable for regular terrain and scenarios with relatively clear task conditions. However, when facing complex terrains such as varying step sizes, varying gap widths, discontinuous support areas, or irregular foot boundaries, it usually requires relying on preset trajectories, manual constraints, or additional planning processes, making it difficult to fully adapt to the constantly changing foot requirements in open environments.

[0004] In recent years, data-driven control methods based on reinforcement learning have gradually become an important direction in the field of legged robot motion control. These methods typically use deep neural networks as controllers, autonomously optimizing motion strategies based on a preset reward function through robot-environment interaction in a simulation environment, thereby reducing reliance on precise models and manual gait planning. Ren Ju (CN111360834A) et al. proposed a humanoid robot motion control method and system based on deep reinforcement learning. This method determines the target angles of each joint based on the current state of the humanoid robot and a preset deep reinforcement learning model, and then controls the joint movements through a PD controller. Zhuang et al., in *Humanoid Parkour Learning*, proposed an end-to-end vision-based whole-body control strategy, enabling humanoid robots to complete complex motion tasks such as jumping onto platforms, crossing obstacles, and traversing gaps. These methods demonstrate that combining reinforcement learning and visual perception can improve the motion capabilities of humanoid robots in complex terrain, but the issue of safe footing in complex terrain remains unresolved.

[0005] For humanoid robots with planar foot structures, the safety of foot landing depends not only on whether the center of the foot is within the accessible area, but also on whether there is sufficient support within the area covered by the foot. When the foot lands on the edge of a step, gap, platform, or discrete support area, only a portion of the foot may be supported, resulting in a significant reduction in the effective support area. This greatly increases the risk of foot slippage, robot instability, or even robot falls. Therefore, safe landing in complex terrain cannot be judged solely by whether the center of the foot contacts the ground; the adequacy of support within the area covered by the foot must also be evaluated.

[0006] To address the issue of foot safety in complex terrain, existing research typically improves foot contact quality by designing foot placement rewards, constraining foot placement areas, tracking pre-defined steps, or incorporating terrain height information. These methods can, to some extent, reduce obviously unsuitable foot placement locations and improve the robot's ability to navigate sparsely supported areas or irregular terrain. For example, Wang et al., in "BeamDojo: Learning Agile Humanoid Locomotion on SparseFootholds," proposed a reinforcement learning framework for humanoid robots in sparsely supported terrain, using sampled foot placement rewards to improve the robot's accurate foot placement ability. However, existing foot placement rewards or safety assessments are usually generated during or after actual foot contact; the robot can only obtain feedback on whether the foot placement is safe and whether the foot support is sufficient after landing. For the swinging foot, the swing trajectory is gradually formed before contact with the ground. If safety feedback only occurs upon contact, the movement strategy will struggle to promptly determine whether the current swinging motion will lead to subsequent stepping on the edge, suspension, or insufficient support. This results in difficulties in allocating time credit and hinders the provision of timely and continuous guidance for corrective footing during the swinging phase. Especially in tasks requiring precise footing, such as descending stairs, crossing gaps, or ascending platforms, delayed feedback makes the strategy more prone to edge stepping or foot deviation, impacting the effective and efficient learning of safe footing behaviors.

[0007] In summary, current methods for motion control of humanoid robots in complex terrain have achieved visual perception and reinforcement learning control to a certain extent, but a forward-looking guidance mechanism for safe landing training is still lacking. Specifically, existing methods struggle to predict the future contact position of the foot during the swing phase and evaluate the degree of support deficiency in the foot-covered area at that position, resulting in sparse and delayed safety feedback. Therefore, it is necessary to propose a humanoid robot safe landing training method based on the imagination of future foot contact. This method predicts the distribution of future foot contact positions during the training phase and calculates the degree of support deficiency based on local terrain height information within the foot-covered area. This shifts the safety feedback generated only upon ground contact forward and densifies it into pre-contact guidance during the swing phase, improving the humanoid robot's safe landing capability in complex terrain. Summary of the Invention

[0008] The purpose of this invention is to address the problems in existing technologies regarding the lag in safety landing feedback for humanoid robots in complex terrain, the lack of forward-looking guidance during the swing phase, and the difficulty in timely evaluation of foot support adequacy. This invention proposes a safety landing training method for humanoid robots based on the imagination of future foot contact. During the training phase, this method predicts the distribution of future foot contact positions and evaluates the degree of support deficiency based on local terrain height information within the foot coverage area. This shifts the sparse safety feedback generated only upon ground contact forward and densifies it into pre-contact guidance during the swing phase. This allows the humanoid robot's motion control strategy to correct the swing trajectory and select a sufficiently supported expected landing position before the foot touches the ground.

[0009] To achieve the above objectives, the present invention provides a training method for safe footing training of humanoid robots based on future foot contact imagination, comprising the following steps: S1. Set up a future foot contact visualization model during the training phase; the future foot contact visualization model is used to generate training guidance signals related to safe foot landing. S2. Input the current privileged state and current action of the humanoid robot into the future foot contact imagination model to predict the future foot contact position distribution; S3. Use the lack of support as an evaluation index for the safety of the foot landing point, and establish a rule for calculating the lack of support: For the foot position to be evaluated, obtain the local terrain height information of the corresponding foot coverage area, and calculate the degree of support loss at the foot position to be evaluated based on the local terrain height information. S4. Determine the foot landing safety guidance evaluation method based on the foot contact state: For the foot in the support phase, calculate the degree of support deficiency based on its current contact position as the foot position to be evaluated; For the foot in the swing phase, perform discrete sampling on the future contact position distribution of the foot as described in step S2, and perform weighted summation on the degree of support deficiency at each sampling position to obtain the expected degree of support deficiency under the future contact distribution. S5. Based on the degree of support deficiency at the foot end of the support phase and the expected degree of support deficiency at the foot end of the swing phase, construct a foot landing guidance signal; S6. The humanoid robot's motion control strategy is trained using the foot guidance signal, so that the humanoid robot's motion control strategy learns to correct the swing trajectory and select a sufficiently supported expected foot landing position before the foot touches the ground.

[0010] Furthermore, the future foot contact imagination model is composed of a multi-layered sensing mechanism.

[0011] Furthermore, the current privileged state includes one or more of the following: robot body state, foot contact state, foot speed, foot position, limb position, terrain height information near the foot, and terrain height information near the body.

[0012] Furthermore, the future contact position distribution of the foot tip is a two-dimensional probability distribution in the current robot base coordinate system, used to characterize the future contact position of the foot tip and its prediction uncertainty; the future contact position distribution of the foot tip is a Gaussian distribution, and the future foot tip contact imagination model outputs the two-dimensional position parameters and uncertainty parameters corresponding to each foot tip. The two-dimensional position parameters are used to determine the mean of the Gaussian distribution, and the uncertainty parameters are used to determine the variance or standard deviation of the Gaussian distribution after positive value mapping.

[0013] Furthermore, the supervision target of the future foot contact imagination model is the first effective future contact position of the corresponding foot. The first effective future contact position is: within the same training trajectory segment, for the foot currently in the swing phase, search backward from the current time for the moment when the foot first enters the contact phase, and transform the foot position at that moment to the two-dimensional plane position obtained in the current robot base coordinate system; the future foot contact imagination model is trained using negative log-likelihood loss so that the predicted future foot contact position distribution covers the first effective future contact position.

[0014] Furthermore, the foot coverage area is set according to the size of the humanoid robot's foot, and local terrain height information is obtained by sampling within the foot coverage area at a preset sampling interval; the degree of support deficiency is determined based on the proportion of sampling points within the foot coverage area that meet the support height condition, wherein the support height condition is that the height difference between the foot reference height and the terrain height of the sampling point is less than a preset height threshold; for the predicted future contact position, the maximum value of the terrain height within the corresponding foot coverage area is used as the foot reference height.

[0015] Furthermore, when evaluating the landing safety of the foot in the swing phase, the sampling range and sampling weight of discrete sampling are determined based on the prediction uncertainty of the future contact position distribution of the foot, and the degree of support deficiency at each sampling position is weighted and summed according to the sampling weight.

[0016] Furthermore, for each foot end, the degree of comprehensive support deficiency is determined based on the foot end contact state: for the foot end in the support phase, the degree of support deficiency at its current contact position is taken as the comprehensive support deficiency degree; for the foot end in the swing phase, the expected degree of support deficiency under the future contact distribution is taken as the comprehensive support deficiency degree.

[0017] Furthermore, the foot placement guidance signal is a reinforcement learning reward item, which is constructed based on the comprehensive support deficiency degree of each foot end and decreases as the support deficiency degree increases; the foot placement guidance signal is activated when the training course level is greater than the preset course level threshold.

[0018] Furthermore, the future foot contact imagination model is only used to generate foot landing guidance signals during the training phase. After training, the humanoid robot motion control strategy obtained is used to output robot control actions based on deployment observations.

[0019] Compared with the prior art, the beneficial effects of the present invention are: (1) The present invention enables the humanoid robot to obtain the support sufficiency information of the expected landing area before the foot actually touches the ground by imagining the future foot contact and evaluating the lack of foot support, thereby moving the sparse safety feedback generated at the time of landing to the pre-contact guidance of the swinging stage; (2) The present invention can improve the credit allocation of the swinging stage decision, enabling the motion control strategy to learn the correspondence between the foot swing trajectory and the subsequent landing safety earlier, and improve the learning efficiency of safe landing behavior; (3) The present invention can prompt the robot to correct the foot trajectory in advance during the swinging stage, reducing edge stepping, suspended landing and landing deviation caused by feedback lag; (4) The present invention can improve the foot support sufficiency of the planar foot structure humanoid robot in complex terrains such as steps, gaps, high platforms, discrete support surfaces, etc., and reduce the risk of slipping, body instability and falling; (5) The present invention can enhance the humanoid robot's ability to pass through complex terrains and motion stability, providing a technical basis for its reliable application in actual scenarios such as industrial inspection, disaster relief, and scientific exploration. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the overall process of a humanoid robot's safe footing training method based on the imagination of future foot contact according to the present invention. Figure 2 This is a schematic diagram of the future foot contact imagery-guided training framework structure in this invention; Figure 3 This is a schematic diagram illustrating the calculation of the expected degree of support loss based on the future contact position distribution of the foot in this invention. Figure 4 This is a schematic diagram illustrating the calculation of the degree of support deficiency in the foot cover area in this invention. Detailed Implementation

[0021] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are for illustrative purposes only and are not intended to limit the scope of protection of the present invention. Equivalent substitutions or modifications made by those skilled in the art without departing from the concept of the present invention should fall within the scope of protection of the present invention.

[0022] like Figure 1As shown, this invention provides a training method for safe foot placement training of humanoid robots based on imagined future foot contact. This method is used in the training phase of the humanoid robot's reinforcement learning motion control strategy. During training, the humanoid robot's motion control strategy outputs the current action based on current observation information. The simulation environment updates the robot's state based on the current action and provides privileged states, foot contact states, and local terrain height information available during the training phase. The method specifically includes the following steps.

[0023] S1. Set up a future foot contact imagination model during the training phase.

[0024] like Figure 2 As shown, a future foot contact imagination model is set up during the training phase. This model predicts the possible contact locations of the foot at subsequent moments based on the humanoid robot's current privileged state and current action. As an auxiliary model during the training phase, this model generates training guidance signals related to safe foot placement.

[0025] In one implementation, the future foot contact imagination model is composed of a multilayer perceptron (MLP), whose inputs are the current privileged state and the current action, and whose outputs are the future foot contact position distribution parameters. For a bipedal humanoid robot, the future foot contact imagination model can output the future contact position distribution parameters for the left and right feet respectively.

[0026] S2. Input the current privileged state and current action of the humanoid robot into the future foot contact imagination model to predict the future foot contact position distribution.

[0027] like Figure 2 As shown, the current privileged state is denoted as The current action is denoted as The current privileged state includes one or more of the following: robot body state, foot contact state, foot speed, foot position, limb position, terrain height information near the foot, and terrain height information near the body.

[0028] Current privilege status and current action Input the future foot contact imagination model to obtain the foot tip in Predicted future contact distribution parameters at time: No. Each foot tip The predicted future contact distribution parameters at time t can be expressed as: in, Indicates the predicted future foot contact position. This indicates prediction uncertainty. Based on the predicted future contact distribution parameters, the future contact position distribution of the foot is constructed as follows: in, This indicates the current two-dimensional planar position in the robot's base coordinate system. Represents the identity matrix.

[0029] In one specific implementation, the future foot contact visualization model outputs two-dimensional position parameters and uncertainty parameters for each foot. The two-dimensional position parameters are used to determine the mean of a Gaussian distribution, and the uncertainty parameters, after positive value mapping, are used to determine the variance or standard deviation of the Gaussian distribution. The positive value mapping can employ exponential mapping to ensure that the variance or standard deviation is positive.

[0030] The supervision target of the future foot contact imagination model is the first effective future contact position of the corresponding foot. Specifically, within the same training trajectory segment, for a foot currently in the swing phase, based on the foot contact state in the training environment, the first moment when the foot enters the contact phase from the swing phase is searched backward from the current moment, and the foot position at that moment is taken as the first effective future contact position. Subsequently, the first effective future contact position is transformed from the world coordinate system to the current robot base coordinate system, and its two-dimensional plane coordinates are taken as the supervision target. If no subsequent effective contact is found within the current training trajectory segment, the corresponding sample does not participate in the supervision loss calculation of the future foot contact imagination model.

[0031] After obtaining the supervised target, the future foot contact imagination model is trained using negative log-likelihood loss, so that the predicted future foot contact position distribution has a high probability of being at the first effective future contact position. If the future foot contact position distribution is Gaussian, the negative log-likelihood loss, ignoring the constant term, is expressed as: in, Indicates the first Each foot tip The first valid future contact position corresponding to a given moment. Through the above training method, the future foot contact imagination model can learn to predict the subsequent foot contact position and its uncertainties based on the current privileged state and current action.

[0032] S3. Use support loss as an evaluation index for the safety of landing points, and establish support loss calculation rules.

[0033] like Figure 4As shown, this invention uses the area covered by the sole of the foot as the object of foot landing safety evaluation. For the foot position to be evaluated... The system determines the foot coverage area corresponding to the size of the humanoid robot's foot, centered on this location, and acquires local terrain height information within this area. This local terrain height information consists of the terrain heights of multiple sampling points, used to determine how much of the foot coverage area can provide effective support.

[0034] In one specific embodiment, the foot cover area is 22.5 cm × 10 cm, and sampling is performed within this area at a preset sampling interval of 2.5 cm to obtain multiple local terrain height sampling points. The above values ​​are only a preferred embodiment; in practical applications, they can be adjusted according to the foot size, terrain resolution, and training task of different humanoid robots.

[0035] Let the position of the foot to be evaluated be... The total area covered by the foot is The sampling point, the first The terrain height of each sampling point is The reference height of the sole is The preset height threshold is .like Then it is considered that the first Each sampling point meets the support height condition and can provide effective support for the sole of the foot; if this condition is not met, the area corresponding to that sampling point is considered not to provide effective support for the sole of the foot. Degree of support deficiency. Represented as: in, Indicates an indicator function. The larger the value, the lower the proportion of sampling points that can provide effective support within the foot's coverage area, and the less sufficient the support at the corresponding foot landing position. The smaller the value, the higher the proportion of sampling points that can provide effective support within the foot's coverage area, and the more adequate the support at the corresponding foot placement location.

[0036] In a preferred embodiment, a preset height threshold is used. The value is 3 cm. For the predicted future contact location, since the foot has not yet actually touched the ground, the maximum terrain height within the corresponding foot-covered area is used as the foot reference height, i.e.: The above methods enable a quantitative evaluation of the adequacy of plantar support at the foot position to be evaluated, and provide a unified evaluation index for the safety of foot landing in both the support phase and swing phases.

[0037] S4. Determine the method for guiding and evaluating foot safety based on the contact state of the foot tip.

[0038] like Figure 2 As shown, this invention determines the landing safety evaluation method for the foot end in the support phase and the foot end in the swing phase based on the foot contact state. For the foot end in the support phase, since the foot end has already made contact with the terrain, its current contact position is used. As the foot position to be evaluated, the degree of support loss at the current contact position is calculated according to the support loss calculation rules in step S3. For the foot in the swing phase, since it has not yet touched the ground, the current contact position cannot be directly used for evaluation. Therefore, this invention performs a forward-looking evaluation based on the distribution of future foot contact positions predicted by the future foot contact imagination model in step S2. Figure 3 As shown, the distribution of future foot contact positions is first output from the future foot contact imagination model. Then, based on the distribution of future contact positions at the foot, discrete sampling is performed to obtain multiple candidate sampling positions: in, This indicates the number of sampling locations. The sampling range and sampling weights for discrete sampling depend on the prediction uncertainty. Confirmed. Greater prediction uncertainty indicates a more dispersed distribution of future contact locations, allowing for a larger sampling range to cover potential landing areas; lower prediction uncertainty results in sampling locations more concentrated around the prediction mean. For each sampling location... Calculate the degree of support deficiency according to the support deficiency calculation rules in step S3. Then, a weighted sum is performed based on the weights of each sampling location under the future contact location distribution to obtain the expected degree of support deficiency under the future contact distribution: in, Indicates the first The sampling weights at each sampling location, and the sampling weights are determined by... The result is obtained through normalization. Using the above method, the adequacy of support in the potential landing area can be prospectively estimated before the swinging foot touches the ground.

[0039] S5. Based on the degree of support deficiency at the foot end of the support phase and the expected degree of support deficiency at the foot end of the swing phase, construct a forward-looking foot guidance signal.

[0040] After obtaining the degree of support deficiency at the current contact position of the support phase foot end and the expected degree of support deficiency under the future contact distribution of the swing phase foot end, the present invention determines the comprehensive degree of support deficiency of each foot end based on the foot end contact state.

[0041] For the At the tip of the foot, The degree of lack of comprehensive support at any given moment is expressed as follows: in, Indicates the first Is each foot in the support phase at time t? Indicates the first Each foot tip The current contact position at any given moment. This indicates the degree of support loss at the current contact point. Indicates the first Each foot tip Distribution of future contact locations at any given time This indicates the degree of expected support deficiency under the future contact distribution.

[0042] Therefore, for the foot in the support phase, the overall degree of support deficiency is determined by the degree of support deficiency at the current contact position; for the foot in the swing phase, the overall degree of support deficiency is determined by the expected degree of support deficiency under the future contact distribution.

[0043] Furthermore, a foot-landing guidance signal is constructed based on the overall degree of support deficiency at each foot. In one embodiment, the foot-landing guidance signal is a reinforcement learning reward term, which decreases as the overall support deficiency increases. For a bipedal humanoid robot, the foot-landing guidance reward is represented as: in, express Timely guidance and rewards. This represents the reward metric parameter. In one specific implementation, Take 0.0625.

[0044] When the overall support deficiency at each foot is small, it indicates that the corresponding current contact position or expected future landing position has sufficient plantar support, resulting in a larger landing guidance reward. Conversely, when the overall support deficiency at each foot is large, it indicates a risk of edge stepping, suspended landing, or insufficient support, resulting in a smaller landing guidance reward. Through this reward, the motor control strategy can learn to pre-correct the foot trajectory during the swing phase and select a sufficiently supported expected landing position.

[0045] In a preferred embodiment, considering that the robot's gait is still unstable in the early stages of training and that there may be significant noise when it comes into contact with the supervised target in the future, the foot guidance signal is activated after the training course level exceeds a preset course level threshold. In a specific embodiment, the preset course level threshold is 2, that is, the foot guidance signal is activated when the training course level is greater than 2.

[0046] S6. The humanoid robot's motion control strategy is trained using the foot guidance signal, so that the humanoid robot's motion control strategy learns to correct the swing trajectory and select a sufficiently supported expected foot landing position before the foot touches the ground.

[0047] like Figure 2 As shown, during the training process, the foot-landing guidance signal and the reward related to the humanoid robot's motion control task are used together for reinforcement learning training. The humanoid robot's motion control strategy outputs actions based on the current observation information, the simulation environment updates the robot's state based on the actions and returns training information, and the reinforcement learning training module updates the motion control strategy based on the task reward and the foot-landing guidance reward.

[0048] In one implementation, the humanoid robot's motion control strategy is trained using a proximal policy optimization (PPO) algorithm. The humanoid robot's motion control strategy employs an asymmetric actor-critic structure. The policy network, acting as the actor, receives ontology-aware history and visual observations available during the deployment phase and outputs target joint actions. The critic network additionally receives privileged states during the training phase to estimate state values ​​and assist in policy updates. The visual observations can be depth images, and the ontology-aware history can include one or more of the following: base angular velocity, gravity projection, velocity commands, joint positions, joint velocities, and actions from the previous moment.

[0049] In one embodiment, the policy network may include a convolutional neural network (CNN) for processing visual observations, a multilayer perceptron for processing ontological perception history, and a gated recurrent unit (GRU) for fusing temporal information; the critic network may be composed of a multilayer perceptron. The above network structure is only one embodiment; the core of this invention lies in generating pre-contact foot placement guidance signals through future foot contact imagination and support deficiency evaluation, and is not limited to a specific motion control policy network structure.

[0050] In one implementation, the training rewards include one or more of the following: task-related rewards, motion stability or gait constraint rewards, regularization rewards, and foot placement guidance rewards. Task-related rewards encourage the humanoid robot to complete motion tasks according to speed commands or terrain traversal instructions; motion stability or gait constraint rewards encourage the robot to maintain reasonable body posture, foot contact, and gait behavior; regularization rewards limit joint movements, motion variations, torques, or other control quantities to a reasonable range; and foot placement guidance rewards guide the robot to select a sufficiently supported expected footing position based on the degree of support deficiency. The foot placement guidance rewards, together with the aforementioned training rewards, are used to update the motion control strategy.

[0051] Since the foot-landing guidance signal can be generated before the foot actually touches the ground, the sparse safety feedback at the time of ground contact can be moved forward and densed into pre-contact guidance during the swing phase, thereby improving the credit allocation of the swing phase decision and increasing the learning efficiency of the motion control strategy for safe foot-landing behavior.

[0052] After training, the resulting humanoid robot motion control strategy outputs robot control actions based on the observation information available during the deployment phase. The future foot contact imagination model is only used to generate foot-landing guidance signals during the training phase and is not a necessary component of the control strategy during the deployment phase after training.

[0053] It will be understood by those skilled in the art that the above descriptions are merely preferred examples of the invention and are not intended to limit the invention. Although the invention has been described in detail with reference to the foregoing examples, those skilled in the art can still modify the technical solutions described in the foregoing examples or make equivalent substitutions for some of the technical features. All modifications and equivalent substitutions made within the spirit and principles of the invention should be included within the scope of protection of the invention.

Claims

1. A method for training humanoid robots to safely land based on future foot contact imagination, characterized in that, Includes the following steps: S1. Set up a future foot contact visualization model during the training phase; the future foot contact visualization model is used to generate training guidance signals related to safe foot landing. S2. Input the current privileged state and current action of the humanoid robot into the future foot contact imagination model to predict the future foot contact position distribution; S3. Use the lack of support as an evaluation index for the safety of the foot landing point, and establish a rule for calculating the lack of support: For the foot position to be evaluated, obtain the local terrain height information of the corresponding foot coverage area, and calculate the degree of support loss at the foot position to be evaluated based on the local terrain height information. S4. Determine the foot landing safety guidance evaluation method based on the foot contact state: For the foot in the support phase, calculate the degree of support deficiency based on its current contact position as the foot position to be evaluated; For the foot in the swing phase, perform discrete sampling on the future contact position distribution of the foot as described in step S2, and perform weighted summation on the degree of support deficiency at each sampling position to obtain the expected degree of support deficiency under the future contact distribution. S5. Based on the degree of support deficiency at the foot end of the support phase and the expected degree of support deficiency at the foot end of the swing phase, construct a foot landing guidance signal; S6. The humanoid robot's motion control strategy is trained using the foot guidance signal, so that the humanoid robot's motion control strategy learns to correct the swing trajectory and select a sufficiently supported expected foot landing position before the foot touches the ground.

2. The method for training humanoid robots to safely land based on future foot contact imagination as described in claim 1, characterized in that, The imagined model of future foot contact is composed of a multi-layered sensory mechanism.

3. The method for training humanoid robots to safely land based on future foot contact imagination as described in claim 1, characterized in that, The current privileged state includes one or more of the following: robot body state, foot contact state, foot speed, foot position, limb position, terrain height information near the foot, and terrain height information near the body.

4. The method for training humanoid robots to safely land based on future foot contact imagination as described in claim 1, characterized in that, The future contact position distribution of the foot tip is a two-dimensional probability distribution in the current robot base coordinate system, used to characterize the future contact position of the foot tip and its prediction uncertainty; the future contact position distribution of the foot tip is a Gaussian distribution, and the future foot tip contact imagination model outputs the two-dimensional position parameters and uncertainty parameters corresponding to each foot tip. The two-dimensional position parameters are used to determine the mean of the Gaussian distribution, and the uncertainty parameters are used to determine the variance or standard deviation of the Gaussian distribution after positive value mapping.

5. The method for training humanoid robots to safely land based on future foot contact imagination as described in claim 1, characterized in that, The supervision target of the future foot contact imagination model is the first effective future contact position of the corresponding foot. The first effective future contact position is: within the same training trajectory segment, for the foot currently in the swing phase, search backward from the current time for the moment when the foot first enters the contact phase, and transform the foot position at that moment to the two-dimensional plane position obtained under the current robot base coordinate system. The future foot contact imagination model is trained using negative log-likelihood loss, so that the predicted future foot contact position distribution covers the first effective future contact position.

6. The method for training humanoid robots to safely land based on future foot contact imagination as described in claim 1, characterized in that, The foot coverage area is set according to the size of the humanoid robot's foot, and local terrain height information is obtained by sampling within the foot coverage area at a preset sampling interval; The degree of support deficiency is determined based on the proportion of sampling points within the foot coverage area that meet the support height condition, wherein the support height condition is that the height difference between the foot reference height and the terrain height of the sampling point is less than a preset height threshold; for the predicted future contact position, the maximum value of the terrain height within the corresponding foot coverage area is used as the foot reference height.

7. The method for training humanoid robots to safely land based on future foot contact imagination according to claim 1, characterized in that, When evaluating the landing safety of the foot in the swing phase, the sampling range and sampling weight of discrete sampling are determined based on the prediction uncertainty of the future contact position distribution of the foot, and the degree of support deficiency at each sampling position is weighted and summed according to the sampling weight.

8. The method for training humanoid robots to safely land based on future foot contact imagination as described in claim 1, characterized in that, For each foot end, the degree of overall support deficiency is determined based on the foot end contact state: for the foot end in the support phase, the degree of support deficiency at its current contact position is taken as the overall support deficiency degree; For the foot in the swing phase, the expected degree of support loss under the future contact distribution is taken as the comprehensive degree of support loss.

9. The method for training humanoid robots to safely land based on future foot contact imagination according to claim 8, characterized in that, The foot placement guidance signal is a reinforcement learning reward, which is constructed based on the comprehensive support deficiency of each foot end and decreases as the support deficiency increases. When the training course level is higher than the preset course level threshold, the landing guidance signal is activated.

10. The method for training humanoid robots to safely land based on future foot contact imagination according to claim 1, characterized in that, The future foot contact imagination model is only used to generate foot landing guidance signals during the training phase. After training, the humanoid robot motion control strategy obtained is used to output robot control actions based on deployment observations.

Images