A method for visual standing-up control of a humanoid robot and a humanoid robot

Abstract

The application discloses a humanoid robot vision standing-up control method and a humanoid robot, which are used for improving the stability and success rate of a standing-up action in a real scene. The method comprises the following steps: if a standing-up trigger instruction of the humanoid robot is received, a standing-up control program of the humanoid robot is called; the center of mass of the humanoid robot and preset standing-up reference trajectories of a plurality of end effectors are obtained according to the standing-up control program of the humanoid robot; perception information of a surrounding environment recognized by the humanoid robot is called; whether the surrounding terrain meets preset support conditions is judged according to the perception information of the surrounding environment; when the terrain does not meet the preset support conditions, adjusted standing-up reference trajectories of the center of mass of the humanoid robot and the plurality of end effectors are obtained according to the perception information of the surrounding environment, the center of mass of the humanoid robot and the preset standing-up reference trajectories of the plurality of end effectors, and the humanoid robot is controlled to transition from a preset posture to a double-foot standing posture.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robot control, and in particular to a method for visual standing up control of a humanoid robot and the humanoid robot. BACKGROUND

[0002] With the development of service robots, special operation robots and companion robots, humanoid robots gradually become the focus of research and application because their appearance and action mode are close to human beings. In a complex environment, a humanoid robot inevitably falls due to external impact, uneven ground or control failure. In order to ensure that the robot can continue to complete the task without human intervention, the humanoid robot needs to have the ability to stand up autonomously after falling.

[0003] In the prior art, the standing up problem of a humanoid robot after falling is mostly dependent on a pre-planned fixed reference trajectory for action control, but the trajectory is usually generated under the assumption of ideal flat terrain and is not dynamically corrected in combination with real-time information of the environment in which the robot is located. Due to the lack of visual perception ability of the surrounding environment, when the robot performs a standing up action on a terrain with height difference, slope or obstacles, it cannot adaptively adjust according to the actual support surface form, and it is easy to cause the support point to deviate, the force to be abnormal or the center of mass to be unbalanced, resulting in slipping, falling or failure in the middle of the standing up action, and the robot cannot realize fast and stable autonomous standing up. SUMMARY

[0004] The present application provides a method for visual standing up control of a humanoid robot and the humanoid robot, for improving the stability and success rate of the standing up action in a real scene.

[0005] The first aspect of the present application provides a method for visual standing up control of a humanoid robot, the humanoid robot comprising at least one visual sensor, a torso, a first upper limb and an end effector thereof, a second upper limb and an end effector thereof, a first lower limb and an end effector thereof, a second lower limb and an end effector thereof, the first upper limb and the second upper limb being connected with the torso respectively, the torso comprising a crotch, the first lower limb and the second lower limb being connected with the crotch respectively, the first lower limb and the second lower limb each comprising a thigh and a shank, the thigh and the shank being connected through a knee joint, a first included angle being formed between the thigh and the shank of the first lower limb, a second included angle being formed between the thigh and the shank of the second lower limb;

[0006] The method comprises:

[0007] If a standing up trigger instruction of the humanoid robot is received, a standing up control program of the humanoid robot is invoked;

[0008] According to the human-shaped robot standing up control program, the preset standing up reference trajectory of the center of mass of the human-shaped robot and each of the plurality of end effectors is acquired;

[0009] The perception information of the surrounding environment recognized by the human-shaped robot is called;

[0010] According to the perception information of the surrounding environment, it is judged whether the surrounding terrain meets the preset support condition;

[0011] When the terrain meets the preset support condition, the preset standing up reference trajectory of the center of mass of the human-shaped robot and each of the plurality of end effectors is directly executed to control the human-shaped robot to transition from the preset posture to the double-foot standing posture;

[0012] When the terrain does not meet the preset support condition, the preset standing up reference trajectory of the center of mass of the human-shaped robot and each of the plurality of end effectors is adjusted according to the perception information of the surrounding environment to control the human-shaped robot to transition from the preset posture to the double-foot standing posture;

[0013] The preset posture includes that the hip is in contact with the ground, the end effector of the first lower limb and / or the end effector of the second lower limb is in contact with the ground, the first included angle is α, 45°<α<180°, and the second included angle is β, 45°<β<180°.

[0014] The second aspect of the present application provides a human-shaped robot vision standing up control system, comprising:

[0015] The first calling unit is configured to call the human-shaped robot standing up control program if the human-shaped robot standing up trigger instruction is received;

[0016] The acquisition unit is configured to acquire the preset standing up reference trajectory of the center of mass of the human-shaped robot and each of the plurality of end effectors according to the human-shaped robot standing up control program;

[0017] The second calling unit is configured to call the perception information of the surrounding environment recognized by the human-shaped robot;

[0018] The judgment unit is configured to judge whether the surrounding terrain meets the preset support condition according to the perception information of the surrounding environment;

[0019] The first control unit is configured to directly execute the preset standing up reference trajectory of the center of mass of the human-shaped robot and each of the plurality of end effectors when the terrain meets the preset support condition, to control the human-shaped robot to transition from the preset posture to the double-foot standing posture;

[0020] a second control unit configured to, when the terrain does not satisfy the preset support condition, obtain an adjusted rising reference trajectory of the centroid of the humanoid robot and each of the plurality of end effectors according to the perception information of the surrounding environment, the centroid, and the preset rising reference trajectory of each of the plurality of end effectors, and control the humanoid robot to transition from the preset posture to the double-foot standing posture;

[0021] The preset posture includes that the hip part touches the ground, the end effector of the first lower limb and / or the end effector of the second lower limb touches the ground, the first included angle is a, 45° < a < 180°, and the second included angle is b, 45° < b < 180°.

[0022] The third aspect of the present application provides a humanoid robot, which comprises:

[0023] a processor, a memory, an input / output unit, and a bus;

[0024] The processor is connected with the memory, the input / output unit, and the bus;

[0025] The memory stores a program, and the processor invokes the program to execute the method for visual rising control of the humanoid robot according to the first aspect and any optional implementation of the first aspect.

[0026] The fourth aspect of the present application provides a computer readable storage medium, which stores a program, and the program executes the method for visual rising control of the humanoid robot according to the first aspect and any optional implementation of the first aspect when executed on a computer.

[0027] As can be seen from the above technical solutions, the present application has the following advantages:

[0028] After receiving the rising trigger instruction, a preset rising control program is invoked and preset reference trajectories of the centroid of the robot and each end effector are obtained, and whether the terrain satisfies a preset support condition is determined based on the surrounding environment perception information collected by the visual sensor. When the terrain satisfies the support condition, the robot can directly execute the preset trajectory to realize rapid rising. When the terrain does not satisfy the condition, the rising trajectories of the centroid and each end effector are adaptively adjusted according to the visual perception result, so as to match the actual terrain characteristics, thereby improving the rising stability and success rate of the robot in a non-ideal terrain. The present application combines visual perception and trajectory correction, so that the humanoid robot can automatically optimize the rising motion path according to the actual terrain, and improve the stability and success rate of the rising motion in a real scene. BRIEF DESCRIPTION OF DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the present application, the drawings needed to be used in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0030] Figure 1 An embodiment flowchart of a method for visual standing-up control of a humanoid robot provided by the present application is shown in the figure.

[0031] Figure 2 A schematic diagram of adjusting the humanoid robot from a fallen state to a preset posture provided by the present application is shown in the figure.

[0032] Figure 3 A schematic diagram of a support state of the humanoid robot in the standing-up process provided by the present application is shown in the figure.

[0033] Figure 4 A schematic diagram of a double-foot standing posture of the humanoid robot provided by the present application is shown in the figure.

[0034] Figure 5 An embodiment flowchart of a method for visual standing-up control of a humanoid robot provided by the present application is shown in the figure.

[0035] Figure 6 A schematic diagram of phase division of a standing-up action sequence provided by the present application is shown in the figure.

[0036] Figure 7 An embodiment flowchart of trajectory adjustment based on global terrain evaluation in a method for visual standing-up control of a humanoid robot provided by the present application is shown in the figure.

[0037] Figure 8 An embodiment flowchart of trajectory adjustment based on local terrain evaluation in a method for visual standing-up control of a humanoid robot provided by the present application is shown in the figure.

[0038] Figure 9 An embodiment structural schematic diagram of a system for visual standing-up control of a humanoid robot provided by the present application is shown in the figure.

[0039] Figure 10 An embodiment hardware structural schematic diagram of a humanoid robot provided by the present application is shown in the figure. DETAILED DESCRIPTION

[0040] The present application provides a method for visual standing-up control of a humanoid robot and a humanoid robot, which are used to improve the stability and success rate of a standing-up action in a real scene.

[0041] It should be noted that the method for visual standing up control of the humanoid robot provided in the present application is applied to a humanoid robot and is designed for a visual standing up scene of the humanoid robot, that is, the humanoid robot realizes autonomous standing up under the condition of relying on a visual sensor. The humanoid robot comprises at least one visual sensor, a trunk, a first upper limb and an end effector thereof, a second upper limb and an end effector thereof, a first lower limb and an end effector thereof, and a second lower limb and an end effector thereof, the first upper limb and the second upper limb are connected with the trunk respectively, the trunk comprises a crotch, the first lower limb and the second lower limb are connected with the crotch respectively, the first lower limb and the second lower limb each comprise a thigh and a shank, the thigh and the shank are connected through a knee joint, a first included angle is formed between the thigh and the shank of the first lower limb, and a second included angle is formed between the thigh and the shank of the second lower limb. Each end effector of the humanoid robot is provided with a force sensor or a tactile sensor for detecting a contact state and a force condition with the ground in real time. The humanoid robot further comprises a posture perception module, the posture perception module is used for acquiring posture data and a center of mass position of the humanoid robot in real time through an inertial measurement unit (IMU), an encoder and a joint angle sensor. For the convenience of description, the embodiments of the present application take a control system of the humanoid robot, referred to as a system, as an execution subject for description, but this is not a limitation on the protection scope of the present application.

[0042] Please refer to Figure 1 , Figure 1 An embodiment of the method for visual standing up control of the humanoid robot provided in the present application comprises the following steps.

[0043] 101. If a standing up trigger instruction of the humanoid robot is received, a standing up control program of the humanoid robot is called.

[0044] When the humanoid robot receives a standing up trigger instruction, a standing up control program of the humanoid robot is called. The standing up trigger instruction can be automatically generated by the robot when it detects that it is in a fallen state, or can be issued by an external control terminal, and the specific implementation is not limited herein. After receiving the standing up trigger instruction, the system will automatically load the standing up control program corresponding to the current posture to execute the subsequent standing up control process. The standing up control program comprises a standing up action sequence for controlling the movement of the center of mass of the robot and the action of the plurality of end effectors, so as to realize the control and adjustment of the standing up action.

[0045] 102. The preset standing up reference trajectories of the center of mass of the humanoid robot and the plurality of end effectors are acquired according to the standing up control program of the humanoid robot.

[0046] The preset standing-up reference trajectory in the standing-up control program includes a movement trajectory of a center of mass of the humanoid robot in the standing-up process, and movement trajectories of end effectors of both hands and both feet. The preset standing-up reference trajectory of the center of mass is used to describe a movement path of the overall center of gravity of the humanoid robot in the standing-up process, so as to ensure that the center of gravity is always in a controllable stable region. The preset standing-up reference trajectories of the end effectors are used to describe movement paths of the end of each limb of the humanoid robot in space, so as to ensure that the humanoid robot can establish a support relationship in turn according to a reasonable action rhythm in the standing-up process. The preset standing-up reference trajectory is to guide the center of mass of the humanoid robot to gradually move from a starting position near the ground to a target position in a standing posture of the feet, and the height of the target position is higher than that of the starting position, so as to realize a dynamic transition from a fallen state to an upright state. In the process, the preset standing-up reference trajectories of the center of mass and the end effectors are optimized to ensure that the center of mass has a smooth rising path and reasonable energy consumption, and to ensure that the end effectors maintain sufficient support stability at each stage.

[0047] 103. retrieving perception information of a surrounding environment recognized by the humanoid robot;

[0048] After obtaining the preset standing-up reference trajectories of the center of mass of the humanoid robot and the end effectors, the system further retrieves perception information of a surrounding environment recognized by the humanoid robot. The perception information of the surrounding environment can be collected by at least one visual sensor arranged on the body of the humanoid robot, which can include a monocular camera, a binocular camera, a depth camera, or an infrared structured light sensor, etc. Through the visual sensor, the humanoid robot can obtain image information or depth information of the current area to perceive the morphological characteristics of the surrounding terrain.

[0049] It should be noted that the perception information of the surrounding environment can not only be the environment data at the current time, but also the continuous environment perception information within a period of time before the current time (for example, from 5 seconds before standing up to the current time). The system can further perform spatial feature extraction and time sequence filtering based on the perception information data within the period of time to obtain more stable and accurate terrain description results.

[0050] 104. determining whether the surrounding terrain meets a preset support condition according to the perception information of the surrounding environment;

[0051] The system analyzes the terrain features of the area where the humanoid robot is located based on the surrounding environment perception information collected by the vision sensor to determine whether the area meets the preset support condition. Specifically, the system can perform spatial feature extraction on the image information or depth information obtained by the vision sensor, including ground normal vector extraction, surface height mapping, local flatness analysis, and inclination calculation, and determine whether the surrounding terrain meets the preset support condition according to these information. The preset support condition refers to the ground feature requirements that need to be met for the humanoid robot to establish stable support during the standing-up process. This requirement can include the flatness, inclination, support height difference, and supportable reaction force of the ground, and can be set according to the actual application scenario of the humanoid robot, which is not limited here. Among them, the commonly used indicators are flatness and inclination. Flatness is used to measure the degree of fluctuation of the ground surface. When the ground surface has large concave-convex or obvious high-low difference, the flatness indicator is low, and unintended contact is likely to occur. Inclination is used to describe the inclination angle of the ground surface relative to the horizontal plane. When the terrain slope is large, the support point is likely to be unevenly stressed, causing the center of mass to shift or the robot to lose stability.

[0052] By comparing the perception information of the surrounding environment with the preset support condition, the system can determine whether the current terrain is suitable for directly executing the preset standing-up reference trajectory, and selectively execute step 105 or step 106 according to the determination result, thereby making adaptive decisions according to the actual terrain.

[0053] 105、When the terrain meets the preset support condition, the preset standing-up reference trajectory of the center of mass of the humanoid robot and each of the plurality of end effectors is directly executed to control the humanoid robot to transition from the preset posture to the double-foot standing posture;

[0054] When the system determines that the terrain meets the preset support condition based on the surrounding environment perception information collected by the vision sensor, for example, when the humanoid robot falls on a smooth indoor floor or a flat road surface, the system can directly call the preset standing-up reference trajectory pre-planned in the standing-up control program to control the center of mass of the humanoid robot and the plurality of end effectors to move according to the trajectory sequence, thereby smoothly transitioning the robot from the preset posture to the double-foot standing posture. Through such control logic, the humanoid robot can complete the standing-up process at a higher response speed in an ideal support environment, avoiding repeated terrain analysis and trajectory calculation operations, thereby shortening the standing-up time and improving the execution efficiency of the standing-up action.

[0055] It should be noted that the preset posture specifically refers to a stable posture of the humanoid robot before performing the standing-up action, and the preset posture requires the crotch to be in contact with the ground to form a reliable initial support surface, and the end effector of at least one lower limb should be in contact with the ground. The humanoid robot should have at least three support points in the preset posture, i.e., the contact points of the crotch and the ground, and other contact points formed by the end effector of the limb, which can be formed by the end effector of the two lower limbs and the ground, or by the end effector of one lower limb and the end effector of at least one upper limb. The body of the humanoid robot can be in a sitting posture or a lying posture, and the specific embodiments are not limited here. In the preset posture, the included angles between the thighs and the shanks of the lower limbs are α and β, respectively, which need to satisfy the constraint range of 45°<α<180° and 45°<β<180°, so as to ensure that each joint has sufficient space and torque output capacity during the standing-up process.

[0056] In some specific embodiments, in addition to the preset standing-up reference trajectories of the center of mass of the humanoid robot and the plurality of end effectors, the standing-up control program can also include a preposed posture adjustment control instruction. The posture adjustment control instruction is used to smoothly adjust each joint of the humanoid robot, so that the humanoid robot is adjusted from an arbitrary falling state to a preset posture, and then the preset standing-up reference trajectories of the center of mass of the humanoid robot and the plurality of end effectors are executed on the basis of the preset posture. Through this step, it can be ensured that the subsequent execution of the standing-up control program has consistent and stable initial conditions, greatly improving the standing-up stability. The posture adjustment control instruction can be realized by a PD position controller. The PD controller calculates the output joint torque instruction JointTorque according to the deviation between the preset target joint position q_des and the real-time detected current joint position q, and the deviation between the target joint speed qd_des and the current joint speed qd. Specifically, JointTorque = Kp * (q_des - q) + Kd * (qd_des - qd). Wherein, Kp is the proportional gain, and Kd is the differential gain. In the parameter design of the PD position controller, it is preferred to use a moderate Kp value and a moderate Kd value, the purpose of which is to preferentially reduce the impact and chattering in the posture adjustment process, rather than pursuing the fastest response speed. The moderate gain ensures that the humanoid robot moves smoothly in the process of transitioning to the preset posture, avoiding secondary rigid collision with the ground or high-frequency vibration.

[0057] Please refer to Figure 2 , Figure 2Fig. 2 is a schematic diagram of the adjustment of the humanoid robot from the prone state to the preset posture. After the posture adjustment control, the upper body of the humanoid robot is gradually lifted from lying, the lower limbs are bent, and the hip is kept in contact with the ground, thereby forming a stable initial support structure, i.e., the preset posture is adjusted. Please refer to Fig. 3. Figures 3-4 , Figure 3 Fig. 4 is a schematic diagram of the support state of the humanoid robot during the standing-up process. Figure 4 Fig. 5 is a schematic diagram of the double-foot standing posture. During the entire standing-up process, the humanoid robot gradually lifts the hip off the ground through the coordinated movement of the upper limbs and the lower limbs, and finally makes the end effectors of the double lower limbs stably contact with the ground, and the torso remains vertical, thereby realizing the posture transition from the prone state to the completely standing posture.

[0058] 106. When the terrain does not satisfy the preset support condition, the adjusted standing-up reference trajectories of the centroid and the multiple end effectors of the humanoid robot are obtained according to the perception information of the surrounding environment, the centroid and the preset standing-up reference trajectories of the multiple end effectors, and the humanoid robot is controlled to transition from the preset posture to the double-foot standing posture.

[0059] When the system determines that the terrain does not satisfy the preset support condition according to the surrounding environment perception information collected by the vision sensor, if the preset standing-up reference trajectory in the standing-up control program is directly executed, it is extremely likely to cause the end effectors to miss or slip, or the robot torso to tilt due to the inconsistent support point height, and finally lead to the failure of the standing-up. At this time, the system can comprehensively utilize the perception information of the surrounding environment, and the preset standing-up reference trajectories of the centroid and the multiple end effectors, adjust the trajectory planning, and obtain the adjusted standing-up reference trajectories of the centroid and the multiple end effectors of the humanoid robot. The adjusted standing-up reference trajectory is a new trajectory that can adapt to the current environment, and through the new trajectory, the humanoid robot can be controlled to stably transition from the preset posture to the double-foot standing posture.

[0060] In the embodiment, after receiving the standing-up trigger instruction, the preset standing-up control program is called and the preset reference trajectories of the robot centroid and the end effectors are obtained, and whether the terrain satisfies the preset support condition is determined based on the surrounding environment perception information collected by the vision sensor. When the terrain satisfies the support condition, the robot can directly execute the preset trajectory to realize the rapid standing-up; and when the terrain does not satisfy the condition, the standing-up trajectories of the centroid and the end effectors are adaptively adjusted according to the vision perception result, so as to match the actual terrain characteristics, thereby improving the standing-up stability and success rate of the robot in the non-ideal terrain. Through the combination of vision perception and trajectory correction, the humanoid robot can automatically optimize the standing-up motion path according to the actual terrain, and improve the stability and success rate of the standing-up motion in the real scene.

[0061] In some specific embodiments, the humanoid robot standing up control program is configured to control the humanoid robot to perform a standing up motion sequence, the standing up motion sequence comprising a plurality of stages of standing up motions, each stage of standing up motion being controlled by a corresponding standing up control sub-program to move the center of mass and the plurality of end effectors of the humanoid robot according to a respective preset standing up reference trajectory. Specifically, the standing up motion sequence can be designed to comprise a plurality of stages of standing up motions, i.e. the standing up control program comprises a plurality of sub-programs, each stage of standing up motion being controlled by a corresponding standing up control sub-program to move the center of mass and the plurality of end effectors of the humanoid robot according to a respective preset standing up reference trajectory.

[0062] In this case, each stage in the standing up motion sequence has a clear time boundary, and each stage corresponds to an independent starting time point and an ending time point. The main basis for stage division is the change in the contact state of the end effectors of the humanoid robot, i.e. when it is detected that the contact mode of the hand or foot end effector with the ground changes relative to the previous stage, the system determines that the current stage ends and enters the next stage. Based on this, please refer to Figure 5 The embodiment proposes a contact abnormality control method, which specifically comprises:

[0063] 501. During the execution of the standing up motion sequence, the contact state signal of the end effector of the humanoid robot with the ground is monitored.

[0064] During the execution of the plurality of stages of standing up motions by the humanoid robot, the system needs to collect the interaction information between the end effector and the ground in real time, including but not limited to torque sensors, tactile sensors, joint current estimation modules, inertial measurement units (IMUs), etc., which are not limited here. In the present embodiment, the monitoring of the contact state signal is preferably realized based on joint torque abnormality detection. Specifically, the system can calculate the expected joint torque required to overcome the gravity and inertia of each limb in a non-contact state according to the dynamics model of the humanoid robot combined with the real-time position, speed and acceleration instructions of each joint of the humanoid robot. When a certain end effector is in contact with the ground, the ground will generate a reaction force, which will be transmitted upwards along the kinematic chain of the limb, causing the actual output torque of one or more joints on the limb chain to deviate from the expected joint torque. Thus, the contact state of the end effector with the ground, i.e. whether the end effector is in contact with the ground, can be determined according to the joint torque feedback on each limb of the humanoid robot.

[0065] During the whole standing-up control process, the system can compare the real-time monitored contact state signal with the pre-stored contact state reference sequence to determine whether the current action is consistent with the planned trajectory. The contact state reference sequence is used to define the timing of the contact or separation of the end effector such as the hand and foot of the humanoid robot with the ground during the whole standing-up action process. The definition of the contact reference timing enables the humanoid robot to orderly establish or release the support point at different stages. By pre-establishing the contact template during the training stage, the actual contact state can be compared and monitored during the execution process, and the corresponding mechanism is triggered when the deviation occurs to ensure the stable execution of the action. The contact state reference sequence pre-sets the touchdown time point and contact duration of each end effector, both of which are determined according to the planning of the multi-stage standing-up action sequence and are fixed timing indicators. During the execution process, the system will determine the establishment and release time of each support point according to these time parameters, thereby coordinating the force and posture changes of each end effector, making the standing-up action rhythm natural, continuous and predictable.

[0066] 502、if it is monitored that at least one end effector does not establish the target contact with the ground at the target time point, triggering contact abnormal control of the at least one end effector;

[0067] If it is monitored that the contact timing of the end effector is consistent with the reference sequence, i.e., the target contact is established with the ground at the target time point, it is considered that the standing-up action is executed normally, and the system does not intervene at this time. However, if it is monitored that at least one end effector does not establish the target contact with the ground at the target time point, it indicates that the end effector has a delayed contact, and the end effector needs to trigger the contact abnormal control of at least one end effector to establish the target contact with the ground. The delayed contact phenomenon is often caused by various reasons, such as soft or slippery support surface, blocked end effector action, or sensor delay, etc. For example, the humanoid robot identifies a flat support area through a visual sensor, but the area may be physically soft or loose (such as thick carpet, grass, gravel or stones). When the end effector reaches the target position according to the planned trajectory, the surface will deform or sink. Although the end effector reaches the target coordinate at the target time point, the force sensor does not monitor the expected support reaction force, and the system therefore determines that there is a delayed contact.

[0068] Specifically, the contact abnormal control includes: acquiring image information of a region corresponding to the current position of the end effector that does not establish the target contact with the ground at the target time point; calculating a contact compensation trajectory of the end effector based on the image information and in combination with the current position of the end effector and controlling the end effector to track the contact compensation trajectory; and ending the contact abnormal control if an effective contact signal of the end effector with the ground is monitored.

[0069] In this embodiment, when the contact abnormality control is triggered, the system first acquires the image information of the area corresponding to the current position of the end effector that fails to establish the target contact with the ground at the target time point, i.e., controls the visual sensor mounted on the head or relevant limb of the robot to align its field of view with the end effector that has the contact abnormality and the area below it. The purpose of this is to obtain real-time image information about the local small range area. Then, according to the real-time image information combined with the current position of the end effector, the contact compensation trajectory of the end effector is calculated. For example, if it is found from the area image information that the actual ground is 5 cm lower than expected, then the contact compensation trajectory is a trajectory that probes 5 cm in the negative direction of the Z axis. The system then controls the end effector to track this contact compensation trajectory. During the tracking process, if the effective contact signal between the end effector and the ground is monitored, the contact abnormality control is ended.

[0070] 503、After the contact abnormality control of at least one end effector is ended, the remaining action sequence of the multi-stage standing-up action sequence corresponding to the at least one end effector is executed until the humanoid robot reaches the double-foot standing posture.

[0071] After the contact abnormality control is ended, the system continues to execute the remaining action sequence of the multi-stage standing-up action of the at least one end effector and continuously monitors the contact state between the end effector of the humanoid robot and the ground. As the subsequent actions continue to be executed, the humanoid robot gradually enters the final double-foot support stage. In the final stage, when the system detects that the double-foot end effector is in stable contact with the ground and the center of mass height reaches the target value of the reference trajectory, it is determined that the standing-up process is completed and the humanoid robot successfully recovers to the double-foot standing posture. At this time, the hip, knee, and ankle angles are all in a balanced state, the center of mass is located in the central region of the double-foot support surface, and the humanoid robot has a stable foundation to continue to execute walking or standing tasks.

[0072] In this embodiment, a standing-up action sequence including multiple stages is introduced, and a closed-loop feedback mechanism for continuously monitoring whether the end effector successfully establishes the target contact is combined. By actively monitoring whether the end effector of the humanoid robot successfully establishes contact with the physical world in each standing-up stage, once an abnormality is detected, the system triggers the contact abnormality control, actively calls the visual sensor to acquire real-time image information of the abnormal area, and calculates an accurate contact compensation trajectory based on the visual information. After the contact abnormality control is ended, the system will continue to execute the remaining action sequence until the double-foot standing posture is reached. This mechanism can effectively suppress errors in subsequent actions caused by slipping or missing of the supporting hand / foot, thereby avoiding accidental interruption or complete failure of the standing-up process, ensuring that the humanoid robot can reliably correct local errors and successfully implement standing-up even in complex situations with perception errors.

[0073] Referring to Figure 6 , Figure 6 is a schematic diagram for dividing the stages of the rising motion sequence. In some specific embodiments, the rising motion sequence includes at least the following stages: single-hand support stage, single-hand-single-leg support stage, double-hand-single-leg support stage, double-hand-double-leg support stage, and double-leg support stage. The stages are described in detail as follows:

[0074] 1. Single-hand support stage: This stage is to establish the initial support from the preset posture (i.e., the rising preparation posture). The system controls the first upper limb of the humanoid robot to move so that its end effector establishes stable contact with the ground, thereby forming a first support point on the ground. In this stage, the system also synchronously controls the torso of the humanoid robot to turn sideways toward the first support point. The purpose is to actively shift the center of gravity of the humanoid robot toward the first support point, thereby creating space for the subsequent movement of the non-supporting side of the body and ensuring posture stability.

[0075] 2. Single-hand-single-leg support stage: This stage is to increase the support of the lower limbs to raise the center of gravity on the basis of the first support point. While maintaining the stable support of the first upper limb (first support point), the system controls the first lower limb (i.e., the lower limb on the same side as the first upper limb) of the humanoid robot to move. Specifically, the end effector of the first lower limb moves from an initial first position to a second position that is closer to the hip of the humanoid robot than the first position, i.e., performs a “leg folding” action. When the end effector of the first lower limb establishes stable contact with the ground at the second position, a second support point is established. It should be noted that during the movement in this stage, to coordinate with the “leg folding” and raise the body, the knee joint of the first lower limb moves away from the ground (i.e., bends), while the knee joint of the second lower limb moves toward the ground. This set of coordinated movements makes the first angle between the thigh and the lower leg of the first lower limb smaller than the second angle between the thigh and the lower leg of the second lower limb (i.e., the first lower limb bends more), thereby creating a favorable geometric configuration for the subsequent raising of the center of gravity and body rotation.

[0076] 3. Two hands and one foot supporting stage: This stage is to establish three points support and prepare for the landing of both feet. The system first controls the second upper limb (i.e. the upper limb of the non-supporting side) of the humanoid robot to move, so that its end effector contacts the ground, thereby establishing a third support point, forming a stable three-point support structure of two hands and one foot. On this basis, the system further controls the second lower limb (i.e. the leg of the non-supporting side) of the humanoid robot to move, so that its end effector moves from the current third position to the fourth position, which is also closer to the waist of the humanoid robot than the third position. During the movement of the second lower limb, the angle between its thigh and calf will gradually decrease (i.e. the leg is gradually bent and folded), and the purpose of this action is to cooperate with the upper body, so that the overall center of gravity of the humanoid robot is inclined forward, preparing for the subsequent transfer of the center of gravity to both feet and lifting the waist.

[0077] 4. Two hands and two feet supporting stage: This stage is the key stage for the humanoid robot to transition from a sitting position to a four-point support. While maintaining the stability of the first, second and third support points that have been established, the system performs two coordinated actions: first, it controls the waist of the humanoid robot to move upward, so that it moves from the fifth position (lower height) to the sixth position, and the height of the sixth position from the ground is greater than that of the fifth position from the ground, i.e. it performs an action similar to "lifting the hips". Second, while lifting the waist, the system controls the end effector (foot) of the second lower limb that has been folded in the previous stage to contact the ground, thereby establishing the fourth support point. After this stage is completed, the humanoid robot forms a stable two hands and two feet four-point support state.

[0078] 5. Two feet supporting stage: This stage is the final stage of the getting-up action, and the goal is to transition to an upright standing position. Based on the "two hands and two feet" four-point support, the system controls the center of mass position of the humanoid robot to transfer from the current seventh position (lower squat center of gravity) to the eighth position (standing center of gravity), and the height of the eighth position from the ground is greater than that of the seventh position from the ground. During the process of lifting and moving forward of the center of mass, as the center of gravity gradually transfers and stabilizes within the two-foot support surface, the system will control the end effector of the first upper limb and the end effector of the second upper limb to separate from the ground. Finally, the humanoid robot relies only on the support of both feet, and completes the entire transition from falling to a standing position with both feet.

[0079] By decomposing the standing-up motion sequence into multiple stages such as single-hand support, single-hand single-leg support, double-hand single-leg support, double-hand double-leg support, and double-leg support, the stability and controllability of the standing-up process can be significantly improved. By decomposing the complex whole-body motion into a series of sub-motions with stable intermediate postures and controlling them through corresponding standing-up control sub-programs, it can be ensured that the humanoid robot always maintains a reliable support polygon during the transition from the preset posture to the final double-leg standing posture. This progressive motion design achieves smooth and progressive lifting of the center of mass, for example, the system uses the support points established by the upper limbs to assist in lifting the hips, rather than relying solely on the lower limbs or waist and abdomen to perform high-torque, high-risk explosive actions. This design avoids sudden changes in posture and instability caused by rapid movement of the center of mass, ensuring that the humanoid robot can safely transfer the center of mass from near the ground to the double-foot support surface in a predictable, low-impact, and efficient manner, thereby greatly improving the robustness and success rate of the standing-up motion.

[0080] The specific ways of determining whether the surrounding terrain meets the preset support condition in step 104 and obtaining the adjusted standing-up reference trajectory in step 106 are further described in detail below.

[0081] In a preferred embodiment, the overall terrain can be evaluated before the standing-up motion is performed, and potential risk areas can be identified in advance to avoid planning the standing-up motion on unstable support surfaces, thereby improving the overall safety and success rate of motion execution. Please refer to Figure 7 , Figure 7 The method is a trajectory adjustment method based on global terrain evaluation, and the method comprises the following steps:

[0082] 701. Calculate the flatness and inclination of the terrain around the position of the humanoid robot according to the perception information of the surrounding environment, and determine whether the flatness and inclination meet the first preset support condition;

[0083] After processing the perception information of the surrounding environment collected by the vision sensor, the terrain height distribution data of the area where the robot is located can be obtained. The system calculates the flatness and inclination of the terrain based on the terrain height distribution data to measure whether the ground has stable support ability. Among them, the flatness is used to describe the uniformity of the height change of the ground surface, when the height difference of the local area is small, and there is no obvious depression or protrusion, the flatness index is higher; the inclination is used to represent the inclination angle of the ground surface relative to the horizontal plane, when the inclination angle is less than the set threshold, the robot can establish stable support in the area.

[0084] The system compares the calculated flatness and inclination values with the stored first preset support condition respectively. When both indicators are within the allowed range, it indicates that the current terrain is flat enough to directly execute the preset standing-up trajectory. If either indicator exceeds the threshold, it indicates that the current terrain has high and low relief or is too steep, which is not suitable for direct standing-up, and subsequent steps 702 and 703 need to be executed. For example, when the humanoid robot falls on the edge of a slope or on a staircase, the system can identify that the terrain flatness is poor or the inclination is large, and at this time the system judges that the current area does not meet the first preset support condition, thereby triggering the identification process of the contact candidate area.

[0085] It should be noted that the first preset support condition is used for global terrain evaluation, and the purpose is to judge whether the current terrain has the basic support characteristics of the executable standing-up action in a macroscopic range. The first preset support condition is specifically used to judge whether the current falling area of the robot meets the environmental conditions for direct standing-up.

[0086] 702、When the flatness and inclination do not meet the first preset support condition, identify at least one contact candidate area for each end effector, the contact candidate area being a local area of a preset area size;

[0087] When it is detected that the flatness and inclination do not meet the first preset support condition, the system locally divides and analyzes the terrain within the reachable range of the robot based on the perception information. In this embodiment, the system identifies the contact candidate area within the reachable range of each end effector. Each contact candidate area is a local flat area of a preset area size, and the preset area size can be set according to the size of the end effector and the force requirement of the standing-up action, which is not limited here. For example, when the humanoid robot falls on the ground with uneven stone blocks, the system can identify several relatively flat small areas in its surrounding area and mark these areas as contact candidate areas.

[0088] Further, the system can also calculate the characteristics of each contact candidate area using the perception information, including average height, surface inclination direction, texture uniformity, etc., to evaluate whether the contact candidate area has the ability to form a stable contact. By screening and sorting multiple contact candidate areas, the system can provide feasible support point selection basis for subsequent trajectory adjustment.

[0089] 703、According to the contact candidate area, the center of mass and the preset standing-up reference trajectory of each end effector, an adjusted standing-up reference trajectory of the center of mass and each end effector is obtained.

[0090] After the contact candidate regions are determined, the system recalculates the optimal standing-up reference trajectory based on the current center of mass position of the robot and the preset standing-up trajectory. Specifically, the system selects a contact candidate region for each end effector, and then adjusts the original preset standing-up trajectory to generate an adjusted standing-up reference trajectory for the center of mass and each end effector. The adjusted standing-up reference trajectory enables the robot to complete the standing-up action at the new support point without breaking the overall action coherence, thereby ensuring the stability and executability of the action. For example, when the system detects that there is an inclination or gap under the originally designated support point of the left foot of the humanoid robot, the system can re-determine the support point position in a flat area near the left foot and re-calculate the movement path of the center of mass to rebalance the overall center of gravity distribution. The humanoid robot will perform the standing-up action according to the adjusted trajectory, and can stably complete the standing-up process even if the terrain is uneven.

[0091] In some specific embodiments, after the global terrain assessment is performed, when the system determines that the current terrain does not meet the first preset support condition, the trajectory adjustment process is entered. The trajectory adjustment process screens the contact candidate regions and recalculates the standing-up trajectory of the center of mass and the end effectors according to the screening results to ensure that the robot can still achieve stable standing-up in a non-planar terrain environment. The trajectory adjustment process specifically includes:

[0092] A1, screening a target support region that meets a second preset support condition in terms of flatness and inclination in the contact candidate region;

[0093] In this embodiment, the system first performs secondary screening on the identified multiple contact candidate regions to find a target support region that meets more stringent support requirements. The second preset support condition is used for screening, which is a terrain constraint condition with higher standards than the first preset support condition, and is used to ensure that the support point established in the robot standing-up action has higher stability. The second preset support condition is specifically used to further screen a target support region that can be used to establish a support point in a local range.

[0094] For example, when the robot falls on a sandy road, there may be both rough areas and relatively smooth stone plate areas in the contact candidate region, and the system will preferentially select the stone plate surface as the target support region, thereby providing a more reliable support point for subsequent trajectory adjustment.

[0095] A2, when the target support region is detected, obtaining an adjusted standing-up reference trajectory for the center of mass and each end effector based on the height difference of the target support region, the center of mass, and the preset standing-up reference trajectory of each end effector;

[0096] When the system screens at least one target support area that meets the second preset support condition, the original standing-up trajectory is dynamically adjusted according to the topographic information of the target support area and the spatial position relationship with each end effector of the robot. During the adjustment process, the system calculates the height difference between the target support area and the preset support point, and adjusts the movement path of the corresponding end effector to achieve stable contact at the new support height. At the same time, the system will also reassign the movement path of the center of mass according to the change of the support point to maintain the balance and coordination of the overall posture. For example, when the ground at the originally designated support point of the right foot of the humanoid robot is slightly concave, and there is a target support area near the right side that is slightly higher but flat, the system will adjust the contact position of the right foot to the high point, and slightly offset the target position of the center of mass to the right to maintain the overall force balance. In this way, the humanoid robot can adaptively correct the support point and action trajectory in uneven terrain, significantly improving the success rate and balance stability of the standing-up action.

[0097] A3, when it is detected that there is no target support area, identifying a non-planar support object in the contact candidate area;

[0098] In some complex environments, for example, when there are only curved objects (such as stones, cylinders or inclined panels) near the position of the humanoid robot falling down and no obvious flat area, the system cannot screen a target support area that meets the second preset support condition. At this time, the system can further identify a non-planar support object in the candidate area. The non-planar support object refers to a three-dimensional support structure with a certain height and shape, whose surface does not meet the requirement of a plane, but can still provide stable support through local contact. For example, the top of a stone, the local protrusion of a slope or the edge of furniture can all be non-planar support objects.

[0099] A4, judging whether the non-planar support object meets the support stability condition according to the surface inclination, support height and friction characteristics of the non-planar support object;

[0100] After identifying the non-planar support object, the system will determine whether the non-planar support object meets the support stability condition according to the surface inclination, support height, and friction characteristics of the non-planar support object. The support stability condition refers to the constraint condition that the support surface or support object can withstand the local load of the robot after contact and maintain stable contact without slipping or toppling. The system calculates the support height, surface inclination, and friction characteristics of the non-planar support object based on the perception information identified by the vision sensor. When the analysis results show that the inclination angle of the support object is small and the surface roughness can provide sufficient friction, it is determined that the support stability condition is met. For example, when a humanoid robot falls on a gravel road, there is a raised stone block whose surface is not completely flat, but its inclination angle is small and the surface friction is large. The system can consider the top of the stone block as a usable support object for a certain end effector to establish a contact point.

[0101] A5、When the non-planar support object meets the support stability condition, based on the support height, center of mass, and preset rising reference trajectory of each end effector of the non-planar support object, the adjusted rising reference trajectory of the center of mass and each end effector is obtained.

[0102] When the judgment result shows that the non-planar support object can provide effective support, the system will adjust the rising reference trajectory of the end effector and the center of mass with the surface height of the support object as the constraint condition to ensure overall balance.

[0103] Through the above steps, the robot can not only achieve stable rising in a planar terrain, but also can flexibly select a support object to complete the action in an environment with complex terrain characteristics, thereby realizing autonomous rising across terrains. Compared with the traditional fixed trajectory rising method, the rising trajectory planning of the present embodiment is more intelligent, stable, and robust.

[0104] Further, in certain non-rigid or loose terrain, the ground state can change slightly with force or collapse. For example, when the robot falls on a thick carpet, gravel or sandy ground, the end effector will cause local subsidence or support surface tilt after contacting the ground. In order to deal with this situation, after screening the target support area, and during the execution of the standing up action sequence, the system can monitor the changes of the target support area in real time. When the system detects that the support height or tilt angle of the target support area has changed, it immediately acquires updated terrain images or depth data through the vision sensor, re-identifies the target support area of the end effector, and calculates the adjusted standing up reference trajectory based on the updated target support area. For example, when the robot is in the process of standing up, its left hand originally supports the soft carpet surface, and due to the action of gravity, the area subsides. The system detects that the height of the target support area decreases and the tilt angle increases. The system updates the target support area of the left hand accordingly, and adjusts the standing up reference trajectory of the end effector to keep the overall posture balanced. Through this dynamic adjustment, the robot can continue to complete the standing up in the case of terrain deformation, without being interrupted or falling down due to the collapse of the support surface.

[0105] In another preferred embodiment, in order to further improve the adaptive ability of humanoid robots in complex environments, the system can adjust the trajectory based on local terrain assessment in addition to global terrain assessment. The local terrain assessment has the characteristics of fast response speed, small adjustment range, strong real-time performance, and can only correct the local abnormalities of a certain support point quickly. Please refer to Figure 8 , Figure 8 A trajectory adjustment method based on local terrain assessment, the method comprises:

[0106] 801, dividing the terrain around the location of the humanoid robot into multiple local areas according to the perception information of the surrounding environment;

[0107] In this embodiment, after obtaining the perception information of the surrounding environment, the system first divides the terrain data around the robot in space to obtain multiple local areas. Among them, the local area refers to a plurality of fixed size ground cells divided within the reachable range of the current posture of the robot. Each local area corresponds to a limited spatial range, for example, a square or rectangular grid with the robot's center of mass as the center and 0.2-0.5 meters as the side length. Through this division method, the system can convert complex three-dimensional terrain data into several structured areas for analysis and evaluation one by one.

[0108] 802, calculating the preset contact points of each end effector according to the current position of the humanoid robot and the preset standing up reference trajectory in the standing up control program;

[0109] After the terrain division is completed, the system calculates the preset contact points of each end effector on the ground during the standing-up action according to the preset standing-up reference trajectory in the standing-up control program and in combination with the current posture of the humanoid robot. The preset contact point refers to a standard support point position planned to ensure the stability of the standing-up action under ideal planar conditions.

[0110] 803. determining whether the flatness and the inclination of the local region where the preset contact point is located satisfy a third preset support condition;

[0111] The system maps each preset contact point into the divided local region and analyzes the flatness and the inclination of the local region to determine whether the third preset support condition is satisfied. The third preset support condition is used to evaluate whether a single local region can provide reliable support under the current posture. If the third preset support condition is satisfied, the preset standing-up reference trajectory of the end effector can be directly used, i.e., no adjustment of the standing-up trajectory of the end effector is needed. If the third preset support condition is not satisfied, step 804 is performed to adjust and correct the standing-up trajectory of the end effector.

[0112] 804. If not, a replaceable support region is searched in other local regions adjacent to the preset contact point, and an adjusted standing-up reference trajectory of the centroid and the plurality of end effectors is obtained based on the perception information, the centroid, and the preset standing-up reference trajectory of the plurality of end effectors of the replaceable support region.

[0113] When the system determines that a local region where a preset contact point is located does not satisfy the third preset support condition, a replaceable support region is searched in the adjacent region. A region with higher flatness and smaller inclination in the search result is selected as a new support region, and then the system adjusts the motion trajectory of the corresponding end effector according to the terrain information of the support region and recalculates the moving path of the centroid to ensure the balance and coordination of the entire standing-up process. For example, when the originally planned contact region of the left foot of the humanoid robot is located at the edge of a slope with an excessively large inclination, the system reselects a support point in a flat region in front of or outside the left foot and adjusts the motion trajectory of the left leg so that the landing position is located in the new support region.

[0114] In this embodiment, through the mechanism of local terrain evaluation, the humanoid robot can perform real-time adjustment for specific contact points during the standing-up process without recalculating the global trajectory to complete action optimization. This local correction method has higher response speed and calculation efficiency compared with global re-planning.

[0115] The system for visual standing-up control of the humanoid robot provided in the present application is described in detail below. Please refer to Figure 9 , Figure 9An embodiment of a system for visual standing-up control of a humanoid robot is provided in the present application, and the system comprises:

[0116] A first calling unit 901 is configured to call a standing-up control program of the humanoid robot if a standing-up trigger instruction of the humanoid robot is received;

[0117] An acquisition unit 902 is configured to acquire a center of mass of the humanoid robot and preset standing-up reference trajectories of a plurality of end effectors of the humanoid robot according to the standing-up control program of the humanoid robot;

[0118] A second calling unit 903 is configured to call perception information of a surrounding environment recognized by the humanoid robot;

[0119] A judging unit 904 is configured to judge whether a surrounding terrain satisfies a preset support condition according to the perception information of the surrounding environment;

[0120] A first control unit 905 is configured to directly execute the center of mass of the humanoid robot and the preset standing-up reference trajectories of the plurality of end effectors of the humanoid robot to control the humanoid robot to transition from a preset posture to a double-foot standing posture when the terrain satisfies the preset support condition;

[0121] A second control unit 906 is configured to acquire adjusted standing-up reference trajectories of the center of mass of the humanoid robot and the plurality of end effectors of the humanoid robot according to the perception information of the surrounding environment, the center of mass of the humanoid robot and the preset standing-up reference trajectories of the plurality of end effectors of the humanoid robot, and control the humanoid robot to transition from the preset posture to the double-foot standing posture when the terrain does not satisfy the preset support condition;

[0122] The preset posture comprises: a hip touching the ground, an end effector of a first lower limb and / or an end effector of a second lower limb touching the ground, a first included angle α satisfying 45° < α < 180°, and a second included angle β satisfying 45° < β < 180°.

[0123] In the system of the embodiment, the functions of each unit correspond to the steps in the method embodiments shown in the foregoing Figure 1 、 Figures 6-8 , and thus will not be described here again.

[0124] The present application also provides a humanoid robot, please refer to Figure 10 , Figure 10 An embodiment of a humanoid robot is provided in the present application, and the humanoid robot comprises:

[0125] A processor 1001, a memory 1002, an input / output unit 1003, and a bus 1004;

[0126] The processor 1001 is connected with the memory 1002, the input / output unit 1003, and the bus 1004;

[0127] The memory 1002 stores a program, and the processor 1001 invokes the program to execute the method of visual standing control of the humanoid robot.

[0128] The application also relates to a computer readable storage medium, which stores a program, and when the program is run on a computer, the computer executes the method of visual standing control of the humanoid robot.

[0129] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0130] In the several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are only schematic. The division of the units is only a logical function division. There can be another division manner for actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between the units can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrical, mechanical or in other forms.

[0131] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, can be located in one place, or can be distributed on a plurality of network units. According to actual needs, some or all of the units can be selected to achieve the purpose of the embodiment scheme.

[0132] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present alone, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0133] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in other words, the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the various embodiments of the method of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, read-only memory), a random access memory (RAM, random access memory), a magnetic disk or an optical disk, and various media that can store program codes.

Claims

1. A method for visual control of standing up in a humanoid robot, characterized in that, The humanoid robot includes at least one visual sensor, a torso, a first upper limb and its end effector, a second upper limb and its end effector, a first lower limb and its end effector, and a second lower limb and its end effector. The first upper limb and the second upper limb are respectively connected to the torso. The torso includes a hip. The first lower limb and the second lower limb are respectively connected to the hip. The first lower limb and the second lower limb each include a thigh and a calf. The thigh and the calf are connected by a knee joint. A first angle is formed between the thigh and the calf of the first lower limb, and a second angle is formed between the thigh and the calf of the second lower limb. The method includes: If a command to make the humanoid robot stand up is received, the humanoid robot standing up control program is invoked. The humanoid robot's center of mass and the preset standing reference trajectories of each of the multiple end effectors are obtained according to the humanoid robot standing control program. Retrieve the sensory information of the surrounding environment as perceived by the humanoid robot; Based on the perceived information of the surrounding environment, determine whether the surrounding terrain meets the preset support conditions; When the terrain meets the preset support conditions, the preset standing reference trajectory of the humanoid robot's center of mass and multiple end effectors is directly executed to control the humanoid robot to transition from a preset posture to a bipedal standing posture. When the terrain does not meet the preset support conditions, the humanoid robot's center of mass and the adjusted standing reference trajectories of the humanoid robot are obtained based on the perceived information of the surrounding environment, the center of mass, and the preset standing reference trajectories of the multiple end effectors, and the humanoid robot is controlled to transition from the preset posture to the bipedal standing posture. The preset posture includes: the hip touching the ground, the end effector of the first lower limb and / or the end effector of the second lower limb touching the ground, the first included angle being α, 45° < α < 180°, and the second included angle being β, 45° < β < 180°.

2. The method according to claim 1, characterized in that, The humanoid robot standing control program is used to control the humanoid robot to execute a sequence of standing actions. The sequence of standing actions includes multiple stages of standing actions. Each stage of standing action is controlled by a corresponding standing control subroutine to move the humanoid robot's center of mass and multiple end effectors according to their respective preset standing reference trajectories. The method further includes: During the execution of the standing action sequence, the contact status signal between the end effector of the humanoid robot and the ground is monitored; If it is detected that at least one of the end effectors fails to establish target contact with the ground at the target time point, contact anomaly control of at least one of the end effectors is triggered; After the contact anomaly control of at least one of the end effectors ends, the remaining action sequence of the multi-stage standing action sequence corresponding to the at least one end effector is executed until the humanoid robot reaches a bipedal standing posture.

3. The method according to claim 2, characterized in that, The contact anomaly control includes: Acquire image information of the area corresponding to the current position of the end effector that has not established target contact with the ground at the target time point; Based on the image information and the current position of the end effector, the contact compensation trajectory of the end effector is calculated and the end effector is controlled to track the contact compensation trajectory; If a valid contact signal is detected between the end effector and the ground, the contact anomaly control is terminated.

4. The method according to claim 1, characterized in that, The humanoid robot standing control program is used to control the humanoid robot to execute a sequence of standing actions. The sequence of standing actions includes multiple stages of standing actions. Each stage of standing action is controlled by a corresponding standing control subroutine to move the center of mass of the humanoid robot and multiple end effectors according to their respective preset standing reference trajectories. The multiple stages of standing actions include at least: a single-hand support stage, a single-hand and single-foot support stage, a double-hand and single-foot support stage, a double-hand and double-foot support stage, and a double-foot support stage. The single-hand support phase includes moving the first upper limb of the humanoid robot so that its end effector contacts the ground, thereby establishing a first support point on the ground; The single-hand, single-leg support phase includes, while maintaining support at the first support point, moving the first lower limb of the humanoid robot, causing the end effector of the first lower limb to move from a first position to a second position. The second position is closer to the humanoid robot's hip than the first position, and the end effector of the first lower limb contacts the ground at the second position, thereby establishing a second support point. During the single-hand, single-leg support phase, the knee joint of the first lower limb moves away from the ground, forming a first angle between the thigh and lower leg of the first lower limb, and a second angle between the thigh and lower leg of the second lower limb, the second angle being greater than the first angle. The two-handed, one-legged support phase includes moving the humanoid robot's second upper limb so that its end effector contacts the ground, thereby establishing a third support point; and moving the humanoid robot's second lower limb so that the end effector of the second lower limb moves from a third position to a fourth position, the fourth position being closer to the humanoid robot's hip than the third position; during the process of the second lower limb moving from the third position to the fourth position, the second angle formed between the thigh and calf of the second lower limb gradually decreases; The two-handed and two-footed support phase includes, while maintaining support at the first, second, and third support points, moving the hips of the humanoid robot from the fifth position to the sixth position, where the sixth position is higher than the fifth position. At the same time, the second lower limb of the humanoid robot is moved so that its end effector contacts the ground, thereby establishing the fourth support point. The bipedal support phase includes controlling the humanoid robot's center of gravity to shift from a seventh position to an eighth position. The height of the eighth position above the ground is greater than that of the seventh position. During the shift of the humanoid robot's center of gravity from the seventh position to the eighth position, the end effectors of the first and second upper limbs separate from the ground.

5. The method according to claim 1, characterized in that, The step of determining whether the surrounding terrain meets the preset support conditions based on the perceived information of the surrounding environment includes: The flatness and inclination of the terrain around the humanoid robot's location are calculated based on the perceived information of the surrounding environment, and it is determined whether the flatness and inclination meet the first preset support conditions. The step of obtaining the humanoid robot's center of mass and the adjusted standing reference trajectories of each of the multiple end effectors based on the perceived information of the surrounding environment, the center of mass, and the preset standing reference trajectories of each of the multiple end effectors includes: When the flatness and tilt do not meet the first preset support conditions, at least one contact candidate area of ​​each of the plurality of end effectors is identified, and the contact candidate area is a local area of ​​a preset area size; Based on the contact candidate area, the centroid, and the preset stand-up reference trajectories of each of the multiple end effectors, the adjusted stand-up reference trajectories of the centroid and the multiple end effectors are obtained.

6. The method according to claim 5, characterized in that, The step of obtaining the adjusted stand-up reference trajectories of the centroid and each of the multiple end effectors based on the contact candidate region, the centroid, and the preset stand-up reference trajectories of the multiple end effectors includes: Within the contact candidate area, target support areas that meet the second preset support conditions in terms of flatness and inclination are selected; When the target support area is detected, the center of mass and the adjusted standing reference trajectory of each of the multiple end effectors are obtained based on the height difference of the target support area, the center of mass, and the preset standing reference trajectory of each of the multiple end effectors.

7. The method according to claim 6, characterized in that, The method further includes: When the absence of the target support area is detected, a non-planar support object within the contact candidate area is identified; Based on the surface inclination, support height, and friction characteristics of the non-planar support object, determine whether the non-planar support object meets the support stability conditions; When the non-planar support object meets the support stability condition, the center of mass and the preset standing reference trajectory of each of the multiple end effectors are obtained based on the support height, center of mass, and preset standing reference trajectory of the non-planar support object.

8. The method according to claim 6, characterized in that, The method further includes: Real-time monitoring of changes in the target support area; When a change in the support height or tilt angle of the target support area is detected, the updated target support area is obtained through the vision sensor, and the adjusted standing reference trajectory of the end effector is obtained based on the updated target support area.

9. The method according to claim 1, characterized in that, The step of obtaining the adjusted stand-up reference trajectories of the center of mass and each of the multiple end effectors based on the perceived information of the surrounding environment, the center of mass, and the preset stand-up reference trajectories of the multiple end effectors includes: Based on the perceived information of the surrounding environment, the terrain around the location of the humanoid robot is divided into multiple local areas; Based on the current position of the humanoid robot and the preset standing reference trajectory in the standing control program, calculate the preset contact point of each of the multiple end effectors; Determine whether the flatness and inclination of the local area where the preset contact point is located meet the third preset support condition; If not, search for alternative support areas in other local areas near the preset contact point, and obtain the adjusted standing reference trajectories of the center of mass and multiple end effectors based on the sensing information of the alternative support areas, the center of mass, and the preset standing reference trajectories of multiple end effectors.

10. A humanoid robot, characterized in that, The humanoid robot includes: Processor, memory, input / output units, and bus; The processor is connected to the memory, the input / output unit, and the bus; The memory stores a program, which the processor invokes to perform the method as described in any one of claims 1 to 9.

Images