Humanoid robot whole body motion control method and device, and storage medium

Abstract

A humanoid robot full-body motion control method and device and storage medium, relate to the technical field of humanoid robot motion control. The method comprises: obtaining the state observation value of the humanoid robot; determining the upper limb joint angle command of the humanoid robot according to the state observation value and the pre-trained upper limb control strategy; determining the lower limb joint angle command of the humanoid robot according to the state observation value, the upper limb joint angle command and the pre-trained lower limb control strategy; and controlling the corresponding joint end of the humanoid robot according to the state observation value, the upper limb joint angle command and the lower limb joint angle command to control the full-body motion of the humanoid robot. In the determination process of the lower limb joint angle command, the joint angle information corresponding to the upper limb motion is introduced, and the lower limb control strategy can obtain information representing the current upper limb motion demand.

Description

Technical Field

[0001] This application relates to the field of humanoid robot motion control technology, and more specifically, to a method, device, and storage medium for controlling the whole-body motion of a humanoid robot. Background Technology

[0002] Humanoid robots typically possess multi-jointed mechanical structures that mimic human movement patterns, and their motion control involves coordinated control of multiple moving parts, including upper limb joints, lower limb joints, and the torso. Compared to robots that control the motion of only a single limb, humanoid robots, when performing tasks such as walking, turning, manipulating, or simultaneously walking and manipulating, must simultaneously meet requirements such as motion stability, motion tracking accuracy, and real-time joint control. Therefore, the whole-body motion control of humanoid robots usually requires generating corresponding joint control commands based on the robot's current motion state and applying these commands to the corresponding joint ends.

[0003] In related technologies, a method for controlling the whole-body motion of a humanoid robot can configure control strategies separately for different body parts. For example, an upper limb control strategy can be set for the robot's upper limb movements to determine the upper limb joint actions based on the robot's current state; a lower limb control strategy can be set for the robot's lower limb movements to determine the lower limb joint actions based on the robot's current state, and then the control results corresponding to different body parts are sent to the corresponding joints for execution. For upper limb control, a robot model-based control method or a machine learning-based control strategy can be used; for lower limb control, a control strategy that can output lower limb actions based on the robot's state can also be obtained through strategy learning.

[0004] However, when a humanoid robot simultaneously performs upper limb manipulations and lower limb movements, these movements are not entirely independent. When the upper limbs perform actions, their joint movements alter the robot's overall center of mass position, momentum, and other states. Simultaneously, the lower limbs' movement speed and support state also affect the robot's overall stability during upper limb manipulations. If the upper limb and lower limb control strategies generate control results independently without fully considering their kinematic relationship, changes in upper limb movements may impact the stability of lower limb movements, or the lower limbs may be unable to adapt to the dynamic changes caused by different upper limb movements, thereby reducing the robot's overall motion stability and the effectiveness of its actions.

[0005] The content of the background section is merely technology known to the public and does not necessarily represent existing technology in the field. Summary of the Invention

[0006] This application aims to provide a method, device, and storage medium for controlling the whole-body motion of a humanoid robot, in order to solve at least one of the above-mentioned technical problems.

[0007] According to one aspect of this application, a method for controlling the whole-body motion of a humanoid robot is provided. The method includes: acquiring state observations of the humanoid robot; determining upper limb joint angle commands of the humanoid robot based on the state observations and a pre-trained upper limb control strategy; determining lower limb joint angle commands of the humanoid robot based on the state observations, upper limb joint angle commands, and a pre-trained lower limb control strategy; and controlling the corresponding joint ends of the humanoid robot based on the state observations, upper limb joint angle commands, and lower limb joint angle commands to control the whole-body motion of the humanoid robot.

[0008] According to some embodiments of this application, the upper limb joint angle command of a humanoid robot is determined based on state observations and a pre-trained upper limb control strategy, including: inputting state observations into the upper limb control strategy to output upper limb joint angle commands; and updating the upper limb joint angle commands to the data cache.

[0009] According to some embodiments of this application, the lower limb joint angle command of a humanoid robot is determined based on state observations, upper limb joint angle commands, and a pre-trained lower limb control strategy. This includes: inputting state observations and upper limb joint angle commands into the lower limb control strategy to output lower limb joint angle commands; and updating the lower limb joint angle commands to the data cache.

[0010] According to some embodiments of this application, before obtaining the state observations of the humanoid robot, the method further includes: pre-training a lower limb control strategy. The pre-training lower limb control strategy includes: establishing a training environment including a terrain model and a humanoid robot model in a dynamic simulation environment; determining training observations based on the humanoid robot state in the training environment, the current desired motion speed, and the current desired upper limb joint angle trajectory; determining the current upper limb course difficulty and the current lower limb course difficulty based on the current desired motion speed and the current desired upper limb joint angle trajectory; determining lower limb movements based on the training observations and the initial lower limb control strategy under the current upper limb course difficulty and the current lower limb course difficulty; driving the humanoid robot movement in the training environment based on the lower limb movements and the current desired upper limb joint angle trajectory; determining the training result based on the state of the humanoid robot after movement; and updating the lower limb control strategy based on the training result and the pre-set upper limb course and lower limb course coupled learning strategy.

[0011] According to some embodiments of this application, the difficulty of the current upper limb course and the difficulty of the current lower limb course are determined based on the current expected movement speed and the current expected upper limb joint angle trajectory, including: determining the difficulty of the current lower limb course based on the current expected movement speed; and determining the difficulty of the current upper limb course based on the current upper limb center of mass position offset and the current upper limb center of mass momentum corresponding to the current expected upper limb joint angle trajectory.

[0012] According to some embodiments of this application, the difficulty of the current upper limb course is determined based on the current upper limb center of mass position offset and the current upper limb center of mass momentum corresponding to the current expected upper limb joint angle trajectory. This includes: determining the current upper limb center of mass position offset and the current upper limb center of mass momentum based on the current expected upper limb joint angle trajectory; performing a weighted calculation on the current upper limb center of mass position offset and the current upper limb center of mass momentum to obtain a weighted calculation result; and determining the difficulty of the current upper limb course based on the weighted calculation result.

[0013] According to some embodiments of this application, the preset upper limb course and lower limb course coupling learning strategy is as follows: when the difficulty of the lower limb course is fixed, the difficulty of the upper limb exercise course is increased according to a first preset method; when the difficulty of the upper limb exercise course reaches a first preset threshold, the difficulty of the upper limb exercise course is fixed, and the difficulty of the lower limb exercise course is increased according to a second preset method until the difficulty of the lower limb exercise course reaches a second preset threshold.

[0014] According to some embodiments of this application, the humanoid robot is controlled by controlling the corresponding joint ends of the humanoid robot according to state observation values, upper limb joint angle commands, and lower limb joint angle commands, respectively. This includes: determining joint torque commands corresponding to multiple joint ends according to state observation values, upper limb joint angle commands, and lower limb joint angle commands through joint proportional-derivative control; and controlling the corresponding joint ends of the humanoid robot according to the corresponding joint torque commands, respectively, to control the whole-body movement of the humanoid robot.

[0015] According to one aspect of this application, a humanoid robot whole-body motion control device is provided. The device includes: a sensor data processing unit, an upper limb control unit, a lower limb control unit, and a real-time control unit. The sensor data processing unit acquires state observations of the humanoid robot; the upper limb control unit determines the upper limb joint angle commands of the humanoid robot based on the state observations and a pre-trained upper limb control strategy; the lower limb control unit determines the lower limb joint angle commands of the humanoid robot based on the state observations, the upper limb joint angle commands, and the pre-trained lower limb control strategy; the real-time control unit controls the corresponding joint ends of the humanoid robot based on the state observations, the upper limb joint angle commands, and the lower limb joint angle commands to control the whole-body motion of the humanoid robot.

[0016] According to another aspect of this application, this application also provides a non-volatile computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can implement the humanoid robot whole-body motion control method as described above.

[0017] The technical solution of this application can obtain the state observation value of the humanoid robot, then determine the upper limb joint angle command based on the state observation value, and further use the upper limb joint angle command to determine the lower limb joint angle command. Finally, the humanoid robot’s whole-body movement is controlled by the state observation value, the upper limb joint angle command and the lower limb joint angle command.

[0018] Since the determination of lower limb joint angle commands incorporates the joint angle information corresponding to upper limb movement (i.e., upper limb joint angle commands), the lower limb control strategy of this application can obtain information characterizing the current upper limb movement requirements, compared to the method of generating control results completely independently of the upper and lower limbs.

[0019] The technical solution of this application uses the same humanoid robot state observation value to determine the upper limb joint angle command and the lower limb joint angle command, and ultimately applies them to the corresponding joint ends. Therefore, it can form an associated whole-body motion control process based on unified state information. This improves the coordination between the upper and lower limb movements of the humanoid robot, enabling it to maintain corresponding lower limb movements while performing upper limb operations, thereby improving the stability and control effect of the humanoid robot's whole-body motion control. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A flowchart illustrating method 1000 according to an embodiment of this application is shown; Figure 2 A flowchart illustrating step S200 according to an embodiment of this application is shown; Figure 3 A flowchart illustrating step S300 according to an embodiment of this application is shown; Figure 4 Another schematic flowchart of method 1000 according to an embodiment of this application is shown; Figure 5 A flowchart illustrating step S110 according to an embodiment of this application is shown; Figure 6A flowchart illustrating step S113 according to an embodiment of this application is shown; Figure 7 A flowchart illustrating step S1132 according to an embodiment of this application is shown; Figure 8 A flowchart illustrating step S400 according to an embodiment of this application is shown; Figure 9 A schematic diagram of the device according to an embodiment of this application is shown; Figure 10 Another structural schematic diagram of an apparatus according to an embodiment of this application is shown.

[0022] Explanation of reference numerals in the attached figures: Device 20; Sensor data processing unit 21; Upper limb control unit 22; Lower limb control unit 23; Real-time control unit 24; Pre-training unit 25. Detailed Implementation

[0023] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, they are provided so that this application will be thorough and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted.

[0024] The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a full understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced without one or more of these specific details, or other methods, components, materials, devices, etc. In these cases, well-known structures, methods, devices, implementations, materials, or operations will not be shown or described in detail.

[0025] Furthermore, the terms “comprising” and “having”, and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus.

[0026] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, rather than to describe a specific order.

[0027] The technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0028] See Figure 9 The humanoid robot whole-body motion control device 20 provided in this application includes a sensor data processing unit 21, an upper limb control unit 22, a lower limb control unit 23, and a real-time control unit 24. See also Figure 10 The humanoid robot's whole-body motion control device 20 may also include a pre-training unit 25.

[0029] The following is combined with Figure 9 and Figure 10 This application describes a method 1000 for controlling the whole-body motion of a humanoid robot. See also... Figure 1 Method 1000 may include steps S100-S400.

[0030] In step S100, the state observation values ​​of the humanoid robot are obtained.

[0031] According to the example embodiment, the state observation value can be information describing the robot's current motion state. For example, the state observation value can include joint angles, joint velocities, trunk posture angles, trunk posture angular velocities, and trunk linear accelerations.

[0032] Joint angles describe the current rotational position of the humanoid robot's joints. Joint velocities describe the change of the humanoid robot's joint angles over time. Torso pose angles describe the orientation of the humanoid robot's torso relative to a reference coordinate system. Torso pose angular velocity describes the rate of change of the humanoid robot's torso orientation. Torso linear acceleration describes the translational motion of the humanoid robot's torso.

[0033] For example, in step S100, the sensor data processing unit 21 acquires the state observation values ​​of the humanoid robot. The sensor data processing unit 21 can initialize a data buffer. The sensor data processing unit 21 can acquire state observation values ​​by periodically reading sensor data. The sensor data processing unit 21 can acquire the torso posture angle, torso posture angular velocity, and torso linear acceleration collected by the humanoid robot's inertial measurement unit. The sensor data processing unit 21 can acquire the joint angles and joint angular velocities collected by the humanoid robot's joint encoders.

[0034] The sensor data processing unit 21 performs data processing such as low-pass filtering on the acquired data to obtain state observation values. The sensor data processing unit 21 then updates the state observation values ​​to the data buffer.

[0035] In step S200, the upper limb joint angle command of the humanoid robot is determined based on the state observation value and the pre-trained upper limb control strategy.

[0036] According to the example embodiment, the upper limb control strategy can be a control rule that determines the upper limb movement result based on state observations. The upper limb control strategy can be determined based on the specific upper limb operant action task. The upper limb control strategy can employ a mature model-based control strategy or a machine learning-based upper limb control strategy. The upper limb joint angle command can be a joint angle command used to control the target position of a specific upper limb joint.

[0037] For example, in step S200, the upper limb control unit 22 determines the upper limb joint angle command of the humanoid robot based on the state observation value and the pre-trained upper limb control strategy.

[0038] When the humanoid robot needs to perform upper limb manipulation tasks, the upper limb control unit 22 can determine the target motion state of each joint of the upper limb based on the state observation value and the corresponding upper limb manipulation task, and output upper limb joint angle commands through the upper limb control strategy.

[0039] In step S300, the lower limb joint angle command of the humanoid robot is determined based on the state observation value, the upper limb joint angle command, and the pre-trained lower limb control strategy.

[0040] According to the example embodiment, the lower limb control strategy determines the control rules for the lower limb movement outcome based on state observations and upper limb joint angle commands. The lower limb joint angle commands can be joint angle commands used to control the target position of a specific lower limb joint.

[0041] For example, in step S300, the lower limb control unit 23 determines the lower limb joint angle command of the humanoid robot based on the state observation value, the upper limb joint angle command and the pre-trained lower limb control strategy.

[0042] When the humanoid robot needs to perform lower limb manipulation tasks, the lower limb control unit 23 can determine the target motion state of each joint of the lower limb based on the state observation value, the upper limb joint angle command and the corresponding lower limb manipulation task, and output the lower limb joint angle command through the lower limb control strategy.

[0043] In step S400, the corresponding joint ends of the humanoid robot are controlled according to the state observation value, the upper limb joint angle command and the lower limb joint angle command, so as to control the whole body movement of the humanoid robot.

[0044] According to the example embodiment, the real-time control unit 24 controls the corresponding joint ends of the humanoid robot based on state observation values, upper limb joint angle commands, and lower limb joint angle commands to control the whole-body movement of the humanoid robot. For example, the real-time control unit 24 can determine the target joint angle corresponding to each joint based on the state observation values, upper limb joint angle commands, and lower limb joint angle commands, calculate the control quantity corresponding to each joint, and send the control quantity to the corresponding joint drive mechanism to make each joint of the humanoid robot move towards the corresponding target state.

[0045] Through the above embodiments, the technical solution of this application can obtain the state observation value of the humanoid robot, determine the upper limb joint angle command based on the state observation value, further use the upper limb joint angle command to determine the lower limb joint angle command, and finally control the whole body movement of the humanoid robot together based on the state observation value, the upper limb joint angle command and the lower limb joint angle command.

[0046] Since the determination of lower limb joint angle commands incorporates the joint angle information corresponding to upper limb movement (i.e., upper limb joint angle commands), the lower limb control strategy of this application can obtain information characterizing the current upper limb movement requirements, compared to the method of generating control results completely independently of the upper and lower limbs.

[0047] The technical solution of this application uses the same humanoid robot state observation value to determine the upper limb joint angle command and the lower limb joint angle command, and ultimately applies them to the corresponding joint ends. Therefore, it can form an associated whole-body motion control process based on unified state information. This improves the coordination between the upper and lower limb movements of the humanoid robot, enabling it to maintain corresponding lower limb movements while performing upper limb operations, thereby improving the stability and control effect of the humanoid robot's whole-body motion control.

[0048] Optionally, see Figure 2 Step S200 includes steps S210 and S220.

[0049] In step S210, the state observation values ​​are input into the upper limb control strategy to output upper limb joint angle commands.

[0050] According to the example embodiment, in step S210, the upper limb control unit 22 inputs the state observation value into the upper limb control strategy to output the upper limb joint angle command.

[0051] The upper limb control unit 22 reads the state observation values ​​from the data buffer and inputs them into the upper limb control strategy. The upper limb control strategy calculates the upper limb joint angle command based on the input state.

[0052] In step S220, the upper limb joint angle command is updated to the data cache.

[0053] According to the example embodiment, in step S220, the upper limb control unit 22 updates the upper limb joint angle command to the data cache.

[0054] The upper limb control unit 22 writes the upper limb joint angle command into the data buffer. The data buffer can use shared memory, a circular cache, a thread-safe data structure, or other data storage methods that can achieve temporary storage and updating of control data.

[0055] Through the above embodiments, the technical solution of this application inputs the state observation values ​​into the upper limb control strategy and writes the obtained upper limb joint angle commands into the data buffer, so that the humanoid robot state data and upper limb control results can form a continuous data transmission relationship in the control process.

[0056] Optionally, see Figure 3 Step S300 includes steps S310 and S320.

[0057] In step S310, the state observation value and the upper limb joint angle command are input into the lower limb control strategy to output the lower limb joint angle command.

[0058] According to the example embodiment, in step S310, the lower limb control unit 23 inputs the state observation value and the upper limb joint angle command into the lower limb control strategy to output the lower limb joint angle command.

[0059] The lower limb control unit 23 can acquire state observation values ​​and upper limb joint angle commands from the data buffer. The lower limb control unit 23 inputs the state observation values ​​and upper limb joint angle commands into the lower limb control strategy. The lower limb control strategy calculates the lower limb joint angle commands based on the input state.

[0060] In step S320, the lower limb joint angle command is updated to the data cache.

[0061] According to the example embodiment, the lower limb control unit 23 writes the lower limb joint angle command into the data buffer. The data buffer can employ shared memory, a circular cache, a thread-safe data structure, or other data storage methods capable of temporarily storing and updating control data.

[0062] Through the above embodiments, the technical solution of this application simultaneously provides state observation values ​​and upper limb joint angle commands to the lower limb control strategy. This enables the lower limb control strategy to not only determine the lower limb movement based on the robot's current actual movement state, but also to adjust the lower limb movement results based on the current upper limb movement requirements. Since the upper limb movement requirements can serve as input information for the lower limb control strategy, the influence of upper limb movement can be introduced during the lower limb movement generation stage, creating a more direct data correlation between the lower limb and upper limb movements.

[0063] Optionally, see Figure 4 Before step S100, method 1000 further includes step S110.

[0064] Step S110 involves pre-training lower limb control strategies. See [link / reference] Figure 5 Step S110 includes steps S111-S117.

[0065] In step S111, a training environment including a terrain model and a humanoid robot model is established in the dynamic simulation environment.

[0066] According to an example embodiment, the dynamics simulation environment can be a virtual environment that simulates the dynamic behavior of a humanoid robot during movement using a computer. For example, the dynamics simulation environment can be a training environment established by a dynamics simulation platform, including a terrain model and a fully free-degree-of-freedom humanoid robot model.

[0067] For example, in step S111, the pre-training unit 25 establishes a training environment in the dynamic simulation environment that includes a terrain model and a humanoid robot model.

[0068] The pre-training unit 25 initializes the training parameters and buffer data for the environment. The training parameters include the environment parameters to be established, the terrain parameters of the environment, the robot model parameters of the environment, motion command parameters, course learning parameters, reward function parameters, etc. The buffer data includes the robot state (base coordinate pose, joint angles, joint angular velocities, joint torques, foot contact forces, etc.), lower limb movements, desired motion speed, desired upper limb joint angles, etc. The pre-training unit 25 establishes N training environments in a dynamic simulation environment based on a high-performance robot simulation platform (e.g., IsaacGym). Each training environment can include a terrain model and a fully free-degree-of-freedom humanoid robot model. N is an integer greater than or equal to 1.

[0069] In step S112, training observations are determined based on the humanoid robot state in the training environment, the current desired motion speed, and the current desired upper limb joint angle trajectory.

[0070] According to the example embodiment, the current desired motion speed can represent the movement speed that the training environment currently requires the humanoid robot to perform. The current desired upper limb joint angle trajectory can represent the trajectory along which the humanoid robot's upper limbs are required to move over time in the training environment.

[0071] Training observations can be the data set input into the lower limb control strategy during the pre-training phase. Training observations may include robot trunk posture angles, trunk posture angular velocities, desired motion velocities, desired upper limb joint angle trajectories, actual joint angles, actual joint angular velocities, and lower limb movements from the previous control cycle.

[0072] For example, in step S112, the pre-training unit 25 determines training observations based on the humanoid robot state in the training environment, the current desired motion speed, and the current desired upper limb joint angle trajectory.

[0073] The pre-training unit 25 can sample the joint angles, joint velocities, torso posture angles, torso posture angular velocities, and torso linear acceleration of the humanoid robot, and perform normalization and coordinate transformation on the sampled data. The normalized data is then vector-concatenated with the current desired motion velocity and the current desired upper limb joint angle trajectory to generate training observations.

[0074] In step S113, the difficulty of the current upper limb course and the difficulty of the current lower limb course are determined based on the current desired movement speed and the current desired upper limb joint angle trajectory.

[0075] According to the example embodiment, the difficulty of the upper limb course can be used to characterize the requirements of the upper limb's own motion accuracy and the severity of the disturbance to the lower limb balance when the humanoid robot performs the operation task.

[0076] The difficulty of the upper limb course is measured by the weighted values ​​of the maximum center of mass position offset, center of mass momentum, and end-effector thrust disturbance corresponding to the expected upper limb joint angle trajectory of the robot. The smaller the weighted value, the lower the difficulty of the upper limb course; the larger the weighted value, the higher the difficulty of the upper limb course. For example, the difficulty of the upper limb course can be divided into three levels: Difficulty 1: small-amplitude upper limb swings (such as small-amplitude swings with the arm hanging naturally), which has a slight impact on upper limb accuracy and lower limb balance; Difficulty 2: medium-amplitude upper limb operations (such as lifting the hand to pick up an object), which has a certain impact on upper limb accuracy and lower limb balance; Difficulty 3: large-amplitude upper limb operations (such as extending both arms, lifting heavy objects, and rapidly swinging the arm), which has a significant impact on upper limb accuracy and lower limb balance.

[0077] The difficulty of the lower limb course can be used to characterize the intensity of the humanoid robot's lower limb movements, that is, the expected level of movement speed that the robot will achieve.

[0078] The difficulty of lower limb courses is measured by the desired robot movement speed command. The lower the speed, the lower the difficulty of the lower limb course; the higher the speed, the higher the difficulty of the lower limb course. For example, the difficulty of lower limb courses can be divided into three levels: Difficulty 1: low-speed walking; Difficulty 2: medium-speed walking or jogging; Difficulty 3: fast running or high-speed movement.

[0079] The current upper limb course difficulty level corresponds to the complexity level of the humanoid robot's current upper limb manipulation task. The current lower limb course difficulty level corresponds to the complexity level of the humanoid robot's current lower limb manipulation task.

[0080] For example, in step S113, the pre-training unit 25 can determine the current upper limb course difficulty and the current lower limb course difficulty based on the current desired movement speed and the current desired upper limb joint angle trajectory.

[0081] The pre-training unit 25 can compare the current desired movement speed with a preset desired speed threshold to determine the difficulty of the current lower limb course. The pre-training unit 25 can also calculate the current upper limb center of mass position offset and current upper limb center of mass momentum based on the current desired upper limb joint angle trajectory to determine the difficulty of the current upper limb course.

[0082] In step S114, the lower limb movements are determined based on training observations and the initial lower limb control strategy, under the current upper limb course difficulty and the current lower limb course difficulty.

[0083] According to an example embodiment, the initial lower limb control strategy can be the initial mechanism of the lower limb control strategy.

[0084] For example, in step S114, the lower limb control unit 23 determines the lower limb movement based on training observations and an initial lower limb control strategy, under the current upper limb course difficulty and the current lower limb course difficulty. The lower limb control unit 23 can initialize a lower limb control strategy optimized based on a proximal strategy to obtain an initial lower limb control strategy. The input of this initial lower limb control strategy is the training observations, and the output is the lower limb movement of the humanoid robot.

[0085] In step S115, the humanoid robot in the training environment is driven to move based on the lower limb movements and the current desired upper limb joint angle trajectory.

[0086] According to the example embodiment, the real-time control unit 24 drives the humanoid robot's movement in the training environment based on lower limb movements and the desired upper limb joint angle trajectory. The lower limb control unit 23 can convert lower limb movements into corresponding lower limb joint angle commands. The upper limb control unit 22 can convert the current desired upper limb joint angle trajectory into corresponding upper limb joint angle commands.

[0087] For example, lower limb joint angle commands It can be represented as follows: ; in, Humanoid robot The initial angle of each joint, For the first Lower limb movements involving each joint The scaling ratio for the motion. This is a command for lower limb joint angles.

[0088] In step S116, the training result is determined based on the state of the humanoid robot after the movement.

[0089] According to the example embodiment, the training result can be a reward function for the state of the humanoid robot after movement. The reward function can include rewards for robot motion speed tracking, robot hand pose tracking, minimization of robot joint torques, and minimization of robot joint angular accelerations. The training result can also be combined with the training environment runtime.

[0090] For example, in step S116, the pre-training unit 25 determines the training result based on the state of the humanoid robot after movement. The sensor data processing unit 21 acquires data such as joint angles, joint velocities, torso posture angles, torso posture angular velocities, and torso linear acceleration of the humanoid robot after movement. The pre-training unit 25 calculates the reward function based on the acquired data.

[0091] In step S117, the lower limb control strategy is updated based on the training results and the preset upper limb course and lower limb course coupling learning strategy.

[0092] According to the example embodiment, the preset upper limb course and lower limb course coupling learning strategy can be a training strategy that combines and progressively increases the difficulty of upper limb courses and lower limb courses according to a preset relationship.

[0093] Optionally, the preset upper limb and lower limb course coupling learning strategy is as follows: with the lower limb course difficulty fixed, the upper limb exercise course difficulty is increased according to a first preset method. When the upper limb exercise course difficulty reaches a first preset threshold, the upper limb exercise course difficulty is fixed, and the lower limb exercise course difficulty is increased according to a second preset method until the lower limb exercise course difficulty reaches a second preset threshold.

[0094] For example, a pre-defined upper limb and lower limb course coupled learning strategy can be specifically as follows: (1) Initialize the difficulty of the upper limb course and the lower limb course, both with a difficulty level of 1; (2) Fix the difficulty of the lower limb courses and increase the difficulty of the upper limb courses, that is, increase the difficulty of the upper limb courses by 1; (3) If the difficulty of the upper limb movement course reaches the first preset threshold, then proceed to (4); otherwise proceed to (2). (4) Initialize the difficulty of the upper limb course, that is, set the course difficulty to 1; (5) Increase the difficulty of the lower limb courses, i.e., increase the difficulty of the lower limb courses by 1; (6) Determine whether the maximum lower limb course difficulty has been reached. If it has, end the course. If it has not, proceed to (2).

[0095] The pre-set upper limb and lower limb course coupling learning strategy can also adjust the difficulty of the upper limb and lower limb courses according to the training duration or control precision.

[0096] For example, in step S117, the pre-training unit 25 updates the lower limb control strategy based on the training results and the preset upper limb course and lower limb course coupling learning strategy.

[0097] Pre-training unit 25 can update the expected movement speed by combining the training environment running time and the current lower limb course difficulty. Pre-training unit 25 can update the expected upper limb joint angle trajectory by combining the training environment running time and the current upper limb course difficulty.

[0098] The pre-training unit 25 can combine the running time of the training environment to apply random force perturbation to the humanoid robot's hand to simulate the interaction perturbation between the humanoid robot's upper limb and the external environment when performing actual limb manipulation meta-action tasks.

[0099] The pre-training unit 25 can combine the running time of the training environment. If the humanoid robot has fallen or the running time exceeds the set value, the corresponding training environment should terminate the training and obtain the sequence number of the training environment that needs to be terminated, which is abbreviated as the termination sequence number.

[0100] The pre-training unit 25 can calculate the cumulative error of the upper limb end operation corresponding to the termination number, and adjust the difficulty of the upper limb course and the lower limb course according to the preset upper limb course and lower limb course coupling learning strategy, based on the training environment runtime.

[0101] The pre-training unit 25 can combine a reward function to optimize and update the lower limb control strategy based on the proximal policy. The pre-training unit 25 can be set to a maximum number of training iterations. When the maximum number of training iterations is reached, the lower limb control strategy is output and the pre-training ends.

[0102] Through the above embodiments, the technical solution of this application can introduce the desired movement speed and the desired upper limb joint angle trajectory into the pre-training process of the lower limb control strategy, so that the lower limb control strategy can be exposed to whole-body movement scenarios formed by different movement speeds and different upper limb movement states during the pre-training process.

[0103] The technical solution of this application can control the difficulty of both upper and lower limb courses by pre-setting a coupled learning strategy of upper and lower limb courses. The pre-training process of the lower limb control strategy can gradually change according to the preset difficulty, and the lower limb control strategy can be updated based on the training results. Simultaneously, the technical solution of this application simulates external interactive disturbances that may occur during actual operation through random force perturbations of the hand, enabling the lower limb control strategy to learn in a perturbed training environment. Therefore, the technical solution of this application can improve the adaptability of the trained lower limb control strategy to upper limb movements and external disturbances, thereby improving the robustness of the humanoid robot when performing whole-body movements.

[0104] Optionally, see Figure 6Step S113 may include steps S1131 and S1132.

[0105] In step S1131, the difficulty of the current lower limb course is determined based on the current desired movement speed.

[0106] According to the example embodiment, in step S1131, the pre-training unit 25 determines the difficulty of the current lower limb course based on the current desired movement speed.

[0107] The pre-training unit 25 can compare the current desired movement speed with the preset desired speed threshold to determine the difficulty of the current lower limb course.

[0108] For example, if the difficulty of lower limb exercise courses is divided into three levels, the pre-training unit 25 can determine the current difficulty of the lower limb course according to the following formula: ; in, Given the current difficulty level of the lower limb courses, For the current desired speed of motion, , and All of these are preset desired speed thresholds.

[0109] In step S1132, the difficulty of the current upper limb course is determined based on the current upper limb center of mass position offset and current upper limb center of mass momentum corresponding to the current expected upper limb joint angle trajectory.

[0110] According to the example embodiment, the upper limb center of mass position offset can describe the degree of positional deviation of the upper limb center of mass relative to the origin of the humanoid robot's floating base coordinate system. The upper limb center of mass momentum can describe the kinetic momentum of the upper limb center of mass relative to the origin of the robot's floating base coordinate system.

[0111] The current upper limb center of mass offset represents the degree of displacement of the upper limb center of mass relative to the origin of the humanoid robot's floating coordinate system during the current upper limb operation task. The current upper limb center of mass momentum represents the kinetic momentum of the upper limb center of mass relative to the origin of the robot's floating coordinate system during the current upper limb operation task.

[0112] For example, in step S1132, the pre-training unit 25 determines the difficulty of the current upper limb course based on the current upper limb center of mass position offset and the current upper limb center of mass momentum corresponding to the current expected upper limb joint angle trajectory.

[0113] The pre-training unit 25 can establish a floating quadruple linkage model of the humanoid robot and perform iterative calculations using the Newton-Euler method to obtain the current upper limb center of mass position offset and current upper limb center of mass momentum. The pre-training unit 25 can determine the current upper limb course difficulty based on the weighted result of the current upper limb center of mass position offset and current upper limb center of mass momentum.

[0114] Through the above embodiments, the technical solution of this application can evaluate the difficulty of the current lower limb movement course and the current upper limb operation course by using the current expected movement speed and the current expected upper limb joint angle trajectory respectively, so that the difficulty of the two courses can reflect the actual movement complexity of the humanoid robot's movement task and upper limb operation task respectively.

[0115] Optionally, see Figure 7 Step S1132 may include steps S11321-S11323.

[0116] In step S11321, the current upper limb center of mass position offset and current upper limb center of mass momentum are determined based on the current desired upper limb joint angle trajectory.

[0117] According to the example embodiment, in step S11321, the pre-training unit 25 determines the current upper limb center of mass position offset and the current upper limb center of mass momentum based on the current desired upper limb joint angle trajectory. The pre-training unit 25 can calculate the motion state of each link of the humanoid robot's upper limb based on the current desired upper limb joint angle trajectory, and calculate the current upper limb center of mass position offset and the current upper limb center of mass momentum iteratively using the Newton-Euler method based on the humanoid robot's whole-body floating fundamental link model.

[0118] In step S11322, the current upper limb center of mass position offset and the current upper limb center of mass momentum are weighted and calculated to obtain the weighted calculation result.

[0119] According to the example embodiment, in step S11322, the pre-training unit 25 performs a weighted calculation on the current upper limb center of mass position offset and the current upper limb center of mass momentum to obtain a weighted calculation result. The pre-training unit 25 multiplies the current upper limb center of mass position offset and the current upper limb center of mass momentum by the corresponding weight coefficients to obtain the weighted calculation result.

[0120] In step S11323, the difficulty of the current upper limb course is determined based on the weighted calculation results.

[0121] According to the example embodiment, in step S11323, the pre-training unit 25 determines the current upper limb course difficulty based on the weighted calculation result.

[0122] The pre-training unit 25 can compare the weighted calculation results with the preset weighted calculation result threshold to determine the difficulty of the current upper limb course.

[0123] For example, if the difficulty of upper limb movement courses is divided into three levels, the pre-training unit 25 can determine the current upper limb course difficulty according to the following formula: ; in, Given the current difficulty level of the upper limb courses, The current position of the upper limb's center of mass is offset. The current upper limb center of mass momentum, This is the weighting coefficient for the current upper limb center of mass position offset. This is the weighting coefficient for the current upper limb center of mass momentum. , and All are preset weighted calculation result thresholds.

[0124] Through the above embodiments, the technical solution of this application uses both the current upper limb center of mass position offset and the current upper limb center of mass momentum to evaluate the difficulty of upper limb courses. Compared with the method of judging training difficulty based solely on changes in the angle of a single upper limb joint, the technical solution of this application can simultaneously consider the impact of upper limb movements on the robot's overall spatial and dynamic state, making the evaluation of upper limb course difficulty more comprehensive, thereby improving the matching degree between course learning and actual whole-body movement difficulty.

[0125] Optionally, see Figure 8 Step S400 includes steps S410 and S420.

[0126] In step S410, joint torque commands corresponding to multiple joint ends are determined by joint proportional-derivative control based on state observation values, upper limb joint angle commands, and lower limb joint angle commands.

[0127] According to the example embodiment, in step S410, the real-time control unit 24 determines the joint torque command corresponding to multiple joint ends by means of joint proportional-derivative control, based on the state observation value, the upper limb joint angle command and the lower limb joint angle command.

[0128] For example, the real-time control unit 24 can read the state observation values, upper limb joint angle commands, and lower limb joint angle commands from the data buffer, and calculate the joint torque commands of the humanoid robot through the joint proportional-differential controller. The joint torque commands can include the torque commands of each joint of the humanoid robot. The joint torque commands can be represented as follows: ; in, For the first Joint torque commands for each joint: For the first The expected joint angle of each joint. For the first The expected angular velocity of each joint, For the first The actual joint angles of each joint. For the first The actual angular velocity of each joint This is the proportionality coefficient. These are the proportional and differential coefficients.

[0129] In step S420, the corresponding joint ends of the humanoid robot are controlled according to the corresponding joint torque command to control the whole body movement of the humanoid robot.

[0130] According to the example embodiment, in step S420, the real-time control unit 24 controls the corresponding joint ends of the humanoid robot according to the corresponding joint torque commands, so as to control the whole-body movement of the humanoid robot. For example, the real-time control unit 24 sends joint torque commands to the motors at the corresponding joint ends of the humanoid robot to control the whole-body movement of the humanoid robot.

[0131] Through the above embodiments, the technical solution of this application converts upper limb joint angle commands and lower limb joint angle commands into actual joint torque commands by using proportional-derivative control, thereby realizing closed-loop control from motion strategy to physical joint execution.

[0132] According to another aspect of this application, this application also provides a non-volatile computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can implement the humanoid robot whole-body motion control method as described above.

[0133] According to another aspect of this application, this application also provides an electronic device, including: one or more processors; and a storage device for storing one or more programs, which, when executed by one or more processors, enable the one or more processors to implement the humanoid robot whole-body motion control method as described above.

[0134] According to another aspect of this application, this application also provides a computer program product, including: a computer program stored on a computer-readable storage medium; the computer program includes program instructions that, when executed by a computer, cause the computer to perform the humanoid robot whole-body motion control method as described above.

[0135] Finally, it should be noted that the above description is merely a preferred embodiment of this application and is not intended to limit this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions of the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for controlling the whole-body motion of a humanoid robot, characterized in that, include: Acquire state observations of the humanoid robot; Based on the state observations and the pre-trained upper limb control strategy, the upper limb joint angle commands of the humanoid robot are determined; The lower limb joint angle command of the humanoid robot is determined based on the state observation value, the upper limb joint angle command, and the pre-trained lower limb control strategy. The humanoid robot's corresponding joint ends are controlled according to the state observation value, the upper limb joint angle command, and the lower limb joint angle command to control the humanoid robot's whole-body movement; Before acquiring the state observations of the humanoid robot, the method further includes: Pre-training the lower limb control strategy includes: Establish a training environment that includes terrain models and humanoid robot models within a dynamic simulation environment; The training observations are determined based on the humanoid robot's state in the training environment, the current desired movement speed, and the current desired upper limb joint angle trajectory. Based on the current expected movement speed and the current expected upper limb joint angle trajectory, determine the current upper limb course difficulty and the current lower limb course difficulty; Under the current upper limb course difficulty and the current lower limb course difficulty, the lower limb movement is determined based on the training observations and the initial lower limb control strategy; The humanoid robot in the training environment is driven to move according to the lower limb movements and the current desired upper limb joint angle trajectory. The training results are determined based on the state of the humanoid robot after the exercise. Based on the training results and the preset upper limb and lower limb course coupled learning strategy, the lower limb control strategy is updated.

2. The method according to claim 1, characterized in that, The step of determining the upper limb joint angle commands of the humanoid robot based on the state observation values ​​and the pre-trained upper limb control strategy includes: The state observation values ​​are input into the upper limb control strategy to output the upper limb joint angle command; Update the upper limb joint angle command to the data cache area.

3. The method according to claim 2, characterized in that, The step of determining the lower limb joint angle command of the humanoid robot based on the state observation value, the upper limb joint angle command, and the pre-trained lower limb control strategy includes: The state observation value and the upper limb joint angle command are input into the lower limb control strategy to output the lower limb joint angle command; Update the lower limb joint angle command to the data cache area.

4. The method according to claim 1, characterized in that, The step of determining the current upper limb course difficulty and the current lower limb course difficulty based on the current expected movement speed and the current expected upper limb joint angle trajectory includes: The difficulty of the current lower limb course is determined based on the current expected movement speed. The difficulty of the current upper limb course is determined based on the current upper limb center of mass position offset and current upper limb center of mass momentum corresponding to the current expected upper limb joint angle trajectory.

5. The method according to claim 4, characterized in that, The step of determining the current upper limb course difficulty based on the current upper limb center of mass position offset and current upper limb center of mass momentum corresponding to the current expected upper limb joint angle trajectory includes: Based on the current desired upper limb joint angle trajectory, determine the current upper limb center of mass position offset and the current upper limb center of mass momentum; The current upper limb center of mass position offset and the current upper limb center of mass momentum are weighted and calculated to obtain a weighted calculation result; The difficulty of the current upper limb course is determined based on the weighted calculation results.

6. The method according to claim 1, characterized in that, The preset upper limb and lower limb course coupled learning strategy is as follows: While keeping the lower limb course difficulty fixed, increase the upper limb exercise course difficulty according to the first preset method; When the difficulty of the upper limb exercise course reaches the first preset threshold, the difficulty of the upper limb exercise course is fixed, and the difficulty of the lower limb exercise course is increased according to the second preset method until the difficulty of the lower limb exercise course reaches the second preset threshold.

7. The method according to claim 1, characterized in that, The step of controlling the humanoid robot by controlling the corresponding joint ends of the humanoid robot according to the state observation value, the upper limb joint angle command, and the lower limb joint angle command, respectively, includes: By using joint proportional-derivative control, joint torque commands corresponding to multiple joint ends are determined based on the state observation values, the upper limb joint angle commands, and the lower limb joint angle commands. The corresponding joint torque commands are used to control the joint ends of the humanoid robot to control the whole-body movement of the humanoid robot.

8. A humanoid robot full-body motion control device, characterized in that, include: The sensor data processing unit acquires state observations of the humanoid robot. The upper limb control unit determines the upper limb joint angle commands of the humanoid robot based on the state observation values ​​and the pre-trained upper limb control strategy. The lower limb control unit determines the lower limb joint angle command of the humanoid robot based on the state observation value, the upper limb joint angle command, and the pre-trained lower limb control strategy. The real-time control unit controls the corresponding joint ends of the humanoid robot according to the state observation value, the upper limb joint angle command and the lower limb joint angle command, so as to control the whole body movement of the humanoid robot; Pre-trained units; The pre-training unit, the lower limb control unit, and the real-time control unit execute the pre-training of the lower limb control strategy, including: The pre-training unit establishes a training environment that includes a terrain model and a humanoid robot model in a dynamic simulation environment; The pre-training unit determines training observations based on the humanoid robot's state in the training environment, the current desired movement speed, and the current desired upper limb joint angle trajectory. The pre-training unit determines the difficulty of the current upper limb course and the difficulty of the current lower limb course based on the current expected movement speed and the current expected upper limb joint angle trajectory. The lower limb control unit determines the lower limb movement based on the training observations and the initial lower limb control strategy under the current upper limb course difficulty and the current lower limb course difficulty. The real-time control unit drives the humanoid robot in the training environment to move according to the lower limb movement and the current desired upper limb joint angle trajectory. The pre-training unit determines the training result based on the state of the humanoid robot after the movement; The pre-training unit updates the lower limb control strategy based on the training results and the preset upper limb and lower limb course coupling learning strategy.

9. A non-volatile computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the humanoid robot whole-body motion control method as described in any one of claims 1-7.

Images