Model training method, motion control method, device and robot for humanoid robot

Abstract

The application provides a model training method, a motion control method, equipment and a robot for a humanoid robot, and relates to the technical field of robot control. The model training method obtains motion data of a source body and training data of a target humanoid robot, obtains a pre-training motion network according to a first historical motion sequence, a first future motion sequence and a source body task condition in a whole-body motion sequence; adopts the pre-training motion network, performs motion prediction according to a second historical motion sequence in a robot motion sequence and a robot task condition, and obtains a future predicted motion sequence; and optimizes the pre-training motion network according to a second future motion sequence in the robot motion sequence and the future predicted motion sequence, and obtains a robot control network including a target motion network. The application solves the defects that the existing model lacks cross-body compatible motion representation, cannot reuse massive low-cost source body motion data, and training is highly dependent on high-cost real robot data.

Description

Technical Field

[0001] This application relates to the field of robot control technology, and more specifically, to a model training method, motion control method, device, and robot for humanoid robots. Background Technology

[0002] Currently, the mainstream humanoid robot full-body motion control strategies typically employ visual language action models or world action models. The former inputs visual observations, task instructions, and the robot's own state into the policy network to directly generate robot actions; the latter adds future image or visual feature predictions in addition to action generation, enabling the model to learn changes in the external scene as the actions occur.

[0003] However, both models have significant shortcomings in terms of data utilization: source body motion data such as Internet human body teaching videos, human motion capture, and heterogeneous simulation robot trajectories have the advantages of large scale and extremely low acquisition cost, but these data have inherent differences in skeleton topology, joint degrees of freedom, and mechanical structure with the target humanoid robot. They lack a unified and compatible motion representation scheme and cannot extract general whole-body limb coordination priors from massive source body motion data without attaching motor motion labels to the target robot.

[0004] In other words, the two models mentioned above can only be trained using real data from the target humanoid robot. Complete real machine data requires the simultaneous collection of robot visual images, body state, and control commands. The costs of collecting data in terms of manpower, hardware, and time are high. The massive amount of low-cost Internet motion data cannot be effectively reused. Model training requires a large number of real machine teaching samples, and the threshold for deployment is high. Summary of the Invention

[0005] This application addresses the shortcomings of the prior art by providing a model training method, motion control method, device, and robot for humanoid robots, in order to solve the problems existing in the prior art.

[0006] The technical solution adopted in the embodiments of this application is as follows: In a first aspect, embodiments of this application provide a model training method for a humanoid robot, the method comprising: The motion data of the source body and the training data of the target humanoid robot are acquired. The motion data of the source body includes: whole-body motion sequence and corresponding source body task conditions. The training data of the target humanoid robot includes: robot task conditions and robot motion sequence. Based on the first historical motion sequence, the first future motion sequence in the whole-body motion sequence and the source ontology task conditions, the initial motion network is pre-trained to obtain the pre-trained motion network. Using the pre-trained motion network, motion prediction is performed based on the second historical motion sequence in the robot motion sequence and the robot task conditions to obtain the future predicted motion sequence; Based on the second future motion sequence and the future predicted motion sequence in the robot motion sequence, the pre-trained motion network is optimized to obtain a robot control network that includes the target motion network.

[0007] In one embodiment, before performing pre-training based on the first historical motion sequence, the first future motion sequence, and the source ontology task conditions in the whole-body motion sequence to obtain the pre-trained motion network, the method further includes: The whole-body motion sequence is converted into the temporal representation of the motion model corresponding to the target humanoid robot to obtain the converted motion sequence of the source body; The first historical motion sequence and the first future motion sequence are obtained from the transformed motion sequence.

[0008] In one embodiment, the training data for the target humanoid robot further includes: a sequence of visual observation images, a sequence of body states, and a corresponding sequence of robot actions; The method further includes: An initial visual network is used to perform visual prediction based on historical visual observation images in the visual observation image sequence to obtain future visual prediction images. An initial action network is used to generate actions based on the future visual prediction image, the future predicted motion sequence, and the body state at the corresponding time step in the body state sequence, thereby obtaining the future predicted action sequence. The step of optimizing the pre-trained motion network based on the second future motion sequence and the future predicted motion sequence in the robot motion sequence to obtain a robot control network including the target motion network includes: Based on the second future motion sequence, the future predicted action sequence, the future visual observation image in the visual observation image sequence, the future visual predicted image, the corresponding future action sequence in the robot action sequence, and the future predicted action sequence, the pre-trained motion network, the initial visual network, and the initial action network are jointly optimized to obtain a robot control network including a target motion network, a target visual network, and a target action network.

[0009] In one embodiment, the step of jointly optimizing the pre-trained motion network, the initial vision network, and the initial action network based on the second future motion sequence, the future predicted motion sequence, the future visual observation image in the visual observation image sequence, the future visual predicted image, the corresponding future action sequence in the robot action sequence, and the future predicted action sequence to obtain a robot control network including a target motion network, a target vision network, and a target action network includes: The motion prediction loss of the pre-trained motion network is calculated based on the second future motion sequence and the future predicted motion sequence. Based on the future visual observation image and the future visual prediction image, calculate the visual prediction loss of the initial visual network; Calculate the action generation loss of the initial action network based on the future action sequence and the future predicted action sequence; Based on the motion prediction loss, the visual prediction loss, and the action generation loss, the pre-trained motion network, the initial visual network, and the initial action network are jointly optimized to obtain a robot control network that includes the target motion network, the target visual network, and the target action network.

[0010] In one embodiment, the step of jointly optimizing the pre-trained motion network, the initial vision network, and the initial action network based on the motion prediction loss, the visual prediction loss, and the action generation loss to obtain a robot control network including the target motion network, the target vision network, and the target action network includes: Calculate the joint optimization loss based on the motion prediction loss, the visual prediction loss, and the action generation loss; Based on the joint optimization loss, the pre-trained motion network, the initial vision network, and the initial action network are jointly optimized to obtain a robot control network including the target motion network, the target vision network, and the target action network.

[0011] Secondly, embodiments of this application provide a motion control method for a humanoid robot, the method comprising: Obtain the target humanoid robot's historical motion sequence before the current time and the robot's task conditions; Using the target motion network in the robot control network, motion prediction is performed based on the historical motion sequence and the robot task conditions to obtain the future predicted action sequence for future time steps after the current time. At the future time step, motion control is performed on the target humanoid robot based on the predicted future action sequence.

[0012] In one embodiment, if the robot control network includes: a target vision network and a target motion network; the method further includes: Acquire the observed image and body state of the target humanoid robot at the current time; Using the target vision network in the robot control network, visual prediction is performed based on the observed image at the current time to obtain the future visual prediction image for the future time step. The step of performing motion control on the target humanoid robot based on the predicted future action sequence at the future time step includes: Using the target action network in the robot control network, action prediction is performed based on the future predicted action sequence at the future time step, the future predicted action sequence, and the body state at the current time to obtain the future predicted action at the future time step. At the future time step, motion control is performed on the target humanoid robot based on the predicted future action.

[0013] Thirdly, embodiments of this application provide a computer device, including: a processor, a storage medium, and a bus. The storage medium stores program instructions executable by the processor. When the computer device is running, the processor communicates with the storage medium via the bus, and the processor executes the program instructions to implement the model training method for humanoid robots described in any of the above embodiments.

[0014] Fourthly, embodiments of this application provide a control device, including: a processor, a storage medium, and a bus. The storage medium stores program instructions executable by the processor. When the control device is running, the processor communicates with the storage medium via the bus, and the processor executes the program instructions to implement the motion control method for the humanoid robot described in any of the above embodiments.

[0015] Fifthly, embodiments of this application provide a humanoid robot, including: a humanoid robot body and a control device, wherein the control device is used to execute the motion control method of the humanoid robot described in any of the above embodiments.

[0016] The beneficial effects of this application are: it provides a model training method for humanoid robots, which independently trains a pre-trained motion network using only low-cost source ontology motion data to extract general limb coordination priors, then transfers the pre-trained motion network to the target humanoid robot, and combines it with real robot motion sequences to adapt and optimize the pre-trained motion network, thus solving the defects of existing models that lack cross-ontology compatible motion representations, cannot reuse massive amounts of low-cost source ontology motion data, and whose training is highly dependent on high-cost real robot data. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A schematic flowchart illustrating a model training method for a humanoid robot provided in an embodiment of this application; Figure 2 This is a schematic flowchart of a method for processing whole-body motion sequences provided in an embodiment of this application; Figure 3 This is a schematic flowchart of a method for obtaining a robot control network provided in an embodiment of this application; Figure 4 A schematic diagram of the joint optimization method provided in the embodiments of this application; Figure 5 A schematic diagram of a joint optimization method provided in another embodiment of this application; Figure 6 A flowchart illustrating the motion control method for a humanoid robot provided in an embodiment of this application; Figure 7 A flowchart illustrating a motion control method for a humanoid robot provided in another embodiment of this application; Figure 8 A schematic diagram of the structure of a model training device for a humanoid robot provided in an embodiment of this application; Figure 9 A schematic diagram of the motion control device for a humanoid robot provided in an embodiment of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of this application, but not all embodiments.

[0020] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0021] Furthermore, the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Additionally, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0022] It should be noted that, where there is no conflict, the features in the embodiments of this application can be combined with each other.

[0023] This application provides a computer device, including a processor, a storage medium, and a bus. The storage medium stores program instructions executable by the processor. When the computer device is running, the processor communicates with the storage medium via the bus, and the processor executes the program instructions to implement the model training method for humanoid robots described in the following embodiments.

[0024] The following examples, in conjunction with the accompanying drawings, provide specific illustrations of the model training method for humanoid robots provided in this application.

[0025] Figure 1 This is a flowchart illustrating a model training method for a humanoid robot provided in an embodiment of this application, as shown below. Figure 1 As shown, the method includes: S11. Obtain motion data of the source body and training data of the target humanoid robot.

[0026] The motion data of the source ontology includes a full-body motion sequence (e.g., the timing of the limb joints of the source ontology) and corresponding source ontology task conditions (e.g., text instructions). The source ontology includes at least one type of source ontology, such as the human body, different types of heterogeneous humanoid robots, simulation robots, and virtual characters. Correspondingly, the full-body motion sequence and the corresponding source ontology task conditions are also the full-body motion sequence and corresponding source ontology task conditions of at least one type of source ontology. The motion data of the source ontology only includes the full-body motion sequence and the corresponding source ontology task conditions, and does not include the limb joint motor control instructions of the target humanoid robot.

[0027] It should be noted that the source body and the target humanoid robot differ in at least one of the following: skeleton topology, body proportions, joint degrees of freedom, state definition, or control interface. Data between multiple source bodies does not need to be paired with each other.

[0028] The training data for the target humanoid robot includes robot task conditions (e.g., natural language instructions) and robot motion sequences (real motion sequences obtained by collecting data from the target humanoid robot, used to describe the posture state of the robot's joints).

[0029] S12. Based on the first historical motion sequence, the first future motion sequence, and the source ontology task conditions in the whole-body motion sequence, the initial motion network is pre-trained to obtain the pre-trained motion network.

[0030] The whole-body motion sequence is split into a first historical motion sequence and a first future motion sequence. For example, the window of the whole-body motion sequence is 80 frames, with the first 16 frames as the first historical motion sequence and the last 64 frames as the first future motion sequence.

[0031] The initial motion network, also known as the Motion-Expert network, is pre-trained by taking the first historical motion sequence and source ontology task conditions as inputs and using the first future motion sequence as the output training target of the network. This process learns the general limb coordination priors of different source ontology, resulting in a pre-trained motion network.

[0032] S13. Using a pre-trained motion network, motion prediction is performed based on the second historical motion sequence in the robot's motion sequence and the robot's task conditions to obtain the future predicted motion sequence.

[0033] The robot motion sequence of the target humanoid robot is split into a second historical motion sequence and a second future motion sequence. For example, the window of the robot motion sequence is 80 frames, the first 16 frames are the second historical motion sequence, and the last 64 frames are the second future motion sequence.

[0034] By inputting the real historical joint timing sequence (second historical motion sequence) of the target humanoid robot and the robot's task conditions into the pre-trained motion network, the network can predict the future motion of the target humanoid robot and obtain the future predicted motion sequence.

[0035] S14. Based on the second future motion sequence and the future predicted motion sequence in the robot motion sequence, optimize the pre-trained motion network to obtain a robot control network that includes the target motion network.

[0036] Using the true future joint timing sequence (second future motion sequence) of the target humanoid robot as the ground truth, the motion prediction loss is calculated based on the second future motion sequence and the future predicted motion sequence. The pre-trained motion network is then updated based on the motion prediction loss to obtain the target motion network, which is adapted to the mechanical structure and dynamic constraints of the target humanoid robot.

[0037] The target motion network is included in the robot control network deployed on the control equipment.

[0038] In summary, this embodiment provides a model training method for humanoid robots. It independently trains a pre-trained motion network using only low-cost source ontology motion data to extract general limb collaboration priors. The pre-trained motion network is then transferred to the target humanoid robot, and the pre-trained motion network is adapted and optimized by combining it with real robot motion sequences. This solves the shortcomings of existing models, such as lack of cross-ontology compatible motion representation, inability to reuse massive amounts of low-cost source ontology motion data, and high dependence on high-cost real robot data for training.

[0039] Figure 2 This is a schematic flowchart of a method for processing whole-body motion sequences provided in an embodiment of this application, as shown below. Figure 2 As shown, before performing step S12, the method of this application further includes: S111. Convert the whole-body motion sequence to the temporal representation of the motion model corresponding to the target humanoid robot to obtain the converted motion sequence of the source body.

[0040] The whole-body motion sequences of each source ontology are preprocessed by quality filtering, coordinate normalization and time resampling to obtain the preprocessed whole-body motion sequences of each source ontology.

[0041] The preprocessed whole-body motion sequences of each source ontology are subjected to joint semantic mapping and skeleton redirection. The preprocessed whole-body motion sequences of different source ontology are then converted to the temporal representation of the motion model corresponding to the target humanoid robot to obtain the converted motion sequence of the source ontology.

[0042] The transformed motion sequence can be either an explicit joint state, a continuous latent feature, or a discrete motion token. For example, in this embodiment, it is an explicit joint state. In the sequence, each frame of data includes a 60-dimensional continuous skeleton state composed of 3D local axis angles of 21 joints.

[0043] S112. Obtain the first historical motion sequence and the first future motion sequence from the transformed motion sequence.

[0044] The transformed motion sequence is split. For example, if the total window length of the transformed motion sequence is 80 frames, the first 16 frames are defined as the first historical motion sequence and the last 64 frames are defined as the first future motion sequence. If the number of remaining frames at the end of the sequence is greater than 80, they are discarded directly to ensure that each pre-trained sample has a complete historical and future motion sequence.

[0045] In one embodiment, the training data for the target humanoid robot also includes a visual observation image sequence, a body state sequence, and a corresponding robot motion sequence. The visual observation image sequence is a time-series set of first-view RGB images continuously acquired at a fixed sampling frequency by the robot's onboard front camera, sequentially stitched together according to timestamps, fully recording the external environment and object position changes throughout the task execution process. The body state sequence is a set of native joint value vectors, continuously acquired at the same sampling frequency and arranged in time sequence, read in real-time from all driven joint angles by the robot's built-in joint encoder, representing only the robot's real-time body configuration and independent of the external environment. The robot motion sequence consists of motor control commands issued by the robot's underlying controller to each joint actuator.

[0046] Figure 3 This is a schematic diagram of the method for obtaining a robot control network provided in an embodiment of this application, such as... Figure 3 As shown, the method of this application further includes: S100. Using an initial visual network, visual prediction is performed based on historical visual observation images in the visual observation image sequence to obtain future visual prediction images.

[0047] Visual observation image sequences can also be divided into historical visual observation images and future visual observation images. For example, a visual observation image sequence consists of 80 frames, with the first 16 frames as historical visual observation images and the last 64 frames as future visual observation images.

[0048] By inputting historical visual observation images into the initial visual network (Video-Expert) for visual prediction, the future visual prediction image output by the initial visual network can be obtained.

[0049] S200. Using an initial action network, actions are generated based on the future visual prediction image, the future predicted motion sequence, and the corresponding time step of the body state sequence to obtain the future predicted action sequence.

[0050] The future visual prediction image output by the initial visual network, the future predicted motion sequence output by the pre-trained motion network, and the corresponding time step of the ontology state sequence are input into the initial action network (Action-Expert) to generate actions and obtain the future predicted action sequence.

[0051] Based on this, step S14 includes: S141. Based on the second future motion sequence, the future predicted motion sequence, the future visual observation image in the visual observation image sequence, the future visual prediction image, the corresponding future action sequence in the robot action sequence, and the future predicted action sequence, the pre-trained motion network, the initial visual network, and the initial action network are jointly optimized to obtain a robot control network including the target motion network, the target visual network, and the target action network.

[0052] Figure 4 A schematic diagram of the joint optimization method provided in the embodiments of this application is shown below. Figure 4 As shown, S141 specifically includes: S1411. Calculate the motion prediction loss of the pre-trained motion network based on the second future motion sequence and the future predicted motion sequence.

[0053] By comparing the predicted future motion sequence output by the pre-trained motion network with the actual second future motion sequence, the motion prediction loss of the pre-trained motion network can be obtained. .

[0054] S1412. Based on the future visual observation images and the future predicted motion sequence, calculate the visual prediction loss of the initial visual network.

[0055] By comparing the predicted motion sequence output by the initial visual network with the actual future visual observation image, the visual prediction loss of the initial visual network is obtained. .

[0056] S1413. Calculate the action generation loss of the initial action network based on the future action sequence and the future predicted action sequence.

[0057] By comparing the predicted future action sequence output by the initial action network with the actual future action sequence, the action generation loss of the initial action network can be obtained. .

[0058] S1414. Based on the motion prediction loss, visual prediction loss, and action generation loss, the pre-trained motion network, the initial visual network, and the initial action network are jointly optimized to obtain a robot control network that includes the target motion network, the target visual network, and the target action network.

[0059] Figure 5 A schematic diagram of the joint optimization method flow provided in another embodiment of this application is shown below. Figure 5 As shown, S1414 specifically includes: S14141. Calculate the joint optimization loss based on the motion prediction loss, visual prediction loss, and action generation loss.

[0060] Loss based on motion prediction Visual prediction loss and action generation loss Calculate the joint optimization loss , .

[0061] S14142. Based on the joint optimization loss, the pre-trained motion network, the initial vision network, and the initial action network are jointly optimized to obtain a robot control network that includes the target motion network, the target vision network, and the target action network.

[0062] According to joint optimization loss The pre-trained motion network, initial vision network, and initial action network are jointly optimized to obtain a robot control network that includes a target motion network, a target vision network, and a target action network.

[0063] In summary, the model training method of this application sets loss functions for the target motion network, target vision network and target action network branches respectively, which can independently supervise the temporal coordination of the robot's limbs and torso, effectively solving the problems of posture imbalance and limb interference in high-degree-of-freedom humanoid robots.

[0064] This application also provides a control device, which can be installed inside the target humanoid robot body or outside the target humanoid robot body.

[0065] The control device includes a processor, a storage medium, and a bus. The storage medium stores program instructions that can be executed by the processor. When the control device is running, the processor communicates with the storage medium through the bus, and the processor executes the program instructions to implement the motion control method for the humanoid robot described in the following embodiments.

[0066] Figure 6 This is a flowchart illustrating the motion control method for a humanoid robot provided in an embodiment of this application, as shown below. Figure 6 As shown, this application also provides a motion control method for a humanoid robot, including: S101. Obtain the historical motion sequence of the target humanoid robot before the current time and the robot's task conditions.

[0067] After the robot control network is deployed to the control device, the control device can execute the motion control method of the humanoid robot according to the robot control network.

[0068] In this embodiment, the historical motion sequence of the target humanoid robot (e.g., Tiangong robot) before the current time (historical 16 frames, i.e., the historical motion sequence of 16 time steps) and the robot task conditions (e.g., natural language command "stack blocks on the table") are first obtained.

[0069] S102. Using the target motion network in the robot control network, motion prediction is performed based on the historical motion sequence and robot task conditions to obtain the future predicted action sequence for future time steps after the current time.

[0070] The target motion network in the robot control network is invoked to predict motion based on historical motion sequences and robot task conditions, and to infer and output a future predicted motion sequence for 64 future time steps.

[0071] S103. In the future time step, based on the predicted action sequence, perform motion control on the target humanoid robot.

[0072] The control equipment issues joint control commands based on the predicted motion sequence to achieve motion control of the target humanoid robot.

[0073] Figure 7 A flowchart illustrating a motion control method for a humanoid robot provided in another embodiment of this application is shown below. Figure 7 As shown, in another embodiment, if the robot control network includes a target vision network and a target motion network, the motion control method for the humanoid robot further includes: S201. Obtain the observation image and body state of the target humanoid robot at the current time.

[0074] The system acquires the observation image of the target humanoid robot at the current time from the onboard front camera, and the body state of the robot at the current time from the robot's built-in joint encoder.

[0075] S202. Using the target vision network in the robot control network, visual prediction is performed based on the observed image at the current time to obtain the future visual prediction image for the next time step.

[0076] By employing the target vision network in the robot control network, visual prediction is performed based on the observed image at the current time, resulting in future visual prediction images for 64 future time steps.

[0077] S103 includes: S203. Using the target action network in the robot control network, the action prediction is performed based on the future visual prediction image of the future time step, the future predicted action sequence, and the current body state to obtain the future predicted action of the future time step.

[0078] The target action network in the robot control network is used to predict the future action at the future time step based on the future visual prediction images at 64 future time steps, the future predicted motion sequence at 64 future time steps, and the body state at the current time.

[0079] S204. In the future time step, based on the predicted future actions, perform motion control on the target robot.

[0080] Only the first 10 control commands of the action segment are executed. After execution, the image and body state are re-acquired, the 16-frame historical motion window is updated, and the data is sent back to the robot control network for inference. The cycle continues to correct the prediction deviation caused by the perspective and object collision, so as to achieve high-frequency and long-term stable control.

[0081] This application also provides a humanoid robot, including a humanoid robot body and a control device, wherein the control device is used for the motion control method of the humanoid robot described in the above embodiments.

[0082] The following will continue to explain the apparatus, device, and storage medium for implementing the method provided in any of the above embodiments of this application. The specific implementation process and the resulting technical effects are the same as those in the corresponding method embodiments. For the sake of brevity, the parts not mentioned in the following embodiments can be referred to the corresponding content in the method embodiments.

[0083] Figure 8 This is a schematic diagram of the structure of a model training device for a humanoid robot provided in an embodiment of this application, as shown below. Figure 8 As shown, this application provides a model training device for humanoid robots, comprising: The first acquisition module 10 is used to acquire motion data of the source body and training data of the target humanoid robot. The motion data of the source body includes: whole-body motion sequence and corresponding source body task conditions. The training data of the target humanoid robot includes: robot task conditions and robot motion sequence.

[0084] The first training module 20 is used to pre-train the initial motion network based on the first historical motion sequence, the first future motion sequence and the source ontology task conditions in the whole-body motion sequence, to obtain a pre-trained motion network.

[0085] The second training module 30 is used to use the pre-trained motion network to predict the future motion sequence based on the second historical motion sequence in the robot motion sequence and the robot task conditions.

[0086] The optimization module 40 is used to optimize the pre-trained motion network based on the second future motion sequence and the future predicted motion sequence in the robot motion sequence, so as to obtain a robot control network including the target motion network.

[0087] Optionally, the first acquisition module 10 is used to convert the whole-body motion sequence into a temporal representation of the motion model corresponding to the target humanoid robot to obtain the converted motion sequence of the source body; and to acquire the first historical motion sequence and the first future motion sequence from the converted motion sequence.

[0088] Optionally, the training data for the target humanoid robot further includes: a sequence of visual observation images, a sequence of body states, and a corresponding sequence of robot actions; the second training module 30 is further configured to use an initial visual network to perform visual prediction based on historical visual observation images in the sequence of visual observation images to obtain a future visual prediction image; and to use an initial action network to generate actions based on the future visual prediction image, the future predicted motion sequence, and the body states at corresponding time steps in the sequence of body states to obtain a future predicted action sequence.

[0089] The optimization module 40 is used to jointly optimize the pre-trained motion network, the initial vision network, and the initial action network based on the second future motion sequence, the future predicted motion sequence, the future visual observation image in the visual observation image sequence, the future visual prediction image, the corresponding future action sequence in the robot action sequence, and the future predicted action sequence, to obtain a robot control network including a target motion network, a target vision network, and a target action network.

[0090] Optionally, the optimization module 40 is configured to calculate the motion prediction loss of the pre-trained motion network based on the second future motion sequence and the future predicted motion sequence; calculate the visual prediction loss of the initial visual network based on the future visual observation image and the future visual predicted image; calculate the action generation loss of the initial action network based on the future action sequence and the future predicted action sequence; and jointly optimize the pre-trained motion network, the initial visual network, and the initial action network based on the motion prediction loss, the visual prediction loss, and the action generation loss to obtain a robot control network including the target motion network, the target visual network, and the target action network.

[0091] Optionally, the optimization module 40 is configured to calculate a joint optimization loss based on the motion prediction loss, the visual prediction loss, and the action generation loss; and to perform joint optimization on the pre-trained motion network, the initial visual network, and the initial action network based on the joint optimization loss to obtain a robot control network including the target motion network, the target visual network, and the target action network.

[0092] Figure 9 This is a schematic diagram of the motion control device for the humanoid robot provided in the embodiments of this application, as shown below. Figure 9 As shown, this application provides a motion control device for a humanoid robot, comprising: The second acquisition module 50 is used to acquire the target robot's historical motion sequence before the current time and the robot's task conditions.

[0093] The third acquisition module 60 is used to perform motion prediction based on the historical motion sequence and the robot task conditions using the target motion network in the robot control network, and obtain the future predicted action sequence for the future time step after the current time.

[0094] The control module 70 is used to perform motion control on the target robot at the future time step based on the future predicted action sequence.

[0095] Optionally, if the robot control network includes a target vision network and a target action network; the third acquisition module 60 is further configured to acquire the observation image and body state of the target robot at the current time; and use the target vision network in the robot control network to perform visual prediction based on the observation image at the current time to obtain the future visual prediction image for the future time step. The control module 70 is further configured to use the target action network in the robot control network to perform action prediction based on the future predicted action sequence of the future time step, the future predicted action sequence, and the body state at the current time, to obtain the future predicted action of the future time step; and to perform motion control on the target robot based on the future predicted action at the future time step.

[0096] The above-described device is used to execute the method provided in the foregoing embodiments, and its implementation principle and technical effect are similar, so they will not be described again here.

[0097] These modules can be one or more integrated circuits configured to implement the above methods, such as one or more Application Specific Integrated Circuits (ASICs), one or more microprocessors, or one or more Field Programmable Gate Arrays (FPGAs). Alternatively, when a module is implemented using processing element scheduler code, the processing element can be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. Furthermore, these modules can be integrated together as a system-on-a-chip (SOC).

[0098] This application also provides a readable storage medium storing program instructions that, when executed by a processor, implement the methods described in any of the above embodiments.

[0099] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0100] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0101] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in a combination of hardware and software functional units.

[0102] The integrated units implemented as software functional units described above can be stored in a computer-readable storage medium. These software functional units, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0103] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A model training method for humanoid robots, characterized in that, The method includes: The motion data of the source body and the training data of the target humanoid robot are acquired. The motion data of the source body includes: whole-body motion sequence and corresponding source body task conditions. The training data of the target humanoid robot includes: robot task conditions and robot motion sequence. Based on the first historical motion sequence, the first future motion sequence in the whole-body motion sequence and the source ontology task conditions, the initial motion network is pre-trained to obtain the pre-trained motion network. Using the pre-trained motion network, motion prediction is performed based on the second historical motion sequence in the robot motion sequence and the robot task conditions to obtain the future predicted motion sequence; Based on the second future motion sequence and the future predicted motion sequence in the robot motion sequence, the pre-trained motion network is optimized to obtain a robot control network that includes the target motion network.

2. The method according to claim 1, characterized in that, Before performing pre-training based on the first historical motion sequence, the first future motion sequence, and the source ontology task conditions in the whole-body motion sequence to obtain the pre-trained motion network, the method further includes: The whole-body motion sequence is converted into the temporal representation of the motion model corresponding to the target humanoid robot to obtain the converted motion sequence of the source body; The first historical motion sequence and the first future motion sequence are obtained from the transformed motion sequence.

3. The method according to claim 1, characterized in that, The training data for the target humanoid robot also includes: visual observation image sequences, body state sequences, and corresponding robot action sequences; The method further includes: An initial visual network is used to perform visual prediction based on historical visual observation images in the visual observation image sequence to obtain future visual prediction images. An initial action network is used to generate actions based on the future visual prediction image, the future predicted motion sequence, and the body state at the corresponding time step in the body state sequence, thereby obtaining the future predicted action sequence. The step of optimizing the pre-trained motion network based on the second future motion sequence and the future predicted motion sequence in the robot motion sequence to obtain a robot control network including the target motion network includes: Based on the second future motion sequence, the future predicted motion sequence, the future visual observation image in the visual observation image sequence, the future visual predicted image, the corresponding future action sequence in the robot action sequence, and the future predicted action sequence, the pre-trained motion network, the initial visual network, and the initial action network are jointly optimized to obtain a robot control network including a target motion network, a target visual network, and a target action network.

4. The method according to claim 3, characterized in that, The step of jointly optimizing the pre-trained motion network, the initial vision network, and the initial action network based on the second future motion sequence, the future predicted motion sequence, the future visual observation image in the visual observation image sequence, the future visual prediction image, the corresponding future action sequence in the robot action sequence, and the future predicted action sequence, to obtain a robot control network including a target motion network, a target vision network, and a target action network, includes: The motion prediction loss of the pre-trained motion network is calculated based on the second future motion sequence and the future predicted motion sequence. Based on the future visual observation image and the future visual prediction image, calculate the visual prediction loss of the initial visual network; Calculate the action generation loss of the initial action network based on the future action sequence and the future predicted action sequence; Based on the motion prediction loss, the visual prediction loss, and the action generation loss, the pre-trained motion network, the initial visual network, and the initial action network are jointly optimized to obtain a robot control network that includes the target motion network, the target visual network, and the target action network.

5. The method according to claim 4, characterized in that, The step of jointly optimizing the pre-trained motion network, the initial vision network, and the initial action network based on the motion prediction loss, the vision prediction loss, and the action generation loss to obtain a robot control network including the target motion network, the target vision network, and the target action network includes: Calculate the joint optimization loss based on the motion prediction loss, the visual prediction loss, and the action generation loss; Based on the joint optimization loss, the pre-trained motion network, the initial vision network, and the initial action network are jointly optimized to obtain a robot control network including the target motion network, the target vision network, and the target action network.

6. A motion control method for a humanoid robot, characterized in that, The method includes: Obtain the target humanoid robot's historical motion sequence before the current time and the robot's task conditions; Using the target motion network in the robot control network, motion prediction is performed based on the historical motion sequence and the robot task conditions to obtain the future predicted action sequence for future time steps after the current time. At the future time step, motion control is performed on the target humanoid robot based on the predicted future action sequence.

7. The method according to claim 6, characterized in that, If the robot control network includes a target vision network and a target motion network, the method further includes: Acquire the observed image and body state of the target humanoid robot at the current time; Using the target vision network in the robot control network, visual prediction is performed based on the observed image at the current time to obtain the future visual prediction image for the future time step. The step of performing motion control on the target humanoid robot based on the predicted future action sequence at the future time step includes: Using the target action network in the robot control network, action prediction is performed based on the future predicted action sequence at the future time step, the future predicted action sequence, and the body state at the current time to obtain the future predicted action at the future time step. At the future time step, motion control is performed on the target humanoid robot based on the predicted future action.

8. A computer device, characterized in that, include: The computer device includes a processor, a storage medium, and a bus. The storage medium stores program instructions executable by the processor. When the computer device is running, the processor communicates with the storage medium via the bus, and the processor executes the program instructions to implement the model training method for a humanoid robot as described in any one of claims 1 to 5.

9. A control device, characterized in that, include: The device includes a processor, a storage medium, and a bus. The storage medium stores program instructions executable by the processor. When the control device is running, the processor communicates with the storage medium via the bus, and the processor executes the program instructions to implement the motion control method for the humanoid robot according to any one of claims 6 to 7.

10. A humanoid robot, characterized in that, include: A humanoid robot body and a control device, wherein the control device is used to execute the motion control method of the humanoid robot according to any one of claims 6 to 7.

Images