Humanoid robot control method and system, storage medium and program product

Abstract

The invention provides a humanoid robot control method and system, a storage medium and a program product, and belongs to the field of computer vision. The method comprises the following steps: preprocessing expert action data, and processing the expert action data into expert data equivalent to a skeleton architecture of a target robot; in a simulation environment, a humanoid structure is adopted to build a robot, joint parameters of the robot are configured, and the degree of freedom of each joint is controlled by an independent physical control module; constructing a strategy representation method of the robot, wherein the strategy representation method comprises a state space, an action space, a reward function and a multi-frame control mode; the robot is initialized; and the difference between the robot action and the expert action on each frame is minimized, the reward function is maximized, and the robot is driven to learn. The method can assist the learning process of the humanoid robot, so that the robot can be personified while completing tasks, and the training speed is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a humanoid robot control method, system, storage medium and program product. Background Art

[0002] With the continuous development of robot-related technologies, users have higher and higher requirements for the functionality and adaptability of robots. Most traditional industrial robots have a single application scenario and cannot be generalized to daily production and life. As a result, humanoid robot technology came into being.

[0003] However, the current humanoid robot control technology cannot control the robot to complete various complex movements and tasks, and cannot be anthropomorphic. When using reinforcement learning as a robot control algorithm, it takes a long time and is inefficient for the robot to produce high-quality movements, which cannot meet the needs of robot control.

[0004] In recent years, imitation learning-related technologies have developed rapidly, and it has been proven to effectively solve the problem of inefficiency caused by too large an exploration space in reinforcement learning. It is of great significance to study the control method of humanoid robots based on imitation learning, so that robots can produce diverse actions and complete complex tasks while being humanized. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention provides a humanoid robot control method, system, storage medium and program product, which adopts imitation learning method for motion control, aiming to solve the problem that it takes a long time and is inefficient to produce high-quality movements of the robot, and cannot meet the control needs of the robot, so as to enable the robot to produce diverse movements and complete complex tasks while being humanized.

[0006] In a first aspect, the present invention provides a method for controlling a humanoid robot, comprising the following steps:

[0007] Preprocess the expert motion data to convert the expert motion data into expert data equivalent to the target robot skeleton structure;

[0008] In the simulation environment, a humanoid structure is used to build a robot, and the robot's joint parameters are configured. Each joint degree of freedom is controlled by an independent physical control module;

[0009] Construct a robot strategy representation method, including state space, action space, reward function, and multi-frame control method;

[0010] Initialize the robot;

[0011] Minimize the difference between the robot's actions and the expert's actions in each frame, maximize the reward function, and drive the robot to learn.

[0012] As a further improvement of the present invention, the steps are also included:

[0013] Put the state in the state space, the action in the action space, and the reward in the reward function into the experience buffer pool. After the experience buffer pool is filled, execute the strategy update;

[0014] Sample data from the experience buffer pool and perform updates on the policy network and value network.

[0015] As a further improvement of the present invention, the preprocessing of the expert motion data, processing the expert motion data into expert data equivalent to the skeleton structure of the target robot, includes:

[0016] Build an expert action dataset;

[0017] Perform skeleton redirection on the expert motion data to adapt the skeleton parameters of the target robot;

[0018] Fine-tuned by the inverse kinematics algorithm, it constitutes an expert dataset for imitation learning.

[0019] As a further improvement of the present invention, the building components of the robot include a rigid body, a collision body, and a configurable joint;

[0020] The joint parameters of the robot include rigid body weight, collision range, joint degrees of freedom, and joint rotation range.

[0021] As a further improvement of the present invention, the state space includes the position of the pelvis relative to the ground origin, the center of mass position of each rigid body, rotation information, linear velocity information, angular velocity information, and time variable.

[0022] As a further improvement of the present invention, the position of the pelvis relative to the origin of the ground is the first three dimensions of the state space, calculated using a world coordinate system, for providing a relative relationship between the robot and the ground;

[0023] The center of mass position of each rigid body segment, the linear velocity information, and the angular velocity information are all three-dimensional information, and are all calculated using the pelvic coordinate system;

[0024] The rotation information is a six-dimensional feature vector, calculated using a pelvic coordinate system;

[0025] The time variable is the last dimension of the state space, which is responsible for providing the time information when the robot performs actions and is used to model timing information.

[0026] As a further improvement of the present invention, the action space generates motion by controlling the rotation of joints on each rotational degree of freedom, and when the robot moves, the rotation of the joints is a continuous variable;

[0027] The method for representing the action space includes:

[0028] Gaussian distribution is used to model the action space, and the rotation on each degree of freedom is modeled as an independent Gaussian distribution;

[0029] After completing the motion distribution modeling, sampling is performed in the distribution, and the sampling result is the target rotation of the joint;

[0030] In order to convert the target rotation value into joint torque, the robot is driven to move through the joint torque.

[0031] As a further improvement of the present invention, in the reward function, the robot's action imitation reward function is expressed as:

[0032]

[0033] w pos =0.2,w rot =0.5,w vel =0.1,w end =0.1,w cof =0.1;

[0034] The reward is in the form of a weighted average of position reward, rotation reward, speed reward, end effector reward, and center of mass reward;

[0035] Among them, w is the weight coefficient of each item, r is the specific reward value of each item; reward Each item from left to right is: position reward, rotation reward, speed reward, end effector reward, center of mass reward.

[0036] As a further improvement of the present invention, the multi-frame control method includes:

[0037] The historical states of n frames before the current state are concatenated together to form a new feature vector, which is used as the input of the policy network to capture the timing information between actions.

[0038] The input of the policy network is represented as:

[0039]

[0040] in, It represents the state vector composed of the concatenation of the previous n frames of historical states of the current state;

[0041] The strategy network includes a fully connected network, several LSTM modules and a linear layer network;

[0042] At each step of the simulation update in the simulation environment, the timing information is fed into the policy network, which outputs the mean of the action distribution and then samples an action a from the action distribution.

[0043] The action distribution is characterized as follows:

[0044]

[0045] Among them, a is the action value, which specifically means joint rotation; s is the robot state value returned by the simulation environment; π θ is the conditional probability distribution from state to action, which is modeled as a Gaussian distribution; the mean is the output of the network The variance is a fixed initial value ∑.

[0046] As a further improvement of the present invention, the initializing the robot includes:

[0047] By using expert state initialization and sampling states from reference motion, the robot can initially sample any state on the motion trajectory.

[0048] As a further improvement of the present invention, the method of minimizing the difference between the robot action and the expert action in each frame, maximizing the reward function, and driving the robot to learn includes:

[0049] Reinforcement learning training, performing policy gradient ascent to maximize the reward function;

[0050] Monitor the training process and perform early stopping when the robot training crashes to prevent the strategy from falling into a trough and being unable to jump out;

[0051] Get the trained model;

[0052] Put it into a simulation environment, and make the robot anthropomorphic while completing the target action.

[0053] In a second aspect, the present invention provides a computer system, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the above method.

[0054] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the above method when executed by a processor.

[0055] In a fourth aspect, the present invention provides a computer program product, comprising a computer program, which implements the steps of the above method when executed by a processor.

[0056] Compared with the prior art, the present invention has the following beneficial effects:

[0057] Using a humanoid robot control method based on imitation learning can assist the learning process of humanoid robots, allowing robots to be anthropomorphic while completing tasks, and the training speed is improved. In various scenarios, robots can learn human actions through imitation learning methods to perform complex actions. When new action requirements arise, the robot can learn new actions with only a piece of expert action data, which is more efficient. In addition, anthropomorphic robots are more acceptable to users and more heartwarming. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 A flow chart of a method for controlling a humanoid robot disclosed in the present invention;

[0059] Figure 2 The present invention is a schematic diagram of the principle of a method for controlling a humanoid robot disclosed in the present invention. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the present invention will combine the accompanying drawings to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments, wherein steps S1, S2... in the embodiments described in the present invention do not limit the only execution steps of the present invention; the various models, simulation environments, and software described in the present invention are not the only limiting methods of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0061] In the present invention, "module", "system", etc. refer to related entities applied to computers, such as hardware, a combination of hardware and software, software or software in execution, etc. In detail, for example, software includes but is not limited to a process running on a processor, a processor, an object, executable software, an execution thread, a program and / or a computer. In addition, an application or script program running on a server, and a server can all be software. One or more software can be in an execution process and / or thread, and the software can be localized on one computer and / or distributed between two or more computers, and can be run by various computer-readable media.

[0062] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application may be combined with each other.

[0063] like Figure 1 and Figure 2 As shown, in an embodiment of the present invention, in a first aspect, a method for controlling a humanoid robot is provided, and the specific process may be as follows:

[0064] S1: Preprocess the expert motion data and process the expert motion data into expert data equivalent to the target robot skeleton architecture;

[0065] Preferably, the preprocessing of the expert motion data, processing the expert motion data into expert data equivalent to the skeleton structure of the target robot, includes:

[0066] S11: Build expert action dataset;

[0067] S12: Performing skeletal redirection on the expert motion data to adapt the skeletal parameters of the target robot;

[0068] S13: Fine-tuned by the inverse kinematics algorithm to form an imitation learning expert dataset.

[0069] S2: Build a robot using a humanoid structure in a simulation environment, configure the robot's joint parameters, and control each joint's degree of freedom by an independent physical control module;

[0070] Preferably, the robot's building components include a rigid body, a collision body, and a configurable joint;

[0071] The joint parameters of the robot include rigid body weight, collision range, joint degrees of freedom, and joint rotation range.

[0072] In one embodiment of the present invention, in the Unity simulation physics engine, a 1.75-meter-high, 65-kg-weight humanoid simulation robot with multiple degrees of freedom and high-dimensional features is built using rigid bodies, collision bodies, configurable joints and other components, and joint parameters such as rigid body weight, collision range, joint degrees of freedom and joint rotation range are configured to ensure that the humanoid simulation robot meets the dynamic requirements. Each joint degree of freedom is controlled by an independent physical control module to ensure the flexibility of the robot. The robot has a total of 18 rigid bodies, 17 joints, and 31 rotational degrees of freedom.

[0073] S3: Construct a robot strategy representation method, including state space, action space, reward function, and multi-frame control method;

[0074] Preferably, the state space includes the position of the pelvis relative to the ground origin, the center of mass position of each rigid body segment, rotation information, linear velocity information, angular velocity information, and time variables.

[0075] Preferably, the position of the pelvis relative to the origin of the ground is the first three dimensions of the state space, calculated using a world coordinate system, for providing a relative relationship between the robot and the ground;

[0076] The center of mass position of each rigid body segment, the linear velocity information, and the angular velocity information are all three-dimensional information, and are all calculated using the pelvic coordinate system;

[0077] The rotation information is a six-dimensional feature vector, calculated using a pelvic coordinate system;

[0078] The time variable is the last dimension of the state space, which is responsible for providing the time information when the robot performs actions and is used to model timing information.

[0079] In one embodiment of the present invention, the robot has a total of 18 rigid bodies, 17 joints, and 31 rotational degrees of freedom. For the state space, the first three dimensions are the position of the pelvis relative to the origin of the ground, which is calculated in the world coordinate system and is used to provide the relative relationship between the robot and the ground; in addition, information on other parts of the robot's body is also considered, including the center of mass position, rotation information, linear velocity information, and angular velocity information of each rigid body. Among them, the center of mass position, linear velocity information, and angular velocity information of each rigid body are three-dimensional information, all calculated in the pelvic coordinate system; the rotation information is a six-dimensional eigenvector, calculated in the pelvic coordinate system. The last dimension is the time variable t, which is responsible for providing the time information when the robot action is executed, and is used to model timing information, so as to capture and simulate more complex behaviors. The value of the time variable t is related to the current system update frequency and the total duration of the current execution action. To summarize the above information, each joint segment has three-dimensional position information, three-dimensional linear velocity information, three-dimensional angular velocity information and six-dimensional state information, so the total dimension of the state space is 18*(3+6+3+3)+1=271, and the last dimension is the time variable dimension. Each segment of the rigid body is expressed with independent features to ensure more accurate description and control of the robot's movements.

[0080] Preferably, the action space generates motion by controlling the rotation of joints on each rotational degree of freedom, and when the robot moves, the rotation of the joints is a continuous variable;

[0081] The method for representing the action space includes:

[0082] S31: Gaussian distribution is used to model the action space, and the rotation on each degree of freedom is modeled as an independent Gaussian distribution;

[0083] S32: After completing the action distribution modeling, sampling is performed in the distribution, and the sampling result is the target rotation of the joint;

[0084] S33: To convert the target rotation value into joint torque, the robot is driven to move through the joint torque.

[0085] In one embodiment of the present invention, the robot has a total of 18 rigid bodies, 17 joints, and 31 rotational degrees of freedom. For the action space, a control method of controlling the rotation of joints on each rotational degree of freedom is used to generate motion. The robot has 31 degrees of freedom, so the action space is 31 dimensions. When the robot moves, the joint rotation is a continuous variable.

[0086] The characterization methods of the motion space include: in order to more accurately describe the joint rotation, Gaussian distribution is used to model the motion space, and the rotation on each degree of freedom is modeled as an independent Gaussian distribution, which provides a continuous and smooth way to describe the action. This construction method is flexible enough to independently adjust the action on each degree of freedom according to different needs and scenarios; after completing the motion distribution modeling, sampling is performed in the distribution, and the sampling result is the target rotation of the joint; finally, in order to convert the target rotation value into joint torque, PD-control is required to perform torque solution, and the robot is driven to move through the joint torque.

[0087] Preferably, in the reward function, the action imitation reward function expression formula of the robot at time t is:

[0088]

[0089] w pos =0.2,w rot =0.5,w vel =0.1,w end =0.1,w cof =0.1;

[0090] The reward is in the form of a weighted average of position reward, rotation reward, speed reward, end effector reward, and center of mass reward;

[0091] Among them, w is the weight coefficient of each item, r is the specific reward value of each item; reward Each item from left to right is: position reward, rotation reward, speed reward, end effector reward, center of mass reward.

[0092] Preferably, the multi-frame control method includes:

[0093] The historical states of n frames before the current state are concatenated together to form a new feature vector as the input of the policy network. Since there is one-dimensional time information in the state space design, the timing information between actions can be captured.

[0094] The input of the policy network is represented as:

[0095]

[0096] in, It represents the state vector composed of the concatenation of the previous n frames of historical states of the current state;

[0097] The policy network includes a fully connected network, several LSTM modules and a linear layer network. In order to better capture the timing information, several LSTM modules are stacked after the fully connected network, and the output layer of the fully connected network is changed to a feature output layer; the output of the LSTM layer passes through a linear layer network, and the final output dimension size is equal to the action space size.

[0098] At each step of the simulation update in the simulation environment, the timing information is fed into the policy network, which outputs the mean of the action distribution and then samples an action a from the action distribution.

[0099] The action distribution is characterized as follows, where:

[0100]

[0101] Among them, a is the action value, which specifically means joint rotation; s is the robot state value returned by the simulation environment; π θ is the conditional probability distribution from state to action, which is modeled as a Gaussian distribution; the mean is the output of the network The variance is a fixed initial value ∑.

[0102] As for the control method, different from the traditional single-frame control, the present invention adopts a multi-frame control method. Considering the correlation of continuous frame information, the policy network representation method that only uses the MLP network and adopts a single-frame input is improved. Multi-frame control introduces more contextual information and enhances the model's understanding of sequential actions.

[0103] S4: Initialize the robot to ensure the optimal training effect of the robot;

[0104] Preferably, the initializing the robot includes:

[0105] By using expert state initialization and sampling states from reference motion, the robot can initially sample any state on the motion trajectory.

[0106] The traditional initial state distribution p(s0) is used to determine the initial state of the robot in each round. Usually, the robot is initialized to a fixed state (Fixed Initialization, FI). The specific implementation of this initialization strategy is to initialize the character to the starting state of the reference motion in each round, and then use the strategy to drive the robot to match the actions in the reference motion frame by frame until the entire action segment ends. This initialization strategy makes the character's initial exploration space smaller, and only after accessing the state in the reference motion trajectory can it obtain a high reward. Before accessing the high-reward state, the strategy cannot determine whether the state is beneficial through the value function, which limits the character's exploration behavior and is not conducive to training.

[0107] The present invention adopts Expert Initialization (EI). By sampling states from reference motion, the robot can sample any states on the motion trajectory at the early stage. Even before the robot explores these states, the EI strategy can find that high returns can be obtained in these states.

[0108] S5: Minimize the difference between the robot's actions and the expert's actions in each frame, maximize the reward function, and drive the robot to learn.

[0109] Preferably, minimizing the difference between the robot action and the expert action in each frame, maximizing the reward function, and driving the robot to learn includes:

[0110] S51: Reinforcement learning training, performing policy gradient ascent to maximize the reward function;

[0111] S52: Monitor the training process and execute early stop when the robot training crashes to prevent the strategy from falling into a trough and being unable to jump out;

[0112] S53: Get the trained model;

[0113] S54: Place the robot in a simulation environment to humanize it while completing the target action.

[0114] In one embodiment of the present invention, the PPO algorithm is used to perform reinforcement learning training on the robot's strategy, and the strategy gradient ascent is performed to maximize the reward function. The PPO algorithm includes a strategy network and a value network. The strategy network is responsible for learning the robot's behavior strategy, and the value network is responsible for evaluating the behavior of the strategy network. At the same time, the training process is monitored, and early stopping is performed when the robot training crashes to prevent the strategy from entering a trough and being unable to jump out. Finally, a trained model is obtained. Put it into the Unity simulation environment, and the robot can be anthropomorphized while completing the target action.

[0115] In another embodiment of the present invention, in addition to steps S1 to S5, the following steps may also be included:

[0116] S6: Put the state in the state space, the action in the action space, and the reward in the reward function into the experience buffer pool. After the experience buffer pool is filled, execute the strategy update;

[0117] S7: Sample data from the experience buffer pool and execute updates of the policy network and value network.

[0118] like Figure 2As shown, the principle of a humanoid robot control method based on imitation learning provided in an embodiment of the present invention is: in the training stage, an initial state s0 is sampled from the expert data set at the beginning of each round, and the state is used to initialize the role in the simulation environment. At each step of the simulation update of the simulation environment, the timing information is sent to the policy network, the policy network outputs the mean of the action distribution, and then an action a is sampled from the action distribution; the state, action, and reward are placed in the experience buffer pool, and after the experience buffer pool is filled, the policy update is executed; a small batch of data is sampled from the experience buffer pool, and the policy network and the value network are updated.

[0119] In a second aspect, in an embodiment of the present invention, the present invention provides a computer system, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the above method.

[0120] In a third aspect, in an embodiment of the present invention, the present invention provides a computer-readable storage medium having a computer program stored thereon, and the computer program implements the steps of the above method when executed by a processor.

[0121] In a fourth aspect, in an embodiment of the present invention, the present invention provides a computer program product, including a computer program, which implements the steps of the above method when executed by a processor.

[0122] The present invention uses a humanoid robot control method based on imitation learning, which is applicable to the fields of virtual human animation production, simulation robot control, etc. It can assist the learning process of humanoid robots, so that robots can be anthropomorphic while completing tasks, and the training speed is improved. For example, in a scene facing catering services, robots can learn human actions through imitation learning methods to perform complex actions such as wiping tables, sweeping floors, and delivering meals. When new action requirements arise, the robot can learn new actions only through a piece of expert action data, which is more efficient. In addition, anthropomorphic robots are more acceptable to users and more heartwarming.

Claims

1. A method for controlling a humanoid robot, characterized in that: The following steps are involved: Preprocess the expert motion data to convert the expert motion data into expert data equivalent to the target robot skeleton structure; In the simulation environment, a humanoid structure is used to build a robot, and the robot's joint parameters are configured. Each joint degree of freedom is controlled by an independent physical control module; Construct a robot strategy representation method, including state space, action space, reward function, and multi-frame control method; Initialize the robot; Minimize the difference between the robot's actions and the expert's actions in each frame, maximize the reward function, and drive the robot to learn.

2. The humanoid robot control method according to claim 1, characterized in that: Also includes the steps: Put the state in the state space, the action in the action space, and the reward in the reward function into the experience buffer pool. After the experience buffer pool is filled, execute the strategy update; Sample data from the experience buffer pool and perform updates on the policy network and value network.

3. The humanoid robot control method according to claim 1, characterized in that: The preprocessing of the expert motion data, processing the expert motion data into expert data equivalent to the skeleton structure of the target robot, includes: Build an expert action dataset; Perform skeleton redirection on the expert motion data to adapt the skeleton parameters of the target robot; Fine-tuned by the inverse kinematics algorithm, it constitutes an expert dataset for imitation learning.

4. The humanoid robot control method according to claim 1, characterized in that: The robot's building components include a rigid body, a collision body, and configurable joints; The joint parameters of the robot include rigid body weight, collision range, joint degrees of freedom, and joint rotation range.

5. The humanoid robot control method according to claim 1 or 2, characterized in that: The state space includes the position of the pelvis relative to the ground origin, the center of mass position of each rigid body segment, rotation information, linear velocity information, angular velocity information, and time variables.

6. The humanoid robot control method according to claim 5, characterized in that: The position of the pelvis relative to the ground origin is the first three dimensions of the state space, calculated using the world coordinate system, and is used to provide the relative relationship between the robot and the ground; The center of mass position of each rigid body segment, the linear velocity information, and the angular velocity information are all three-dimensional information, and are all calculated using the pelvic coordinate system; The rotation information is a six-dimensional feature vector, calculated using a pelvic coordinate system; The time variable is the last dimension of the state space, which is responsible for providing the time information when the robot performs actions and is used to model timing information.

7. The humanoid robot control method according to claim 1 or 2, characterized in that: The action space generates motion by controlling the rotation of joints on each rotational degree of freedom. When the robot moves, the rotation of joints is a continuous variable. The method for representing the action space includes: Gaussian distribution is used to model the action space, and the rotation on each degree of freedom is modeled as an independent Gaussian distribution; After completing the motion distribution modeling, sampling is performed in the distribution, and the sampling result is the target rotation of the joint; In order to convert the target rotation value into joint torque, the robot is driven to move through the joint torque.

8. The humanoid robot control method according to claim 1 or 2, characterized in that: In the reward function, the robot's action imitation reward function is expressed as: In pos =0.2,in rot =0.5,in vel =0.1,in end =0.1,in cof =0.1; The reward is in the form of a weighted average of position reward, rotation reward, speed reward, end effector reward, and center of mass reward; Among them, w is the weight coefficient of each item, r is the specific reward value of each item; reward Each item from left to right is: position reward, rotation reward, speed reward, end effector reward, center of mass reward.

9. The humanoid robot control method according to claim 1, characterized in that: The multi-frame control method includes: The historical states of n frames before the current state are concatenated together to form a new feature vector, which is used as the input of the policy network to capture the timing information between actions. The input of the policy network is represented as: in, It represents the state vector composed of the concatenation of the previous n frames of historical states of the current state; The strategy network includes a fully connected network, several LSTM modules and a linear layer network; At each step of the simulation update in the simulation environment, the timing information is fed into the policy network, which outputs the mean of the action distribution and then samples an action a from the action distribution. The action distribution is characterized as follows: Among them, a is the action value, which specifically means joint rotation; s is the robot state value returned by the simulation environment; π θ is the conditional probability distribution from state to action, which is modeled as a Gaussian distribution; the mean is the output of the network The variance is a fixed initial value ∑.

10. The humanoid robot control method according to claim 1, characterized in that: The initialization of the robot includes: By using expert state initialization and sampling states from reference motion, the robot can initially sample any state on the motion trajectory.

11. The humanoid robot control method according to claim 1, characterized in that: The method of minimizing the difference between the robot action and the expert action in each frame, maximizing the reward function, and driving the robot to learn includes: Reinforcement learning training, performing policy gradient ascent to maximize the reward function; Monitor the training process and perform early stopping when the robot training crashes to prevent the strategy from falling into a trough and being unable to jump out; Get the trained model; Put it into a simulation environment, and make the robot anthropomorphic while completing the target action.

12. A computer system comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1-11.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

14. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

Images