Humanoid robot motion imitation method and system based on human linear model

Abstract

The application provides a humanoid robot action imitation method and system based on a human linear model, comprising the following steps: S1, collecting video input of human action and extracting action information frame by frame; S2, modeling based on the action information and obtaining SMPL representation and three-dimensional representation of human posture; S3, generating SMPL sequence data in the time dimension and three-dimensional representation of human action based on the SMPL representation and three-dimensional representation of human posture, and constructing a robot dynamics tree model; S4, planning a robot motion trajectory through the dynamics tree model and generating corresponding control instructions. The application accurately captures human dynamics by mesh modeling and extracting an SMPL model for human action. Based on the SMPL sequence of motion, a dynamics tree model suitable for humanoid robot motion is constructed, real-time imitation of human action by the robot is realized, and the accuracy and real-time performance of the action are improved.

Description

Technical Field

[0001] The present invention belongs to the field of human-computer interaction technology, and in particular, relates to a method and system for imitating the motion of a humanoid robot based on a human linear model. More particularly, it relates to a method for imitating the motion of a humanoid robot based on a skinned human linear model. Background Art

[0002] Currently, most humanoid robot methods for imitating human motion are limited to localized movements, such as gait and bimanual manipulation. These methods typically focus on only a portion of the human body, such as the simulation of arms or a single limb. These methods often neglect the dynamic modeling of the entire humanoid, failing to fully simulate complex, dynamic human motion. Consequently, these methods lack the ability to accurately capture and simulate the entire human motion process in real time, hindering the ability to achieve more natural and smooth humanoid robot motion imitation.

[0003] Patent document CN116079727A discloses a method and device for humanoid robot motion imitation based on 3D human body posture estimation. This method collects human body posture point cloud data and estimates the positions of human skeletal points to obtain the coordinates of key human body points. These coordinates are then mapped to the robot's arm joint angles and filtered and smoothed, thereby enabling the humanoid robot to imitate the instructor's movements. However, this type of method mainly targets static human body movements and can only imitate the movements of the arms. Its disadvantage is that it cannot capture the dynamic changes during human movement and lacks humanoid robot motion dynamics modeling, resulting in the inability to achieve real-time and continuous imitation of human movements.

[0004] Patent document CN106078752A discloses a Kinect-based method for mimicking human behavior in a humanoid robot. This method involves: 1) extracting human skeleton information from RGBD images captured by Kinect to construct a human motion model; 2) constructing a humanoid motion model based on the robot's own limb structure; 3) using a mapping model to convert the captured human motion into motion in the robot's motion model; 4) using balance control technology to adjust the robot's joint angles to maintain stability during the imitation process; and 5) performing self-collision avoidance on the robot's state and sending the final joint configuration data to the robot to complete the imitation. This method fails to enable the robot to more accurately imitate complex human movements and improve the naturalness and smoothness of the movements. This problem urgently needs to be addressed. Summary of the Invention

[0005] In view of the defects in the prior art, the purpose of the present invention is to provide a method and system for imitating the movements of a humanoid robot based on a human linear model.

[0006] According to the present invention, a method for imitating humanoid robot motion based on a human linear model comprises:

[0007] Step S1: Collecting video input of human body movements and extracting movement information frame by frame;

[0008] Step S2: Modeling is performed based on the motion information, and a SMPL representation and a three-dimensional representation of the human body posture are obtained;

[0009] Step S3: Based on the SMPL representation and the three-dimensional representation of the human body posture, SMPL sequence data in the time dimension and the three-dimensional representation of the human body motion are generated to construct a dynamic tree model of the robot;

[0010] Step S4: planning the robot motion trajectory through the dynamic tree model and generating corresponding control instructions.

[0011] Preferably, in step S1, the action information is the visual features of the human body shape and the motion features of the changes between frames; the visual features include the human body contour and joint morphology; the motion features include the movement and posture of the human body;

[0012] The action information is extracted based on a deep network.

[0013] Preferably, in the step S2, the modeling is to construct a Mesh model;

[0014] In step S2, based on the motion information, a multi-layer perceptron is used to estimate the human body posture and obtain a Mesh model of the human body; the Mesh model of the human body is mapped to a three-dimensional space through an SMPL model to generate a three-dimensional representation of the human body posture;

[0015] In step S3, the SMPL sequence data is a sequence consisting of three-dimensional posture parameters output by the SMPL model in each frame and the motion trajectory of each joint; the three-dimensional posture parameters include joint positions, rotation matrices, and shape parameters of the human body;

[0016] The process of establishing and optimizing the kinetic tree model includes:

[0017] Step A1: Based on the SMPL sequence data output by the human motion modeling module, construct the robot's dynamic tree model; use the Newton-Lagrange method to obtain the relationship between joint torque and joint motion;

[0018] Step A2: constructing a complete robot dynamics model based on the relationship between the joint torque and joint motion and the robot's dynamics tree model;

[0019] The complete robot dynamics model includes the physical connection relationship between the robot joints, inertia parameters and mass distribution;

[0020] Step A3: Based on the complete robot dynamics model, extract the target joint positions from the SMPL sequence data of each frame, and calculate the angles of each joint according to the target joint positions, thereby generating a motion trajectory as the excitation trajectory; the target joint positions include the positions of the hands and feet;

[0021] Step A4: Based on the motion trajectory, collecting the robot's joint position, velocity, and torque as training data;

[0022] Step A5: performing low-pass filtering on the training data to obtain filtered training data;

[0023] Step A6: training a deep learning network using the filtered training data; the deep learning network outputs a kinetic tree model; optimizing and updating the control strategy based on the parameters of the kinetic tree model, and determining whether the optimization error is less than a preset threshold. If yes, the process ends; if no, the process re-executes step A3.

[0024] The control strategy includes: adjusting the torque control parameters of the joint, optimizing the motion trajectory and redefining the rules of feedback control;

[0025] In the step A1, the dynamic tree model provides the kinetic energy and gravitational potential energy of the robot joints;

[0026] The mathematical expression of the kinetic energy is:

[0027]

[0028] Where T represents the kinetic energy of the robot joint, n represents the total number of robot joints; m i represents the mass of the i-th joint; v i represents the velocity of the i-th joint;

[0029] The mathematical expression of the gravitational potential energy is:

[0030]

[0031] Where V represents the gravitational potential energy of the robot joint; g represents the acceleration due to gravity; h i represents the height of the center of gravity of the i-th joint;

[0032] The relationship between the joint torque and joint motion is expressed mathematically as follows:

[0033]

[0034] Among them, qj represents the generalized coordinates of the j-th joint, represents the generalized velocity of the jth joint, τ j is the joint torque of the jth joint;

[0035] In step A2, the complete robot dynamics model is expressed as follows:

[0036]

[0037] in, are the Coriolis and centrifugal force matrices; M(q) represents the mass matrix; G(q) represents the gravity term; q represents the generalized coordinates of the joint, represents the generalized velocity of the joint; represents the generalized acceleration of the joint.

[0038] Preferably, the step A1 includes:

[0039] Step A1.1: Each frame of SMPL sequence data is used as input data and fed into a deep network composed of multi-layer perceptrons, i.e., a deep neural network.

[0040] Step A1.2: Extracting features of the input data based on the deep network and reducing the dimensionality to obtain first-layer features; mapping the first-layer features into dynamic tree model parameters; the dynamic tree model parameters include: joint mass, inertia matrix, damping coefficient, and stiffness coefficient;

[0041] Step A1.3: Outputting a dynamic tree model of the robot based on the dynamic tree model parameters;

[0042] Step A1.4: Based on the dynamic tree model parameters, calculate and obtain the kinetic energy and potential energy of each joint; solve the kinetic energy and potential energy using the Lagrange equation to obtain the explicit relationship between joint torque and joint position and the explicit relationship between velocity and acceleration, and then obtain the relationship between joint torque and joint motion;

[0043] In step A3, it includes:

[0044] Step A3.1: Extract target joint positions from the SMPL pose data of each frame; the target joint positions include the positions of the hands and feet;

[0045] Step A3.2: Based on the target joint position, obtain the angles of each joint, and obtain a set of joint angle values ​​that meet the target joint position based on inverse kinematics;

[0046] Step A3.3: Optimizing the joint angle values ​​based on the dynamic constraints and kinematic restrictions of each joint; the dynamic constraints include velocity restrictions, acceleration restrictions, and torque restrictions; the kinematic restrictions include joint angle ranges and degrees of freedom; the optimization involves setting numerical boundaries for the joint angle values;

[0047] Step A3.4: Connect the optimized joint angle values ​​into a smooth and continuous trajectory using cubic spline interpolation, i.e., the target motion trajectory;

[0048] The smooth and continuous trajectory refers to a trajectory with a smoothness index less than or equal to 0.1 and continuous;

[0049] The mathematical expression of the smoothness index is:

[0050]

[0051] Among them, S represents the smoothness index, t0 represents the starting time of the trajectory; t f represents the end time of the trajectory; θ represents the joint angle; t represents time;

[0052] In step A6, the process of optimizing the parameters of the kinetic tree model includes:

[0053] Step B1: Derivation of predicted values ​​through the dynamic tree model; the predicted values ​​include: joint position, velocity and torque;

[0054] Step B2: Calculating the loss function value between the predicted value and the actual data using a loss function;

[0055] Step B3: Calculating the gradient of the loss function with respect to the network parameters through a back-propagation algorithm, and updating the network parameters through a gradient descent method, thereby obtaining an optimized dynamic tree model;

[0056] In step B2, the mathematical expression of the loss function is:

[0057]

[0058] Among them, N represents the number of motion trajectory frames, represents the predicted joint position of the i-th frame, q i Indicates the actual joint position; Indicates the prediction speed, v i Indicates actual speed; Denotes the predicted moment, τ i Represents the actual torque. α, β, and γ are a weight coefficient, another weight coefficient, and another weight coefficient respectively; || represents the L2 norm of the vector, that is, the Euclidean distance, Represents the final loss function value;

[0059] In step B3, the mathematical expression of the gradient of the loss function with respect to the network parameters is:

[0060]

[0061] in, Represents the gradient of the loss function with respect to the network parameters, and the superscript T represents the transpose of the matrix or vector; represents the predicted joint position of the i-th frame; q i represents the actual joint position of the i-th frame; Indicates the predicted speed of the i-th frame; v i Indicates the actual speed of the i-th frame.

[0062] The gradient of the joint position is expressed mathematically as:

[0063]

[0064] in, Gradient representing joint position;

[0065] The gradient of velocity is expressed mathematically as:

[0066]

[0067] in, represents the gradient of velocity;

[0068] The gradient of the torque is expressed mathematically as:

[0069]

[0070] in, represents the gradient of the moment, and W represents the network parameter;

[0071] The mathematical expression of the network parameter, that is, the optimization error, is:

[0072]

[0073] Among them, W represents the network parameters, that is, the optimization error, ← represents the assignment operation, represents the partial derivative, Represents the final loss function value, and η represents the learning rate.

[0074] According to the present invention, a humanoid robot motion imitation system based on a human linear model is provided, comprising:

[0075] Module M1: collects video input of human movements and extracts movement information frame by frame;

[0076] Module M2: Modeling based on the motion information and obtaining SMPL representation and three-dimensional representation of human posture;

[0077] Module M3: Based on the SMPL representation and the three-dimensional representation of the human body posture, generate SMPL sequence data in the time dimension and the three-dimensional representation of the human body motion, and construct a dynamic tree model of the robot;

[0078] Module M4: plans the robot motion trajectory through the dynamic tree model and generates corresponding control instructions.

[0079] Preferably, in the module M1, the motion information is the visual features of the human body shape and the motion features of the changes between frames; the visual features include the human body contour and joint morphology; the motion features include the movement and posture of the human body;

[0080] The action information is extracted based on a deep network.

[0081] Preferably, in the module M2, the modeling is to construct a Mesh model;

[0082] In the module M2, based on the motion information, a multi-layer perceptron is used to estimate the human body posture and obtain a mesh model of the human body; the mesh model of the human body is mapped to a three-dimensional space through an SMPL model to generate a three-dimensional representation of the human body posture;

[0083] In the module M3, the SMPL sequence data is a sequence consisting of three-dimensional posture parameters output by the SMPL model in each frame and the motion trajectory of each joint; the three-dimensional posture parameters include joint positions, rotation matrices, and shape parameters of the human body;

[0084] The establishment and optimization of the kinetic tree model includes:

[0085] Module A1: Construct a dynamic tree model of the robot based on the SMPL sequence data output by the human motion modeling module; use the Newton-Lagrange method to obtain the relationship between joint torque and joint motion;

[0086] Module A2: Based on the relationship between joint torque and joint motion and the robot's dynamic tree model, a complete robot dynamic model is constructed;

[0087] The complete robot dynamics model includes the physical connection relationship between the robot joints, inertia parameters and mass distribution;

[0088] Module A3: Based on the complete robot dynamics model, extract the target joint positions from each frame of the SMPL sequence data, and calculate the angles of each joint based on the target joint positions, thereby generating a motion trajectory as the excitation trajectory; the target joint positions include the positions of the hands and feet;

[0089] Module A4: Based on the motion trajectory, collect the robot's joint position, velocity, and torque as training data;

[0090] Module A5: performing low-pass filtering on the training data to obtain filtered training data;

[0091] Module A6: Train a deep learning network using filtered training data; the deep learning network outputs a kinetic tree model; optimize and update the control strategy based on the parameters of the kinetic tree model, and determine whether the optimization error is less than a preset threshold. If yes, the process ends; if no, the process re-triggers Module A3.

[0092] The control strategy includes: adjusting the torque control parameters of the joint, optimizing the motion trajectory and redefining the rules of feedback control;

[0093] In the module A1, the dynamic tree model provides the kinetic energy and gravitational potential energy of the robot joints;

[0094] The mathematical expression of the kinetic energy is:

[0095]

[0096] Where T represents the kinetic energy of the robot joint, n represents the total number of robot joints; m i represents the mass of the i-th joint; v i represents the velocity of the i-th joint;

[0097] The mathematical expression of the gravitational potential energy is:

[0098]

[0099] Where V represents the gravitational potential energy of the robot joint; g represents the acceleration due to gravity; h i represents the height of the center of gravity of the i-th joint;

[0100] The relationship between the joint torque and joint motion is expressed mathematically as follows:

[0101]

[0102] Among them, q j represents the generalized coordinates of the j-th joint, represents the generalized velocity of the jth joint, τ jis the joint torque of the jth joint;

[0103] In the module A2, the complete robot dynamics model is expressed as follows:

[0104]

[0105] in, are the Coriolis and centrifugal force matrices; M(q) represents the mass matrix; G(q) represents the gravity term; q represents the generalized coordinates of the joint, represents the generalized velocity of the joint; represents the generalized acceleration of the joint.

[0106] Preferably, the module A1 includes:

[0107] Module A1.1: Each frame of SMPL sequence data is input into a deep network composed of multi-layer perceptrons, i.e., a deep neural network.

[0108] Module A1.2: Extracting features of the input data based on the deep network and reducing its dimensionality to obtain first-layer features; mapping the first-layer features into dynamic tree model parameters; the dynamic tree model parameters include: joint mass, inertia matrix, damping coefficient, and stiffness coefficient;

[0109] Module A1.3: outputting a dynamic tree model of the robot based on the dynamic tree model parameters;

[0110] Module A1.4: Based on the dynamic tree model parameters, calculate and obtain the kinetic energy and potential energy of each joint; solve the kinetic energy and potential energy using the Lagrange equation to obtain the explicit relationship between joint torque and joint position and the explicit relationship between velocity and acceleration, and then obtain the relationship between joint torque and joint motion;

[0111] Module A3 includes:

[0112] Module A3.1: Extract target joint positions from the SMPL pose data of each frame; the target joint positions include the positions of the hands and feet;

[0113] Module A3.2: Based on the target joint position, obtain the angles of each joint, and obtain a set of joint angle values ​​that meet the target joint position based on inverse kinematics;

[0114] Module A3.3: Optimize the joint angle values ​​based on the dynamic constraints and kinematic restrictions of each joint; the dynamic constraints include velocity restrictions, acceleration restrictions, and torque restrictions; the kinematic restrictions include joint angle ranges and degrees of freedom; the optimization involves setting numerical boundaries for the joint angle values;

[0115] Module A3.4: Connect the optimized joint angle values ​​into a smooth and continuous trajectory using cubic spline interpolation, i.e., the target motion trajectory;

[0116] The smooth and continuous trajectory refers to a trajectory with a smoothness index less than or equal to 0.1 and continuous;

[0117] The mathematical expression of the smoothness index is:

[0118]

[0119] Among them, S represents the smoothness index, t0 represents the starting time of the trajectory; t f represents the end time of the trajectory; θ represents the joint angle; t represents time;

[0120] In the module A6, the process of optimizing the parameters of the kinetic tree model includes:

[0121] Module B1: Derives predicted values ​​through the dynamic tree model; the predicted values ​​include: joint position, velocity and torque;

[0122] Module B2: Calculate the loss function value of the predicted value and the actual data through the loss function;

[0123] Module B3: Calculate the gradient of the loss function with respect to the network parameters through the back-propagation algorithm, and update the network parameters through the gradient descent method to obtain the optimized dynamic tree model;

[0124] In the module B2, the mathematical expression of the loss function is:

[0125]

[0126] Among them, N represents the number of motion trajectory frames, represents the predicted joint position of the i-th frame, q i Indicates the actual joint position; Indicates the prediction speed, v i Indicates actual speed; Denotes the predicted moment, τ i Represents the actual torque. α, β, and γ are a weight coefficient, another weight coefficient, and another weight coefficient respectively; || represents the L2 norm of the vector, that is, the Euclidean distance, Represents the final loss function value;

[0127] In the module B3, the mathematical expression of the gradient of the loss function to the network parameters is:

[0128]

[0129] in, Represents the gradient of the loss function with respect to the network parameters, and the superscript T represents the transpose of the matrix or vector; represents the predicted joint position of the i-th frame; q i represents the actual joint position of the i-th frame; Indicates the predicted speed of the i-th frame; v i Indicates the actual speed of the i-th frame.

[0130] The gradient of the joint position is expressed mathematically as:

[0131]

[0132] in, Gradient representing joint position;

[0133] The gradient of velocity is expressed mathematically as:

[0134]

[0135] in, represents the gradient of velocity;

[0136] The gradient of the torque is expressed mathematically as:

[0137]

[0138] in, represents the gradient of the moment, and W represents the network parameter;

[0139] The mathematical expression of the network parameter, that is, the optimization error, is:

[0140]

[0141] Among them, W represents the network parameters, that is, the optimization error, ← represents the assignment operation, represents the partial derivative, Represents the final loss function value, and η represents the learning rate.

[0142] According to a computer-readable storage medium storing a computer program provided by the present invention, when the computer program is executed by a processor, the steps of the humanoid robot motion imitation method based on the human linear model are implemented.

[0143] According to an electronic device provided by the present invention, it includes a memory, a processor and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the steps of the humanoid robot motion imitation method based on the human linear model are implemented.

[0144] Compared with the prior art, the present invention has the following beneficial effects:

[0145] 1. This paper accurately captures human motion by mesh modeling and extracting SMPL models. Based on the SMPL sequences of motion, a dynamic tree model adapted to humanoid robot motion is constructed, enabling the robot to mimic human motion in real time, improving both accuracy and real-time performance.

[0146] 2. The present invention can accurately capture human motion data by combining RGB cameras with computer vision algorithms.

[0147] 3. The present invention uses the SMPL model to achieve high-quality three-dimensional human posture representation, ensuring the accuracy and efficiency of input data.

[0148] 4. The key innovation of this invention lies in the integration of motion optimization units and deep learning methods, which effectively solves the accuracy problem in robot motion control. Especially when faced with the complexity of robot structure and inconsistency of dynamic parameters, the optimized robot dynamics model significantly improves the accuracy of motion control.

[0149] 5. This invention optimizes robot dynamic parameters by incorporating deep learning network training. This allows for automatic adaptation and precise adjustment based on the specific characteristics of humanoid robots, further enhancing the precision and stability of motion control. During the optimization process, the natural and fluid imitation of robot movements was significantly improved.

[0150] 6. In terms of the control module, the present invention successfully overcomes external interference and internal structural errors through real-time motion planning and error compensation mechanism, significantly improving the robot's execution ability in complex environments. Experiments show that the robot's accuracy and execution efficiency in performing imitation actions have been significantly improved.

[0151] 7. The method proposed in the present invention is highly scalable and versatile and is applicable to various types of humanoid robots. It not only reduces the difficulty of robot programming, but also improves the flexibility and adaptability of robot motion imitation. It can achieve efficient and accurate motion imitation in a variety of scenarios and has broad application prospects and market potential. BRIEF DESCRIPTION OF THE DRAWINGS

[0152] Other features, objects and advantages of the present invention will become more apparent upon reading the detailed description of non-limiting embodiments with reference to the following drawings:

[0153] Figure 1 A schematic diagram of the workflow provided by the present invention;

[0154] Figure 2This is a schematic diagram of the humanoid robot's motion imitation effect of the human linear model provided by the present invention. DETAILED DESCRIPTION

[0155] The present invention will be described in detail below with reference to specific embodiments. The following examples will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those skilled in the art, several changes and improvements can be made without departing from the scope of the present invention. These all fall within the scope of protection of the present invention.

[0156] The present invention proposes an imitation learning scheme for a humanoid robot. Specifically, the present invention combines the multi-module collaboration of human motion acquisition, motion modeling, dynamics optimization and control strategy to propose an efficient and accurate robot motion imitation method.

[0157] The present invention solves the problems in the prior art that humanoid robots cannot capture dynamic changes when imitating human movements and lack dynamic modeling of the entire humanoid body by adopting a comprehensive solution of a human motion acquisition module, a human motion modeling module, a robot power tree modeling module and a robot motion control module.

[0158] This invention uses an RGB camera to collect human motion data and combines it with a deep learning algorithm to extract and optimize the SMPL model, enabling the robot to more accurately imitate complex human movements. Furthermore, by constructing a dynamic tree model tailored to the humanoid robot and integrating it with a deep learning network to optimize the robot's motion control precision, the robot is able to continuously and accurately imitate human movements. Ultimately, this invention achieves the effect of enabling the robot to accurately imitate human dynamic movements, effectively improving the smoothness and naturalness of the imitated movements.

[0159] According to the present invention, a method for imitating humanoid robot motion based on a human linear model, i.e., a method for directly imitating human motion by a humanoid robot, the software system process includes:

[0160] Step S1: collecting human motion data, and obtaining motion information based on the human motion data;

[0161] Step S2: constructing a mesh model based on the motion information, extracting and constructing a three-dimensional representation of the human body motion through the SMPL model;

[0162] Step S3: outputting SMPL sequence data based on the three-dimensional representation of the human body motion;

[0163] Step S4: constructing a dynamic tree model of the robot based on the SMPL sequence data;

[0164] Step S5: Plan the robot's motion trajectory and generate control instructions through the dynamic tree model.

[0165] In other words, a humanoid robot motion imitation method based on a human linear model provided by the present invention includes:

[0166] Step S1: Collect video input of human movements and extract appearance and motion features frame by frame through a deep network;

[0167] Step S2: Based on the appearance and motion features extracted frame by frame, human body mesh modeling is performed, and the SMPL representation and the three-dimensional representation of the human body posture are obtained using the SMPL model;

[0168] Step S3: Based on the frame-by-frame constructed SMPL representation and the three-dimensional representation of the human body posture, generate SMPL sequence data in the time dimension and a three-dimensional representation of the human body motion;

[0169] Step S4: constructing a dynamic tree model of the robot based on the SMPL sequence and the three-dimensional representation of the human body motion;

[0170] Step S5: Plan the robot's motion trajectory through the dynamic tree model and generate corresponding control instructions.

[0171] In step S1, the motion information refers to the visual features of the human body shape and the motion features of the changes between frames;

[0172] The visual features include: human appearance, outline and joint morphology;

[0173] The motion characteristics include: movement and posture of the human body;

[0174] Specifically, the action information is extracted based on a deep network.

[0175] Specifically, the motion information is extracted based on a deep network, which means that by inputting RGB video of human movements and using a deep learning algorithm of computer vision, the motion features therein are analyzed and extracted. This feature is represented in the form of an intermediate vector and is ultimately mapped to the parameters of the SMPL model.

[0176] Motion optimization refers to optimizing the robot's dynamics. It uses SMPL model parameters as input and the robot's dynamics tree parameters as output. A mapping relationship between the two is established through a deep neural network. With the help of deep learning optimization algorithms, this mapping is continuously optimized to generate dynamics tree parameters that better fit human motion, ultimately building a more accurate robot dynamics model that better conforms to human motion characteristics.

[0177] In the step S2, human Mesh modeling is performed, and a SMPL representation and a three-dimensional representation of a human pose are obtained by using a SMPL model, which is implemented in the following manner:

[0178] In the step S2, the human Mesh is subjected to pose estimation in combination with visual features and motion features, which is implemented in the following manner:

[0179] Firstly, based on the visual features, i.e. motion information, extracted in the step S1, a multi-layer perception, i.e. MLP, is used to estimate the Mesh appearance form of the human body;

[0180] Then, the motion features are combined to further estimate the pose of the human Mesh;

[0181] Finally, the estimated human Mesh is mapped to a three-dimensional space by using the SMPL model to generate a three-dimensional representation of the human pose;

[0182] In the step S3, the SMPL sequence data is a sequence composed of three-dimensional pose parameters output by the SMPL model of each frame in the time dimension, including joint positions, rotation matrices, shape parameters of the human body, and motion trajectories of each joint.

[0183] Specifically, the three-dimensional pose parameters of each frame obtained in the step S2 are processed to extract the joint positions and rotation matrices of each frame, and the shape parameters of the human body are combined to construct a three-dimensional representation of the human pose of each frame. Then, based on the joint position information, the motion trajectories of each joint are calculated and time-sequenced, and finally all the three-dimensional poses and motion trajectories of the frames are connected in time sequence to form a complete SMPL sequence data representing the evolution of human motion.

[0184] In the step S4, the dynamics tree model is established in the following manner:

[0185] Based on the SMPL sequence data, Newton-Lagrange method and deep network are used to establish the relationship between joint torque and motion to obtain a preliminary dynamics tree model.

[0186] Then, the action optimization unit is used to iteratively optimize the dynamics tree. In the action optimization unit, joint position, velocity and other signals are used to generate excitation trajectory data to train the deep network. At the same time, a low-pass filter is used to filter out high-frequency noise and burr signals to reduce interference.

[0187] The signal data processed by the filter is transmitted to the optimization unit for further optimization. The iteration process will end when the optimization error of the dynamics tree model is lower than the preset threshold, indicating that the accuracy of the model has reached the requirement.

[0188] In the step S5, the motion trajectory is planned by the dynamic tree model, specifically: according to the relationship between joint torque and motion in the dynamic tree model, combining the dynamics constraints and kinematics restrictions of each joint, using trajectory planning algorithm to calculate the optimal motion trajectory of the robot from the initial position to the target position.

[0189] Specifically, this trajectory planning algorithm is the motion trajectory generation method mentioned in step A4, and the specific steps of the trajectory algorithm are as follows:

[0190] First, based on the given end effector, such as the target position of the hand and foot, combined with the kinematics model and joint degrees of freedom of the humanoid robot, the inverse kinematics algorithm is used to calculate the angle of each joint; the inverse kinematics equation is solved by Newton iteration method or analytical method to generate a set of joint angle values that satisfy the end effector target position;

[0191] Then, according to the relationship between joint torque and motion in the dynamic tree model, combining the dynamics constraints of each joint, such as speed, acceleration and torque limit, and kinematics restrictions, such as joint angle range and degrees of freedom, the joint angle is optimized and adjusted to ensure that the generated motion trajectory conforms to the dynamics characteristics of human motion;

[0192] Next, the cubic spline interpolation method is used to connect the optimized joint angle values into a smooth and continuous trajectory;

[0193] Finally, through feedback control combined with PID controller to adjust the joint motion in real time, to ensure that the robot can accurately track the generated trajectory.

[0194] Specifically, the control instructions are generated by the motion trajectory: the motion trajectory generated by the trajectory planning module will be discretized and transformed by the control module to form a series of control instructions, which define the motion trajectory and speed of each joint or effector of the robot. The control instructions are then transmitted to the robot through the drive system, and the execution unit monitors the posture through real-time sensor feedback, and combines the error compensation unit to correct the deviation in the execution process.

[0195] The optimization step of the dynamic tree model, the software flow includes:

[0196] Step A1: According to the SMPL sequence data output by the human motion modeling module, combined with the deep network, a preliminary dynamic tree model suitable for the motion of humanoid robot is constructed. Newton-Lagrange method is used to establish the relationship between joint torque and joint motion, to ensure that the robot motion is highly consistent with human motion.

[0197] In the step A1, it includes:

[0198] Step A1.1: input each frame of SMPL sequence data as input data into a deep network composed of multiple layers of perception mechanism, i.e. a deep neural network;

[0199] Step A1.2: extract features of the input data and reduce dimensionality based on the deep network to obtain first layer features; map the first layer features to dynamic tree model parameters; the dynamic tree model parameters include joint mass, inertia matrix, damping coefficient and stiffness coefficient;

[0200] Step A1.3: output a dynamic tree model of the robot based on the dynamic tree model parameters;

[0201] Step A1.4: calculate and obtain kinetic energy and potential energy of each joint based on the dynamic tree model parameters; obtain explicit relationships of joint torque and joint position and explicit relationships of velocity and acceleration through Lagrange equation, and then obtain the relationship between joint torque and joint motion;

[0202] Specifically, in the step A1, according to the SMPL sequence data output by the human motion modeling module, the process of constructing the motion dynamic tree model of the humanoid robot;

[0203] First, input each frame of SMPL sequence data into a deep network composed of two layers of multiple perception mechanisms, the first layer network extracts features of the input data through a nonlinear activation function and reduces dimensionality, and the second layer network maps the extracted features to parameters required by the dynamic tree model, including joint mass, inertia matrix, damping coefficient and stiffness coefficient, etc., which completely describes the dynamic characteristics of each joint of the humanoid robot. Through this process, the deep network directly outputs the initial dynamic tree model of the humanoid robot motion;

[0204] After obtaining the initial dynamic tree model, the relationship between the joint torque of the humanoid robot and the joint motion is established by using the Newton-Lagrange method. Specifically, based on the parameters of the dynamic tree model, first, the kinetic energy and potential energy of each joint are calculated, the kinetic energy is determined by the joint mass and velocity, and the potential energy is determined by the joint position and external force field such as gravity, then the kinetic energy and potential energy are substituted into the Lagrange equation, and through the solution of the Lagrange equation, the explicit relationship between the joint torque and the joint position, velocity and acceleration is obtained. This relationship can be described as the dynamic equation of the humanoid robot, so that the required joint torque can be directly calculated according to the motion state of the robot.

[0205] The entire method establishes a precise mapping between human motion and humanoid robot motion. The human motion features extracted from SMPL sequence data are mapped into parameters of the robot dynamics tree model through a deep network, and the dynamic equations based on the Newton-Lagrange method further map the human joint motion into the robot joint torque, thereby ensuring that the motion of the humanoid robot can accurately imitate human motion and meet the physical constraints and control requirements of the robot, ultimately achieving efficient conversion and precise control from human motion to humanoid robot motion.

[0206] Specifically, the Lagrangian function is first defined: L = TV, where the kinetic energy T and potential energy V accumulate the kinetic energy and gravitational potential energy of all joints.

[0207] T and V are expressed as:

[0208]

[0209]

[0210] Among them, m i is the mass of the i-th joint, v i is its linear velocity, g is the acceleration due to gravity, h i is the center of gravity height.

[0211] Then, through the Lagrange equation Derive the dynamic equations to describe the relationship between the joints.

[0212] In other words, in step A1, the dynamic tree model provides the kinetic energy and gravitational potential energy of the robot joints;

[0213] The mathematical expression of the kinetic energy is:

[0214]

[0215] Where T represents the kinetic energy of the robot joint, n represents the total number of robot joints; m i represents the mass of the i-th joint; v i represents the velocity of the i-th joint;

[0216] The mathematical expression of the gravitational potential energy is:

[0217]

[0218] Where V represents the gravitational potential energy of the robot joint; g represents the acceleration due to gravity; h i represents the height of the center of gravity of the i-th joint;

[0219] The relationship between the joint torque and joint motion is expressed mathematically as follows:

[0220]

[0221] Among them, q j represents the generalized coordinates of the j-th joint, represents the generalized velocity of the jth joint, τ j is the joint torque of the jth joint;

[0222] In step A2, the complete robot dynamics model is expressed as follows:

[0223]

[0224] in, are the Coriolis and centrifugal force matrices; M(q) represents the mass matrix; G(q) represents the gravity term; q represents the generalized coordinates of the joint, represents the generalized velocity of the joint; represents the generalized acceleration of the joint.

[0225] Step A2: Based on the SMPL sequence data, a complete robot dynamics model is constructed, including the physical connection relationship between joints, inertia parameters, mass distribution, etc.

[0226] Inertial parameters such as mass, center of mass position and moment of inertia are combined in the dynamic equations through the mass matrix M(q) and the gravity term G(q), which ultimately form

[0227] in, are the Coriolis and centrifugal force matrices.

[0228] This model comprehensively reflects the motion and torque relationship of each joint of the robot, and combines the physical connection and mass distribution of each joint.

[0229] Step A3: Generate an excitation trajectory based on the SMPL sequence data. This is done through an inverse kinematics algorithm, also known as inverse kinematics, which extracts the target joint positions from each frame of the SMPL pose data. Then, based on the robot's degrees of freedom and physical constraints, the motion trajectory of each joint is solved using inverse kinematics. Inverse kinematics learns to calculate the angles of each joint based on the position of a given end effector, such as a hand or foot, thereby generating a motion trajectory that matches human motion, ensuring that the robot accurately mimics the human motion in the SMPL model.

[0230] In step A3, it includes:

[0231] Step A3.1: Extract target joint positions from the SMPL pose data of each frame; the target joint positions include the positions of the hands and feet;

[0232] Step A3.2: Based on the target joint position, obtain the angles of each joint, and obtain a set of joint angle values ​​that meet the target joint position based on inverse kinematics;

[0233] Step A3.3: Optimizing the joint angle values ​​based on the dynamic constraints and kinematic restrictions of each joint; the dynamic constraints include velocity restrictions, acceleration restrictions, and torque restrictions; the kinematic restrictions include joint angle ranges and degrees of freedom; the optimization sets numerical boundaries for the joint angle values;

[0234] Step A3.4: Connect the optimized joint angle values ​​into a smooth and continuous trajectory, i.e., the target motion trajectory, by using the cubic spline interpolation method; in other words, connect the optimized joint angle values ​​into a smoothness index that satisfies And the continuous trajectory is the target motion trajectory;

[0235] Step A4: Collect training data such as joint position, velocity, and torque;

[0236] Step A5: low-pass filter the collected data to remove high-frequency noise and burr signals to reduce interference;

[0237] Step A6: Use the processed data to train the deep learning network, optimize the parameters of the kinetic tree model, and update the control strategy.

[0238] In step A6, the preprocessed joint position, velocity, and torque data from step A5 are used as true labels to optimize the deep network. The optimization process first calculates the output of the deep network, i.e., the dynamic tree model, through forward propagation, and derives the corresponding predicted values ​​of the joint position, velocity, and torque based on this model. The predicted values ​​are then compared with the true label data, and a loss function is calculated to measure the difference between the predicted values ​​and the true values. The gradient of the loss function with respect to the network parameters is then calculated using a backpropagation algorithm, and the network parameters are updated using a gradient descent method to reduce the loss function value and improve the model accuracy. Furthermore, based on the optimized dynamic tree model, the robot's joint control strategy is updated, including adjusting the torque control parameters of the joints, optimizing the motion trajectory, and redefining the rules of feedback control.

[0239] In step A6, the process of optimizing the parameters of the kinetic tree model includes:

[0240] Step B1: Derivation of predicted values ​​through the dynamic tree model; the predicted values ​​include: joint position, velocity and torque;

[0241] Step B2: Calculating the loss function value between the predicted value and the actual data using a loss function;

[0242] Step B3: Calculate the gradient of the loss function with respect to the network parameters through the back propagation algorithm, and update the network parameters through the gradient descent method to obtain the optimized dynamic tree model;

[0243] In step B2, the mathematical expression of the loss function is:

[0244]

[0245] Among them, N represents the number of motion trajectory frames, represents the predicted joint position of the i-th frame, q i Indicates the actual joint position; Indicates the prediction speed, v i Indicates actual speed; Denotes the predicted moment, τ i represents the actual torque. α, β, and γ are weight coefficients, respectively, used to balance the weights of joint position, velocity, and torque in the loss function. |·| represents the L2 norm of the vector, i.e., the Euclidean distance, and L is the final loss function value. α, β, and γ can be adjusted based on specific task requirements.

[0246] In step B3, the mathematical expression of the gradient of the loss function to the network parameters is:

[0247]

[0248] in, Represents the gradient of the loss function with respect to the network parameters, and the superscript T represents the transpose of the matrix or vector; represents the predicted joint position of the i-th frame; q i represents the actual joint position of the i-th frame; Indicates the predicted speed of the i-th frame; v i Indicates the actual speed of the i-th frame.

[0249] The gradient of the joint position is expressed mathematically as:

[0250]

[0251] in, Gradient representing joint position;

[0252] The gradient of velocity is expressed mathematically as:

[0253]

[0254] in, represents the gradient of velocity;

[0255] Gradient of the torque, mathematically expressed as:

[0256]

[0257] wherein, represents the gradient of the torque, and W represents the network parameters.

[0258] Specifically, the main role of solving the position gradient, velocity gradient and torque gradient is to calculate the gradient of the loss function with respect to the network parameters through the chain rule, so as to update the network parameters and optimize the model performance.

[0259] In the optimization process of deep learning, the chain rule is the core method of calculating the gradient, which needs to pass the error of the loss function back to the network parameters step by step. Position, velocity and torque are the key outputs of model prediction, which are directly related to the loss function, therefore, their gradients need to be calculated first. The gradient acts as a "bridge" to pass the error of the loss function to the network parameters, helping the operator to understand the contribution of each parameter to the error.

[0260] Finally, through the calculation of these gradients, the network parameters can be updated using the gradient descent method to optimize the performance of the model. Therefore, the fundamental purpose of solving these gradients is to obtain the network parameters and optimize the model.

[0261] The network parameters, i.e. the mathematical expression of the optimization error, are:

[0262]

[0263] wherein, W represents the network parameters, i.e. the optimization error; <- represents the assignment operation, i.e. updating the variable on the left side with the value on the right side; represents the partial derivative, which is used to calculate the gradient; represents the final loss function value; η is the learning rate, which is used to control the step size of parameter update.

[0264] Step A7: Loop iteration until the optimization error of the dynamics tree model is lower than the preset threshold, indicating that the accuracy of the model has reached the requirement, and the optimization process ends.

[0265] The human action acquisition module acquires human action data through an RGB camera and uses computer vision algorithms such as human pose estimation to extract two-dimensional or three-dimensional joint data of the human body from video frames. The video data captured by the RGB camera will be transmitted to the computer vision algorithm, and the pose and action information of the human body will be obtained through algorithm processing.

[0266] To ensure data quality, the collected data is further processed by the pre-processing unit, including noise removal, smoothing, and normalization, to ensure that the data is suitable for subsequent modeling. The pre-processed data is then passed to the human motion modeling module for subsequent operations.

[0267] The Human Motion Modeling module meshes the collected human motion data and extracts the Skinned Multi-Person Linear Model (SMPL) to construct a three-dimensional representation of human motion. This module uses a deep learning algorithm to extract the SMPL model from the preprocessed data using the Data Modeling Unit, accurately representing the human body's postures and movements. The extracted SMPL model data is then passed to the Motion Optimization Unit for further processing. The Motion Optimization Unit optimizes the SMPL model, adjusting joint angles and smoothing transitions, making the robot's imitation movements more natural and fluid. This optimized data is then passed to the Robot Power Tree Modeling Module to ensure accurate and smooth motion control.

[0268] The Robot Dynamic Tree Modeling Module constructs a dynamic tree model suitable for humanoid robot motion based on the SMPL sequence data output by the Human Motion Modeling Module, thereby optimizing the robot's motion control algorithm. This module's dynamic modeling unit constructs the humanoid robot's dynamic model based on the SMPL model sequence and employs the Newton-Lagrange method to establish the relationship between joint torque and joint motion, ensuring that the robot's movements are highly consistent with those of the human body.

[0269] However, due to the complexity of humanoid robot design, there are deviations in elastic components, mass and inertia parameters in the mechanical structure, and inevitable vibration, friction, coupling effects and noise may lead to inaccurate dynamic calculations of theoretical models.

[0270] To address this issue, the system uses deep learning to optimize the dynamic parameters of humanoid robots. The optimization unit uses deep learning to train a network, adapting it to different humanoid robots and accurately obtaining their dynamic parameters, thereby improving motion control accuracy.

[0271] The training data includes information such as joint position, velocity, and current. The deep learning network is trained based on this data, ultimately outputting optimized dynamic parameters. These optimized parameters are then passed to the robot's motion control module to update the control strategy. During the training process, the system utilizes an excitation trajectory design unit to generate sufficient data for the deep learning network training. This unit designs appropriate excitation trajectories based on system requirements to ensure the generation of recognizable dynamic parameters. The data generated by these excitation trajectories enables better training and optimization of the deep learning network. The collected robot joint torque data is then passed to a low-pass filter for signal processing, filtering out high-frequency noise and glitches to reduce interference. After filtering, the signal data is passed to the optimization unit for further model optimization and training.

[0272] The control module is responsible for generating control instructions for the robot based on the robot's dynamic model and hardware configuration. These instructions are then executed by the execution unit, thereby completing motion simulation and ensuring synchronization between the robot's movements and those of the target human. The control module consists of multiple functional units.

[0273] First, the real-time motion planning unit plans a reasonable motion trajectory based on the dynamic model, real-time sensor data, and task requirements. Motion planning takes into account the reachability, stability, and dynamic constraints of the robot's joints to ensure the accuracy and efficiency of the robot's movements.

[0274] Next is the execution unit, which generates control instructions based on the planned trajectory and executes these instructions through the robot's drive system, ensuring the robot accurately executes the movement. To further improve the robot's motion accuracy, the control module also includes an error compensation unit. This unit monitors and adjusts any deviations during execution in real time, ensuring the robot can still perform high-precision movements in complex environments. The error compensation unit uses sensor feedback to adjust the control strategy, effectively overcoming the effects of external interference or internal structural errors.

[0275] In addition, the control module also integrates an adaptive learning mechanism, which enables the robot to continuously optimize the control strategy during long-term training and adapt to different motion tasks.

[0276] The key innovation of this method lies in the integration of motion optimization units and deep learning methods, which effectively solves the accuracy problem in robot motion control. Especially in the face of robot structural complexity and inconsistent dynamic parameters, the optimized robot dynamics model significantly improves the accuracy of motion control, and the error is reduced to within 5%.

[0277] The present invention also provides a humanoid robot motion imitation system based on a human body linear model. The humanoid robot motion imitation system based on a human body linear model can be realized by executing the process steps of the humanoid robot motion imitation method based on a human body linear model, that is, those skilled in the art can understand the humanoid robot motion imitation method based on a human body linear model as a preferred embodiment of the humanoid robot motion imitation system based on a human body linear model.

[0278] According to the present invention, a humanoid robot motion imitation system based on a human linear model is provided, comprising:

[0279] Module M1: collects video input of human movements and extracts movement information frame by frame;

[0280] Module M2: Modeling based on the motion information and obtaining SMPL representation and three-dimensional representation of human posture;

[0281] Module M3: Based on the SMPL representation and the three-dimensional representation of the human body posture, generate SMPL sequence data in the time dimension and the three-dimensional representation of the human body motion, and construct a dynamic tree model of the robot;

[0282] Module M4: plans the robot's motion trajectory through the dynamic tree model and generates corresponding control instructions.

[0283] The modules M1, M2, M3 and M4 correspond to the human motion acquisition module, human motion modeling module, robot power tree modeling module and robot motion control module in the embodiment respectively;

[0284] The robot is a humanoid robot.

[0285] Those skilled in the art will appreciate that, in addition to implementing the system and its various devices, modules, and units provided by the present invention in purely computer-readable program code, it is entirely possible to implement the same functions of the system and its various devices, modules, and units provided by the present invention in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered as structures within the hardware component; the devices, modules, and units for implementing various functions can also be considered as both software modules implementing the method and structures within the hardware component.

[0286] In the description of this application, it should be understood that the terms "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on this application.

[0287] The above describes specific embodiments of the present invention. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art may make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. The embodiments of this application and the features in the embodiments may be combined with each other in any manner unless there is a conflict.

Claims

1. A method for imitating humanoid robot motion based on a human linear model, characterized in that: include: Step S1: Collecting video input of human body movements and extracting movement information frame by frame; Step S2: Modeling is performed based on the motion information, and a SMPL representation and a three-dimensional representation of the human body posture are obtained; Step S3: Based on the SMPL representation and the three-dimensional representation of the human body posture, SMPL sequence data in the time dimension and the three-dimensional representation of the human body motion are generated to construct a dynamic tree model of the robot; Step S4: planning the robot motion trajectory through the dynamic tree model and generating corresponding control instructions; In the step S2, the modeling is to construct a Mesh model; In step S2, based on the motion information, a multi-layer perceptron is used to estimate the human body posture and obtain a Mesh model of the human body; the Mesh model of the human body is mapped to a three-dimensional space through an SMPL model to generate a three-dimensional representation of the human body posture; In step S3, the SMPL sequence data is a sequence consisting of three-dimensional posture parameters output by the SMPL model in each frame and the motion trajectory of each joint; The three-dimensional posture parameters include joint positions, rotation matrices, and shape parameters of the human body; The process of establishing and optimizing the kinetic tree model includes: Step A1: Based on the SMPL sequence data output by the human motion modeling module, construct the robot's dynamic tree model; use the Newton-Lagrange method to obtain the relationship between joint torque and joint motion; Step A2: constructing a complete robot dynamics model based on the relationship between the joint torque and joint motion and the robot's dynamics tree model; The complete robot dynamics model includes the physical connection relationship between the robot joints, inertia parameters and mass distribution; Step A3: Based on the complete robot dynamics model, extract the target joint positions from the SMPL sequence data of each frame, and calculate the angles of each joint according to the target joint positions, thereby generating a motion trajectory as the excitation trajectory; the target joint positions include the positions of the hands and feet; Step A4: Based on the motion trajectory, collecting the robot's joint position, velocity, and torque as training data; Step A5: performing low-pass filtering on the training data to obtain filtered training data; Step A6: training a deep learning network using the filtered training data; the deep learning network outputs a kinetic tree model; optimizing and updating the control strategy based on the parameters of the kinetic tree model, and determining whether the optimization error is less than a preset threshold. If yes, the process ends; if no, the process re-executes step A3. The control strategy includes: adjusting the torque control parameters of the joint, optimizing the motion trajectory and redefining the rules of feedback control; In the step A1, it includes: Step A1.1: Each frame of SMPL sequence data is used as input data and fed into a deep network composed of multi-layer perceptrons, i.e., a deep neural network. Step A1.2: Extracting features of the input data based on the deep network and reducing the dimensionality to obtain first-layer features; mapping the first-layer features into dynamic tree model parameters; the dynamic tree model parameters include: joint mass, inertia matrix, damping coefficient, and stiffness coefficient; Step A1.3: Outputting a dynamic tree model of the robot based on the dynamic tree model parameters; Step A1.4: Based on the dynamic tree model parameters, calculate and obtain the kinetic energy and potential energy of each joint; solve the kinetic energy and potential energy using the Lagrange equation to obtain the explicit relationship between joint torque and joint position and the explicit relationship between velocity and acceleration, and then obtain the relationship between joint torque and joint motion; In step A6, the process of optimizing the parameters of the kinetic tree model includes: Step B1: Derivation of predicted values ​​through the dynamic tree model; the predicted values ​​include: joint position, velocity and torque; Step B2: Calculating the loss function value between the predicted value and the actual data using a loss function; Step B3: The gradient of the loss function with respect to the network parameters is calculated by the back propagation algorithm, and the network parameters are updated by the gradient descent method to obtain the optimized dynamic tree model.

2. The humanoid robot motion imitation method based on the human body linear model according to claim 1, characterized in that: In step S1, the action information is the visual features of the human body shape and the motion features of the changes between frames; the visual features include the human body contour and joint shape; the motion features include the movement and posture of the human body; The action information is extracted based on a deep neural network.

3. The humanoid robot motion imitation method based on the human body linear model according to claim 2, characterized in that: In step A1, the dynamic tree model provides the kinetic energy and gravitational potential energy of the robot joints; The mathematical expression of the kinetic energy is: in, represents the kinetic energy of the robot joints, represents the total number of joints of the robot; Indicates the The quality of each joint; i Indicates the The speed of each joint; The mathematical expression of the gravitational potential energy is: in, represents the gravitational potential energy of the robot joints; represents the acceleration due to gravity; Indicates the The height of the center of gravity of the joint; The relationship between the joint torque and joint motion is expressed mathematically as follows: in, Representative The generalized coordinates of the joints, Representative The generalized velocity of each joint, It is Joint torque of each joint; In step A2, the complete robot dynamics model is expressed as follows: in, is the Coriolis and centrifugal force matrix; represents the mass matrix; represents the gravity term; represents the generalized coordinates of the joint, represents the generalized velocity of the joint; represents the generalized acceleration of the joint.

4. The humanoid robot motion imitation method based on the human body linear model according to claim 3, characterized in that: In step A3, it includes: Step A3.1: Extract target joint positions from the SMPL pose data of each frame; the target joint positions include the positions of the hands and feet; Step A3.2: Based on the target joint position, obtain the angles of each joint, and obtain a set of joint angle values ​​that meet the target joint position based on inverse kinematics; Step A3.3: Optimizing the joint angle values ​​based on the dynamic constraints and kinematic restrictions of each joint; the dynamic constraints include velocity restrictions, acceleration restrictions, and torque restrictions; the kinematic restrictions include joint angle ranges and degrees of freedom; the optimization involves setting numerical boundaries for the joint angle values; Step A3.4: Connect the optimized joint angle values ​​into a smooth and continuous trajectory using cubic spline interpolation, i.e., the target motion trajectory; The smooth and continuous trajectory refers to a trajectory with a smoothness index less than or equal to 0.1 and continuous; The mathematical expression of the smoothness index is: in, represents the smoothness index, Indicates the starting time of the trajectory; Indicates the end time of the trajectory; represents the joint angle; Indicates time; In step B2, the mathematical expression of the loss function is: in, Indicates the number of motion trajectory frames, Indicates the The predicted joint positions for a frame, Indicates the actual joint position; represents the prediction speed, Indicates actual speed; represents the predicted moment, Indicates the actual torque; 、 and They are respectively a weight coefficient, another weight coefficient and another weight coefficient; represents the L2 norm of the vector, that is, the Euclidean distance, Represents the final loss function value; In step B3, the mathematical expression of the gradient of the loss function with respect to the network parameters is: in, Represents the gradient of the loss function with respect to the network parameters, and the superscript Represents the transpose of a matrix or vector; Indicates the Predicted joint positions for a frame; Indicates the Actual joint positions of the frame; Indicates the The predicted speed of the frame; Indicates the The actual speed of the frame; The gradient of the joint position is expressed mathematically as: in, Gradient representing joint position; The gradient of velocity is expressed mathematically as: in, represents the gradient of velocity; The gradient of the torque is expressed mathematically as: in, represents the gradient of the torque, Represents network parameters; The mathematical expression of the network parameter, that is, the optimization error, is: in, represents the network parameters, i.e., the optimization error, Represents an assignment operation, represents the partial derivative, Represents the final loss function value, Represents the learning rate.

5. A humanoid robot motion imitation system based on a human linear model, characterized in that: include: Module M1: collects video input of human movements and extracts movement information frame by frame; Module M2: Modeling based on the motion information and obtaining SMPL representation and three-dimensional representation of human posture; Module M3: Based on the SMPL representation and the three-dimensional representation of the human body posture, generate SMPL sequence data in the time dimension and the three-dimensional representation of the human body motion, and construct a dynamic tree model of the robot; Module M4: plans the robot motion trajectory through the dynamic tree model and generates corresponding control instructions; In the module M2, the modeling is to construct a Mesh model; In the module M2, based on the motion information, a multi-layer perceptron is used to estimate the human body posture and obtain a mesh model of the human body; the mesh model of the human body is mapped to a three-dimensional space through an SMPL model to generate a three-dimensional representation of the human body posture; In the module M3, the SMPL sequence data is a sequence consisting of three-dimensional posture parameters output by the SMPL model in each frame and the motion trajectory of each joint; The three-dimensional posture parameters include joint positions, rotation matrices, and shape parameters of the human body; The establishment and optimization of the kinetic tree model includes: Module A1: Construct a dynamic tree model of the robot based on the SMPL sequence data output by the human motion modeling module; use the Newton-Lagrange method to obtain the relationship between joint torque and joint motion; Module A2: Based on the relationship between joint torque and joint motion and the robot's dynamic tree model, a complete robot dynamic model is constructed; The complete robot dynamics model includes the physical connection relationship between the robot joints, inertia parameters and mass distribution; Module A3: Based on the complete robot dynamics model, extract the target joint positions from each frame of the SMPL sequence data, and calculate the angles of each joint based on the target joint positions, thereby generating a motion trajectory as the excitation trajectory; the target joint positions include the positions of the hands and feet; Module A4: Based on the motion trajectory, collect the robot's joint position, velocity, and torque as training data; Module A5: performing low-pass filtering on the training data to obtain filtered training data; Module A6: Train a deep learning network using filtered training data; the deep learning network outputs a kinetic tree model; optimize and update the control strategy based on the parameters of the kinetic tree model, and determine whether the optimization error is less than a preset threshold. If yes, the process ends; if no, the process re-triggers Module A3. The control strategy includes: adjusting the torque control parameters of the joint, optimizing the motion trajectory and redefining the rules of feedback control; The module A1 includes: Module A1.1: Each frame of SMPL sequence data is input into a deep network composed of multi-layer perceptrons, i.e., a deep neural network. Module A1.2: Extracting features of the input data based on the deep network and reducing its dimensionality to obtain first-layer features; mapping the first-layer features into dynamic tree model parameters; the dynamic tree model parameters include: joint mass, inertia matrix, damping coefficient, and stiffness coefficient; Module A1.3: outputting a dynamic tree model of the robot based on the dynamic tree model parameters; Module A1.4: Based on the dynamic tree model parameters, calculate and obtain the kinetic energy and potential energy of each joint; solve the kinetic energy and potential energy using the Lagrange equation to obtain the explicit relationship between joint torque and joint position and the explicit relationship between velocity and acceleration, and then obtain the relationship between joint torque and joint motion; In the module A6, optimizing the parameters of the kinetic tree model includes: Module B1: Derives predicted values ​​through the dynamic tree model; the predicted values ​​include: joint position, velocity and torque; Module B2: Calculate the loss function value of the predicted value and the actual data through the loss function; Module B3: The gradient of the loss function with respect to the network parameters is calculated by the back-propagation algorithm, and the network parameters are updated by the gradient descent method to obtain the optimized dynamic tree model.

6. The humanoid robot motion imitation system based on human linear model according to claim 5, characterized in that: In the module M1, the action information is the visual features of the human body shape and the motion features of the changes between frames; the visual features include the human body contour and joint shape; the motion features include the movement and posture of the human body; The action information is extracted based on a deep neural network.

7. The humanoid robot motion imitation system based on human linear model according to claim 6, characterized in that: In the module A1, the dynamic tree model provides the kinetic energy and gravitational potential energy of the robot joints; The mathematical expression of the kinetic energy is: in, represents the kinetic energy of the robot joints, represents the total number of joints of the robot; Indicates the The quality of each joint; i Indicates the The speed of each joint; The mathematical expression of the gravitational potential energy is: in, represents the gravitational potential energy of the robot joints; represents the acceleration due to gravity; Indicates the The height of the center of gravity of the joint; The relationship between the joint torque and joint motion is expressed mathematically as follows: in, Representative The generalized coordinates of the joints, Representative The generalized velocity of each joint, It is Joint torque of each joint; In the module A2, the complete robot dynamics model is expressed as follows: in, is the Coriolis and centrifugal force matrix; represents the mass matrix; represents the gravity term; represents the generalized coordinates of the joint, represents the generalized velocity of the joint; represents the generalized acceleration of the joint.

8. The humanoid robot motion imitation system based on human linear model according to claim 7, characterized in that: Module A3 includes: Module A3.1: Extract target joint positions from the SMPL pose data of each frame; the target joint positions include the positions of the hands and feet; Module A3.2: Based on the target joint position, obtain the angles of each joint, and obtain a set of joint angle values ​​that meet the target joint position based on inverse kinematics; Module A3.3: Optimize the joint angle values ​​based on the dynamic constraints and kinematic restrictions of each joint; the dynamic constraints include velocity restrictions, acceleration restrictions, and torque restrictions; the kinematic restrictions include joint angle ranges and degrees of freedom; the optimization involves setting numerical boundaries for the joint angle values; Module A3.4: Connect the optimized joint angle values ​​into a smooth and continuous trajectory using cubic spline interpolation, i.e., the target motion trajectory; The smooth and continuous trajectory refers to a trajectory with a smoothness index less than or equal to 0.1 and continuous; The mathematical expression of the smoothness index is: in, represents the smoothness index, Indicates the starting time of the trajectory; Indicates the end time of the trajectory; represents the joint angle; Indicates time; In the module B2, the mathematical expression of the loss function is: in, Indicates the number of motion trajectory frames, Indicates the The predicted joint positions for a frame, Indicates the actual joint position; represents the prediction speed, Indicates actual speed; represents the predicted moment, Indicates the actual torque; 、 and They are respectively a weight coefficient, another weight coefficient and another weight coefficient; represents the L2 norm of the vector, that is, the Euclidean distance, Represents the final loss function value; In the module B3, the mathematical expression of the gradient of the loss function to the network parameters is: in, Represents the gradient of the loss function with respect to the network parameters, and the superscript Represents the transpose of a matrix or vector; Indicates the Predicted joint positions for a frame; Indicates the Actual joint positions of the frame; Indicates the The predicted speed of the frame; Indicates the The actual speed of the frame; The gradient of the joint position is expressed mathematically as: in, Gradient representing joint position; The gradient of velocity is expressed mathematically as: in, represents the gradient of velocity; The gradient of the torque is expressed mathematically as: in, represents the gradient of the torque, Represents network parameters; The mathematical expression of the network parameter, that is, the optimization error, is: in, represents the network parameters, i.e., the optimization error, Represents an assignment operation, represents the partial derivative, Represents the final loss function value, Represents the learning rate.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the humanoid robot motion imitation method based on a human linear model according to any one of claims 1 to 4 are implemented.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the computer program is executed by a processor, the steps of the humanoid robot motion imitation method based on a human linear model according to any one of claims 1 to 4 are implemented.

Images