A humanoid robot motion real-time following method and system
By using a neural network mapping method guided by a humanoid robot and driven by a nine-axis sensor, the limitations of expensive equipment and traditional algorithms are overcome, enabling low-cost, high-precision, real-time human-computer interaction and stable following in complex environments.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHINA JILIANG UNIV
- Filing Date
- 2026-03-02
- Publication Date
- 2026-07-24
AI Technical Summary
Existing methods rely on expensive optical motion capture equipment to limit the operator's range and duration of activity. Traditional inverse kinematics algorithms face multiple solutions, singularities, and large computational loads, making it difficult to achieve real-time and smooth human-computer interaction.
A humanoid robot guidance strategy is adopted, which combines a nine-axis sensor and a neural network. Calibration data is generated by the robot's leading actions. Visual methods are used to calibrate and train the neural network for accurate mapping. A multi-level data verification mechanism is introduced to ensure safety and stability.
It achieves high-precision human-computer interaction under low-cost conditions, reduces computational complexity, improves real-time performance and robustness, avoids loss of control of actions, and enables smooth interaction in complex environments.
Smart Images
Figure CN122442610A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of robot research and development technology, specifically to a method and system for real-time motion tracking of a humanoid robot. Background Technology
[0002] In the fields of embodied intelligence and humanoid robot research and development, achieving accurate mapping of human actions to robots is a key step in building robot mobility and collecting training data. Data collection can effectively solve the error problems that may occur in the process of humanoid robots following actions.
[0003] Patent CN119175719B discloses a humanoid robot control method and system based on motion mapping. It can directly map joint muscles and use 3D animation software to optimize the mapped motion to achieve smooth motion mapping. It can obtain the motion trajectory information and foot force of each bone in the humanoid robot animation model through motion data extraction script, which is convenient for subsequent control of the humanoid robot using motion data. By prioritizing the humanoid robot in multiple tasks and using zero-space mapping to achieve whole-body control, the captured human motion can be accurately reproduced while ensuring the stability of the humanoid robot.
[0004] Patent CN119188692A discloses a humanoid robot motion training method based on wearable limb motion recognition. It uses a full-body dynamic capture device based on MEMS posture sensors to acquire real-time motion data of real people. This data is fully compliant with human kinematics and ergonomics, with extremely high accuracy, high data generation efficiency, and extremely low data acquisition cost. A huge motion database can be formed in a short period of time. Furthermore, human stress response data can also be acquired very directly. The motion data of all joints in the whole body can be obtained by simply wearing a wearable limb motion recognition device based on MEMS posture sensing, which can complete all-round motion data in any location and any scene, indoors or outdoors. The acquired data is real and fully compliant with humanoid robot ergonomics, meets the real humanoid robot training requirements, and has significant advantages in acquiring full-body motion coordination data under human stress.
[0005] In the aforementioned patent, the method in the patent can obtain human motion data. However, in the implementation of the above method, motion capture usually relies on expensive optical motion capture equipment, which limits the operator's range of motion and duration. Under the vision solution, directly calculating joint angles through traditional inverse kinematics algorithms often faces problems such as multiple solutions, singularities, and delays caused by large computational loads, making it difficult to meet the needs of real-time and smooth human-computer interaction.
[0006] To address the aforementioned issues, there is an urgent need for innovative designs based on existing methods. Summary of the Invention
[0007] The purpose of this invention is to provide a method and system for real-time motion tracking of humanoid robots, in order to solve the problems mentioned in the background art. In the existing methods, motion capture usually relies on expensive optical motion capture equipment, which limits the operator's range of motion and duration. In the vision solution, joint angle calculation directly through traditional inverse kinematics algorithms often faces problems such as multiple solutions, singularities, and delays caused by large computational load, making it difficult to meet the requirements of real-time and smooth human-computer interaction.
[0008] To achieve the above objectives, the present invention provides the following technical solution: a method and system for real-time motion tracking of a humanoid robot, comprising the following steps:
[0009] S1. Establish a human posture model and a humanoid robot model;
[0010] Define the location of human joints as The coordinates of key points are obtained through a pose extraction module after photos are captured by a camera. The angle of the humanoid robot's motor is defined as... Let the coordinates of the 12 motor units be... The motor angle O is the actual rotation angle of the motor, which can be obtained by querying the motor status.
[0011] S2, System state initialization;
[0012] Define the initial state motor vector of the humanoid robot. It is a vector of all zeros, that is At this time, the robot is in a static, upright position with its arms hanging naturally at its sides.
[0013] S3. Calibrate the human body posture increment and the change in the angle of the humanoid robot motor;
[0014] The system generates calibration data by employing a strategy of humanoid robot guidance combined with human follow-up. First, it randomly generates a set of motor angle increments that conform to the mechanical limits of the humanoid robot. Drive the robot to perform action changes;
[0015] S4. Real-time posture adjustment of humanoid robots driven by wearable nine-axis sensors;
[0016] The incremental data from the nine-axis sensors is calculated and input into the neural network to control the robot's angle and verify the limit. Multiple sets of data are integrated and processed to improve the reliability of the data.
[0017] Preferably, in step S1;
[0018] Define the location of human joints as The coordinates of key points were obtained through a pose extraction module after the camera captured images, extracting the positions of 12 key nodes in total, such as... Figure 1 As shown, the left shoulder joint is denoted as... The right shoulder joint is The left elbow joint is The right elbow joint is The left wrist joint is The right wrist joint is The left hip joint is The right hip joint is The left knee joint is The right knee joint is The left ankle joint is The right ankle joint is , ,in, Indicates the direction vector of the left upper arm; This represents the direction vector of the left forearm. Indicates the direction vector of the right upper arm; This represents the direction vector of the right forearm; This represents the direction vector of the left thigh. This represents the direction vector of the left lower leg; This represents the direction vector of the right thigh. This represents the direction vector of the right lower leg; This represents the shoulder direction vector; This represents the hip joint direction vector at each node of the human body. A nine-axis sensor from Shenzhen Weite Intelligent Technology Co., Ltd., model WT901WiFi, is worn at the device. The readings from each sensor are recorded as follows: ,remember .
[0019] Preferably, in step S1;
[0020] Define the angle of the motor of the humanoid robot (Universal Tree Technology G1-edu as an example) as follows: Let the coordinates of the 12 motor units be... motor angle The actual rotation angle of the motor can be obtained by querying the motor status; coordinates are also available. This refers to the coordinates of the joint points in the camera's image, obtained through a pose extraction module after the camera captures the image. The angle of the left shoulder joint motor is recorded as... Its spatial location is The angle of the right shoulder joint motor is Its spatial location is The angle of the left elbow joint motor is Its spatial location is The angle of the right elbow joint motor is Its spatial location is The angle of the left wrist joint motor is Its spatial location is The angle of the right wrist joint motor is Its spatial location is The angle of the left hip joint motor is Its spatial location is The angle of the right hip joint motor is Its spatial location is The angle of the left knee joint motor is Its spatial location is The angle of the right knee joint motor is Its spatial location is The angle of the left ankle joint motor is Its spatial location is The angle of the right ankle joint motor is Its spatial location is The humanoid robot's pose vector is denoted as:
[0021]
[0022] in, ; ; ; ; ; ; ; ; ; All are three-dimensional vectors.
[0023] Preferably, in step S2:
[0024] Define the initial motor angle vector for motion control of the humanoid robot. It is a vector of all zeros, that is This initial vector corresponds to the robot's standard upright and static initial posture with its arms hanging naturally and no deflection at any joint.
[0025] Preferably, in step S2:
[0026] Before operation, the operator must maintain a perfectly aligned upright reference posture with the robot, remain stable in this static posture for a preset time, and wait until the state of the human joints shows no significant fluctuations. Then, read the data collected by the nine-axis sensors worn on the human joints at this moment. The actual readings of the human body's nine-axis sensors in this stable initial state are defined as the initial reference data for human motion perception. This serves as a reference value for subsequent motion mapping control.
[0027] Preferably, in step S3;
[0028] The system generates calibration data by employing a strategy of humanoid robot guidance combined with human follow-up. First, it randomly generates a set of motor angle increments that conform to the mechanical limits of the humanoid robot. Drive the robot to perform action transformations, define Time's up The changes in the humanoid robot's motors at any given moment are as follows:
[0029]
[0030] The operator observes and imitates the humanoid robot's movements, adjusting their own until the human posture matches the robot's posture to the required consistency (see step S3 below for the specific calculation method). Simultaneously, the system records the data captured by the sensors. Human pose vector at any time Increment The calculation formula is:
[0031] .
[0032] Preferably, in step S3;
[0033] The two sets of data obtained from the calculation constitute a paired sample. Store the data in the dataset, and repeat this process until multiple sets of samples covering different magnitudes and dimensions are collected, denoted as the dataset. ,by As input, with As output, train a neural network net. The suggested neural network structure is as follows:
[0034] Input layer: 108 units, representing the increment of the human pose vector. ;
[0035] First layer: Fully connected layer, 128 units, LeakyReLU activation function;
[0036] The second layer is a fully connected layer with 256 cells and the activation function is LeakyReLU.
[0037] The third layer: a fully connected layer with 128 units and LeakyReLU activation function;
[0038] Output layer: Fully connected layer, 12 units, activation function is a linear function, applied to the 12 motor angle increments of the humanoid robot. For regression prediction, the loss function is the least squares error loss function, and the training algorithm can be SGD, SGDM, Adagrad, RMSProp, or Adam.
[0039] Preferably, in step S3;
[0040] The steps for determining the consistency between human pose and humanoid robot pose are as follows: Take photos of the human pose and humanoid robot pose using the same type of camera; then extract key points for both the human body and the humanoid robot using a human pose extraction algorithm; finally, calculate the key points using the method in step S1. and ,like and If the deviations between the 12 corresponding vectors and the vertical angle are all less than a certain threshold, it is recommended that... At that time, the consistency between the human body posture and the humanoid robot posture meets the requirements.
[0041] Preferably, in step S4;
[0042] Data from the nine-axis sensor worn on the human body as described in step S1 is collected in real time at a fixed sampling frequency. Each collected data point is recorded as follows: ,calculate ,Will The input is fed into the neural network trained in step 3, and the result is calculated. In the middle, the verification prediction angle is increased. Is it within the robot's physical limits?
[0043] Preferably, in step S4;
[0044] The data after the prediction angle was increased was verified. Within the calculated angle, the angle command is sent to the humanoid robot's motor controller, driving the robot's motors to the calculated angle. This completes the mapping and control from the human body to the humanoid robot, among which... , See step S2 for reference.
[0045] Compared with the prior art, the beneficial effects of the present invention are:
[0046] 1. A data acquisition strategy is adopted that uses robot-led actions and human follow-up imitation. This method uses motor angle increments that conform to mechanical limits to generate samples, thereby preventing training data from exceeding the robot's physical activity range from the source, ensuring the safety of model training and control stability.
[0047] 2. A visual method is adopted as the calibration method for the equipment, which is more scientific. During the training phase, a nine-axis sensor is used to ensure the accuracy of the true data. During the calibration phase, projection mapping technology is used to drive the process, which takes into account the advantages of high-precision control and low-cost deployment.
[0048] 3. Furthermore, by introducing a neural network model, a precise mapping from human posture to robot motion is achieved. This method not only solves the problems of multiple solutions and singularities in traditional inverse kinematics algorithms, but also significantly reduces computational complexity, thereby improving real-time performance. The training process of the neural network makes full use of multi-dimensional and multi-amplitude sample data to ensure the stability of the model under various posture changes.
[0049] 4. Furthermore, by introducing a multi-layered data verification mechanism, physical limit verification is performed on the results after each angle prediction, avoiding robot motion loss due to model prediction errors or abnormal data. This design significantly improves the system's robustness, enabling it to maintain stable performance in complex and ever-changing real-world application scenarios.
[0050] 5. Furthermore, by calculating the difference between the nine-axis sensor data and integrating multiple sets of data, the reliability and anti-interference ability of the system are further improved, providing technical support for the smooth interaction of humanoid robots in complex environments. Attached Figure Description
[0051] Figure 1 The joint location of the present invention Schematic diagram of the marked state structure;
[0052] Figure 2 This is a schematic diagram of the coordinate P-mark state structure of the motor unit in this invention;
[0053] Figure 3 This is a schematic diagram of the coordinate P-mark state structure of the motor unit in this invention; Figure 4 This is a schematic diagram of the coordinate P-mark state structure of the motor unit in this invention; Figure 5 This is a schematic diagram of the state structure of the motor unit coordinate O mark in this invention; Figure 6 This is a schematic diagram of the state structure for constructing the model of the present invention. Detailed Implementation
[0054] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0055] Example 1: In a specific embodiment, the present invention provides the following technical solution: a method and system for real-time motion tracking of a humanoid robot, such as... Figure 1 - Figure 3 The specific operation process of the method and system is disclosed as shown.
[0056] S1. Establish a human posture model and a humanoid robot model;
[0057] Define the location of human joints as The coordinates of key points are obtained through a pose extraction module after photos are captured by a camera. The angle of the humanoid robot's motor is defined as... Let the coordinates of the 12 motor units be... The motor angle O is the actual rotation angle of the motor, which can be obtained by querying the motor status.
[0058] Define the location of human joints as The coordinates of key points were obtained through a pose extraction module after the camera captured images, extracting the positions of 12 key nodes in total, such as... Figure 1 As shown, the left shoulder joint is denoted as... The right shoulder joint is The left elbow joint is The right elbow joint is The left wrist joint is The right wrist joint is The left hip joint is The right hip joint is The left knee joint is The right knee joint is The left ankle joint is The right ankle joint is Define the three-dimensional vector of the human body. ,in, Indicates the direction vector of the left upper arm; This represents the direction vector of the left forearm. Indicates the direction vector of the right upper arm; This represents the direction vector of the right forearm; This represents the direction vector of the left thigh. This represents the direction vector of the left lower leg; This represents the direction vector of the right thigh. This represents the direction vector of the right lower leg; This represents the shoulder direction vector; This represents the hip joint direction vector at each node of the human body. A nine-axis sensor is worn at the device, and the readings from each sensor are recorded as follows: ,remember .
[0059] Define the angle of the humanoid robot motor as Let the coordinates of the 12 motor units be... motor angle The actual rotation angle of the motor can be obtained by querying the motor status; coordinates are also available. This refers to the coordinates of the joint points in the camera's image, obtained through a pose extraction module after the camera captures the image. The angle of the left shoulder joint motor is recorded as... Its spatial location is The angle of the right shoulder joint motor is Its spatial location is The angle of the left elbow joint motor is Its spatial location is The angle of the right elbow joint motor is Its spatial location is The angle of the left wrist joint motor is Its spatial location is The angle of the right wrist joint motor is Its spatial location is The angle of the left hip joint motor is Its spatial location is The angle of the right hip joint motor is Its spatial location is The angle of the left knee joint motor is Its spatial location is The angle of the right knee joint motor is Its spatial location is The angle of the left ankle joint motor is Its spatial location is The angle of the right ankle joint motor is Its spatial location is The humanoid robot's pose vector is denoted as:
[0060]
[0061] in, ; ; ; ; ; ; ; ; ; All are three-dimensional vectors.
[0062] S2, System state initialization;
[0063] Define the initial state motor vector of the humanoid robot. It is a vector of all zeros, that is At this time, the robot is in a static, upright position with its arms hanging naturally at its sides.
[0064] Define the initial motor angle vector for motion control of the humanoid robot. It is a vector of all zeros, that is This initial vector corresponds to the robot's standard upright and static initial posture with its arms hanging naturally and no deflection at any joint.
[0065] Before operation, the operator must maintain a perfectly aligned upright reference posture with the robot, remain stable in this static posture for a preset time, and wait until the state of the human joints shows no significant fluctuations. Then, read the data collected by the nine-axis sensors worn on the human joints at this moment. The actual readings of the human body's nine-axis sensors in this stable initial state are defined as the initial reference data for human motion perception. This serves as a reference value for subsequent motion mapping control.
[0066] S3. Calibrate the human body posture increment and the change in the angle of the humanoid robot motor;
[0067] The system generates calibration data by employing a strategy of humanoid robot guidance combined with human follow-up. First, it randomly generates a set of motor angle increments that conform to the mechanical limits of the humanoid robot. Drive the robot to perform action changes;
[0068] The system generates calibration data by employing a strategy of humanoid robot guidance combined with human follow-up. First, it randomly generates a set of motor angle increments that conform to the mechanical limits of the humanoid robot. Drive the robot to perform action transformations, define Time's up The changes in the humanoid robot's motors at any given moment are as follows:
[0069]
[0070] The operator observes and imitates the humanoid robot's movements, adjusting their own actions until the human posture matches the robot's posture to the required consistency (see below for specific calculation methods). Simultaneously, the system records the data captured by the sensors. Human pose vector at any time Increment The calculation formula is:
[0071] .
[0072] The two sets of data obtained from the calculation constitute a paired sample. Store the data in the dataset, and repeat the above process until multiple sets of samples covering different magnitudes and dimensions are collected, denoted as the dataset. ,by As input, with As output, train a neural network net. The suggested neural network structure is as follows:
[0073] Input layer: 108 units, representing the increment of the human pose vector. ;
[0074] First layer: Fully connected layer, 128 units, LeakyReLU activation function;
[0075] The second layer is a fully connected layer with 256 cells and the activation function is LeakyReLU.
[0076] The third layer: a fully connected layer with 128 units and LeakyReLU activation function;
[0077] Output layer: Fully connected layer, 12 units, activation function is a linear function, applied to the 12 motor angle increments of the humanoid robot. For regression prediction, the loss function is the least squares error loss function, and the training algorithm can be SGD, SGDM, Adagrad, RMSProp, or Adam.
[0078] The steps for determining the consistency between human pose and humanoid robot pose are as follows: Take photos of the human pose and humanoid robot pose using the same type of camera; then extract key points for both the human body and the humanoid robot using a human pose extraction algorithm; finally, calculate the key points using the method in step S1. and ,like and If the deviations between the 12 corresponding vectors and the vertical angle are all less than a certain threshold, it is recommended that... At that time, the consistency between the human body posture and the humanoid robot posture meets the requirements.
[0079] S4. Real-time posture adjustment of humanoid robots driven by wearable nine-axis sensors;
[0080] The robot's angle is controlled by a neural network using nine-axis data to calculate the difference and sensor data to verify the limit. Multiple sets of data are integrated and processed to improve the reliability of the data.
[0081] Data from the nine-axis sensor worn on the human body as described in step S1 is collected in real time at a fixed sampling frequency. Each collected data point is recorded as follows: ,calculate ,Will The input is fed into the neural network trained in step S3, and the result is calculated. In the middle, the verification prediction angle is increased. Whether it is within the robot's physical limits, verify the data after increasing the predicted angle. Within the calculated angle, the angle command is sent to the humanoid robot's motor controller, driving the robot's motors to the calculated angle. This completes the mapping and control from the human body to the humanoid robot, among which... , See step S2 for reference.
[0082] The system includes: a humanoid robot with a left shoulder joint, a right shoulder joint, and a left elbow joint. The robot features a right elbow joint, left wrist joint, right wrist joint, left hip joint, right hip joint, left knee joint, right knee joint, left ankle joint, and right ankle joint, with joint rotation positions driven by motors; a camera for capturing the humanoid robot's posture; and a nine-axis sensor for collecting and recording human posture data.
[0083] Example 2: In one specific embodiment, such as Figure 1 - Figure 3 As shown, this method can be performed quickly.
[0084] In the actual operation of this method, in order to further improve the system's response speed, the data from the nine-axis sensor can be filtered to reduce noise interference and improve data reliability. After the neural network predicts the output, a smoothing module is added to avoid the occurrence of discontinuous or unstable robot movements due to sudden angle changes.
[0085] Considering the complex motion scenarios that humanoid robots may face, the system also introduces a dynamic adjustment mechanism. When the rate of change of human posture is detected to exceed a preset threshold, the system will automatically reduce the sampling frequency to reduce the computational burden and ensure smooth overall operation. When the human posture tends to stabilize, high-frequency sampling will be restored to capture more detailed motion.
[0086] To enhance the accuracy of the measurement results, the system prioritizes ensuring the accuracy of action reproduction, making it suitable for application scenarios that require a high degree of consistency. In "Fast Mode," the system focuses on shortening the latency, making it suitable for interactive scenarios with high real-time requirements.
[0087] To facilitate subsequent optimization and debugging, the system records key data during each run, including nine-axis sensor readings, neural network inputs and outputs, and motor execution results, and generates log files for analysis. This data not only helps to identify potential problems, but also serves as supplementary material for training new models, thereby continuously improving the system's performance.
[0088] Example 3: Based on the above examples, such as... Figure 1 - Figure 3 As shown, the process by which this method can be accurately applied to a variety of situations is disclosed.
[0089] In practical applications, to ensure the accuracy and real-time performance of humanoid robot movements, the system needs to further optimize the data processing flow. During the sensor data acquisition stage, a Kalman filter algorithm is introduced to preprocess the raw data from the nine-axis sensors. This algorithm can effectively fuse the data from the accelerometer and gyroscope to obtain more accurate attitude estimates.
[0090] In the neural network prediction stage, an anomaly detection mechanism is added. When the motor angle increment output by the neural network... When an abnormal value occurs, such as exceeding the preset reasonable range or deviating significantly from the historical data trend, the system will automatically trigger the protection mechanism, suspend the sending of instructions and prompt the operator to check. This design can avoid robot loss of control or mechanical damage caused by data anomalies.
[0091] To improve system stability, it is recommended to introduce adversarial example augmentation techniques when training neural networks. By adding small perturbations to samples in the original dataset to generate adversarial examples and incorporating them into the training process, the model can have stronger generalization ability and thus exhibit higher adaptability when facing complex and ever-changing real-world scenarios.
[0092] For operators with different body types or movement habits, the system should support personalized calibration functions. During the initialization phase, operators should be able to adjust key parameters, such as joint mapping relationships or motion sensitivity, through a simple interactive interface to achieve optimal tracking performance. This flexibility not only enhances the user experience but also expands the system's applicability and increases its overall usability.
[0093] The contents not described in detail in this specification are existing technologies known to those skilled in the art.
[0094] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.
Claims
1. A method for real-time motion tracking of a humanoid robot, characterized in that: Includes the following steps: S1. Establish a human posture model and a humanoid robot model; Define the location of human joints as The coordinates of key points are obtained through a pose extraction module after photos are captured by a camera. The angle of the humanoid robot's motor is defined as... Let the coordinates of the 12 motor units be... The motor angle O is the actual rotation angle of the motor, which can be obtained by querying the motor status. S2, System state initialization; Define the initial state motor vector of the humanoid robot. It is a vector of all zeros, that is At this time, the robot is in a static, upright position with its arms hanging naturally at its sides. S3. Calibrate the human body posture increment and the change in the angle of the humanoid robot motor; The system generates calibration data by employing a strategy of humanoid robot guidance combined with human follow-up. First, it randomly generates a set of motor angle increments that conform to the mechanical limits of the humanoid robot. Drive the robot to perform action changes; S4. Real-time posture adjustment of humanoid robots driven by wearable nine-axis sensors; The incremental data from the nine-axis sensors is calculated and input into the neural network to control the robot's angle and verify the limit. Multiple sets of data are integrated and processed to improve the reliability of the data.
2. The method for real-time motion tracking of a humanoid robot according to claim 1, characterized in that: In step S1; Define the location of human joints as The coordinates of key points were obtained by capturing images with a camera and then using the MediaPipe module in Python to extract the positions of 12 key nodes, denoted as the left shoulder joint. The right shoulder joint is The left elbow joint is The right elbow joint is The left wrist joint is The right wrist joint is The left hip joint is The right hip joint is The left knee joint is The right knee joint is The left ankle joint is The right ankle joint is Define the three-dimensional vector of the human body. ,in, Indicates the direction vector of the left upper arm; This represents the direction vector of the left forearm. Indicates the direction vector of the right upper arm; This represents the direction vector of the right forearm; This represents the direction vector of the left thigh. This represents the direction vector of the left lower leg; This represents the direction vector of the right thigh. This represents the direction vector of the right lower leg; This represents the shoulder direction vector; This represents the hip joint direction vector at each node of the human body. A nine-axis sensor is worn at the device, and the readings from each sensor are recorded as follows: , remember .
3. The method for real-time motion tracking of a humanoid robot according to claim 1, characterized in that: In step S1; Define the angle of the humanoid robot motor as Let the coordinates of the 12 motor units be... motor angle The actual rotation angle of the motor can be obtained by querying the motor status; coordinates are also available. This refers to the coordinates of the joint points in the camera's image, obtained through a pose extraction module after the camera captures the image. The angle of the left shoulder joint motor is recorded as... Its spatial location is The angle of the right shoulder joint motor is Its spatial location is The angle of the left elbow joint motor is Its spatial location is The angle of the right elbow joint motor is Its spatial location is The angle of the left wrist joint motor is Its spatial location is The angle of the right wrist joint motor is Its spatial location is The angle of the left hip joint motor is Its spatial location is The angle of the right hip joint motor is Its spatial location is The angle of the left knee joint motor is Its spatial location is The angle of the right knee joint motor is Its spatial location is The angle of the left ankle joint motor is Its spatial location is The angle of the right ankle joint motor is Its spatial location is The humanoid robot's pose vector is denoted as: ； in, ; ; ; ； ； ； ； ; ; All are three-dimensional vectors.
4. The method for real-time motion tracking of a humanoid robot according to claim 1, characterized in that: In step S2: Define the initial motor angle vector for motion control of the humanoid robot. It is a vector of all zeros, that is This initial vector corresponds to the robot's standard upright and static initial posture with its arms hanging naturally and no deflection at any joint.
5. The method for real-time motion tracking of a humanoid robot according to claim 1, characterized in that: In step S2: Before operation, the operator must maintain a perfectly aligned upright reference posture with the robot, remain stable in this static posture for a preset time, and wait until the state of the human joints shows no significant fluctuations. Then, read the data collected by the nine-axis sensors worn on the human joints at this moment. The actual readings of the human body's nine-axis sensors in this stable initial state are defined as the initial reference data for human motion perception. This serves as a reference value for subsequent motion mapping control.
6. The method for real-time motion tracking of a humanoid robot according to claim 1, characterized in that: In step S3; The system generates calibration data by employing a strategy of humanoid robot guidance combined with human follow-up. First, it randomly generates a set of motor angle increments that conform to the mechanical limits of the humanoid robot. Drive the robot to perform action transformations, define Time's up The changes in the humanoid robot's motors at any given moment are as follows: ； The operator observes and imitates the humanoid robot's movements, adjusting their own until the human posture matches the robot's posture to the required consistency (see step S3 below for the specific calculation method). Simultaneously, the system records the data captured by the sensors. Human pose vector at any time Increment The calculation formula is: 。 7. The method for real-time motion tracking of a humanoid robot according to claim 1, characterized in that: In step S3; The two sets of data obtained from the calculation constitute a paired sample. Store the data in the dataset, and repeat the above process until multiple sets of samples covering different magnitudes and dimensions are collected, denoted as the dataset. ,by As input, with As output, train a neural network net. The suggested neural network structure is as follows: Input layer: 108 units, representing the increment of the human pose vector. ; First layer: Fully connected layer, 128 units, LeakyReLU activation function; The second layer is a fully connected layer with 256 cells and the activation function is LeakyReLU. The third layer: a fully connected layer with 128 units and LeakyReLU activation function; Output layer: Fully connected layer, 12 units, activation function is a linear function, applied to the 12 motor angle increments of the humanoid robot. For regression prediction, the loss function is the least squares error loss function, and the training algorithm can be SGD, SGDM, Adagrad, RMSProp, or Adam.
8. The method for real-time motion tracking of a humanoid robot according to claim 1, characterized in that: In step S3; The steps for determining the consistency between human pose and humanoid robot pose are as follows: Take photos of the human pose and humanoid robot pose using the same type of camera; then extract key points for both the human body and the humanoid robot using a human pose extraction algorithm; finally, calculate the key points using the method in step S1. and ,like and If the deviations between the 12 corresponding vectors and the vertical angle are all less than a certain threshold, it is recommended that... At that time, the consistency between the human body posture and the humanoid robot posture meets the requirements.
9. The method for real-time motion tracking of a humanoid robot according to claim 1, characterized in that: In step S4; Data from the nine-axis sensor worn on the human body as described in step S1 is collected in real time at a fixed sampling frequency. Each collected data point is recorded as follows: ,calculate ,Will The input is fed into the neural network trained in step S3, and the result is calculated. Verify the increase in prediction angle Whether it is within the robot's physical limits, verify the data after increasing the predicted angle. Within the calculated angle, the angle command is sent to the humanoid robot's motor controller, driving the robot's motors to the calculated angle. This completes the mapping and control from the human body to the humanoid robot, among which... , See step S2 for reference.
10. A real-time motion tracking system for a humanoid robot, wherein the real-time motion tracking method for a humanoid robot as described in any one of claims 1-9 is characterized in that: The system includes: The humanoid robot has a left shoulder joint, a right shoulder joint, and a left elbow joint. The robot features a right elbow joint, left wrist joint, right wrist joint, left hip joint, right hip joint, left knee joint, right knee joint, left ankle joint, and right ankle joint, with joint rotation positions driven by motors; a camera for capturing the humanoid robot's posture; and a nine-axis sensor for collecting and recording human posture data.