Humanoid robot whole body control method based on human body action collection

Abstract

The invention provides a humanoid robot whole body control method based on human motion collection. The humanoid robot whole body control method comprises the steps of 1, collecting multi-modal human motion data; step 2, carrying out format standardization on the acquired multi-modal human body motion data; and step 3, a repositioning method of reference motion data of the humanoid robot. Human body motion data sequence input is utilized, key motion information is extracted from the human body motion data sequence, the information is repositioned to the humanoid robot, human body motion data of different modes can be fully adapted, information such as corresponding joint positions and rotation can be analyzed, a reference data set for simulating human motion by the humanoid robot is generated, and a reinforcement learning algorithm is deployed. The robot can realize robust whole-body control in diversified and dynamically changing environments, so that the control robustness is improved, and various human motion data paradigms acting on humanoid robot learning are effectively unified.

Description

Technical Field

[0001] The present invention belongs to the technical field of humanoid robots, and in particular relates to a whole-body control method of a humanoid robot based on human motion collection. Background Art

[0002] In recent years, humanoid robots, as an intelligent entity that can imitate and replicate human behavior, have attracted widespread attention. Humanoid robots have become an important research direction in robotics due to their potential applications in many fields such as service, industry, medical care, and family companionship. Humanoid robots are able to perform complex tasks such as walking, running, jumping, and carrying objects, thanks to their human-like appearance and multi-degree-of-freedom joint design. However, how to enable humanoid robots to achieve robust whole-body control and learning in diverse and dynamically changing environments remains a huge challenge.

[0003] Humanoid robots are a natural hardware platform for general-purpose robots, as they are similar in appearance to humans, and can potentially solve all tasks that humans can accomplish. Real-world human equipment, tasks, and tools are built and designed based on human form, which provides a unique opportunity for humanoid robots to be trained using a large amount of human motion and skill data, making up for the scarcity of traditional robot data. By imitating humans, humanoid robots can potentially take advantage of the rich skills and movements exhibited by humans, providing a promising path to achieving general-purpose robot intelligence.

[0004] The existing humanoid robot control has the following problems:

[0005] 1. In the field of robot control, current control still relies on model-driven methods, which decouples the problem from physical modeling, mechanical analysis, and predefined control rules to achieve robot motion. However, these methods are usually sensitive to environmental changes and the diversity of tasks, and it is difficult to maintain stability in complex practical scenarios. In addition, traditional methods rely on high-precision physical models, are highly dependent on the environment and tasks, and are difficult to adapt to changes in uncertain environments.

[0006] 2. Although humanoid robots are very similar to humans compared to other forms of robots, there are still significant differences in physical structure and dynamic characteristics between humanoid robots and humans in terms of morphology and drive, including the number of degrees of freedom, connecting rod length, height, weight, visual parameters and mechanism, drive strength and responsiveness, which makes it impossible for humanoid robots to effectively read and interpret human motion data, resulting in unstable and unnatural movements, which creates obstacles for humanoid robots to effectively use and learn human data.

[0007] 3. In recent years, data-driven control has attracted widespread attention. In particular, the use of reinforcement learning methods enables robots to autonomously learn and optimize their control through interaction with the environment. However, reinforcement learning faces huge challenges in the high-dimensional and continuous control space of humanoid robots, mainly in terms of algorithm convergence, learning efficiency and robustness. In addition, the lack of high-quality training data and safety issues in real environments are also bottlenecks that limit the application of reinforcement learning methods in actual robot control.

[0008] Therefore, developing a general method to transmit rich human motion data to robots so that they can learn and imitate human actions in simulated and real environments is a practical and effective path to achieve robust control of humanoid robots. Summary of the invention

[0009] Purpose of the invention: In view of the above-mentioned prior art, a method for controlling the whole body of a humanoid robot based on human motion acquisition is proposed;

[0010] Technical solution:

[0011] A humanoid robot full body control method based on human motion acquisition, comprising:

[0012] Step 1: Multimodal human motion data collection;

[0013] Step 2: Normalize the format of the collected multimodal human motion data;

[0014] Step 3: Relocalization method of humanoid robot with reference motion data.

[0015] Preferably, multimodal human motion data collection includes: extracting human motion data from a public human motion data set, collecting human motion data from an IMU inertial measurement unit, and collecting human motion data from pure video.

[0016] Preferably, formatting the collected multimodal human motion data includes adopting npy or npz format, converting the collected FBX format to npy or npz, or BVH format.

[0017] Preferably, a method for repositioning a humanoid robot with reference to motion data is as follows:

[0018] Step 3.1: URDF file processing: URDF is an XML format file used to describe the structure of the robot. It defines information including the geometry, connection method, joint type, position, rotation axis, and range limit of each joint of the robot. The joint position and angle in the human motion data are matched with the parameters defined in the URDF file.

[0019] Step 3.2: Joint mapping and conversion: determine the number and type of the humanoid robot joints, including rotational joints, translational joints, and motion axes; establish a one-to-one mapping relationship between human joints and robot joints according to the number and type of the humanoid robot joints, including rotational joints, translational joints, and motion axes;

[0020] Step 3.3: Local and global coordinate system conversion: Since the coordinate systems used by the human body and the robot are different, the coordinate system conversion is required;

[0021] Step 3.4: Motion data optimization and correction: After completing the basic joint mapping, the motion data needs to be further corrected to ensure its consistency and physical feasibility;

[0022] Step 3.5: Write the converted motion data into the control system of the humanoid robot in a format supported by URDF.

[0023] Preferably, the number of joints and degrees of freedom of human motion data will have more redundancy than that of humanoid robots. Some degrees of freedom will be deleted in a heuristic way to achieve complete alignment of the two data information and ensure that the data conforms to the joint motion range defined in URDF; if it exceeds the range, the data is adjusted using scaling or threshold limits to avoid compromising the authenticity and coherence of the motion.

[0024] Preferably, smoothing filtering technology is applied, including low-pass filtering or Kalman filtering to reduce high-frequency noise and smooth the data curve, thereby eliminating unnecessary jitter; in addition, artifacts are detected by analyzing the abnormal values ​​in the human motion data, and interpolation correction is performed using adjacent normal data points.

[0025] Preferably, the motion data is sent to the control system of the humanoid robot using the relevant library of URDF to issue real-time motion instructions so that the robot moves according to a predetermined motion trajectory; after verifying the validity of the data, the actual motion data is collected and a reference motion data set is generated, and the reference motion data set is used as training data for machine learning or reinforcement learning models to improve the robustness and adaptability of humanoid robot control; the reference motion data set is also used to verify the performance of new controls; by comparing the ideal motion trajectory and the actual motion trajectory in the reference motion data set, the accuracy and stability of the humanoid robot control system are evaluated.

[0026] Beneficial effects: The present invention discloses a general system and implementation path for a humanoid robot to imitate human motion to achieve whole-body control. The system utilizes human motion data sequence input, extracts key motion information therefrom, and relocates the information to the humanoid robot. The system can fully adapt to human motion data of different modalities, parse corresponding joint positions, rotations and other information, generate a reference data set for humanoid robots to imitate human motion, and deploy reinforcement learning algorithms, so that the robot can achieve robust whole-body control in a diverse and dynamically changing environment. This not only improves the robustness of the control, but also effectively unifies a variety of human motion data paradigms that act on humanoid robot learning, thereby enhancing the robot's adaptability in real environments. DETAILED DESCRIPTION

[0027] The present invention will be further explained below.

[0028] In order to achieve the above technical objectives, the present invention adopts the following technical solutions:

[0029] A general implementation path for whole-body control learning of a humanoid robot, including a humanoid robot motion data acquisition and normalization path and a whole-body control learning method. The humanoid robot motion data acquisition and normalization path includes a multimodal human motion data acquisition scheme, a human motion data format normalization method, and a general repositioning method for humanoid robot reference motion data. The whole-body control learning method is a task-independent low-level method for whole-body control of humanoid motion based on reinforcement learning algorithm training in simulation.

[0030] 1. The multi-modal human motion data acquisition scheme:

[0031] 1. Extracting motions from public human motion datasets: Public human motion datasets are usually released by research institutions or commercial companies and contain a large amount of processed and annotated motion data. Common public datasets include CMU MotionCapture Database, Human3.6M, AMASS, etc. These datasets contain a variety of motion types, such as walking, running, jumping, weightlifting, etc., covering a variety of human postures and movements. The advantages of public datasets are large data volume, rich data types, and detailed annotations, which are suitable for preliminary training and verification of models. However, its limitation is that the data collection environment is usually a laboratory scene, which lacks adaptability to complex real environments, and the data may not fully meet the needs of specific application scenarios.

[0032] 2. Collecting human motion data from IMU (Inertial Measurement Unit): IMU is a sensor widely used in human motion data collection. It infers the motion state of the human body by detecting physical quantities such as acceleration and angular velocity. IMU is usually composed of accelerometers, gyroscopes and magnetometers. When collecting human motion data, IMU sensors need to be fixed on multiple key parts of the human body, such as limbs, waist, chest, etc. These sensors can record the motion information of each part in real time and infer the overall motion trajectory of the human body through data fusion technology. The data collected by IMU includes three-axis acceleration, three-axis angular velocity and magnetic field intensity. These raw data can be integrated to calculate the posture change, speed and displacement of each part. Combined with the human skeletal model, IMU data can be used to reconstruct the motion trajectory of the human body. The advantages of IMU are strong portability, low cost, and ability to be used in complex environments. It is suitable for long-term monitoring and large-scale data collection. However, the accuracy of IMU is lower than that of motion capture systems, and due to sensor drift, long-term data collection may lead to error accumulation. In addition, it is difficult for IMU to accurately capture the relative motion of various parts of the human body, and it is usually necessary to combine other sensor data to improve accuracy.

[0033] 3. Collecting human motion data from pure video: Pure video acquisition is another method to obtain human motion data, which uses computer vision technology to extract motion information from video. Video data can be obtained by shooting with a monocular or multi-camera. The scene of video acquisition can be a laboratory, outdoors, or any other practical application scene. The camera settings and resolution directly affect the quality and accuracy of the data. Through computer vision technology, especially deep learning models, the joint positions and motion trajectories of the human body can be extracted from the video. Common methods include pose estimation and depth estimation. Models such as OpenPose and HRNet can identify and track human key points from monocular or multi-camera videos and generate motion sequences of human skeletons. In the case of multi-camera videos, the accuracy of three-dimensional motion data can be further improved by triangulation. The advantage of pure video acquisition is that it is non-invasive, does not require any equipment to be attached to the human body, and can collect data in natural scenes. In addition, video acquisition can capture rich environmental context information, which is convenient for subsequent behavior recognition and scene understanding.

[0034] 2. Method for normalizing the human motion data format

[0035] Normalizing the format of human motion data is a key step. This process ensures that motion data from different sources can be processed and used uniformly, facilitating subsequent algorithm development and model training. Data format normalization involves not only the storage and reading of data, but also the conversion of the format so that the data can be directly used by a wide range of machine learning and reinforcement learning tools.

[0036] 1.npy / npz format: a commonly used data storage format in Python, especially suitable for processing large-scale numerical data such as human motion data types. NPY and NPZ formats are directly supported by Python's NumPy library. These formats are very efficient in storing and reading data, and can quickly load large-scale multidimensional array data. This efficiency is particularly important for application scenarios that process large amounts of motion data, such as control learning of humanoid robots.

[0037] 2. Conversion from FBX format to NPY / NPZ format: FBX (Filmbox) format is a universal 3D animation format, widely used to store skeletal animation data and 3D models. Use specialized libraries (such as pyfbx or Autodesk's FBXSDK) to parse FBX files. During the parsing process, extract the joint position, rotation, and bone hierarchy information related to human motion. The extracted joint data usually exists in the form of a local coordinate system. In order to facilitate subsequent processing, the local coordinates need to be converted to global coordinates. This step can be completed by recursively calculating the transformation matrix of the joint relative to the parent node. Once the global coordinate data of all joints is extracted and calculated, the data can be organized into a multidimensional array. The data of each frame can be stored as a three-dimensional array (number of joints × 3 (X, Y, Z) coordinates). Finally, use NumPy to save these multidimensional arrays in NPY or NPZ format. For example, use numpy.savez_compressed() to compress and store multiple frames of motion data into an NPZ file for subsequent use.

[0038] 3.BVH (Biovision Hierarchy) format is a common 3D animation format, especially in motion capture data. BVH files usually contain two parts: bone structure information and motion data. Use a dedicated parsing library (such as bvh-python) to read BVH files. The parsing process includes extracting the bone hierarchy and the motion data of each frame. The bone structure information usually defines the parent-child relationship of each joint and its initial position, while the motion data records the rotation or displacement information of the joint in each frame. Convert the motion data in BVH from the rotation angle (usually Euler angle) to the joint position in the global coordinate system. This usually requires recursive calculation of the transformation matrix of each joint relative to the root node. Convert the motion data of each frame into a three-dimensional array containing the positions of all joints in space. Similarly, the processed data will be organized into a multidimensional array representing the entire motion sequence. The converted multidimensional array can be saved in NPY or NPZ format using NumPy. In this process, you can choose to compress and store multiple action sequences to reduce the file size.

[0039] 3. General Relocalization Method for Humanoid Robots with Reference Motion Data

[0040] In the process of learning humanoid robot control, human motion data needs to be adapted to different robot platforms. This process involves converting normalized human motion data (usually stored in NPY / NPZ format) into a data format suitable for the specific robot structure and control system. The core of the general relocalization method is to adjust and map the human motion data according to the robot's physical structure (such as joints, links) and control requirements to ensure that the robot can effectively imitate and execute these movements. The following details the relocalization method for motion data converted from NPY / NPZ format to different humanoid robots.

[0041] 1. Detailed explanation of URDF file: URDF (Unified Robot Description Format) is an XML format file used to describe the structure of a robot. It defines the geometry, connection method, joint type, etc. of each part of the robot. The position, rotation axis, range limit, etc. of each joint. The joint positions and angles in the human motion data need to match these parameters defined in URDF.

[0042] 2. Joint mapping and conversion: Determine the number, type (such as rotational joints, translational joints) and motion axes of the humanoid robot joints. Based on this information, establish a one-to-one mapping relationship between human joints and robot joints. For example, if the human shoulder joint corresponds to the robot shoulder joint, the shoulder joint position and rotation information in the human motion data needs to be mapped to the motion parameters of the robot shoulder joint. In most cases, the number of joints and degrees of freedom of human motion data will have more redundancy than that of humanoid robots. Some degrees of freedom need to be deleted in a heuristic way to achieve complete alignment of the two data information and ensure that the data conforms to the joint motion range defined in the URDF. If it exceeds the range, the data can be adjusted using scaling or threshold limits to avoid compromising the authenticity and coherence of the motion.

[0043] 3. Local and global coordinate system conversion: Since the coordinate systems used by the human body and the robot may be different, coordinate system conversion is usually required. For example, the global coordinate system and local coordinate system of each joint of some humanoid robots may be different from the natural coordinate system of human body data. In this case, the motion data needs to be rotated and flipped to adapt to the robot coordinate system to ensure that the motion data can be correctly mapped to the robot.

[0044]

[0045] This formula is a mapping of 3D posture and action of a human model extracted from a monocular video. (t) is the video sequence, θ is the joint pose parameter of the human body posture model, K is the number of joint degrees of freedom, +1 is the global rotation, and β is the shape parameter of each rigid body structure block.

[0046]

[0047] This formula is the process of extracting and mapping the joint parameters of the human body to the joint parameters of the humanoid robot (retargeting). The parameters include the global three-dimensional rigid body position p of all rigid bodies in the robot. (t) , direction θ (t) , Linear speed v (t) and angular velocity ω (t) .

[0048] 4. Motion data optimization and correction: After completing the basic joint mapping, the motion data needs to be further corrected to ensure its consistency and physical feasibility. Due to the noise of the data acquisition equipment or the accuracy limitation of the posture estimation, the motion data mapped to the robot may have jitter, foot sliding, artifacts and other unreasonable data. Smoothing filtering technology should be used, such as low-pass filtering or Kalman filtering to reduce high-frequency noise and smooth the data curve to eliminate unnecessary jitter. In addition, by analyzing the outliers in the motion data (such as excessive angular velocity, angle change, etc.), artifacts can be detected and interpolated and corrected using adjacent normal data points. For example, for a joint angle that suddenly jumps, the smooth data segments before and after can be used to interpolate it to eliminate mutations.

[0049] 5. Write the converted motion data to the robot control system in a format supported by URDF. You can use URDF-related libraries (such as pybullet or ROS) to send motion data to the humanoid robot controller (usually using the sensor_msgs / JointState message type to describe the state of the joint) to issue real-time motion instructions so that the robot moves according to the predetermined motion trajectory. After verifying the validity of the data, the actual motion data is collected and a reference motion dataset is generated. The reference motion dataset can be used as training data for machine learning or reinforcement learning models to improve the robustness and adaptability of robot control. The dataset can also be used to verify the performance of new controls. By comparing the ideal motion trajectory and the actual motion trajectory in the Reference Motion dataset, the accuracy and stability of the controller can be evaluated.

[0050] 1) Training environment

[0051] Physical simulation environments simulate the physical laws of the real world, such as gravity, friction, collision detection, etc. In these environments, humanoid robots can perform various actions and get feedback based on the results of the actions. Common physical simulation environments include PyBullet, MuJoCo, Isaac Gym, etc. These environments support complex rigid body and soft body simulation and are suitable for training low-level strategies of humanoid robots. In order to make the trained strategies task-independent, it is necessary to design a variety of tasks in the training environment. For example, for different task scenarios such as walking, running, jumping, and obstacle avoidance, the robot needs to learn to perform corresponding actions under different conditions. During the training process, the parameters of the environment (such as terrain, friction coefficient, obstacle position, etc.) can be randomly changed to improve the generalization ability of the strategy and ensure that the robot can perform well in different environments.

[0052] 2) Deep reinforcement learning algorithm

[0053] In the present invention, a deep reinforcement learning algorithm based on Decoder Transformer is used to train the low-level strategy of the humanoid robot. Decoder Transformer is a powerful sequence model that can process time series data and learn complex temporal dependencies, and is very suitable for capturing the correlation between robot actions. When training low-level strategies, the Decoder Transformer model is used to predict the action of the robot joint at the next moment. Specifically, the input is the current state of the robot (such as joint position, speed, sensor data, etc.), and the current state of the robot is encoded into a vector sequence of fixed dimension. These vector sequences will be used as the input of the Transformer. The model captures the dependency between the current state and the past state through the self-attention mechanism, thereby determining the next action. Finally, the model outputs an action vector, which will be directly applied to the joint control of the robot.

[0054]

[0055] This formula is the task-independent underlying behavioral strategy of the humanoid robot, which enables it to always maintain stability, coordination, and anti-interference when imitating various (large-amplitude) human movements in the later stage. is the proprioception state input, is the target state input, for the positions of eight selected reference body positions (shoulder, elbow, hand, ankle); is the position difference between the reference joint and the humanoid joint; is the linear velocity of the reference motion joint, v, ω, a are joint position, joint velocity, root linear velocity, root angular velocity, and the previous action respectively.

[0056] 3) Reinforcement Learning Training Method

[0057] In order to obtain task-independent low-level strategies, the proximal policy optimization algorithm PPO (Proximal Policy Optimization) is used to implement strategy training. PPO is a widely used reinforcement learning algorithm that introduces a clipping term to limit the update amplitude, constrains the degree of strategy change, and uses the gradient ratio to ensure the effectiveness of the update. The algorithm can achieve higher learning efficiency and stability without sacrificing efficiency. In robot operation and control, the PPO algorithm can optimize the robot's action strategy to enable it to achieve precise gait control and operation, providing an effective method and technical support for solving the task execution of robots in complex environments. During the training process, the generalization ability of the strategy is improved through data enhancement technology. For example, different initial conditions and different target positions are used in the same task to expand the training data set so that the model can adapt to more variable environments. In a physical simulation environment, training can be accelerated by running multiple instances in parallel. Each instance can use different environmental parameters and initial conditions to improve the convergence speed of the strategy.

[0058] 4) Reward design

[0059] The design of the reward function needs to take into account multiple factors to guide the robot to learn a stable and universal motion strategy. Task-independent rewards for whole-body motion control strategies include balance rewards: the longer the robot maintains balance, the higher the reward it receives. This reward can help the robot maintain balance in various motion tasks; joint stability rewards: if the robot's joints change smoothly during movement and do not shake violently, an additional reward will be given; reference motion imitation rewards: the robot copies the reference motion sequence as much as possible within the controllable range, and the closer it can be to the reference motion while maintaining stability, the higher the reward it receives, so as to achieve the purpose of imitating human movements while ensuring stability; collision penalties: if the robot collides with obstacles or other objects in the environment during movement, a penalty will be given to encourage the robot to learn to avoid obstacles.

[0060] By using a deep reinforcement learning algorithm based on Decoder Transformer, task-independent low-level policies can be effectively trained, enabling humanoid robots to maintain stable motion control in different environments and tasks. The diversity of training environments, reasonable reward design, and scientific training methods are key factors for success. Ultimately, the trained policy model has good generalization capabilities and can cope with complex real-world application scenarios.

[0061] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A humanoid robot full body control method based on human motion acquisition, characterized in that: include: Step 1: Multimodal human motion data collection; Step 2: Normalize the format of the collected multimodal human motion data; Step 3: Relocalization method of humanoid robot with reference motion data.

2. A humanoid robot whole body control method based on human motion acquisition as claimed in claim 1, characterized in that: Multimodal human motion data collection includes: extracting human motion data from public human motion data sets, collecting human motion data from IMU inertial measurement units, and collecting human motion data from pure videos.

3. A humanoid robot whole body control method based on human motion acquisition as claimed in claim 1, characterized in that: The collected multimodal human motion data is formatted in normalized form, including using npy or npz format, and converting the collected FBX format to npy or npz, or BVH format.

4. A humanoid robot whole body control method based on human motion acquisition as claimed in claim 1, characterized in that: A method for relocalizing a humanoid robot with reference to motion data; specifically: Step 3.1: URDF file processing: URDF is an XML format file used to describe the structure of the robot. It defines information including the geometry, connection method, joint type, position, rotation axis, and range limit of each joint of the robot. The joint position and angle in the human motion data are matched with the parameters defined in the URDF file. Step 3.2: Joint mapping and conversion: determine the number and type of the humanoid robot joints, including rotational joints, translational joints, and motion axes; establish a one-to-one mapping relationship between human joints and robot joints according to the number and type of the humanoid robot joints, including rotational joints, translational joints, and motion axes; Step 3.3: Local and global coordinate system conversion: Since the coordinate systems used by the human body and the robot are different, the coordinate system conversion is required; Step 3.4: Motion data optimization and correction: After completing the basic joint mapping, the motion data needs to be further corrected to ensure its consistency and physical feasibility; Step 3.5: Write the converted motion data into the control system of the humanoid robot in a format supported by URDF.

5. A humanoid robot whole body control method based on human motion acquisition as claimed in claim 4, characterized in that: The number of joints and degrees of freedom of human motion data are more redundant than those of humanoid robots. Some degrees of freedom are deleted in a heuristic way to achieve complete alignment of the two data information and ensure that the data conforms to the joint motion range defined in URDF. If out of range, use scaling or thresholding to adjust the data to avoid compromising the realism and coherence of the motion.

6. A method for controlling the whole body of a humanoid robot based on human motion acquisition as claimed in claim 4, characterized in that: Apply smoothing filtering techniques, including low-pass filtering or Kalman filtering, to reduce high-frequency noise and smooth the data curve, thereby eliminating unnecessary jitter; in addition, artifacts are detected by analyzing the outliers in the human motion data and interpolated and corrected using adjacent normal data points.

7. A method for controlling the whole body of a humanoid robot based on human motion acquisition as claimed in claim 4, characterized in that: Use the URDF library to send motion data to the control system of the humanoid robot, issue real-time motion instructions, and make the robot move according to the predetermined motion trajectory; After verifying the validity of the data, the actual motion data is collected and a reference motion dataset is generated. The reference motion dataset is used as training data for machine learning or reinforcement learning models to improve the robustness and adaptability of humanoid robot control. The reference motion dataset is also used to verify the performance of new controls. By comparing the ideal motion trajectory and the actual motion trajectory in the reference motion dataset, the accuracy and stability of the humanoid robot control system are evaluated.

Images