Low-cost robot imitation learning method and system based on human video

Abstract

The invention discloses a low-cost robot imitation learning method and system based on a human video. The method comprises the following steps: S1, data acquisition; s2, data extraction and physical alignment are carried out to eliminate man-machine physical differences; the step is divided into two parallel processing modules of action space alignment and visual space alignment; s3, data set construction: mixing the aligned human data with real robot teleoperation data, carrying out balanced sampling, and constructing a mixed data set Dmix; and S4, cooperative training: constructing a strategy network based on diffusion Transform for training. According to the method, data can be acquired only through the monocular RGB camera, expensive robot teleoperation data are replaced with cheap and easily available human videos, and the data acquisition threshold is greatly reduced. Through a visual alignment strategy of random color grid rendering, a network can learn neglect skin color textures and pay attention to geometric structures without a complex generative model, so that the robot can be seamlessly migrated to robots in different forms.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of robot control and artificial intelligence, and in particular to a low-cost robot imitation learning method and system based on human video. BACKGROUND

[0002] Currently, the main ways to obtain robot training data include:

[0003] 1. Teleoperation: Using exoskeletons, VR handles, or master-slave robotic arms, human experts control the robot for operation, and directly record the joint state and image data of the robot.

[0004] 2. Sim-to-Real: Train the strategy in a simulation environment, and then migrate to the real machine.

[0005] 3. Video-based Retargeting: Use computer vision algorithms to extract hand movements from human videos, and map them to robot action space through inverse kinematics (IK).

[0006] The existing technology has the following significant shortcomings:

[0007] 1. High cost and low efficiency of data collection: Teleoperation relies on expensive professional hardware (such as VR devices, teleoperation arms), and the collection process is cumbersome, making it difficult to scale up.

[0008] 2. There is a serious appearance difference: Human hands and robot grippers are completely different in texture and shape. Models trained directly using human videos often overfit human skin features, resulting in recognition failure on the robot real machine. Existing solutions often rely on complex generative models (such as diffusion models) for image style transfer, with huge computational overhead.

[0009] 3. There is a serious action space difference: The kinematic structure of human arms and mechanical arms is different. Existing retargeting methods usually take the wrist or fingertip center as the anchor point, but in complex operations, these anchor points are unstable (for example, the wrist is too far from the interaction point, and the fingertip changes with the grasping motion), resulting in unnatural robot actions and even collisions.

[0010] The present application aims to solve the problems of high cost of data collection, large human-machine appearance difference leading to poor model generalization, and inaccurate action mapping in existing robot imitation learning. Through a low-cost framework, the present application directly uses ordinary RGB cameras to collect human videos, and through innovative physical alignment (including action and vision) and collaborative training strategies, it achieves efficient, low-cost, and high-precision robot operation skill learning.

[0011] The information disclosed in this Background section is only for the purpose of enhancing the understanding of the general background of the application, and should not be taken as admitting that the information forms the prior art that is already known to those skilled in the art. SUMMARY

[0012] The purpose of the present application is to provide a low-cost robot imitation learning method and system based on human video, aiming to solve the problems of high data collection cost, poor model generalization caused by large appearance difference between man and machine, and inaccurate action mapping in existing robot imitation learning.

[0013] In order to achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0014] A low-cost robot imitation learning method based on human video, comprising the following steps:

[0015] S1, data collection, specifically including: human operation video collection: using a consumer-grade RGB camera to shoot a video of human hand operating an object; robot teleoperation data collection: collecting a small amount of real robot teleoperation data, images and corresponding joint actions;

[0016] S2, data extraction and physical alignment, used to eliminate the physical difference between man and machine; this step is divided into two parallel processing modules: action space alignment and visual space alignment;

[0017] S3, data set construction: mixing the aligned human data and real robot teleoperation data, and performing balanced sampling to construct a mixed data set ;

[0018] S4, cooperative training: constructing a strategy network based on diffusion Transformer for training.

[0019] Further, the action space alignment is used to convert the human hand trajectory into the pose of the robot end effector, and the specific implementation steps are as follows:

[0020] 1) 3D key point extraction: using a hand pose estimation model to extract 3D hand key points from each frame of the video;

[0021] 2) Determine the position anchor point: select the center of the large thenar as the anchor point for reorientation; the specific algorithm is: calculate the midpoint of the proximal interphalangeal joint of the thumb and the metacarpophalangeal joint of the index finger; this point is the most stable relative to the palm center during the grabbing process;

[0022] 3) Construct a 3D coordinate system and calculate the pose: select five points including the four joints of the index finger and the PIP joint of the thumb to fit a plane, determine the Z axis through the normal vector of the plane, and determine the X axis through the direction of the index finger, and then calculate the rotation matrix and quaternion;

[0023] 4) Compute gripper state: Compute the Euclidean distance between the thumb tip and the index tip, and normalize it to the opening degree of the robot gripper:

[0024] ;

[0025] wherein, and correspond to the distance threshold for fully closed and fully open, respectively;

[0026] 5) Coordinate system conversion: Convert the pose in the camera coordinate system to the robot base coordinate system through the hand-eye calibration matrix.

[0027] Further, the visual space alignment is used to eliminate the visual appearance difference between the human hand and the mechanical gripper, and the specific implementation steps are as follows:

[0028] 1) Hand mesh extraction: Obtain the 3D mesh model of the hand in the video frame;

[0029] 2) Random color parameter generation: Randomly generate RGB color parameters;

[0030] 3) Mesh color rendering: Render the extracted hand mesh into the original image with random colors, covering the original human hand;

[0031] 4) Enhanced view output: Output the image after "visual enhancement".

[0032] Further, the specific implementation process of step S4 is as follows:

[0033] 1) Input alignment and feature extraction: Human data and robot data share the same visual encoder and language encoder;

[0034] 2) Embodied independent coding: Design an independent action encoder to process human mapped actions and robot real actions respectively;

[0035] 3) Shared policy learning: The intermediate layer uses a shared DiT module to learn the general operation logic;

[0036] 4) Embodied independent decoding: Use an independent action decoder to output the predicted action;

[0037] 5) Model optimization: Calculate the loss function between the predicted action and the real / mapped action, and update the network parameters.

[0038] ​​The system of a low-cost robot imitation learning method based on human video comprises an RGB camera acquisition module, and a data processing module and a control module for realizing the low-cost robot imitation learning method based on human video.

[0039] By adopting the technical scheme, the application has the following beneficial effects:

[0040] 1) Low cost and high expansibility: only a monocular RGB camera is needed to acquire data, and cheap and easily available human video is used to replace expensive robot remote operation data, thereby greatly reducing the data acquisition threshold.

[0041] 2) Improve cross-possessing generalization ability (solve appearance gap): through the visual alignment strategy of "random color grid rendering", the network can learn to ignore skin color texture and focus on geometric structure without complex generative models, so as to be seamlessly migrated to different robots.

[0042] 3) Accurate and stable motion mapping (solve motion gap): the anchor point selection method based on "large thenar center" is proposed, which is more stable than the traditional wrist or fingertip midpoint when the hand grasps the object and deforms, and the generated robot trajectory is smoother and more useful.

[0043] 4) High data utilization rate: the cooperative training strategy combines the independent action head (Heads) design, which not only learns high-level semantics from human data, but also retains the bottom-level control characteristics learned from robot data, and can achieve high success rate with a small amount of robot data. BRIEF DESCRIPTION OF DRAWINGS

[0044] In order to more clearly illustrate the technical scheme in the specific embodiments or prior art of the application, the drawings needed in the description of the specific embodiments or prior art will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0045] Figure 1 The overall flowchart of the low-cost robot imitation learning method based on human video provided for the embodiments of the application is shown in the figure.

[0046] Figure 2 The flowchart of motion alignment and visual alignment provided for the embodiments of the application is shown in the figure.

[0047] Figure 3 The training flowchart provided for the embodiments of the application is shown in the figure. DETAILED DESCRIPTION

[0048] The technical solutions of the present application will be described clearly and completely in conjunction with the drawings. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0049] The specific embodiments of the present application will be described in detail below in conjunction with the drawings. It should be understood that the specific embodiments described here are only used to illustrate and explain the present application, and are not used to limit the present application.

[0050] Before a detailed introduction of the present application, the professional terms involved in the present application are first explained as follows:

[0051] VLA (Vision-Language-Action Model): Vision-Language-Action Model, a multi-modal large model capable of understanding images, text instructions and outputting robot control actions at the same time.

[0052] DiT (Diffusion Transformer): a network structure combining the generation capability of diffusion model and the advantages of Transformer architecture, commonly used for generating high-quality action sequences.

[0053] Retargeting (re-targeting): in computer graphics and robotics, it refers to the process of transferring the motion of one skeleton (such as human hand) to another different skeleton (such as mechanical hand).

[0054] End-effector (end-effector): the tool at the end of the robot arm, such as gripper, suction cup, etc., the component directly interacting with the environment.

[0055] PIP / MCP: hand anatomy term, PIP refers to the proximal interphalangeal joint, and MCP refers to the metacarpophalangeal joint.

[0056] As shown in Figures 1-3 The present application provides a low-cost robot imitation learning method based on human video, which includes the following steps:

[0057] Step S1: data acquisition (in Figure 1 102, 103);

[0058] Human operation video acquisition: use a consumer-grade RGB camera to shoot a video of human hand operating an object.

[0059] Robot teleoperation data acquisition: collect a small amount of real robot teleoperation data (images and corresponding joint actions).

[0060] Step S2: Data extraction and physical alignment Figure 1 Step S3: Coordinate transformation Figure 2 This is the core processing link of the application, aiming to eliminate the physical differences between man and machine. This step is divided into two parallel processing modules: action space alignment and visual space alignment.

[0061] Action Alignment (AA): Convert human hand trajectory to robot end-effector pose.

[0062] 3D key point extraction: Use a hand pose estimation model (such as HaMeR) to extract 3D hand key points (21 joint coordinates) from each frame of video.

[0063] Determine the position anchor point Figure 2 Step S3: Coordinate transformation

[0064] Construct a 3D coordinate system and calculate the pose Figure 2 Step S3: Coordinate transformation

[0065] Calculate the gripper state Figure 2 Step S3: Coordinate transformation , :

[0066] ;

[0067] where and correspond to the distance thresholds for complete closure and complete opening, respectively.

[0068] Coordinate transformation Figure 2 Step S3: Coordinate transformation

[0069] Visual Alignment (VA): Eliminate the visual appearance differences between human hand and mechanical claw.

[0070] Hand mesh extraction: Extract 3D mesh model of hand in video frame.

[0071] Random color parameter generation Figure 2 Step 207: Randomly generate RGB color parameters.

[0072] Mesh color rendering Figure 2 Step 208: Render the extracted hand mesh into the original image with random colors, covering the original human hand (especially the skin texture).

[0073] Augmented view output Figure 2 Step 209: Output the "visually augmented" image. This method forces the model to ignore surface features such as skin color and texture, and instead learn the geometric structure of the hand and object interaction logic, thus adapting to non-human appearances (i.e. robotic arms).

[0074] Step S3: Dataset construction Figure 1 Step 105: Mix and balance sample the aligned human data (augmented images + mapped robot actions) with real robot teleoperation data to construct a mixed dataset .

[0075] Step S4: Collaborative training Figure 1 Step 105, detailed architecture see Figure 3 ) Construct a strategy network based on diffusion Transformer (DiT) for training.

[0076] Input alignment and feature extraction Figure 3 Steps 303, 304: Human data and robot data share the same vision encoder (Vision Encoder) and language encoder.

[0077] Embodiment-specific encoding Figure 3 Step 305: Design an independent action encoder (Action Encoder): process human mapped actions and real robot actions respectively. Because even after alignment, there may be slight differences in the numerical distribution of the two, independent encoding heads can prevent gradient conflicts.

[0078] Shared strategy learning Figure 3 Step 306: Use shared DiT modules in the middle layer to learn general operation logic.

[0079] Embodiment-specific decoding Figure 3 Step 307: Use independent action decoder to output predicted actions.

[0080] Model optimization (s) Figure 3 308): Calculate the loss function between the predicted action and the real / mapped action, update the network parameters.

[0081] Preferably, regarding action alignment: anchor point selection can also use "palm geometric center" or "middle finger root joint", but the stability is slightly worse than the thenar center of the present solution. Pose calculation can use SVD (singular value decomposition) instead of plane fitting.

[0082] Regarding visual alignment: in addition to random color rendering, "random texture mapping" or "wireframe mode (Wireframe) rendering" can also be used, or the hand area can be completely masked out, but completely masking out will lose hand geometry information, and the effect is not as good as the present solution.

[0083] Regarding network architecture: the action head can not be independent, that is, human data and robot data are forced to share the same action head, but this requires more stringent normalization of action data, and may result in performance degradation.

[0084] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A low-cost robot imitation learning method based on human videos, characterized in that, Includes the following steps: S1. Data acquisition, specifically including: human operation video acquisition: using a consumer-grade RGB camera to capture videos of human hands operating objects; robot teleoperation data acquisition: acquiring a small amount of real robot teleoperation data, images and corresponding joint movements; S2. Data extraction and physical alignment are used to eliminate physical differences between humans and machines; this step is divided into two parallel processing modules: motion space alignment and visual space alignment. S3. Dataset Construction: Aligned human data is mixed with real robot teleoperation data, and balanced sampling is performed to construct a hybrid dataset. ; S4. Collaborative Training: Construct a policy network based on Diffusion Transformer for training.

2. The low-cost robot imitation learning method based on human videos according to claim 1, characterized in that, The motion space alignment is used to convert the human hand trajectory into the pose of the robot's end effector, and its specific implementation steps are as follows: 1) 3D key point extraction: Extract 3D hand key points from each frame of the video using a hand pose estimation model; 2) Determine the anchor point: Select the center of the thenar eminence as the anchor point for redirection; Specific algorithm: Calculate the midpoint between the proximal interphalangeal joint of the thumb and the metacarpophalangeal joint of the index finger; this point is the most stable relative to the center of mass of the palm during the grasping process; 3) Construct a 3D coordinate system and calculate the pose: Select five points, namely the four joints of the index finger and the PIP joint of the thumb, to fit a plane. Determine the Z-axis through the normal vector of the plane and the X-axis through the direction of the index finger. Then calculate the rotation matrix and quaternion. 4) Calculate the gripper state: Calculate the Euclidean distance between the tips of the thumb and index finger. This is normalized to the degree of opening and closing of the robot gripper. : ； in, and These correspond to the distance thresholds for fully closed and fully open positions, respectively. 5) Coordinate system transformation: The pose in the camera coordinate system is transformed to the robot base coordinate system through the hand-eye calibration matrix.

3. The low-cost robot imitation learning method based on human videos according to claim 1, characterized in that, The visual spatial alignment is used to eliminate the visual appearance difference between the human hand and the robotic gripper, and its specific implementation steps are as follows: 1) Hand mesh extraction: Obtain a 3D mesh model of the hand from the video frame; 2) Random color parameter generation: Randomly generate RGB color parameters; 3) Mesh Shading Rendering: The extracted hand mesh is rendered back into the original image with random colors, covering the original hand; 4) Enhanced View Output: Outputs an image that has been "visually enhanced".

4. The low-cost robot imitation learning method based on human videos according to claim 1, characterized in that, The specific implementation process of step S4 is as follows: 1) Input alignment and feature extraction: Human and robot data share the same visual encoder and language encoder; 2) Embodied Independent Encoding: Design independent motion encoders to process human-mapped motions and robot-real motions separately; 3) Shared strategy learning: The middle layer uses a shared DiT module to learn common operational logic; 4) Embodied Independent Decoding: Uses an independent action decoder to output the predicted action; 5) Model optimization: Calculate the loss function between the predicted action and the actual / mapped action, and update the network parameters.

5. A system employing the low-cost robot imitation learning method based on human videos as described in any one of claims 1-4, characterized in that, It includes an RGB camera acquisition module, as well as a data processing module and a control module for implementing the methods of claims 1-4.

Images