Robot data acquisition and training method based on exoskeleton and data glove

Abstract

The invention discloses a robot data acquisition and training method based on an exoskeleton and a data glove, and the method comprises the following steps: firstly, collecting human upper limb and hand motion data through the data glove and the exoskeleton at the same time; secondly, performing time alignment, coordinate conversion and Kalman filtering processing on the acquired data, and unifying the acquired data into a high-precision state vector; and finally, training a Transform model by using the state sequence sample to predict a human action trend so as to realize simulation learning of actions demonstrated by experts. According to the method, multi-modal sensor data is combined, high-precision acquisition and modeling of human complex actions are achieved, and the method has good generalization ability and practicability.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of robot control and data acquisition, in particular to a robot data acquisition and training method based on exoskeleton and data glove. BACKGROUND

[0002] With the development of deep reinforcement learning, the ability of robots in operation, grasping and collaboration tasks has gradually improved. However, direct use of reinforcement learning for policy learning usually faces problems such as long training time, low exploration efficiency, and easy to fall into failure strategy in the early stage. At the same time, human operation experience contains highly abstracted strategy priori, which can be used to guide the initialization of robot strategy.

[0003] Existing methods mostly use visual imitation, action demonstration and other forms for human demonstration, but due to the structural difference between man and machine, the imitation accuracy and universality are insufficient. In addition, a large amount of data relies on actual robot execution for collection, which is costly and risky.

[0004] Therefore, a method is needed to extract a universal strategy representation from human actions and use it in the pre-training process of robot strategy to improve the performance and learning efficiency of strategy initialization. SUMMARY

[0005] The present application aims to provide a robot data acquisition and pre-training method based on exoskeleton and data glove, which uses exoskeleton and data glove to collect multi-granularity robot demonstration data, trains a strategy function through imitation learning, and improves the speed and generalization ability of strategy learning.

[0006] The technical solution adopted by the present application to achieve the above-mentioned purpose is:

[0007] A robot data acquisition and training method based on exoskeleton and data glove, comprising the following steps:

[0008] 1) Collecting action data of human upper limbs and hands through data glove and exoskeleton at the same time;

[0009] 2) sequentially performing time alignment, coordinate conversion and Kalman filtering on the action data, and splicing all the processed data into a unified state vector;

[0010] 3) constructing the state vector into a state sequence sample, constructing an imitation learning model based on Transformer, and using the model to predict the state sequence sample to obtain the trend of human action.

[0011] The step 1) comprises the following steps:

[0012] 1.1) Collect the shoulder, elbow, wrist large joint angle, velocity and end position through the IMU inertial measurement unit in the exoskeleton as the human upper limb large arm data x arm :

[0013] x arm =[θ1,…,θ i ,ω1,…,ω i ,q1,…,q i ,x,y,z] T

[0014] Wherein θ i represents the angle of the i th joint, ω i represents the angular velocity of the i th joint, q i represents the quaternion of the i th joint, x, y, z represents the position coordinates of the human upper limb end in three-dimensional space rectangular coordinate system;

[0015] 1.2) Collect the finger angle and palm posture through the angle sensor and IMU inertial measurement unit in the data glove as the human hand data x glove :

[0016]

[0017] Wherein, represents the angle of the k th joint of the j th finger, j = 1 ~ 5, k = 1 ~ 3, x p , y p , z p represents the position of the palm center, q p represents the quaternion of the palm.

[0018] The step 2) comprises the following steps:

[0019] 2.1) Time align them. Use interpolation method to complete the missing frames of x arm and x glove , and the completed data is aligned to the unified time line;

[0020] 2.2) Map x arm and x glove to the unified spatial coordinate system, and establish the spatial relationship between the arm end and the palm;

[0021] x end =x p +Δx

[0022] Wherein x end represents the spatial position of the arm end, x p represents the spatial position of the palm, and Δx represents the displacement vector connecting the palm to the arm end;

[0023] 2.3) mapping the x arm and x glove perform Kalman filtering processing;

[0024] 2.4) splice the processed x arm and x glove into a unified vector s t :

[0025]

[0026] wherein, represents the angle of the i-th joint after mapping, represents the angular velocity of the i-th joint after mapping, represents the quaternion of the i-th joint after mapping, represents the position coordinates of the end of the upper limb of the human body in the three-dimensional space rectangular coordinate system after mapping, represents the angle of the k-th joint of the j-th finger after mapping, represents the position of the palm center after mapping, represents the quaternion of the palm after mapping.

[0027] The step 3) comprises the following steps:

[0028] 3.1) using a behavior cloning strategy, combining a Transformer model to construct a mapping function from state to action trend, as a model of imitation learning;

[0029] 3.2) by a time window mechanism, each input sample is expanded into a state sequence, and through the imitation learning model, the value of the state s t and all predicted actions .

[0030] The step 3.1) is specifically:

[0031] define the target action a t as the difference between the next state s t+1 and the current state s t , that is:

[0032] a t =s t+1 -s t

[0033] save each (s t , a t ) pair at each time step to form a training data set D:

[0034] D={(s1,a1),(s2,a2),…,(s N ,aN )

[0035] Through the supervised training strategy network π θ , the state to action mapping learning is completed, that is:

[0036] π θ (s t )≈a t =s t+1 -s t .

[0037] The step 3.2) is specifically:

[0038] Each input sample is expanded into a state sequence X t-T+1:t using a time window mechanism:

[0039] X t-T+1:t =(s t-T+1 ,s t-T+2 ,…,s t )

[0040] If there are not enough T frames at the initial time, the sliding time window is traversed through the entire demonstration trajectory using static state repetition or zero motion padding to generate training samples;

[0041] The state sequence is input into π θ , and the predicted action trend is obtained

[0042]

[0043] The mean square error (MSE) loss is used as the supervised objective function, and π θ is trained by minimizing the MSE loss:

[0044]

[0045] Wherein, N is the total number of training samples.

[0046] The present application has the following beneficial effects and advantages:

[0047] 1. The present application cooperates with the exoskeleton and the data glove to completely cover the high degree of freedom motion of the upper limbs to the hands, combines the IMU, position coding and hand joint angle information, realizes high-precision and time-consistent data acquisition, and improves the signal-to-noise ratio and motion restoration degree through Kalman filtering and spatial fusion.

[0048] 2. The present application uses the Transformer model to learn the high-order time sequence mapping relationship of human state to action trend, which can accurately capture the behavior intention of human in continuous state compared with the traditional single-frame state supervised model. BRIEF DESCRIPTION OF DRAWINGS

[0049] Figure 1 is a flow chart of the method of the present application;

[0050] Figure 2 is a flow chart of the data processing process of the present application;

[0051] Figure 3 is a flow chart of the prediction of robot action of the present application;

[0052] Figure 4 is a schematic diagram of the wearing of the equipment of the present application. DETAILED DESCRIPTION

[0053] The present application will be further described in detail below in combination with the drawings and embodiments.

[0054] The present application uses an exoskeleton to collect information such as shoulder, elbow, wrist, etc. large joint angles, speeds and end positions of upper limbs, etc. to realize data collection of large limbs and large joints. At the same time, a data glove is used to collect hand data, collect palm posture information and the bending angle of each finger. The collected data is time-aligned, the coordinate system is converted and processed through Kalman filtering. The processed data is trained by imitation learning to form a pre-trained model, to obtain a predicted action and improve the generalization performance of the model.

[0055] As shown in Figure 1 , a robot data collection and training method based on an exoskeleton and a data glove, comprising the following steps:

[0056] 1) simultaneously acquiring human upper limb and hand action data using a data glove and an exoskeleton;

[0057] 2) time-aligning, converting the coordinate system and processing the collected data through Kalman filtering, and splicing all the sensor data after processing into a unified vector;

[0058] 3) constructing a state sequence sample as an input of an imitation learning model, and training a Transformer model to predict human action trends.

[0059] The step 1) comprises the following steps:

[0060] 1.1) using an exoskeleton to collect data of the upper limb of the human body, using an IMU inertial measurement unit to collect angles, speeds and end positions of large joints such as shoulder, elbow, wrist, etc.

[0061] X arm =[θ1,…,θ i ,ω1,…,ω i ,q1,…,q i ,x,y,z] T

[0062] Where, θ i ω represents the angle of the i-th joint. i q represents the angular velocity of the i-th joint. i A quaternion representing each joint;

[0063] 1.2) Data is collected from the human hand using a data glove, with angle sensors and an IMU (Inertial Measurement Unit) collecting finger angles and palm postures.

[0064]

[0065] in, The angle of the k-th joint of the j-th finger, x p ,y p ,z p q represents the center of the palm. p A quaternion representing the palm of a hand.

[0066] Figure 4 The exoskeleton used in this study employs the Xsens MVN Awinda system, which utilizes wireless MTw inertial sensors deployed at key nodes of the upper limbs. These sensors are secured using specialized clothing and straps, enabling precise acquisition of the angles, postures, and spatial positions of the shoulder, elbow, and wrist joints. arm =[θ1,…,θ i ,ω1,…,ω i ,q1,…,q i [x,y,z] T The system supports a data sampling rate of 240 frames per second and outputs standard skeletal topology data, facilitating subsequent data fusion and imitation learning input.

[0067] Figure 4 The data acquisition module in this system uses the Manus Prime II data glove, which acquires the bending degree of each finger and the palm posture through fiber optic bending sensors and an IMU. The output format includes joint angle vectors and palm quaternions. Hand and upper limb data are fused at the main control end via a synchronized clock, and the data accuracy is uniformly optimized using a Kalman filter algorithm to construct a joint imitation learning dataset.

[0068] Data processing process such as Figure 2 As shown:

[0069] Because the data sampling frequencies are different, they cannot be directly spliced ​​together and must be time-aligned. Interpolation is used to fill in missing frames, aligning the collected data to a unified timeline to ensure data matches every action frame.

[0070] The exoskeleton and the glove are in different body coordinate systems, data of the hand and the upper limb are mapped into a unified spatial coordinate system, and a spatial relationship between the arm end and the palm is established;

[0071] x end =x p +Δx

[0072] Wherein, x end is a spatial position of the arm end, and x p is a spatial position of the palm.

[0073] The collected data information is subjected to Kalman filtering, data acquired through the exoskeleton and the IMU sensor are optimized, noise is eliminated, and more accurate state estimation is obtained;

[0074] All the processed sensor data are spliced into a unified vector s t for inputting into a strategy model:

[0075]

[0076] A flow of predicting robot action is shown in Figure 3 , the application uses a behavior cloning strategy, combines a Transformer model to construct a mapping function from state to action trend, so as to realize high-precision imitation of expert action.

[0077] The target action a t is a difference between a next time state s t+1 and a current state s t :

[0078] a t =s t+1 -s t

[0079] A pair of (s t , a t ) at each time step is saved to form a training data set:

[0080] D={(s1,a1),(s2,a2),…,(s N , a N )}

[0081] A strategy network pi θ is used for supervised training, and mapping learning from state to action is completed.

[0082] pi θ (s t )≈a t =s t+1 -s t

[0083] Considering the significant time continuity and context dependence of human motion, single-frame state is difficult to accurately predict motion. Therefore, this method introduces a time window mechanism to expand each input sample into a state sequence

[0084] X t-T+1:t =(s t-T+1 ,s t-T+2 ,…,s t )

[0085] If there are less than T frames at the initial time, the static state can be repeated or filled with zero motion. The sliding time window traverses the entire demonstration trajectory, generating a large number of training samples.

[0086] The state sequence is input into the Transformer network π θ , and the predicted motion trend is output:

[0087]

[0088] The mean square error loss is used as the supervised objective function:

[0089]

[0090] N is the total number of training samples, a t is the true state difference of the t-th sample in the demonstration data, and is the model prediction. By minimizing the MSE loss, the model learns to output a prediction that is as close as possible to the human expert motion change given the state.

[0091] A robot imitation learning training system, comprising:

[0092] An exoskeleton acquisition module for acquiring the angles, angular velocities and end positions of upper limb joints;

[0093] A data glove acquisition module for acquiring finger angles and palm poses;

[0094] A data processing module for time alignment, coordinate conversion and Kalman filtering of the collected data, and outputting a unified state vector;

[0095] A model training module for constructing a state sequence sample based on the state vector and training a Transformer model to predict human motion trends.

Claims

1. A robot data collection and training method based on exoskeleton and data glove, characterized in that, The method comprises the following steps: 1) simultaneously collecting motion data of human upper limbs and hands through a data glove and an exoskeleton; 2) sequentially performing time alignment, coordinate conversion and Kalman filtering on the motion data, and splicing all the processed data into a unified state vector; 3) constructing the state vector into a state sequence sample, constructing a Transformer-based imitation learning model, and using the model to predict the state sequence sample to obtain a human motion trend.

2. The robot data collection and training method based on exoskeleton and data glove according to claim 1, characterized in that, The step 1) comprises the following steps: 1.1) Collect the shoulder, elbow, wrist, large joint angle, velocity and end position data of the upper limb of the human body as the upper limb data x of the human body through the IMU inertial measurement unit in the exoskeleton arm : x arm = [θ1,..., θ i , ω1,..., ω i , q1,..., q i , x, y, z] T where θ i represents the angle of the i-th joint, ω i represents the angular velocity of the i-th joint, q i represents the quaternion of the i-th joint, and x, y, z represent the position coordinates of the end of the upper limb of the human body in a three-dimensional space rectangular coordinate system. 1.2) Collect finger angles and palm pose by angle sensors, IMU inertial measurement unit in data glove, as human hand data x glove : wherein, denotes the k-th joint angle of the j-th finger, j = 1 ~ 5, k = 1 ~ 3, x p ,y p ,z p denotes the position of the palm center, q p denotes the quaternion of the palm.

3. The robot data collection and training method based on exoskeleton and data glove according to claim 1, characterized in that, The step 2) comprises the following steps: 2.1) Align their time. Interpolate missing frames of x arm and x glove and align the completed data to a uniform timeline; 2.2) x arm and x glove mapping into a unified spatial coordinate system, establishing the spatial relationship between the end of the arm and the palm; x end = x p + Δx where x end represents the spatial position of the hand tip, x p represents the spatial position of the palm, and Δx represents the displacement vector connecting the palm to the hand tip; 2.3) on the mapped x arm and x glove Kalman filter processing is performed; 2.4) Concatenate the processed x arm and x glove into a unified vector s t : wherein, represents the angle of the i-th joint after mapping, represents the angular velocity of the i-th joint after mapping, represents the quaternion of the i-th joint after mapping, represents the position coordinates of the end of the upper limb of the human body in a three-dimensional space rectangular coordinate system after mapping, represents the angle of the k-th joint of the j-th finger after mapping, represents the position of the center of the palm after mapping, represents the quaternion of the palm after mapping.

4. The robot data collection and training method based on exoskeleton and data glove according to claim 1, characterized in that, The step 3) The method comprises the following steps: 3.1) using a behavior cloning strategy to construct a mapping function from a state to a motion trend in combination with a Transformer model as an imitation learning model; 3.2) Extend each input sample into a state sequence by time window mechanism, and get the state s by imitation learning model t The values of all predicted actions below.

5. The robot data collection and training method based on exoskeleton and data glove according to claim 4, characterized in that, The step 3.1) is specifically as follows: Define target action a t For next time state s t+1 Difference from current state s t That is: a t = s t+1 -s t Save (s t ,a t ) pairs at each time step, compose training dataset D: D = {(s1, a1), (s2, a2),..., (s N , a N )} By supervised training policy network π θ , complete state to action mapping learning, namely: π θ (s t )≈a t =s t+1 -s t 。 6. The robot data collection and training method based on exoskeleton and data glove according to claim 4, characterized in that, The step 3.2) is specifically as follows: Each input sample is extended into a state sequence X using a time window mechanism t-T+1:t : X t-T+1:t = (s t-T+1 ,s t-T+2 ,…,s t ) If there are less than T frames at the initial moment, a stationary state is repeated or filled with zero motion, a sliding time window is traversed through the entire demonstration trajectory, and a training sample is generated; inputting a state sequence π θ , obtaining a predicted action trend Using mean square error (MSE) loss as the supervised objective function, the MSE loss is minimized with respect to π θ training: wherein N is the total number of training samples.

Images