A force feedback exoskeleton and humanoid robot imitation learning method

Abstract

The present invention discloses a force feedback exoskeleton and a humanoid robot imitation learning method. The force feedback exoskeleton includes a back-carrying part, an arm exoskeleton part and a data glove part. There are two arm exoskeleton parts, and the two arm exoskeleton parts are symmetrically connected to the back-carrying part. There are two data glove parts to respectively collect the position and posture information of the left and right hands. The humanoid robot imitation learning method is based on the same dual-arm features and operator perspective image input, combined with the noded topological correlation organization of human joints, integrates multimodal data information, and outputs control quantities that are almost the same as expert operations. The present invention is applied to the field of exoskeleton technology, and can achieve high-quality data collection while making the exoskeleton lightweight and integrated, greatly broadening the number of degrees of freedom of data collection of traditional exoskeletons, and can be used to guide the coordinated movement of dual robotic arms, and then effectively applied to the dexterous operation tasks of humanoid robots.

Description

Technical Field

[0001] The present invention relates to the technical field of exoskeletons, in particular to a force feedback exoskeleton and a humanoid robot imitation learning method. Background Art

[0002] Imitation is an important mechanism for human learning. By watching others demonstrate, humans can learn the corresponding task operations. Similarly, the imitation learning of the robot arm enables it to learn complex and diverse task actions through human operation data. This method avoids the tedious manual programming process and enables the robot arm to have the ability to learn new behaviors, enhancing its generalization and scalability, and improving the efficiency of the robot arm's learning of new behaviors.

[0003] Most existing robotic arm teleoperators are based on single-arm operation, which only controls one robotic arm at a time to complete a specific task. Such teleoperators cannot simultaneously control both arms of a humanoid robot to complete a task. Compared with dual-arm operation, single-arm operation has a simpler system structure and lower implementation cost. However, most tasks in life require the coordination of both hands, so the tasks that can be accomplished by a single arm are limited and cannot complete complex tasks.

[0004] Most of the few existing dual-arm teleoperation devices are unable to simultaneously collect multimodal data for imitation learning. At present, dual-arm teleoperation devices mainly include VR handles, motion capture systems (visual motion capture, inertial motion capture) and encoder exoskeletons. Although these operation methods can control the movement of both arms at the same time, since the teleoperation system itself does not have active force-applying devices such as motors, the master end cannot sense the force state of the slave end manipulator. Since the joint resistance cannot be sensed, it is difficult for the operator to accurately sense whether the manipulator has collided and the magnitude of the collision, so it is impossible to perform tasks in complex environments with limited space. In addition, most of these teleoperation devices only obtain single-mode operation data such as the spatial position of the end or the angles of each joint, and the effect of using these data for imitation learning training is limited.

[0005] At present, there are two main devices that have both multi-modal operation data collection functions and can perform dual-arm teleoperation, namely, isomorphic robotic arms similar to mobile-aloha and upper limb exoskeletons with motors and force sensors. However, due to the different configurations of mobile-aloha robotic arms and human arms, it is difficult to directly apply them to dual-arm teleoperation and data collection of humanoid robots. The upper limb exoskeleton with motors and force sensors has a high degree of adaptability to human joints, can transmit force feedback information, and can collect multi-modal information such as encoders, torque, and control current. It can be used as an ideal humanoid robot imitation learning training data collection device. However, this exoskeleton has a more complex electrical system, a large number of exoskeleton subsystem equipment models and quantities, a large overall weight, is difficult to carry, has poor flexibility, and is difficult to deploy. At present, there is no carryable force feedback upper limb dual-arm exoskeleton system at home and abroad.

[0006] In terms of data collection, most of the current dexterous manipulation tasks use the visual system as input to control the robot arm to complete specific tasks. Although the visual system can provide unparalleled information richness, it cannot fully adapt to all scenarios (such as complex backgrounds, transparent objects, occlusion and darkness, irrelevant feature interference, etc.), and it is difficult to adapt to all manipulation tasks, especially for multi-contact manipulation tasks. In addition, the current imitation learning method lacks the collection and use of force interaction information, and does not take into account the motion logic and topological correlation between the joints of the human body. The actual output can only control specific actuators and lacks generalization. Summary of the invention

[0007] In view of the deficiencies in the above-mentioned prior art, the present invention provides a force feedback exoskeleton and a humanoid robot imitation learning method, which can achieve high-quality data acquisition while making the exoskeleton lightweight and integrated, greatly broadening the number of degrees of freedom for data acquisition in traditional exoskeletons, and can be used to guide the coordinated movement of dual robotic arms, and then effectively applied to the dexterous operation tasks of humanoid robots.

[0008] To achieve the above object, the present invention provides a force feedback exoskeleton, comprising a back-carrying part, an arm exoskeleton part and a data glove part;

[0009] The number of the arm exoskeleton parts is two, and the two arm exoskeleton parts are symmetrically connected to the back part, and the number of the data glove parts is two, so as to respectively collect the position and posture information of the left and right hands;

[0010] The arm exoskeleton part includes a shoulder joint module, an upper arm joint module and a lower arm joint module, and the shoulder joint module includes a first drive, a second drive, a first joint, a second joint and a third joint;

[0011] The first joint is rotatably connected to the back-bearing part, and the first drive is provided between the first joint and the back-bearing part to realize the lifting and lowering of the shoulder glenohumeral joint;

[0012] The second joint is rotatably connected to the first joint, and the second drive is provided between the first joint and the second joint to achieve forward flexion and backward flexion of the glenohumeral joint of the shoulder;

[0013] The third joint is rotatably connected to the second joint, the upper arm joint module includes an upper arm drive, and the upper arm drive is rotatably connected to the third joint, so that the upper arm can track the forward flexion and backward flexion of the shoulder glenohumeral joint, and the forearm joint module is connected to the upper arm joint module;

[0014] A synchronous transmission mechanism is provided between the second joint, the third joint and the upper arm drive to realize driving the upper arm to track the movement of the shoulder glenohumeral joint.

[0015] In one embodiment, the second joint includes a first link and a second link;

[0016] The first ends of the first connecting rod and the second connecting rod are hinged on the first joint, the second ends of the first connecting rod and the second connecting rod are hinged on the third joint, and the first connecting rod, the second connecting rod, the first joint, and the third joint form a parallelogram linkage mechanism.

[0017] In one embodiment, the third joint is rotatably connected to a rotatable driving base, and the upper arm driving device is fixedly mounted on the driving base;

[0018] The synchronous transmission mechanism includes a transmission member, a first transmission wheel and a second transmission wheel, the first transmission wheel is arranged on the first connecting rod or the second connecting rod, the second transmission wheel is arranged on the third joint, and the transmission member is arranged between the first transmission wheel, the second transmission wheel and the rotating shaft of the driving base.

[0019] In one embodiment, the third joint is rotatably connected to a driving base, the upper arm drive is fixedly arranged on the driving base, and the third joint is located between the first connecting rod and the second connecting rod;

[0020] The synchronous transmission mechanism comprises a first transmission member, a second transmission member, a first transmission wheel and a second transmission wheel, wherein the first transmission wheel is arranged on the first connecting rod or the second connecting rod, and the second transmission wheel is arranged on the third joint;

[0021] The first transmission member is arranged between the first transmission wheel and the second transmission wheel, and the second transmission member is arranged between the second transmission wheel and the rotating shaft of the driving base.

[0022] In one embodiment, the upper arm joint module further includes an upper arm plate, an upper arm binding structure, an elbow joint and an elbow drive;

[0023] The first end of the upper arm plate is fixedly connected to the output end of the upper arm drive, the elbow joint is rotationally connected to the second end of the upper arm plate, and the elbow drive is arranged between the upper arm plate and the elbow joint to drive the elbow joint to rotate;

[0024] The upper arm binding structure is arranged on the upper arm flat plate, and an upper arm six-dimensional force sensor is arranged between the upper arm binding structure and the upper arm flat plate.

[0025] In one embodiment, the forearm joint module includes a forearm plate, a forearm drive and a forearm binding structure;

[0026] The forearm drive is arranged between the forearm flat plate and the elbow joint, the forearm binding structure is arranged on the forearm flat plate, and a forearm six-dimensional force sensor is arranged between the forearm binding structure and the forearm flat plate.

[0027] In one embodiment, the backpack portion comprises:

[0028] A back plate, on which the two arm exoskeleton parts are symmetrically connected, and a carrying strap is connected to the back plate;

[0029] An electrical box, fixedly connected to the back plate;

[0030] A back IMU sensor, fixedly connected to the electrical box or the back panel, for measuring the posture of the operator's upper body;

[0031] The depth camera is fixedly connected to the back plate through a bracket to collect video information and depth information in front of the operator.

[0032] In one embodiment, the data glove portion includes:

[0033] The data glove body is used to collect the five degrees of freedom information of the operator's five fingers and the one degree of freedom information of the thumb rotation;

[0034] The hand IMU sensor is fixedly connected to the data glove body and is used to collect information on the three rotational degrees of freedom of the operator's wrist.

[0035] To achieve the above object, the present invention also provides a humanoid robot imitation learning method, comprising the following steps:

[0036] Construct an imitation learning model, including a dexterous hand state encoding module, a robotic arm and waist state encoding module, an image and video reasoning module, an upper limb interactive motion graph convolutional network, and a multi-layer perceptron;

[0037] The operator wears the force feedback exoskeleton to perform actions, and the force feedback exoskeleton is used to collect the real action data of the operator's upper limbs and the operator's perspective image sequence, wherein the real action data includes the operator's hand joint angle, hand joint movement speed data, finger force data, arm joint angle, arm joint movement speed data, arm active force data and waist movement angle;

[0038] Inputting the operator's hand joint angle, hand joint movement speed data, and finger force data into the dexterous hand state coding module to extract the corresponding dexterous hand state coding map;

[0039] Input the operator's arm joint angle, arm joint movement speed data, arm active force data and waist movement angle into the manipulator and waist state coding module, and extract the corresponding manipulator and waist state coding diagram;

[0040] Inputting the operator's view image sequence into the image and video reasoning module to obtain a spatial motion feature interaction coding map;

[0041] The dexterous hand state coding map, the robot arm and waist state coding map, and the spatial motion feature interaction coding map are input into the upper limb interactive motion graph convolutional network, and after L layers of graph convolution layers and graph activation layers connected in series, multi-level feature extraction and feature aggregation related to the graph structure are realized to obtain upper limb motion features and matching spatial interaction features;

[0042] Inputting upper limb motion features and matching spatial interaction features into the multi-layer perceptron to obtain prediction values ​​of upper limb motion control instructions of the humanoid robot;

[0043] Calculating a loss function based on the real motion data and the predicted value of the upper limb motion control instruction of the humanoid robot, and feeding back to update the network parameters of the imitation learning model, and completing the training of the imitation learning model through multiple iterations;

[0044] Obtaining a robot-view image sequence captured by a camera on a humanoid robot, and inputting the robot-view image sequence into an image and video reasoning module in the trained imitation learning model to obtain a robot spatial motion feature interaction coding diagram;

[0045] The robot spatial motion feature interaction coding graph is input into the upper limb interactive motion graph convolutional network, and after L layers of graph convolution layers and graph activation layers connected in series, multi-level feature extraction and feature aggregation related to the graph structure are realized to obtain the robot spatial interaction feature;

[0046] The robot spatial interaction features are input into the multi-layer perceptron to obtain control instructions for upper limb movements of the humanoid robot.

[0047] Compared with the prior art, the present invention has the following beneficial technical effects:

[0048] 1. The exoskeleton of the present invention can not only be carried on the back and has a large range of joint motion, but also takes into account the movement of the shoulder blade joint, and has higher mobility than the traditional exoskeleton mechanical arm. At the same time, the exoskeleton is light in weight and flexible in movement, and can be worn in most human activity scenes, which greatly broadens the use scenarios of the exoskeleton and increases the diversity of humanoid robot imitation learning training data collection;

[0049] 2. The exoskeleton of the present invention is lightweight and integrated while achieving high-quality data collection. Through the use of data gloves and IMU and other sensors, it can collect position data of 28 joint degrees of freedom, force data of 8 joint degrees of freedom, video teaching data and video depth information data, which greatly broadens the data collection richness of traditional exoskeletons;

[0050] 3. The humanoid robot imitation learning method of the present invention constructs an imitation learning model including a neural network, based on the same dual-arm features and operator perspective image input, combined with the noded topological correlation organization of the human joints, integrates multimodal data information, extracts the spatiotemporal features of the corresponding data, and outputs control quantities that are almost the same as expert operations, thereby achieving end-to-end mapping from original feature input to control instructions, thereby guiding the coordinated movement of dual robotic arms, and can be effectively applied to humanoid robot dexterous operation tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0052] Figure 1 Schematic diagram of the structure of the force feedback exoskeleton in Example 1 of the present invention;

[0053] Figure 2 This is a schematic diagram of the structure of the force feedback exoskeleton after being worn in Example 1 of the present invention;

[0054] Figure 3 This is a schematic diagram of the structure of the carrying part in Example 1 of the present invention;

[0055] Figure 4 This is a schematic diagram of the structure of the arm exoskeleton in Example 1 of the present invention;

[0056] Figure 5 It is a partial schematic diagram of the shoulder joint module in Embodiment 1 of the present invention;

[0057] Figure 6 It is a partial schematic diagram of the upper arm joint module and the lower arm joint module in Example 1 of the present invention;

[0058] Figure 7 This is a schematic diagram of the structure of the data glove part in Embodiment 1 of the present invention;

[0059] Figure 8 Schematic diagram of the degree of freedom of operation in Example 2 of the present invention;

[0060] Fig. 9 This is a schematic diagram of multimodal data collection in Example 2 of the present invention;

[0061] Fig.10 The flowchart of the humanoid robot imitation learning method in Embodiment 2 of the present invention;

[0062] Fig.11 Schematic diagram of the framework of the dual-arm operation cloning method based on expert action sampling in Example 2 of the present invention;

[0063] Fig.12 is a state topology diagram of the robot arm in Example 2 of the present invention;

[0064] Fig.13 is a state topology diagram of the dexterous hand in Embodiment 2 of the present invention;

[0065] Fig.14 This is a graph convolutional network structure diagram in Example 2 of the present invention.

[0066] Figure Number:

[0067] Back-carrying part 10, arm exoskeleton part 20, data glove part 30;

[0068] Back plate 101, back strap 102, electrical box 103, back IMU sensor 104, depth camera 105, bracket 106;

[0069] Shoulder joint module 2, first drive 201, second drive 202, first joint 203, second joint 204, first connecting rod 2041, second connecting rod 2042, third joint 205, driving base 206;

[0070] Upper arm joint module 3, upper arm drive 301, upper arm plate 302, upper arm binding structure 303, elbow joint 304, elbow drive 305;

[0071] Forearm joint module 4, forearm plate 401, forearm drive 402, forearm binding structure 403;

[0072] A first transmission wheel 501, a second transmission wheel 502, a first transmission member 503, and a second transmission member 504;

[0073] Data glove body 601 and hand IMU sensor 602.

[0074] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0075] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0076] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0077] In addition, in the present invention, descriptions such as "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0078] In the present invention, unless otherwise clearly specified and limited, the terms "connection", "fixation", etc. should be understood in a broad sense. For example, "fixation" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, an electrical connection, a physical connection, or a wireless communication connection; it can be a direct connection, or an indirect connection through an intermediate medium, or it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0079] In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the fact that ordinary technicians in the field can implement it. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0080] Example 1

[0081] This embodiment discloses a force feedback exoskeleton, which has a force feedback function and a multi-modal control information collection function, wherein the multi-modality includes operating force information, joint position speed information, gyroscope information, and visual information. The exoskeleton integrating the above functions can enable the operator to perform remote operation and human behavior data collection in most human activity scenes.

[0082] refer to Figure 1 , Figure 2 The force feedback exoskeleton in this embodiment mainly includes a back-carrying part 10, an arm exoskeleton part 20 and a data glove part 30. There are two arm exoskeleton parts 20, corresponding to the left arm and the right arm of the operator, that is, the two arm exoskeleton parts 20 are symmetrically connected to the back-carrying part 10. There are also two data glove parts 30, corresponding to the left hand and the right hand of the operator, and the application collects the finger positions and wrist posture information of the left and right hands of the operator respectively.

[0083] refer to Figure 3 The back-carrying part 10 mainly includes a back plate 101, an electrical box 103, a back IMU sensor 104 and a depth camera 105. The back plate 101 is connected with a back strap 102, which can be carried on the back of the operator, and the two arm exoskeleton parts 20 are respectively hinged on the left and right sides of the back plate 101 through bearings. The electrical box 103 is rigidly fixed to the back plate 101 by screws or welding. Electrical equipment such as a force sensor data acquisition box and a signal conversion box are placed inside the electrical box 103. An emergency stop switch and a battery are fixed next to the electrical box 103 for power supply and braking the exoskeleton in special circumstances. The back IMU sensor 104 is rigidly fixed to the electrical box 103. Since the electrical box 103 and the back plate 101 are rigidly connected, the back IMU sensor 104 directly measures the posture of the operator's upper body, which is used to control the movement of the waist motor of the humanoid robot. The back plate 101 is rigidly connected to a bracket 106 by bolts or welding, and the depth camera 105 is fixed on the bracket 106 to collect video information and depth information in front of the operator during the exoskeleton operation.

[0084] refer to Figure 4The arm exoskeleton part 20 includes a shoulder joint module 2, an upper arm joint module 3 and a forearm joint module 4. The shoulder joint module 2 includes a first drive 201, a second drive 202, a first joint 203, a second joint 204 and a third joint 205. The upper arm joint module 3 includes an upper arm drive 301, an upper arm plate 302, an upper arm binding structure 303, an elbow joint 304 and an elbow drive 305. The forearm joint module 4 includes a forearm plate 401, a forearm drive 402 and a forearm binding structure 403.

[0085] refer to Figure 5 The first end of the first joint 203 is rotatably connected to the back plate 101 through a bearing. When the shoulder joint module 2 is lifted horizontally, the shaft of the bearing is perpendicular to the back plate 101 in the horizontal direction. The first drive 201 is a motor fixed to the back plate 101. At the same time, the rotation output end of the first drive 201 is also parallel to the back plate 101 in the horizontal direction. The first drive 201 and the first joint 203 are driven by a helical gear, thereby driving the first joint 203 to rotate on the back plate 101, that is, the lifting and lowering of the shoulder glenohumeral joint is realized.

[0086] refer to Figure 5 The first end of the second joint 204 is rotatably connected to the second end of the first joint 203 through the first hinge. When the shoulder joint module 2 is horizontal, the rotation axis direction of the first hinge is the gravity direction. The second drive 202 can also be set as a motor. The main body of the second drive 202 is fixedly set at the second end of the first joint 203. The rotation output end of the second drive 202 is fixedly connected to the first end of the second joint 204. The second drive 202 can drive the second joint 204 to move forward and backward, that is, to achieve horizontal abduction and horizontal adduction of the shoulder glenohumeral joint.

[0087] refer to Figure 5 , the first end of the third joint 205 is rotatably connected to the second end of the second joint 204 through the second hinge. Similarly, when the shoulder joint module 2 is horizontal, the rotation axis direction of the second hinge is the gravity direction. The second end of the third joint 205 is provided with a driving base 206, and the driving base 206 is rotatably connected to the third joint 205 through a rotating support. The upper arm drive 301 is a motor fixedly connected to the driving base 206. Among them, when the shoulder joint module 2 is horizontal, the rotation axis direction of the rotating support is the gravity direction, and the axial direction of the upper arm drive 301 is perpendicular to the gravity direction where the third joint 205 is located.

[0088] Combination Figure 4The second joint 204 includes a first connecting rod 2041 and a second connecting rod 2042. The first ends of the first connecting rod 2041 and the second connecting rod 2042 are hinged on the first joint 203, and the second ends of the first connecting rod 2041 and the second connecting rod 2042 are hinged on the third joint 205. The first connecting rod 2041, the second connecting rod 2042, the first joint 203, and the third joint 205 form a parallelogram linkage mechanism, that is, when the second joint 204 makes the shoulder swing back and forth under the action of the second drive 202, the third joint 205 follows the movement of the second joint 204. After a synchronous transmission mechanism is set between the second joint 204, the third joint 205, and the upper arm drive 301, it can be achieved that while tracking the 2-degree-of-freedom movement of the glenohumeral joint, the upper arm drive 301 can also track the movement of the shoulder, that is, two motors are used to approximately fit the 3-degree-of-freedom movement of the shoulder.

[0089] The synchronous transmission mechanism includes a transmission member, a first transmission wheel 501 and a second transmission wheel 502, wherein the transmission member can be a synchronous belt, a transmission chain or a gear set, etc. In this embodiment, a synchronous belt is used as the transmission member, and the first transmission wheel 501 and the second transmission wheel 502 are both synchronous belt wheels. In the specific implementation process, the first transmission wheel 501 is arranged on the first connecting rod 2041 or the second connecting rod 2042, and the second transmission wheel 502 is arranged on the third joint 205. The transmission member is wound around the first transmission wheel 501, the second transmission wheel 502 and the rotating support of the driving base 206, and at this time, it is suitable for the first transmission wheel 501 and the second transmission wheel 502 to be at the same height. Among them, the first transmission wheel 501 and the second transmission wheel 502 can be set in multiple according to the shape and size of the second joint 204 and the third joint 205, and a guide tensioning wheel that can tighten the transmission member can also be arranged on the second joint 204 and the third joint 205.

[0090] In the specific application process, in order to reduce the volume of the arm exoskeleton part 20, the third joint 205 is located between the first connecting rod 2041 and the second connecting rod 2042. Figure 4 As shown, there is a height difference between the third joint 205 and the first connecting rod 2041 and the second connecting rod 2042. In this case, it is difficult to use a synchronous belt to connect the first transmission wheel 501, the second transmission wheel 502 and the rotating support of the driving base 206. In this case, the first transmission member 503 and the second transmission member 504 are provided. The first transmission member 503 and the second transmission member 504 are both synchronous belts. The first transmission member 503 is wound around the first transmission wheel 501 and the second transmission wheel 502, and the second transmission member 504 is wound around the rotating shaft of the second transmission wheel 502 and the driving base 206. Similarly, the first transmission wheel 501 and the second transmission wheel 502 can be provided in plurality according to the shape and size of the second joint 204 and the third joint 205, and a guide tensioning wheel that can tighten the transmission member can also be arranged on the second joint 204 and the third joint 205.

[0091] It is worth noting that sprockets can also be used as the first transmission member 503 and the second transmission member 504. In this case, the first transmission wheel 501 and the second transmission wheel 502 need to be replaced with sprockets accordingly, or other forms of transmission structures can be used as the first transmission member 503, the second transmission member 504, the first transmission wheel 501, and the second transmission wheel 502.

[0092] refer to Figure 6 , the first end of the upper arm plate 302 is fixedly connected to the rotation output end of the upper arm drive 301, and the elbow joint 304 is rotationally connected to the second end of the upper arm plate 302. When the upper arm joint module 3 is drooping, the rotation axis direction of the elbow joint 304 is the gravity direction. The elbow drive 305 is a motor, and the main body of the elbow drive 305 is fixed on the back of the upper arm plate 302. The rotation output end of the elbow drive 305 is connected to the elbow joint 304 through a parallelogram linkage mechanism, thereby driving the elbow joint 304 to rotate, and the upper arm binding structure 303 is fixed on the upper arm plate 302 to fix the upper arm. The forearm drive 402 is also a motor, which is fixed at the bottom of the elbow joint 304, and the rotation axis direction of the forearm drive 402 is perpendicular to the gravity direction; at the same time, the forearm plate 401 is fixed to the rotation output end of the forearm drive 402, thereby driving the forearm joint module 4 to rotate, and the forearm binding structure 403 is fixed on the forearm plate 401 to fix the forearm. An upper arm six-dimensional force sensor is provided between the upper arm binding structure 303 and the upper arm flat plate 302, and a lower arm six-dimensional force sensor is provided between the lower arm binding structure 403 and the lower arm flat plate 401, which can collect force information between the exoskeleton and the operator.

[0093] It is worth noting that in this embodiment Figures 4 to 6 The arm exoskeleton part 20 shown in the figure is for the operator's left arm. The arm exoskeleton part 20 for the operator's right arm is similar in structure and hardware system to the arm exoskeleton part 20 shown in the figure. Figures 4 to 6 The structure is completely symmetrical, so it will not be described in detail in this embodiment.

[0094] refer to Figure 7 The data glove part 30 can be mainly divided into a data glove body 601 and a hand IMU sensor 602. The data glove body 601 can collect position information of 6 degrees of freedom, including 5 degrees of freedom for each finger and 1 degree of freedom for thumb rotation, so as to collect the operator's grasping data in the process of completing the task. The hand IMU sensor 602 can collect information on the three degrees of freedom of wrist rotation. Figure 7 The data glove part 30 for the operator's left hand is shown. The data glove part 30 for the operator's right hand is similar in structure and hardware system to the data glove part 30 for the operator's left hand. Figure 7 The structure is completely symmetrical, so it will not be described in detail in this embodiment.

[0095] Example 2

[0096] Based on the force feedback exoskeleton in Example 1, this example discloses a humanoid robot imitation learning method, which is mainly used for the humanoid robot to imitate and learn the operator wearing the force feedback exoskeleton.

[0097] In this embodiment, each arm of the humanoid robot has 7 degrees of freedom, each dexterous hand has 6 degrees of freedom, and the waist has 2 degrees of freedom, for a total of 28 degrees of freedom. The force feedback exoskeleton in Example 1 can independently control all degrees of freedom of the arms and waist of the humanoid robot, wherein: 2 degrees of freedom of the glenohumeral joint, 3 degrees of freedom of the shoulder and 1 degree of freedom of the elbow (a total of 6 degrees of freedom) are controlled by motors bound to the upper arm and forearm, 3 degrees of freedom of a single wrist are controlled by the hand IMU sensor on each data glove part, 6 degrees of freedom of the fingers of a single dexterous hand are controlled by each data glove body, and 2 degrees of freedom of the waist are controlled by the back IMU sensor on the back plate, as shown in the figure. Figure 8 It should be noted that the glenohumeral joint is not designed for general humanoid robots, so the 2 degrees of freedom of the glenohumeral joint of the force feedback exoskeleton in Example 1 are auxiliary degrees of freedom, and the design purpose is to make the exoskeleton more suitable for human movement.

[0098] refer to Fig. 9 , when the expert operator wears the force feedback exoskeleton and performs actions, the real action data of the operator's upper limbs are collected by using the upper arm six-dimensional force sensor, forearm six-dimensional force sensor, back IMU sensor, data glove body and hand IMU sensor on the force feedback exoskeleton, which is equivalent to the true value of the upper limb action control command. While collecting the real action data, the depth camera synchronously collects the operator's perspective image with time synchronization, and the operator's perspective image contains the operator's arm features and hand features. Combined with the graph neural network, based on the node-based topological correlation organization of human joints, the operator's perspective image and motor angle information, IMU posture information (including the operator's hand joint angle, hand joint motion speed data, finger force data, arm joint angle, arm joint motion speed data, arm main force data and waist motion angle) are used to predict the upper limb action control command of the humanoid robot. The network parameters are adjusted and optimized by the error between the predicted value and the true value of the upper limb action control command, and finally the end-to-end mapping with proprioceptive tactile perception and proprioceptive motion perception as input and the control amount of each joint motion of the humanoid robot's upper limb as output is realized.

[0099] refer to Fig.10 , Fig.11 The humanoid robot imitation learning method in this embodiment specifically includes the following steps:

[0100] Construct an imitation learning model, including a dexterous hand state encoding module, a robotic arm and waist state encoding module, an image and video reasoning module, an upper limb interactive motion graph convolutional network, and a multi-layer perceptron;

[0101] The operator wears the force feedback exoskeleton in Example 1 and performs actions, and the force feedback exoskeleton collects the real action data of the operator's upper limbs and the operator's perspective image sequence, wherein the real action data includes the operator's hand joint angle, hand joint movement speed data, finger force data, arm joint angle, arm joint movement speed data, arm main force data and waist movement angle;

[0102] The operator's hand joint angle, hand joint movement speed data, and finger force data are input into the dexterous hand state coding module to extract the corresponding dexterous hand state coding map;

[0103] The operator's arm joint angle, arm joint movement speed data, arm active force data and waist movement angle are input into the robot arm and waist state coding module to extract the corresponding robot arm and waist state coding map;

[0104] The operator's viewpoint image sequence is input into the image and video reasoning module to obtain a spatial motion feature interaction coding map;

[0105] The dexterous hand state coding map, the robot arm and waist state coding map, and the spatial motion feature interaction coding map are input into the upper limb interactive motion graph convolutional network. After L layers of graph convolutional layers and graph activation layers connected in series, multi-level feature extraction and feature aggregation related to the graph structure are realized to obtain the upper limb motion features and the matching spatial interaction features.

[0106] The upper limb motion features and the matching spatial interaction features are input into a multi-layer perceptron to obtain the predicted values ​​of the upper limb motion control instructions of the humanoid robot;

[0107] The loss function is calculated based on the real action data and the predicted value of the humanoid robot's upper limb action control instructions, and the network parameters of the imitation learning model are updated through feedback, and the training of the imitation learning model is completed through multiple iterations;

[0108] Obtain the robot-view image captured by the camera on the humanoid robot, and input the robot-view image into the image-video reasoning module in the trained imitation learning model to obtain the robot's spatial motion feature interaction coding diagram;

[0109] The robot spatial motion feature interaction coding graph is input into the upper limb interactive motion graph convolutional network. After L layers of graph convolutional layers and graph activation layers are connected in series, multi-level feature extraction and feature aggregation related to the graph structure are achieved to obtain the robot spatial interaction features.

[0110] The robot's spatial interaction features are input into a multi-layer perceptron to obtain control instructions for the upper limb movements of the humanoid robot.

[0111] It is worth noting that in the specific application process, the training process in the above-mentioned humanoid robot imitation learning method can be carried out in the simulation software of the computer, or pre-training can be carried out in the simulation software first, and then the pre-trained model can be mounted on the humanoid robot for online reinforcement learning.

[0112] Different from the existing research on teleoperation imitation learning that only considers the state correspondence of the end of the robot arm, this embodiment hopes to simulate the position and posture of key joints such as the shoulder, elbow, and wrist during the expert operation, in order to learn more labor-saving and convenient operations from the operation experience of human experts. Therefore, in addition to the state quantity of the terminal dexterous hand, the state vectors corresponding to other nodes such as the shoulder, elbow, and wrist are expressed as ,in The corresponding node three-dimensional position coordinates, Corresponding to the node Euler angle, the characteristic matrix of the robot state encoding diagram in the above step is expressed as The state topology of the humanoid robot's arms is as follows: Fig.12 As shown in the figure, the lines between nodes represent the flow of information during the graph convolution process. A graph convolution process will cause the node state features to flow to its first-order neighbors, and so on. Considering the possible cooperation and collision avoidance requirements during the dual-arm collaborative operation, in addition to the nodes with direct physical links in the state diagram, considering the possible cooperation and competition relationship between the corresponding joints of the left and right arms during the operation, edges are also added between the corresponding joints of the left and right arms for direct connection.

[0113] The state vector corresponding to the dexterous hand node is expressed as ,in Corresponding to the joint position (bone length), angle, and angular velocity in polar coordinates, Corresponding to the tactile feedback value, the characteristic matrix of the dexterous hand state encoding diagram in the above step is expressed as The state topology of the dexterous hand is as follows Fig.13 As shown in the figure, the lines between nodes represent the flow of information during the graph convolution process. A graph convolution process will cause the node state features to flow to its first-order neighbors, and so on.

[0114] Adjacency matrices and Laplace matrices are often used to formally describe the communication relationship between graph nodes, and can also represent transformation operations acting on the first-order subgraph of each node. Such matrices are collectively referred to as graph displacement operators. These common operators assign the same weight to each communication link, that is, as long as there is an edge link between the nodes, the information between the nodes will be interactively aggregated with equal weights. For two-arm operations, the importance of the feature information of the corresponding joints of the different arms (left and right elbow joints, left and right wrist joints, left and right hands) is inversely proportional to the distance between the two parties. That is to say, only when the distance between the corresponding joints is small, the necessity of considering the competition and cooperation relationship will arise, and when the distance itself is far, the influence of the corresponding joints of the different arms on the movement of the arm can be ignored. Therefore, this embodiment converts the graph displacement operator in the upper limb interactive motion graph convolutional network into Defined as:

[0115] ;

[0116] in, Representation Node With Node The graph displacement operator between Representation Node With Node The distance between.

[0117] The upper limb interactive motion graph convolutional network in this embodiment is composed of The network consists of a series of layers, each of which includes a graph convolution layer and a graph activation layer, that is, Fig.14 As shown in Figure 2. The graph convolution layer is defined as a linear combination of sub-graph convolutions of different orders, namely:

[0118] ;

[0119] in, is the output feature of the graph convolution layer, is the maximum number of neighbor orders that need to be referenced for a decision, is the parameter matrix to be optimized, shared by all nodes, for The feature matrix corresponding to the moment;

[0120] The graph activation layer can be expressed as:

[0121] ;

[0122] in, For the The output of the layer map activation layer, To indicate the The output of the graph convolutional layer, Represents the activation function.

[0123] In the specific application process, the number of network neurons or layers of the graph neural network can be adjusted according to actual needs, as well as the order of the feature matrix data of the robotic arm and dexterous hand.

[0124] The training process of the imitation learning model is to minimize the error between the predicted value and the true value of the upper limb movement control command, that is, ,in, represents the parameter matrix to be trained, represents the optimal parameter matrix after training, Represents the loss function, generally expressed as mean square error, represents the true value of the upper limb movement control command, Represents the predicted value of the upper limb action control command. By minimizing the cumulative error between the predicted value and the expert control value, the imitation learning model can output a control value that is almost the same as the expert operation based on the same dual-arm features and operator perspective image input, realizing end-to-end mapping from original feature input to control command.

[0125] It is worth noting that although this embodiment Fig. 9 The steps in the process are shown in sequence as indicated by the arrows, but the steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of the steps, and the steps can be executed in other orders. Fig. 9 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0126] The above description is only a preferred embodiment of the present invention, and does not limit the scope of the present invention. All equivalent structural changes made by using the contents of the present invention specification and drawings under the inventive concept of the present invention, or directly / indirectly applied in other related technical fields are included in the protection scope of the present invention.

Claims

1. A force feedback exoskeleton, characterized in that: It includes a back-carrying part, an arm exoskeleton part and a data glove part; The number of the arm exoskeleton parts is two, and the two arm exoskeleton parts are symmetrically connected to the back part, and the number of the data glove parts is two, so as to respectively collect the position and posture information of the left and right hands; The arm exoskeleton part includes a shoulder joint module, an upper arm joint module and a lower arm joint module, and the shoulder joint module includes a first drive, a second drive, a first joint, a second joint and a third joint; The first joint is rotatably connected to the back-bearing part, and the first drive is provided between the first joint and the back-bearing part to realize the lifting and lowering of the shoulder glenohumeral joint; The second joint is rotatably connected to the first joint, and the second drive is provided between the first joint and the second joint to achieve forward flexion and backward flexion of the glenohumeral joint of the shoulder; The third joint is rotatably connected to the second joint, the upper arm joint module includes an upper arm drive, and the upper arm drive is rotatably connected to the third joint so that the upper arm can track the forward flexion and backward flexion of the shoulder glenohumeral joint, and the forearm joint module is connected to the upper arm joint module; A synchronous transmission mechanism is provided between the second joint, the third joint and the upper arm drive to achieve driving the upper arm to track the movement of the shoulder glenohumeral joint; The second joint includes a first connecting rod and a second connecting rod; The first ends of the first connecting rod and the second connecting rod are hinged on the first joint, the second ends of the first connecting rod and the second connecting rod are hinged on the third joint, and the first connecting rod, the second connecting rod, the first joint, and the third joint form a parallelogram linkage mechanism.

2. The force feedback exoskeleton according to claim 1, characterized in that: The third joint is rotatably connected to a rotatable driving base, and the upper arm driving device is fixedly mounted on the driving base; The synchronous transmission mechanism includes a transmission member, a first transmission wheel and a second transmission wheel, the first transmission wheel is arranged on the first connecting rod or the second connecting rod, the second transmission wheel is arranged on the third joint, and the transmission member is arranged between the first transmission wheel, the second transmission wheel and the rotating shaft of the driving base.

3. The force feedback exoskeleton according to claim 1, characterized in that: The third joint is rotatably connected to a driving base, the upper arm drive is fixedly arranged on the driving base, and the third joint is located between the first connecting rod and the second connecting rod; The synchronous transmission mechanism comprises a first transmission member, a second transmission member, a first transmission wheel and a second transmission wheel, wherein the first transmission wheel is arranged on the first connecting rod or the second connecting rod, and the second transmission wheel is arranged on the third joint; The first transmission member is arranged between the first transmission wheel and the second transmission wheel, and the second transmission member is arranged between the second transmission wheel and the rotating shaft of the driving base.

4. The force feedback exoskeleton according to any one of claims 1 to 3, characterized in that: The upper arm joint module also includes an upper arm plate, an upper arm binding structure, an elbow joint and an elbow drive; The first end of the upper arm plate is fixedly connected to the output end of the upper arm drive, the elbow joint is rotationally connected to the second end of the upper arm plate, and the elbow drive is arranged between the upper arm plate and the elbow joint to drive the elbow joint to rotate; The upper arm binding structure is arranged on the upper arm flat plate, and an upper arm six-dimensional force sensor is arranged between the upper arm binding structure and the upper arm flat plate.

5. The force feedback exoskeleton according to claim 4, characterized in that: The forearm joint module includes a forearm plate, a forearm drive and a forearm binding structure; The forearm drive is arranged between the forearm flat plate and the elbow joint, the forearm binding structure is arranged on the forearm flat plate, and a forearm six-dimensional force sensor is arranged between the forearm binding structure and the forearm flat plate.

6. The force feedback exoskeleton according to any one of claims 1 to 3, characterized in that: The carrying part comprises: A back plate, on which the two arm exoskeleton parts are symmetrically connected, and a carrying strap is connected to the back plate; An electrical box, fixedly connected to the back plate; A back IMU sensor, fixedly connected to the electrical box or the back plate, for measuring the posture of the operator's upper body; The depth camera is fixedly connected to the back plate through a bracket to collect video information and depth information in front of the operator.

7. The force feedback exoskeleton according to any one of claims 1 to 3, characterized in that: The data glove part includes: The data glove body is used to collect the five degrees of freedom information of the operator's five fingers and the one degree of freedom information of the thumb rotation; The hand IMU sensor is fixedly connected to the data glove body and is used to collect information on the three rotational degrees of freedom of the operator's wrist.

8. A humanoid robot imitation learning method, characterized in that: The steps include: Construct an imitation learning model, including a dexterous hand state encoding module, a robotic arm and waist state encoding module, an image and video reasoning module, an upper limb interactive motion graph convolutional network, and a multi-layer perceptron; The operator wears the force feedback exoskeleton according to any one of claims 1 to 7 to perform actions, and the force feedback exoskeleton is used to collect the real action data of the operator's upper limbs and the operator's perspective image sequence, wherein the real action data includes the operator's hand joint angles, hand joint movement speed data, finger force data, arm joint angles, arm joint movement speed data, arm active force data and waist movement angle; Inputting the operator's hand joint angle, hand joint movement speed data, and finger force data into the dexterous hand state coding module to extract the corresponding dexterous hand state coding map; Input the operator's arm joint angle, arm joint movement speed data, arm active force data and waist movement angle into the manipulator and waist state coding module, and extract the corresponding manipulator and waist state coding diagram; Inputting the operator's view image sequence into the image and video reasoning module to obtain a spatial motion feature interaction coding map; The dexterous hand state coding map, the robot arm and waist state coding map, and the spatial motion feature interaction coding map are input into the upper limb interactive motion graph convolutional network, and after L layers of graph convolution layers and graph activation layers connected in series, multi-level feature extraction and feature aggregation related to the graph structure are realized to obtain upper limb motion features and matching spatial interaction features; Inputting upper limb motion features and matching spatial interaction features into the multi-layer perceptron to obtain prediction values ​​of upper limb motion control instructions of the humanoid robot; Calculating a loss function based on the real motion data and the predicted value of the upper limb motion control instruction of the humanoid robot, and feeding back to update the network parameters of the imitation learning model, and completing the training of the imitation learning model through multiple iterations; Obtaining a robot-view image sequence captured by a camera on a humanoid robot, and inputting the robot-view image sequence into an image and video reasoning module in the trained imitation learning model to obtain a robot spatial motion feature interaction coding diagram; The robot spatial motion feature interaction coding graph is input into the upper limb interactive motion graph convolutional network, and after L layers of graph convolution layers and graph activation layers connected in series, multi-level feature extraction and feature aggregation related to the graph structure are realized to obtain the robot spatial interaction feature; The robot spatial interaction features are input into the multi-layer perceptron to obtain control instructions for upper limb movements of the humanoid robot.

Images