Mechanical arm-dexterous hand system tool intelligent grabbing strategy generation method oriented to space microgravity environment

Abstract

The invention relates to the technical field of intelligent robots, in particular to a mechanical arm-dexterous hand system tool intelligent grabbing strategy generation method oriented to a space microgravity environment. Based on installation configuration of a mechanical arm-dexterous hand system, gravity coefficients are set for a tool to be grabbed and an environment scene, and a simulation environment under space microgravity is constructed; on the basis of configuration of a tool to be grabbed and the dexterous hand, a grabbing configuration of the dexterous hand is generated, and a fixed grabbing test is carried out in the simulation environment; and on the basis of the simulation environment, the pose information of the tool to be grabbed and the real-time states of the mechanical arm and the dexterous hand serve as input of neural network training, actions of the mechanical arm and the dexterous hand serve as output of neural network training, and a grabbing strategy is obtained through neural network training. According to the scheme, an intelligent grabbing strategy of a mechanical arm-dexterous hand system tool can be generated for the space microgravity environment.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent robots, and in particular to a method for generating an intelligent grasping strategy for a tool of a manipulator-dexterous hand system in a space microgravity environment. Background Art

[0002] At present, with the gradual development of space technology and the gradual maturity of robotics technology, the use of manipulator-dexterous hand systems to work with astronauts in the daily operation and maintenance of large facilities such as space stations has a wide range of application prospects. Due to the dynamic and complex characteristics of scenes and targets in space missions, intelligent methods are needed to achieve flexible control, autonomous collision avoidance planning and stable operation. For the task of robot grasping and transferring tools, due to the type of targets, the space environment has the characteristics of microgravity, and its physical behavior is quite different from that on the ground. The intelligent model trained by the ground scene is difficult to be directly applied to space control tasks. The operation actions similar to those on the ground are easy to cause floating objects to be knocked away, which leads to mission failure. At the same time, the above problems also become the training difficulties of the intelligent model. Due to the low fault tolerance of the operation and the large space of the dexterous hand action, the model is difficult to converge quickly. Therefore, it is of great significance to develop a method for generating intelligent grasping strategies for tools in the manipulator-dexterous hand system for the space microgravity environment and form intelligent and dexterous operation capabilities in the space environment. Summary of the invention

[0003] The embodiment of the present invention provides a method for generating an intelligent grasping strategy for a tool of a manipulator-dexterous hand system in a space microgravity environment, which can generate an intelligent grasping strategy for a tool of a manipulator-dexterous hand system in a space microgravity environment.

[0004] The embodiment of the present invention provides a method for generating an intelligent grasping strategy for a tool of a manipulator-dexterous hand system in a space microgravity environment, comprising:

[0005] The gravity coefficient is set based on the installation configuration of the robot arm-dexterous hand system, the tool to be grasped and the environmental scene to build a simulation environment under space microgravity;

[0006] generating a grasping configuration of the dexterous hand based on the tool to be grasped and the configuration of the dexterous hand, so as to perform a fixed grasping test under the simulation environment;

[0007] Based on the simulation environment, the posture information of the tool to be grasped, the real-time status of the robotic arm and the dexterous hand are used as the input of the neural network training, and the movements of the robotic arm and the dexterous hand are used as the output of the neural network training, and the grasping strategy is obtained through the neural network training.

[0008] Optionally, the reward function of the neural network training includes an approach reward function and a grasping reward function.

[0009] Optionally, the proximity reward function is as follows:

[0010]

[0011] Among them, α, β, γ, ε, ρ, μ are parameters greater than 0, and p h is the vector of the tool relative to the palm, v is the speed of the tool relative to the palm, a x ,a y ,a z Corresponding to the three-axis unit vector in the three-dimensional model of the palm, a ox Corresponding to the x-axis of the tool, a hx Thumb tip x-axis, p ho is the orientation unit vector of the tool relative to the palm.

[0012] Optionally, the grab reward function is as follows:

[0013]

[0014] Among them, r a To be close to the reward function, μ is a parameter greater than 0, p H is the deviation vector between the current joint angle of the dexterous hand and the grasping configuration angle.

[0015] Optionally, the neural network training includes:

[0016] Approaching stage: guiding the robot arm-dexterous hand system to approach the tool to be grasped. When the relative position and posture of the tool to be grasped and the palm of the dexterous hand are within the set first threshold for a preset time, the approaching task is judged to be completed, and the robot arm-dexterous hand system is guided to grasp the tool to be grasped; when the relative position and posture of the tool to be grasped and the palm exceed the set second threshold or the palm end force is greater than the preset force, the task is judged to have failed, and the penalty value of the approaching reward function is increased;

[0017] Grasping stage: when guiding the robot arm-dexterous hand system to grasp the tool to be grasped, when the relative position and posture of the tool to be grasped and the palm are within the third threshold, and the relative distance between the current dexterous hand and the target configuration is within the set fourth threshold for a preset time, the grasping task is judged to be completed, and the reward value of the grasping reward function is increased; when the relative position and posture of the tool to be grasped and the palm exceeds the set second threshold or the palm end force is greater than the preset force, the task is judged to have failed, and the penalty value of the grasping reward function is increased.

[0018] Optionally, the reset strategy of the neural network training is as follows:

[0019] When the judgment of the approaching stage and the grasping stage is completed, resetting the simulation environment;

[0020] In the reset simulation environment, the probability of 1 / 10 is controlled so that the tool to be grasped is initialized in the palm of the hand, and its position and posture are close to the correct grasping state;

[0021] In the reset environment, the probability of 1 / 10 is controlled so that the tool to be grasped is initialized near the palm of the hand, and its posture is close to the correct grasping posture;

[0022] In the reset environment, the probability of 4 / 5 is controlled to make the position and posture of the tool to be grasped random.

[0023] Optionally, the movements of the dexterous hand and the robotic arm include six-axis movements of the robotic arm and seven-degree-of-freedom joint movements of the dexterous hand.

[0024] Optionally, the neural network training adopts LSTM-PPO strategy, and the network architecture includes:

[0025] LSTM layer, 512 neurons;

[0026] MLP first layer, 256 neurons;

[0027] MLP second layer, 128 neurons;

[0028] MLP third layer, 64 neurons;

[0029] The output layer results in 13-dimensional actions.

[0030] Optionally, it also includes:

[0031] Perform inference verification on the trained neural network.

[0032] Compared with the prior art, the present invention has at least the following beneficial effects:

[0033] In this embodiment, a simulation environment is obtained by simulating a space microgravity environment, and a neural network training is performed in the simulation environment to obtain an intelligent grasping strategy of the pick-up / unload ratio-dexterous hand system. The input of the neural network training is the posture information of the tool to be grasped, the real-time status of the robotic arm and the dexterous hand, and the output of the neural network training is the movement of the robotic arm and the dexterous hand. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0035] Figure 1This is a flow chart of the multi-robot multi-mode arm-hand-eye relationship intelligent calibration method based on online reconstruction-rendering-matching of the present invention;

[0036] Figure 2 The initial arm-eye calibration rendering image of this embodiment of the present invention;

[0037] Figure 3 The actual arm-eye calibration image acquired by the camera of this embodiment of the present invention;

[0038] Figure 4 The comparison images are actually taken and rendered under the initial value of the arm-eye relationship in this embodiment of the present invention;

[0039] Figure 5 It is a comparison image actually shot and rendered under the final arm-eye calibration result in an embodiment of the present invention;

[0040] Figure 6 It is a comparison image actually shot and rendered under the final hand-eye calibration result in an embodiment of the present invention;

[0041] Figure 7 This is a comparison image of the actual shooting and rendering under the final arm-to-arm calibration result in an embodiment of the present invention. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0043] In the description of the embodiments of the present invention, unless otherwise clearly specified and limited, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance; unless otherwise specified or explained, the term "plurality" refers to two or more; the terms "connected" and "fixed" should be understood in a broad sense, for example, "connected" can be a fixed connection, a detachable connection, an integral connection, or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0044] In the description of this specification, it should be understood that the directional words such as "upper" and "lower" described in the embodiments of the present invention are described at the angles shown in the drawings and should not be understood as limiting the embodiments of the present invention. In addition, in the context, it should also be understood that when it is mentioned that an element is connected to another element "upper" or "lower", it can not only be directly connected to another element "upper" or "lower", but also indirectly connected to another element "upper" or "lower" through an intermediate element.

[0045] like Figure 1 As shown, an embodiment of the present invention provides a method for generating an intelligent grasping strategy for a tool of a manipulator-dexterous hand system in a space microgravity environment, comprising:

[0046] S1, based on the installation configuration of the robot arm-dexterous hand system, the tool to be grasped and the environmental scene, the gravity coefficient is set to build a simulation environment under space microgravity;

[0047] S2, based on the tool to be grasped and the configuration of the dexterous hand, the grasping configuration of the dexterous hand is generated to perform a fixed grasping test in a simulation environment;

[0048] S3, based on the simulation environment, takes the posture information of the tool to be grasped, the real-time status of the robotic arm and the dexterous hand as the input of the neural network training, and takes the movements of the robotic arm and the dexterous hand as the output of the neural network training, and obtains the grasping strategy through the neural network training.

[0049] The robotic arm has more than 6 active degrees of freedom, which can realize 3 degrees of translation and 3 degrees of rotation at the end of the robotic arm; the dexterous hand has 5 fingers, whose joint distribution is similar to that of a human hand, each finger has 4 degrees of freedom, of which no less than 2 are active degrees of freedom; the manipulated target is a common tool such as a hammer, a tool with a certain initial velocity and linear velocity, whose position and velocity can be obtained by camera calculation or directly in a simulation environment; the space floating environment can be set to 0 to 1g.

[0050] In this embodiment, a simulation environment is obtained by simulating a space microgravity environment, and a neural network training is performed in the simulation environment to obtain an intelligent grasping strategy of the pick-up / unload ratio-dexterous hand system. The input of the neural network training is the posture information of the tool to be grasped, the real-time status of the robotic arm and the dexterous hand, and the output of the neural network training is the movement of the robotic arm and the dexterous hand.

[0051] For S1, the simulation environment can build a simulation model based on the general description file of the robotic arm and dexterous hand, parameterize the gravity parameters, and support parallel training in multiple simulation environments.

[0052] The steps for setting up the simulation environment are as follows: Figure 2 As shown:

[0053] (1) The simulation environment uses IsaacGym, the simulation physics engine uses PhysX, and the interface program with the intelligent grasping network reinforcement learning training uses IsaacGymEnvs;

[0054] (2) The UR5 robot arm, which has 6 active degrees of freedom, is used, and its kinematic and dynamic models are configured based on the URDF general description file;

[0055] (3) A five-finger dexterous hand is used, each finger has 4 degrees of freedom, including active degrees of freedom and passive coupled degrees of freedom, and its simplified kinematic and dynamic models are configured based on the URDF general description file. For the passive coupled degrees of freedom, an online solution is adopted based on the given values ​​of the active degrees of freedom and the active value is assigned in the simulation environment to avoid the situation where the complex hybrid mechanism cannot be solved in the simulation engine;

[0056] (4) Setting the robot base coordinate system and the world coordinate system to be fixedly connected;

[0057] (5) The tool to be grasped is set as a hammer. Its 3D model is constructed using RGB camera and real-object image data, and is manually optimized and adjusted in conjunction with the NeRF 3D reconstruction network. Its 3D model is configured based on the URDF general description file, and its pose is initialized in the simulation environment. It is set not to be fixed to any object to achieve the simulation of the floating state.

[0058] (6) Set simulation parameters such as simulation step size and PID parameters of position control.

[0059] For S2, the grasping configuration of the dexterous hand can obtain the reasonable configuration of human hand grasping according to the target, and based on this, the grasping configuration of the dexterous hand is generated and adjusted.

[0060] Based on the configuration of the manipulated object and the dexterous hand, the grasping configuration of the dexterous hand is generated, including the relative posture between the root of the dexterous hand and the manipulated object and the angles of each joint of the dexterous hand;

[0061] The method for generating the grasping configuration is as follows:

[0062] (1) Obtain the typical configuration of human hand grasping based on OakInk, including relative posture and values ​​of each joint of the hand, such as Figure 3 As shown;

[0063] (2) Perform a fixed grasping test in a simulation environment according to the configuration obtained in step (1), and manually optimize and adjust the configuration to ensure that it is suitable for the five-finger dexterous hand.

[0064] For S3, specifically, the observation input may include the shape information of the target (tool to be grasped), the position, posture, linear velocity, and angular velocity of the target relative to the base coordinate system of the robotic arm, the angle, angular velocity, and position, posture, linear velocity, and angular velocity of each joint of the robotic arm, the angle, angular velocity, and position of each finger end point of each joint of the dexterous hand, the six-dimensional force sensor at the wrist of the robotic arm, the force sensor of the fingers of the dexterous hand, and other partial information in the state information.

[0065] The reward function calculates the reward function based on the input state information, according to the gap between the current state and the ideal approach state, sets penalties for factors such as collisions causing the tool to move away, and determines the scope of the current reward function.

[0066] The intelligent network observation input is determined as follows and can be adjusted according to the actual training effect:

[0067] (1) The first to third dimensions are the tool positions, and the fourth to seventh dimensions are the tool attitude quaternions, all with respect to the robot base coordinate system;

[0068] (2) The 8th to 10th dimensions are the linear velocity of the tool, and the 11th to 13th dimensions are the angular velocity of the tool, and the values ​​are all relative to the robot base coordinate system;

[0069] (3) The 14th to 19th dimensions are the angles of the joints of the robot arm, and the 19th to 24th dimensions are the angular velocities of the joints of the robot arm;

[0070] (4) The 25th to 44th dimensions are the angles of the joints of the dexterous hand, and the 45th to 64th dimensions are the angular velocities of the joints of the dexterous hand;

[0071] (5) The 65th to 67th dimensions are the positions of the base of the dexterous hand, and the 68th to 71st dimensions are the attitude quaternions of the base of the dexterous hand, and the values ​​are all relative to the base coordinate system of the robot arm;

[0072] (6) Dimensions 71 to 75 are the contact forces at the ends of each finger of the dexterous hand;

[0073] (7) Dimensions 76 to 78 are the dimensions of the tool;

[0074] (8) Dimensions 79 to 91 are the output action values ​​of the previous network;

[0075] (9) Dimensions 92 to 111 are the joint angle values ​​of the dexterous hand configuration calculated by S2.

[0076] In some embodiments of the present invention, the reward function for neural network training includes an approach reward function and a grasping reward function.

[0077] In some embodiments of the invention, the proximity reward function is as follows:

[0078]

[0079] Among them, α, β, γ, ε, ρ, μ are parameters greater than 0, and p h is the vector of the tool relative to the palm, v is the speed of the tool relative to the palm, a x ,a y ,a z Corresponding to the three-axis unit vector in the three-dimensional model of the palm, a ox Corresponding to the x-axis of the tool, a hx Thumb tip x-axis, p ho is the orientation unit vector of the tool relative to the palm.

[0080] Specifically, The term is the change in the relative distance between the palm and the target, which encourages the operator to approach the target.

[0081] v·(-a x ) is the speed bonus for approaching in the normal direction of the palm;

[0082] v·(-a z ) is the velocity bonus towards the end normal;

[0083] |a ox ·a y | is a rotation posture bonus, so that the direction of the tool hammer handle corresponds to the direction of the palm;

[0084] p ho ·a x Item is relative to the palm a x Towards reward;

[0085] p ho ·a hx Item is relative to the palm fingertip a hx Towards the reward.

[0086] In some embodiments of the invention, the grab reward function is as follows:

[0087]

[0088] Among them, r a To be close to the reward function, μ is a parameter greater than 0, p H is the deviation vector between the current joint angle of the dexterous hand and the grasping configuration angle.

[0089] The term is the change in the relative distance between the current dexterous hand and the target configuration, which encourages the dexterous hand to grasp the desired configuration.

[0090] In some embodiments of the invention, neural network training includes:

[0091] Approaching stage: guide the robot arm-dexterous hand system to approach the tool to be grasped. When the relative position and posture of the tool to be grasped and the palm of the dexterous hand are within the set first threshold for a preset time, the approach task is judged to be completed, and the robot arm-dexterous hand system is guided to grasp the tool to be grasped; when the relative position and posture of the tool to be grasped and the palm exceed the set second threshold or the palm end force is greater than the preset force, the task is judged to have failed, and the penalty value of the approach reward function is increased;

[0092] Grasping stage: when guiding the robot arm-dexterous hand system to grasp the tool to be grasped, when the relative position and posture of the tool to be grasped and the palm are within the third threshold, and the relative distance between the current dexterous hand and the target configuration is within the set fourth threshold for a preset time, the grasping task is judged to be completed and the reward value of the grasping reward function is increased; when the relative position and posture of the tool to be grasped and the palm exceeds the set second threshold or the palm end force is greater than the preset force, the task is judged to have failed and the penalty value of the grasping reward function is increased.

[0093] In some embodiments of the present invention, the reset strategy for neural network training is as follows:

[0094] When the judgment of the approach phase and the grasping phase is completed, the simulation environment is reset;

[0095] In the reset simulation environment, the probability of 1 / 10 is controlled so that the tool to be grasped is initialized in the palm of the hand, and its position and posture are close to the correct grasping state;

[0096] In the reset environment, the probability of 1 / 10 is controlled so that the tool to be grasped is initialized near the palm of the hand, and its posture is close to the correct grasping posture;

[0097] In the reset environment, the probability of 4 / 5 is controlled to make the position and posture of the tool to be grasped random.

[0098] In some embodiments of the present invention, the movements of the dexterous hand and the robotic arm include six-axis movements of the robotic arm and seven-degree-of-freedom joint movements of the dexterous hand.

[0099] Specifically, the output action contains a total of 6 degrees of freedom, corresponding to the six-axis action of the robot arm, all of which are relative incremental values, that is, the expected joint position of the robot arm is set to the sum of the current position and the output action value of the intelligent grasping network;

[0100] For the joint movements of the dexterous hand, in order to reduce the exploration space, each flexion and extension joint in each finger is set as a virtual coupling. The output movement contains a total of 7 degrees of freedom, including the closing and extension of all flexion and extension joints, the side swing of the five fingers, and the rotation degree of freedom of the thumb along the root. All of them are relative incremental values, that is, the expected joint position of the dexterous hand is set to the sum of the current position and the output action value of the intelligent grasping network.

[0101] In some embodiments of the present invention, the neural network training adopts the LSTM-PPO strategy, and the network architecture ( Figure 4 )include:

[0102] LSTM layer, 512 neurons;

[0103] MLP first layer, 256 neurons;

[0104] MLP second layer, 128 neurons;

[0105] MLP third layer, 64 neurons;

[0106] The output layer results in 13-dimensional actions.

[0107] In some embodiments of the present invention, it further includes:

[0108] Perform inference verification on the trained neural network.

[0109] In this embodiment, the initial position of the tool is randomized, and the trained intelligent grasping model is used for reasoning verification. The obstacle avoidance grasping of the tool can be completed. The tool with a shape similar to that used in the training process can be used to test the generalization ability of the intelligent method. The specific process is as follows: Figure 5 , Figure 6 , Figure 7 shown.

[0110] The present invention can be widely used in microgravity environments such as space stations, where space robots can work autonomously or in collaboration with astronauts, such as repairing and replacing key components.

[0111] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating intelligent grasping strategies for tools in a robot arm-dexterous hand system in a space microgravity environment, characterized in that: include: The gravity coefficient is set based on the installation configuration of the robot arm-dexterous hand system, the tool to be grasped and the environmental scene to build a simulation environment under space microgravity; generating a grasping configuration of the dexterous hand based on the tool to be grasped and the configuration of the dexterous hand, so as to perform a fixed grasping test under the simulation environment; Based on the simulation environment, the posture information of the tool to be grasped, the real-time status of the robotic arm and the dexterous hand are used as the input of the neural network training, and the movements of the robotic arm and the dexterous hand are used as the output of the neural network training, and the grasping strategy is obtained through the neural network training.

2. The method according to claim 1, characterized in that The reward function of the neural network training includes an approach reward function and a grasping reward function.

3. The method according to claim 2, characterized in that The proximity reward function is as follows: Among them, α, β, γ, ε, ρ, μ are parameters greater than 0, and p h is the vector of the tool relative to the palm, v is the speed of the tool relative to the palm, a x ,a y ,a z Corresponding to the three-axis unit vector in the three-dimensional model of the palm, a ox Corresponding to the x-axis of the tool, a hx Thumb tip x-axis, p ho is the orientation unit vector of the tool relative to the palm.

4. The method according to claim 2, characterized in that: The grab reward function is as follows: Among them, r a To be close to the reward function, μ is a parameter greater than 0, p H is the deviation vector between the current joint angle of the dexterous hand and the grasping configuration angle.

5. The method according to claim 2, characterized in that: The neural network training includes: Approaching stage: guiding the robot arm-dexterous hand system to approach the tool to be grasped. When the relative position and posture of the tool to be grasped and the palm of the dexterous hand are within the set first threshold for a preset time, the approaching task is judged to be completed, and the robot arm-dexterous hand system is guided to grasp the tool to be grasped; when the relative position and posture of the tool to be grasped and the palm exceed the set second threshold or the palm end force is greater than the preset force, the task is judged to have failed, and the penalty value of the approaching reward function is increased; Grasping stage: when guiding the robot arm-dexterous hand system to grasp the tool to be grasped, when the relative position and posture of the tool to be grasped and the palm are within the third threshold, and the relative distance between the current dexterous hand and the target configuration is within the set fourth threshold for a preset time, the grasping task is judged to be completed, and the reward value of the grasping reward function is increased; when the relative position and posture of the tool to be grasped and the palm exceeds the set second threshold or the palm end force is greater than the preset force, the task is judged to have failed, and the penalty value of the grasping reward function is increased.

6. The method according to claim 5, characterized in that The reset strategy for the neural network training is as follows: When the judgment of the approaching stage and the grasping stage is completed, resetting the simulation environment; In the reset simulation environment, the probability of 1 / 10 is controlled so that the tool to be grasped is initialized in the palm of the hand, and its position and posture are close to the correct grasping state; In the reset environment, the probability of 1 / 10 is controlled so that the tool to be grasped is initialized near the palm of the hand, and its posture is close to the correct grasping posture; In the reset environment, the probability of 4 / 5 is controlled to make the position and posture of the tool to be grasped random.

7. The method according to claim 1, characterized in that The movements of the dexterous hand and the robotic arm include six-axis movements of the robotic arm and seven-degree-of-freedom joint movements of the dexterous hand.

8. The method according to claim 1, characterized in that The neural network training adopts the LSTM-PPO strategy, and the network architecture includes: LSTM layer, 512 neurons; MLP first layer, 256 neurons; MLP second layer, 128 neurons; MLP third layer, 64 neurons; The output layer results in 13-dimensional actions.

9. The method according to claim 1, characterized in that: Also includes: Perform inference verification on the trained neural network.

Images