Five-fingered robot hand grasping control method and system with teaching-learning capability

Abstract

The present application relates to the technical field of robot learning, and provides a five-fingered hand grasping control method and system with teaching learning capability. The method comprises the following steps: under a three-dimensional motion capture system, a world coordinate system is established; under the condition of wearing a data glove, hand posture information data when a hand grips different objects is collected; based on the hand posture information data, a mapping relationship between the hand and the five-fingered hand is established; based on the hand posture information data, a generative adversarial network is used to analyze the hand posture, a coordination matrix is constructed, and a pre-grasping posture is generated according to different grasped objects; through the motion capture system, the object to be gripped is identified, the closest preset posture is matched for the current pre-grasping posture of the five-fingered hand based on the mapping relationship between the hand and the five-fingered hand, and a quadratic programming-based inverse dynamics control method is used to control the five-fingered hand to grip.

Description

Technical Field

[0001] This invention relates to the field of robot learning technology, and in particular to a five-finger robotic hand grasping control method and system with teaching and learning capabilities. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Robotics technology is advancing rapidly, and its applications have expanded from traditional industrial production to everyday life. Manipulator grasping control is a crucial direction in robot development, involving multiple aspects such as perception, planning, and control. It has already played a significant role in industrial sectors like manufacturing and logistics, as well as in civilian applications such as prosthetics and exoskeletons for assisted grasping. However, in practical applications, designing controllers for manipulators with human-like five fingers remains challenging. The accuracy (such as grasp success rate and grasping posture stability) and precision (grasping force accuracy and trajectory accuracy) of their grasping remain insufficient, severely impacting the grasping efficiency and widespread application of multi-fingered manipulators. Therefore, designing and developing high-performance manipulator grasping controllers to improve their grasping performance has become a current research hotspot.

[0004] Traditional robotic gripper control methods are mostly based on rules and experience, and rely on manual parameter adjustment. When facing unstructured environments, they cannot adapt to the physical properties of objects (such as material, shape, surface smoothness, etc.), initial state (such as flat, oblique, initial velocity, etc.), motion state (such as speed, acceleration, angular acceleration, etc. when moving in space), and task objectives (such as stable holding, accelerated lifting, movement between two points, etc.), resulting in poor gripping control flexibility, low stability, and weak adaptability.

[0005] In recent years, with the development of artificial intelligence technologies such as deep learning and reinforcement learning, data-driven robotic arm grasping control methods have received widespread attention. These methods can train the robotic arm's grasping strategy using a large amount of data and can adaptively adjust the grasping strategy, improving the accuracy and precision of grasping. However, these methods still have many drawbacks, mainly including the need for large amounts of training data, the requirement for accurate data labeling, excessively long training time, poor learning generalization ability, and low online control performance.

[0006] To address the aforementioned problems in the grasping control of anthropomorphic five-fingered robotic hands, numerous control methods have been proposed, including: vision-based grasping control methods, object shape-based grasping control methods, and force feedback-based grasping control methods. These methods can effectively improve the accuracy and precision of robotic hand grasping, but shortcomings remain, particularly in how to achieve human-like control strategies when grasping complex objects, and how to ensure short transient transitions and high control precision during the grasping process. Therefore, research on five-fingered robotic hand grasping control methods remains extremely challenging. Summary of the Invention

[0007] In order to solve the technical problems existing in the background art, the present invention provides a five-finger robotic hand grasping control method and system with teaching and learning capabilities, which aims to achieve human-like grasping while autonomously learning the grasping control method, thereby improving the control effect.

[0008] To achieve the above objectives, the present invention adopts the following technical solution:

[0009] The first aspect of the present invention provides a five-finger robotic hand grasping control method with teaching and learning capabilities.

[0010] A grasping control method for a five-finger robotic hand with teach-and-learn capabilities, comprising:

[0011] In a 3D motion capture system, a world coordinate system is established, and hand posture information data when a person grasps different objects is collected while wearing data gloves;

[0012] Based on hand posture information data, a mapping relationship between the human hand and a five-finger robotic hand is established;

[0013] Based on hand posture information data, a generative adversarial network (GAN) is used to analyze hand posture, construct a collaboration matrix, and generate pre-grasping postures according to different grasping objects.

[0014] The motion capture system identifies the object to be grasped. Based on the mapping relationship between the human hand and the five-finger robotic hand, the system matches the closest preset posture to the current pre-grasping posture of the five-finger robotic hand. The inverse dynamics control method based on quadratic programming is used to control the grasping of the five-finger robotic hand.

[0015] The process of using the inverse dynamics control method based on quadratic programming includes: constructing the motion equations and constraint equations of the five-finger manipulator, introducing a cost function, and defining the desired acceleration of the fingertips of the five-finger manipulator; calculating the acceleration of the fingertips of the five-finger manipulator based on the desired acceleration and feedback terms, with the goal of minimizing the cost function; and controlling the five-finger manipulator's gripping according to the torque required by the joint and the acceleration.

[0016] Furthermore, the hand posture information data includes the position of each finger joint in all grasping postures, the position of the wrist in the world coordinate system, and the angular position of the joint connecting the fingers and the palm.

[0017] Furthermore, the process of employing a generative adversarial network includes: expanding the hand pose information data to construct input data; using a generator based on the input data to obtain generated data; and using a discriminator to continuously optimize the generator network with the goal of minimizing the difference between the generated data and the hand pose information data; wherein the generative adversarial network includes a generator and a discriminator.

[0018] Furthermore, the process of using generative adversarial networks to analyze hand pose and construct a cooperative matrix includes: obtaining principal component data of hand pose based on hand pose information data using an optimized generator network; performing principal component analysis on the principal component data of hand pose to construct a cooperative matrix, wherein the cooperative matrix is ​​the product of cooperative weights and cooperative effect subspaces.

[0019] Furthermore, the equations of motion and constraint equations are as follows:

[0020]

[0021]

[0022] Where, q∈R j Let j represent the state of the system, j represent the degree of freedom of the linkage of the five-fingered robotic hand, and x∈R. l It is a cascade of the attitudes of all contact points in Cartesian space, M(q)∈R j×j Represents the inertia matrix. S represents the sum of gravity, centrifugal force, and Coriolis force, where S∈R j×j It is a selection matrix. It is a cascade of all contact torque vectors. It is a concatenation of the Jacobian matrices of all contact points, τ∈R j It is a vector containing the control torques of all degrees of freedom of the system.

[0023] Furthermore, the mathematical model expression for the quadratic programming is:

[0024]

[0025] stA E θ=a E

[0026] A I θ≥a I

[0027] The optimized cost function is 1 / 2||Aθ-a|| 2 ,in, A E and a E A represents an equality constraint. I and a I This represents an inequality constraint, and G = A. T A, g = -A T a, A and a are both column vectors of the desired behavior.

[0028] Furthermore, the input of the fingertips of the five-finger robotic hand is solved.

[0029]

[0030] in, For the desired acceleration of the end joint, This is the feedback term for the closed-loop control circuit.

[0031] Furthermore, the process of solving the feedback term includes: establishing a desired reference model and a dynamic model of the distal joints of the five-finger robotic hand; and, according to preset control rules, making the dynamic model follow the desired reference model to solve the feedback term.

[0032] Furthermore, the desired reference model is:

[0033]

[0034] Where r(t) is a bounded adjustment signal, B r C r r(t) is the design parameter of the reference model. From the Lyapunov perspective, we have t→∞, ||ζ r -ζ d ||→0;

[0035] Furthermore, the dynamic model of the distal joint is as follows:

[0036]

[0037] in, Parameters B and C are both unknown and determined by the adaptive law; F(·)∈Rn , being a smooth function, is approximated by an adaptive RBF neural network.

[0038] Furthermore, the control rules are as follows:

[0039] υ=ψ r ζ+ψ r r(t)+ψ φ φ(e)

[0040] Where, ψ ζ ,ψ r ,ψ φ All parameters are online approximations; the adaptive controller in the task space will use ζ, ζ d As input, output Input QP optimization;

[0041]

[0042] in, It is an approximation matrix of the above equation and follows the following adaptive law:

[0043]

[0044]

[0045]

[0046] Among them, Λ ζ ,Λ r ,Λ φ It is a positive definite matrix that adjusts the convergence speed of the adaptive gain; as a necessary and sufficient stability condition for the cost function, P and C r satisfy

[0047] A second aspect of the present invention provides a five-finger robotic gripping control system with teaching and learning capabilities.

[0048] A five-finger robotic hand grasping control system with teach-and-learn capabilities includes:

[0049] The teaching module is configured to: establish a world coordinate system under a 3D motion capture system, and collect hand posture information data when a human hand grasps different objects while wearing data gloves;

[0050] The mapping module is configured to establish a mapping relationship between a human hand and a five-finger robotic hand based on hand posture information data.

[0051] The pose generation module is configured to: analyze hand poses using generative adversarial networks based on hand pose information data, construct a cooperative matrix, and generate pre-grasping poses according to different grasping objects;

[0052] The control module is configured to: identify the object to be grasped through the motion capture system; match the closest preset posture to the current pre-grasping posture of the five-finger robotic hand based on the mapping relationship between the human hand and the five-finger robotic hand; and control the grasping of the five-finger robotic hand using an inverse dynamics control method based on quadratic programming.

[0053] The process of using the inverse dynamics control method based on quadratic programming includes: constructing the motion equations and constraint equations of the five-finger manipulator, introducing a cost function, and defining the desired acceleration of the fingertips of the five-finger manipulator; calculating the acceleration of the fingertips of the five-finger manipulator based on the desired acceleration and feedback terms, with the goal of minimizing the cost function; and controlling the five-finger manipulator's gripping according to the torque required by the joint and the acceleration.

[0054] Compared with the prior art, the beneficial effects of the present invention are:

[0055] (1) The principal component analysis method based on generative adversarial networks in this invention no longer relies on statistical principles and mathematical model derivation, but directly learns the data distribution by training the generator network, which can better adapt to the nonlinearity and complexity of the data.

[0056] (2) The present invention transfers the synergistic effect to the robot, making it more adaptable and flexible, because it allows the generalization of learned skills to be extended to grasping new objects, and the transfer of the synergistic effect is simpler.

[0057] (3) The inverse dynamics (ID) controller based on quadratic programming (QP) of this invention can calculate the optimal joint torque distribution according to the dynamic model and performance index of the system, so as to achieve accurate motion tracking and torque control, and can dynamically adjust the control strategy according to the real-time measured data.

[0058] (4) Through online learning and adaptive technology, the controller can update the model parameters at runtime to adapt to changes in system dynamics and uncertainties.

[0059] (5) The present invention can better adapt to nonlinear and complex data, enabling the robot to be more adaptable and flexible, complete precise motion tracking and torque control, and dynamically adjust the control strategy.

[0060] (6) This invention can handle complex data and dynamic environments, achieving precise motion control and flexible task execution, providing a higher level of control and adaptability for robot systems. It enables robots to cope with diverse task and environmental requirements and possess the ability to learn and adjust autonomously. This is of great significance for improving the practicality, application scope, and performance of robots. Attached Figure Description

[0061] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0062] Figure 1 This is a flowchart of a five-finger robotic hand grasping control method with teaching and learning capabilities, as shown in this invention.

[0063] Figure 2 This is a flowchart of the principal component analysis method based on generative adversarial networks shown in this invention;

[0064] Figure 3 This is a diagram of the inverse dynamics control framework based on quadratic programming shown in this invention;

[0065] Figure 4 This is a diagram showing the calculation results of the synergistic effect of the present invention. The center shows the initial posture of the five-finger robotic hand. The lower left to upper right corner Syergy2 represents the hand posture generated along the synergistic coefficient 2. The middle from bottom to top Syergy1 represents the hand posture generated along the synergistic coefficient 1. The lower right to upper left corner Syergy3 represents the hand posture generated along the synergistic coefficient 3.

[0066] Figure 5 This is a diagram illustrating the system integration and application scenarios of this invention;

[0067] Among them, 1. Motion capture system, 2. Five-finger robotic hand, 3. Cylinder object grasping, 4. Sphere object grasping, 5. Cube object grasping, 6. Disc object grasping, 7. Test bench, 8. Data glove. Detailed Implementation

[0068] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0069] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0070] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0071] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of methods and systems according to various embodiments of this disclosure. It should be noted that each block in a flowchart or block diagram may represent a module, segment, or portion of code, which may include one or more executable instructions for implementing the logical functions specified in the various embodiments. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that shown in the drawings. For example, two consecutively represented blocks may actually be executed substantially in parallel, or they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, may be implemented using a dedicated hardware-based system that performs the specified functions or operations, or using a combination of dedicated hardware and computer instructions.

[0072] Example 1

[0073] This embodiment provides a five-finger robotic hand grasping control method with teaching and learning capabilities, the method including the following steps:

[0074] In a 3D motion capture system, a world coordinate system is established, and hand posture information data when a person grasps different objects is collected while wearing data gloves;

[0075] Based on hand posture information data, a mapping relationship between the human hand and a five-finger robotic hand is established;

[0076] Based on hand posture information data, generative adversarial networks are used to analyze hand posture, construct a cooperative matrix, and generate a grasping posture according to different grasping objects.

[0077] The motion capture system identifies the object to be grasped. Based on the mapping relationship between the human hand and the five-finger robotic hand, the system matches the closest preset posture to the current pre-grasping posture of the five-finger robotic hand. The inverse dynamics control method based on quadratic programming is used to control the grasping of the five-finger robotic hand.

[0078] The process of using the inverse dynamics control method based on quadratic programming includes: constructing the motion equations and constraint equations of the five-finger manipulator, introducing a cost function, and defining the desired acceleration of the fingertips of the five-finger manipulator; calculating the acceleration of the fingertips of the five-finger manipulator based on the desired acceleration and feedback terms, with the goal of minimizing the cost function; and controlling the five-finger manipulator's gripping according to the torque required by the joint and the acceleration.

[0079] The technical solution of this embodiment will be described in detail below with reference to the accompanying drawings, such as... Figure 1 As shown:

[0080] Step 1: Mapping the human hand's grasping posture to a five-fingered robotic hand to achieve human-like collaborative grasping, mainly including the following steps:

[0081] Step (1-1): As Figure 5 As shown, 1 represents the motion capture system, 2 represents the five-fingered robotic hand, 3 represents the cylindrical object being grasped, 4 represents the spherical object being grasped, 5 represents the cubic object being grasped, 6 represents the disc object being grasped, 7 represents the experimental platform, and 8 represents the data glove. Under the 3D motion capture system 1, a world coordinate system needs to be established first to record the subject's hand posture when grasping different objects. The subject wears a data glove and grasps various objects under the motion capture system, such as bananas, apples, balls, pens, books, cylinders, etc., a total of 18 objects. Hand posture information is obtained by recording the positions of each finger joint and the wrist's position in the world coordinate system during all grasping postures. During the recording process, attention needs to be paid to the angle position of the joint connecting each finger and palm, because the active degrees of freedom of the five-fingered robotic hand to be mapped are provided by the joints connecting the fingers and palm, and the thumb also has an active degree of freedom at the wrist. The recorded hand posture information will become the data source for subsequent principal component analysis based on generative adversarial networks. To improve the results of component analysis, it is also necessary to perform appropriate preprocessing and cleaning of the data, remove invalid or abnormal data, and normalize the data.

[0082] Step (1-2): Calculate the synergistic effect subspace to reduce the dimensionality of the dataset. To calculate the synergistic effect subspace, a principal component analysis algorithm based on generative adversarial networks is proposed. The basic idea is to adjust the input vector of the generator network so that the distribution of the output samples of the generator network closely matches the distribution of the original data, while preserving the information of the principal components. The process is as follows: Figure 2 As shown.

[0083] Step (1-2-1): First, construct the generator network. This embodiment uses a convolutional neural network to construct the generator network. The generator network maps the input latent variables to the space of hand pose data and outputs the generated data. The input latent variables are the data obtained after augmenting the preceding hand pose information.

[0084] Step (1-2-2): The next step is to construct the discriminator network. In this embodiment, a convolutional neural network is used to construct the discriminator network. The discriminator network is mainly used to distinguish between real data and data generated by the generator network, and outputs a probability value representing the probability that the input data is real data.

[0085] Step (1-2-3): Obtain the intermediate layer features of the discriminator network. Input the hand pose data samples into the discriminator network and obtain the feature vectors of its intermediate layers.

[0086] Step (1-2-4): Calculate the variance of the principal components. For the eigenvectors of the intermediate layer, calculate their covariance matrix and perform eigenvalue decomposition to obtain the eigenvectors and eigenvalues, which are the variances of the principal components and their corresponding values.

[0087] Step (1-2-5): Optimize the input vector. By optimizing the input vector of the generator network, the distribution of the data samples generated by the generator network is made to fit the distribution of the original data as closely as possible, while retaining the information of the principal components. Here, optimization algorithms such as gradient descent are used to iteratively optimize the input vector of the generator network to minimize the difference between the generated data and the real data.

[0088] Step (1-2-6) applies principal component analysis. The optimized generator network can generate a large amount of hand pose principal component data. This data can be used for principal component analysis to obtain the weights of each principal component and the reconstructed hand pose, enabling automated recognition, classification, and transformation of hand poses.

[0089] Using steps (1-2): Analyze the acquired hand grasping data, and based on this, convert the relevant variables of the 6 degrees of freedom of the finger joints into a small number of unrelated variables n, i.e., the synergistic effect in this embodiment, where n≤3, and the component of n degrees of freedom should be greater than or equal to 90%. The basis vector of the unrelated variable n is E={m1,m2,...,m n} represents the collaborative weight values ​​α of n variables. i ,i=1,2,3,...,n, such as Figure 4As shown, the initial posture of the five-finger robotic hand is in the middle. The Syergy2 from the lower left to the upper right represents the hand posture generated along the synergy coefficient 2. The Syergy1 from the bottom to the top in the middle represents the hand posture generated along the synergy coefficient 1. The Syergy3 from the lower right to the upper left represents the hand posture generated along the synergy coefficient 3.

[0090] Steps (1-3): Establish a mapping matrix between the human hand and the robotic hand, and establish a mapping relationship between the joint data obtained through the data glove and the five-fingered robotic hand:

[0091]

[0092] Where J = [J1, J2, J3, J4, J5, J6] T This indicates the joint angles of the six joints in a five-finger robotic hand (the metacarpophalangeal joints of the index, middle, ring, and little fingers, the middle joint of the thumb, and the joint near the wrist). This represents the mechanical offset of the joint angle when the motor is set to zero, also known as the initial position of the five-fingered robotic hand. k = diag(ki), i = 1, 2, ..., 6 represents the mapping matrix from human hand movements to the five-fingered robotic hand, and S = [S1, S2, S3, S4, S5, S6]. T This refers to motion data collected by a person wearing data gloves.

[0093] Steps (1-4): The pre-grasp posture is generated through the constructed cooperative effect subspace and cooperative weights. Cooperative control relies on the fact that for each grasping posture c... i Approximating the pose coordination effect subspace by appropriately selecting suitable coordination weights.

[0094]

[0095] The motion capture system identifies the object to be grasped and informs the five-finger robotic hand. The hand opens as it reaches out to grasp the object. Using equation (2), the system determines the grasping posture closest to the preset posture and sends it to the five-finger robotic hand. When the hand approaches the object, it closes, achieving a grasp. The control algorithm for the entire grasping process of the five-finger robotic hand adopts the content of Part Two.

[0096] Step 2: Regarding the grasping control method of the five-finger robotic arm, in this embodiment, an inverse dynamics controller based on quadratic programming is constructed. By updating the cost function, system stability is ensured and the accuracy of the ID model is effectively improved. The control flow diagram is as follows: Figure 3As shown, this method can overcome problems such as model inaccuracy and large model mismatch by combining slow ID model learning with fast online adaptive control law in the task space to adjust the cost function of QP. The implementation steps are as follows.

[0097] Step (2-1): Establish the motion equations and constraint equations for the five-fingered robotic hand:

[0098]

[0099] Where, q∈R j Represent the state of the system (j represents the degree of freedom of the linkage of the five-fingered manipulator), x∈R l It is a cascade of the attitudes (including position and orientation) of all contact points in Cartesian space, M(q)∈R j×j Represents the inertia matrix. S represents the sum of gravity, centrifugal force, and Coriolis force, where S∈R j×j It is a selection matrix where the first 6 rows are all zeros, and the rest are identity matrices. It is a cascade of all contact moment vectors (in world coordinates). It is a concatenation of the Jacobian matrices of all contact points, τ∈R j It is a vector containing the control torques of all degrees of freedom of the system.

[0100] Step (2-2): Modify formula (3) to obtain formula (4), by giving a state. Then the equation of motion is relative to It is linear.

[0101]

[0102] definition The ID formula can be expressed as a QP-based control problem.

[0103]

[0104] Among them, we optimized the form as follows: The cost function, and θ is related to constraint A. E ,a E A I ,a I It is problem-specific. Therefore, in the equation, G = A T A,g=-A T b. Both A and b are column vectors, represented by the following formula, where each row emphasizes a specific desired behavior with a weight of w. i .

[0105]

[0106] In addition, we transform the equations of motion into equality constraints (A E and a E Furthermore, it transforms joint limits and other constraints (such as friction cone or pressure center constraints) into inequality constraints (A). I and a I Then, the desired acceleration (QP task objective) of the end effector is defined by appropriately filling in G and g.

[0107] Step (2-3): Solve for the input of the fingertips of the five-finger robotic hand. For the acceleration of the fingertips of a five-finger robotic hand The calculation method is as follows:

[0108]

[0109] in, For the desired acceleration of the distal joint, This refers to the feedback term of the closed-loop control circuit. In this embodiment, model reference adaptive control is used instead of the traditional control scheme, which can be modulated online. This follows the dynamic system, thereby controlling the transient behavior of the error signal and ensuring the stability of the system on the feedback line through an adaptive law.

[0110] Steps (2-4): For Solving for . Definition This describes the state of the end joints of a five-fingered robotic hand in the task space. The desired end state. Consider a general case, taking equation (8) as the desired reference model that the finger end joints of a five-finger robotic hand need to follow.

[0111]

[0112] Where r(t) is a bounded adjustment signal, B r C r r(t) is the design parameter of the reference model, and the selection of the parameter must make the reference model (8) stable. From the Lyapunov point of view, we have t→∞, ||ζ r -ζ d ||→0.

[0113] Since ζ is a series of the position and velocity of the end-joint, a dynamic model of the end-joint can be established to control the robot's state, and the values ​​in equation (7) can be used to control the robot's state. As input, establish the dynamic model of the end effector:

[0114]

[0115] in, Parameters B and C are both unknown and determined by the adaptive law. F(·)∈R n , being a smooth function, is approximated by an adaptive RBF neural network. The purpose of equations (8) and (9) is to find a solution for equation (7).

[0116] To ensure that the system state can smoothly track the reference model (8), the following control rules are proposed for the dynamic model of (9):

[0117] υ=ψ r ζ+ψ r r(t)+ψ φ φ(e) (10)

[0118] Where, ψ ζ ,ψ r ,ψ φ All parameters are online approximations; the adaptive controller in the task space will use ζ, ζ d As input, output Input QP optimization.

[0119]

[0120] in, It is an approximation matrix of equation (10) and follows the following adaptive law:

[0121]

[0122] Among them, Λ ζ ,Λ r ,Λ φ It is a positive definite matrix that adjusts the convergence speed of the adaptive gain. As a necessary and sufficient stability condition for the reference model (5), P and C r satisfy

[0123] Steps (2-5): Learning the inverse dynamics model. ID's task is to provide a model h. * This model can be applied to the system under specific conditions of the robot to provide the torque required for certain joints.

[0124]

[0125] Equation (13) is a classic restatement of equation (3). The purpose is to treat the equation as a data-driven model to be learned, which is a supervised learning task. Any suitable learning algorithm can be used. Since Gaussian process regression has good generality and is suitable for learning with few data samples, this embodiment uses Gaussian process regression for learning the inverse dynamics model.

[0126] In summary, the execution steps of the inverse dynamics controller based on quadratic programming are as follows:

[0127] Step (2-5-1): Given the task objective: x d (t),

[0128] Step (2-5-2): Configure the adaptive controller and model learning program.

[0129] Step (2-5-3): Perform once at each sampling time.

[0130] Step (2-5-3-1): Calculate the reference acceleration using a nonlinear adaptive controller.

[0131] Step (2-5-3-2): According to Calculate the cost function of QP.

[0132] Step (2-5-3-3): Use the updated cost function and the ID model obtained through linearization. The torque τ is obtained from QP.

[0133] Steps (2-5-3-4): Apply torque to the robot and collect data.

[0134] Step (2-5-3-5): Update the adaptive controller at each sampling time.

[0135] Step (2-5-4): Learn the ID model using a Gaussian process and all the collected data.

[0136] Step (2-5-5): Return to step (2-5-3) until the system converges.

[0137] Example 2

[0138] This embodiment provides a five-finger robotic gripper control system with teaching and learning capabilities.

[0139] A five-finger robotic hand grasping control system with teach-and-learn capabilities includes:

[0140] The teaching module is configured to: establish a world coordinate system under a 3D motion capture system, and collect hand posture information data when a human hand grasps different objects while wearing data gloves;

[0141] The mapping module is configured to establish a mapping relationship between a human hand and a five-finger robotic hand based on hand posture information data.

[0142] The pose generation module is configured to: analyze hand poses using generative adversarial networks based on hand pose information data, construct a cooperative matrix, and generate pre-grasping poses according to different grasping objects;

[0143] The control module is configured to: identify the object to be grasped through the motion capture system; match the closest preset posture to the current pre-grasping posture of the five-finger robotic hand based on the mapping relationship between the human hand and the five-finger robotic hand; and control the grasping of the five-finger robotic hand using an inverse dynamics control method based on quadratic programming.

[0144] The process of using the inverse dynamics control method based on quadratic programming includes: constructing the motion equations and constraint equations of the five-finger manipulator, introducing a cost function, and defining the desired acceleration of the fingertips of the five-finger manipulator; calculating the acceleration of the fingertips of the five-finger manipulator based on the desired acceleration and feedback terms, with the goal of minimizing the cost function; and controlling the five-finger manipulator's gripping according to the torque required by the joint and the acceleration.

[0145] It should be noted that the teaching module, mapping module, attitude generation module, and control module described above are the same examples and application scenarios implemented in the steps of Embodiment 1, but are not limited to the content disclosed in Embodiment 1. It should also be noted that these modules, as part of the system, can be executed in a computer system such as a set of computer-executable instructions.

[0146] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A five-fingered hand grasping control method with teaching-learning capability, characterized in that, The application relates to a method for generating a pre-grasping posture of a five-fingered robot hand. The method comprises the following steps: A world coordinate system is established under a three-dimensional motion capture system, and hand posture information data of a human hand when the human hand grasps different objects is collected under the condition that a data glove is worn; A mapping relationship between the human hand and the five-fingered robot hand is established based on the hand posture information data; A generative adversarial network is used to analyze a hand posture based on the hand posture information data, a coordination matrix is constructed, and a pre-grasping posture is generated according to different grasped objects; The process of using the generative adversarial network comprises the following steps: the hand posture information data is expanded to construct input data; a generator is used to obtain generated data based on the input data; and a discriminator is used to continuously optimize the generator network with the minimum difference between the generated data and the hand posture information data as the target; wherein the generative adversarial network comprises the generator and the discriminator; The process of using the generative adversarial network to analyze the hand posture and construct the coordination matrix comprises the following steps: the hand posture principal component data is obtained by using the optimized generator network based on the hand posture information data; and principal component analysis is performed on the hand posture principal component data to construct the coordination matrix, wherein the coordination matrix is the product of coordination weights and a coordination effect subspace; An object to be grasped is recognized by a motion capture system, the closest preset posture is matched to the current pre-grasping posture of the five-fingered robot hand based on the mapping relationship between the human hand and the five-fingered robot hand, and a quadratic programming-based inverse dynamics control method is used to control the five-fingered robot hand to grasp; The process of using the quadratic programming-based inverse dynamics control method comprises the following steps: a motion equation and a constraint equation of the five-fingered robot hand are constructed, a cost function is introduced, and the expected acceleration of a finger tip joint of the five-fingered robot hand is defined; the acceleration of the finger tip joint of the five-fingered robot hand is calculated based on the expected acceleration and a feedback term with the minimum cost function as the target; and the five-fingered robot hand is controlled to grasp according to the required torque of the joint and the acceleration. wherein, represents the state of the system, represents the link degrees of freedom of the five-fingered hand, is the concatenation of the poses of all contact points in Cartesian space, represents the inertia matrix, represents the sum of gravitational, centrifugal and Coriolis forces, is a selection matrix, is the concatenation of all contact force torque vectors, is the concatenation of Jacobians of all contact points, is the vector containing the control torques of all degrees of freedom of the system; The motion equation and the constraint equation are The cost function to be optimized is where , and denote equality constraints, and denote inequality constraints, and , and are column vectors of desired behavior.

2. The five-fingered hand grasping control method with teaching-learning ability according to claim 1, wherein, The mathematical model expression of the quadratic programming is 3. The five-fingered hand grasping control method with teaching-learning capability according to claim 1, wherein, The hand posture information data comprises the positions of all grasping postures of finger joints, the position of a wrist in a world coordinate system and the angular position of a finger and palm connecting joint.

4. The five-fingered hand grasping control method with teaching-learning ability according to claim 3, wherein, The solving process of the feedback term comprises the following steps: an expected reference model to be followed by the finger tip joint of the five-fingered robot hand and a dynamics model of the end joint are established; and the dynamics model is made to follow the expected reference model according to a preset control rule to solve the feedback term. wherein is a bounded regulating signal, is a design parameter of the reference model, from the Lyapunov point of view, then , ; The expected reference model is Or wherein , the parameters are unknown and determined by an adaptive law; is a smooth function approximated by an adaptive RBF neural network, .

5. The five-fingered hand grasp control method with teaching-learning capability according to claim 4, wherein, The dynamics model of the end joint is wherein, are online approximated parameters, an adaptive controller of the task space will as input, and will output input QP optimization; where is an approximated matrix of the above equation and follows the following adaptive law: wherein, is a positive definite matrix that adjusts the adaptive gain convergence speed; as a sufficient and necessary stability condition of the cost function, and satisfies .

6. A five-fingered hand grasping control system with the ability of teaching learning, characterized in that, The control rule is The application further relates to a device for generating a pre-grasping posture of a five-fingered robot hand. The device comprises the following modules: A teaching module configured to establish a world coordinate system under a three-dimensional motion capture system, and collect hand posture information data of a human hand when the human hand grasps different objects under the condition that a data glove is worn; A mapping module configured to establish a mapping relationship between the human hand and the five-fingered robot hand based on the hand posture information data; A posture generation module configured to use a generative adversarial network to analyze a hand posture based on the hand posture information data, construct a coordination matrix, and generate a pre-grasping posture according to different grasped objects. The process of adopting the generative adversarial network comprises: expanding the hand posture information data to construct input data; adopting a generator based on the input data to obtain generated data; and adopting a discriminator to continuously optimize the generator network with the objective of minimizing the difference between the generated data and the hand posture information data; wherein the generative adversarial network comprises the generator and the discriminator; The process of adopting the generative adversarial network to analyze the hand posture and construct a coordination matrix comprises: adopting the optimized generator network to obtain hand posture principal component data based on the hand posture information data; and performing principal component analysis on the hand posture principal component data to construct a coordination matrix, wherein the coordination matrix is the product of coordination weights and a coordination effect subspace; The control module is configured to: identify an object to be gripped by a motion capture system, match the closest preset posture to the current pre-gripping posture of the five-fingered robot hand based on the mapping relationship between the human hand and the five-fingered robot hand, and control the five-fingered robot hand to grip by using a quadratic programming-based inverse dynamics control method; The process of adopting the quadratic programming-based inverse dynamics control method comprises: constructing a motion equation and a constraint equation of the five-fingered robot hand, introducing a cost function, and defining the expected acceleration of the finger tip joint of the five-fingered robot hand; calculating the acceleration of the finger tip joint of the five-fingered robot hand based on the expected acceleration and a feedback term with the objective of minimizing the cost function; and controlling the five-fingered robot hand to grip according to the required torque of the joint and the acceleration; The motion equation and the constraint equation are wherein, represents the state of the system, represents the link degrees of freedom of the five-fingered hand, is the concatenation of the poses of all contact points in Cartesian space, represents the inertia matrix, represents the sum of gravitational, centrifugal and Coriolis forces, is a selection matrix, is the concatenation of all contact force torque vectors, is the concatenation of Jacobians of all contact points, is the vector containing the control torques of all degrees of freedom of the system; The mathematical model expression of the quadratic programming is The cost function to be optimized is where , and denote equality constraints, and denote inequality constraints, and , and are column vectors of desired behavior.

Images