Dexterous hand control method, device, system and medium

Abstract

The present application provides a dexterous hand control method, device, system and medium, which relate to the field of dexterous hand technology. The dexterous hand control method comprises: obtaining the current state quantity of the fingers of the dexterous hand and the reference state quantity of the fingers at a future moment; according to the current state quantity and the reference state quantity, using the continuous generalized minimum residual method to iteratively solve the optimization problem of the fingers to obtain the target control quantity of the fingers at a future moment, and the optimization problem is determined according to the dynamic model of the fingers; the fingers are controlled according to the target control quantity. The present application improves the control accuracy of the fingers of the dexterous hand by solving the optimization problem, and improves the solution rate of the optimization problem by the continuous generalized minimum residual method, thereby improving the efficiency of the control of the fingers of the dexterous hand, thereby improving the accuracy and response speed of the control of the dexterous hand.

Description

Technical Field

[0001] The present application relates to the field of dexterous hand technology, and in particular to a dexterous hand control method, device, system and medium. Background Art

[0002] With the development of robotics technology, dexterous hands are increasingly used in various complex operating tasks. How to accurately and smoothly control the finger joint angles of dexterous hands is crucial to achieving fine operations of dexterous hands in complex operating tasks.

[0003] When a dexterous hand performs delicate operations, such as picking up a small object, the control amount of its finger joints determines the accuracy of the finger joint angles and the degree of control stability. In related technologies, iterative linearization methods are used to deal with the motion planning problem of robot control systems. Due to the characteristics of multiple degrees of freedom and strong nonlinearity of dexterous hands, iterative linearization methods cannot meet the high response speed and high control accuracy requirements of dexterous hands in complex operation scenarios.

[0004] Therefore, the response speed and accuracy of dexterous hand control need to be improved. Summary of the invention

[0005] The present application provides a dexterous hand control method, device, system and medium to solve the problem of how to improve the response speed and accuracy of dexterous hand control.

[0006] In a first aspect, the present application provides a method for controlling a dexterous hand, comprising: obtaining a current state quantity of a finger of the dexterous hand and a reference state quantity of the finger at a future moment, the current state quantity being related to the joint angle of the finger at the current moment, the reference state quantity being related to the joint angle of the finger at a future moment, the future moment being located after the current moment; based on the current state quantity and the reference state quantity, a continuous generalized minimum residual method is used to iteratively solve an optimization problem of the finger to obtain a target control quantity of the finger at the future moment, the optimization problem being determined based on a dynamic model of the finger; and according to the target control quantity, the joint angle of the finger is controlled.

[0007] In some embodiments, the optimization problem includes an objective function and constraints, the constraints include equality constraints, the equality constraints include dynamic constraint formulas obtained based on a dynamic model, and the dynamic constraint formulas are used to represent the dynamic relationship between the state quantity of the finger and the control quantity of the finger.

[0008] In some embodiments, based on the current state quantity and the reference state quantity, the optimization problem of the finger is iteratively solved using the continuous generalized minimum residual method to obtain the target control quantity of the finger at the future moment, including: inputting the reference state quantity into the objective function, transforming the optimization problem according to the constraints and the objective function, and obtaining the control quantity solution equation; based on the current state quantity, the control quantity solution equation is iteratively solved using the continuous generalized minimum residual method to obtain the target control quantity.

[0009] In some embodiments, the constraints also include inequality constraints, and the optimization problem is transformed according to the constraints and the objective function to obtain the control quantity solution equation, including: converting the inequality constraint into a corresponding penalty term, adding the penalty term to the objective function, and obtaining an equivalent optimization problem corresponding to the optimization problem, the equivalent optimization problem including the objective function and the equality constraint with the penalty term added; establishing a Hamiltonian function corresponding to the equivalent optimization problem; and determining the control quantity solution equation according to the equality constraint and the Hamiltonian function.

[0010] In some embodiments, the inequality constraint is converted into a corresponding penalty term, including: determining the penalty term by processing the inequality constraint using a barrier function method; establishing a Hamiltonian function corresponding to the equivalent optimization problem by processing the equality constraint in the equivalent optimization problem, including: processing the equality constraint in the equivalent optimization problem using a Lagrange multiplier method to obtain a Hamiltonian function, wherein the Hamiltonian function includes an objective function with a penalty term added and an equality constraint weighted by a Lagrange multiplier; determining a control quantity solution equation based on the equality constraint and the Hamiltonian function, including: respectively deriving the state quantity of the finger and the control quantity of the finger in the Hamiltonian function to obtain derivative formulas corresponding to the state quantity of the finger and the control quantity of the finger respectively; constructing a control quantity solution equation based on the derivative formula and the equality constraint.

[0011] In some embodiments, according to the current state quantity, the control quantity solution equation is iteratively solved by the continuous generalized minimum residual method to obtain the target control quantity, including: using Newton's method to determine the initial approximate solution of the control quantity solution equation; determining the initial residual vector based on the initial approximate solution; in a low-dimensional subspace, according to the initial approximate solution, the initial residual vector and the current state quantity, the approximate solution of the control quantity solution equation is iteratively solved until the iteration end condition is met to obtain the target control quantity, and the iteration end condition includes: the number of iterations is greater than or equal to the number threshold, or the residual vector during the iteration process is less than the vector threshold.

[0012] In a second aspect, the present application provides a dexterous hand control device, comprising: an information acquisition module, used to obtain the current state quantity of the fingers of the dexterous hand and the reference state quantity of the fingers at a future moment, the current state quantity is related to the joint angle of the fingers at the current moment, the reference state quantity is related to the joint angle of the fingers at a future moment, and the future moment is located after the current moment; a control quantity optimization solution module, used to iteratively solve the optimization problem of the fingers according to the current state quantity and the reference state quantity, using a continuous generalized minimum residual method to obtain the target control quantity of the fingers at the future moment, and the optimization problem is determined according to the dynamic model of the fingers; a control module, used to control the joint angle of the fingers according to the target control quantity.

[0013] In a third aspect, the present application provides a dexterous hand control system, comprising: a processor, disposed on the dexterous hand or separately from the dexterous hand, for implementing the dexterous hand control method as described in the first aspect of the present application; and a dexterous hand, for performing corresponding actions according to the control of the processor.

[0014] In a fourth aspect, the present application provides a computer-readable storage medium, in which computer program instructions are stored. When the computer program instructions are executed by a processor, the dexterous hand control method described in the first aspect of the present application is implemented.

[0015] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the dexterous hand control method as described in the first aspect of the present application.

[0016] The dexterous hand control method, device, system and medium provided by the present application optimize and solve the control amount of the dexterous hand fingers at the future moment by iteratively solving the optimization problem, thereby improving the accuracy of the control amount of the dexterous hand fingers, reducing the steady-state error of the dexterous hand finger control, and making the movement of the dexterous hand fingers smoother; by adopting the continuous generalized minimum residual method to iteratively solve the optimization problem, the amount of calculation in the solution process is reduced, the solution efficiency of the optimization problem is improved, and the control frequency of the dexterous hand finger movements is increased. Thus, the accuracy and response speed of the dexterous hand control are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0018] Figure 1 This is an example diagram of the structure of the index finger in a dexterous hand;

[0019] Figure 2 The process of the dexterous hand control method provided in the embodiment of the present application Figure 1 ;

[0020] Figure 3 The process of the dexterous hand control method provided in the embodiment of the present application Figure 2 ;

[0021] Figure 4 A schematic diagram of the structure of a dexterous hand control device provided in an embodiment of the present application;

[0022] Figure 5 This is a structural block diagram of a dexterous hand control system provided in one embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0024] Different from traditional end effectors, dexterous hands have multiple degrees of freedom and strong nonlinearity. Traditional control methods are easily affected by the nonlinearity of dexterous hand control, the coupling between finger joints, and external disturbances. They are not capable of precise control and rapid response of finger joints, resulting in low control accuracy and response speed of dexterous hands.

[0025] In order to improve the accuracy and response speed of dexterous hand control, the embodiments of the present application provide a dexterous hand control method, device, system and medium. In the dexterous hand control method, the control problem of the fingers of the dexterous hand is converted into an optimization problem of the control amount of the fingers. By iteratively solving the optimization problem, the optimal control amount of the fingers is obtained, and smoother and more accurate angle control of the finger joints is achieved; the optimization problem is solved by the continuity generalized minimum residual method, which reduces the amount of calculation for solving the optimization problem, improves the efficiency of solving the optimization problem, and improves the control frequency of the finger movements of the dexterous hand. Thereby, the accuracy and response speed of the dexterous hand control are improved, the high-precision and high-response speed requirements of the dexterous hand in complex operation scenarios are met, and the task completion effect of the dexterous hand is improved.

[0026] Among them, the above-mentioned dexterous hand control method can be executed by a control unit in the dexterous hand, such as a microcontroller unit (MCU) deployed in the dexterous hand.

[0027] Among them, the technical means adopted by the devices, equipment and media and the technical effects produced can refer to the technical means adopted and the technical effects produced by the above-mentioned scheme, and will not be repeated here.

[0028] First, the fingers, joints, and joint angles of the dexterous hand are introduced.

[0029] Take the index finger in the dexterous hand as an example. Figure 1 This is an example diagram of the structure of the index finger in a dexterous hand. Figure 1 As shown, the index finger includes two joints: a first joint 101 and a second joint 102. The first joint angle θ1 and the second joint angle θ2 are active degrees of freedom, and the third joint angle θ3 is a passive degree of freedom. Among them, the active degree of freedom refers to the degree of freedom that can be controlled by a drive device to move independently, and the passive degree of freedom refers to the degree of freedom that changes position and / or posture under the action of external force or its own structure without the control of a drive device.

[0030] The index finger also includes a finger base 104 and a finger tip 103. The first joint angle θ1 is the angle between the first joint 101 and the finger base 104 relative to the first coordinate system (i.e. Figure 1 The X-axis (i.e. Figure 1 The second joint angle θ2 is the angle between the first joint 101 and the second joint 102 relative to the second coordinate system (ie Figure 1 The X-axis (i.e. Figure 1 The third joint angle θ3 is the angle between the line between the second joint 102 and the finger tip 103 relative to the third coordinate system (i.e. Figure 1 The X-axis in the X3Y3 coordinate system (i.e. Figure 1 The angle between the X3 axis and the line between the second joint 102 and the first joint 101 is parallel.

[0031] It is understandable that the structure of the middle finger, ring finger and pinky finger of the dexterous hand can refer to the above index finger result, and also include two finger joints and two joint angles belonging to active degrees of freedom. The thumb of the dexterous hand can include one finger joint and one joint angle belonging to active degrees of freedom.

[0032] Next, Figure 1 Taking the finger structure shown as an example, the dexterous hand control method is introduced through specific embodiments and drawings related to the specific embodiments.

[0033] Figure 2 The process of the dexterous hand control method provided in the embodiment of the present application Figure 1 .like Figure 2As shown, the dexterous hand control method provided in the embodiment of the present application includes:

[0034] S201, obtain the current state quantity of the fingers of the dexterous hand and the reference state quantity of the fingers at a future moment, the current state quantity of the fingers is related to the joint angle of the fingers at the current moment, the reference state quantity of the fingers at a future moment is related to the joint angle of the fingers at the future moment, and the future moment is located after the current moment.

[0035] The current state quantity of the finger refers to the state quantity of the finger at the current moment, and the reference state quantity of the finger at the future moment refers to the ideal state quantity required for the finger to perform a task at the future moment.

[0036] Optionally, the current state quantity of the finger includes the joint angle of the finger at the current moment, and the reference state quantity of the finger at the future moment includes the reference joint angle of the finger at the future moment.

[0037] Furthermore, the joint angles included in the current state quantity of the finger and the joint angles included in the reference state quantity of the finger belong to active degrees of freedom.

[0038] Optionally, the current state quantity of the finger includes the derivative of the joint angle of the finger at the current moment, and the reference state quantity of the finger at the future moment includes the derivative of the reference joint angle of the finger at the future moment.

[0039] Furthermore, the derivative of the joint angle of the finger at the current moment includes the first-order derivative of the joint angle; the derivative of the reference joint angle of the finger at the future moment includes the first-order derivative of the reference joint angle.

[0040] by Figure 1 Taking the index finger shown as an example, the current state quantity of the finger includes the first joint angle, the first-order derivative of the first joint angle, the second joint angle and the first-order derivative of the second joint angle.

[0041] In this embodiment, the current posture information of the dexterous hand can be obtained, and the current posture information includes the current state quantity of the fingers of the dexterous hand; the motion path planning information of the dexterous hand can be obtained, and the motion path planning information includes multiple groups of posture information of the dexterous hand during the task process, one group of posture information corresponds to a future moment, and one group of posture information includes the reference state quantity of the fingers of the dexterous hand, that is, the reference state information of the fingers at the future moment corresponding to the group of posture information. Therefore, the current state quantity of the fingers of the dexterous hand can be obtained from the current posture information of the dexterous hand; the reference state information of the fingers of the dexterous hand at the future moment can be obtained from the motion planning information of the dexterous hand.

[0042] Optionally, the dexterous hand includes multiple fingers. Considering that the task performed by the dexterous hand may involve each finger of the dexterous hand or some fingers of the dexterous hand, the current state quantity of the fingers related to the target task may be obtained; the reference state information of the fingers related to the target task at a future moment may be obtained. The target task may be a task to be executed or a task being executed.

[0043] Optionally, considering that the task execution time of the target task may include multiple future moments, the reference state information of the fingers of the dexterous hand at N future moments can be obtained, where N is greater than 1, so as to obtain the target control amount of the fingers of the dexterous hand at N future moments by solving the optimization problem.

[0044] Furthermore, the task execution time of the dexterous hand (i.e., the time to execute the target task) may include M future moments, and the reference state information of the fingers of the dexterous hand at N consecutive future moments following the current moment may be obtained, where M is greater than N. After determining the target control amount of the finger at N future moments, the target control amount of the finger at the next moment of the current moment is obtained from the target control amounts at the N future moments, and the joint angle of the finger is controlled according to the target control amount at the next moment. In this way, a round of control process of the joint angle of the finger is completed. Multiple rounds of control of the joint angle of the finger can be achieved according to this process. Thus, through the rolling optimization process, the optimal control amount of the finger can be obtained in each round, thereby improving the control effect of the joint angle of the finger of the dexterous hand.

[0045] As an example, the target task needs to be completed in 20 moments. When solving the optimal problem for the first time, the target control quantities corresponding to the 1st to 10th moments can be solved, and the fingers of the dexterous hand are controlled according to the control quantity at the first moment, and the target control quantities corresponding to the 2nd to 10th moments are not used; when solving the optimal problem for the second time, the target control quantities corresponding to the 2nd to 11th moments are solved, and so on.

[0046] S202, based on the current state quantity and the reference state quantity, the continuous generalized minimum residual method is used to iteratively solve the optimization problem of the finger to obtain the target control quantity of the finger at the future moment. The optimization problem is determined based on the dynamic model of the finger.

[0047] Among them, the target control quantity includes the control torque of the finger joints.

[0048] by Figure 1 Taking the index finger as an example, the target control amount of the finger at the future moment includes the target control torque of the first joint at the future moment and the target control torque of the second joint at the future moment.

[0049] Among them, as an improved least squares method, the continuity generalized minimal residual method (GMRES) can effectively deal with the unstable problem of nonlinear equations and has a certain robustness. Using the continuity generalized minimal residual method for iterative solution of finger optimization problems can avoid the huge inverse matrix calculation in solving optimization problems and improve the solution efficiency.

[0050] In this embodiment, the dynamic model of the finger can be constructed by analyzing the motion characteristics and force relationship of the finger of the dexterous hand. For example, the dynamic model of the finger is constructed by comprehensively considering the mass, inertia, force conditions, etc. of each component of the finger. Based on the dynamic model of the finger, an optimization problem of the finger can be pre-constructed, and the optimization problem is used to optimize the target control amount of the finger at a future moment. When solving the target control amount at a certain future moment, the current state quantity and the reference state quantity can be used as input data of the optimization problem, and the continuous generalized minimum residual method can be used to iteratively solve the optimization problem until the iteration end condition is met, and the solution obtained by the last iterative solution of the optimization problem is obtained, that is, the target control amount of the finger at the future moment.

[0051] S203, controlling the joint angles of the fingers according to the target control amount.

[0052] In this embodiment, a control instruction may be generated according to the target control amount at the future moment, and the control instruction may be sent to the driver, and the driver may be instructed by the control instruction to control the joint angle of the finger at the future moment.

[0053] Optionally, the number of future moments is N, that is, N future moments, and the target control amount of the finger at the next moment of the current moment is obtained from the target control amount of the finger at the N future moments, and the joint angle of the finger is controlled according to the target control amount at the next moment. Wherein, N is greater than 1.

[0054] In the embodiment of the present application, the optimal control amount of the finger is obtained by iteratively solving the optimization problem of the control amount of the dexterous hand fingers, so as to achieve smoother and more accurate angle control of the finger joints; the optimization problem is solved by using the continuous generalized minimum residual method, which reduces the amount of calculation for solving the optimization problem, improves the efficiency of solving the optimization problem, and improves the control frequency of the dexterous hand finger movements. Thus, the control accuracy and response speed of the dexterous hand are improved, the high-precision and high-response speed requirements of the dexterous hand in complex operation scenarios are met, and the task completion effect of the dexterous hand is improved.

[0055] Below, corresponding embodiments are provided for the dynamic model of the fingers of a dexterous hand.

[0056] In some embodiments, the dynamic model of the finger is constructed based on the Lagrangian equation and the parameter information of the finger, wherein the parameter information of the finger includes the structural information of the finger and the state quantity information of the finger. Thus, the dynamic model of the finger is constructed by using the Lagrangian equation, which is more suitable for processing motion systems with multiple degrees of freedom, nonlinearity, coupling and other characteristics, while taking into account the structure and state quantity of the dexterous finger, effectively improving the accuracy of the dynamic model of the finger.

[0057] Among them, the prerequisites for constructing the dynamic model of the finger may include: 1. The finger is an ideal rigid structure, and the deformation of the finger is not considered; 2. The friction between the various parts of the finger is ignored; 3. It is assumed that each joint of the finger is a rigid body with uniform density; 4. Considering that the angle at the fingertip (i.e., θ3 in the aforementioned embodiment) is a passive degree of freedom, the first joint angle and the second joint angle are active degrees of freedom (the first joint angle and the second joint angle can refer to Figure 1 The fingers are simplified into a planar elbow-type robot arm.

[0058] Optionally, when the joint angles of the fingers include a first joint angle and a second joint angle, the dynamic model of the fingers of the dexterous hand is expressed as:

[0059] Formula (1)

[0060] in, represents the first joint angle, represents the first derivative of the first joint angle,

[0061] represents the second derivative of the first joint angle, represents the second joint angle, represents the first derivative of the second joint angle, represents the second-order derivative of the second joint angle, represents the control torque at the first joint, represents the control torque at the second joint.

[0062] in, and It is a coefficient related to the parameter information of the finger. It is an element in the inertia matrix in the Lagrange equation (representing the mass distribution of the finger and the influence of the mass distribution on the angular acceleration of the first joint angle and the angular acceleration of the second joint angle); It is the rate of change of the elements in the inertia matrix in the Lagrange equation with respect to the generalized coordinates of the finger. It is the Coriolis term coefficient related to the Coriolis force and centrifugal force, and is used to describe the nonlinear dynamic coupling effect caused by the change of the inertia matrix.

[0063] in, and The formula is as follows:

[0064] Formula (2)

[0065] in, represents the mass of the first finger segment (i.e. the segment between the first joint and the root of the finger), It represents the mass of the second finger segment (i.e. the finger segment between the first joint and the second joint);

[0066] Indicates the length of the first finger segment; represents the distance from the mass point of the first finger segment to the origin (the root of the finger), Indicates the distance from the mass point of the second finger segment to the origin (the base of the finger); represents the moment of inertia of the first finger segment about the first axis, the first axis passes through the center of mass of the first finger segment and points to the side (left side or right side) of the first finger segment; represents the moment of inertia of the second finger segment about the second axis, the second axis passes through the center of mass of the second finger segment and points to the side (left side or right side) of the second finger segment; represents the potential energy of the first finger segment, It indicates the potential energy of the second finger segment. It can be seen that the above parameter information of the finger is used in the process of constructing the dynamic model.

[0067] Furthermore, the dynamic model can be written in the following matrix form:

[0068] Formula (3)

[0069] Set the finger state to , the control amount of the finger is , then the above matrix form of the dynamic model can be converted into:

[0070] Formula (4)

[0071] in, represents the first-order derivative of the state quantity X, X(t) represents the state quantity of the finger at the future time t, and U(t) represents the control quantity of the finger at the future time t.

[0072] Below, corresponding embodiments are provided for the optimization problem.

[0073] In some embodiments, the optimization problem includes an objective function and constraints, the constraints include equality constraints, the equality constraints include a dynamic constraint formula obtained based on a dynamic model of the finger, and the dynamic constraint formula is used to express the dynamic relationship between the state quantity of the finger and the control quantity of the finger. Thus, the equality constraints in the optimization problem are designed based on the dynamic model of the finger, so that the optimization solution of the optimization problem satisfies the dynamic model of the finger, thereby improving the accuracy of solving the optimization problem.

[0074] In this embodiment, the objective function reflects the difference between the state quantity of the finger at the future moment and the reference state quantity of the finger at the future moment, and the difference between the control quantity of the finger at the future moment and the reference control quantity of the finger at the future moment. The optimization problem is solved, that is, the objective function is minimized under the constraint condition to reduce the difference between the state quantity of the finger at the future moment and the reference state quantity of the finger at the future moment, and the difference between the control quantity of the finger at the future moment and the reference control quantity of the finger at the future moment, and obtain the optimal control quantity of the finger at the future moment.

[0075] Optionally, the objective function is constructed by weighting the first difference result and the second difference result, wherein the first difference result is related to the following content: the difference between the state quantity of the finger at the future moment and the reference state quantity of the finger at the future moment; the second difference result is related to the following content: the difference between the control quantity of the finger at the future moment and the reference control quantity of the finger at the future moment.

[0076] Furthermore, the first difference result refers to the binary norm of the difference between the state quantity of the finger at the future moment and the reference state quantity of the finger at the future moment, and the second difference result refers to the binary norm of the difference between the control quantity of the finger at the future moment and the reference control quantity of the finger at the future moment.

[0077] Furthermore, the objective function is expressed as:

[0078] Formula (5)

[0079] Among them, X r (t) represents the reference state of the finger at the future time t, X(t) represents the state of the finger at the future time t; U r (t) represents the reference control amount of the finger at the future time t, U(t) represents the control amount of the finger at the future time t; Q and R represent the weight coefficient matrix, express The second norm of express The second norm of .

[0080] The reference control amount of the finger at the future time t may be a preset value.

[0081] Furthermore, the reference control amount of the finger at the future time t is 0, that is, it is hoped that the task can be completed without control, and it is required that the control amount of the finger does not need to change significantly, which would cause excessive burden on the driver.

[0082] When the reference control amount is 0, the objective function is optionally expressed as:

[0083] Formula (6)

[0084] in, Represents the change in the control amount of the finger at the future time t. By minimizing the objective function, the change range of the state amount of the finger at the future time t is reduced, the control amount of the finger at the future time t and the change range of the control amount are reduced, and the smoothness and stability of the finger movement are improved.

[0085] Optionally, the dynamic constraint formula is the above formula (4).

[0086] Optionally, the optimization problem also includes inequality constraints, and the equality constraints may include other equality constraints in addition to the dynamic constraint formulas.

[0087] The inequality constraints are expressed as:

[0088] Formula (7)

[0089] Furthermore, the inequality constraint is expressed as:

[0090] Formula (8)

[0091] Where q represents the joint angle (the joint angle of the finger in the state quantity at the future time t, which can be the first joint angle and / or the second joint angle), q min represents the minimum threshold corresponding to the joint angle, q max Indicates the maximum threshold corresponding to the joint angle; represents the first-order derivative of the joint angle, Indicates the minimum threshold corresponding to the first-order derivative of the joint angle, Indicates the maximum threshold corresponding to the first-order derivative of the joint angle; represents the second-order derivative of the joint angle, Indicates the minimum threshold corresponding to the first-order derivative of the joint angle, Indicates the maximum threshold corresponding to the first-order derivative of the joint angle; U min (t) represents the minimum threshold value of the finger control amount U(t) at the future time t, U max (t) represents the maximum threshold value corresponding to the control amount U(t) of the finger at the future time t.

[0092] Among them, other equality constraints except the dynamic constraint formula are expressed as:

[0093] Formula (9)

[0094] Furthermore, the equality constraint is expressed as:

[0095] Formula (10)

[0096] Among them, t s represents the end time of the future time t. The equality constraint expressed by formula (10) belongs to the terminal constraint, which refers to the conditions that the fingers of the dexterous hand need to meet at the end time.

[0097] Figure 3 Schematic diagram of a dexterous hand control method provided in an embodiment of the present application Figure 2 When the optimization problem includes an objective function and constraints, such as Figure 3 As shown, the dexterous hand control method provided in the embodiment of the present application includes:

[0098] S301, obtaining the current state quantity of the fingers of the dexterous hand and the reference state quantity of the fingers at a future moment.

[0099] The implementation principle and technical effects of S301 may refer to the aforementioned embodiments and will not be described in detail.

[0100] S302, inputting the reference state quantity of the finger at the future moment into the objective function, transforming the optimization problem according to the constraint conditions and the objective function, and obtaining the control quantity solution equation.

[0101] In this embodiment, in the objective function, the reference state quantity of the finger at the future time can be used as a reference for the state quantity of the finger at the future time, so that the state quantity of the finger at the future time is close to the reference state quantity of the finger at the future time during the optimization solution of the objective function. Therefore, the reference state quantity of the finger at the future time can be input into the objective function. Afterwards, in order to facilitate the solution of the control quantity, the optimization problem is transformed into a control quantity solution equation by integrating the constraint conditions with the objective function.

[0102] Optionally, when the objective function is expressed as formula (5) or formula (6), X in formula (5) or formula (6) is r (t) is the reference state of the input finger at the future moment.

[0103] In one possible implementation, the constraints include equality constraints and inequality constraints. In this case, Figure 3As shown, S302 includes S3021~S3023: S3021, converting the inequality constraints in the constraints into corresponding penalty terms, adding the penalty terms to the objective function, and obtaining an equivalent optimization problem corresponding to the optimization problem, the equivalent optimization problem includes the objective function with the penalty terms added and the equality constraints in the constraints; S3022, establishing the Hamiltonian function corresponding to the equivalent optimization problem; S3023, determining the control quantity solution equation according to the equality constraints and the Hamiltonian function in the constraints.

[0104] The constraint condition may include one or more inequality constraints. If the constraint condition includes multiple inequality constraints, each inequality constraint may be converted into a corresponding penalty term, and the penalty term corresponding to each inequality constraint may be included in the objective function.

[0105] In this implementation, by converting the inequality constraint in the optimization problem into a penalty term and then adding the penalty term to the objective function in the optimization problem, the inequality constraint and the objective function are integrated to obtain a changed optimization problem. In the changed optimization problem, the objective function integrates the penalty term corresponding to the inequality constraint, and only the equality constraint remains in the constraint condition. The changed optimization problem is the equivalent optimization problem corresponding to the original optimization problem. Afterwards, according to the equivalent optimization problem, the corresponding Hamiltonian function is established, and the equation for solving the control quantity is obtained by transforming the equality constraint in the constraint condition and the Hamiltonian function. Thus, the constraint condition and the objective function are integrated by means of the penalty term, the Hamiltonian function, etc., and the optimization problem is transformed into the equation for solving the control quantity.

[0106] Optionally, in S3021, the inequality constraint in the constraint condition is processed by using a barrier function method to determine the penalty term corresponding to the inequality constraint. The barrier function method is also called the internal penalty function method (or the internal penalty function method). When the constrained term in the inequality constraint approaches the constraint boundary, the value of the penalty term corresponding to the inequality constraint increases; when the constrained term in the inequality constraint is far away from the constraint boundary, the value of the penalty term corresponding to the inequality constraint gradually decreases to zero. Thus, through the barrier function method, the penalty term corresponding to the inequality constraint is designed for the objective function that needs to be minimized in the optimization problem. The numerical variation characteristics of the penalty term meet the minimization requirements of the objective function, that is, as the optimization proceeds, the value of the penalty term corresponding to the inequality constraint should gradually decrease to zero, and the value of the objective function also gradually approaches zero.

[0107] Furthermore, the barrier function method is used to process the inequality constraints in the constraints and obtain:

[0108] Formula (11)

[0109] in, For the penalty item, is the activation function, is a hyperparameter that can be adjusted during the experiment to improve the accuracy of solving the control quantity through optimization problem.

[0110] Furthermore, based on formula (11), a penalty term is added to the objective function expressed in formula (5), and the objective function in the equivalent optimization problem can be expressed as:

[0111] Formula (12)

[0112] in, represents the penalty term, is the coefficient of the penalty term.

[0113] The constraints in the equivalent optimization problem may include formula (4) and formula (9).

[0114] In a possible implementation method, S3022 includes: using the Lagrange multiplier method to process the equality constraints in the equivalent optimization problem to obtain a Hamiltonian function, wherein the Hamiltonian function includes an objective function with a penalty term added and equality constraints weighted by Lagrange multipliers.

[0115] Among them, the Lagrange multiplier method is used to associate the constraints with the objective function to form an equality equation.

[0116] In this implementation, the effect of the Lagrange multiplier method is used to associate the equality constraint in the equivalent optimization problem with the objective function in the equivalent optimization problem to obtain a Hamiltonian function. Therefore, the Hamiltonian function includes the objective function with a penalty term added (i.e., the objective function in the equivalent optimization problem) and the equality constraint weighted by the Lagrange multiplier.

[0117] Optionally, when the equality constraints include formula (4) and formula (9), the Hamiltonian function obtained by the Lagrange multiplier method is expressed as:

[0118] Formula (13)

[0119] in, represents the Hamiltonian function, represents the objective function in the equivalent optimization problem, such as formula (12), and represents the Lagrange multiplier, Is an equality constraint

[0120] The corresponding Lagrange multiplier is, Is an equality constraint The corresponding Lagrange multiplier.

[0121] In a possible implementation, S3023 includes: respectively differentiating the state quantity of the finger and the control quantity of the finger in the Hamiltonian function to obtain derivative formulas corresponding to the state quantity of the finger and the control quantity of the finger respectively; and constructing a control quantity solution equation based on the derivative formula and the equation constraint.

[0122] In this implementation, the state quantity of the finger is differentiated in the Hamiltonian function to obtain the derivative formula corresponding to the state quantity of the finger; the control quantity of the finger is differentiated in the Hamiltonian function to obtain the derivative formula corresponding to the control quantity of the finger; the derivative formula of the state quantity of the finger, the derivative formula of the control quantity of the finger and the equality constraint can be combined according to the Karush-Kuhn-Tucker Conditions (KKT Conditions) to form an equation group; the equation group is converted into a matrix equation to obtain the control quantity solution equation. Thus, by the method of differentiation, construction of the equation group and conversion of the matrix equation, the control quantity derivative equation is simplified and the amount of calculation is reduced.

[0123] Optionally, according to the KKT condition, the derivative formula of the state quantity of the finger, the derivative formula of the control quantity of the finger and the equality constraint are combined to form a set of equations expressed as:

[0124] Formula (14)

[0125] in, The derivative formula representing the state quantity of the finger is obtained by differentiating the Hamiltonian function; The derivative formula representing the control amount of the finger is obtained by differentiating the Hamiltonian function.

[0126] Furthermore, by converting formula (14) into a matrix equation, the control quantity solution equation is expressed as:

[0127] Formula (15)

[0128] S303, according to the current state quantity of the finger, the control quantity solution equation is iteratively solved by the continuous generalized minimum residual method to obtain the target control quantity.

[0129] In this embodiment, after obtaining the control quantity solution equation, the current state quantity of the finger is taken as the initial value of the state quantity of the finger in the control quantity equation at the future moment, and the control quantity solution equation is iteratively solved through the continuous generalized minimum residual method, that is, the control quantity of the finger at the future moment is solved to obtain the target control quantity of the finger at the future moment.

[0130] In one possible implementation, S303 includes S3031~S3033: S3031, using Newton's method to determine the initial approximate solution of the control quantity solution equation; S3032, determining the initial residual vector based on the initial approximate solution; S3033, in a low-dimensional subspace, based on the initial approximate solution, the initial residual vector and the current state quantity, iteratively solving the approximate solution of the control quantity solution equation until the iteration end condition is met to obtain the target control quantity, and the iteration end condition includes: the number of iterations is greater than or equal to the number threshold, or the residual vector during the iteration process is less than the vector threshold.

[0131] The low-dimensional space is the Krylov subspace in the continuous generalized minimum residual method.

[0132] Among them, if the control quantity solution equation is solved according to the conventional method, the inverse matrix needs to be calculated, which consumes a lot of computing resources. For example, solving formula (15) in the conventional way will result in:

[0133] Formula (16)

[0134] in, is the stability matrix obtained by treating the control quantity solution equation as a dynamic system. Optionally, if formula (15) is regarded as a dynamic system, its expression is . yes Abbreviation of yes The partial derivative of X(t).

[0135] From formula (16), it can be seen that the conventional method requires calculating the inverse matrix , the amount of calculation is large.

[0136] In order to reduce the amount of calculation, this implementation method uses the continuous generalized minimum residual method to solve the control quantity solution equation. In this implementation method, in the solution process, the Newton method is first used to determine the initial approximate solution of the control quantity solution equation, that is, to determine the initial value of the control quantity of the finger at the future moment; according to the initial approximate solution and the relationship between the residual vector and the approximate solution, the initial residual vector is calculated; then, in the low-dimensional subspace, by calculating the residual vector in the kth iteration and the unit vector corresponding to the residual vector, the approximate solution in the kth iteration is calculated using the unit vector and the initial approximate solution, and the objective function in the continuous generalized minimum residual method is minimized through the iterative process, and finally, when the iteration end condition is met, the target control quantity of the finger at the future moment is obtained.

[0137] In this implementation, the continuous generalized minimum residual method can be written as:

[0138] Formula (17)

[0139] in, represents the continuous generalized minimum residual method, X0 represents the initial approximate solution, and k represents the number of iterations.

[0140] Optionally, when using the continuous generalized minimum residual method, the control quantity solution equation can be converted to:

[0141] Formula (18)

[0142] It can be seen from formula (17) that there is no need to solve the inverse matrix when using the continuous generalized minimum residual method.

[0143] Optionally, the relationship between the residual vector and the approximate solution is expressed as:

[0144] Formula (19)

[0145] Among them, r0 represents the initial residual vector, and b is the vector parameter in the continuous generalized minimum residual method. Represents the control quantity to solve the equation.

[0146] Optionally, the unit vector corresponding to the residual vector is expressed as:

[0147] Formula (20)

[0148] Among them, r k represents the residual vector in the kth iteration, v k Represents the unit vector corresponding to the residual vector in the kth iteration.

[0149] Optionally, the calculation formula of the approximate solution in the kth iteration is expressed as:

[0150] Formula (21)

[0151] Among them, x k represents the approximate solution in the kth iteration, y k is the vector parameter of length k in the continuous generalized minimum residual method.

[0152] Alternatively, the objective function in the continuous generalized minimum residual method is expressed as:

[0153] Formula (22)

[0154] Therefore, in the continuous generalized minimum residual method, by continuously iteratively solving the above formula to minimize it, we can get the appropriate r k According to formula (21), we can get x k . Finally, the target control quantity is obtained.

[0155] S304: Control the joint angles of the fingers according to the target control amount.

[0156] The implementation principle and technical effects of S304 may refer to the aforementioned embodiments and will not be described in detail.

[0157] In the embodiment of the present application, a series of transformations are performed on the optimization problem of the control quantity of the fingers of the dexterous hand to obtain the control quantity solution equation; the continuity generalized minimum residual method is used to iteratively solve the control quantity solution equation, which reduces the amount of calculation for solving the control quantity solution equation, improves the efficiency of solving the optimization problem, and improves the control frequency of the finger movements of the dexterous hand. Thus, the control accuracy and response speed of the dexterous hand are improved, the high-precision and high-response speed requirements of the dexterous hand in complex operation scenarios are met, and the task completion effect of the dexterous hand is improved.

[0158] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0159] Figure 4 A schematic diagram of the structure of the dexterous hand control device provided in the embodiment of the present application is shown in FIG. Figure 4 As shown, the dexterous hand control device 400 of the embodiment of the present application includes: an information acquisition module 401, a control amount optimization solution module 402 and a control module 403. Among them:

[0160] The information acquisition module 401 is used to obtain the current state quantity of the fingers of the dexterous hand and the reference state quantity of the fingers in the future. The current state quantity is related to the joint angle of the fingers at the current moment, and the reference state quantity is related to the joint angle of the fingers in the future, and the future moment is located after the current moment; the control quantity optimization solution module 402 is used to iteratively solve the optimization problem of the fingers according to the current state quantity and the reference state quantity using the continuous generalized minimum residual method to obtain the target control quantity of the fingers in the future, and the optimization problem is determined based on the dynamic model of the fingers; the control module 403 is used to control the joint angle of the fingers according to the target control quantity.

[0161] In some embodiments, the optimization problem includes an objective function and constraints, the constraints include equality constraints, the equality constraints include dynamic constraint formulas obtained based on a dynamic model, and the dynamic constraint formulas are used to represent the dynamic relationship between the state quantity of the finger and the control quantity of the finger.

[0162] In some embodiments, the control quantity optimization solution module 402 is specifically used to: input the reference state quantity into the objective function, transform the optimization problem according to the constraints and the objective function, and obtain the control quantity solution equation; according to the current state quantity, through the continuous generalized minimum residual method, iteratively solve the control quantity solution equation to obtain the target control quantity.

[0163] In some embodiments, the constraint conditions also include inequality constraints, and the control quantity optimization solution module 402 is specifically used to: convert the inequality constraints into corresponding penalty terms, add the penalty terms to the objective function, and obtain an equivalent optimization problem corresponding to the optimization problem, the equivalent optimization problem includes the objective function and the equality constraint with the penalty terms added; establish the Hamiltonian function corresponding to the equivalent optimization problem; and determine the control quantity solution equation based on the equality constraints and the Hamiltonian function.

[0164] In some embodiments, the control quantity optimization solution module 402 is specifically used to: determine the penalty term by processing the inequality constraint by adopting the barrier function method; process the equality constraint in the equivalent optimization problem by adopting the Lagrange multiplier method to obtain the Hamiltonian function, which includes the objective function with the penalty term added and the equality constraint weighted by the Lagrange multiplier; derive the state quantity of the finger and the control quantity of the finger in the Hamiltonian function respectively to obtain the derivative formulas corresponding to the state quantity of the finger and the control quantity of the finger respectively; construct the control quantity solution equation according to the derivative formula and the equality constraint.

[0165] In some embodiments, the control quantity optimization solution module 402 is specifically used to: use Newton's method to determine the initial approximate solution of the control quantity solution equation; determine the initial residual vector based on the initial approximate solution; in a low-dimensional subspace, based on the initial approximate solution, the initial residual vector and the current state quantity, iteratively solve the approximate solution of the control quantity solution equation until the iteration end condition is met to obtain the target control quantity, and the iteration end condition includes: the number of iterations is greater than or equal to the number threshold, or the residual vector during the iteration process is less than the vector threshold.

[0166] The device of this embodiment can be used to execute the technical solution of any of the above-mentioned method embodiments. Its implementation principle and technical effects are similar and will not be repeated here.

[0167] Figure 5 This is a structural block diagram of a dexterous hand control system provided in one embodiment of the present application.

[0168] Reference Figure 5The dexterous hand control system 500 includes a processor 501 and a dexterous hand 502. The processor 501 is disposed on the dexterous hand 502 or is disposed separately from the dexterous hand 502, and is used to implement the dexterous hand control method provided in any of the above embodiments; the dexterous hand 502 is used to perform corresponding actions according to the control of the processor 501.

[0169] The embodiment of the present application also provides a dexterous hand finger, which can implement any of the above embodiments under the control of a processor.

[0170] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, a solution provided in any of the above embodiments is implemented.

[0171] The present application also provides a computer program product, including a computer program, which implements the solution provided in any of the above embodiments when executed by a processor.

[0172] The computer-readable storage medium mentioned above may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium may be any available medium that can be accessed by a general or special-purpose computer.

[0173] An exemplary readable storage medium is coupled to the processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can also exist in the dexterous hand control system as discrete components.

[0174] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0175] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A dexterous hand control method, characterized in that: include: Acquire a current state quantity of a finger of the dexterous hand and a reference state quantity at a future moment after the current moment, wherein the current state quantity is related to a joint angle of the finger at the current moment, and the reference state quantity is related to a joint angle of the finger at the future moment; According to the current state quantity and the reference state quantity, the optimization problem of the finger is iteratively solved by using a continuous generalized minimum residual method to obtain a target control quantity of the finger at the future moment, wherein the optimization problem includes an objective function and constraints, wherein the constraints include equality constraints, and the equality constraints include a dynamic constraint formula obtained by a dynamic model for representing a dynamic relationship between the state quantity and the control quantity of the finger; the objective function is obtained based on a difference between the state quantity of the finger at the future moment and the reference state quantity, and a difference between the control quantity of the finger at the future moment and the reference control quantity; Controlling the joint angle of the finger according to the target control amount; The target control amount includes the control torque of the joint of the finger; The obtaining of the target control amount of the finger at the future moment includes: The reference state quantity is input into the objective function, and the optimization problem is transformed according to the constraint condition and the objective function to obtain a control quantity solution equation; the control quantity solution equation includes the derivative formulas of the state quantity and the control quantity of the finger and the equality constraint; Using Newton's method, determine the initial approximate solution of the control quantity solution equation and determine the initial residual vector; In the low-dimensional subspace, the approximate solution of the control quantity solution equation is iteratively solved according to the initial approximate solution, the initial residual vector and the current state quantity to obtain the target control quantity.

2. The dexterous hand control method according to claim 1, characterized in that: The constraint conditions also include inequality constraints. The optimization problem is transformed according to the constraint conditions and the objective function to obtain a control quantity solution equation, including: Converting the inequality constraint into a corresponding penalty term, adding the penalty term to the objective function, and obtaining an equivalent optimization problem corresponding to the optimization problem, wherein the equivalent optimization problem includes the objective function with the penalty term added and the equality constraint; Establishing a Hamiltonian function corresponding to the equivalent optimization problem; The control variable is determined to solve the equation according to the equality constraint and the Hamiltonian function.

3. The dexterous hand control method according to claim 2, characterized in that: The converting the inequality constraint into a corresponding penalty term comprises: The penalty term is determined by processing the inequality constraint using a barrier function method; The process of processing the equality constraint in the equivalent optimization problem to establish a Hamiltonian function corresponding to the equivalent optimization problem includes: The equality constraint in the equivalent optimization problem is processed by a Lagrange multiplier method to obtain the Hamiltonian function, wherein the Hamiltonian function includes the objective function with the penalty term added thereto and the equality constraint weighted by the Lagrange multiplier; Determining the control quantity to solve the equation according to the equality constraint and the Hamiltonian function includes: Derivatives are respectively performed on the state quantity of the finger and the control quantity of the finger in the Hamiltonian function to obtain derivative formulas corresponding to the state quantity of the finger and the control quantity of the finger respectively; According to the derivative formula and the equality constraint, the control quantity solving equation is constructed.

4. The dexterous hand control method according to any one of claims 1 to 3, characterized in that: The determining of the initial residual vector comprises: Determining an initial residual vector according to the initial approximate solution; The iterative solution of the control quantity solution equation to obtain the target control quantity includes: The approximate solution of the control quantity solving equation is iteratively solved until the iteration end condition is met to obtain the target control quantity. The iteration end condition includes: the number of iterations is greater than or equal to the number threshold, or the residual vector in the iteration process is less than the vector threshold.

5. A dexterous hand control device, characterized in that: include: An information acquisition module, used to acquire a current state quantity of a finger of the dexterous hand and a reference state quantity at a future moment after the current moment, wherein the current state quantity is related to a joint angle of the finger at the current moment, and the reference state quantity is related to a joint angle of the finger at the future moment; a control quantity optimization solution module, for iteratively solving the optimization problem of the finger by using a continuous generalized minimum residual method according to the current state quantity and the reference state quantity, so as to obtain the target control quantity of the finger at the future time, wherein the optimization problem includes an objective function and constraints, wherein the constraints include equality constraints, and the equality constraints include a dynamic constraint formula obtained by a dynamic model for representing the dynamic relationship between the state quantity and the control quantity of the finger; the objective function is obtained based on the difference between the state quantity of the finger at the future time and the reference state quantity, and the difference between the control quantity of the finger at the future time and the reference control quantity; A control module, used to control the joint angle of the finger according to the target control amount; the target control amount includes the control torque of the joint of the finger; The control quantity optimization solution module is specifically used to input the reference state quantity into the objective function, transform the optimization problem according to the constraint condition and the objective function, and obtain the control quantity solution equation; the control quantity solution equation includes the derivative formula of the state quantity and the control quantity of the finger and the equality constraint; The Newton method is used to determine the initial approximate solution of the control quantity solution equation and the initial residual vector; in a low-dimensional subspace, the approximate solution of the control quantity solution equation is iteratively solved according to the initial approximate solution, the initial residual vector and the current state quantity to obtain the target control quantity.

6. A dexterous hand control system, characterized in that: include: A processor, arranged on the dexterous hand or separately arranged from the dexterous hand, for implementing the dexterous hand control method according to any one of claims 1 to 4; The dexterous hand is used to perform corresponding actions according to the control of the processor.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by a processor, the dexterous hand control method according to any one of claims 1 to 4 is implemented.

8. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the dexterous hand control method according to any one of claims 1 to 4 is implemented.

Images