Dexterous hand system control method, device, system and storage medium

Abstract

This application relates to the field of robotics technology and discloses a dexterous hand system control method, device, system, and storage medium. It aims to improve the realism and feedback speed of force feedback information, precisely control the slave dexterous hand, and ensure the consistency and safety of the master and slave dexterous hand movements. The dexterous hand system control method includes: acquiring key hand angle information collected by the master dexterous hand; inputting the key hand angle information into a preset safe collaborative hand mapping model and outputting slave hand control information to the slave dexterous hand; controlling the slave dexterous hand according to the slave hand control information and acquiring initial force sensing data collected by the slave dexterous hand; inputting the initial force sensing data into a preset exponential force feedback mapping model and outputting target force feedback information to the master dexterous hand; controlling the master dexterous hand to update the slave hand control information based on the target force feedback information, and repeating this process until the operation task is completed.

Description

Technical Field

[0001] This application relates to the field of robotics technology, and in particular to a dexterous hand system control method, device, system, and storage medium. Background Technology

[0002] In recent years, with the increasing maturity of robotics and artificial intelligence technologies, robots have gradually become a focus in modern scientific research, manufacturing, and service industries. As the application fields of robots expand, traditional two-finger grippers or humanoid robotic hands can no longer meet people's requirements for robots to perform various dexterous and precise maneuvers.

[0003] Dexterous hand system control technology captures the operator's hand movements using high-precision sensors and, combined with human-machine interaction technology, translates them into control commands for the robotic arm, enabling remote operation of the dexterous hand and allowing for the execution of complex and delicate tasks under remote control. However, due to the complexity and variability of the external environment, traditional dexterous hand system control methods often struggle to adapt to changes in the external environment. Force feedback is delayed and lacks realism, preventing operators from promptly perceiving environmental changes. Furthermore, due to structural differences between the master and slave dexterous hands and inaccurate motion mapping between them, unnecessary collisions can easily occur between the fingertips and adjacent joints of the slave dexterous hand. This not only hinders the successful completion of tasks but may also damage the dexterous hand's precision components. Summary of the Invention

[0004] This application provides a dexterous hand system control method, device, system, and storage medium to solve the problems of force feedback delay and unnecessary collisions between fingertips and adjacent joints in traditional dexterous hand systems, which lead to the inability to detect changes in the remote operating environment in a timely manner and low safety of the dexterous hand system.

[0005] The first aspect of this application provides a control method for a dexterous hand system, comprising: acquiring hand key point angle information collected by a master dexterous hand; inputting the hand key point angle information into a preset safe collaborative hand mapping model, and outputting slave hand control information to a slave dexterous hand; controlling the slave dexterous hand according to the slave hand control information, and acquiring initial force sensing data collected by the slave dexterous hand; inputting the initial force sensing data into a preset exponential force feedback mapping model, and outputting target force feedback information to the master dexterous hand; controlling the master dexterous hand to update the slave hand control information based on the target force feedback information, and repeating this process until the operation task is completed.

[0006] A second aspect of this application provides a dexterous hand system control device, comprising: an acquisition module for acquiring hand key point angle information collected by a master dexterous hand; a motion mapping module for inputting the hand key point angle information into a preset safe collaborative hand mapping model and outputting slave hand control information to a slave dexterous hand; a processing module for controlling the slave dexterous hand according to the slave hand control information and acquiring initial force sensing data collected by the slave dexterous hand; a force feedback module for inputting the initial force sensing data into a preset exponential force feedback mapping model and outputting target force feedback information to the master dexterous hand; and a bidirectional control module for controlling the master dexterous hand to update the slave hand control information based on the target force feedback information, thereby repeating the cycle until the operation task is completed.

[0007] A third aspect of this application provides a remote sensing manipulation dexterous hand system, comprising: a master dexterous hand, a slave dexterous hand, a memory, and at least one processor; the memory and at least one processor store instructions; the at least one processor invokes the instructions in the memory to cause the master dexterous hand and the slave dexterous hand to execute the aforementioned dexterous hand system control method; the master dexterous hand is used to receive target force feedback information output by an exponential force feedback mapping model, collect hand key point angle information, input the hand key point angle information into a safe collaborative hand mapping model, and output slave hand control information to the slave dexterous hand; the slave dexterous hand is used to receive slave hand control information output by the safe collaborative hand mapping model, control the slave dexterous hand to perform operation actions according to the slave hand control information, acquire initial force sensing data, input it into the exponential force feedback mapping model, and output target force feedback information to the master dexterous hand.

[0008] A fourth aspect of this application provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the dexterous hand system control method described above.

[0009] The technical solution provided in this application achieves precise tracking of the master dexterity's movements by the slave dexterity through a safe collaborative hand mapping model between the master and slave dexterity hands, which addresses the differences and safe distances between them. This avoids unnecessary collisions between adjacent fingertips and joints of the slave dexterity, extending the lifespan of the dexterity system. Furthermore, the exponential force feedback mapping model exponentially processes the target force sensing data collected by the slave dexterity, enhancing the realism and timeliness of the force feedback to the master dexterity. This enables the master dexterity to perceive the operating environment in real time and achieves bidirectional precise control between the master and slave dexterity hands. By using a remote sensing operation dexterity system, the operational dexterity for complex tasks is improved, along with the efficiency and safety of remote sensing operations. Attached Figure Description

[0010] Figure 1 This is a schematic diagram of an embodiment of the dexterous hand system control method of this application;

[0011] Figure 2 This is a schematic diagram of another embodiment of the dexterous hand system control method of this application;

[0012] Figure 3 This is a schematic diagram of one embodiment of the dexterous hand system control device of this application;

[0013] Figure 4 This is a schematic diagram of one embodiment of the dexterous hand system control device of this application;

[0014] Figure 5 This is a schematic diagram of one embodiment of the remote sensing operation dexterous hand system of this application. Detailed Implementation

[0015] This application provides a dexterous hand system control method, device, system, and storage medium to improve the realism and feedback speed of force feedback information, accurately control the slave dexterous hand, and ensure the consistency and safety of the master and slave dexterous hand movements.

[0016] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms “comprising” or “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0017] For ease of understanding, the specific process of the embodiments of this application is described below. Please refer to [link / reference]. Figure 1 One embodiment of the dexterous hand system control method in this application includes:

[0018] It is understood that the executing entity of this application can be a dexterous hand system control device, or it can be a remote sensing operated dexterous hand system, a terminal, or a server; no specific limitation is made here. This application's embodiments use a remote sensing operated dexterous hand system as an example for illustration.

[0019] Between acquiring the key hand angle information collected by the master dexterous hand, a remote sensing operation dexterous hand system is built. The remote sensing operation dexterous hand system includes a master dexterous hand for performing remote sensing operations and a slave dexterous hand for receiving remote sensing operation information and performing operations.

[0020] In this embodiment, both the master dexterous hand and the slave dexterous hand have *e* degrees of freedom, with the thumb, index finger, middle finger, ring finger, and little finger each having *f* degrees of freedom, where *e* > 0 and *f* > 0. Both the master and slave dexterous hands use the center of the palm as the origin of their world coordinate system. Adhesive straps for securing the palm to the master dexterous hand are located at the fingertips, the center of the palm, and the midpoints of the finger joints. At least one piezoresistive force sensor is positioned at the midpoint of each finger joint of the slave dexterous hand (e.g., one sensor). Therefore, the slave dexterous hand has a total of *θ* piezoresistive force sensors at each finger joint. At least one piezoresistive force sensor is also positioned at the center of the palm (e.g., one sensor). The degrees of freedom can be selected according to the actual situation. The piezoresistive force sensors can be arranged in a g*h element array, i.e., a two-dimensional array composed of *g* rows and *h* columns. Each element can individually sense the distribution of pressure or force applied to its surface. This matrix layout allows the sensors to capture the details of the applied force with high spatial resolution. By analyzing the signal strength received by each of these g*h units, the specific direction, magnitude, and location of the force can be inferred, thereby improving the system's ability to perceive environmental interactions and the finesse of its operation.

[0021] To facilitate understanding, an example of a remote sensing-operated dexterous hand system is provided: The degrees of freedom are set as e=20 and f=4. Both the master dexterous hand and the slave dexterous hand have 20 degrees of freedom, with the thumb, index finger, middle finger, ring finger, and little finger each having 4 degrees of freedom. Both the master and slave dexterous hands use the center of the palm as the origin of the world coordinate system. Adhesive straps are used to secure the palm to the master dexterous hand at the fingertips, the center of the palm, and the midpoints of the finger joints.

[0022] One 5×5 unit piezoresistive force sensor is placed at the middle of the knuckle of each finger of the dexterous hand, for a total of 14 sensors, and one 5×5 unit piezoresistive force sensor is placed at the center of the palm. Each piezoresistive force sensor contains a total of 25 independent sensing units (or sensing pixels) for collecting initial force sensing data.

[0023] 101. Obtain the angle information of key hand points collected by the main dexterous hand.

[0024] Specifically, the inertial measurement unit (IMU) of the master dexterity hand acquires the hand movement data of the human body to obtain the initial joint angle information; based on the forward kinematics calculation method, the initial joint angle information is converted into target pose information, wherein the target pose information is the pose information of each joint of the master dexterity hand projected onto the world coordinate system.

[0025] In this embodiment, the primary dexterous hand can acquire data on human hand movements through an inertial measurement unit, or it can collect hand movement information through other information.

[0026] In this embodiment, the key points of the hand include the fingertip key points, joint key points and palm key points. The palm is the origin of the world coordinate system. Through forward kinematics calculation, the exact position and posture of the dominant dexterous hand in the world coordinate system can be calculated through a series of matrix transformations.

[0027] 102. Input the key hand angle information into the preset safe collaborative hand mapping model, and output the hand control information to the dexterous hand.

[0028] Specifically, a safe collaborative hand mapping model is established between the master dexterity hand and the slave dexterity hand. The joint angle information of the hand movements obtained by the master dexterity hand is used as input, and the output is used to send control information from the slave hand to the slave dexterity hand.

[0029] The safe collaborative hand mapping model in this embodiment is a motion mapping model of the master dexterity and the fingertips of the slave dexterity based on the differences between the master and slave dexterity. The differences include the ratio of the two, as well as the safe distance between any two fingertips and any two finger joints of the slave dexterity. The safe collaborative hand mapping model of this application can ensure that there is a safe distance between fingertips and between adjacent joints to avoid collisions.

[0030] Optionally, the secure collaborative hand mapping model can be expressed by the following formula:

[0031]

[0032] in, The angle of the fingertip joints of the primary dexterous hand relative to the world coordinate system; The angle of the fingertip joints relative to the world coordinate system; For the master's skillful hands in time The angle of the fingertip joint; To use skillful hands in time The angle of the fingertip joint; The vector of the dominant dexterous hand pointing from one fingertip to another in the world coordinate system; The vector from one fingertip to another in the world coordinate system of a dexterous hand; The total number of vectors; The angle of the finger joints in the primary dexterous hand; From the angle of the finger joints of a dexterous hand; This is a function representing the safe distance coefficient between fingers; This is the first weighting coefficient function; This is the second weighting coefficient function.

[0033] It is understood that the training process of the secure collaborative hand mapping model in this embodiment before deployment can be performed with reference to existing technologies, and will not be described in detail here.

[0034] 103. Control the dexterous hand according to the control information of the slave hand, and acquire the initial force sensing data collected by the dexterous hand.

[0035] Specifically, the movement of each joint and fingertip of the dexterous hand is controlled according to the control information of the dexterous hand, so as to realize remote sensing operation from the dexterous hand. The dexterous hand collects initial force sensing data in each direction through multiple piezoresistive force sensors.

[0036] The initial force sensing data mentioned above is three-dimensional force data, which is based on the world coordinate system constructed from the palm of the dexterous hand. Specifically, the force in each direction on the three-dimensional coordinate axis is determined by the change in resistance value collected by each piezoresistive force sensor.

[0037] Optionally, the formula for calculating the initial force sensing data is:

[0038]

[0039] in, for Force in the axial direction; for Force in the axial direction; for Force in the axial direction; The calibration coefficients for the force sensor are... For the first The change in resistance of a force sensor relative to its initial value.

[0040] 104. Input the initial force sensing data into the preset exponential force feedback mapping model, and output the target force feedback information to the master dexterous hand.

[0041] Specifically, an exponential force feedback mapping model is established between the dexterous hand and the master dexterous hand. The initial force sensing data obtained from the dexterous hand is used as input, and the output is used to send target force feedback information to the master dexterous hand.

[0042] In practical applications, there is a proportional relationship between perceived intensity and actual changes in physical stimuli. The exponential force feedback mapping model in this embodiment uses an exponential function model as the feedback model to be sensitive under small forces and insensitive under large forces, so as to provide a more realistic sensory experience.

[0043] Optionally, the exponential force feedback mapping model can be represented by the following formula:

[0044]

[0045] in, To provide feedback information on the target force to the main dexterous hand; Initial force sensing data collected from a dexterous hand; , , All are fixed constant values.

[0046] The training process of the exponential force feedback mapping model before deployment in this embodiment can be performed with reference to existing technologies, and will not be described in detail here.

[0047] 105. Based on the target force feedback information, the master dexterous hand updates the slave hand control information, and this cycle continues until the operation task is completed.

[0048] Specifically, based on the target force feedback information, the feedback torque of each joint of the slave dexterity hand is reproduced at the master hand end. The master dexterity hand updates the slave hand control information at the next moment based on the target force feedback information, and this cycle continues until the operation task is completed.

[0049] In this embodiment, the operation task can be to grab objects such as balloons, mineral water bottles and rectangular wooden blocks, or to perform other complex operation tasks.

[0050] The technical solution provided in this application achieves precise tracking of the master dexterity's movements by the slave dexterity through a safe collaborative hand mapping model between the master and slave dexterity hands, which addresses the differences and safe distances between them. This avoids unnecessary collisions between adjacent fingertips and joints of the slave dexterity, extending the lifespan of the dexterity system. Furthermore, the exponential force feedback mapping model exponentially processes the target force sensing data collected by the slave dexterity, enhancing the realism and timeliness of the force feedback to the master dexterity. This enables the master dexterity to perceive the operating environment in real time and achieves bidirectional precise control between the master and slave dexterity hands. By using a remote sensing operation dexterity system, the operational dexterity for complex tasks is improved, along with the efficiency and safety of remote sensing operations.

[0051] Please see Figure 2 Another embodiment of the dexterous hand system control method in this application includes:

[0052] 201. Obtain the angle information of key hand points collected by the master dexterity hand.

[0053] Step 201 can be performed by referring to step 101, and will not be repeated here.

[0054] 202. Based on the key hand point angle information, determine the first fingertip vector information and the first joint angle information of the master dexterous hand using the safe collaborative hand mapping model.

[0055] In this embodiment, the first fingertip vector information includes multiple fingertip vectors at the current moment, with a total number of n. Each fingertip vector is a vector pointing from one fingertip key point to another in the world coordinate system at the current moment. The first joint angle information is used to indicate multiple current joint angles of the master dexterity hand. The current joint angle is the joint angle of the master dexterity hand in the world coordinate system at time t.

[0056] 203. Determine the target distance fingertip vector deviation based on the first fingertip vector information corresponding to the master dexterous hand, the preset first weight coefficient function, the preset safety distance coefficient function, and the initial fingertip vector information corresponding to the slave dexterous hand.

[0057] Specifically, the target safety distance coefficient and the target first weight coefficient are determined based on the current distance coefficient between the target fingertips, the preset safety distance coefficient function, and the preset first weight coefficient function, respectively; the target fingertip vector information corresponding to the master dexterity hand is determined based on the target safety distance coefficient and the initial fingertip vector information; the initial distance fingertip vector deviation between the master and slave dexterity hands is determined based on the first joint angle information and the target fingertip vector information; and the target distance fingertip vector deviation is determined based on the initial distance fingertip vector deviation and the target first weight coefficient.

[0058] The current distance coefficient between the aforementioned target fingertips This is used to indicate the distance coefficient between fingers, specifically the distance from the tip of the thumb of the dexterous hand to the tip of another finger. This embodiment dynamically determines the safety distance, the first target weight coefficient, and the second target weight coefficient based on the current distance coefficient, ensuring both consistency in the dexterous hand's movement and the dexterous hand's safety.

[0059] To facilitate understanding, the following is an example of a safety distance coefficient function:

[0060]

[0061] in, This is the distance threshold parameter; The size ratio factor between the primary dexterous hand and the secondary dexterous hand; The distance coefficient between the two fingertips; To maintain a safe distance from the tip of the thumb of a dexterous hand to the tip of another finger; The safe distance between fingertips; This refers to the safe distance between adjacent finger joints; The set of maximum distances from the tip of the thumb of a dexterous hand to the tips of the other four fingers; The set of maximum distances from the tips of the little finger, ring finger, middle finger, and index finger of a dexterous hand; It is the set of maximum distances from the joints of the little finger, ring finger, middle finger, and index finger of a dexterous hand.

[0062] For ease of understanding, the following is an example of a first weighting coefficient function:

[0063]

[0064] in, , , , All are first weight coefficients The constant value of .

[0065] 204. Determine the target distance joint angle deviation based on the first joint angle information, the preset second weighting coefficient function, and the initial joint angle information corresponding to the dexterous hand.

[0066] Specifically, the target second weighting coefficient is determined based on the current distance coefficient and the second weighting coefficient function; the initial joint angle deviation between the master and slave dexterous hands is determined based on the first joint angle information and the initial joint angle information; and the target distance joint angle deviation is determined based on the initial joint angle deviation and the target second weighting coefficient.

[0067] For ease of understanding, the following is an example of a second weighting coefficient function:

[0068]

[0069] in, , , , All are second weighting coefficients The constant value of .

[0070] 205. Determine the slave hand control information by using the preset minimum cost function, the target distance fingertip vector deviation, and the target distance joint angle deviation, and send the slave hand control information to the slave dexterous hand.

[0071] The cost minimization function in this embodiment can be executed with reference to the expression of the secure collaborative hand mapping model in step 102, which will not be repeated here.

[0072] 206. Control the dexterous hand according to the control information of the slave hand, and acquire the initial force sensing data collected by the dexterous hand.

[0073] Taking a 5x5 unit piezoresistive force sensor as an example, the formula for calculating the initial force sensing data is:

[0074]

[0075] in, The initial force sensing data is as follows: for Force in the axial direction; for Force in the axial direction; for Force in the axial direction; The calibration coefficients for the force sensor are... ~ This represents the change in resistance of a 5×5 unit force sensor relative to its initial value.

[0076] 207. Initial force feedback information is obtained by performing exponential calculations based on the initial force sensing data using an exponential force feedback mapping model.

[0077] The target force sensing data is obtained by multiplying the initial force sensing data with the first adjustment coefficient; the initial force feedback information is obtained by performing an exponential operation on the target force sensing data through an exponential force feedback mapping model.

[0078]

[0079] in, This provides initial force feedback information. Initial force sensing data collected from a dexterous hand; It is the first adjustment coefficient, used to control the rate of exponential growth.

[0080] This embodiment uses exponential operations to nonlinearly amplify or reduce the magnitude of the initial force sensing data, so as to adjust the model to have different responses to different magnitudes of force, thereby improving the realism of force feedback.

[0081] 208. Adjust according to the initial force feedback information, the preset second adjustment coefficient and the preset target offset value to obtain the target force feedback information.

[0082] Candidate force feedback information is obtained by summing the negative force feedback information and the preset target offset value; target force feedback information is obtained by multiplying the candidate force feedback information with the preset second adjustment coefficient.

[0083] In this embodiment, the negative force feedback information is the negative value of the initial force feedback information. This embodiment uses the negative force feedback information to adjust the model so that it has different responses to different magnitudes of force, so that it is sensitive to small forces and insensitive to large forces.

[0084] Specifically, it can be expressed by the following force feedback formula:

[0085]

[0086] in, To provide feedback information on the target force to the main dexterous hand; For the second adjustment coefficient, This is the target offset value.

[0087] The aforementioned target offset value can balance the baseline value of the force feedback, ensuring that the force feedback will maintain a certain initial value even when the force sensing data is zero; while the second adjustment coefficient is a global adjustment parameter for the candidate force feedback information, used to control the amplitude of the final feedback force.

[0088] 209. Based on the target force feedback information, the master dexterous hand updates the slave hand control information, and this cycle continues until the operation task is completed.

[0089] Based on the target force feedback information control, the operator performs the action at the next moment to determine the second fingertip vector information and the second joint angle information, and repeats the above steps 201-209 until the operation task is completed.

[0090] In this embodiment, a safe collaborative hand mapping model between the master and slave dexterous hands enables precise tracking of the master dexterous hand's movements by the slave dexterous hand, based on the differences and safe distance between them. The safe distance coefficient, first weight coefficient, and second weight coefficient are dynamically determined using the slave dexterous hand's current distance coefficient, avoiding unnecessary collisions between adjacent fingertips and joints and extending the lifespan of the dexterous hand system. An exponential force feedback mapping model is used to exponentially process the target force sensing data collected by the slave dexterous hand, improving the realism and timeliness of the force feedback from the master dexterous hand. This enables the master dexterous hand to perceive the operating environment in real time and achieves bidirectional precise control between the master and slave dexterous hands. The remote sensing operation dexterous hand system improves the dexterity of complex tasks and enhances the efficiency and safety of remote sensing operations.

[0091] The control method of the dexterous hand system in the embodiments of this application has been described above. The control device of the dexterous hand system in the embodiments of this application is described below. Please refer to [link / reference]. Figure 3 One embodiment of the dexterous hand system control device in this application includes:

[0092] The acquisition module 301 is used to acquire the angle information of key hand points collected by the main dexterous hand;

[0093] The motion mapping module 302 is used to input the key hand point angle information into a preset safe collaborative hand mapping model and output the hand control information to the dexterous hand.

[0094] The processing module 303 is used to control the dexterous hand according to the slave hand control information and to acquire the initial force sensing data collected by the dexterous hand;

[0095] Force feedback module 304 is used to input initial force sensing data into a preset exponential force feedback mapping model and output target force feedback information to the master dexterous hand;

[0096] The bidirectional control module 305 is used to control the master dexterous hand to update the slave hand control information based on the target force feedback information, and so on, until the operation task is completed.

[0097] The technical solution provided in this application achieves precise tracking of the master dexterity's movements by the slave dexterity through a safe collaborative hand mapping model between the master and slave dexterity hands, which addresses the differences and safe distances between them. This avoids unnecessary collisions between adjacent fingertips and joints of the slave dexterity, extending the lifespan of the dexterity system. Furthermore, the exponential force feedback mapping model exponentially processes the target force sensing data collected by the slave dexterity, enhancing the realism and timeliness of the force feedback to the master dexterity. This enables the master dexterity to perceive the operating environment in real time and achieves bidirectional precise control between the master and slave dexterity hands. By using a remote sensing operation dexterity system, the operational dexterity for complex tasks is improved, along with the efficiency and safety of remote sensing operations.

[0098] Another embodiment of the dexterous hand system control device in this application includes:

[0099] The acquisition module 301 is used to acquire the angle information of key hand points collected by the main dexterous hand;

[0100] The motion mapping module 302 is used to input the key hand point angle information into a preset safe collaborative hand mapping model and output the hand control information to the dexterous hand.

[0101] The processing module 303 is used to control the dexterous hand according to the slave hand control information and to acquire the initial force sensing data collected by the dexterous hand;

[0102] Force feedback module 304 is used to input initial force sensing data into a preset exponential force feedback mapping model and output target force feedback information to the master dexterous hand;

[0103] The bidirectional control module 305 is used to control the master dexterous hand to update the slave hand control information based on the target force feedback information, and so on, until the operation task is completed.

[0104] Optionally, the motion mapping module 302 includes:

[0105] The determining unit 3021 is used to determine the first fingertip vector information and the first joint angle information corresponding to the master dexterous hand based on the hand key point angle information through the safe collaborative hand mapping model;

[0106] The fingertip deviation unit 3022 is used to determine the target distance fingertip vector deviation based on the first fingertip vector information corresponding to the master dexterity hand, the preset first weight coefficient function, the preset safety distance coefficient function, and the initial fingertip vector information corresponding to the slave dexterity hand.

[0107] The joint deviation unit 3023 is used to determine the target distance joint angle deviation based on the first joint angle information, the preset second weighting coefficient function, and the initial joint angle information corresponding to the dexterous hand.

[0108] The mapping unit 3024 is used to determine the slave hand control information by means of a preset minimum cost function, the target distance fingertip vector deviation and the target distance joint angle deviation, and to send the slave hand control information to the slave dexterous hand.

[0109] Optionally, the fingertip deviation unit 3022 is specifically used to: determine the target safety distance coefficient and the target first weight coefficient based on the current distance coefficient between the target fingertips, the preset safety distance coefficient function, and the preset first weight coefficient function, respectively;

[0110] Determine the target fingertip vector information corresponding to the dexterous hand based on the target safety distance coefficient and the initial fingertip vector information;

[0111] The initial distance fingertip vector deviation between the master and slave dexterous hands is determined based on the first joint angle information and the target fingertip vector information.

[0112] The target distance fingertip vector deviation is determined based on the initial distance fingertip vector deviation and the target first weight coefficient.

[0113] Optionally, the joint deviation unit 3023 is specifically used to: determine the target's second weighting coefficient based on the current distance coefficient and the second weighting coefficient function;

[0114] The initial joint angle deviation between the master and slave dexterous hands is determined based on the first joint angle information and the initial joint angle information.

[0115] The target distance joint angle deviation is determined based on the initial joint angle deviation and the target's second weighting coefficient.

[0116] Optionally, the force feedback module 304 includes:

[0117] The exponential adjustment unit 3041 is used to perform exponential calculations based on the initial force sensing data through the exponential force feedback mapping model to obtain the initial force feedback information.

[0118] The coefficient adjustment unit 3042 is used to adjust the target force feedback information based on the initial force feedback information, the preset second adjustment coefficient and the preset target offset value.

[0119] Optionally, the index adjustment unit 3041 is specifically used to: perform a multiplication operation based on the initial force sensing data and the first adjustment coefficient to obtain the target force sensing data;

[0120] The initial force feedback information is obtained by performing exponential calculations on the target force sensing data using an exponential force feedback mapping model.

[0121] In this embodiment, a safe collaborative hand mapping model between the master and slave dexterous hands is used to accurately track the movements of the master dexterous hand by the slave dexterous hand, taking into account the differences and safe distance between them. The safe distance coefficient, first weight coefficient, and second weight coefficient are dynamically determined based on the current distance coefficient of the slave dexterous hand, avoiding unnecessary collisions between adjacent fingertips and joints and extending the lifespan of the dexterous hand system. An exponential force feedback mapping model is used to exponentially process the target force sensing data collected by the slave dexterous hand, improving the realism and timeliness of the force feedback to the master dexterous hand. This enables the master dexterous hand to perceive the operating environment in real time and achieves bidirectional precise control between the master and slave dexterous hands. The remote sensing operation dexterous hand system improves the dexterity of complex tasks and enhances the efficiency and safety of remote sensing operations.

[0122] above Figure 3 , Figure 4 The dexterous hand system control device in the embodiments of this application is described in detail from the perspective of modular functional entities. The remote sensing operation dexterous hand system in the embodiments of this application is described in detail below from the perspective of hardware processing.

[0123] See Figure 5 As shown, the remote sensing operation dexterous hand system includes a master dexterous hand 500, a slave dexterous hand 501, a processor 502, and a memory 503. The memory 503 stores machine-executable instructions that can be executed by the processor 502. The processor 502 executes the machine-executable instructions to implement the above-mentioned dexterous hand system control method.

[0124] Furthermore, Figure 5 The remote sensing operation dexterous hand system shown also includes a bus 504 and a communication interface 505. The processor 502, the communication interface 505 and the memory 503 are connected through the bus 504.

[0125] The memory 503 may include high-speed random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 505 (which can be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc. The bus 504 may be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 5 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0126] Processor 502 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of processor 502 or by instructions in software form. The processor 502 can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this disclosure. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this disclosure can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 503. Processor 502 reads the information in memory 503 and, in conjunction with its hardware, completes the method steps of the aforementioned embodiment.

[0127] The master dexterous hand 500 is used to receive the target force feedback information output by the exponential force feedback mapping model, collect the hand key point angle information, input the hand key point angle information into the safe collaborative hand mapping model, and output the slave hand control information to the slave dexterous hand 501.

[0128] The slave dexter 501 is used to receive slave control information output from the safe collaborative hand mapping model, control the slave dexter to perform operation actions according to the slave control information, obtain initial force sensing data input to the exponential force feedback mapping model, and output target force feedback information to the master dexter 500.

[0129] This application also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, storing instructions that, when executed on a computer, cause the computer to perform the steps of a dexterous hand system control method.

[0130] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0131] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0132] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A dexterous hand system control method characterized by, The dexterous hand system control method comprises: Obtaining hand key point angle information collected by a master dexterous hand; Inputting the hand key point angle information into a preset safe cooperative hand mapping model, and outputting slave hand control information to a slave dexterous hand; Controlling the slave dexterous hand according to the slave hand control information, and obtaining initial force sensing data collected by the slave dexterous hand; Inputting the initial force sensing data into a preset exponential force feedback mapping model, and outputting target force feedback information to the master dexterous hand; Controlling the master dexterous hand to update the slave hand control information based on the target force feedback information, and repeating the above steps until the operation task is completed. The step of inputting the hand key point angle information into a preset safe cooperative hand mapping model, and outputting slave hand control information to a slave dexterous hand, comprises: Determining first fingertip vector information and first joint angle information corresponding to the master dexterous hand according to the hand key point angle information through the safe cooperative hand mapping model; Determining target distance fingertip vector deviation according to the first fingertip vector information corresponding to the master dexterous hand, a preset first weight coefficient function, a preset safe distance coefficient function, and initial fingertip vector information corresponding to the slave dexterous hand; Determining target distance joint angle deviation according to the first joint angle information, a preset second weight coefficient function, and initial joint angle information corresponding to the slave dexterous hand; Determining slave hand control information through a preset minimization cost function, the target distance fingertip vector deviation, and the target distance joint angle deviation, and sending the slave hand control information to the slave dexterous hand.

2. The dexterous hand system control method of claim 1, wherein The step of determining target distance fingertip vector deviation according to the first fingertip vector information corresponding to the master dexterous hand, a preset first weight coefficient function, a preset safe distance coefficient function, and initial fingertip vector information corresponding to the slave dexterous hand, comprises: Determining target safe distance coefficient and target first weight coefficient according to current distance coefficient between target fingertips, the preset safe distance coefficient function, and the preset first weight coefficient function, respectively; Determining target fingertip vector information corresponding to the slave dexterous hand according to the target safe distance coefficient and the initial fingertip vector information; Determining initial distance fingertip vector deviation between the master and slave dexterous hands based on the first joint angle information and the target fingertip vector information; Determining target distance fingertip vector deviation based on the initial distance fingertip vector deviation and the target first weight coefficient.

3. The dexterous hand system control method of claim 2, wherein The step of determining target distance joint angle deviation according to the first joint angle information, a preset second weight coefficient function, and initial joint angle information corresponding to the slave dexterous hand, comprises: Determining target second weight coefficient according to the current distance coefficient and the second weight coefficient function; Determining initial joint angle deviation between the master and slave dexterous hands based on the first joint angle information and the initial joint angle information; Determining target distance joint angle deviation based on the initial joint angle deviation and the target second weight coefficient.

4. The method of controlling a dexterous hand system of claim 1, wherein, The step of inputting the initial force sensing data into a preset exponential force feedback mapping model, and outputting target force feedback information to the master dexterous hand, comprises: The initial force feedback information is obtained by performing exponential operation on the initial force sensing data based on the exponential force feedback mapping model; The target force feedback information is obtained by adjusting the initial force feedback information, a preset second adjustment coefficient, and a preset target offset value.

5. The dexterous hand system control method of claim 4, wherein The initial force feedback information is obtained by performing exponential operation on the initial force sensing data based on the exponential force feedback mapping model, including: The target force sensing data is obtained by performing multiplication operation on the initial force sensing data and a first adjustment coefficient; The initial force feedback information is obtained by performing exponential operation on the target force sensing data by the exponential force feedback mapping model.

6. A dexterous hand system control device characterized by comprising: The dexterous hand system control device includes: An acquisition module configured to acquire hand key point angle information collected by a master dexterous hand; An action mapping module configured to input the hand key point angle information into a preset safe cooperative hand mapping model, and output slave hand control information to a slave dexterous hand; A processing module configured to control the slave dexterous hand according to the slave hand control information, and acquire initial force sensing data collected by the slave dexterous hand; A force feedback module configured to input the initial force sensing data into a preset exponential force feedback mapping model, and output target force feedback information to the master dexterous hand; A bidirectional control module configured to control the master dexterous hand to update the slave hand control information based on the target force feedback information, and repeat the operation until an operation task is completed; The hand key point angle information is input into a preset safe cooperative hand mapping model, and slave hand control information is output to a slave dexterous hand, including: First fingertip vector information and first joint angle information corresponding to the master dexterous hand are determined according to the hand key point angle information by the safe cooperative hand mapping model; A target distance fingertip vector deviation is determined according to the first fingertip vector information corresponding to the master dexterous hand, a preset first weight coefficient function, a preset safe distance coefficient function, and initial fingertip vector information corresponding to the slave dexterous hand; A target distance joint angle deviation is determined according to the first joint angle information, a preset second weight coefficient function, and initial joint angle information corresponding to the slave dexterous hand; The slave hand control information is determined by a preset minimization cost function, the target distance fingertip vector deviation, and the target distance joint angle deviation, and the slave hand control information is sent to the slave dexterous hand.

7. A tele-sensory dexterous hand system, comprising: The master dexterous hand, the slave dexterous hand, a memory, and at least one processor are included, and the memory stores instructions; the at least one processor invokes the instructions in the memory, so that the master dexterous hand and the slave dexterous hand perform the dexterous hand system control method in any one of claims 1-5; The master dexterous hand is configured to receive target force feedback information output by an exponential force feedback mapping model, collect hand key point angle information, input the hand key point angle information into a safe cooperative hand mapping model, and output slave hand control information to the slave dexterous hand; The slave hand receives the slave hand control information output by the safety collaborative hand mapping model, controls the slave hand to perform an operation action according to the slave hand control information, acquires initial force sensing data, inputs the initial force sensing data into the index force feedback mapping model, and outputs target force feedback information to the master hand.

8. The tele-sensory dexterous hand system of claim 7, wherein, The master hand and the slave hand are both e degrees of freedom, wherein the thumb, the index finger, the middle finger, the ring finger and the little finger are all f degrees of freedom; The master hand and the slave hand both take a palm center point as an origin of a world coordinate system; The master hand is provided with a pasting type band for fixing a human hand and the master hand at a finger tip of a finger, a palm center and a joint intermediate position of the finger; The slave hand is provided with at least one piezoresistive force sensing sensor at a joint intermediate position of each finger and a palm center.

9. A computer-readable storage medium having stored thereon instructions, the computer-readable storage medium comprising: The instructions perform the control method of the dexterous hand system as claimed in any one of claims 1-5 when read and run.

Images