Dexterous hand perception control method, device and equipment and storage medium

Abstract

The invention discloses a dexterous hand perception control method and device, equipment and a storage medium, and relates to the technical field of artificial intelligence. The method comprises the steps that under the condition that a target multi-fingered dexterous hand grabs a target object is recognized through a pressure sensor array, pressure collection data are obtained from a pressure sensor array, and current joint data of all finger joints of the target multi-fingered dexterous hand are obtained from an encoder; according to the pressure collection data and the current joint data, the current grabbing state of the target multi-fingered dexterous hand is determined; when it is recognized that the current grabbing state does not meet the safe grabbing condition, dexterous hand pose data of the target multi-fingered dexterous hand are obtained from an inertial sensor, and a target grabbing mode of the target multi-fingered dexterous hand is determined according to the current space pose data, the pressure collection data, the current joint data and the dexterous hand pose data; and the target multi-fingered dexterous hand is controlled to grab in a target grabbing mode. According to the scheme, the sensing grabbing efficiency and accuracy of the dexterous hand are improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more particularly to the field of artificial intelligence technology, specifically to a dexterous hand sensing and control method, apparatus, device, and storage medium. Background Technology

[0002] In the field of robotics, dexterous hands, as the core execution component for achieving precise operations, play a crucial role in scenarios such as industrial precision assembly, minimally invasive medical surgery, and home services. The development of its grasping and control technology directly determines the robot's ability to adapt to complex environments and diverse tasks.

[0003] The grasping control process of a dexterous hand is similar to the operation logic of a conventional robot, requiring the sequential completion of three stages: environmental perception, grasping decision-making, and action execution. Currently, dexterous hand grasping control schemes based on different technical approaches can be mainly divided into the following categories: first, dexterous hand grasping control based on rigid transmission; second, dexterous hand grasping control based on tendon-wire transmission; and third, grasping control algorithms based on multi-sensor fusion. While these methods achieve intelligent grasping control of dexterous hands to a certain extent, they have shortcomings in structural flexibility, transmission reliability, and algorithm real-time performance. They are difficult to fully adapt to complex grasping tasks requiring high precision and adaptability. For example, multi-sensor fusion grasping control algorithms have extremely high computational resource requirements, and on edge computing devices (such as embedded controllers), data processing latency is often significant, making it difficult to meet the real-time requirements of dynamic grasping scenarios. Summary of the Invention

[0004] This application provides a dexterous hand sensing and control method, device, equipment, and storage medium to improve the efficiency and accuracy of grasping by multi-finger dexterous hands.

[0005] According to one aspect of this application, a dexterous hand sensing and control method is provided, which is applied to a dexterous hand control system for a target multi-fingered dexterous hand; the fingertips of the target multi-fingered dexterous hand are equipped with pressure sensor arrays; the finger joints of the target multi-fingered dexterous hand are equipped with encoders; and the palm of the target multi-fingered dexterous hand is equipped with inertial sensors; the method includes:

[0006] Based on the received current spatial pose data of the target object, the initial grasping method of the target multi-fingered dexterous hand is determined, and the target multi-fingered dexterous hand is controlled to grasp the target object using the initial grasping method.

[0007] When the target multi-fingered dexterous hand is identified as grasping the target object by the pressure sensor array, pressure acquisition data is obtained from the pressure sensor array, and current joint data of each finger joint of the target multi-fingered dexterous hand is obtained from the encoder;

[0008] Based on the pressure acquisition data and the current joint data, the current grasping state of the target multi-fingered dexterous hand is determined; wherein, the current joint data includes current joint position data and current joint angle data;

[0009] If it is detected that the current grasping state does not meet the safe grasping conditions, the dexterity hand pose data of the target multi-fingered dexterity hand is obtained from the inertial sensor, and the target grasping method of the target multi-fingered dexterity hand is determined based on the current spatial pose data, the pressure acquisition data, the current joint data and the dexterity hand pose data.

[0010] Using the aforementioned target grasping method, the target multi-finger dexterous hand is controlled to complete the grasping operation of the target object.

[0011] According to another aspect of this application, a dexterous hand sensing and control device is provided, which is configured in a dexterous hand control system of a target multi-fingered dexterous hand; the fingertips of the target multi-fingered dexterous hand are equipped with pressure sensor arrays; the finger joints of the target multi-fingered dexterous hand are equipped with encoders; and the palm of the target multi-fingered dexterous hand is equipped with inertial sensors; the device includes:

[0012] The grasping method determination module is used to determine the initial grasping method of the target multi-finger dexterity hand based on the received current spatial pose data of the target object, and to control the target multi-finger dexterity hand to grasp the target object using the initial grasping method;

[0013] The sensor data acquisition module is used to acquire pressure acquisition data from the pressure sensor array and acquire current joint data of each finger joint of the target multi-fingered dexterous hand from the encoder when the target multi-fingered dexterous hand is identified by the pressure sensor array as grasping the target object.

[0014] The grasping state determination module is used to determine the current grasping state of the target multi-fingered dexterous hand based on the pressure acquisition data and the current joint data; wherein, the current joint data includes current joint position data and current joint angle data;

[0015] The grasping method correction module is used to acquire the dexterity hand pose data of the target multi-fingered dexterity hand from the inertial sensor when it is detected that the current grasping state does not meet the safe grasping conditions, and to determine the target grasping method of the target multi-fingered dexterity hand based on the current spatial pose data, the pressure acquisition data, the current joint data and the dexterity hand pose data.

[0016] The dexterous hand control module is used to control the target multi-finger dexterous hand to complete the grasping operation of the target object using the target grasping method.

[0017] According to another aspect of this application, an electronic device is provided, the electronic device comprising:

[0018] One or more processors;

[0019] Memory, used to store one or more programs;

[0020] When the one or more programs are executed by the one or more processors, the one or more processors implement any of the dexterous hand sensing and control methods provided in the embodiments of this application.

[0021] According to another aspect of this application, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements any of the dexterous hand sensing and control methods provided in the embodiments of this application.

[0022] According to another aspect of this application, a computer program product is provided, including a computer program that, when executed by a processor, implements any of the dexterous hand sensing and control methods provided in the embodiments of this application.

[0023] This application determines the initial grasping method of a target multi-fingered dexterous hand based on the received current spatial pose data of the target object, and controls the target multi-fingered dexterous hand to grasp the target object using the initial grasping method. When the target multi-fingered dexterous hand is detected to have grasped the target object by a pressure sensor array, pressure acquisition data is obtained from the pressure sensor array, and current joint data of each finger joint of the target multi-fingered dexterous hand is obtained from the encoder. Based on the pressure acquisition data and the current joint data, the current grasping state of the target multi-fingered dexterous hand is determined. If the current grasping state does not meet the safe grasping conditions, the dexterous hand pose data of the target multi-fingered dexterous hand is obtained from an inertial sensor, and the target grasping method of the target multi-fingered dexterous hand is determined based on the current spatial pose data, pressure acquisition data, current joint data, and dexterous hand pose data. Using the target grasping method, the target multi-fingered dexterous hand is controlled to complete the grasping operation of the target object. The above technical solution, by judging the grasping state of the dexterous hand during grasping and adjusting the grasping method of the dexterous hand in real time, helps to improve the grasping accuracy and efficiency of multi-finger dexterous hands. Attached Figure Description

[0024] Figure 1a This is a flowchart of a dexterous hand sensing and control method according to Embodiment 1 of this application;

[0025] Figure 1b This is a schematic diagram of a current capture state determination logic provided according to Embodiment 1 of this application;

[0026] Figure 2 This is a flowchart of a dexterous hand sensing and control method according to Embodiment 2 of this application;

[0027] Figure 3 This is a schematic diagram of the structure of a dexterous hand sensing and control device according to Embodiment 3 of this application;

[0028] Figure 4 This is a schematic diagram of the structure of an electronic device that implements the dexterous hand sensing and control method of Embodiment 4 of this application. Detailed Implementation

[0029] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0030] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0031] Furthermore, it should be noted that the collection, storage, use, processing, transmission, provision, and disclosure of relevant data such as current spatial pose data and initial grasping method involved in the technical solution of this application all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0032] Example 1

[0033] Figure 1aThis is a flowchart of a dexterous hand sensing and control method according to Embodiment 1 of this application. This embodiment is applicable to real-time detection and adjustment during the grasping process of a multi-fingered dexterous hand. It can be executed by a dexterous hand sensing and control device, which can be implemented in hardware and / or software. This device can be configured in a computer device, such as a dexterous hand control system for a target multi-fingered dexterous hand. The fingertips of the target multi-fingered dexterous hand are equipped with pressure sensor arrays; the finger joints of the target multi-fingered dexterous hand are equipped with encoders; and the palm of the target multi-fingered dexterous hand is equipped with inertial sensors. Figure 1a As shown, the method includes:

[0034] S110. Based on the received current spatial pose data of the target object, determine the initial grasping method of the target multi-finger dexterous hand, and use the initial grasping method to control the target multi-finger dexterous hand to grasp the target object.

[0035] The target object refers to the object that the target multi-fingered dexterous hand needs to grasp. It can be any three-dimensional object, such as tools, food, or toys. Current spatial pose data refers to the position and orientation information of the target object in three-dimensional space. It is usually composed of position coordinates (x, y, z) and an orientation description (such as rotation angle or quaternions), used to determine the object's exact state in space. A target multi-fingered dexterous hand is a robotic hand device with multiple flexible fingers that can mimic the grasping and manipulation capabilities of a human hand. It can achieve complex grasping tasks by controlling the movement of each finger. The initial grasping method refers to the strategy or plan initially determined for grasping the target object based on its current spatial pose data; this includes parameters such as finger position, orientation, and grasping force to ensure successful grasping of the target object.

[0036] For example, in response to an object grasping command for a target object, the grasping action of the target multi-fingered dexterous hand is determined based on the received current spatial pose data of the target object, and the target multi-fingered dexterous hand is controlled to begin performing the grasping action.

[0037] S120. When the target multi-fingered dexterous hand is identified as grasping a target object by the pressure sensor array, pressure acquisition data is obtained from the pressure sensor array, and the current joint data of each finger joint of the target multi-fingered dexterous hand is obtained from the encoder.

[0038] A pressure sensor array refers to a network or array of multiple pressure sensors used to simultaneously monitor the pressure distribution at multiple contact points. A pressure sensor is a device used to measure the pressure or force applied to its surface, detecting the contact force between a finger and a target object to determine the stability of the grip. Pressure acquisition data refers to real-time pressure information about the contact between the finger and the target object collected from the pressure sensor array. An encoder is a device used to measure motion, typically for detecting rotational or linear displacement; in dexterous hands, encoders are often used to record the angular positions of various joints for precise control of finger movement. Current joint data refers to real-time angle and position data of various joints in the target multi-fingered dexterous hand obtained from the encoder; this joint data may include at least one of current joint position data and current joint angle data.

[0039] S130. Based on the pressure acquisition data and current joint data, determine the current grasping state of the target multi-fingered dexterous hand.

[0040] The current grasping state refers to the overall description of the dexterous hand's grasping of the target object. It integrates pressure acquisition data and current joint data to assess the effectiveness and stability of the grasp, including information such as whether the grasp was successful, whether the grasping force was appropriate, and whether the finger position was correct. This state can include at least one of the following: normal grasping state, abnormal grasping point state, abnormal grasping position state, and abnormal object pose state. A normal grasping state refers to the state in which the dexterous hand successfully and stably grasps the target object; in this state, the contact force between the fingers and the object is appropriate, the object's position and posture are within controllable limits, and the grasping process meets expectations. An abnormal grasping point state refers to an abnormal situation at a certain contact point during the grasping process; this may manifest as insufficient contact force, fingers failing to correctly contact the object, or unstable contact points, leading to unsatisfactory grasping results. An abnormal grasping position state refers to the dexterous hand's overall pose or path failing to correctly align with the target object, potentially causing grasping failure or instability; this state may be caused by sensor errors, inaccurate calculations, or external interference. An abnormal object pose refers to a situation where the actual position and orientation of a target object are inconsistent with the expected state. This may be due to the object moving, tilting, or failing to be placed in the intended manner.

[0041] Optionally, based on the pressure acquisition data, determine the total fingertip pressure and the center position of the fingertip pressure of the target multi-fingered dexterous hand; based on the total fingertip pressure and the center position of the fingertip pressure, determine the candidate grasping states of each fingertip of the target multi-fingered dexterous hand; based on the candidate grasping states and the current joint position data in the current joint data, determine the current grasping state of the target multi-fingered dexterous hand.

[0042] The total fingertip pressure refers to the sum of the pressure exerted on the target object by all the fingertips of the dexterous hand; it reflects the overall force application of the dexterous hand during the grasping process and helps to determine the stability of the grasp. The center of fingertip pressure is a calculated geometric center point among the pressure distribution applied by all the fingertips; this location represents the area of ​​concentrated pressure and helps to understand the point of application and equilibrium state of the object during grasping. Candidate grasping states refer to possible grasping force states derived from the total fingertip pressure and the center of fingertip pressure; these candidate grasping states can include at least one of the following: uniform force, forward tilting force, backward tilting force, left tilting force, and right tilting force. A uniform force state means that during the grasping or supporting process, the pressure applied by the dexterous hand's fingertips on the target object is relatively evenly distributed, without significant concentration or deviation. A forward tilting force state means that during the grasping process, the pressure applied by the dexterous hand's fingertips is mainly concentrated on the front of the object, causing the object to tilt forward. The "backward tilt" state refers to a grasping state where, during the grasping process, the pressure applied by the dexterous hand's fingertips is mainly concentrated on the rear of the object, causing it to tilt backward. The "leftward tilt" state refers to a grasping state where, during the grasping process, the pressure applied by the dexterous hand's fingertips is mainly concentrated on the left side of the object, causing it to tilt to the left. The "rightward tilt" state refers to a grasping state where, during the grasping process, the pressure applied by the dexterous hand's fingertips is mainly concentrated on the right side of the object, causing it to tilt to the right. Current joint position data refers to the actual position of each joint during the grasping process.

[0043] It should be noted that determining the current grasping state of the target multi-fingered dexterous hand based on candidate grasping states and the current joint position data in the current joint data is done according to preset rules. These rules are manually preset through a large number of experiments or empirical values; for example, using... Figure 1b For example, for any fingertip in the dexterous hand, if the candidate grasping state of the fingertip is a state of uniform force and the current joint position data of the fingertip is that it has reached the designated position, then the fingertip is in a normal grasping state; if the candidate grasping state of the fingertip is a state of forward tilt under force and the current joint position data of the fingertip is that it has reached the designated position, then the fingertip is in a state of lagging grasping point; for abnormal state of part shape, it is a special case and is not within the scope of the dexterous hand grasping automation adjustment of this application. If this state occurs, the subsequent work will be stopped and an abnormal signal will be sent to the management personnel.

[0044] Furthermore, determining the candidate grasping states of each fingertip of the target multi-fingered dexterous hand based on the total fingertip pressure and the fingertip pressure center position can be achieved as follows: if the total fingertip pressure satisfies the fingertip contact condition, then the center distance between the fingertip pressure center position and the geometric center position is determined based on the fingertip pressure center position and the geometric center position of the pressure sensor array; the positional relationship between the fingertip pressure center position and the geometric center position is determined based on the fingertip pressure center position and the geometric center position; and the candidate grasping states of each fingertip of the target multi-fingered dexterous hand are determined based on the positional relationship.

[0045] In this context, the geometric center refers to the spatial geometric center of all the sensors in the pressure sensor array that make up the dexterous hand; this position represents the centroid of the entire sensor array. The center distance is the Euclidean distance between the fingertip pressure center and the geometric center of the pressure sensor array; this distance is used to assess the relationship between the concentration of pressure applied by the fingertip and the overall layout of the sensor array. The fingertip contact condition refers to the criteria or rules for evaluating the effectiveness of contact between the fingertips and the target object; meeting these conditions usually means that there is contact force between the fingertip and the target object. The positional relationship refers to the relative positional characteristics of the fingertip pressure center relative to the geometric center of the pressure sensor array; it can be coincident, offset, or other geometrical relationships.

[0046] For example, the total pressure at the fingertips can be determined using the following formula:

[0047] ;

[0048] in, This refers to the total pressure exerted by the fingertips on the target object at time k. This refers to the pressure value read by the i-th pressure sensor in the fingertip at the moment of calculation.

[0049] For example, the location of the center of pressure at the fingertip can be determined by the following formula:

[0050] ;

[0051] in, It refers to the x-coordinate of the center of gravity of the pressure distribution applied to the target object by the fingertips of the dexterous hand at time k. It refers to the centroidal coordinate of the pressure distribution applied to the target object by the fingertips of the dexterous hand at time k. It refers to the horizontal coordinate of the fixed position in space of each pressure sensor in the current calculation fingertip. It refers to the vertical coordinate of the fixed position in space of each pressure sensor in the current calculation fingertip.

[0052] For example, the location of the geometric center can be determined by the following formula:

[0053] ;

[0054] in, It refers to the x-coordinate of the sensor array center position, which is obtained by averaging the coordinates of all sensors in the current calculation fingertip. It refers to the ordinate of the sensor array center position, which is obtained by averaging the coordinates of all sensors in the current calculation fingertip.

[0055] For example, positional relationships can be calculated by the fingertips of a dexterous hand. and respectively with Comparison to determine, It is a threshold preset based on actual conditions or experience; if and If , it indicates that the candidate grasping state of the fingertip is a state of uniform force; if This indicates that the candidate grasping state of the fingertip is a forward-leaning state under force; if This indicates that the candidate grasping state of the fingertip is a backward-leaning state under force; if This indicates that the candidate grasping state of the fingertip is a leftward tilted state under force; if This indicates that the candidate grasping state of the fingertip is a rightward tilting state under force.

[0056] For example, the candidate crawl state can also be determined by the following formula:

[0057] ;

[0058] ;

[0059] in, This refers to a value used to evaluate whether a candidate grab state is left-right lightweight. This refers to a numerical value used to evaluate whether the candidate grab state is lightweight before and after. The maximum value of the x-coordinate layout in the current calculation fingertip. The maximum value of the y-coordinate layout in the current calculation fingertip.

[0060] Furthermore, if If the value is negative, it indicates that the candidate grasping state of the fingertip is a leftward tilted state under force; if If it is a positive number, it indicates that the candidate grasping state of the fingertip is a rightward tilted state under force; if If the value is negative, it indicates that the candidate grasping state of the fingertip is a backward-leaning state under force; if If the value is positive, it indicates that the candidate grasping state of the fingertip is a forward-leaning state under force. If and If all values ​​are 0, it indicates that the candidate grasping state of the fingertip is a state of uniform force.

[0061] It should be noted that the candidate grasping state can be a combination of one of the following: forward tilting state, backward tilting state, forward tilting state, and right tilting state.

[0062] S140. If it is found that the current grasping state does not meet the safe grasping conditions, the dexterity hand pose data of the target multi-fingered dexterity hand is obtained from the inertial sensor, and the target grasping method of the target multi-fingered dexterity hand is determined based on the current spatial pose data, pressure acquisition data, current joint data and dexterity hand pose data.

[0063] Among these, the safe grasping condition is used to determine whether the current grasping state is in a normal grasping state; if it is, the safe grasping condition is met; otherwise, it is not. Inertial sensors are sensors used to measure information such as the acceleration, angular velocity, and magnetic field of an object, typically including accelerometers and gyroscopes. Dexterous hand pose data refers to the data on the position and orientation of the dexterous hand in space, usually composed of position (coordinates) and orientation (rotation). Target grasping method refers to the specific grasping strategy or method developed for the target object based on given pose data, pressure acquisition data, and joint data.

[0064] For example, based on the current spatial pose data and the dexterous hand pose data, the spatial pose data of the target object is regenerated, and based on the new spatial pose data, pressure acquisition data and current joint data, a new grasping method of the dexterous hand is recalculated.

[0065] S150. Using a target grasping method, the target multi-finger dexterous hand is controlled to complete the grasping operation of the target object.

[0066] This application embodiment determines the initial grasping method of the target multi-fingered dexterous hand based on the received current spatial pose data of the target object, and controls the target multi-fingered dexterous hand to grasp the target object using the initial grasping method. When the target multi-fingered dexterous hand is detected to have grasped the target object by the pressure sensor array, pressure acquisition data is obtained from the pressure sensor array, and current joint data of each finger joint of the target multi-fingered dexterous hand is obtained from the encoder. The current grasping state of the target multi-fingered dexterous hand is determined based on the pressure acquisition data and the current joint data. If the current grasping state is found to not meet the safe grasping conditions, the dexterous hand pose data of the target multi-fingered dexterous hand is obtained from the inertial sensor, and the target grasping method of the target multi-fingered dexterous hand is determined based on the current spatial pose data, pressure acquisition data, current joint data, and dexterous hand pose data. The target grasping method is then used to control the target multi-fingered dexterous hand to complete the grasping operation of the target object. The above technical solution, by judging the grasping state of the dexterous hand during grasping and adjusting the grasping method of the dexterous hand in real time, helps to improve the grasping accuracy and efficiency of multi-finger dexterous hands.

[0067] Example 2

[0068] Figure 2 This is a flowchart of a dexterous hand perception and control method according to Embodiment 2 of this application. Based on the technical solutions of the above embodiments, this embodiment refines the process of "determining the target grasping method of the target multi-fingered dexterous hand based on current spatial pose data, pressure acquisition data, current joint data, and dexterous hand pose data" into "determining at least one candidate contact point between each fingertip of the target multi-fingered dexterous hand and the target object based on pressure acquisition data; determining candidate fingertip pose data of each fingertip of the target multi-fingered dexterous hand based on the dexterous hand pose data and at least one candidate contact point using a forward kinematics calculation method; and determining the target grasping method of the target multi-fingered dexterous hand based on the current spatial pose data, pressure acquisition data, candidate fingertip pose data, dexterous hand pose data, and current joint data." It should be noted that for parts not detailed in this embodiment, please refer to the relevant descriptions in other embodiments. Figure 2 As shown, the method includes:

[0069] S210. Based on the received current spatial pose data of the target object, determine the initial grasping method of the target multi-finger dexterous hand, and use the initial grasping method to control the target multi-finger dexterous hand to grasp the target object.

[0070] S220: When the target multi-fingered dexterous hand is identified to have grasped a target object by the pressure sensor array, pressure acquisition data is obtained from the pressure sensor array, and current joint data of each finger joint of the target multi-fingered dexterous hand is obtained from the encoder.

[0071] S230. Based on the pressure data and current joint data, determine the current grasping state of the target multi-fingered dexterous hand.

[0072] S240. If it is found that the current grasping state does not meet the safe grasping conditions, the dexterity pose data of the target multi-fingered dexterity hand is obtained from the inertial sensor, and at least one candidate contact point between each fingertip of the target multi-fingered dexterity hand and the target object is determined based on the pressure acquisition data.

[0073] Among them, the candidate contact point refers to the specific position where each fingertip contacts the surface of the target object during the process of a dexterous hand grasping the target object.

[0074] For example, based on pressure acquisition data, the total fingertip pressure of each fingertip of the target multi-fingered dexterous hand is determined; for each fingertip, if the total fingertip pressure meets the fingertip contact condition, the fingertip is determined as the contact fingertip, and based on the pressure distribution of the fingertip in the pressure acquisition data, a region with uniform and relatively high pressure distribution is selected as at least one candidate contact point between the fingertip and the target object.

[0075] S250. Based on the positive kinematics calculation method, the candidate fingertip pose data of each fingertip of the target multi-finger dexterity hand are determined according to the dexterity hand pose data and at least one candidate contact point.

[0076] Among them, forward kinematics computation is a fundamental concept in robotics, referring to the process of calculating the spatial position and orientation of an end effector (such as a finger or manipulator) based on the robot's joint parameters (such as angles and positions). Candidate fingertip pose data refers to the position and orientation data of each fingertip in three-dimensional space, calculated given the dexterous hand pose and candidate contact points.

[0077] S260. Based on the current spatial pose data, pressure acquisition data, candidate fingertip pose data, dexterous hand pose data, and current joint data, determine the target grasping method of the target multi-fingered dexterous hand.

[0078] Optionally, the candidate fingertip pose data is matched with the current spatial pose data to obtain the target fingertip pose data of each fingertip of the target multi-fingered dexterous hand; based on the Bayesian principle, the corrected spatial pose data of the target object is determined according to the target fingertip pose data and the current spatial pose data; based on the corrected spatial pose data, the target fingertip pose data, the pressure acquisition data, the dexterous hand pose data and the current joint data, the target grasping method of the target multi-fingered dexterous hand is determined.

[0079] Among them, target fingertip pose data refers to the expected position and orientation of each fingertip on the target object obtained after coordinate system matching. Bayesian principle refers to a probabilistic reasoning method that updates the probability of a belief or hypothesis using prior knowledge and observational data. Corrected spatial pose data refers to the spatial position and orientation data of the target object obtained through Bayesian inference.

[0080] In one alternative implementation, before matching the candidate fingertip pose data with the current spatial pose data to obtain the target fingertip pose data of each fingertip of the target multi-fingered dexterous hand, the current spatial pose data can be quantized into a Gaussian distribution to obtain a candidate normal distribution. Accordingly, based on the candidate normal distribution, the candidate fingertip pose data is matched with the candidate spatial pose data to obtain the target fingertip pose data of each fingertip of the target multi-fingered dexterous hand.

[0081] For example, the corrected spatial pose data can be determined by constructing and solving an objective function based on Bayesian principles. The objective function is constructed as follows:

[0082] ;

[0083] in, It refers to correcting spatial pose data. This refers to finding a parameter X that maximizes the posterior probability. This refers to a given initial pose. The prior probability described describes the likelihood of the current pose X, and this probability follows a normal distribution obtained by quantizing the Gaussian distribution of the current spatial pose data. . refers to the fingertip position Given the pose X, the likelihood probability describes the probability of the fingertip position in that pose. This refers to the current spatial pose data. This refers to the target fingertip pose data.

[0084] For example, the objective function can be solved by transforming it into a linear least squares problem and using normal equations or other numerical methods to obtain a closed-form solution for the corrected spatial pose data; the closed-form solution can be expressed by the following formula:

[0085] ;

[0086] in, It refers to the mean value of the current spatial pose data after Gaussian distribution quantization. This refers to the covariance of the current spatial pose data after Gaussian distribution quantization. H refers to the Jacobian matrix, which describes the relationship between observations and states, and is used to map the state X to the observation space. Each row corresponds to an observation, and the corresponding column is the partial derivative of the state variable. Z refers to the covariance matrix of the observation noise, representing the uncertainty or noise characteristics of the observation data. It is preset based on the understanding of sensor characteristics and environmental noise. Z refers to the actual measurement data obtained through sensors or other means, that is, the target fingertip pose data of each fingertip.

[0087] Furthermore, the target grasping method includes target joint angle data and target control torque data. Accordingly, with the joint angle limitation range of the target multi-fingered dexterous hand as a constraint and minimizing the target joint angle data as the objective, the target joint angle data of the target multi-fingered dexterous hand is determined based on the corrected spatial pose data, the target fingertip pose data, the dexterous hand pose data, and the current joint data. Based on the preset dexterous hand dynamic model of the target multi-fingered dexterous hand, with the fingertip contact force in the pressure acquisition data as a constraint and minimizing the target control torque data as the objective, the target control torque data of the target multi-fingered dexterous hand is determined based on the fingertip contact force and the target joint angle data.

[0088] Among these, target joint angle data refers to the desired angle settings of each joint of the dexterous hand when performing a grasping task. Target control torque data refers to the control torque data that needs to be applied to each joint to achieve the target joint angles. Joint angle limitation range refers to the range of motion limitations of each joint of the dexterous hand, usually determined by the mechanical structure and design. Dexterous hand dynamic model refers to a mathematical model describing the dynamic behavior of the dexterous hand during movement (such as joint movement, torque changes, etc.). Fingertip contact force refers to the contact force generated when the fingertips of the dexterous hand come into contact with the target object.

[0089] For example, the optimization function for the target joint angle data can be represented as follows:

[0090] ;

[0091] in, This refers to the target joint angle data, which can be in vector form. This refers to the joint angle data in the current joint data, which can be in vector form. This refers to the weighting coefficient, used to balance the penalty for abrupt changes in joint motion and the penalty for contact point alignment. m represents the number of candidate contact points. This refers to the target fingertip pose data for each fingertip. It refers to the hand posture in dexterity hand pose data. This refers to the location of the target contact point. It refers to the rotation matrix or transformation matrix associated with the target contact point, used to adjust the position of the target contact point. It refers to correcting spatial pose data.

[0092] For example, the dynamic model of the target control torque data can be represented by the following formula:

[0093] ;

[0094] in, This refers to the mass matrix (inertia matrix), which represents the mass at joint angles. The inertial characteristics of the lower robotic arm are a positive definite matrix. It refers to the acceleration vector of the target joint angle obtained after optimization, which represents the degree to which the joint angle changes faster over time. It refers to the Coriolis and centrifugal force matrix, which describes the additional torque caused by the movement of joints. It refers to the joint angular velocity vector of the target joint angle obtained after optimization, which represents the rate of change of the joint angle over time. This refers to the gravity matrix, which represents the force at joint angles. The torque generated by gravity. It refers to the joint torque vector of the target joint angle obtained after optimization, which represents the torque applied to the joint. It refers to the Jacobian matrix, which describes the relationship between joint angles and the position / attitude of the end effector, reflecting the influence of joint motion on the end effector. The fingertip contact force, typically a vector, represents the force applied to the end effector. Based on this, the optimization objective can be expressed as... .

[0095] In one alternative implementation, the target grasping method of the target multi-fingered dexterous hand is re-determined based on the corrected spatial pose data, target control torque data, and target joint angle data.

[0096] S270. Using a target grasping method, the target multi-finger dexterous hand is controlled to complete the grasping operation of the target object.

[0097] This application embodiment determines the initial grasping method of the target multi-fingered dexterous hand based on the received current spatial pose data of the target object, and controls the target multi-fingered dexterous hand to grasp the target object using the initial grasping method; when the target multi-fingered dexterous hand is detected to have grasped the target object by the pressure sensor array, pressure acquisition data is obtained from the pressure sensor array, and current joint data of each finger joint of the target multi-fingered dexterous hand is obtained from the encoder; the current grasping state of the target multi-fingered dexterous hand is determined based on the pressure acquisition data and the current joint data; if the current grasping state is detected to not meet the safe grasping conditions, ... This process involves acquiring dexterity hand pose data from an inertial sensor and, based on pressure data, determining at least one candidate contact point between each fingertip of the dexterity hand and the target object. Using forward kinematics calculations, candidate fingertip pose data is determined based on the dexterity hand pose data and at least one candidate contact point. The target grasping method of the dexterity hand is then determined based on the current spatial pose data, pressure data, candidate fingertip pose data, dexterity hand pose data, and current joint data. This grasping method is then used to control the dexterity hand to complete the grasping operation on the target object. This technical solution, through judging the grasping state of the dexterity hand during grasping and adjusting the grasping method in real time, helps improve the grasping accuracy and efficiency of the multi-fingered dexterity hand.

[0098] Example 3

[0099] Figure 3 This is a schematic diagram of a dexterous hand sensing and control device according to Embodiment 3 of this application. It is applicable to real-time detection and adjustment during the grasping process of a multi-fingered dexterous hand. This dexterous hand sensing and control device can be implemented in hardware and / or software and can be configured in a computer device, such as a dexterous hand control system for a target multi-fingered dexterous hand. The fingertips of the target multi-fingered dexterous hand are equipped with pressure sensor arrays; the finger joints of the target multi-fingered dexterous hand are equipped with encoders; and the palm of the target multi-fingered dexterous hand is equipped with inertial sensors. Figure 3 As shown, the device includes:

[0100] The grasping method determination module 310 is used to determine the initial grasping method of the target multi-finger dexterous hand based on the received current spatial pose data of the target object, and to control the target multi-finger dexterous hand to grasp the target object using the initial grasping method.

[0101] The sensor data acquisition module 320 is used to acquire pressure acquisition data from the pressure sensor array and acquire current joint data of each finger joint of the target multi-fingered dexterous hand from the encoder when the target multi-fingered dexterous hand is identified by the pressure sensor array as grasping the target object.

[0102] The grasping state determination module 330 is used to determine the current grasping state of the target multi-fingered dexterous hand based on the pressure acquisition data and the current joint data; wherein, the current joint data includes the current joint position data and the current joint angle data;

[0103] The grasping method correction module 340 is used to obtain the dexterity hand pose data of the target multi-fingered dexterity hand from the inertial sensor when it is detected that the current grasping state does not meet the safe grasping conditions, and to determine the target grasping method of the target multi-fingered dexterity hand based on the current spatial pose data, pressure acquisition data, current joint data and dexterity hand pose data.

[0104] The dexterous hand control module 350 is used to control the target multi-finger dexterous hand to complete the grasping operation of the target object using the target grasping method.

[0105] This application embodiment determines the initial grasping method of the target multi-fingered dexterous hand based on the received current spatial pose data of the target object, and controls the target multi-fingered dexterous hand to grasp the target object using the initial grasping method. When the target multi-fingered dexterous hand is detected to have grasped the target object by the pressure sensor array, pressure acquisition data is obtained from the pressure sensor array, and current joint data of each finger joint of the target multi-fingered dexterous hand is obtained from the encoder. The current grasping state of the target multi-fingered dexterous hand is determined based on the pressure acquisition data and the current joint data. If the current grasping state is found to not meet the safe grasping conditions, the dexterous hand pose data of the target multi-fingered dexterous hand is obtained from the inertial sensor, and the target grasping method of the target multi-fingered dexterous hand is determined based on the current spatial pose data, pressure acquisition data, current joint data, and dexterous hand pose data. The target grasping method is then used to control the target multi-fingered dexterous hand to complete the grasping operation of the target object. The above technical solution, by judging the grasping state of the dexterous hand during grasping and adjusting the grasping method of the dexterous hand in real time, helps to improve the grasping accuracy and efficiency of multi-finger dexterous hands.

[0106] Optionally, the capture method correction module 340 includes:

[0107] The contact point determination unit is used to determine at least one candidate contact point between the fingertips of the target multi-fingered dexterous hand and the target object based on the pressure acquisition data.

[0108] The pose determination unit is used to determine the candidate fingertip pose data of each fingertip of the target multi-finger dexterity hand based on the positive kinematics calculation method, according to the dexterity hand pose data and at least one candidate contact point.

[0109] The grasping method determination unit is used to determine the target grasping method of the target multi-finger dexterity hand based on the current spatial pose data, pressure acquisition data, candidate fingertip pose data, dexterity hand pose data, and current joint data.

[0110] Optionally, the capture method determination unit includes:

[0111] The pose matching subunit is used to match the candidate fingertip pose data with the current spatial pose data in the coordinate system to obtain the target fingertip pose data of each fingertip of the target multi-finger dexterous hand.

[0112] The pose correction subunit is used to determine the corrected spatial pose data of the target object based on the Bayesian principle, according to the target fingertip pose data and the current spatial pose data.

[0113] The grasping method determination subunit is used to determine the target grasping method of the target multi-fingered dexterity hand based on the corrected spatial pose data, target fingertip pose data, pressure acquisition data, dexterity hand pose data, and current joint data.

[0114] Optionally, the target grasping method includes target joint angle data and target control torque data; correspondingly, the grasping method determination sub-unit is specifically used for:

[0115] The target joint angle data of the target multi-fingered dexterous hand is determined based on the joint angle limitation range of the target multi-fingered dexterous hand, the objective of minimizing the target joint angle data, the corrected spatial pose data, the target fingertip pose data, the dexterous hand pose data, and the current joint data.

[0116] Based on the pre-defined dexterity hand dynamic model of the target multi-fingered dexterity hand, the fingertip contact force of each fingertip in the pressure acquisition data is used as a constraint, and the target control torque data is minimized as the objective. The target control torque data of the target multi-fingered dexterity hand is determined according to the fingertip contact force and the target joint angle data.

[0117] Optionally, the grasping status determination module 330 includes:

[0118] The pressure analysis unit is used to determine the total fingertip pressure and the center position of fingertip pressure of the target multi-fingered dexterous hand based on the pressure acquisition data.

[0119] The candidate state determination unit is used to determine the candidate grasping states of each fingertip of the target multi-finger dexterity hand based on the total fingertip pressure and the position of the fingertip pressure center.

[0120] The target state determination unit is used to determine the current grasping state of the target multi-fingered dexterous hand based on the candidate grasping states and the current joint position data in the current joint data; wherein, the current grasping state includes normal grasping state, grasping point abnormal state, grasping position abnormal state, and object pose abnormal state.

[0121] Optionally, a candidate state determination unit is used for:

[0122] If the total fingertip pressure meets the fingertip contact condition, then the center distance between the fingertip pressure center position and the geometric center position is determined based on the fingertip pressure center position and the geometric center position of the pressure sensor array.

[0123] Determine the positional relationship between the fingertip pressure center and the geometric center based on the fingertip pressure center position and the geometric center position;

[0124] Based on positional relationships, candidate grasping states of each fingertip of the target multi-fingered dexterous hand are determined; among them, candidate grasping states include uniform force application, forward tilting force application, backward tilting force application, left tilting force application, and right tilting force application.

[0125] The dexterous hand sensing and control device provided in this application embodiment can execute the dexterous hand sensing and control method provided in any embodiment of this application, and has the corresponding functional modules and beneficial effects for executing each dexterous hand sensing and control method.

[0126] According to embodiments of this application, this application also provides an electronic device, a readable storage medium, and a computer program product.

[0127] Example 4

[0128] Figure 4 This is a schematic diagram of the structure of an electronic device 410 implementing the dexterous hand sensing control method of the embodiments of this application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0129] like Figure 4As shown, the electronic device 410 includes at least one processor 411 and a memory, such as a read-only memory (ROM) 412 or a random access memory (RAM) 413, communicatively connected to the at least one processor 411. The memory stores computer programs executable by the at least one processor. The processor 411 can perform various appropriate actions and processes based on the computer program stored in the ROM 412 or loaded from storage unit 418 into the RAM 413. The RAM 413 may also store various programs and data required for the operation of the electronic device 410. The processor 411, ROM 412, and RAM 413 are interconnected via a bus 414. An input / output (I / O) interface 415 is also connected to the bus 414.

[0130] Multiple components in electronic device 410 are connected to I / O interface 415, including: input unit 416, such as keyboard, mouse, etc.; output unit 417, such as various types of displays, speakers, etc.; storage unit 418, such as disk, optical disk, etc.; and communication unit 419, such as network card, modem, wireless transceiver, etc. Communication unit 419 allows electronic device 410 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0131] Processor 411 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 411 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 411 performs the various methods and processes described above, such as dexterous hand perception control methods.

[0132] In some embodiments, the dexterous hand sensing control method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 418. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 410 via ROM 412 and / or communication unit 419. When the computer program is loaded into RAM 413 and executed by processor 411, one or more steps of the dexterous hand sensing control method described above may be performed. Alternatively, in other embodiments, processor 411 may be configured as the dexterous hand sensing control method by any other suitable means (e.g., by means of firmware).

[0133] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0134] Computer programs used to implement the methods of this application may be written in any combination of one or more programming languages. These computer programs may be provided to the processor of a general-purpose computer, a special-purpose computer, or other programmable dexterous hand sensing and control device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0135] In the context of this application, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0136] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0137] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0138] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0139] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this application can be achieved, and this is not limited herein.

[0140] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A dexterous hand sensing and control method, characterized in that, A dexterous hand control system applied to a target multi-fingered dexterous hand; the fingertips of the target multi-fingered dexterous hand are equipped with an array of pressure sensors; The finger joints of the target multi-fingered dexterous hand are equipped with encoders; The palm of the target multi-fingered dexterous hand is equipped with an inertial sensor; the method includes: Based on the received current spatial pose data of the target object, the initial grasping method of the target multi-fingered dexterous hand is determined, and the target multi-fingered dexterous hand is controlled to grasp the target object using the initial grasping method. When the target multi-fingered dexterous hand is identified as grasping the target object by the pressure sensor array, pressure acquisition data is obtained from the pressure sensor array, and current joint data of each finger joint of the target multi-fingered dexterous hand is obtained from the encoder; Based on the pressure acquisition data and the current joint data, the current grasping state of the target multi-fingered dexterous hand is determined; wherein, the current joint data includes current joint position data and current joint angle data; If it is detected that the current grasping state does not meet the safe grasping conditions, the dexterity hand pose data of the target multi-fingered dexterity hand is obtained from the inertial sensor, and the target grasping method of the target multi-fingered dexterity hand is determined based on the current spatial pose data, the pressure acquisition data, the current joint data and the dexterity hand pose data. Using the aforementioned target grasping method, the target multi-finger dexterous hand is controlled to complete the grasping operation of the target object.

2. The method according to claim 1, characterized in that, Based on the current spatial pose data, the pressure acquisition data, the current joint data, and the dexterous hand pose data, the target grasping method of the target multi-fingered dexterous hand is determined, including: Based on the pressure acquisition data, at least one candidate contact point between the fingertips of the target multi-fingered dexterous hand and the target object is determined; Based on the forward kinematics calculation method, the candidate fingertip pose data of each fingertip of the target multi-fingered dexterous hand are determined according to the dexterous hand pose data and the at least one candidate contact point; Based on the current spatial pose data, the pressure acquisition data, the candidate fingertip pose data, the dexterous hand pose data, and the current joint data, the target grasping method of the target multi-fingered dexterous hand is determined.

3. The method according to claim 2, characterized in that, The step of determining the target grasping method of the target multi-fingered dexterous hand based on the current spatial pose data, the pressure acquisition data, the candidate fingertip pose data, the dexterous hand pose data, and the current joint data includes: The candidate fingertip pose data is matched with the current spatial pose data in a coordinate system to obtain the target fingertip pose data of each fingertip of the target multi-finger dexterous hand. Based on Bayesian principles, the corrected spatial pose data of the target object is determined according to the target fingertip pose data and the current spatial pose data. Based on the corrected spatial pose data, the target fingertip pose data, the pressure acquisition data, the dexterous hand pose data, and the current joint data, the target grasping method of the target multi-fingered dexterous hand is determined.

4. The method according to claim 3, characterized in that, The target grasping method includes target joint angle data and target control torque data; correspondingly, determining the target grasping method of the target multi-fingered dexterous hand based on the corrected spatial pose data, target fingertip pose data, pressure acquisition data, dexterous hand pose data, and current joint data includes: The target joint angle data of the target multi-fingered dexterous hand is determined based on the corrected spatial pose data, the target fingertip pose data, the dexterous hand pose data, and the current joint data, with the constraint of the joint angle limitation range of the target multi-fingered dexterous hand and the objective of minimizing the target joint angle data. Based on the preset dexterity dynamic model of the target multi-fingered dexterity hand, with the fingertip contact force of each fingertip in the pressure acquisition data as a constraint, and with minimizing the target control torque data as the objective, the target control torque data of the target multi-fingered dexterity hand is determined according to the fingertip contact force and the target joint angle data.

5. The method according to claim 1, characterized in that, Based on the pressure acquisition data and the current joint data, the current grasping state of the target multi-fingered dexterous hand is determined, including: Based on the pressure data collected, the total fingertip pressure and the center position of the fingertip pressure of the target multi-fingered dexterous hand are determined. Based on the total fingertip pressure and the position of the fingertip pressure center, the candidate grasping states of each fingertip of the target multi-finger dexterity hand are determined; The current grasping state of the target multi-fingered dexterous hand is determined based on the candidate grasping states and the current joint position data in the current joint data; wherein, the current grasping state includes normal grasping state, grasping point abnormal state, grasping position abnormal state, and object pose abnormal state.

6. The method according to claim 5, characterized in that, The step of determining the candidate grasping states of each fingertip of the target multi-fingered dexterous hand based on the total fingertip pressure and the position of the fingertip pressure center includes: If the total fingertip pressure meets the fingertip contact condition, then the center distance between the fingertip pressure center position and the geometric center position is determined based on the fingertip pressure center position and the geometric center position of the pressure sensor array. Based on the position of the fingertip pressure center and the position of the geometric center, determine the positional relationship between the fingertip pressure center and the geometric center. Based on the positional relationship, candidate grasping states of each fingertip of the target multi-fingered dexterous hand are determined; wherein, the candidate grasping states include uniform force state, forward force state, backward force state, left force state, and right force state.

7. A dexterous hand sensing and control device, characterized in that, A dexterous hand control system configured on a target multi-fingered dexterous hand; the fingertips of the target multi-fingered dexterous hand are equipped with pressure sensor arrays; the finger joints of the target multi-fingered dexterous hand are equipped with encoders; The palm of the target multi-fingered dexterous hand is equipped with an inertial sensor; the device includes: The grasping method determination module is used to determine the initial grasping method of the target multi-finger dexterity hand based on the received current spatial pose data of the target object, and to control the target multi-finger dexterity hand to grasp the target object using the initial grasping method; The sensor data acquisition module is used to acquire pressure acquisition data from the pressure sensor array and acquire current joint data of each finger joint of the target multi-fingered dexterous hand from the encoder when the target multi-fingered dexterous hand is identified by the pressure sensor array as grasping the target object. The grasping state determination module is used to determine the current grasping state of the target multi-fingered dexterous hand based on the pressure acquisition data and the current joint data; wherein, the current joint data includes current joint position data and current joint angle data; The grasping method correction module is used to acquire the dexterity hand pose data of the target multi-fingered dexterity hand from the inertial sensor when it is detected that the current grasping state does not meet the safe grasping conditions, and to determine the target grasping method of the target multi-fingered dexterity hand based on the current spatial pose data, the pressure acquisition data, the current joint data and the dexterity hand pose data. The dexterous hand control module is used to control the target multi-finger dexterous hand to complete the grasping operation of the target object using the target grasping method.

8. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the dexterous hand sensing control method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the dexterous hand perception control method as described in any one of claims 1-6.

10. A computer program product comprising a computer program that, when executed by a processor, implements the dexterous hand sensing and control method according to any one of claims 1-6.

Images