Dexterous hand / clamping jaw integrated module and method for grabbing object by using dexterous hand / clamping jaw integrated module

Abstract

The invention relates to the technical field of robot grabbing and sensing, in particular to a dexterous hand / clamping jaw integrated module and a method for grabbing an object through the dexterous hand / clamping jaw integrated module. An existing robot dexterous hand / clamping jaw depends on external vision and touch perception, a system is complex, a view blind area exists, adaptability is poor, and cost is high. The integrated module is arranged at the fingertip of the dexterous hand or the claw tip of the clamping claw and comprises an elastic body, a cover plate and a module base which are sequentially attached to one another from front to back, the elastic body is provided with a through hole, a forward depth camera module is arranged on the front face of the module base, and the elastic body is provided with a through hole. A window hole matched with the depth camera module is formed in the cover plate, when an object is grabbed, the front face, back to the cover plate, of the elastic body makes contact with the object, and the forward depth camera module forwards obtains image and depth information through the window hole and the through hole. The integrated module simultaneously has visual positioning and force perception, does not need to depend on an external sensor, and is simplified in system, good in visual field and low in cost.

Description

Technical Field

[0001] This invention relates to the field of robot grasping and sensing technology, and in particular to an integrated module of a dexterous hand / gripper and a method for grasping objects using it. Background Technology

[0002] With the rapid development of intelligent robot technology, robots are widely used in intelligent grasping and industrial sorting, warehousing, and supermarket services, employing dexterous hands or grippers as end effectors. To achieve stable grasping, environmental adaptation, and intelligent grasping during the grasping process, the dexterous hand or gripper, as the robot's end effector, needs to possess perception capabilities throughout the entire manipulating task. To achieve dexterous control, the dexterous hand / gripper needs to have perception capabilities covering all stages of the entire operation, especially for tasks requiring precise control of contact position and release force. For example, existing dual-arm robots have a single working mode, only able to pick up and move objects using a pair of robotic arms. When objects are difficult for the robotic arms to move due to their shape, size, or weight, the robots often struggle to do so.

[0003] Current robotic dexterity hand or gripper manipulation strategies primarily rely on two types of end-effector sensing in dexterity scenarios. The first type is external vision, using hardware such as environmental cameras or wrist cameras to acquire information about the surrounding environment. The second type is tactile sensing, using hardware that integrates vision and tactile sensing with fingertip sensors to provide rich contact information after contact. In practical engineering, most end-effector sensing solutions for dexterity hands or grippers combine these two types of hardware. However, this approach still has the following drawbacks and limitations.

[0004] Problem 1: System complexity. Multiple independent sensors need to be integrated into the dexterous hand. For example, a depth camera for vision is installed on the arm or wrist of the robotic hand for global path planning and coarse positioning; encoders are installed at the finger joints to measure joint angle coordinates; single-point force / torque sensors or thin-film pressure sensors are installed at the fingertips to measure the grasping force; and tactile sensors are used to detect contact points, friction states, and whether slippage has occurred. Finally, a unified and collaborative perception of vision, touch, and force is achieved. This results in low hardware integration, large space requirements, high system redundancy, and the discrete distribution of multiple sensors leads to a complex overall structure and difficult wiring. Integration and design are difficult to achieve in small dexterous hands, and the technical challenges of structural integration design are significant.

[0005] Problem 2, blind spots: Existing dexterous hands lack environmental perception and interaction capabilities, have insufficient environmental adaptability, and their external sensor view is easily obstructed. When working in narrow or complex spaces, they cannot determine whether their own structure is restricted or whether they will have an unexpected collision with the environment, making continuous monitoring difficult and leading to damage to the mechanical structure. Frequent collisions with obstacles will cause the dexterous hand to deviate from the planned path, resulting in unstable subsequent actions. Moreover, collisions will also affect the accuracy of the hand's end-effector pose, further reducing the repeatability of operations and the success rate of grasping. Efficient and accurate operations must rely on continuous collision-free movement as a prerequisite.

[0006] Problem 3: Calibration difficulties. During the calibration process, multiple heterogeneous sensors are constrained by factors such as mechanical assembly errors, sensor installation position errors, and deformation of flexible materials, making it difficult to guarantee high accuracy and establish a unified spatial reference system, thus hindering data fusion. Furthermore, due to different sampling frequencies and transmission delays among different sensors, sensors may exhibit "asynchrony" under the same event, and misaligned timestamps can lead to incorrect state estimations during data fusion. Significant differences in the data dimensions of each sensor can cause data fusion to overwhelm effective data or require complex feature extraction and normalization. Different sensors have different noise characteristics, making it difficult to establish a unified reliability model during fusion. Multiple sensors may provide different results for the same physical quantity, making it difficult to determine which sensor's information is more reliable, which can lead to subsequent decision-making errors, thus involving issues of sensor trust and weight allocation.

[0007] Question 4: High cost: The use of multiple high-performance, high-precision sensors increases the overall cost. In subsequent data processing, multimodal sensing technology is required, which involves point cloud processing, image segmentation, signal filtering, feature extraction, and complex calculations such as deep learning or filters. This consumes a lot of computing power and poses a high challenge to the real-time processing capabilities of embedded controllers, which may lead to control loop delays and affect the stability of real-time grasping. Summary of the Invention

[0008] To address the system complexity, insufficient adaptability to blind spots, calibration difficulties, transmission asynchrony, and high cost inherent in existing robot dexterous hands / grippers that combine external vision and tactile sensing, this invention provides an integrated module for dexterous hands / grippers and its usage method. This module integrates visual positioning and force sensing modules onto the end effector, reducing reliance on other discrete external sensors, simplifying the sensor structure of the dexterous hand or gripper, lowering wiring and installation complexity, and reducing system redundancy. The integrated module is installed... Mounted on the fingertips of end effectors such as dexterous hands or grippers, it offers excellent field of view and adaptability for viewing the workspace ahead. Visual and force information throughout the operation comes from the same "fingertip" perspective, facilitating the organic fusion of multimodal sensor information, enhancing hand-eye coordination of the dexterous hand / gripper, and forming robust, closed-loop manipulation throughout the entire process. The main body of the integrated module is made of low-cost miniature cameras and elastic materials. Compared to using expensive high-precision force / torque sensors and independent tactile array sensors (matrix sensors), it ensures performance while significantly reducing system costs, resulting in a higher cost-performance ratio.

[0009] The technical solution adopted by the present invention to solve the technical problem is: an integrated module for a dexterous hand / gripper, comprising an integrated module disposed on the dexterous hand / gripper of a robot, characterized in that the integrated module is disposed at the fingertip of the front end of the dexterous hand or the claw tip of the front end of the gripper, the integrated module comprising an elastic body, a cover plate and a module base that are sequentially and tightly attached from front to back, the elastic body having a through hole, a forward depth camera module disposed on the front side of the module base, and a viewing window that matches the depth camera module on the cover plate, wherein when the robot grasps an object, the front side of the elastic body facing away from the cover plate contacts the object, and the forward depth camera module obtains images and depth information through the viewing window and the through hole.

[0010] The integrated module described in this invention contains only a depth camera module for acquiring visual information. Compared to existing combinations of camera vision and torque / tactile sensors for grasping, it has a simpler structure and easier wiring, making it easier to integrate into small dexterous hands or grippers, reducing the difficulty of structural integration design. The integrated module is located on the fingertip or claw tip at the foremost point of the dexterous hand or gripper that contacts the object. The depth camera module, through a through-hole in the elastomer, provides a clear and unobstructed imaging field of view towards the object to be grasped, covering the surrounding environment in front of the fingertip or claw tip. This allows for precise provision of relative position, depth, and the attitude information of the object to be grasped, guiding the dexterous hand or gripper to perform fine adjustments and grasping, and observing the entire process from the fingertip or claw tip approaching the object to grasping and manipulating the moving object. A cover plate acts as a transition between the elastomer and the module base, used to adjust the distance between the lens of the forward-facing depth camera module and the elastomer, preventing the lens from getting too close to or even touching the elastomer, which could affect operation. During use, the distance can be adjusted according to the selected camera. The module is equipped with cover plates of different thicknesses, eliminating the need to replace the module base, thus facilitating assembly and controlling costs. The depth camera module contains homogeneous sensors, which can acquire images and depth information using existing technologies such as binocular cameras or RGB cameras + TFO area array cameras. With the sensors integrated into one module, information from the same "fingertip" perspective helps ensure high precision and a unified spatial reference system. The depth camera module's sensors can perform position recognition and guidance before grasping, and can also confirm the precise position and whether the grasp is in place during grasping. After grasping, the depth camera module captures images of the grasped object, obtaining the object's position information relative to the depth camera. After the grasping begins, the object's position will exceed the original position of the elastic body relative to the depth camera in its free state. By comparing the difference between the position of the outer surface of the elastic body in its free state before contacting the object and the current position of the object, the deformation of the elastic body after contacting the object is used as the deformation. Combined with the force-deformation curve of the elastic body calibrated in advance, the force information of the grasped object is calculated and output. The integrated module enables the calculation of force magnitude through distance. The entire process uses a sensor at the same location, avoiding the inconsistencies in the spatial coordinate systems of individual sensors found in distributed multi-sensor combinations. This avoids the complex calculations required to fuse data from multiple sensors to complete the corresponding recognition, grasping, and force states. No additional hand-eye calibration is needed; the information fusion process is direct and accurate, helping to improve the hand-eye coordination of the dexterous hand / gripper. The integrated module mainly consists of a low-cost depth camera module made of miniature cameras and an elastomer made of elastic materials. Compared to existing robotic grasping methods that rely on distributed and expensive high-precision force / torque sensors and independent tactile array sensors (matrix sensors), the integrated module of this invention effectively reduces system costs while ensuring performance requirements, significantly improving cost-effectiveness.During operation, the forward depth camera module on the integrated module faces the inside of the fingertip / gripper towards the object being grasped. The forward depth camera module acquires images and depth information of the side of the fingertip facing the object, providing this spatial information to the control module of the dexterous hand / gripper. The controller outputs this spatial information as obstacle avoidance information to the drive module operating the dexterous hand / gripper, thereby controlling the dexterous hand or gripper to approach the object to be grasped and guiding the global path to avoid collisions. During the grasping process, the forward depth camera module compares the position of the outer surface of the elastic body before contact with the object in its free state with its position after contact. Here, the position of the outer surface of the elastic body after contact is considered the position of the target object. Combined with the pre-calibrated force-deformation curve of the elastic body, it calculates and outputs the force information of the grasped object, thus providing continuous force feedback sensing and control. The control module continuously adjusts the drive module of the dexterous hand or gripper based on force feedback results, adaptively adjusting the appropriate gripping force. This is achieved by adjusting the tightness of the gripper's grasp, preventing over-gripping or failure to grasp the object, ultimately achieving a closed-loop control effect of "vision-force perception-drive". The integration module is integrated into the robot system's dexterous hand / gripper and connects with the corresponding control module and drive module (end-effector drive mechanism) to realize path planning and obstacle avoidance before grasping, target localization and environmental monitoring during grasping, and state evaluation and force feedback after grasping. This achieves a perception chain covering the entire grasping process, avoiding the remote information transmission delay caused by the mixed use of existing scattered sensors. The dexterous hand or gripper can respond quickly in dynamic environments, improving the stability, safety, and real-time performance of grasping operations. In addition, the integration module uses only vision for full-field, non-contact measurement, avoiding the hysteresis and creep problems caused by the contact required by traditional force sensors. It has higher potential in terms of force perception accuracy and resolution, and can further enhance the grasping effect through algorithm improvements and sensor accuracy enhancements. This integrated module can be used in robot dexterous hands or grippers, and features an integrated sensing module and policy network application that provides perception throughout the entire target grasping process.

[0011] As a further improvement and supplement to the above technical solution, the present invention adopts the following technical measures: a rearward depth camera module for acquiring images and depth information is set on the reverse side of the module base. The field of view of the rearward depth camera module is exactly opposite to that of the forward depth camera module. During use, the back of the fingertip / gripper is facing away from the object, observing the back of the dexterous hand or gripper and the surrounding external environment. Its main function is to enable the dexterous hand or gripper to monitor the environmental status in real time during movement, identify potential obstacles in advance, and determine whether there is contact or collision with the surrounding environment. This provides auxiliary protection for the dexterous hand or gripper when working in complex and narrow spaces, achieving environmental safety protection and collision avoidance. Furthermore, when the forward depth camera module may not be able to fully acquire information about the surroundings of the target object due to its viewing angle, the back depth camera module's field of view is directed away from the grasped object. Using a "fingernail-sized view," it provides close-up, unobstructed stereoscopic observation of the surrounding objects of the grasped target, providing precise relative position and depth information, as well as the object's attitude information. This guides the dexterous hand or gripper to complete the final fine adjustments and grasp point alignment, helping to improve the robustness of path planning and prevent unnecessary collisions.

[0012] Both the forward depth camera module and the backward depth camera module are binocular cameras consisting of two RGB cameras, or a combination of an RGB camera and a TOF area array camera. Regardless of whether it is a forward depth camera module or a backward depth camera module, it can use existing dual RGB cameras, or a combination of an RGB camera and a TOF area array camera, to acquire image and depth information.

[0013] The elastomer consists of an annular elastic grid and two annular plates connected to its two ends, with the elastic grid and the two annular plates being integrally formed. The through hole is formed by connecting the inner holes of the elastic grid and the inner holes of the two annular plates. One of the two annular plates is fixedly attached to the cover plate, while the other annular plate serves as the front of the elastomer to contact the object being grasped. The viewing window on the cover plate is located inside the through hole. The elastomer is formed by connecting the integrally formed annular elastic grid to the two annular plates at its two ends. It can be made using 3D printing technology with suitable elastic materials such as TPU plastic raw material (thermoplastic polyurethane) or other materials with good elasticity and wear resistance. The two annular plates are respectively bonded and fixed to the elastomer and the cover plate. During operation, the grasped object contacts the outer elastic plate, and as it is gripped and pressed, the elastic plate deforms under pressure.

[0014] The module base has a front-mounted fill light mounted on the side of the forward depth camera module. The front-mounted fill light is used to provide localized illumination when the ambient light is insufficient or when the target object is in contact with an elastomer, enabling the vision sensor of the forward depth camera module to acquire images with a high signal-to-noise ratio and improving the robustness of perception.

[0015] A method for grasping objects using an integrated module of a dexterous hand / gripper, wherein the grasping method enables the dexterous hand / gripper to grasp objects through an integrated module, a control module, and a drive module disposed thereon. Both the integrated module and the drive module are communicatively connected to the control module. The control module receives data information from the integrated module, processes the data information, and outputs instructions to the drive module to control the dexterous hand / gripper to perform actions. The method is characterized in that the integrated module is one of the aforementioned dexterous hand / gripper integrated modules, and the dexterous hand / gripper grasps the object through the following sequential steps:

[0016] Step S1: Environmental perception loop. After the capture task starts, the surrounding environment information is perceived in real time. The environmental information includes three-dimensional data, including the position and posture of the target object in front, which are obtained in real time using the forward depth camera module. The control module fuses the three-dimensional data of the environmental information to form an environmental point cloud map and corrects the obstacle avoidance path to prevent collision.

[0017] Step S2: Approaching loop, the dexterous hand / gripper gradually approaches the target object according to the corrected obstacle avoidance path in step S1. During the approach, the control module calculates and outputs instructions to the drive module based on the measurement data of the forward depth camera module, and controls the dexterous hand / gripper to gradually approach the target object.

[0018] Step S3: Grasping control loop, which involves real-time force feedback adjustment from the moment the fingertips of the dexterous hand / the tips of the grippers come into contact with the target object until the grasping is completed.

[0019] The grasping method described in this invention achieves full perception of the entire grasping process by a dexterous hand / gripper through sequential steps, and has the following advantages:

[0020] (1) Closed-loop perception throughout the entire process: From the start of the task to the end of the grasping, there is real-time perception data support, realizing a complete closed loop of "environmental perception - proximity positioning - grasping control - task completion";

[0021] (2) Multi-source information fusion: Forward and backward vision sensors work together and are combined with elastic body deformation detection to achieve multi-level perception of far-field path planning, near-field positioning and contact force feedback;

[0022] (3) Adaptive gripping control: Based on the real-time force feedback adjustment mechanism of the force-deformation curve, the gripping force can stably hold the object while avoiding excessive force that could damage the object.

[0023] (4) High environmental adaptability and success rate: Even in dynamic or complex environments, dexterous hands can still improve the success rate and safety of grasping operations by means of real-time perception and adaptive control mechanisms.

[0024] The continuous loop of step S1 enables the dexterous hand / gripper to perceive potential risks in advance, avoid collisions with the surrounding environment during movement, and ensure high adaptability, high safety, and high stability of the dexterous hand / gripper during the grasping process; the continuous loop of S2 enables the dexterous hand / gripper to accurately align with the position where the target object is located, providing a position guarantee for subsequent stable grasping; the continuous loop of step S3 ensures the compliance and safety of the grasping process, enabling the dexterous hand / gripper to grasp the target object with appropriate tightness.

[0025] The measurement data and output instructions in step S2 include the following:

[0026] The measurement data is the position and distance information of the target object continuously measured by the forward depth camera module;

[0027] The output instruction is that the control module calculates the displacement and angle required for the dexterous hand / gripper to reach the target object according to the measurement data.

[0028] The real-time force feedback regulation in step S3 includes the following:

[0029] (1) The elastomer of the integrated module is deformed under pressure after contacting the target object;

[0030] (2) The forward depth camera module detects the deviation between the position of the target object and the initial position of the elastomer surface in real time as the deformation amount of the elastomer and sends it to the control module;

[0031] (3) The control module calculates the actual normal force F at the contact position between the fingertip / claw tip and the target object according to the deformation amount of the elastomer and the preset force-deformation curve of the elastomer;

[0032] (4) The control module compares the actual normal force F with the preset target grasping force F_target and executes the following control according to the result:

[0033] (4-1) If F < F_target, the control module sends an instruction to tighten the dexterous hand / gripper to the drive module to grasp the target object tightly;

[0034] (4-2) If F = F_target, the dexterous hand / gripper maintains the current grasping state and completes stable clamping;

[0035] (4-3) If F > F_target, the control module sends an instruction to relax the dexterous hand / gripper to the drive module to avoid damaging the target object.

[0036] The role of force feedback regulation is to monitor the change of force in real time during the grasping process for lower-level closed-loop grasping, aiming to perform robust closed-loop control considering the actual contact situation.

[0037] The environmental information in step S1 also includes the real-time perception of the depth information of the surrounding environment of the target behind and to the side of the dexterous hand / gripper by the back depth camera module.

[0038] Step S1 is followed by step S2: crawling and pre-crawling calculation. Steps S and S2 are performed in parallel and are completed before step S2 ends. The crawling and pre-crawling calculation includes the following:

[0039] (1) Optimization variable definition: Let the grasping parameter be g=(T,R,θ), where T is the wrist translation distance of the dexterous hand / gripper, R is the wrist rotation angle of the dexterous hand / gripper, and θ is the set of joint angles of all joints of the dexterous hand / gripper. Select n candidate contact points x on the front of the dexterous fingertip / gripper gripping surface. i And obtain the corresponding normal n from the object point cloud O. i Contact force f at each contact point i Constrained by Coulomb friction cone: ||f i t ||≤μ·f i n μ is the coefficient of friction, f i t For tangential friction, f i n The normal friction force is denoted as N. The units of both the tangential and normal friction forces are N. Here, the objective function below is optimized by constraining the optimization variables.

[0040] (2) Definition of objective function:

[0041] The overall objective is E total =E fc +w dis ·E dis +w pen ·E pen +w spen ·E spen +w joints ·E joints E fc E represents the energy of a differentiable force closing the loop. dis E represents the contact attraction term. pene E represents the penetration penalty. spen E represents the self-penetration penalty term. joints Represents joint limit penalty item, w dis w pen w spen w joints These represent the weight coefficients for the contact attraction term, penetration penalty term, self-penetration penalty term, and joint limitation penalty term, respectively.

[0042] The energy required for a small force to close the loop is G = [I3…I3; [x1] × …[x n ] × G is a linear mapping matrix from contact forces to the total six-dimensional forces and torques of the object, mapping the local forces applied at all contact points to the total forces and torques in the object's coordinate system. i ] × For x i The antisymmetric cross product matrix, c is the vector after stacking the contact forces in order, minimize E fc This is equivalent to bringing the resultant force and resultant moment generated by the friction cone close to zero, thereby tending towards a force-closed grasping action;

[0043] The contact attraction term is E dis =∑ i d(x i ,O), used to attract the contact point to the object surface, d(x i O) represents the distance from point x i The shortest distance to O;

[0044] Penalty for penetration is To achieve a geometric constraint that fits snugly but doesn't penetrate, S(H) is the set of vertices of the mesh of the robotic hand or gripper, and v is the position of a sampling point on the surface of the hand. It is an indicator function that acts as a switch. It is 1 when the hand point falls inside the object and 0 when the hand point is outside the object; d(v,O) represents the minimum penetration distance from point v to the surface of the object.

[0045] Self-collision penalty term E spen =∑ p,q∈P(H),p≠q `max(δ-‖pq‖,0)` is used to suppress self-collisions, where `P(H)` is the surface point cloud of the dexterous hand in the current state, and `p` and `q` are two non-overlapping sampling points on the surface point cloud; the joint limit penalty term is also used. Used to restrict joints from going out of bounds Let be the maximum joint angle of the i-th joint. Let be the minimum joint angle of the i-th joint;

[0046] (3) Optimize the solution method: For (T,R,θ,{x i The gradient method is used to update and iterate the solution to obtain the grasping parameter g that satisfies the requirements of friction cone force closure, low penetration and natural feasibility.

[0047] After environmental perception is completed, the grasping and pre-grasping steps begin. This step and the approach loop step are performed in parallel. Before the approach loop ends and the position is reached, the grasping posture of the dexterous hand / gripper is adjusted based on the final path and the actual position reached. This adjustment involves opening the dexterous hand / gripper and gradually adjusting it to a suitable grasping posture. The aim is to perform upper-level grasping planning when considering the geometry of the object, so that the calculated grasping parameter g meets the condition of force closure. The grasping parameter is updated and iterated before the actual grasping contact. The joints in the objective function are the various movable joints of the dexterous hand or gripper, which is the prior art.

[0048] The integrated module of the present invention has the following advantages:

[0049] (1) High integration and simplification: Visual positioning and force perception are both achieved through integrated cameras, reducing dependence on external sensors, greatly simplifying the sensor system structure of dexterous hands or grippers, reducing wiring and installation complexity, and reducing system redundancy.

[0050] (2) Enhanced perception and real-time performance throughout the entire process: As long as the integration module, control module, and drive module are all integrated on the dexterous hand / gripper, a single integrated hardware platform can realize path planning and obstacle avoidance before grasping, target positioning and environmental monitoring during grasping, and state evaluation and force feedback after grasping, forming a perception chain covering the entire process; avoiding the delay caused by remote information transmission required by the existing distributed sensor architecture, the system can respond quickly in dynamic environments, improving operational stability and real-time performance;

[0051] (3) Enhanced perception accuracy and reliability: The module uses a visual method to perform full-field, non-contact measurement of the deformation of the elastic body, avoiding problems such as hysteresis and creep that may occur in traditional force sensors, thus having higher potential in terms of force perception accuracy and resolution; at the same time, by continuously monitoring the relative position of the target and the finger during the grasping process, it can effectively prevent slippage, misalignment and unintended contact, and improve the grasping success rate.

[0052] (4) Enhanced safety: Back-view vision perception can continuously monitor the back of the dexterous hand and the surrounding environment, detect potential obstacles in advance, and effectively avoid unexpected collisions between the dexterous hand, gripper and other end effectors and the environment during the movement, thereby ensuring operational safety and the structural integrity of the end effector.

[0053] (5) Simplified hand-eye coordination and information fusion: Since visual and force information naturally come from the same perspective (i.e., fingertip perspective), their coordinate system is the same, and no additional hand-eye calibration operation is required, making the information fusion process more direct and accurate, and improving hand-eye coordination ability.

[0054] (6) Significant cost-effectiveness: The integrated module has good cost optimization potential. It can use relatively low-cost micro-cameras and elastomer materials to replace expensive high-precision force / torque sensors and independent tactile arrays, thereby reducing hardware costs and improving the system's cost-effectiveness while ensuring performance.

[0055] (7) Wide range of applications: The modular design and high integration characteristics enable it to be flexibly applied to various robot end effectors, including industrial grippers, service robot dexterous hands and medical auxiliary manipulators, and are suitable for complex and ever-changing application scenarios.

[0056] Using the aforementioned integrated module's grasping method, "through-the-loop manipulation perception" can be achieved throughout the entire grasping process, with continuous perception and evaluation carried out in the three stages of pre-contact, instantaneous contact, and post-contact. Attached Figure Description

[0057] Figure 1 : Exploded view of the integrated module structure described in Example 1.

[0058] Figure 2 : A schematic diagram of the integrated module after assembly.

[0059] Figure 3 Top view of the integrated module.

[0060] Figure 4 : Schematic diagram of the integrated module structure used by the dexterous hand.

[0061] Figure 5 : Schematic diagram of the integrated module structure of the gripper.

[0062] Figure 6 Example 2: Flowchart of the method for grasping objects.

[0063] In the diagram: 1. Module base, 2. Elastomer, 2-1. Elastic grid, 2-11. Grid strip, 2-2. Annular piece, 3. Cover plate, 3-1. Viewing window, 3-2. Lamp hole, 4. RGB camera, 5. Front fill light, 6. Integrated module, 7. First interphalangeal joint, 8. Second interphalangeal joint, 9. Finger base, 10. Hand and robotic arm connector, 11. Palm, 12. Motor, 13. Integrated module and knuckle connector, 14. Gripper body. Detailed Implementation

[0064] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0065] like Figures 1-3As shown, Embodiment 1 is an integrated module of a dexterous hand / gripper, including an integrated module 6 disposed on the dexterous hand / gripper of a robot. The integrated module can be disposed at the fingertip of the dexterous hand or the claw tip of the gripper. The integrated module 6 includes an elastic body 2, a cover plate 3, and a module base 1, which are sequentially and tightly fixed from front to back. The elastic body 2 has a hollow structure with a through hole in the center. The elastic body 2 is glued to the cover plate 3. The cover plate 3 is fixed to the module base 1 by screws in a detachable connection for replacement and adjustment. A forward depth camera module is disposed on the front of the module base 1. The cover plate 3 has a viewing window 3-1 that matches the depth camera module. When a person grasps an object, the elastic body 2 faces away from the cover plate 3 and contacts the object. The forward depth camera module obtains images and depth through the viewing window and through hole. The reverse side of the module base 1 is provided with a backward depth camera module that obtains images and depth. In this embodiment, the forward depth camera module and the backward depth camera module are each a binocular camera composed of two RGB cameras 4 arranged side by side. The binocular camera can also be replaced by a combination of an RGB camera and an area array TOF camera. A front fill light 5 is set between the two RGB cameras where the forward depth camera module is located in the module base 1. The cover plate 3 has a lamp hole 3-2 that matches the front fill light 5. The elastic body 2 includes two parallel annular plates 2-2 and an annular elastic grid 2-1. The two ends of the elastic grid are fixed to the two annular plates respectively. The three are integrally formed. The through hole is formed by connecting the inner hole of the elastic grid and the inner holes of the two annular plates. One of the two annular plates is attached and fixed to the cover plate 3. The other annular plate serves as the front contact of the elastic body 2 to grasp the object. The viewing window 3-1 on the cover plate 3 is located inside the through hole. The elastic grid 2-1 is composed of multiple grid strips 2-11 continuously distributed around the circumference. Two adjacent grid strips are connected to form a V-shape. The middle part of each grid strip 2-11 bends inward toward the annular inner hole area of ​​the elastic grid. The common connecting end at the bottom of the V-shape is alternately connected to the two annular plates 2-2 around the circumference.

[0066] like Figure 4As shown, taking the application of integrated modules to a four-fingered dexterous hand as an example, the dexterous hand includes a palm 11, an integrated module 6, and an integrated module 6 at the tip of each finger, totaling four sets of integrated modules 6 with full-process perception capabilities. Each finger has four degrees of freedom, and the structure of each finger is the same. Each finger also has four joints, resulting in a total of 16 degrees of freedom for the dexterous hand. Each joint is driven by an independent servo motor. The palm 11 also contains a drive module and a control module. Each servo motor is connected to the drive module, and the control module is connected to the sensors of the two binocular cameras and the drive module via signal lines. It is used to collect sensor unit signals and perform closed-loop control of the dexterous hand through the drive module. The fingers of the dexterous hand also include finger bases 9, second interphalangeal joints 8, first interphalangeal joints 7, integrated modules and knuckle connectors 13, and motors 12, as shown in the figure. The integrated modules 6 are fixed on the corresponding integrated modules and knuckle connectors 13. The palm 11 is connected to the robot's robotic arm through the hand-robotic arm connector 10.

[0067] In practical applications, the aforementioned integrated module can realize sensing and control functions in three stages according to the work process.

[0068] Phase 1: Far-field approximation and path planning. In this phase, the binocular camera of the forward depth camera module is in its primary working state. It uses stereo vision algorithms to acquire and process images of the environment surrounding the robot's hand movement, constructing a 3D point cloud environment model in real time and identifying information about the working environment, the target object to be grasped, and surrounding obstacles. Based on the environmental perception results, using the geometric features of the point cloud surface and combining the friction cone theory, a suitable grasping contact point is selected on the object surface, outputting the grasping point coordinates, finger grasping posture, and grasping force magnitude. Simultaneously, the spatial information provided by this module provides obstacle avoidance information to the robot controller, enabling far-field approximation and global path guidance for the hand, thereby avoiding collisions with the external environment. During this process, the binocular camera of the backward depth camera module can assist in detecting finger contact with the environment, improving the robustness of path planning and preventing unnecessary collisions during finger movement.

[0069] Phase Two: Near-Field Pre-Grasp and Precise Positioning. When the hand approaches the target, the binocular camera in the forward depth camera module may not be able to fully acquire information about the surrounding object due to its viewing angle. At this time, the binocular camera in the backward depth camera module performs obstacle recognition and approach guidance. From the fingernail's perspective, it performs close-range, unobstructed stereo observation of the objects around the target, providing precise relative position and depth information, as well as the object's posture information, thereby guiding the fingers to complete the final fine adjustment and grasping point alignment. The back sensor mainly undertakes the task of collecting obstacle information from the back during this process to avoid the risk of collision caused by hand posture adjustment. The two sets of binocular cameras, forward and backward, allow the four fingers to move in narrow spaces for auxiliary side and rear collision protection, while also providing auxiliary monitoring after the object touches the fingers. For the need for large-area environmental detection, sensors installed on the robot's chest or head that can obtain more information can also be used. The integrated module of this invention mainly completes the acquisition and recognition of environmental data in a small area of ​​tens of centimeters around the fingertips.

[0070] Phase Three: Grasping Execution and Post-Grasping Monitoring. During the grasping action, the binocular camera of the forward depth camera module works in conjunction with the hollowed-out elastic body. When the target object comes into contact with the elastic body, the binocular camera obtains the real-time deformation by comparing the positional difference of the outer surface of the elastic body in the free state and the contact state. Combined with the pre-calibrated force-deformation curve, it outputs the object's force information, thereby achieving continuous force feedback perception. Based on this feedback result, the control module outputs instructions to the drive module to perform closed-loop adjustment of the servo drive, achieving adaptive adjustment of the gripping force and avoiding over-gripping or gripping failure. At the same time, the binocular camera of the forward depth camera module can also continuously monitor the spatial position and posture of the object to confirm whether the grasping is successful and to determine whether slippage or drop has occurred during transportation. This phase forms a closed-loop control of "vision-force perception-drive" to ensure the stability and safety of the grasping process. The rear binocular vision sensor observes whether the sides and top of the fingers are in contact or collision with the surrounding environment, providing necessary auxiliary protection for dexterous hands working in complex and narrow spaces.

[0071] like Figure 5 As shown, taking the integrated module in Embodiment 1 as an example of a gripper with two claws, each claw tip of the gripper body 14 has an integrated module 6. The gripper body 14 also integrates a drive module and a control module. The control module is connected to the two binocular cameras of the integrated module 6 and the drive module, respectively. Similar to the operation process in Embodiment 1, the gripper is controlled in a closed loop by the images and data from the front and rear directions fed back by the integrated module. Similarly, the integrated module in Embodiment 1 can also be used on other end effectors such as two-finger, three-finger, and other multi-finger grippers of different shapes.

[0072] like Figure 6As shown, Embodiment 2 is a method for grasping objects using the integrated module described in Embodiment 1. This grasping method enables a dexterous hand / gripper to grasp objects through the integrated module, control module, and drive module mounted on it. Both the integrated module and the drive module are communicatively connected to the control module. The control module receives data information sent by the integrated module, processes the data information, and outputs instructions to the drive module to control the dexterous hand / gripper to perform actions. The integrated module adopts the integrated module of the dexterous hand / gripper described in Embodiment 1. After the task starts, the dexterous hand / gripper grasps the object through the following steps in sequence:

[0073] Step S1: Environmental perception loop. After the grasping task starts, the surrounding environment information is perceived in real time. The environmental information includes the three-dimensional data of the target object's position and posture in front of it, which is obtained in real time by the binocular camera of the forward depth camera module, and the depth information of the surrounding environment of the target behind the dexterous hand / gripper in the back and side rear of the binocular camera of the rear depth camera module. The control module fuses the three-dimensional data obtained by the two binocular cameras to form an environmental point cloud map and corrects the obstacle avoidance path to prevent collision.

[0074] Step S: Grasping and pre-grasping calculation. This step optimizes the grasping process by focusing on the friction cone and the estimation of differentiable force closure. Steps S and S2 are performed in parallel and are completed before the end of step S2. This includes the following:

[0075] (1) Optimization variable definition: Let the grasping parameter be g=(T,R,θ), where T is the wrist translation distance of the dexterous hand / gripper, R is the wrist rotation angle of the dexterous hand / gripper, and θ is the set of joint angles of all joints of the dexterous hand / gripper. Select n candidate contact points x on the front of the dexterous fingertip / gripper gripping surface. i And obtain the corresponding normal n from the object point cloud O. i Contact force f at each contact point i Constrained by Coulomb friction cone: ||f i t ||≤μ·f i n μ is the coefficient of friction, f i t For tangential friction, f i n The normal friction force is denoted as N. The units of both the tangential and normal friction forces are N. Here, the objective function below is optimized by constraining the optimization variables.

[0076] (2) Definition of objective function:

[0077] The overall objective is E total =E fc +w dis ·E dis +w pen·E pen +w spen ·E spen +w joints ·E joints E fc E represents the energy of a differentiable force closing the loop. dis E represents the contact attraction term. pen E represents the penetration penalty. spen E represents the self-penetration penalty term. joints Represents joint limit penalty item, w dis w pen w spen w joints These represent the weight coefficients for contact attraction, penetration penalty, self-penetration penalty, and joint limitation penalty, respectively. These four weight coefficients can be adjusted according to specific task requirements and user preferences. This embodiment provides default settings of 600, 20000, 5000, and 10000 as initial values. After obtaining optimized grasping results, these can be fine-tuned as needed. If it is desired to further reduce object penetration, the weight coefficient w of the penetration penalty can be appropriately increased. pen Conversely, the weights of the other three energy terms can be adjusted according to the desired effect. For example, if you want to further enhance the attraction of contact and make your fingers adhere more stably to the surface of the object, you can appropriately increase the weight coefficient w of the attraction of contact term. dis To reduce self-penetration, the weighting coefficient w of the self-penetration penalty term can be increased. spen If you want your hand joint movements to be more conservative and avoid pushing them to their limits, you can increase the weighting coefficient w of the joint limit penalty item. joints These coefficients can all be started from the default settings and then fine-tuned according to task requirements after observing the optimization results;

[0078] The energy required for a small force to close the loop is G = [I3…I3; [x1] × …[x n ] × G is a linear mapping matrix from contact forces to the total six-dimensional forces and torques of the object, mapping the local forces applied at all contact points to the total forces and torques in the object's coordinate system. i ] × For x i The antisymmetric cross product matrix, c is the vector after stacking the contact forces in order, minimize E fc This is equivalent to bringing the resultant force and resultant moment generated by the friction cone close to zero, thereby tending towards a force-closed grasping action;

[0079] The contact attraction term is E dis =∑ i d(x i ,O), used to attract the contact point to the object surface, d(xi O) represents the distance from point x i The shortest distance to O;

[0080] Penalty for penetration is To achieve a geometric constraint that fits snugly but doesn't penetrate, S(H) is the set of vertices of the mesh of the robotic hand or gripper, and v is the position of a sampling point on the surface of the hand. It is an indicator function that acts as a switch. It is 1 when the hand point falls inside the object and 0 when the hand point is outside the object; d(v,O) represents the minimum penetration distance from point v to the surface of the object.

[0081] Self-collision penalty term E spen =∑ p,q∈P(H),p≠q `max(δ-‖pq‖,0)` is used to suppress self-collisions, where `P(H)` is the surface point cloud of the dexterous hand in the current state, and `p` and `q` are two non-overlapping sampling points on the surface point cloud; the joint limit penalty term is also used. Used to restrict joints from going out of bounds Let be the maximum joint angle of the i-th joint. Let be the minimum joint angle of the i-th joint, where the joints are the various movable joints of a dexterous hand or gripper;

[0082] (3) Optimize the solution method: For (T,R,θ,{x i The gradient method is used to update and iterate the solution to obtain the grasping parameter g that satisfies the requirements of friction cone force closure, low penetration and natural feasibility.

[0083] Step S2: Approaching loop. The dexterous hand / gripper gradually approaches the target object according to the corrected obstacle avoidance path in step S1. During the approach, the control module calculates and outputs commands to the drive module based on the measurement data of the forward depth camera module, controlling the dexterous hand / gripper to gradually approach the target object. The measurement data is the position and distance information of the target object continuously measured by the forward depth camera module. The output commands are the displacement and angle required for the dexterous hand / gripper to reach the target object based on the measurement data.

[0084] Step S3: Grasping control loop, from the moment the fingertips of the dexterous hand / the tips of the grippers come into contact with the target object until the grasping is completed, real-time force feedback adjustment is performed. The real-time force feedback adjustment includes the following:

[0085] (1) The elastic body of the integrated module deforms under pressure after contacting the target object;

[0086] (2) The forward depth camera module detects the deviation between the position of the target object and the initial position of the elastic body surface in real time and sends it to the control module as the deformation of the elastic body.

[0087] (3) The control module calculates the actual normal force F at the contact position between the fingertip / claw tip and the target object according to the deformation amount of the elastomer and the preset force-deformation curve of the elastomer, unit: N;

[0088] (4) The control module compares the actual normal force F with the preset target grasping force F_target, and performs the following controls according to the result:

[0089] (4-1) If F < F_target, the control module sends an instruction to tighten the dexterous hand / claw to the drive module to grasp the target object tightly;

[0090] (4-2) If F = F_target, the dexterous hand / claw maintains the current grasping state to complete stable clamping;

[0091] (4-3) If F > F_target, the control module sends an instruction to relax the dexterous hand / claw to the drive module to avoid damaging the target object.

[0092] Taking the application of Example 2 to the dexterous hand for picking grapes as an example: In the pre-contact stage, visual positioning is performed on small target fruits, and the approaching trajectory is finely corrected through closed-loop control while avoiding adjacent fruit grains and branches to prevent non-target collisions; at the moment of contact, it is confirmed that multiple fingers reach the target simultaneously to achieve stable grasping under possible conditions of premature contact or environmental disturbances; in the post-contact stage, the contact state is continuously monitored and the actions are adjusted in a timely manner to maintain stable grasping and avoid damage.

[0093] After completing the stable grasping of the target object, the control module can perform operations such as handling, moving or releasing the object according to the task requirements. These are conventional operations of the existing technology. When the subsequent handling and other operations described above are completed, the entire operation task ends.

[0094] This embodiment takes the dexterous hand with four fingers and sixteen joints in Example 1 as an example, selects 287 candidate contact points, and calculates the following grasping parameters g = (T, R, θ), where

[0095] The wrist translation distance T of the dexterous hand is

[0096] [-0.020926635975536147, -0.019662993385016966, 0.06556714289032296]

[0097] The wrist rotation angle R (represented by quaternion) of the dexterous hand is

[0098] [-0.6971721649169922, 0.6863731145858765, 0.08780419826507568, -0.18743787705898285]

[0099] The set of joint angles θ of all joints in a dexterous hand is

[0100] [-0.03374765,1.44347535,0.31446635,1.17196065,-0.04601945,1.38365035,0.33440765,1.08299035,-0.27918465,1.71805795,0.10891235,1.01856335,1.63522435,0.48473825,-0.04141725,1.11827235]

[0101] The above-mentioned expressions for translation distance, rotation angle, and joint angle set are the common formats used for grasping in existing technologies.

[0102] The object-grabbing method described in Example 2 enables a dexterous hand to independently complete a complete chain of operational perception tasks without relying on an external sensing system, and has the following advantages.

[0103] 1. Far-field perception and planning: Through forward vision, the system acquires real-time 3D information of target objects and environmental obstacles in the workspace to complete target detection, pose estimation and scene modeling; combined with path planning and obstacle avoidance algorithms, it provides a safe and feasible movement trajectory for the dexterous hand.

[0104] 2. Environmental safety protection: As the dexterous hand gradually approaches the target object, it continuously perceives the spatial information behind and to the sides of the hand using back vision, identifies potential obstacles in advance, dynamically adjusts the movement trajectory, and avoids collisions with the surrounding environment; at the same time, it further corrects the precise position and posture of the target object to ensure safety and accuracy during the approach phase.

[0105] 3. Grasping process monitoring: During the gripping or grasping action, the module continuously monitors the relative positional relationship between the fingertip and the target object through multimodal perception (fusion of vision and force perception), judges the contact point, contact surface and relative movement in real time, identifies whether slippage, misalignment, empty grasp or unexpected contact occurs, and triggers protective actions when necessary.

[0106] 4. Grasping force feedback and adaptive adjustment: Combining tactile and force sensing mechanisms, the distribution and changes of grasping contact force are obtained in real time, and the grasping force and finger posture are dynamically adjusted to achieve safe and compliant grasping of objects of different materials, thereby improving the grasping success rate and task adaptability.

[0107] 5. Grasping Result Confirmation and Task Closure: After the grasping is completed, the module verifies the grasping result through visual and force information, including whether the grasp is stable, whether there is a slipping trend, and whether the grasped object meets the expected target; at the same time, it provides task completion status feedback to the upper control system to realize closed-loop perception and control throughout the entire process.

Claims

1. An integrated module for a dexterous hand / gripper, comprising an integrated module disposed on a robot's dexterous hand / gripper, characterized in that: The integrated module is set at the fingertip of the dexterous hand or the claw tip of the gripper. The integrated module includes an elastic body (2), a cover plate (3) and a module base (1) that are fixed in close contact from front to back. The elastic body (2) has a through hole. A forward depth camera module is set on the front of the module base (1). The cover plate (3) has a viewing window (3-1) that matches the depth camera module. When the robot grasps an object, the elastic body (2) contacts the object with its back to the front of the cover plate (3). The forward depth camera module obtains images and depth information through the viewing window (3-1) and the through hole.

2. The integrated module of dexterous hand / gripper according to claim 1, characterized in that: The reverse side of the module base (1) is provided with a rearward depth camera module that acquires images and depth information.

3. The integrated module of dexterous hand / gripper according to claim 2, characterized in that: Both the forward depth camera module and the backward depth camera module are binocular cameras composed of two RGB cameras (4), or a combination of an RGB camera (4) and a TOF area array camera.

4. The integrated module of dexterous hand / gripper according to claim 1, characterized in that: The elastic body (2) consists of an annular elastic grid (2-1) and two annular pieces (2-2) connected to its two ends. The elastic grid (2-1) and the two annular pieces (2-2) are integrally formed. The through hole is formed by connecting the inner hole of the elastic grid (2-1) and the inner holes of the two annular pieces (2-2). One of the two annular pieces (2-2) is attached and fixed to the cover plate (3). The other annular piece (2-2) serves as the front contact of the elastic body (2) to grasp the object. The viewing window on the cover plate (3) is located inside the through hole.

5. The integrated module of dexterous hand / gripper according to claim 1, characterized in that: The module base (1) has a front fill light (5) on the side of the forward depth camera module.

6. A method for grasping an object using an integrated module of a dexterous hand / gripper, wherein the grasping method enables the dexterous hand / gripper to grasp the object through an integrated module, a control module, and a drive module disposed thereon, wherein the integrated module and the drive module are both communicatively connected to the control module, the control module receives data information sent by the integrated module, processes the data information, and outputs instructions to the drive module to control the dexterous hand / gripper to perform activities, characterized in that... The integrated module is the dexterous hand / gripper integrated module as described in claims 1 to 5. The dexterous hand / gripper grasps an object through the following steps performed in sequence: Step S1: Environmental perception loop. After the capture task starts, the surrounding environment information is perceived in real time. The environmental information includes three-dimensional data, including the position and posture of the target object in front, which are obtained in real time using the forward depth camera module. The control module fuses the three-dimensional data of the environmental information to form an environmental point cloud map and corrects the obstacle avoidance path to prevent collision. Step S2: Approaching loop, the dexterous hand / gripper gradually approaches the target object according to the corrected obstacle avoidance path in step S1. During the approach, the control module calculates and outputs instructions to the drive module based on the measurement data of the forward depth camera module, and controls the dexterous hand / gripper to gradually approach the target object. Step S3: Grasping control loop, which involves real-time force feedback adjustment from the moment the fingertips of the dexterous hand / the tips of the grippers come into contact with the target object until the grasping is completed.

7. The method for grasping objects using an integrated module of a dexterous hand / gripper according to claim 6, characterized in that: The measurement data and output instructions in step S2 include the following: The measurement data consists of the target object's position and distance information continuously measured by the forward depth camera module; The output command is for the control module to calculate the displacement and angle required for the dexterous hand / gripper to catch the target object based on the measurement data.

8. The method for grasping objects using an integrated module of a dexterous hand / gripper according to claim 6, characterized in that: The real-time force feedback adjustment in step S3 includes the following: (1) The elastic body of the integrated module deforms under pressure after contacting the target object; (2) The forward depth camera module detects the deviation between the position of the target object and the initial position of the elastic body surface in real time and sends it to the control module as the deformation of the elastic body. (3) The control module calculates the actual normal force F at the contact position between the fingertip / claw tip and the target object according to the deformation amount of the elastomer and the preset force-deformation curve of the elastomer. (4) The control module compares the actual normal force F with the preset target grasping force F_target, and performs the following controls according to the result: (4-1) If F < F_target, the control module sends an instruction to tighten the dexterous hand / claw to the drive module to grasp the target object tightly. (4-2) If F = F_target, the dexterous hand / claw maintains the current grasping state to complete stable clamping. (4-3) If F > F_target, the control module sends an instruction to relax the dexterous hand / claw to the drive module to avoid damaging the target object.

9. The method for grasping objects using an integrated module of a dexterous hand / gripper according to claim 6, characterized in that: The environmental information in step S1 further includes the depth information of the surrounding environment of the target behind and on the side of the dexterous hand / claw sensed by the depth camera module facing away.

10. The method for grasping objects using an integrated module of a dexterous hand / gripper according to claims 6-9, characterized in that: After step S1, step S: grasping and pre-grasping calculation is performed. Step S is parallel to step S2 and is completed before the end of step S2. The grasping and pre-grasping calculation includes the following: (1) Optimization variable definition: Let the grasping parameter be g=(T,R,θ), where T is the wrist translation distance of the dexterous hand / gripper, R is the wrist rotation angle of the dexterous hand / gripper, and θ is the set of joint angles of all joints of the dexterous hand / gripper. Select n candidate contact points x on the front of the dexterous fingertip / gripper gripping surface. i And obtain the corresponding normal n from the object point cloud O. i Contact force f at each contact point i Constrained by Coulomb friction cone: ||f i t ||≤μ·f i n μ is the coefficient of friction, f i t For tangential friction, f i n The unit for both normal friction and tangential friction is N; (2) Definition of the objective function: The overall objective is E total =E fc +w dis ·E dis +w pen ·E pen +w spen ·E spen +w joints ·E joints E fc E represents the energy of a differentiable force closing the loop. dis E represents the contact attraction term. pen E represents the penetration penalty. spen E represents the self-penetration penalty term. joints Represents joint limit penalty item, w dis w pen w spen w joints These represent the weight coefficients for the contact attraction term, penetration penalty term, self-penetration penalty term, and joint limitation penalty term, respectively. The energy required for a small force to close the loop is G = [I3…I3; [x1] × …[x n ] × G is a linear mapping matrix from contact forces to the total six-dimensional forces and torques of the object, mapping the local forces applied at all contact points to the total forces and torques in the object's coordinate system. i ] × For x i The antisymmetric cross product matrix, c is the vector after stacking the contact forces in order, minimize E fc This is equivalent to bringing the resultant force and resultant moment generated by the friction cone close to zero, thereby tending towards a force-closed grasping action; The contact attraction term is E dis =∑ i d(x i ,O), used to attract the contact point to the object surface, d(x i O) represents the distance from point x i The shortest distance to O; Penalty for penetration is To achieve a geometric constraint that fits snugly but doesn't penetrate, S(H) is the set of vertices of the mesh of the robotic hand or gripper, and v is the position of a sampling point on the surface of the hand. It is an indicator function that acts as a switch. It is 1 when the hand point falls inside the object and 0 when the hand point is outside the object; d(v,O) represents the minimum penetration distance from point v to the surface of the object. Self-collision penalty term E spen =∑ p,q∈P(H),p≠q `max(δ-‖pq‖,0)` is used to suppress self-collisions, where `P(H)` is the surface point cloud of the dexterous hand in the current state, and `p` and `q` are two non-overlapping sampling points on the surface point cloud; the joint limit penalty term is also used. Used to restrict joints from going out of bounds Let be the maximum joint angle of the i-th joint. Let be the minimum joint angle of the i-th joint; (3) Optimize the solution method: For (T,R,θ,{x i The gradient method is used to update and iterate the solution to obtain the grasping parameter g that satisfies the requirements of friction cone force closure, low penetration and natural feasibility.

Images