A robotic gripper perception system based on proximity and tactile synergy for adaptive grasping

Abstract

The present invention proposes a robotic gripper perception system that combines proximity and tactile sensing for adaptive grasping. The system belongs to the field of robotic sensing technology and includes a perception module, an execution module, and a data transmission and processing module. The perception module includes a flexible transparent fingertip, a proximity sensor array, and a multiplexer. The execution module includes a three-fingered manipulator and a robotic arm. The data transmission and processing module includes a controller and a host computer. Compared with traditional proximity-tactile grasping perception schemes, a proximity-tactile collaborative perception framework with spatial consistency is established, which effectively reduces information alignment errors and significantly improves the spatiotemporal consistency and fusion efficiency of perception. It can more comprehensively characterize the multi-dimensional characteristics of the target object and adaptively adjust the grasping posture and force accordingly, achieving high-precision and high-robust grasping in a dynamic environment. In addition, the system achieves soft grasping through flexible fingers, avoiding object damage and improving grasping efficiency.

Description

Technical Field

[0001] The present invention relates to the field of robot sensing technology, and in particular to a robotic gripper sensing system with coordinated proximity and tactile senses for adaptive grasping. Background Art

[0002] In recent years, active sensing technology for robotic grippers has been widely used in complex manipulation tasks. Joint vision-tactile perception, a mainstream approach, combines the long-range information capture capabilities of vision with the fine-grained contact information perception capabilities of touch, improving the adaptive grasping performance of robotic grippers to a certain extent. However, visual perception is easily affected by illumination variations, occlusions, and background complexity in complex scenes, making it difficult to provide stable and reliable prior information. In contrast, the combination of proximity and tactile perception demonstrates significant technical advantages, enabling contactless perception while effectively avoiding occlusions. However, current sensing schemes often spatially separate proximity and tactile perception, resulting in temporal and spatial mismatches when fusing the two modalities. Furthermore, the high computational complexity makes it difficult to meet the requirements for efficient operation in dynamic environments, significantly limiting the performance of the grasping system. Furthermore, many existing grasping systems rely on rigid fingers, which present challenges when handling fragile, flexible, or irregularly shaped objects and cannot effectively avoid damage. Summary of the Invention

[0003] An embodiment of the present invention provides a robotic gripper sensing system that coordinates proximity and tactile perception for adaptive grasping. It solves the problem in traditional proximity-tactile grasping sensing schemes that proximity and tactile perception are spatially separated, resulting in temporal and spatial mismatches and high computational complexity when the two modal information is fused. It also solves the problem that rigid fingers are prone to damage to objects when handling fragile, flexible or irregularly shaped objects.

[0004] On the one hand, an embodiment of the present invention provides a proximity and tactile collaborative robotic gripper perception system for adaptive grasping, including a perception module, an execution module, and a data transmission and processing module;

[0005] The sensing module includes a flexible transparent fingertip, a proximity sensor array, and a multiplexer;

[0006] The execution module includes a three-finger manipulator and a robotic arm;

[0007] The data transmission and processing module includes a controller and a host computer;

[0008] The flexible transparent fingertips are made of highly elastic and ductile PETG material through 3D printing, imitating human fingers to achieve soft grasping of objects;

[0009] The proximity sensor array is composed of three proximity sensors arranged in an equilateral triangle on the PCB board, and the multiplexer is placed in the center of the equilateral triangle array;

[0010] The proximity sensor array feeds back proximity information and tactile information, which are spatially consistent and used for adaptive adjustment of the gripping posture and strength of the manipulator;

[0011] The proximity sensor array is tightly embedded in the flexible transparent fingertip, worn on the three-fingered robotic finger, and communicates with the controller using the IIC protocol;

[0012] The controller is connected to the host computer.

[0013] Preferably, the proximity sensor adopts a highly integrated infrared proximity sensor chip.

[0014] Preferably, the data transmission and processing module uses a UART communication protocol to realize communication between the controller and the host computer, and uses a UART communication protocol to realize communication between the controller and the execution module.

[0015] On the other hand, an embodiment of the present invention further provides a method for operating the proximity and tactile collaborative robotic gripper sensing system for adaptive grasping as described in the aforementioned embodiment, the method mainly comprising the following steps:

[0016] S1: system initialization;

[0017] S2: Proximity information collection and adaptive adjustment of grasping posture;

[0018] S3: tactile information collection and adaptive adjustment of grasping force;

[0019] S4: Grasp the target object.

[0020] Preferably, S2 includes the following steps:

[0021] S2.1: Collect raw values ​​of 9 proximity sensors from three proximity sensor arrays ,in , represents the group number of the proximity sensor array, , represents the number of the proximity sensor; the host computer uses the pre-trained neural network model to Converted to the distance to the surface of the target object ;

[0022] S2.2: Calculate the roll angles of the three groups of flexible transparent fingertips respectively and pitch angle , calculate the closest distance between three sets of flexible transparent fingertips The calculation formula is as follows:

[0023] ,

[0024] ,

[0025] ,

[0026] in, It is The rolling angle of the flexible transparent fingertips, It is The pitch angle of the flexible transparent fingertips, They are The distance from the three proximity sensors of the flexible transparent fingertips to the surface of the target object, is the side length of the equilateral triangle proximity sensor array, It is The shortest distance between the sensor array of a flexible transparent fingertip and the surface of the target object;

[0027] S2.3: , For the goal, through error feedback control, the grasping posture of the three-finger manipulator is adjusted in time so that the target object is basically located in the middle of the three fingers and the contact surfaces of the three fingers and the target object are basically parallel.

[0028] Preferably, the S3 is specifically:

[0029] When the robotic arm equipped with flexible transparent fingertips touches the surface of the target object, the proximity sensor array indirectly reflects the tactile information, which is transmitted to the host computer for sorting and analysis, and the grasping force is controlled within an appropriate range.

[0030] One or more technical solutions provided in the embodiments of the present invention have at least the following technical effects or advantages:

[0031] (1) The present invention designs a robotic gripper sensing system that integrates proximity and tactile perception for adaptive grasping. By embedding a proximity sensor array into a flexible, transparent fingertip, a spatially consistent proximity-tactile collaborative perception framework is established. Before and after contact with a target object, proximity and tactile data can be collected at the same spatial location, effectively reducing information alignment errors and significantly improving the spatiotemporal consistency and fusion efficiency of perception. This system can more comprehensively characterize the multi-dimensional characteristics of the target object and adaptively adjust the grasping posture and force accordingly, achieving high-precision and high-robust grasping in dynamic environments.

[0032] (2) The proximity sensor array designed for this system is arranged in an equilateral triangle. The positions of the three proximity sensors are relatively fixed and concise, which simplifies the calculation process of information processing and improves the real-time and efficiency of robot perception.

[0033] (3) The system uses PETG material to prepare flexible transparent fingertips with high elasticity and high ductility, simulating human fingers to achieve soft grasping, effectively improving grasping flexibility, reducing damage, and enhancing the adaptability of the grasping process. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is an overall structural diagram of a robotic gripper sensing system with proximity and tactile coordination for adaptive grasping provided in one embodiment of the present invention;

[0035] Figure 2 This is a structural diagram of a perception module of a robotic gripper perception system with proximity and tactile collaboration for adaptive grasping provided in one embodiment of the present invention;

[0036] Figure 3 This is a flowchart of a working method of a robotic gripper sensing system with proximity and tactile coordination for adaptive grasping provided in one embodiment of the present invention;

[0037] In the figure: 1. Flexible transparent fingertip; 2. Proximity sensor array; 3. Multiplexer; 4. Three-finger manipulator; 5. Robotic arm; 6. Controller; 7. Host computer. DETAILED DESCRIPTION

[0038] An embodiment of the present invention provides a robotic gripper sensing system that coordinates proximity and tactile perception for adaptive grasping. It solves the problem in traditional proximity-tactile grasping sensing schemes that proximity and tactile perception are spatially separated, resulting in temporal and spatial mismatches and high computational complexity when the two modal information is fused. It also solves the problem that rigid fingers are prone to damage to objects when handling fragile, flexible or irregularly shaped objects.

[0039] In order to more clearly illustrate the purpose, technical solutions and advantages of the present invention, the present invention will be described in detail below with reference to specific embodiments and drawings. It should be noted that the embodiments and descriptions described herein are only illustrative and do not limit the present invention in any form.

[0040] like Figure 1-Figure 3 As shown, the present invention provides a proximity and tactile collaborative robotic gripper perception system for adaptive grasping, including a perception module, an execution module, and a data transmission and processing module;

[0041] The sensing module includes a flexible transparent fingertip 1, a proximity sensor array 2, and a multiplexer 3;

[0042] The execution module includes a three-finger manipulator 4 and a manipulator arm 5;

[0043] The data transmission and processing module includes a controller 6 and a host computer 7;

[0044] The flexible transparent fingertip 1 is made of highly elastic and ductile PETG (polyethylene terephthalate glycol) material through 3D printing, imitating human fingers to achieve soft grasping of objects;

[0045] The proximity sensor array 2 consists of three proximity sensors arranged in an equilateral triangle on the PCB, with the multiplexer 3 placed in the center of the equilateral triangle array.

[0046] The proximity sensor array 2 feeds back proximity information and tactile information, which are spatially consistent and used for adaptive adjustment of the gripping posture and force of the manipulator 2;

[0047] The proximity sensor array 2 is tightly embedded in the flexible transparent fingertip 1, which is worn on the fingers of the three-fingered manipulator 4 and communicates with the controller 6 using the IIC protocol.

[0048] The controller 6 is connected to the host computer 7 .

[0049] Preferably, the proximity sensor 2 adopts a highly integrated infrared proximity sensor chip;

[0050] Preferably, the data transmission and processing module uses the UART communication protocol to realize the communication between the controller 6 and the host computer 7, and uses the UART communication protocol to realize the communication between the controller 6 and the execution module.

[0051] An embodiment of the present invention further provides a method for operating the proximity and tactile collaborative robotic gripper sensing system for adaptive grasping as described in the aforementioned embodiment, comprising the following steps:

[0052] S1: System initialization;

[0053] S2: Proximity information collection and adaptive adjustment of grasping posture;

[0054] S3: tactile information collection and adaptive adjustment of grasping force;

[0055] S4: Grasp the target object.

[0056] Preferably, S2 includes the following steps:

[0057] S2.1: Collect raw values ​​of 9 proximity sensors in three proximity sensor arrays 2 ,in , represents the group number of proximity sensor array 2, , represents the number of the proximity sensor; the host computer 7 uses the pre-trained neural network model to Converted to the distance to the surface of the target object ;

[0058] S2.2: Calculate the roll angles of the three sets of flexible transparent fingertips 1 and pitch angle , calculate the closest distance between the three groups of flexible transparent fingertips 1, the calculation formula is as follows:

[0059] ,

[0060] ,

[0061] ,

[0062] in, It is The rolling angle of the flexible transparent fingertips, It is The pitch angle of the flexible transparent fingertips, ( ) are respectively The distance from the three proximity sensors of the flexible transparent fingertips to the surface of the target object, is the side length of the equilateral triangle proximity sensor array, It is The shortest distance between the sensor array of a flexible transparent fingertip and the surface of the target object;

[0063] S2.3: , For the goal, through error feedback control, the grasping posture of the three-finger manipulator 4 is adjusted in time, so that the contact surface of the three fingers and the target object is basically parallel, and the target object is basically located in the middle of the three fingers, which can prevent the target object from being pushed down while improving the grasping stability.

[0064] Preferably, S3 is specifically as follows: when the manipulator 2 equipped with the flexible transparent fingertip 1 contacts the surface of the target object, the proximity sensor array 2 indirectly reflects the tactile information, which is transmitted to the host computer 7 for sorting and analysis, and the grasping force is controlled within an appropriate range.

[0065] This invention proposes a robotic gripper sensing system that integrates proximity and tactile perception for adaptive grasping. By embedding a proximity sensor array within a flexible, transparent fingertip, this system establishes a spatially consistent collaborative proximity-tactile perception framework. Before and after contact with a target object, proximity and tactile data are collected at the same spatial location, effectively reducing information alignment errors and significantly improving the spatiotemporal consistency and fusion efficiency of perception. This system can more comprehensively characterize the multidimensional characteristics of the target object, adaptively adjusting the grasping posture and force accordingly, achieving high-precision and robust grasping in dynamic environments.

[0066] The proximity sensor array designed for this system is arranged in an equilateral triangle. The positions between the three proximity sensors are relatively fixed and concise, which simplifies the calculation process of information processing and improves the real-time performance of the robot's perception.

[0067] The system uses PETG material to make flexible transparent fingertips with high elasticity and high ductility, simulating human fingers to achieve soft grasping, effectively improving grasping flexibility, reducing damage, and enhancing the adaptability of the grasping process.

[0068] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A robotic gripper sensing system that combines proximity and tactile sensing for adaptive grasping, characterized by: It includes a perception module, an execution module, and a data transmission and processing module; The sensing module includes a flexible transparent fingertip, a proximity sensor array, and a multiplexer; The execution module includes a three-finger manipulator and a robotic arm; The data transmission and processing module includes a controller and a host computer; The flexible transparent fingertips are made of highly elastic and ductile PETG material through 3D printing, imitating human fingers to achieve soft grasping of objects; The proximity sensor array is composed of three proximity sensors arranged in an equilateral triangle on the PCB board, and the multiplexer is placed in the center of the equilateral triangle array; The proximity sensor array feeds back proximity information and tactile information, which are spatially consistent and used for adaptive adjustment of the gripping posture and strength of the manipulator; The proximity sensor array is tightly embedded in the flexible transparent fingertip, worn on the three-fingered robotic finger, and communicates with the controller using the IIC protocol; The controller is connected to the host computer.

2. The adaptive grasping-oriented proximity and tactile collaborative robotic gripper sensing system according to claim 1, characterized in that: The proximity sensor adopts a highly integrated infrared proximity sensor chip.

3. The adaptive grasping-oriented proximity and tactile collaborative robotic gripper sensing system according to claim 1, characterized in that: The data transmission and processing module uses the UART communication protocol to realize the communication between the controller and the host computer, and uses the UART communication protocol to realize the communication between the controller and the execution module.

4. A method for operating a proximity and tactile collaborative robotic gripper sensing system for adaptive grasping according to any one of claims 1 to 3, the method comprising the following steps: S1: system initialization; S2: Proximity information collection and adaptive adjustment of grasping posture; S3: tactile information collection and adaptive adjustment of grasping force; S4: Grasp the target object.

5. The working method according to claim 4, characterized in that: The S2 comprises the following steps: S2.1: Collect raw values ​​of 9 proximity sensors from three proximity sensor arrays ,in , represents the group number of the proximity sensor array, , represents the number of the proximity sensor; the host computer uses the pre-trained neural network model to Converted to the distance to the surface of the target object ; S2.2: Calculate the roll angles of the three groups of flexible transparent fingertips respectively and pitch angle , calculate the closest distance between the three groups of flexible transparent fingertips, the calculation formula is as follows: ， ， ， in, It is The rolling angle of the flexible transparent fingertips, It is The pitch angle of the flexible transparent fingertips, They are The distance from the three proximity sensors of the flexible transparent fingertips to the surface of the target object, is the side length of the equilateral triangle proximity sensor array, It is The shortest distance between the sensor array of a flexible transparent fingertip and the surface of the target object; S2.3: , For the goal, through error feedback control, the grasping posture of the three-finger manipulator is adjusted in time so that the target object is basically located in the middle of the three fingers and the contact surfaces of the three fingers and the target object are basically parallel.

6. The working method according to claim 4, characterized in that: The S3 is specifically: When the robotic arm equipped with flexible transparent fingertips touches the surface of the target object, the proximity sensor array indirectly reflects the tactile information, which is transmitted to the host computer for sorting and analysis, and the grasping force is controlled within an appropriate range.

Images