A dual-modal sensing system for proximity and touch for dexterous fingertips

Through the combination of flexible photoelectric proximity sensing and flexible tactile sensing, the cross-fitting gold embedded electrode and piezoresistive piezoelectric tactile array sensor are used to solve the ability to perceive flexible small objects in an unstructured environment. The problem of the skilled hands achieves high flexibility and precise operation capabilities.

CN115674231BActive Publication Date: 2024-11-26NANCHANG UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202211431914.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-11-16
Publication Date
2024-11-26
Estimated Expiration
2042-11-16

AI Technical Summary

Technical Problem

The existing smart hand sensing technology is difficult to determine the position of flexible and small objects and control the grip force in an unstructured environment, and it faces challenges in identifying, sorting and assembly of different parts.

Method used

Adopt a proximity and tactile dual-mode sensing system combining flexible photoelectric proximity sensing and flexible tactile sensing to adjust the hand posture through non-contact and contact exploratory movements, using polydimethylsiloxane PDMS elastic film, cross-fitting gold embedded electrodes, piezoresistive and piezoelectric haptic array sensors combine tricolor high-efficiency polymerized light emitting diodes and organic photodetectors to achieve high-resolution static pressure and vibration haptic perception.

Benefits of technology

It improves the accurate perception and precise operation ability of the object to be tested by a clever hand, makes up for the limitations of the position and posture determination of the blocked object, and has the characteristics of light weight, small size and fit, suitable for integration into multiple clever fingertips.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115674231B_ABST
    Figure CN115674231B_ABST
Patent Text Reader

Abstract

The present invention discloses a proximity and touch dual-mode sensing system applied to dexterous fingertips, belonging to the field of robot sensing technology, including a conformable protective layer and a dielectric layer, and a data acquisition and processing device; the protective layer is provided with an embedded electrode I and a tactile array sensor I, both of which are bonded to the dielectric layer; the dielectric layer is provided with an embedded electrode II and a tactile array sensor II, wherein a flexible and conformable three-color high-efficiency polymerized light-emitting diode (PLED) and an organic photodetector (OPD) are provided between the embedded electrode I and the embedded electrode II, and the data acquisition and processing device is mounted at the end of the sensing system. The present invention realizes the dexterous detection of finger proximity and contact measurement by preparing a highly integrated flexible proximity and touch fingertip sensing system, making up for the limitations of the robot's visual touch grasping perception, and the multi-modal fingertip sensor also has the characteristics of softness, conformability, simple structure, and easy integration, which is very attractive for future close-range active interaction between machines and objects.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the field of robot sensing technology, in particular to a proximity and touch dual-mode sensing system applied to dexterous fingertips. Background Art

[0002] Most of the dexterous hands currently available on the market are two-fingered or three-fingered based on tactile perception, and object perception and grasping control are greatly limited. In unstructured, cluttered and crowded environments, dexterous hands need to have more flexibility to adapt to the grasping requirements of different objects. This requires the dexterous hand to actively perceive the grasped object, complete the object posture estimation and hand posture adjustment before grasping. At present, the existing sensing technology for the active perception of objects by dexterous hands is to use the combined perception of vision and touch, identify the position and posture of the grasped object through machine vision, and control the movement of the dexterous hand to complete the touch and grasp. However, the existing sensing technology not only has a large amount of calculation and slow recognition speed, but also it is usually difficult to determine the position and control the grasping force of flexible and small objects when they are occluded; in the recognition, sorting and assembly of different parts, the existing sensing technology still faces challenges. Summary of the invention

[0003] In view of the single sensing mode of dexterous fingertips on the market and the shortcomings of existing sensing technology, the present invention provides a proximity and touch dual-modal sensing system applied to dexterous fingertips. The system provides a lightweight, small and highly integrated approach and touch sensing solution. Compared with traditional single-function tactile sensing, the present invention combines flexible photoelectric proximity sensing with flexible tactile sensing, and can adjust the hand posture of the dexterous hand through non-contact and contact exploratory movements of the occluded area of ​​the object to be measured, thereby achieving accurate perception and precise operation of the dexterous hand on the object to be measured.

[0004] The present invention has the following characteristics:

[0005] A proximity and touch dual-mode sensing system for dexterous fingertips includes a dual-mode fingertip sensing unit and a data acquisition and processing device. The dual-mode fingertip sensing unit includes a conformable protective layer and a dielectric layer, wherein the protective layer is provided with an embedded electrode I and a tactile array sensor I, and the dielectric layer is provided with an embedded electrode II and a tactile array sensor II, and the data acquisition and processing device is mounted at the end of the sensing system.

[0006] The protective layer and the dielectric layer are laminated up and down, and the material of the protective layer and the dielectric layer is polydimethylsiloxane (PDMS) elastic film.

[0007] The embedded electrode I and the embedded electrode II are cross-laminated, and both are made of gold; the tactile array sensor I is an n×n piezoresistive tactile array, composed of a flexible force-sensitive material loaded with silver nanofibers; the tactile array sensor II is an n×n piezoelectric tactile array; the tactile array sensor I and the tactile array sensor II are laminated up and down, together forming a piezoresistive and piezoelectric hybrid pressure sensor array for high-resolution detection of the spatial distribution of static pressure levels and the frequency information of vibration tactile stimulation.

[0008] The embedded electrodes I contain flexible and conformable three-color high-efficiency polymerized light-emitting diodes (PLEDs), and the embedded electrodes II contain organic photodetectors (OPDs) to form a proximity array sensor;

[0009] The tactile array sensor I and the tactile array sensor II are provided with 0° soft glue with a thickness of 3-8 mm to ensure that the height of the tactile force measurement is higher than other devices, thereby reducing the measurement error.

[0010] The data acquisition and processing device hardware is mounted on a flexible printed circuit (FPC) made of polyimide film. The data acquisition and processing device includes a positive and negative power supply interface, an n+4-way AD sampling unit, an n-way bidirectional integrated switch, a low-power computing unit, and an IIC data transmission unit.

[0011] The dual-mode fingertip sensing unit and the data acquisition and processing device are connected via a flexible connection film printed with conductive ink.

[0012] Preferably, in the dual-modal fingertip sensing unit, the tactile array sensor I and the tactile array sensor II are placed in the middle, and the proximity array sensor is placed around, and they are symmetrically placed up and down and left and right.

[0013] Preferably, the flexible connecting film printed with conductive ink is a film material such as PI film, PET film, etc., which has good flexibility, high insulation and is easy to adhere to the corresponding conductive ink.

[0014] The specific preparation process of the present invention is as follows:

[0015] 1) The flexible insulating silicon substrate I and the silicon substrate II are cleaned with acetone, ethanol and deionized water in turn, the surface is treated with an oxygen ion cleaner, and the silicon substrate is modified with octadecyltrichlorosilane (i.e., an OTS-modified substrate);

[0016] 2) preparing an embedded electrode I on the insulating silicon substrate I and an embedded electrode II on the insulating silicon substrate II by using vacuum evaporation technology;

[0017] 3) preparing a PDMS solution by spin coating and curing with a volume ratio of 10:1 between PDMS and a curing agent (Dow Corning, silicone resin 184), and evenly dropping the solution onto the surface of the silicon substrate to prepare the protective layer and the dielectric layer;

[0018] 4) Using double-sided adhesive to align and bond the protective layer and the dielectric layer;

[0019] 5) using a mechanical probe transfer method to transfer the three-color high-efficiency polymerized light-emitting diode, the organic photodetector and the flexible force-sensitive tactile array;

[0020] 6) Encapsulating the wiring harness of the dual-modal fingertip sensing unit using PDMS and connecting it to a flexible circuit board through a flexible film printed with conductive ink;

[0021] 7) The data acquisition and processing device adopts the general flexible circuit board preparation process;

[0022] Preferably, the operation of the modal fingertip sensing system mainly includes the following steps:

[0023] Step 1: When the object is close to the dual-mode fingertip sensor, the organic photodetector detects the intensity change of the reflected three-color high-efficiency polymer light-emitting diode light signal and outputs the corresponding voltage value. The output signal V at both ends of the proximity sensor S1 、V S3 , the output signal V at both ends of the vertical direction S2 、V S4 , sensor array internal resistance r, sensor array external resistance R0; proximity sensor output signal V S1 、V S2 、V S3 、V S4 get:

[0024]

[0025]

[0026]

[0027] I all is the output parameter of the approach distance, x c and c The four-channel AD sampling unit collects the average value of the proximity voltage output, and the formula 1-3 is used to recursively calculate x in real time. c and c ;

[0028] Step 2: When the object under test contacts the dual-modal fingertip sensor, the double-layer n×n piezoresistive and piezoelectric hybrid pressure sensor array is deformed. The force-sensitive tactile array rows are selected through n-way bidirectional integrated switches, and then the row voltages are collected k times in time periods using n-way AD sampling units and the average value is output, and the n×n force-sensitive measurement array value is calculated by real-time weighted recursion;

[0029] Step 3: Transmit the dual-modal fingertip sensing data to the terminal in real time through IIC;

[0030] The beneficial effects of the present invention are:

[0031] (1) Compared with the traditional dexterous hand with single tactile perception, the highly integrated proximity and tactile combined perception method can improve the flexibility of the dexterous hand before and after contact with the object to be measured, and make up for the limitation that the current position and posture cannot be determined due to the obstruction of the object.

[0032] (2) The sensor system is light, soft, small, and conformable, and can be easily integrated into multiple dexterous fingertips through data acquisition and processing devices; it has a detachable, easy-to-integrate structure and data transmission characteristics.

[0033] (3) It provides new ideas and inspiration for robots to use the combined perception of proximity and touch to improve their ability to accurately perceive and precisely manipulate objects. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a front composition block diagram of the present invention;

[0035] Figure 2 It is a block diagram of the back of the present invention;

[0036] Figure 3 It is the working wiring diagram of the present invention;

[0037] Figure 4 It is a schematic diagram of the structure of the present invention;

[0038] Figure 5 The present invention is a kind of work flow chart.

[0039] Among them, 1-dual-modal fingertip sensing unit, 11-proximity array sensing, 121-tactile array sensing I, 122-tactile array sensing II, 13-0° soft glue, 14-protective layer, 15-dielectric layer, 2-data acquisition and processing device, 21-data acquisition and processing device hardware, 22-flexible circuit board, 3-flexible connecting film, 4-terminal. DETAILED DESCRIPTION

[0040] The embodiments of the present invention are further described below in conjunction with the accompanying drawings.

[0041] like Figure 1-2As shown, a proximity and touch dual-mode sensing system for dexterous fingertips includes a dual-mode fingertip sensing unit 1 and a data acquisition and processing device 2. The dual-mode fingertip sensing unit 1 includes a conformable protective layer 14 and a dielectric layer 15, wherein the protective layer 14 is provided with an embedded electrode I and a tactile array sensor I 121, and the dielectric layer 15 is provided with an embedded electrode II and a tactile array sensor II 122, and the data acquisition and processing device 2 is mounted at the end of the sensing system.

[0042] The protective layer 14 and the dielectric layer 15 are laminated up and down, and the material of the protective layer 14 and the dielectric layer 15 is polydimethylsiloxane (PDMS) elastic film.

[0043] The embedded electrode I and the embedded electrode II are cross-laminated, and both are made of gold; the tactile array sensor I 121 is a 4×4 piezoresistive tactile array, composed of a flexible force-sensitive material loaded with silver nanofibers; the tactile array sensor II 122 is a 4×4 piezoelectric tactile array; the tactile array sensor I 121 and the tactile array sensor II 122 are laminated up and down, together forming a piezoresistive and piezoelectric hybrid pressure sensor array for high-resolution detection of the spatial distribution of static pressure levels and the frequency information of vibration tactile stimulation.

[0044] The embedded electrodes I contain flexible and conformable three-color high-efficiency polymerized light-emitting diodes (PLEDs), and the embedded electrodes II contain organic photodetectors (OPDs) to form a proximity array sensor 11;

[0045] The tactile array sensor I 121 and the tactile array sensor II 122 are provided with 0° soft glue 13 with a thickness of 3-8 mm, so as to ensure that the height of the tactile force measurement is higher than other devices and reduce the measurement error.

[0046] The data acquisition and processing device hardware is mounted on a flexible circuit board (FPC) 22 made of polyimide film. The data acquisition and processing device includes a positive and negative power supply interface, an eight-way AD sampling unit, a four-way bidirectional integrated switch, a low-power computing unit, and an IIC data transmission unit.

[0047] The dual-mode fingertip sensing unit 1 and the data acquisition and processing device 2 are connected via a flexible connection film 3 printed with conductive ink.

[0048] Preferably, in the dual-modal fingertip sensing unit, the tactile array sensor I 121 and the tactile array sensor II 122 are placed in the middle, and the proximity array sensor 11 is placed around, and they are placed symmetrically up and down and left and right.

[0049] Preferably, the flexible connecting film 3 printed with conductive ink is a film material such as PI film, PET film, etc., which has good flexibility, high insulation and is easy to adhere to the corresponding conductive ink.

[0050] The specific preparation process of the present invention is as follows:

[0051] 1) The flexible insulating silicon substrate I and the silicon substrate II are cleaned with acetone, ethanol and deionized water in turn, the surface is treated with an oxygen ion cleaner, and the silicon substrate is modified with octadecyltrichlorosilane (i.e., an OTS-modified substrate);

[0052] 2) preparing an embedded electrode I on the insulating silicon substrate I and an embedded electrode II on the insulating silicon substrate II by using vacuum evaporation technology;

[0053] 3) preparing a PDMS solution by spin coating and curing with a volume ratio of 10:1 of PDMS and a curing agent (Dow Corning, silicone resin 184), and evenly dropping the solution onto the surface of the silicon substrate to prepare the protective layer 14 and the dielectric layer 15;

[0054] 4) Using a double-sided adhesive to align and bond the protective layer 14 and the dielectric layer 15;

[0055] 5) using a mechanical probe transfer method to transfer the three-color high-efficiency polymerized light-emitting diode, the organic photodetector and the flexible force-sensitive tactile array;

[0056] 6) Encapsulating the wiring harness of the dual-mode fingertip sensing unit 1 using PDMS, and connecting it to the flexible circuit board 22 via a flexible connecting film 3 printed with conductive ink;

[0057] 7) The data acquisition and processing device 2 adopts a common flexible circuit board 22 preparation process;

[0058] Preferably, combined Figure 3-5 The dual-mode fingertip sensing system mainly includes the following steps:

[0059] Step 1: Install multiple dual-modal fingertip sensing systems on the Barrett hand, and connect the multiple dual-modal fingertip sensing systems through hardware IIC to establish a communication relationship;

[0060] Step 2: Grasp the object using the Barrett hand equipped with the fingertip sensing system of the present invention;

[0061] Step 3: When the object is close to the dual-mode fingertip sensor, the organic photodetector detects the intensity change of the reflected three-color high-efficiency polymer light-emitting diode light signal and outputs the corresponding voltage value. The output signal V at both ends of the proximity sensor S1 、V S3 , the output signal V at both ends of the vertical direction S2 、V S4 , sensor array internal resistance r, sensor array external resistance R0; proximity sensor output signal V S1、V S2 、V S3 、V S4 get:

[0062]

[0063]

[0064]

[0065] I all is the output parameter of the approach distance, x c and c The four-channel AD sampling unit collects the average value of the proximity voltage output, and the formula 1-3 is used to recursively calculate x in real time. c and c ;

[0066] Step 4: When the object under test contacts the dual-modal fingertip sensor, the double-layer 4×4 force-sensitive measurement array deforms. The force-sensitive tactile array rows are selected through four-way bidirectional integrated switches, and then the four-way AD sampling units collect k row voltages in time periods and output the average value, and the 4×4 force-sensitive measurement array value is calculated by real-time weighted recursion;

[0067] Step 5: The fingertip sensing data is transmitted to the Barrett hand dexterous hand terminal 4 in real time through the IIC;

[0068] Step 6: The Barrett hand uses finger proximity detection and contact measurement to accurately sense and grasp the object to be measured;

[0069] Compared with the traditional dexterous hand with single tactile perception, the present invention adopts a highly integrated approach of combined proximity and tactile perception to improve the flexibility of the dexterous hand before and after contacting the object to be measured, and makes up for the limitation that the current position and posture cannot be determined due to the obstruction of the object.

[0070] The invention has the characteristics of light weight, softness, small size and adaptability, and can be conveniently integrated into multiple dexterous fingertips through a data acquisition and processing device; it has a detachable, easy-to-integrate structure and data transmission characteristics.

[0071] The present invention brings new ideas and inspirations to the robot to improve the accurate perception and precise operation capabilities of the object to be measured by using the combined perception of proximity sense and touch sense.

[0072] The specific embodiments of the present invention are described in detail above, but they are only examples, and the present invention is not limited to the specific embodiments described above. For those skilled in the art, any equivalent modifications and substitutions made to the present invention are also within the scope of the present invention. Therefore, the equalization changes and modifications made without departing from the spirit and scope of the present invention are all within the scope of the present invention.

Claims

1. A proximity and touch dual-mode sensing system for dexterous fingertips, characterized by: It includes a dual-mode fingertip sensing unit and a data acquisition and processing device. The dual-mode fingertip sensing unit includes a conformable protective layer and a dielectric layer. The protective layer is provided with an embedded electrode I and a tactile array sensor I. The dielectric layer is provided with an embedded electrode II and a tactile array sensor II. The data acquisition and processing device is mounted at the end of the sensing system. The protective layer and the dielectric layer are laminated up and down, and the material of the protective layer and the dielectric layer is polydimethylsiloxane (PDMS) elastic film; The embedded electrode I and the embedded electrode II are cross-laminated, and the materials of both are gold; The embedded electrodes I contain flexible and conformable three-color high-efficiency polymerized light-emitting diodes, and the embedded electrodes II contain organic photodetectors to form a proximity array sensor; Tactile array sensor I The piezoresistive tactile array is composed of a flexible force-sensitive material loaded with silver nanofibers; Tactile array sensor II The piezoelectric tactile array, tactile array sensor I and tactile array sensor II are laminated together to form a piezoresistive and piezoelectric hybrid pressure sensor array for high-resolution detection of the spatial distribution of static pressure levels and the frequency information of vibration tactile stimulation; The tactile array sensor I and the tactile array sensor II are mounted on Thick soft glue; The dual-mode fingertip sensing unit and the data acquisition and processing device are connected via a flexible connection film printed with conductive ink; The hardware of the data acquisition and processing device is mounted on a flexible circuit board made of polyimide film; The data acquisition and processing device includes a positive and negative power supply interface, AD sampling unit, Bidirectional integrated switch, low power computing unit, Data transmission unit; The invention also includes a method for a proximity and touch dual-mode sensing system applied to dexterous fingertips, which uses a proximity and touch dual-mode sensing system applied to dexterous fingertips, and includes the following steps: Step 1: When the object is close to the dual-mode fingertip sensor, the organic photodetector detects the intensity change of the reflected three-color high-efficiency polymer light-emitting diode light signal and outputs the corresponding voltage value, and the proximity sensor outputs signals at both ends. , output signal at both ends of the vertical direction , the resistance within the sensor array , the external resistance of the sensor array ; Output signal from proximity sensor And the positive reference voltage V0 is obtained: Formula 1: Formula 2: Formula 3: To approach the distance output parameter, and for and The position of the direction is collected through the four-way AD sampling unit to collect the average value of the proximity voltage output, and the real-time weighted recursion is performed through formula 1-3 and ; Step 2: When the object to be measured contacts the dual-mode fingertip sensor, the double layer The force sensitive tactile array is deformed by measuring the force sensitive array. The four-way bidirectional integrated switch is used to select the force sensitive tactile array row, and then the AD sampling unit collects data in different time periods The row voltage is output and the average value is calculated by real-time weighted recursion. Force sensitive measurement array value; Step 3: Transmit the dual-modal fingertip sensing data through Real-time transmission to the terminal.

2. The proximity and touch dual-mode sensing system for dexterous fingertips according to claim 1, characterized in that: In the dual-modal fingertip sensing unit, the tactile array sensor I and the tactile array sensor II are placed in the middle, and the proximity array sensors are placed around them, and are symmetrically placed up and down and left and right.

3. The proximity and touch dual-mode sensing system for dexterous fingertips according to claim 1, characterized in that: The flexible connecting film is a PI film or a PET film.