Multi-modal fusion touch fingertip module

Abstract

The utility model discloses a multi-mode fusion touch fingertip module which comprises a supporting layer (107). The outer surface of the supporting layer (107) is provided with a proximity sense sensing layer (101) used for sensing the distance, a temperature site (102) used for sensing the temperature, a touch sense / slip sense sensing layer (103) used for sensing the touch sense, a control element (104), an electrode layer (105) used for transmitting signals and a force sensing layer (106) used for sensing the force. According to a three-dimensional force three-dimensional tactile sensing network, multi-mode sensor signals can be obtained, a multi-branch tactile neural network is constructed by using the signals, and a tactile fingertip module is formed, so that the shortages of two-dimensional tactile sensing can be made up, and three-dimensional characteristics of tactile sensing can be obtained; that is to say, richer haptic information such as output magnitude, direction, acting position, spatial distribution of force, dynamic force time sequence characteristics, spatial distribution of temperature and the like can be decoupled, various characteristics of the object are extracted, and the accuracy of object haptic recognition is improved.

Description

Technical Field

[0001] The utility model relates to the field of terminal technology, and in particular to a multi-modal fusion tactile fingertip module. Background Art

[0002] As an actuator mounted on the end of a humanoid robot, the dexterous hand mimics the structure and function of the human hand, enabling manipulation of objects in various scenarios. Its sensitive external sensing capabilities (such as proximity, multi-dimensional force detection, temperature, and touch / slip sensing) ensure precise detection of fingertip contact force and joint position, enabling precise position and force control of objects.

[0003] A Chinese patent (publication number: CN108673540A) discloses a mechanical finger and manipulator with flexible freedom and dexterity, but lacks tactile perception. A Chinese patent (publication number: CN112025750A) discloses a piezoelectric-piezoresistive composite humanoid tactile finger and its preparation method. While the finger has sensory capabilities, its sensory parameters are limited, and the finger is manufactured separately, resulting in poor integration and limited versatility. Utility Model Content

[0004] In response to the problem of low perception ability of robot fingers in the existing technology, the utility model proposes a multi-modal fusion tactile fingertip module.

[0005] In order to achieve the above objectives, the present invention provides the following technical solutions:

[0006] A multimodal fusion tactile fingertip module includes a support layer 107, the outer surface of which is installed with a proximity sensing layer 101 for sensing distance, a temperature site 102 for sensing temperature, a tactile / slip sensing layer 103 for sensing touch, a control element 104, an electrode layer 105 for transmitting signals, and a force sensing layer 106 for sensing force.

[0007] Preferably, the support layer 107 is a finger structure.

[0008] Preferably, the proximity sensing layer 101 is disposed on the upper surface of the tactile / slip sensing layer 103 , and the proximity sensing layer 101 is connected to the electrode layer 105 ; the tactile / slip sensing layer 103 is disposed on the upper surface of the electrode layer 105 .

[0009] Preferably, the touch / slip sensing layer 103 includes a first substrate 1031 , a first microstructure 1032 and a second microstructure 1033 ; wherein the first microstructure 1032 and the second microstructure 1033 are alternately arranged on the lower surface of the first substrate 1031 .

[0010] Preferably, the height of the first microstructure 1032 is smaller than the height of the second microstructure 1033 .

[0011] Preferably, the temperature site 102 penetrates the proximity sensing layer 101 and the touch / slip sensing layer 103 and contacts the upper surface of the electrode layer 105 ; the control element 104 penetrates the force sensing layer 106 and contacts the lower surface of the electrode layer 105 .

[0012] Preferably, the electrode layer 105 includes an FPC as a substrate and electrodes; the electrodes are a mesh structure.

[0013] Preferably, the force sensing layer 106 includes a second substrate 1061 and a third microstructure 1062 , and the third microstructure 1062 is distributed at the upper end, left end, lower end, and right end of the second substrate 1061 .

[0014] In summary, due to the adoption of the above technical solution, compared with the prior art, the present invention has at least the following beneficial effects:

[0015] The present invention is based on a three-dimensional force-stereoscopic tactile perception network, which can obtain multimodal sensor signals and use these signals to construct a multi-branch tactile neural network (tactile fingertip module), including a proximity sensing layer 101, a temperature site 102, a touch / slip sensing layer 103, a controller 104, an electrode layer 105, and a force sensing layer 106. It can not only make up for the shortcomings of two-dimensional tactile sensing, but also obtain three-dimensional characteristics of tactile perception, that is, it can decouple the force size, direction, action position, spatial distribution of force, dynamic force timing characteristics, spatial distribution of temperature, etc. to enrich force tactile information, extract multiple characteristics of objects, and improve the accuracy of tactile recognition of objects.

[0016] At the same time, the curved surface design of the electrode layer 105 increases the sensing range of the finger, reduces the difficulty of installation, and improves the sensing range and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 Schematic diagram of a multimodal fusion tactile fingertip module according to an exemplary embodiment of the present invention.

[0018] Figure 2 Schematic diagram of the planar structure of each module in a multimodal fusion tactile fingertip module according to an exemplary embodiment of the present invention.

[0019] Figure 3 Schematic diagram of an electrode structure according to an exemplary embodiment of the present invention.

[0020] Figure 4 Schematic diagram of the force sensing layer structure according to an exemplary embodiment of the present invention.

[0021] Figure 5 The figure is a flow chart of a method for preparing a multimodal fusion tactile fingertip module according to an exemplary embodiment of the present invention.

[0022] Figure 6 Schematic diagram of a mathematical model of a finger surface structure according to an exemplary embodiment of the present invention.

[0023] Figure 7 It is a schematic diagram showing the principle of unfolding the finger surface structure according to an exemplary embodiment of the present utility model.

[0024] Figure 8 Schematic diagram of electrode installation according to an exemplary embodiment of the present invention.

[0025] Figure 9 Schematic diagram of the first sub-electrode structure according to an exemplary embodiment of the present invention.

[0026] Figure 10 Schematic diagram of the second sub-electrode structure according to an exemplary embodiment of the present invention. DETAILED DESCRIPTION

[0027] The present invention will be further described in detail below with reference to the following examples and specific implementation methods. However, this should not be construed as limiting the scope of the present invention to the following examples. All technologies implemented based on the present invention fall within the scope of the present invention.

[0028] In the description of the present invention, it should be understood that the terms "longitudinal", "transverse", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0029] like Figure 1 As shown, the present invention provides a multimodal fusion tactile fingertip module, including a proximity sensing layer 101, a temperature site 102, a touch / slip sensing layer 103, a control element 104, an electrode layer 105, a force sensing layer 106 and a support layer 107.

[0030] In this embodiment, the support layer 107 is a simulated finger structure, and the proximity sensing layer 101, temperature site 102, touch / slip sensing layer 103, control element 104, electrode layer 105, and force sensing layer 106 are all installed on the outer surface of the support layer 107.

[0031] In this embodiment, the proximity sensing layer 101 is the first layer, and the capacitive coupling effect is used to judge the distance based on different coupling capacitance values. Setting the proximity sensing layer 101 as the first layer can more directly sense the capacitance of approaching objects, obtain the distance information of the approaching objects, and transmit it to the electrode layer 105. When applied to a dexterous hand, it can assist in judging the distance between the grasped object and the grasped object, help determine the object's position, and grasp it accurately.

[0032] The touch / slip sensing layer 103 is the second layer (connected to the lower surface of the proximity sensing layer 101 ) and is used to detect touch / slip signals according to the change in array resistance and transmit them to the electrode layer 105 .

[0033] The electrode layer 105 is the third layer (located on the lower surface of the touch / slip sensing layer 103), which is used to arrange the electrode array and play a role in transmitting signals (distance signals, touch / slip signals, temperature signals, force signals, etc.);

[0034] The force sensing layer 106 is the fourth layer (located on the lower surface of the electrode layer 105), and the force sensing layer 106 is located on the outer surface of the support layer 107; the force sensing layer 106 includes multiple force sensing points, which are used to decouple the magnitude and direction of the force according to the resistance difference of different sensors to obtain a force signal.

[0035] The temperature site 102 penetrates the proximity sensing layer 101 and the touch / slip sensing layer 103 and contacts the upper surface of the electrode layer 105 .

[0036] The control element 104 penetrates the force sensing layer 106 and contacts the lower surface of the electrode layer 105. The control element 104 includes an MCU and an operational amplifier circuit for data acquisition and processing. Therefore, there are multiple groups of control elements 104, each responsible for a corresponding function.

[0037] In this embodiment, since the force sensing layer 106 itself is relatively thin, and the control elements 104 with different functions have different thicknesses and are greater than the thickness of 106, a corresponding second hole is provided in the force sensing layer 106 to accommodate the control element 104. At the same time, the support layer 107 is also provided with a corresponding hole or inward depression corresponding to the control element 104 to allow the control element 104 to be embedded.

[0038] like Figure 2 As shown, the proximity sensing layer 101 is an electrode coated on the upper surface of the touch / slip sensing layer 103, and the material can be silver paste. The proximity sensing layer 101 is connected to the proximity interface on the electrode layer 105 (not shown). Figure 2 shown in ).

[0039] For example, a regular area on the upper surface of the touch / slip sensing layer 103 is coated with silver paste as a sensing electrode, and a circle of electrically isolated shielding is also coated with silver paste around the regular area to form the proximity sensing layer 101; the sensing electrode can sense the coupling capacitance of the approaching object and sense the distance based on the size of the capacitance.

[0040] In this embodiment, the temperature site 102 is connected to the temperature interface on the electrode layer 105; the temperature site 102 can use a temperature-variable material (NTC material, the resistance decreases as the temperature increases), and the temperature signal is determined based on the change in resistance; for example, when objects with different temperatures transfer temperature to the tactile fingertip module, it will cause a change in the resistance of the temperature-variable material. The temperature of the object can be determined by calculating the change in resistance through the control element 104, which is convenient for avoiding secondary damage caused by temperature in the process of grasping the object, and can also protect the tactile fingertip module.

[0041] In this embodiment, the tactile / slip sensing layer 103 includes a first substrate 1031, a first microstructure 1032 (an inverted pyramid structure), and a second microstructure 1033 (an inverted pyramid structure). The first microstructures 1032 and the second microstructures 1033 are alternately arranged on the lower surface of the first substrate 1031, with the first microstructures 1032 being shorter than the second microstructures 1033. The surfaces of the first and second microstructures 1032, 1033 are coated with a conductive layer of carbon nanomaterial. When pressure is applied, the flexible material is compressed, causing the accumulated resistance of the conductive material to change, reflecting the change in force.

[0042] Under the action of the first microstructure 1032 and the second microstructure 1033, wide-range detection can be achieved: when contacting the outside world, the second microstructure 1033 performs detection first; when the second microstructure 1033 is saturated, the first microstructure 1032 performs detection within a larger range, thereby expanding the detection range.

[0043] The inverted pyramid structure can provide more surface area, allowing more carbon nanomaterials to contact the electrodes on the electrode layer 105, thereby improving the detection sensitivity. The good mechanical stability it provides enables the sensor to work stably for a long time. Finally, more tactile points (100 / 1cm2) are distributed in a limited area through masking to achieve high-resolution detection.

[0044] In this embodiment, the control element 104 is located on the lower surface of the electrode layer 105 and fixed on the support layer 107, and is used to process and transmit distance signals, tactile / slip signals, temperature signals, force signals, etc.

[0045] In this embodiment, the electrode layer 105 can be formed using a flexible FPC layer as a substrate, with an electrode array disposed on the upper surface of the substrate. Inspired by spider webs, the electrodes of this invention adopt a mesh structure design, with areas without electrode lines hollowed out. This adapts to the curved surface structure of the robotic hand while maintaining good stretchability.

[0046] In this embodiment, the structure of each electrode is the same, so a single electrode is used as an example. Figure 3 As shown, the electrode includes a first sub-electrode 1051 and a second sub-electrode 1052 . The first sub-electrode 1051 is located inside the second sub-electrode 1052 and cooperates with each other to form a mesh structure.

[0047] like Figure 9 As shown, the first sub-electrode 1051 includes a first fixing structure 10511, and a plurality of first branches 10512 with the same structure are evenly distributed around the periphery of the first fixing structure 10511. The first fixing structure 10511 is a polygon, preferably an octave; the number of first branches 10512 is preferably four, and the angle between each two first branches 10512 is 90 degrees.

[0048] The first branch 10512 includes a first main trunk 105121 , and multiple groups (preferably four groups) of first branch trunks 105122 are symmetrically arranged at different positions of the first main trunk 105121 , and the lengths of the first branch trunks 105122 decrease from the outside to the inside.

[0049] A first signal line 10513 is provided at the top of the first trunk 105121 of any first branch 10512 .

[0050] like Figure 10 As shown, the second sub-electrode 1052 includes a second fixing structure 10521, and a plurality of second branches 10522 of the same structure are evenly distributed inside the second fixing structure 10521. The second fixing structure 10521 is a polygon, preferably an octave; the number of second branches 10522 is preferably four, and the angle between each two second branches 10522 is 90 degrees.

[0051] The second branch 10522 includes a second main trunk 105221 , and multiple groups (preferably three groups) of second branch trunks 105222 are symmetrically arranged at different positions of the first main trunk 105221 , and the lengths of the second branch trunks 105222 decrease from the outside to the inside.

[0052] In this embodiment, the first sub-electrode 1051 is located inside the second sub-electrode 1052 , and a second branch 10512 is provided between every two first branches 10512 . Meanwhile, a second branch 105222 is provided between every two first branches 105122 , thereby forming a mesh structure.

[0053] In this embodiment, the force sensing layer 106 is located on the lower surface of the substrate of the electrode layer 105, and includes a second base 1061 and a third microstructure 1062. The third microstructure 1062 is pyramid-shaped, with the top of the pyramid in contact with the lower surface of the electrode layer 105 and the bottom of the pyramid located on the second base 1061.

[0054] In this embodiment, the structure of the support layer 107 can be designed according to actual needs and belongs to the existing technology.

[0055] like Figure 4 As shown, there is at least one, and preferably four, third microstructures 1062, located at the top, left, bottom, and right ends of the second substrate 1061. The surfaces of the third microstructures 1062 are coated with a carbon nanomaterial for conductivity. When a directional force acts on the force sensing layer 106, the third microstructures 1062 in different directions experience different forces and thus different resistances. Therefore, the difference in resistance of the third microstructures 1062 in different directions can be used to decouple the magnitude and direction of the force.

[0056] Furthermore, a second hole 1063 is set at other positions of the second substrate 1061 (without the third microstructure 1062) (for allowing the control element 104 to pass through to prevent the element from lifting the force sensitive layer and affecting sensing). The shape of the second hole 1063 includes rectangle, square, etc.

[0057] The utility model imitates the structure of human skin and distributes sensory information such as proximity, temperature, touch / slip and three-dimensional force in bionic "skin" layers of different depths, realizing multimodal integration of sensory functions.

[0058] Based on the above multimodal fusion tactile fingertip module, such as Figure 5 As shown, the present invention also provides a method for preparing a multimodal fusion tactile fingertip module, which specifically includes the following steps:

[0059] S1: First, prepare the electrode layer 105.

[0060] S1-1: In this embodiment, Figure 6 As shown in FIG, the finger surface structure (support layer 107) of the dexterous hand is relatively complex. If electrodes and electronic components are directly processed and welded on the finger surface structure, the process is complicated and tedious. However, the spatial structure of the finger surface structure is approximately ellipsoidal. The circumscribed cylindrical surface of the existing spherical surface can be used to unfold the flexible FPC bonded to the finger surface structure into an umbrella-shaped (bionic lotus-shaped) structure, as shown in FIG. Figure 7 shown.

[0061] In this embodiment, the approximate structure of the finger is as follows Figure 6 As shown, the expanded shape is actually only Figure 7Half of c is 1, 2, 3, 4; 1, 2, 3, 4 is to divide the finger surface into four equal parts, the part numbered 1 is the position of the finger close to the rotation joint ( Figure 6 The rearmost area in the middle is approximately unfolded as a cylindrical surface; the part numbered 4 is the fingertip position ( Figure 6 The front area shown in the figure is unfolded as a right cone; the parts numbered 2 and 3 are the finger pads ( Figure 6 The middle area shown in FIG2 is divided into two equal parts and unfolded as a truncated right cone; s1', s2', and s3' are the vertices of each cone.

[0062] The expansion method is to use the horizontal plane to spherical the ball ( Figure 7 a, i.e. the finger surface structure) is divided into several equal parts ( Figure 7 b is divided into seven equal parts). Then, except for the part numbered 1 which is approximately unfolded as a cylindrical surface, the rest are approximately unfolded as their inscribed right circular cone surfaces. The four parts numbered 2, 3, 5, and 6 are unfolded as truncated right circular cone surfaces, and the parts numbered 4 and 7 are unfolded as right circular cone surfaces. The vertices of each cone surface are points s1', s2', s3', etc. The resulting unfolded diagram is shown in the figure below. Figure 7 As shown in c.

[0063] S1-2: If Figure 8 As shown, electrodes are evenly spaced and mounted on different parts of the umbrella structure after the finger surface structure is unfolded, with a density of 100 electrodes / cm 2 The electrodes at each location are connected via wires, ultimately obtaining an electrode layer 105 .

[0064] In this embodiment, multiple through-hole structures are provided in places where there are no wires and electrodes to increase flexibility and conformability, so that the device can adapt to curved structures while having good tensile properties.

[0065] S2: Prepare the tactile / slip sensing layer 103 and the force sensing layer 106, then place the tactile / slip sensing layer 103 on the upper surface of the electrode layer 105 (possibly aligned and bonded with conductive adhesive), and place the force sensing layer 106 on the lower surface of the electrode layer 106 (possibly aligned and bonded with conductive adhesive).

[0066] S2-1: Preparation of the tactile / slip sensing layer 103:

[0067] A first microstructure 1032 (an inverted pyramid) and a second microstructure 1033 (an inverted pyramid) are etched on a first substrate 1031, where the height of the first microstructure 1032 is smaller than that of the second microstructure 1033. A layer of carbon nanomaterial is then deposited after masking the surface of the first microstructure 1032 so that the height of the second microstructure 1033 is smaller. The carbon nanomaterial is then transferred using a PDMS301 (polydimethylsiloxane) to curing agent ratio of 10:1 to obtain a tactile / slip sensing layer 103.

[0068] A first hole for the temperature site 102 to pass through is reserved on the touch / slip sensing layer 103 .

[0069] S2-2: Preparation of force sensing layer 106:

[0070] A third microstructure 1062 is etched on the second substrate 1061 . The third microstructure 1062 is pyramid-shaped. The surface of the third microstructure 1062 is coated with carbon nanomaterial for conductivity.

[0071] There is at least one, and preferably four, third microstructures 1062, distributed at the top, left, bottom, and right ends of the second substrate 1061. When a directional force acts on the force sensing layer 106, the third microstructures 1062 in different directions experience different forces and thus have different resistances. Therefore, the difference in resistance of the third microstructures 1062 in different directions can be used to decouple the magnitude and direction of the force.

[0072] Furthermore, a second hole 1063 is provided at other positions of the second substrate 1061 (where there is no third microstructure 1062). The shape of the second hole 1063 includes a rectangle, a square, etc., and is used to control the penetration of the component 104.

[0073] S2 - 3 : The control element 104 is welded to the lower surface of the electrode layer 105 , and the temperature point 102 is welded to the upper surface of the electrode layer 105 .

[0074] S2-4: The tactile / slip sensing layer 103 is then placed on the upper surface of the electrode layer 105 after the S2-3 treatment (the temperature site 102 passes through the first hole), and the force sensing layer 106 is placed on the lower surface of the electrode layer 105 after the S2-3 treatment (the control element 104 passes through the second hole), thereby forming an integral component; the integral component is then mounted on the outer surface of the support layer 107 to complete the mounting.

[0075] S3: a proximity sensing layer 101 is provided on the upper surface of the mounted tactile / slip sensing layer 103 .

[0076] In this embodiment, silver paste is coated on the upper surface of the touch / slip sensing layer 103 to form the proximity sensing layer 101 , and the proximity sensing layer 101 is connected to the proximity interface of the electrode layer 105 .

[0077] S4: The injection molding process is used for sealing, thereby encapsulating the proximity sensing layer 101, the temperature site 102, the touch / slip sensing layer 103, the controller 104, the electrode layer 105, the force sensing layer 106 and the support layer 107 into one, thereby obtaining an integrated multi-modal fusion tactile fingertip module.

[0078] Those skilled in the art will appreciate that the above-mentioned embodiments are specific examples for implementing the present invention, and in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present invention.

Claims

1. A multimodal fusion tactile fingertip module, comprising a support layer (107), characterized in that: The outer surface of the support layer (107) is provided with a proximity sensing layer (101) for sensing distance, a temperature site (102) for sensing temperature, a tactile / slip sensing layer (103) for sensing touch, a control element (104), an electrode layer (105) for transmitting signals, and a force sensing layer (106) for sensing force.

2. The multimodal fusion tactile fingertip module according to claim 1, characterized in that: The support layer (107) is a finger structure.

3. The multimodal fusion tactile fingertip module according to claim 1, characterized in that: The proximity sensing layer (101) is arranged on the upper surface of the tactile / slip sensing layer (103), and the proximity sensing layer (101) is connected to the electrode layer (105); the tactile / slip sensing layer (103) is arranged on the upper surface of the electrode layer (105).

4. The multimodal fusion tactile fingertip module according to claim 3, characterized in that: The tactile / slip sensing layer (103) comprises a first substrate (1031), a first microstructure (1032) and a second microstructure (1033); wherein the first microstructure (1032) and the second microstructure (1033) are alternately arranged on the lower surface of the first substrate (1031).

5. The multimodal fusion tactile fingertip module according to claim 4, characterized in that: The height of the first microstructure (1032) is smaller than the height of the second microstructure (1033).

6. The multimodal fusion tactile fingertip module according to claim 1, characterized in that: The temperature site (102) penetrates the proximity sensing layer (101) and the touch / slip sensing layer (103) and contacts the upper surface of the electrode layer (105); the control element (104) penetrates the force sensing layer (106) and contacts the lower surface of the electrode layer (105).

7. The multimodal fusion tactile fingertip module according to claim 1, characterized in that: The electrode layer (105) includes an FPC as a substrate and electrodes; the electrodes are in a mesh structure.

8. The multimodal fusion tactile fingertip module according to claim 1, characterized in that: The force sensing layer (106) includes a second substrate (1061) and a third microstructure (1062), and the third microstructure (1062) is distributed at the upper end, left end, lower end, and right end of the second substrate (1061).

Images