Multi-curvature high-resolution bionic miniature visual tactile sensor for fingertips

Abstract

The invention discloses a multi-curvature high-resolution bionic miniature visual tactile sensor for fingertips, which relates to the technical field of sensors and comprises a fingertip structure, a transparent support structure, a flexible silica gel sensing layer and an image acquisition module. A cavity is formed in the fingertip structure; the transparent supporting structure is clamped in the cavity; the flexible silica gel sensing layer is adhered to the transparent supporting structure, and black circular array mark points are attached to the surface of the flexible silica gel sensing layer; the image acquisition module is arranged on the fingertip structure, the acquisition end of the image acquisition module is arranged above the transparent supporting structure, and the view field area of the acquisition end covers the whole flexible silica gel sensing layer area; compared with other fingertip tactile sensors, the visual tactile sensor is a human fingertip simulated visual tactile sensor with the highest spatial resolution, the smallest overall size and the lowest manufacturing cost.

Description

Technical Field

[0001] The invention relates to the technical field of sensors, and in particular to a multi-curvature high-resolution bionic micro visual-tactile sensor for fingertips. Background Art

[0002] In recent years, with the development of science and technology, intelligent robots have more application scenarios and development directions. Intelligent robots rely on a variety of sensors to sense and analyze the environment, thus having senses similar to humans. Tactile sensors are as important as human skin organs, because they not only read physical characteristics such as position, temperature and shape, but also can perform various operations by sensing hardness and pressure. Tactile sensors are an important medium for robots to perceive the external environment, and they are extremely important for robots to correctly operate target objects. Under the premise that the robot moves flexibly and freely, the tactile sensor is required to accurately perceive the external environment in order to achieve various precise operations on the target object. When applied to humanoid five-finger dexterous hands, higher requirements are placed on tactile sensors. For example, performance requirements such as high integration of sensors, high spatial resolution of sensors, high repeatability of sensors, and low hysteresis of sensors.

[0003] The sensor solutions currently used in robot touch are divided into traditional solutions and new solutions. Among them, the sensors of the traditional solution mainly include piezoresistive tactile sensors, capacitive tactile sensors, etc. Visual tactile sensors are a type of tactile sensor and belong to the new solution.

[0004] The representative product of the traditional tactile sensor is the array tactile sensor product of Tekscan Company in the United States. This type of product consists of two very thin polyester films, one of which has several rows of strip conductors on its inner surface and the other has several columns of strip conductors on its inner surface. The width of the conductor itself and the distance between the conductors can be designed according to different measurement needs. The surface of the conductor is coated with a special pressure-sensitive semiconductor material coating. When the two films are combined into one, the intersections of these transverse and longitudinal conductors form an array of pressure sensing points in a grid shape. When an external force is applied to the sensing point, the resistance of the semiconductor will change proportionally with the external force. When the pressure is zero, the resistance is the largest, and the greater the pressure, the smaller the resistance. Tactile perception is obtained by converting physical signals such as pressure and touch into measurable electrical signals.

[0005] In the field of visual tactile sensing research, the more famous sensors include GelSightmini from MIT, TacTip series visual tactile sensors from the University of Bristol, Soft-bubble sensors from Toyota Research Institute, and DIGIT visual tactile sensors from Meta. The typical structure of the current mainstream visual tactile sensor consists of a contact module, a lighting module, a camera module, and an information processing module. When the light emitted from the light source is reflected by the reflective layer or the marking point layer and transmitted to the camera, the camera captures the brightness difference to form a tactile image. The marking point used as a reference can relatively reflect and measure the deformation state of the contact body. However, different components have different design emphases to meet different tactile sensing mode requirements.

[0006] There are two difficulties in the research of fingertip tactile sensors: (1) Spatial resolution: The spatial resolution of the tactile sensor array refers to the minimum distance between different points in the surrounding area that the sensor can distinguish and sense. High spatial resolution means that the sensor array can detect touch or pressure changes more accurately, thereby providing more delicate tactile feedback and obtaining the specific spatial position of tiny objects at the fingertips when they are grasped. This is crucial for the performance of dexterous hands in complex operations. (2) Size: The size definition of a tactile sensor usually refers to the physical size of the sensor itself, including length, width, and thickness. The size of the sensor directly affects its feasibility and performance when integrated into the dexterous fingertips.

[0007] Since it is difficult for the signal processing circuit that converts multiple physical signals into measurable electrical signals to be of a size that can be accommodated by the fingertips, the tactile sensors of traditional solutions usually find it difficult to achieve a high spatial resolution. For example, a dexterous fingertip tactile sensor based on a silicon micro pressure sensitive chip (CN116973015A) has only one tactile sensing unit on the entire fingertip instead of an array of sensing units. There is also a dexterous fingertip with multi-point tactile perception (CN115139322A), which is made based on the principle of resistance and has 9 sensing units on the entire fingertip. Due to the limitations of the preparation process and the sensing mechanism, its spatial resolution can only stay at about 5mm. When faced with more delicate operation tasks, such as grasping needles and threads and identifying textures, the spatial resolution of fingertip tactile sensors of this type of traditional solution is difficult to meet the task requirements.

[0008] From the perception mechanism of the visual touch sensor, it can be known that in order to obtain a high-resolution and clear tactile image, there is a certain distance between the camera module and the flexible contact layer so that the camera module can focus, and an LED light source is required to illuminate the contact layer. Existing visual touch sensors, such as GelSight Mini, Digit, etc., use cameras and lighting solutions that make their overall size larger. The overall size of the mainstream commercial visual touch sensor GelSight Mini reaches 30x 30x 75 cm, which is suitable for two-finger grippers or end effectors with larger overall sizes, but the larger size makes it difficult to apply to humanoid five-finger dexterous hands with smaller overall sizes. For example, a tactile sensor based on structural color (CN118655997A) has a high spatial resolution at the pixel level based on the perception mechanism of the visual touch sensor. However, due to the perception mechanism of the visual touch sensor, the flexible contact module needs to be at a certain distance from the embedded camera module to ensure the focusing effect, which makes the thickness of the visual touch sensor very thick and the signal processing module of the camera module too large. Compared with smaller humanoid dexterous hands, its overall structural size is too large, just like GelSight Mini, Digit and other visual-tactile sensors, and its appearance structure is difficult to integrate into dexterous fingertips.

[0009] Therefore, the spatial resolution of fingertip tactile sensors based on traditional principles is not high enough, and the appearance, structure and size of existing tactile sensors are difficult to apply to dexterous fingertips. Summary of the invention

[0010] Based on the technical problems existing in the background technology, the present invention proposes a multi-curvature high-resolution bionic miniature visual-tactile sensor for fingertips. The industrial endoscope module has the characteristics of small size, large field of view and short focal length, making it very suitable for application scenarios in narrow spaces.

[0011] The present invention proposes a multi-curvature high-resolution bionic micro visual-tactile sensor for fingertips, comprising a fingertip structure, a transparent support structure, a flexible silicone sensing layer and an image acquisition module;

[0012] A cavity is provided inside the fingertip structure;

[0013] The transparent support structure is clamped in the cavity;

[0014] The flexible silicone sensing layer is bonded to the transparent support structure, and has black circular array marking points on the surface;

[0015] The image acquisition module is arranged on the fingertip structure, the acquisition end of the image acquisition module is arranged above the transparent support structure, and the field of view area of ​​the acquisition end covers the entire flexible silicone sensing layer area.

[0016] Furthermore, the image acquisition module adopts an industrial endoscope module, the industrial endoscope module is arranged on the fingertip structure, and the lens of the industrial endoscope module serves as the acquisition end of the image acquisition module.

[0017] Furthermore, the tactile sensor also includes a lighting module, which includes an LED light source array, a microcontroller and a power supply, the microcontroller is electrically connected to the power supply and the LED light source array respectively, and the LED light source array is arranged around the lens of the industrial endoscope module to provide lighting for the flexible silicone sensing layer;

[0018] The lamp beads of the LED light source array are neutral light LEDs. The light source is controlled by the adjustment knob of the microcontroller to dynamically adjust the brightness of the LED light source array.

[0019] Furthermore, the LED light source array includes 6 light sources, the 6 light sources are evenly arranged, the control frequency of the LED light source array is greater than or equal to 1 kHz, and the power is 0.1 W±10%.

[0020] Furthermore, the transparent support structure is embedded in the fingertip structure, supported by a card slot provided in the fingertip structure, and fixed by UV light-curing glue.

[0021] Further, the flexible silicone sensing layer includes transparent silicone rubber, black screen-printed marking dots, and opaque silver-grey silicone paint;

[0022] The transparent silicone rubber is bonded and fixed on the transparent supporting structure;

[0023] Black screen-printed marking dots are set between the transparent silicone rubber and the opaque silver-grey silicone paint, so that black circular array marking dots are attached to the surface of the transparent silicone rubber;

[0024] Opaque silver-grey silicone paint is sprayed on the transparent silicone rubber with black circular array marking dots to act as a reflective layer to prevent light from being diffusely reflected inside the sensor and to act as a protective film during the contact between the sensor and the target.

[0025] Furthermore, the diameter of each black circular marking point in the black circular array marking point is 0.5mm-1.5mm; the maximum thickness of the transparent silicone rubber on the surface of which the black circular array marking points are attached is less than or equal to 5mm; the contact life of the opaque film formed by the opaque silver-gray silicone paint is greater than or equal to 10,000 times, and the contact life of the coating formed by the opaque silver-gray silicone paint corresponding to the black circular array marking points is greater than or equal to 10,000 times.

[0026] Furthermore, the flexible silicone sensing layer is in a non-planar structure, and the image acquisition module disposed above the flexible silicone sensing layer is illuminated obliquely. The raw data of the flexible silicone sensing layer collected by the image acquisition module is a deformed image, and the deformed image has uneven illumination due to the uneven illumination intensity. The de-distortion process of the deformed image is as follows:

[0027] Select four control points in the target image and the original image respectively, and use the perspective transformation method to perform image correction;

[0028] The perspective transformation matrix H is calculated using the control points of the original image. The coordinates of each pixel in the original image after perspective transformation are calculated based on the perspective transformation matrix H. Bilinear interpolation is used to calculate the pixel values ​​in the transformed coordinates. All transformed pixel points are filled into the new image to generate a dedistorted corrected image.

[0029] Furthermore, the light intensity value of the entire dedistorted image is equalized by using an adaptive histogram equalization method to obtain a tactile image with balanced light intensity, specifically:

[0030] The dedistorted image is divided into multiple small areas, a histogram is calculated for each small area, a maximum value constraint of the histogram is set, and each restricted histogram is normalized and summed to obtain a cumulative histogram;

[0031] The original pixel value in each small area is mapped to the enhanced pixel value using the cumulative histogram, and then all the small area images are merged into a complete image. Bilinear interpolation is used to make a smooth transition between adjacent small areas to obtain the final tactile image with no distortion and balanced light intensity.

[0032] The advantage of the multi-curvature high-resolution bionic micro visual-tactile sensor for fingertips provided by the present invention is that a micro visual-tactile sensor suitable for imitating humanoid dexterous fingertips is made by using a micro light source endoscope, 3D printed transparent resin, a high-transparency bionic structure silicone layer, etc. The visual-tactile sensor of this embodiment has great advantages in spatial resolution, overall size, repeatability, and production cost compared to other tactile sensors for dexterous fingertips. It plays an important role in dexterous hand grasping, grasping state recognition, etc. Compared with other fingertip tactile sensors, this fingertip visual-tactile sensor should be the humanoid fingertip visual-tactile sensor with the highest spatial resolution, the smallest overall volume, and the lowest production cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a structural schematic diagram of the present invention;

[0034] Figure 2 It is a front view of the fingertip visual tactile sensor;

[0035] Figure 3 It is the left view of the fingertip visual tactile sensor;

[0036] Figure 4 for Figure 3 Cross-sectional view of the middle BB;

[0037] Figure 5 The production and workflow diagram of the fingertip visual tactile sensor;

[0038] Among them, 1-fingertip structure, 2-transparent support structure, 3-flexible silicone sensing layer, 4-image acquisition module, 31-transparent silicone rubber, 32-black screen-printed marking point, 33-opaque silver-grey silicone paint, 41-industrial endoscope module. DETAILED DESCRIPTION

[0039] Below, the technical solution of the present invention is described in detail through specific embodiments. Many specific details are set forth in the following description to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific implementation disclosed below.

[0040] The present embodiment proposes a miniature visual-tactile sensor based on a miniature industrial endoscope module, a vertical neutral light source solution, and an anthropomorphic multi-curvature elastomer. The industrial endoscope module has the characteristics of small size, large field of view, and short focal length, making it very suitable for application scenarios in which space is limited.

[0041] When an object comes into physical contact with the tactile skin, the distribution of light will change accordingly. Subsequently, the camera captures these changing images and processes them through a computer to obtain tactile information. Such a system realizes visual-based tactile sensing. The method of converting physical contact into optical image information in this embodiment is: a marker point displacement method plus a reflective film method. First, since a black marker point array and a reflective pigment that reflects light from a light source are printed and sprayed on the surface of an optically transparent elastomer. When an object comes into contact with the surface of the elastomer, the elastomer will deform, and the intensity of the reflected light is related to the gradient of the surface of the elastomer after deformation. Therefore, the image captured by the acquisition end of the image acquisition module 4 can be used to estimate the surface gradient, and then the surface height information is obtained by gradient reconstruction, thereby performing three-dimensional reconstruction of the contact object. When an external object acts on the surface of the elastomer, the elastomer deforms, causing the marker point to move. By capturing the two-dimensional displacement of the marker point through the acquisition end of the image acquisition module 4, the system can obtain a variety of tactile information, including shear force and sliding.

[0042] The visual tactile sensor of this embodiment directly integrates the sensor into the fingertip structure, and the image acquisition module 4 adopts the industrial endoscope module 41. The lighting module is a circle of neutral white light LED light ring, that is, an LED light source array, surrounding the lens of the industrial endoscope module 41. The flexible silicone sensing layer 3 is designed to imitate the shape of a human fingertip. By marking array marking points on the surface of the flexible silicone sensing layer 3 and coating a reflective film, the contact image of the contact surface is accurately captured, the change of the contact image is analyzed, and the tactile information such as force, shape, and slippage is accurately estimated.

[0043] The appearance of the visual tactile sensor provided in this embodiment is highly similar to the human fingertip. Figures 1 to 4 As shown in the figure, the design of bionic fingertips can make the grasping method of the dexterous hand very similar to that of humans, and can make the five-fingered dexterous hand more human-like and complete more dexterous operation tasks. The rear end of the multi-curvature high-resolution bionic fingertip micro visual tactile sensor is equipped with two connectors fixed by cylindrical pins, which can be easily installed, disassembled and maintained.

[0044] like Figures 1 to 5 As shown, the present invention proposes a multi-curvature high-resolution bionic micro visual-tactile sensor for fingertips, comprising a fingertip structure 1, a transparent supporting structure 2, a flexible silicone sensing layer 3 and an image acquisition module 4; a cavity is arranged inside the fingertip structure 1; the transparent supporting structure 2 is clamped in the cavity; the flexible silicone sensing layer 3 is bonded to the transparent supporting structure 2, and black circular array marking points are attached to the surface; the image acquisition module 4 is arranged on the fingertip structure 1, and the acquisition end of the image acquisition module 4 is arranged above the transparent supporting structure 2, and the field of view area of ​​the acquisition end covers the entire flexible silicone sensing layer 3 area.

[0045] The fingertip structure 1 is made of aluminum alloy, and its structural dimensions can be adjusted according to the actual size requirements of the dexterous fingertips.

[0046] The transparent support structure 2 is embedded in the fingertip structure 1, supported by the card slot in the fingertip structure 1, and fixed by UV light-curing glue. The material is made of highly transparent acrylic and is used to support the outermost flexible silicone sensing layer 3.

[0047] The flexible silicone sensing layer 3 includes a transparent silicone rubber 31, a black screen-printed marking point 32 and an opaque silver-gray silicone paint 33; the transparent silicone rubber 31 is bonded and fixed on the transparent support structure 2; the black screen-printed marking point 32 is arranged between the transparent silicone rubber 31 and the opaque silver-gray silicone paint 33, so that the black circular array marking point is attached and arranged on the surface of the transparent silicone rubber 31; the opaque silver-gray silicone paint 33 is sprayed on the surface of the black screen-printed marking point 32 away from the transparent silicone rubber 31 to serve as a reflective layer. Among them, the diameter of each black circular marking point in the black circular array marking point is 0.5mm-1.5mm, preferably 1mm; the maximum thickness of the transparent silicone rubber 31 with the black circular array marking point attached to the surface is less than or equal to 5mm; the contact life of the opaque film formed by the opaque silver-gray silicone paint 33 is greater than or equal to 10,000 times, and the contact life of the coating formed by the opaque silver-gray silicone paint 33 corresponding to the black circular array marking point is greater than or equal to 10,000 times.

[0048] The transparent silicone rubber 31, the black screen-printed marking points 32 and the opaque silver-grey silicone paint 33 in the flexible silicone sensing layer 3 can be changed in terms of material and color, but in essence it is still a solution of marking points plus a reflective layer.

[0049] The image acquisition module 4 is fixed to the bottom and the central circular hole of the fingertip structure 1. The signal processing circuit of the image acquisition module 4 is placed in the palm of the dexterous hand of the fingertip structure 1, and the two are connected by a soft FPCB. The image acquisition module 4 adopts an industrial endoscope module 41. In order to save the space occupied by the industrial endoscope module 41 inside the fingertip structure 1, the CMOS sensor model of the industrial endoscope module 41 is OV9734, the pixel size is 1.4um×1.4um, the image output format is MJPG, the focusable focal length is 5-10mm, the maximum pixel is 1280×720, and it supports the UVC driver-free protocol. The image information is transmitted through a USB cable. The pixel of the micro embedded camera module is greater than or equal to 1080P, the frame rate is YUV2 30fps, and a wide field of view of 120 degrees is adopted.

[0050] In addition, a lighting module is provided, which includes an LED light source array, a microcontroller and a power supply. The microcontroller is electrically connected to the power supply and the LED light source array respectively. The LED light source array is arranged around the lens of the industrial endoscope module 41 and fixed by HY 705 sealant adhesive to provide lighting for the flexible silicone sensing layer 3. The lamp beads of the LED light source array are neutral light LEDs. The light source is controlled by the adjustment knob of the microcontroller, and the brightness of the LED light source array is dynamically adjusted. The LED light source array includes 6 light sources, which are evenly arranged. The spacing between adjacent light sources is 360 / 6 degrees, that is, 60°. The control frequency of the LED light source array is greater than or equal to 1kHz, the power is 0.1W±10%, and the size of the light strip FPCB board is 1mm in width and 0.5mm in thickness.

[0051] The tactile perception frequency of the fingertip visual tactile sensor is greater than or equal to 30Hz, the fingertip visual tactile sensor covers the entire fingertip for the surface perception range of the grasped target, the mechanical measurement resolution of the surface marker points of the fingertip visual tactile sensor is greater than or equal to 1mm, and the minimum position spatial resolution of the grasped object is 0.6mm. The perceived tactile information is generated by the industrial endoscope module 41 located at the dexterous fingertips tracking the black circular array marker points on the flexible silicone sensing layer 3. A finite element model (FEM) is developed to describe the relationship between the applied force and the marker displacement, allowing detailed tactile information to be derived simultaneously.

[0052] The perception frequency of the fingertip visual tactile sensor is mainly determined by the image acquisition module, and tactile perception is achieved by observing the deformation of the silicone during contact with the target. The perception range of the fingertip visual tactile sensor is determined by the mechanical properties of the flexible silicone sensing layer 3. The visual tactile sensor on the dexterous fingertip achieves perception by contacting and deforming the target object. The mechanical measurement resolution of the sensor's surface marking is better than 1mm, which is jointly determined by the distance between the image acquisition module 4 and the marking layer of the flexible silicone sensing layer 3. Its resolution can be calculated by the size of each pixel corresponding to the physical size in real space. The opaque silver-gray silicone paint 33 is specifically sprayed on the transparent silicone rubber 31 with black circular array marking points to prevent light from diffusely reflecting inside the sensor, and at the same time act as a protective film during the contact between the sensor and the target.

[0053] Since the flexible silicone sensing layer 3 is not a planar structure, and the industrial endoscope module 41 is placed in an oblique illumination mode, and the lighting module is also obliquely illuminated, the original tactile image is a distorted image, and the illumination intensity is uneven, which will cause the tactile image closer to the LED light strip to be brighter, and the tactile image farther away to be darker. Therefore, after obtaining the initial tactile image, it is necessary to perform orthogonal distortion and light balance on the tactile image; the dedistortion process of the deformed image is as follows:

[0054] (a1) Select four control points in the target image and the deformed image respectively, and use the perspective transformation method to perform image correction;

[0055] Select between the source image and the destination image:

[0056] Original image: P 1 (x 1 ,y 1 ),P 2 (x 2 ,y 2 ),P 3 (x 3 ,y 3 ),P 4 (x 4 ,y 4 );

[0057] Target image: Q 1 (0,0),Q 2 (w,0),Q 3 (w,h),Q 4 (0,h);

[0058] Among them, (x 1 ,y 1 )、(x 2 ,y 2 )、(x 3 ,y 3 )、(x 4 ,y 4 ) are four pixels in the original image, w and h are the width and height of the image selected in the target image respectively.

[0059] (a2) Use the control points to calculate the perspective transformation matrix H, calculate the coordinates of each pixel in the distorted image after perspective transformation based on the perspective transformation matrix H, use bilinear interpolation to calculate the pixel value in the transformed coordinates, fill all the transformed pixel points into the new image, and generate the final corrected image.

[0060]

[0061] Among them, the first row h 11 、h 12 、h 13 , affects the mapping of the horizontal coordinates, controls the scaling, translation and rotation of the horizontal coordinates of the output points; the second line h 21 、h 22 、h 23 , affects the mapping of the ordinate, controls the scaling, translation and rotation of the ordinate of the output point; the third line h 31 、h 32 Used to introduce perspective effect, 1 is mainly used to ensure the homogeneity of the matrix.

[0062] By knowing four point pairs, we can establish a linear equation system to solve the eight unknowns. Solve the linear equation system to determine the elements of the perspective transformation matrix H. After calculating H, we calculate the transformed coordinates. For each pixel (x, y) in the original image, we use the perspective transformation formula to calculate the transformed coordinates (x′, y′):

[0063]

[0064] Bilinear interpolation is used to calculate the pixel values ​​in the transformed coordinates. Finally, all the transformed pixel values ​​are filled into the new image to generate the final corrected image.

[0065] (a3) using an adaptive histogram equalization method to equalize the light intensity value of the entire dedistorted image to obtain a dedistorted tactile image;

[0066] Divide the dedistorted image into multiple small areas, for example, into 16×16 areas, and calculate the histogram for each small area:

[0067]

[0068] Where H(k) is the histogram of gray level k, f(i) is the gray value of the i-th pixel, and δ is the Kronecker function.

[0069] Next, set the histogram maximum constraint H clamped (k) Prevent over-enhancement:

[0070]

[0071] Among them, C max is the contrast limiting factor, which controls the maximum range of the pixel histogram distribution in each local area and selects a reasonable C max This increases contrast while preventing noise from increasing.

[0072] Each limited histogram is then normalized and summed to ensure that the sum is N, where N is the number of pixels in the original area, j represents the index of the gray level, usually an integer value from 0 to L, and L is the total gray level.

[0073]

[0074] Use the cumulative histogram to map the pixel values ​​of the original image to the enhanced pixel values:

[0075] f′(i)=round(C(f(i))×(L-1))

[0076] Finally, all small area images are merged into a complete image. For adjacent small areas, bilinear interpolation is used to perform a smooth transition to obtain the final tactile image without distortion and with balanced light intensity.

[0077] In this embodiment, if Figure 5 As shown, the production and working process of the fingertip visual tactile sensor is as follows:

[0078] (b1) calibrating the internal parameters of the industrial endoscope module 41, including the posture relationship between the dexterous fingertips and the industrial endoscope module 41, and the internal parameters of the industrial endoscope module 41 include camera size, camera viewing angle, baseline distance, distortion condition, shooting frame rate and imaging size;

[0079] (b2) performing 3D printing based on a photosensitive resin material to obtain a fingertip structure 1 of the sensor and a contact layer silicone mold, and finely polishing the contact layer silicone mold and the overall structural frame of the sensor;

[0080] (b3) making a transparent silicone rubber 31 (i.e., elastic layer) based on the contact layer silicone mold, printing black circular array marking points on the outer surface of the transparent silicone rubber 31 by screen printing technology, and finally spraying a reflective layer at the position of the outermost black circular array marking points by silicone spraying technology;

[0081] (b4) making a transparent support structure 2, i.e. a transparent acrylic support layer;

[0082] (b5) Using UV high-transmittance glue to bond the acrylic support layer to the fingertip structure 1, using silicone-specific adhesive to bond the acrylic support layer to the transparent silicone rubber 31, the industrial endoscope module 41 is inserted through the small hole at the rear of the fingertip structure 1 and fixed with silicone rubber;

[0083] (b6) Using an illumination equalization algorithm to equalize the illumination of the original tactile image where the brightness is uneven, and using an image dedistortion algorithm to dedistort the original tactile image.

[0084] Therefore, in this embodiment, a miniature visual tactile sensor suitable for humanoid dexterous fingertips is manufactured using a micro light source endoscope, 3D printed transparent resin, and a high-transparency bionic structure silicone layer. Compared with other tactile sensors for dexterous fingertips, the visual tactile sensor of this embodiment has great advantages in spatial resolution, overall size, repeatability, and production cost. It plays an important role in dexterous hand grasping, grasping state recognition, etc. Compared with other fingertip tactile sensors, this fingertip visual tactile sensor should be the humanoid fingertip visual tactile sensor with the highest spatial resolution, the smallest overall volume, and the lowest production cost.

[0085] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A multi-curvature high-resolution bionic micro visual-tactile sensor for fingertips, comprising a fingertip structure (1), a transparent support structure (2), a flexible silicone sensing layer (3) and an image acquisition module (4); A cavity is provided inside the fingertip structure (1); The transparent support structure (2) is clamped in the cavity; The flexible silicone sensing layer (3) is bonded to the transparent support structure (2), and has black circular array marking points on its surface; The image acquisition module (4) is arranged on the fingertip structure (1), the acquisition end of the image acquisition module (4) is arranged above the transparent support structure (2), and the field of view area of ​​the acquisition end covers the entire flexible silicone sensing layer (3) area.

2. The multi-curvature high-resolution bionic micro visual-tactile sensor for fingertips according to claim 1, characterized in that: The image acquisition module (4) adopts an industrial endoscope module (41), the industrial endoscope module (41) is arranged on the fingertip structure (1), and the lens of the industrial endoscope module (41) serves as the acquisition end of the image acquisition module (4).

3. The multi-curvature high-resolution bionic micro visual-tactile sensor for fingertips according to claim 1, characterized in that: The tactile sensor also includes a lighting module, which includes an LED light source array, a microcontroller and a power supply, wherein the microcontroller is electrically connected to the power supply and the LED light source array respectively, and the LED light source array is arranged around the lens of the industrial endoscope module (41) to provide lighting for the flexible silicone sensing layer (3); The lamp beads of the LED light source array are neutral light LEDs. The light source is controlled by the adjustment knob of the microcontroller to dynamically adjust the brightness of the LED light source array.

4. The multi-curvature high-resolution bionic micro visual-tactile sensor for fingertips according to claim 3, characterized in that: The LED light source array includes 6 light sources, which are evenly arranged. The control frequency of the LED light source array is greater than or equal to 1 kHz, and the power is 0.1 W±10%.

5. The multi-curvature high-resolution bionic micro visual-tactile sensor for fingertips according to claim 1, characterized in that: The transparent support structure (2) is embedded in the fingertip structure (1), supported by a card slot provided in the fingertip structure (1), and fixed by UV light-curing glue.

6. The multi-curvature high-resolution bionic micro visual-tactile sensor for fingertips according to claim 1, characterized in that: The flexible silicone sensing layer (3) comprises transparent silicone rubber (31), black screen-printed marking dots (32) and opaque silver-grey silicone paint (33); The transparent silicone rubber (31) is bonded and fixed on the transparent supporting structure (2); Black screen-printed marking dots (32) are arranged between the transparent silicone rubber (31) and the opaque silver-grey silicone paint (33) so as to attach and arrange black circular array marking dots on the surface of the transparent silicone rubber (31); Opaque silver-grey silicone paint (33) is sprayed on the transparent silicone rubber (31) with black circular array marking points, serving as a reflective layer to prevent light from being diffusely reflected inside the sensor, and also serving as a protective film during the contact between the sensor and the target.

7. The multi-curvature high-resolution bionic micro visual-tactile sensor for fingertips according to claim 6, characterized in that: The diameter of each black circular array marking point is 0.5 mm to 1.5 mm; the maximum thickness of the transparent silicone rubber (31) on the surface of which the black circular array marking points are attached is less than or equal to 5 mm; the contact life of the opaque film formed by the opaque silver-grey silicone paint (33) is greater than or equal to 10,000 times, and the contact life of the coating formed by the opaque silver-grey silicone paint (33) corresponding to the black circular array marking points is greater than or equal to 10,000 times.

8. The multi-curvature high-resolution bionic micro visual-tactile sensor for fingertips according to claim 1, characterized in that: The flexible silicone sensing layer (3) is in a non-planar structure, and the image acquisition module (4) arranged above the flexible silicone sensing layer (3) is illuminated obliquely. The original image of the flexible silicone sensing layer (3) acquired by the image acquisition module (4) is a deformed image, and due to the uneven illumination intensity, the original image has uneven illumination. The dedistortion process of the original image is as follows: Select four control points in the target image and the original image respectively, and use the perspective transformation method to perform image correction; The perspective transformation matrix H is calculated using the control points. The coordinates of each pixel in the original image after perspective transformation are calculated based on the perspective transformation matrix H. Bilinear interpolation is used to calculate the pixel values ​​in the transformed coordinates. All transformed pixel points are filled into the new image to generate a dedistorted corrected image.

9. The multi-curvature high-resolution bionic micro visual-tactile sensor for fingertips according to claim 8, characterized in that: The adaptive histogram equalization method is used to equalize the light intensity value of the entire dedistorted image to obtain a tactile image with balanced light intensity, specifically: The dedistorted image is divided into multiple small areas, the histogram is calculated for each small area, the maximum value constraint of the histogram is set, and each restricted histogram is normalized and summed to obtain a cumulative histogram; The original pixel value in each small area is mapped to the enhanced pixel value using the cumulative histogram, and then all the small area images are merged into a complete image. Bilinear interpolation is used to make a smooth transition between adjacent small areas to obtain the final tactile image with no distortion and balanced light intensity.

Images