Visual tactile sensor for realizing dynamic force high-sampling-rate measurement

Abstract

The invention relates to a visual tactile sensor for realizing dynamic force high sampling rate measurement, which comprises an image acquisition module, a support shell, an illumination module and a contact module, and is characterized in that the contact module comprises a transparent elastic layer of an inner cavity optical mark point, and further comprises a light-proof elastic layer; the light-proof elastic layer is embedded in the transparent elastic layer, the flexible metal supporting net is embedded in the light-proof elastic layer, the outer side bottom face of the transparent elastic layer is attached to and connected with the inner side bottom face of the light-proof elastic layer, the inner side bottom face is suspended in the air and faces the optical incidence end of the image acquisition module, and the outer side bottom face of the light-proof elastic layer makes contact with an object to be detected. Compared with the prior art, the device can be lightened and thinned, and the high-sampling-rate measurement of the dynamic force is realized.

Description

Technical Field

[0001] This invention relates to the field of robotics, and in particular to a visual-tactile sensor that enables high sampling rate measurement of dynamic force. Background Technology

[0002] Vision-tactile sensors have significant application value in fields such as precision robotic manipulation and human-computer interaction. Existing vision-tactile sensors mainly sense external forces by capturing optical information about the deformation of an elastic body under stress.

[0003] Typical visual-tactile sensors include a camera, an opaque housing, and a transparent elastomer. The transparent elastomer is mostly made of silicone. Currently, visual-tactile sensors primarily serve two functions:

[0004] 1. Measurement of three-dimensional forces

[0005] 2. Reconstruction of object surfaces, such as fingerprint recognition.

[0006] In response to this, existing technologies typically require visual-tactile sensors to be made relatively thin. For example, Chinese patent CN119723627A discloses a multi-curvature, high-resolution bionic micro visual-tactile sensor for fingertips, relating to the field of sensor technology. This sensor includes a fingertip structure, a transparent support structure, a flexible silicone sensing layer, and an image acquisition module. The fingertip structure has an internal cavity; the transparent support structure is fitted within the cavity; the flexible silicone sensing layer is bonded to the transparent support structure and has a black circular array of markers on its surface; the image acquisition module is mounted on the fingertip structure, with its acquisition end positioned above the transparent support structure, and its field of view covering the entire flexible silicone sensing layer area. To reduce the overall thinness of the elastomer, an opaque silver-gray silicone paint is sprayed onto the transparent silicone rubber with the black circular array of markers. While other existing technologies may use slightly different coatings, they still maintain the characteristics of being opaque and sufficiently thin.

[0007] While such visual-tactile sensors can meet the needs of both three-dimensional force measurement and three-dimensional reconstruction, they have the following drawbacks:

[0008] 1. Due to its thinness, it requires a transparent acrylic sheet for rigid support. This acrylic sheet is usually located on the side closer to the camera, and the acrylic sheet increases the thickness of the product.

[0009] 2. Due to the slow deformation recovery rate and excessive deformation of silicone, the measurement speed is too slow in large-scale measurements. Summary of the Invention

[0010] The purpose of this invention is to provide a visual-tactile sensor that enables high sampling rate measurement of dynamic force, thereby overcoming the shortcomings of the prior art.

[0011] The objective of this invention can be achieved through the following technical solutions:

[0012] A visual-tactile sensor for measuring dynamic force at a high sampling rate includes an image acquisition module, a supporting shell, an illumination module, and a contact module. The contact module includes a transparent elastic layer with internal optical markers, an opaque elastic layer, and a flexible metal support mesh embedded in the opaque elastic layer. The outer bottom surface of the transparent elastic layer is attached to and connected to the inner bottom surface of the opaque elastic layer. The inner bottom surface is suspended and faces the optical incident end of the image acquisition module. The outer bottom surface of the opaque elastic layer is in contact with the object to be measured.

[0013] Both the opaque elastic layer and the transparent elastic layer are made of silicone, and the hardness of the opaque elastic layer is greater than that of the transparent elastic layer.

[0014] There are multiple optical markers, and each optical marker is evenly distributed at various positions in the transparent elastic layer.

[0015] The inner bottom surface of the transparent elastic layer is sanded to achieve diffuse reflection, wherein the sandpaper has a mesh size of 1500.

[0016] One surface of the supporting shell is provided with a notch for exposing the outer bottom surface of the opaque elastic layer, and the image acquisition module, the illumination module and the contact module are located in the supporting shell.

[0017] The image acquisition module includes an event camera and an optical lens.

[0018] The illumination module includes an LED circuit board and a drive control circuit.

[0019] The flexible metal support mesh is a stainless steel wire mesh with a mesh size of 0.9 mm and a wire diameter of 0.18 mm.

[0020] The preparation process of the transparent elastic layer includes:

[0021] Add a thickener to liquid silicone;

[0022] Optical markers are added to liquid silicone with a thickener; the mixture is then processed using a mechanical stirring combined with a vacuum pump for degassing.

[0023] The ratio of the thickener to the liquid silicone is 1:200.

[0024] Compared with the prior art, the present invention has the following beneficial effects:

[0025] 1. The overall design employs a two-layer structure: a transparent elastic layer and an opaque elastic layer. A flexible metal support mesh is embedded within the opaque elastic layer. Firstly, the flexible metal support mesh provides rigid support, eliminating the need for the existing transparent acrylic sheet and resolving the increased thickness issue associated with acrylic sheets. This makes it suitable for applications with strict size and weight limitations, such as the fingertips of humanoid robot dexterity hands, the ends of medical endoscopes, or minimally invasive surgical instruments. This significantly improves the ease of sensor installation and integration, and broadens the range of applications. Secondly, the flexible metal support mesh also provides rigidity to the opaque elastic layer, thereby increasing the force measurement range and providing faster elastic deformation recovery, enabling high-rate dynamic force measurement.

[0026] 2. First, by adding a thickener to liquid silicone and combining it with multi-stage vacuum degassing technology, the sedimentation problem of optical markers during the curing process is effectively solved, achieving a truly three-dimensional random and uniform distribution of markers within the elastic body. This provides a richer and more reliable depth strain information field than traditional processes. Second, by precisely polishing the inner surface of the transparent elastic sensing layer, interfering specular reflections are cleverly transformed into uniform diffuse reflections. This eliminates light spot artifacts and related signal noise caused by internal light reflections at their physical source. The combination of these two key processes significantly improves the quality and signal-to-noise ratio of the sensor's original optical signal, laying a solid physical foundation for subsequent algorithms to achieve high-precision three-dimensional force estimation and dynamic tracking.

[0027] 3. Due to the specific mechanical response characteristics of the thin, flexible contact module, the three-dimensional spatial distribution of optical markers in the transparent elastomer achieved through a special process, and the resulting clear and low-noise internal imaging conditions, a single-event camera installed inside the sensor can effectively capture high-quality, complementary optical data streams from the movement of these optical markers. These data streams reflect both the rapid dynamic changes in contact force and characterize the quasi-static contact morphology. Therefore, the mechanical structure of this invention provides an excellent raw signal foundation for subsequent use of advanced data fusion processing algorithms to achieve comprehensive, accurate, and long-term stable perception of complex forces containing both high-frequency dynamic components and static / gradually varying components. This is expected to overcome the inherent limitations of traditional single-sensing modes (pure frame-based or pure event-based) in terms of dynamic response range or static force perception stability. Attached Figure Description

[0028] Figure 1 This is a schematic diagram of the external structure of the present invention;

[0029] Figure 2 This is a schematic diagram of the internal structure of the present invention;

[0030] Figure 3 This is a schematic diagram of the flexible metal support mesh structure;

[0031] Figure 4 This is a schematic diagram showing the arrangement of the transparent elastic layer;

[0032] Figure 5 A schematic diagram of the supporting and raised structure;

[0033] The components include: 1. an opaque elastic layer, 2. a supporting shell, 3. an optical lens, 4. an event camera, 5. a flexible metal support mesh, 6. a transparent elastic layer, and 7. a supporting and elevating structure. Detailed Implementation

[0034] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0035] A visual-tactile sensor that enables high sampling rate measurement of dynamic force, such as Figures 1 to 5 As shown, it includes an image acquisition module, a supporting shell, a lighting module, and a contact module. The contact module includes a transparent elastic layer with internal optical markers, an opaque elastic layer, and a flexible metal support mesh embedded in the opaque elastic layer. The outer bottom surface of the transparent elastic layer is attached to and connected to the inner bottom surface of the opaque elastic layer. The inner bottom surface is suspended and faces the optical incident end of the image acquisition module. The outer bottom surface of the opaque elastic layer is in contact with the object to be measured.

[0036] The overall design employs a two-layer structure: a transparent elastic layer and an opaque elastic layer. A flexible metal support mesh is embedded within the opaque elastic layer. Firstly, the flexible metal support mesh provides rigid support, eliminating the need for the existing transparent acrylic sheet and resolving the increased thickness issue associated with acrylic sheets. This makes it suitable for applications with strict size and weight limitations, such as the fingertips of humanoid robot dexterity hands, the ends of medical endoscopes, or minimally invasive surgical instruments. This significantly improves the ease of sensor installation and integration, as well as the versatility of its applications. Secondly, the flexible metal support mesh also provides rigidity to the opaque elastic layer, thereby increasing the force measurement range and providing faster elastic deformation recovery, enabling high-rate dynamic force measurement.

[0037] Both the opaque elastic layer and the transparent elastic layer are made of silicone. The hardness of the opaque elastic layer is greater than that of the transparent elastic layer. In this embodiment, the opaque elastic layer can be made of black silicone material with a specific first preset hardness (e.g., Shore A-30) to achieve light protection and durability. The transparent elastic layer can be made of transparent silicone material with a specific second preset hardness (e.g., Shore A-5, which is usually softer than the first preset hardness). Tiny optical markers are randomly and uniformly distributed in three dimensions inside the transparent elastic layer, and the inner surface facing the image acquisition module is mechanically polished to form a diffuse reflection surface, thereby reducing the noise generated by the reflection of internal light.

[0038] The preparation process of the transparent elastic layer includes:

[0039] Add a thickener to liquid silicone;

[0040] Optical markers are added to liquid silicone containing a thickener;

[0041] The process employs a degassing technique combining mechanical stirring and a vacuum pump.

[0042] The ratio of thickener to liquid silicone is 1:200.

[0043] Specifically, the preparation process is crucial for achieving a three-dimensional random and uniform distribution of optical markers (e.g., glass microspheres) within the transparent elastic layer. This process includes the following steps: First, to overcome the problem of optical markers settling during curing due to density differences between the optical markers and the liquid silicone matrix, a specific proportion of thickener is added to the liquid silicone to increase the overall viscosity of the mixture. This ensures that the markers can be stably suspended in the matrix after mechanical stirring, and the thickener does not alter the hardness of the silicone molding. Second, since increased viscosity makes it easier to introduce and trap air bubbles during stirring, a degassing process combining mechanical stirring and a vacuum pump is used to eliminate the interference of air bubbles on imaging quality. For example, while mechanically stirring the mixture in a vacuum chamber, a bipolar vacuum pump is used to thoroughly vacuum the mixture to completely remove all residual air bubbles, thereby obtaining a high-quality transparent elastic sensing layer with no air bubbles and truly uniform three-dimensional distribution of optical markers.

[0044] Furthermore, to address the issue of specular reflection caused by excessive smoothness on the inner surface of the transparent elastic sensing layer (i.e., the surface facing the image acquisition module), which leads to bright spot artifacts and a large amount of non-motion-related pseudo-event noise in the image, this invention employs a key surface treatment process: microscopic polishing with high-grit sandpaper to transform the original smooth specular surface into a uniform diffuse reflective surface. This treatment effectively eliminates specular reflection without significantly affecting the sensor's spatial resolution, resulting in softer and more uniform internal illumination. This significantly improves the optical signals received by the event camera and image sensor, providing high-quality raw data for subsequent accurate force estimation. In this embodiment, the sandpaper has a grit of 1500.

[0045] The flexible metal support mesh is made of stainless steel wire mesh with a mesh size of 0.9 mm and a wire diameter of 0.18 mm. This mesh structure is directly embedded in the opaque elastic surface layer during the manufacturing process, serving as the main structural support for the contact module and replacing the traditional rigid support plate (such as acrylic plate), thereby significantly reducing the overall thickness of the contact module.

[0046] The image acquisition module includes an event camera and an optical lens. The event camera in the image acquisition module is a camera (e.g., the DAVIS346 model) that can simultaneously acquire asynchronous event signal streams with high temporal resolution and standard image frames with low frame rates. Its dual-modal data output characteristics provide the necessary raw signal input for subsequent sensor fusion estimation of high-frequency dynamic force and long-term stable force.

[0047] The aforementioned illumination module includes multiple LED light-emitting devices and corresponding constant current drive control circuits. These devices are integrated on a custom printed circuit board (PCB) and arranged in a structured manner, such as in a ring or a specific array, around the field of view of the optical lens to ensure uniform, stable, and sufficient illumination of the transparent elastic sensing layer and the optical markers inside it.

[0048] One surface of the support housing has a notch for exposing the outer bottom surface of the opaque elastic layer. The image acquisition module, illumination module, and contact module are housed within the support housing. The support housing can be manufactured using additive manufacturing technologies such as 3D printing, and is integrally molded from lightweight, high-strength engineering plastic. Its internal design incorporates precisely fitted positioning structures, mounting interfaces, and fastening features based on the geometry and dimensions of the contact module, image acquisition module, and illumination module, ensuring the relative positional accuracy and structural stability of each optical and sensing component after assembly.

[0049] When the visual-tactile sensor of this invention is in operation, its front-end contact module comes into contact with an external object to be measured. An external force is applied to the opaque elastic surface of the contact module, causing elastic deformation of the composite flexible structure composed of the surface, the flexible metal support mesh structure, and the transparent elastic sensing layer. As the composite flexible structure deforms, the optical markers (glass microspheres) inside the transparent elastic sensing layer undergo corresponding three-dimensional spatial displacement. Under the continuous illumination of the illumination module, the image acquisition module (event camera) captures and outputs the displacement information of these optical markers in real time. This displacement information is recorded simultaneously in the form of a high-temporal-resolution event signal stream and a low-frame-rate image frame, which together characterize the dynamic and static deformation features of the contact module caused by the external force.

[0050] This invention replaces the traditional rigid support plate with an embedded flexible metal support mesh structure. This allows the contact module to achieve a significantly thinner design and excellent overall deformation compliance while maintaining necessary structural support. This makes the sensor not only easy to integrate into compact spaces such as a robot's fingertip, but also enables a more sensitive and natural response to minute force changes. Simultaneously, the unique transparent elastic sensing layer fabrication process employed in this invention—namely, ensuring a uniform three-dimensional suspension distribution of optical markers through the addition of thickeners and vacuum degassing, and eliminating internal reflection noise through inner surface polishing—is key to achieving high signal-to-noise ratio optical sensing. The uniformly distributed markers ensure rich and reliable displacement field information regardless of depth and position during deformation under stress. The polished diffuse reflective surface purifies the signal at its physical source, ensuring that the image acquisition module captures the true signal purely caused by marker movement, rather than interference caused by optical artifacts. These integrated design features in mechanical structure, material selection, and key manufacturing processes enable a single image acquisition module to effectively capture high-quality, complementary deformation data from the complex motion of marker points. This lays a crucial physical foundation and signal source for subsequent high-frequency, high-precision, and long-term stable three-dimensional force sensing through specific algorithms.

[0051] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A visual-tactile sensor for achieving high sampling rate measurement of dynamic force, comprising an image acquisition module, a supporting shell, an illumination module, and a contact module, wherein the contact module includes a transparent elastic layer with internal optical markers, characterized in that, The contact module also includes an opaque elastic layer and a flexible metal support mesh embedded in the opaque elastic layer. The outer bottom surface of the transparent elastic layer is attached to and connected to the inner bottom surface of the opaque elastic layer. The inner bottom surface is suspended and faces the optical incident end of the image acquisition module. The outer bottom surface of the opaque elastic layer is in contact with the object to be measured.

2. The visual-tactile sensor for realizing high sampling rate measurement of dynamic force according to claim 1, characterized in that, Both the opaque elastic layer and the transparent elastic layer are made of silicone, and the hardness of the opaque elastic layer is greater than that of the transparent elastic layer.

3. The visual-tactile sensor for realizing high sampling rate measurement of dynamic force according to claim 1, characterized in that, There are multiple optical markers, and each optical marker is evenly distributed at various positions in the transparent elastic layer.

4. A visual-tactile sensor for achieving high sampling rate measurement of dynamic force according to claim 1, characterized in that, The inner bottom surface of the transparent elastic layer is sanded to achieve diffuse reflection, wherein the sandpaper has a mesh size of 1500.

5. A visual-tactile sensor for realizing high sampling rate measurement of dynamic force according to claim 1, characterized in that, One surface of the supporting shell is provided with a notch for exposing the outer bottom surface of the opaque elastic layer, and the image acquisition module, the illumination module and the contact module are located in the supporting shell.

6. A visual-tactile sensor for realizing high sampling rate measurement of dynamic force according to claim 1, characterized in that, The image acquisition module includes an event camera and an optical lens.

7. A visual-tactile sensor for realizing high sampling rate measurement of dynamic force according to claim 1, characterized in that, The illumination module includes an LED circuit board and a drive control circuit.

8. A visual-tactile sensor for realizing high sampling rate measurement of dynamic force according to claim 1, characterized in that, The flexible metal support mesh is a stainless steel wire mesh with a mesh size of 0.9 mm and a wire diameter of 0.18 mm.

9. A visual-tactile sensor for realizing high sampling rate measurement of dynamic force according to claim 1, characterized in that, The preparation process of the transparent elastic layer includes: Add a thickener to liquid silicone; Optical markers are added to liquid silicone with a thickener; the mixture is then processed using a mechanical stirring combined with a vacuum pump for degassing.

10. A visual-tactile sensor for realizing high sampling rate measurement of dynamic force according to claim 9, characterized in that, The ratio of the thickener to the liquid silicone is 1:200.

Images