Flexible visual tactile finger and gripper based on multi-mirror optical system

Abstract

The invention belongs to the technical field of flexible mechanical arms and intelligent sensing, and relates to a flexible visual tactile finger and gripper based on a multi-mirror optical system.The finger comprises a finger body base and a flexible finger body framework installed on the finger body base, and the flexible finger body framework is of a bionic fin ray effect structure; the bionic fin ray effect structure is formed by hinging a front beam, a rear beam and rigid side beams arranged on the two sides of the front beam and the two sides of the rear beam. The front beam is provided with a touch sensing pad; a plurality of reflectors are arranged in the bionic fin ray effect structure and on the rear beam, and the reflectors independently face the tactile sensing pad; the finger body base is further provided with an image collector, and the image collector obtains multi-angle visual information through the body and the multiple reflectors. The invention innovatively designs a self-adaptive optical system with multiple sections of mirror surfaces capable of passively deforming along with the structure and a multifunctional sensing layer without sacrificing flexibility, so that the finger still keeps extreme flexibility while having high-performance sensing.

Description

Technical Field

[0001] This invention belongs to the field of flexible robotic arms and intelligent sensing technology, and relates to a flexible visual-tactile finger and gripper based on a multi-mirror optical system. Background Technology

[0002] Visual tactile sensors have become an important solution for robot tactile perception due to their high spatial resolution, low cost, and rich information carrying capacity.

[0003] Chinese patent application CN202510824840.8 relates to the design of a rigid-structure visual-tactile sensor. This technology discloses a visual-tactile sensor module for a robot end effector. The sensor structure proposed in this technical document includes: a closed rigid housing; a transparent light-transmitting plate embedded in the front end of the housing; an elastomer contact layer covering the outer surface of the light-transmitting plate; and an image acquisition unit precisely installed within the housing, which includes a camera, an LED light source, and a specially designed light-diffusing plate assembly. Its core technical feature is that the light emitted from the light source is converted into a uniform surface light source by the light-diffusing plate, enters from the side of the light-transmitting plate, and after internal conduction, is uniformly emitted from the front end surface, thereby improving the imaging quality and measurement accuracy of the elastomer deformation image. However, this existing technology has the following inherent technical limitations: 1) Completely rigid structure: The sensor is a standalone, enclosed rigid module, with all optical and structural components securely encapsulated and positioned within a rigid housing. This design means the sensor cannot withstand any form of bending or torsional deformation other than localized elastomer deformation.

[0004] 2) It fundamentally conflicts with the concept of flexible grasping: its rigid structure determines that it cannot be deeply and efficiently integrated with flexible grippers. If it is attached to a flexible gripper, it will become the hardest component in the system, severely damaging the inherent compliance and safe grasping ability of the flexible gripper, causing the grasping operation to degenerate into "rigid grasping", and making it impossible to achieve adaptive envelope grasping of fragile and irregular objects.

[0005] Another Chinese patent application, CN202411099226.1, also pertains to flexible visual-tactile sensors and gripper designs. This technology discloses a flexible visual-tactile finger based on a fin-effect structure, attempting to combine a visual-tactile sensor with a biomimetic fin-effect flexible gripper. The proposed finger structure includes: a fin-effect structure finger body made of a transparent flexible material; a sensing layer and an array of marker points on the finger body surface; and an image acquisition unit fixed to the base of the finger root, containing a camera and a light source. It reconstructs the overall deformation of the contact surface by tracking the displacement of the marker points using the camera. However, this existing technology has the following pressing technical problems that need to be addressed: 1) Single perception modality: Existing technologies mainly rely on geometric deformation perception, which is limited to detecting contact position and force, and lacks the ability to perceive multimodal tactile information such as temperature and fine texture.

[0006] 2) Insufficient dynamic sensing coverage: Existing optical systems are fixed to the base of the finger, resulting in a static and limited field of view (FOV). When the finger body undergoes significant deformation, the camera's field of view cannot follow the movement of the contact surface. This causes some sensing areas to be obstructed or moved out of the camera's field of view due to finger body deformation, creating a sensing blind spot. Consequently, the effective tactile sensing area is reduced, and it is impossible to provide continuous and stable visual-tactile feedback throughout the entire grasping interaction process. Summary of the Invention

[0007] To address the aforementioned technical problems in existing technologies, this invention proposes a flexible visual-tactile finger and gripper based on a multi-mirror optical system. This solution achieves stable tactile perception over a large area, in multiple modalities, and throughout the entire dynamic process while maintaining the extreme flexibility of the finned fingers. The specific technical solution is as follows: A flexible visual-tactile finger based on a multi-mirror optical system includes: a finger body base; a flexible finger body skeleton mounted on the finger body base; the flexible finger body skeleton adopts a biomimetic fin-effect structure; the biomimetic fin-effect structure is composed of a front beam, a rear beam, and rigid side beams hinged on both sides of the front and rear beams; the front beam is provided with a tactile sensing pad; inside the biomimetic fin-effect structure, multiple reflectors are arranged on the rear beam, each of the multiple reflectors independently facing the tactile sensing pad; an image acquisition device is also provided on the finger body base, the image acquisition device acquires multi-angle visual information through the finger body and the multiple reflectors.

[0008] Furthermore, the front beam and the rear beam are connected at a V-shaped angle at the top, and corresponding hinge parts are arranged at intervals from top to bottom. The rigid side beam is connected between the front beam and the rear beam through the hinge parts.

[0009] Furthermore, each reflector includes a T-shaped bracket and a mirror surface disposed on top of the T-shaped bracket.

[0010] Furthermore, the bottom of each T-shaped bracket is fixed to the midpoint between two adjacent hinges of the rear beam.

[0011] Furthermore, each mirror has an independently designed initial deflection angle.

[0012] Furthermore, the front beam includes a front beam frame, and the tactile sensing pad is solidified in the front beam frame.

[0013] Furthermore, the tactile sensing pad comprises, from the inside out, an elastic support layer, an elastic deformation layer, a reflective layer, a temperature-sensitive layer, and a light-shielding layer.

[0014] Furthermore, a flexible RGBWLED light strip is encapsulated on the elastic support layer by casting with a mixture of polydimethylsiloxane (PDMS).

[0015] Furthermore, the elastic support layer is a transparent flexible substrate made of polydimethylsiloxane; the elastic deformation layer is a layer made of ultra-soft transparent silicone with a Shore hardness of 0A, used to transmit surface texture information of the contact object; the reflective layer is a thin film made of silicone doped with aluminum powder and aluminum flakes and cut with an array of circular marker dots, used to provide high-contrast visual texture features; the temperature-sensitive layer is a thin film made of silicone doped with thermochromic pigments, using color changes to provide temperature sensing; and the light-shielding layer is a thin film made of silicone doped with black pigments, used to isolate ambient light interference.

[0016] A flexible visual-tactile gripper based on a multi-mirror optical system includes a motor housing, a lead screw drive mechanism, and two opposing flexible visual-tactile fingers. The motor housing contains a motor, the lead screw drive mechanism is mounted on the top of the motor housing and connected to the motor output, and the two flexible visual-tactile fingers are connected to the lead screw drive mechanism via their finger body bases and are driven by the motor to move in opposite directions.

[0017] Beneficial effects: 1) It resolves the contradiction between rigidity and flexibility: The finger body has no rigid shell. Its rigid-flexible finger skeleton and fully flexible material sensing pads enable the finger to maintain the original excellent flexibility of the fin effect while gaining multimodal visual and tactile perception capabilities.

[0018] 2) Achieved dynamic full-coverage perception: The innovative design of the multi-segment independent mirror system allows the optical field of view to passively adjust to the deformation of the finger. Even under large deformations, the finger can still maintain a high effective perception coverage, eliminating perception blind spots.

[0019] 3) Achieved multimodal high-precision sensing: The five-layer multifunctional sensing pad integrates texture, temperature, and mechanical sensing functions. Combined with deep learning algorithms to process the rich information captured by the camera, the finger can accurately sense contact force, contact position, and body deformation state, and can predict temperature changes.

[0020] 4) Advanced Interaction Capabilities: Based on multimodal perception information, the gripper of this invention can perform complex tasks such as distinguishing between hot and cold objects, force-closed-loop grasping, proprioception, and human-computer interaction. The flexible gripper's natural proprioceptive compliance and high-precision visual and tactile perception capabilities make it have great application potential in a wide range of applications, including general daily life scenarios, service robots, and medical and elderly care. Attached Figure Description

[0021] Figure 1 This is a three-dimensional view of a flexible visual-tactile finger based on a multi-mirror optical system according to this embodiment; Figure 2 Exploded view of the front beam of the flexible visual-tactile finger in this embodiment; Figure 3 An exploded view of a flexible visual-tactile finger based on a multi-mirror optical system in this embodiment; Figure 4 A side cross-sectional view of a flexible visual-tactile finger based on a multi-mirror optical system according to this embodiment; Figure 5 This embodiment presents a schematic diagram of a two-finger robotic gripper structure constructed using a flexible visual-tactile finger based on a multi-mirror optical system. In the diagram: 1—Front beam; 11—Light-shielding layer; 12—Temperature-sensitive layer; 13—Reflective layer; 14—Elastic deformation layer; 15—Flexible RGBWLED light strip; 16—Front beam frame; 17—Elastic support layer; 2—Rear beam; 21—Hinged joint; 22—T-shaped bracket; 23—Mirror surface; 3—Side beam; 4—Acquirer base; 5—Image acquirer; 6—Screw drive mechanism; 7—Motor box. Detailed Implementation

[0022] To make the objectives, technical solutions, and technical effects of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0023] like Figures 1-4 As shown, this embodiment provides a flexible visual-tactile finger based on a multi-mirror optical system, including: a finger body base, a flexible finger body skeleton mounted on the finger body base, and a multi-mirror optical system integrated inside the finger body skeleton.

[0024] Specifically, the flexible finger skeleton is a biomimetic fin-effect structure made of thermoplastic polyurethane (TPU) material through 3D printing. It includes a front beam 1 and a rear beam 2 that provide elastic support, and a rigid side beam 3 that provides a hinged connection. The front beam 1 and the rear beam 2 are hinged at the top in a V-shape. The rigid side beam can be made of resin 3D printing or other materials with equivalent stiffness support capabilities.

[0025] The front beam 1 includes a front beam frame 16 and a multifunctional tactile sensing pad solidified within the finger body front beam frame 16. The multifunctional tactile sensing pad is the core component of the front beam 1 and is the area of ​​direct contact between the finger and the environment. From the inside out, the multifunctional tactile sensing pad includes an elastic support layer 17, an elastic deformation layer 14, a reflective layer 13, a temperature-sensitive layer 12, and a light-shielding layer 11. The elastic support layer 17 is a transparent flexible substrate made of polydimethylsiloxane (PDMS); the elastic deformation layer 14 is a layer made of ultra-soft transparent silicone with a Shore hardness of 0A, used to transmit surface texture information of objects; the reflective layer 13 is a thin film made of silicone doped with aluminum powder and aluminum flakes, used to provide high-contrast visual texture features; the temperature-sensitive layer 12 is a thin film made of silicone doped with three thermochromic pigments, which change color from colorless to blue, green to yellow, and colorless to magenta at 18°C, 38°C, and 60°C, respectively, thereby realizing temperature sensing; and the light-shielding layer 11 is a thin film made of silicone doped with black pigment, used to isolate ambient light interference.

[0026] More specifically, the reflective layer 13 is a 0.25mm thin film formed by coating a mixture of silicone, aluminum powder, and aluminum sheet in a mass ratio of 400:20:3 onto a glass substrate and curing it. A circular marking array is cut out on the film using a laser cutting machine.

[0027] The thermosensitive layer 12 is a 0.15mm thin film formed by mixing silicone and three thermochromic pigments in a mass ratio of 20:1:1:1 and coating it with a scraping agent. The thermosensitive layer 12 is bonded to the outer surface of the reflective layer 13.

[0028] The light-shielding layer 11 is made of a mixture of silicone and black pigment, and is coated on the temperature-sensitive layer 12 with a thickness of 0.15 mm.

[0029] The multi-mirror optical system includes: a collector base 4 fixed to the base of the finger body; an image collector 5 mounted on the collector base 4, the image collector 5 employing a high-resolution miniature camera; and multiple independent reflectors, each independently fixed to the inner surface of the rear beam 2. Each independent reflector includes a T-shaped bracket 22 and a mirror surface 23 disposed on the top of the T-shaped bracket 22. The bottoms of the multiple T-shaped brackets 22 are spaced apart and connected to the midpoint between two hinge portions 21 on the inner surface of the rear beam 2, meaning that each rear beam segment divided by the hinge portion 21 has an independent reflector surface 23. 3. Furthermore, the deformation of the rear beam section will passively change the orientation of the mirror 23; the reflector has a unique angle layout, that is, each mirror 23 has an independently designed initial deflection angle, which is maintained by each rigid T-shaped bracket 22. The rigid T-shaped bracket 22 enables each mirror 23 to passively follow the deformation of the flexible rear beam 2 and change its angle synchronously, thereby dynamically reconstructing the internal light path; the lighting unit adopts a flexible RGBWLED light strip 15, which is encapsulated on the elastic support layer 17 of the front beam 1 by mixing PDMS, to provide internal lighting for the system.

[0030] The core advantage of the multi-mirror optical system lies in transforming the unavoidable large deformations during finger manipulation from a factor interfering with optical perception into a functional advantage that drives adaptive adjustment of the optical path. In the absence of deformation, the initial angle distribution of the pre-designed mirrors passively changes with finger deformation. The passive deflection of the mirror angles driven by finger deformation compensates for the loss of camera field of view caused by structural deformation, intelligently bringing the original blind spots back into the perception range, thus achieving adaptive perception coverage to deformation.

[0031] like Figure 5 As shown, a flexible visual-tactile gripper based on a multi-mirror optical system includes a motor housing 7, a lead screw transmission mechanism 6, and two opposing fingers. The motor housing 7 contains a motor, the lead screw transmission mechanism 6 is mounted on the top of the motor housing 7 and connected to the motor output, and the two fingers are connected to the lead screw transmission mechanism 6 via their finger body bases and are driven by the motor to move in opposite directions.

[0032] In summary, this invention creates a flexible finger that integrates "sensing and execution," and innovatively designs an adaptive optical system with multi-segment mirrors that can passively deform with the structure and a multifunctional sensing layer that does not sacrifice flexibility. This enables the finger to have the ability to perceive in multiple modes and map fine textures, fundamentally unifying high-performance sensing and ultimate flexibility.

[0033] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any way. Although the implementation process of the present invention has been described in detail above, those skilled in the art can still modify the technical solutions described in the foregoing examples or make equivalent substitutions for some of the technical features. All modifications and equivalent substitutions made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A flexible visual-tactile finger based on a multi-mirror optical system, comprising: A finger body base, and a flexible finger body skeleton mounted on the finger body base, wherein the flexible finger body skeleton adopts a biomimetic fin-effect structure, characterized in that... The biomimetic fin effect structure is composed of a front beam (1), a rear beam (2), and rigid side beams (3) hinged on both sides of the front beam (1) and the rear beam (2); The front beam (1) is provided with a tactile sensing pad; Inside the biomimetic fin effect structure, multiple reflectors are arranged on the rear beam (2), and each of the multiple reflectors faces the tactile sensing pad independently. An image acquisition device (5) is also provided on the base of the finger body. The image acquisition device (5) acquires multi-angle visual information through the body and multiple reflectors.

2. The flexible visual-tactile finger as described in claim 1, characterized in that, The front beam (1) and the rear beam (2) are connected at a V-shaped angle at the top, and corresponding hinge parts are arranged at intervals from top to bottom. The rigid side beam (3) is connected between the front beam (1) and the rear beam (2) through the hinge parts.

3. The flexible visual-tactile finger as described in claim 2, characterized in that, Each reflector includes a T-shaped bracket (22) and a mirror (23) disposed on top of the T-shaped bracket (22).

4. The flexible visual-tactile finger as described in claim 3, characterized in that, The bottom of each T-shaped bracket (22) is fixed at the midpoint between two adjacent hinges of the rear beam (2).

5. The flexible visual-tactile finger as described in claim 3, characterized in that, Each mirror (23) has an independently designed initial deflection angle.

6. The flexible visual-tactile finger as described in claim 1, characterized in that, The front beam (1) includes a front beam frame (16), and the tactile sensing pad is solidified in the front beam frame (16).

7. The flexible visual-tactile finger as described in claim 1, characterized in that, The tactile sensing pad comprises, from the inside out, an elastic support layer (17), an elastic deformation layer (14), a reflective layer (13), a temperature-sensitive layer (12), and a light-shielding layer (11).

8. The flexible visual-tactile finger as described in claim 7, characterized in that, A flexible RGBWLED light strip (15) is encapsulated on the elastic support layer (17) by casting with mixed polydimethylsiloxane PDMS.

9. The flexible visual-tactile finger as described in claim 7, characterized in that, The elastic support layer (17) is a transparent flexible substrate made of polydimethylsiloxane; the elastic deformation layer (14) is a layer made of ultra-soft transparent silicone with a Shore hardness of 0A, used to transmit the surface texture information of the contact object; the reflective layer (13) is a thin film made of silicone doped with aluminum powder and aluminum flakes and cut with an array of circular marker dots, used to provide high-contrast visual texture features; the temperature-sensitive layer (12) is a thin film made of silicone doped with thermochromic pigments, using color changes to provide temperature perception; the light-shielding layer (11) is a thin film made of silicone doped with black pigments, used to isolate ambient light interference.

10. A flexible visual-tactile gripper based on a multi-mirror optical system, characterized in that, The device includes a motor housing (7), a lead screw transmission mechanism (6), and two flexible visual-tactile fingers arranged opposite each other, which are described in any one of claims 1 to 9. The motor housing (7) contains a motor, the lead screw transmission mechanism (6) is mounted on the top of the motor housing (7) and connected to the output of the motor, and the two flexible visual-tactile fingers are connected to the lead screw transmission mechanism (6) by means of their finger body base and are driven by the motor to move in opposite directions.

Images