Fingertip-imitating sensor with multi-modal information

Abstract

The invention discloses a multi-modal information imitated fingertip sensor. The multi-modal information imitated fingertip sensor comprises a six-dimensional force sensor, a finger root connecting part and a finger tip part which are connected in sequence, the six-dimensional force sensor is used for detecting the stress condition of the finger tip part; the finger tip part comprises a transparent finger tip shell, a soft film finger tip sleeve, an image acquisition mechanism, a temperature sensor and a nail cover; the transparent finger tip shell is fixedly connected with the finger root connecting part, and a finger tip cavity is formed in the transparent finger tip shell; the wall surface of the soft film finger tip sleeve is covered with a diffuse reflection coating and a plurality of mark points; the soft film finger tip sleeve surrounds the transparent finger tip shell, and the ice and the transparent finger tip shell are enclosed and filled with heat conduction liquid. The image acquisition mechanism is used for detecting and shooting the change state of the plurality of mark points; the temperature sensor is used for detecting the temperature of the heat-conducting liquid; the nail cover is arranged outside the finger end part; the scheme has the characteristics of modularization, easiness in implementation, multiple applicable scenes and the like, and can further improve the skillful operation capability of the robot after being integrated on the robot end effector.

Description

Technical Field

[0001] This invention relates to the field of sensor technology, and in particular to a fingertip-like sensor for multimodal information. Background Technology

[0002] In early sensor research, end effectors mostly relied on single-modal visual or tactile information for environmental perception and operational control, which was insufficient to meet the demands of complex tasks. Recently, high-resolution visual-tactile sensors have attracted widespread attention. These sensors can acquire tactile information simultaneously with visual information; however, most can only perceive the planar or hemispherical sensor surface corresponding to their camera. This not only suffers from the limitation of a single visual-tactile perception range but also presents difficulties in object grasping, especially for small, thin card-like objects that are often difficult to pick up from a table. Faced with increasingly complex task requirements, higher demands are being placed on the information perception and structural design of visual-tactile sensors. Therefore, to address the aforementioned shortcomings, a high-resolution sensor design scheme with multi-angle visual-tactile perception range and strong grasping capabilities is needed. Summary of the Invention

[0003] The purpose of this invention is to provide a multimodal information-inspired fingertip sensor, which provides a high-resolution sensor design scheme with multi-angle visual and tactile perception range and strong grasping ability.

[0004] To address the aforementioned technical problems, this invention provides a multimodal information-based fingertip sensor, comprising a six-dimensional force sensor, a finger root connection portion, and a fingertip portion connected in sequence; the six-dimensional force sensor is used to detect the force applied to the fingertip portion; the fingertip portion includes a transparent fingertip shell, a soft membrane fingertip sleeve, an image acquisition mechanism, a temperature sensor, and a nail plate; the transparent fingertip shell is connected and fixed to the finger root connection portion, and a fingertip cavity is formed inside the transparent fingertip shell; the wall surface of the soft membrane fingertip sleeve is covered with a diffuse reflection coating, and the reflective surface of the diffuse reflection coating faces... The soft membrane fingertip sleeve surrounds an internal space, and the reflective surface of the diffuse reflection coating is provided with multiple marker points; the soft membrane fingertip sleeve surrounds the outside of the transparent fingertip shell, and the soft membrane fingertip sleeve and the transparent fingertip shell enclose a sealed cavity, which is filled with a heat-conducting liquid; the imaging part of the image acquisition mechanism is aligned with the multiple marker points through the fingertip cavity, and the image acquisition mechanism is used to detect and capture the changing state of the multiple marker points; the temperature sensor is used to detect the temperature of the heat-conducting liquid; the nail plate is located on the outside of the fingertip.

[0005] In one embodiment, the finger root connection includes an annular finger root shell and a force-bearing connector; the annular finger root shell is fitted over the force-bearing connector and the transparent finger tip shell, and both ends of the annular finger root shell are threadedly connected to the force-bearing connector and the transparent finger tip shell, respectively; the force-bearing connector is connected to the six-dimensional force sensor.

[0006] In one embodiment, the image acquisition mechanism is connected to the force-bearing connector, and the imaging part of the image acquisition mechanism is aligned with the fingertip cavity through the hollow part of the annular finger root shell.

[0007] In one embodiment, the image acquisition mechanism includes a camera and a supplementary lighting unit; the camera is located at the opening of the fingertip cavity; the supplementary lighting unit is used to provide supplementary lighting for the shooting position of the camera.

[0008] In one embodiment, the diffuse reflection coating and the plurality of the marker points cover the fingertip and the pad of the soft membrane fingertip sleeve; the camera's shooting range covers the fingertip and the pad of the soft membrane fingertip sleeve.

[0009] In one embodiment, the camera's shooting position is aligned with the fingertip of the soft membrane fingertip sleeve; a reflector is provided inside the fingertip cavity, the reflector is located within the camera's shooting range, the reflective surface of the reflector faces the fingertip of the soft membrane fingertip sleeve and the camera's shooting position, and the reflector is used to reflect the image of the fingertip of the soft membrane fingertip sleeve to the camera.

[0010] In one embodiment, the reflector is arranged at an angle of 8° to 12° relative to the camera's shooting axis.

[0011] In one embodiment, the supplementary lighting unit includes a first circuit board and a second circuit board with light-emitting function; the first circuit board is arranged in a ring around the periphery of the camera, and the light-emitting direction of the first circuit board is aligned with the fingertip of the soft membrane fingertip sleeve; the second circuit board is located at the intersection of the fingertip and the finger pad of the soft membrane fingertip sleeve, and the light-emitting direction of the second circuit board is aligned with the finger pad of the soft membrane fingertip sleeve.

[0012] In one embodiment, the second circuit board is arranged adjacent to the sealed cavity, and the temperature sensor is provided on the second circuit board.

[0013] In one embodiment, the outer peripheral wall of the transparent fingertip shell is provided with an annular groove, the opening of the soft membrane fingertip sleeve is inserted into the annular groove, and a sealing ring is provided at the connection between the soft membrane fingertip sleeve and the annular groove.

[0014] The beneficial effects of this invention are as follows:

[0015] 1. Since the reflective surface of the mirror faces the fingertip of the soft membrane fingertip sleeve and the shooting position of the camera, the mirror is used to reflect the image of the fingertip of the soft membrane fingertip sleeve to the camera. Therefore, the camera can not only shoot the fingertip of the soft membrane fingertip sleeve that is directly opposite it, but also shoot the fingertip of the soft membrane fingertip sleeve that is shot by the mirror. This realizes dual-view perception of the fingertip and fingertip of the fingertip sleeve by a single camera, and realizes a multi-angle visual and tactile perception range.

[0016] 2. This invention incorporates a fingernail cover, thus enabling the use of fingernails or fingertips to grasp objects of different sizes and obtain force information while maintaining a small overall size. This helps the end effector complete more precise operational tasks and also effectively prevents the simulated fingertip sensor from being damaged by collisions with hard objects, thus protecting the simulated fingertip sensor.

[0017] 3. Since the soft membrane fingertip sleeve and the transparent fingertip shell form a sealed cavity, which is filled with a heat-conducting liquid, and the temperature sensor is used to detect the temperature of the heat-conducting liquid, when the simulated fingertip sensor comes into contact with objects of different temperatures, the temperature of the heat-conducting liquid will also change accordingly. Therefore, at this time, it is only necessary to use the temperature sensor to detect the temperature of the heat-conducting liquid to obtain the corresponding temperature information. Attached Figure Description

[0018] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of the structure provided in an embodiment of the present invention;

[0020] Figure 2 yes Figure 1 A schematic diagram of the disassembled structure;

[0021] Figure 3 yes Figure 1 A schematic diagram of the cross-sectional structure;

[0022] Figure 4 yes Figure 3 A diagram illustrating the camera's shooting path.

[0023] The attached figures are labeled as follows:

[0024] 100. Six-dimensional force sensor;

[0025] 200. Finger root connection part; 210. Annular finger root shell; 220. Force-bearing connecting part;

[0026] 300. Finger tip; 310. Transparent fingertip shell; 311. Finger tip cavity; 312. Annular groove; 313. Sealing ring; 320. Soft film fingertip sleeve; 321. Diffuse reflection coating; 322. Marking point; 331. First circuit board; 332. Second circuit board; 333. Camera; 340. Fingernail; 350. Sealing cavity; 360. Thermally conductive liquid; 370. Reflector. Detailed Implementation

[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0028] This invention provides a fingertip-like sensor for multimodal information, an embodiment of which is as follows: Figures 1 to 3 As shown, the device includes a six-dimensional force sensor 100, a finger root connection portion 200, and a fingertip 300 connected in sequence. The six-dimensional force sensor 100 is used to detect the force on the fingertip 300. The fingertip 300 includes a transparent fingertip shell 310, a soft film fingertip sleeve 320, an image acquisition mechanism, a temperature sensor, and a nail plate 340. The transparent fingertip shell 310 is connected and fixed to the finger root connection portion 200, and a fingertip cavity 311 is formed inside the transparent fingertip shell 310. The wall surface of the soft film fingertip sleeve 320 is covered with a diffuse reflection coating 321, and the reflective surface of the diffuse reflection coating 321 faces the soft film fingertip sleeve 320. The internal space is surrounded by a diffuse reflection coating 321 with multiple markers 322 on its reflective surface; the soft film fingertip sleeve 320 surrounds the outside of the transparent fingertip shell 310, and the soft film fingertip sleeve 320 and the transparent fingertip shell 310 enclose a sealed cavity 350, which is filled with a heat-conducting liquid 360; the imaging part of the image acquisition mechanism is aligned with the multiple markers 322 through the fingertip cavity 311, and the image acquisition mechanism is used to detect the changing state of the multiple markers 322; the temperature sensor is used to detect the temperature of the heat-conducting liquid 360; and the nail cap 340 is located on the outside of the fingertip 300.

[0029] After adopting the above settings, multiple modal information can be acquired using a simulated fingertip sensor.

[0030] For example, when the simulated fingertip sensor comes into contact with an object, the soft membrane fingertip sleeve 320 will deform, causing changes in the position and state of multiple marker points 322. Therefore, after the image acquisition mechanism acquires images of multiple marker points 322, the relevant image information can be sent to the computer. The computer can obtain the corresponding tactile information by analyzing the changes in the state of multiple marker points 322 before and after.

[0031] When the simulated fingertip sensor comes into contact with objects of different temperatures, the thermally conductive liquid 360 will also change in temperature. Therefore, at this time, the corresponding temperature information can be obtained by simply using a temperature sensor to detect the temperature of the thermally conductive liquid 360.

[0032] Furthermore, when the simulated fingertip sensor comes into contact with an object, not only will the deformation of the soft membrane fingertip sleeve 320 cause multiple marker points 322 to change state, but the force on the fingertip 300 will be transmitted to the six-dimensional force sensor 100 through the finger root connection 200. Therefore, after comprehensively analyzing the image information obtained by the image acquisition mechanism and the force information measured by the six-dimensional force sensor 100 using a computer, the corresponding force information can be obtained.

[0033] Furthermore, because this embodiment includes a fingernail 340, it enables the use of fingernails and fingertips to grasp objects of different sizes and obtain force information while maintaining a small overall size. This helps the end effector complete more precise operational tasks and also effectively prevents the simulated fingertip sensor from being damaged by collisions with hard objects, thus protecting the simulated fingertip sensor.

[0034] like Figure 1 and Figure 2 As shown, in this embodiment, the finger root connecting part 200 includes an annular finger root shell 210 and a force-bearing connector 220; the annular finger root shell 210 is fitted over the force-bearing connector 220 and the transparent finger tip shell 310, and the two ends of the annular finger root shell 210 are threadedly connected to the force-bearing connector 220 and the transparent finger tip shell 310, respectively; the force-bearing connector 220 is connected to the six-dimensional force sensor 100.

[0035] With this configuration, the inner peripheral walls at both ends of the annular finger root housing 210 are provided with internal threads, and the outer peripheral walls of the force-bearing connector 220 and the transparent finger tip housing 310 are provided with external threads. This allows the annular finger root housing 210 to be threadedly connected and fixed with the force-bearing connector 220 and the transparent finger tip housing 310, which not only makes installation convenient but also ensures high connection stability.

[0036] Specifically, in this embodiment, the image acquisition mechanism is connected to the force-bearing connector 220, and the shooting part of the image acquisition mechanism is aligned with the fingertip cavity 311 through the hollow part of the annular finger root shell 210.

[0037] With this setup, the image acquisition mechanism can lock itself onto the force-bearing connector 220 using screws to ensure a stable connection between the two; and the shooting part of the image acquisition mechanism is aligned with the fingertip cavity 311 through the hollow part of the ring-shaped finger root housing 210, which also ensures accurate alignment of the shooting direction.

[0038] like Figure 2 and Figure 3 As shown, this embodiment sets the image acquisition mechanism to include a camera 333 and a supplementary lighting unit; the camera 333 is located at the opening of the fingertip cavity 311; the supplementary lighting unit is used to provide supplementary lighting for the shooting position of the camera 333.

[0039] With this setting, the fill light unit can be used to provide fill light for the camera 333's shooting, ensuring that the camera 333 can obtain clear image information.

[0040] like Figure 2 and Figure 3 As shown, in this embodiment, the diffuse reflection coating 321 and multiple marker points 322 are configured to cover the fingertip and finger pad of the soft membrane fingertip sleeve 320; the shooting range of the camera 333 covers the fingertip and finger pad of the soft membrane fingertip sleeve 320.

[0041] By adopting this setting, it is ensured that the coverage of the diffuse reflection coating 321 and the multiple marker points 322 is sufficiently wide. For the diffuse reflection coating 321, setting its coverage wide ensures that the diffuse reflection coating 321 can achieve diffuse reflection on all parts of the soft film fingertip sleeve 320, so that the camera 333 can clearly capture all parts of the soft film fingertip sleeve 320. For the multiple marker points 322, setting its coverage wide ensures that after the camera 333 captures the multiple marker points 322, it can know the state changes of each part of the soft film fingertip sleeve 320.

[0042] like Figure 2 and Figure 3 As shown, in this embodiment, the shooting area of ​​the camera 333 is aligned with the fingertip of the soft membrane fingertip sleeve 320; a reflector 370 is provided in the fingertip cavity 311, and the reflector 370 is located within the shooting range of the camera 333. The reflecting surface of the reflector 370 faces the fingertip of the soft membrane fingertip sleeve 320 and the shooting area of ​​the camera 333. The reflector 370 is used to reflect the image of the fingertip of the soft membrane fingertip sleeve 320 to the camera 333.

[0043] With this setup, the camera 333 can not only capture the fingertip of the soft membrane fingertip sleeve 320 directly opposite it, but also capture the fingertip of the soft membrane fingertip sleeve 320 using the reflector 370. This enables a single camera 333 to perceive both the fingertip and the fingertip, achieving a multi-angle visual and tactile perception range.

[0044] In order to ensure better shooting effect of camera 333 on the fingertip of soft membrane finger sleeve 320, it is recommended to set the reflector 370 at an angle of 8° to 12° relative to the shooting axis of camera 333. For example, in this embodiment, the reflector 370 is set at an angle of 10° relative to the shooting axis of camera 333.

[0045] Of course, it is better to arrange the reflector 370 at an angle of 8°, 9°, 11° and 12° relative to the shooting axis of the camera 333. The specific choice can be made according to the actual needs.

[0046] like Figure 2 and Figure 3 As shown, in this embodiment, the supplementary lighting unit includes a first circuit board 331 and a second circuit board 332 with light-emitting function. The first circuit board 331 is arranged in a ring around the periphery of the camera 333, and the light-emitting direction of the first circuit board 331 is aligned with the fingertip of the soft membrane fingertip sleeve 320. The second circuit board 332 is located at the intersection of the fingertip and the finger pad of the soft membrane fingertip sleeve 320, and the light-emitting direction of the second circuit board 332 is aligned with the finger pad of the soft membrane fingertip sleeve 320.

[0047] With this setup, the first circuit board 331 can be used to provide supplemental lighting to the fingertip of the soft membrane fingertip sleeve 320, and the second circuit board 332 can be used to provide supplemental lighting to the fingertip of the soft membrane fingertip sleeve 320, so that the camera 333 can achieve clear shooting of both the fingertip and the fingertip of the soft membrane fingertip sleeve 320.

[0048] Among them, programmable LEDs can be set on the first circuit board 331 and the second circuit board 332 to emit light. The programmable LEDs are used to provide uniform RGB three-color illumination at equal intervals to realize the dual-view illumination field of the fingertip and finger pad of the soft film fingertip sleeve 320.

[0049] like Figure 3 As shown, in this embodiment, the second circuit board 332 is arranged adjacent to the sealing cavity 350, and a temperature sensor is provided on the second circuit board 332.

[0050] With this setup, the second circuit board 332 can be used for supplementary lighting and temperature detection, thereby improving the compactness of the simulated fingertip sensor.

[0051] like Figure 2 As shown, in this embodiment, the outer peripheral wall of the transparent fingertip shell 310 is provided with an annular groove 312, the opening of the soft membrane fingertip sleeve 320 is inserted into the annular groove 312, and a sealing ring 313 is provided at the connection between the soft membrane fingertip sleeve 320 and the annular groove 312.

[0052] With this setup, the sealing ring 313 can be used to seal the connection between the soft membrane finger sleeve 320 and the annular groove 312 to prevent leakage.

[0053] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A fingertip-like sensor for multimodal information, characterized in that, It includes a six-dimensional force sensor, a finger root connector, and a finger tip connected in sequence; The six-dimensional force sensor is used to detect the force applied to the fingertip; The fingertip includes a transparent fingertip shell, a soft film fingertip sleeve, an image acquisition mechanism, a temperature sensor, and a nail cover; The transparent fingertip shell is connected and fixed to the finger root connection part, and the transparent fingertip shell has a fingertip cavity inside; The wall surface of the soft membrane fingertip sleeve is covered with a diffuse reflection coating, the reflective surface of which faces the internal space enclosed by the soft membrane fingertip sleeve, and multiple marking points are provided on the reflective surface of the diffuse reflection coating; and the soft membrane fingertip sleeve surrounds the outside of the transparent fingertip shell, and the soft membrane fingertip sleeve and the transparent fingertip shell enclose a sealed cavity, which is filled with a heat-conducting liquid. The image acquisition mechanism's shooting part is aligned with multiple marker points through the fingertip cavity, and the image acquisition mechanism is used to detect and capture the changing state of the multiple marker points. The temperature sensor is used to detect the temperature of the heat-conducting liquid; The nail cover is located on the outside of the fingertip.

2. The fingertip-like sensor according to claim 1, characterized in that, The finger root connection includes an annular finger root shell and a force-bearing connector; The annular finger root shell is fitted over the force-bearing connector and the transparent finger tip shell, and the two ends of the annular finger root shell are threadedly connected to the force-bearing connector and the transparent finger tip shell, respectively. The force-bearing connector is connected to the six-dimensional force sensor.

3. The fingertip-like sensor according to claim 2, characterized in that, The image acquisition mechanism is connected to the force-bearing connector, and the imaging part of the image acquisition mechanism is aligned with the fingertip cavity through the hollow part of the ring-shaped finger root shell.

4. The fingertip-like sensor according to claim 1, characterized in that, The image acquisition mechanism includes a camera and a supplementary lighting unit; The camera is located at the opening of the fingertip cavity; The supplementary lighting unit is used to provide supplementary lighting for the shooting position of the camera.

5. The fingertip-like sensor according to claim 4, characterized in that, The diffuse reflection coating and the multiple marking points cover the fingertip and finger pad of the soft membrane fingertip sleeve; The camera's shooting range covers the fingertip and finger pad of the soft membrane fingertip sleeve.

6. The fingertip-like sensor according to claim 5, characterized in that, The camera's shooting area is aligned with the fingertip of the soft membrane fingertip sleeve; A reflector is provided inside the fingertip cavity. The reflector is located within the shooting range of the camera. The reflective surface of the reflector faces the fingertip area of ​​the soft membrane fingertip sleeve and the shooting area of ​​the camera. The reflector is used to reflect the image of the fingertip area of ​​the soft membrane fingertip sleeve to the camera.

7. The fingertip-like sensor according to claim 6, characterized in that, The reflector is arranged at an angle of 8° to 12° relative to the shooting axis of the camera.

8. The fingertip-like sensor according to claim 6, characterized in that, The supplementary lighting unit includes a first circuit board and a second circuit board with light-emitting function; The first circuit board is arranged in a ring around the periphery of the camera, and the light emission direction of the first circuit board is aligned with the fingertip of the soft film fingertip sleeve. The second circuit board is located at the junction of the fingertip and the finger pad of the soft membrane fingertip sleeve, and the light emission direction of the second circuit board is aligned with the finger pad of the soft membrane fingertip sleeve.

9. The fingertip-like sensor according to claim 8, characterized in that, The second circuit board is arranged adjacent to the sealed cavity, and the temperature sensor is provided on the second circuit board.

10. The fingertip-like sensor according to claim 1, characterized in that, The outer peripheral wall of the transparent fingertip shell is provided with an annular groove, the opening of the soft membrane fingertip sleeve is inserted into the annular groove, and a sealing ring is provided at the connection between the soft membrane fingertip sleeve and the annular groove.

Images