Visual-tactile sensor

Abstract

To provide a visual-tactile sensor that has a simple structure and can be made thin.SOLUTION: The visual-tactile sensor includes: an elastic body part 12 deformed by applied pressure; a marker 13 for detecting a pressing force; a pinhole body 15 provided in a lower surface of the elastic body part 12 and provided with a small hole 15a; and an image sensor 16 provided in a lower surface of the pinhole body 15 for receiving light transmitted through the small hole 15a. The image sensor 16 is capable of capturing a positional change of the marker 13 and calculating a vertical pressure and a horizontal pressure corresponding to an applied pressure on the basis of the positional change of the marker 13, and is also capable of capturing the entire upper surface of the elastic body part 12.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a sensor that combines the functions of a visual sensor and a tactile sensor (hereinafter referred to as a "visual-tactile sensor"). [Background technology]

[0002] 2. Description of the Related Art Conventionally, tactile sensors are known that detect a force applied to each point on a contact surface based on the contact state of the contact surface. For example, an optical tactile sensor has been disclosed that detects movement information of a marker within a force-receiving area corresponding to the area on the force-receiving layer where force is applied based on the results of photographing with a CCD camera, and determines the applied force.

[0003] That is, the device comprises a force-receiving layer to which a force is applied from the outside, an elastic and translucent marker layer having a plurality of marker groups therein each containing a different type of marker, a sheet layer disposed between the force-receiving layer and the marker layer, and when a force is applied to the force-receiving layer, the area to which the force is applied changes in brightness according to the magnitude of the force, holes formed so as to penetrate a portion of each of the force-receiving layer, the marker layer, and the sheet layer in a plan view in the thickness direction of the layer, a light-shielding member that blocks light from the holes toward the marker layer and the sheet layer, and a light-shielding member disposed on the opposite side of the force-receiving layer from the marker layer, and that photographs each marker in the marker layer and the sheet layer and also photographs the image through the holes. There is an optical tactile sensor that is characterized by comprising an imaging device that can image the area on the opposite side of the hole from the imaging device, a determination means that determines a force-receiving area in the sheet layer that is brighter than a predetermined brightness based on the imaging results from the imaging device, a determination means that determines a marker located in an area that is determined by the determination means to be a force-receiving area brighter than the predetermined brightness as the marker to be used in calculations to determine the force distribution in the force-receiving layer from among the markers in the marker layer, and a calculation means that calculates the force distribution in the force-receiving layer based on the movement information of the marker determined by the determination means from the imaging results from the imaging device (Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2010-190770 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in conventional tactile sensors, a lens is provided between the marker layer and the imaging device (not explicitly stated in Patent Document 1 above), which makes the lens thick, which poses a problem in that it is difficult to miniaturize the entire device.

[0006] The present invention has been made to solve the above-mentioned problems, and aims to provide a visual-tactile sensor that has the functions of both a visual sensor and a tactile sensor, has a simple structure, and can be made thin. [Means for solving the problem]

[0007] In order to achieve the above-mentioned object, the tactile sensor of the present invention (hereinafter, the present invention may be referred to as "the tactile sensor") comprises a transparent elastic body portion that is translucent and deforms when pressure is applied, one or more markers provided on the elastic body portion for detecting the applied pressure, a pinhole body with small holes provided on the underside of the elastic body portion, and an image sensor provided on the underside of the pinhole body for receiving light transmitted through the small holes, wherein the image sensor is capable of capturing images of changes in the position of the markers, and is capable of calculating the vertical and horizontal pressures corresponding to the applied pressure based on the changes in the position of the markers, and is capable of capturing images of the entire upper surface of the elastic body portion (the side (surface side) that is pressed by a contacting object (hereinafter, referred to as "contact object")).

[0008] In this tactile sensor, if the elastic body portion is formed by laminating a plurality of layers with different elastic moduli, it is preferable because the pressure can be calculated in accordance with the hardness of the contact object.

[0009] Here, the elastic body portion needs to have elasticity as well as translucency and transparency, and is preferably made of a gel-like material (preferable from the viewpoint of processability), silicone resin, etc. The elastic body portion needs to be made of a material having an appropriate elastic modulus corresponding to the hardness of the contact object.

[0010] The marker is an identifier that moves in response to the deformation of the elastic body, and is a component used to accurately determine the pressure and direction of the elastic body by detecting the relative movement (including positional change and direction, etc.) before and after the movement. The marker may be of any shape as long as its position can be recognized, such as a sphere, cube, or line (e.g., cross), and there is no restriction on the material.

[0011] Furthermore, the color of the marker is preferably a surface state or color (e.g., black) that has a high light absorption rate in order to reduce light reflection from the marker (strong reflected light increases measurement errors of the marker position), but other colors may also be used. Furthermore, the number of markers is not important, and the markers encapsulated in the elastic body can be arranged in one or more positions on the surface of the elastic body, or in a grid pattern. Note that by providing multiple markers, the surface unevenness shape can be measured in more detail.

[0012] The pinhole body is a plate-like body with small holes (through holes) formed therein. The structure of the pinhole body and the small holes must be determined so that the pressure force can be detected and the captured image can be recognized effectively, and the shape can be determined to be optimal, such as a shape with a constant width in the depth direction of the hole, a shape that widens as it progresses in the depth direction, a polygonal shape, or a funnel-, step-, or wedge-shaped shape in side view.

[0013] The image sensor is a device for capturing and imaging the shape change caused by the pressure acting on the elastic body portion, and various optical detection devices can be used, such as a CMOS (Complementary Metal Oxide Semiconductor) sensor or a CCD (Charge Coupled Device) sensor.

[0014] While it is desirable for the image quality of the captured image by the image sensor to be as accurate as possible, in reality, this goal can be achieved by adopting an image quality accurate enough to achieve the sensing function appropriate for the application.This visual and tactile sensor is an invention that combines a pinhole body and an image sensor within various constraints to ensure image quality that is capable of detecting changes in the position of a marker and identifying the object being contacted.

[0015] As a standard for satisfying the above, for example, when fingerprint authentication is performed, the imaging performance must be sensitive enough to resolve the pattern created by the fine ridges on the inside of the fingertip, which is the contact object, and to recognize the pattern created by differences in the transparent surface color or gloss of the skin of the fingertip, which has little difference in surface color (fingerprint patterns created by differences in light reflection due to the unevenness of the fingertip surface).For example, when biometric authentication is performed by sensing invisible light, it is necessary to design the device as optimal for recognizing specific wavelengths (selection and design of the image sensor, design of the pinhole body (plate thickness, hole diameter, etc.), and the optical properties of the elastic body (refractive index, transmittance, shape, etc.), as well as the design of the following structure and specifications of this visual and tactile sensor).

[0016] To achieve the above, it is preferable that the dimensions of the main parts of this visual-tactile sensor, namely the working distance (L1) between the surface of the elastic body and the pinhole body, the distance (L2) between the image sensor and the pinhole body, and the distance (L3) between the pinhole body and the underside of the elastic body, each satisfy the following (see Figure 2). 14mm ≥ L1 ≥ 5mm 2 mm ≥ L2 ≥ 250 micrometers 2 mm ≥ L3

[0017] To make this tactile sensor thinner, the above (L1 + L2) must be as short as possible. The value of L1 is determined by the imaging area (size), imaging characteristics required such as resolution, and the amount of change in the elastic body (thickness and elastic modulus of the elastic body). After repeated prototyping of L1 while taking into consideration the required imaging characteristics, it became clear that the optimum upper limit for thinning the device was 14 mm. The lower limit for L1 was determined by the processing capabilities of this tactile sensor, and it was difficult to make it less than 5 mm.

[0018] It was also found that the optimum upper limit for L2 to thin this tactile sensor is 2 mm, taking into account the required imaging characteristics.The lower limit for L2 is the value when the pinhole body is in contact with the surface of the image sensor (a state in which the image sensor and pinhole body are in contact with each other with no gap), and considering the dimensions of the image sensor and the thickness of the pinhole body, it was found to be difficult to make the value less than 250 micrometers.

[0019] Furthermore, it was found that the optimum upper limit for L3 for thinning this tactile sensor was 2 mm, taking into account the required imaging characteristics. Note that there does not need to be a gap between the pinhole body and the underside of the elastic body.

[0020] This visual-tactile sensor detects position changes (changes in horizontal and vertical positions) based on captured image data of the marker and calculates horizontal and vertical pressures corresponding to the applied pressure by combining a pinhole body and an image sensor. This allows the lens thickness to be thinner than conventional tactile sensors, making it possible to significantly reduce the thickness of the entire device.

[0021] Furthermore, since the elastic body portion is translucent and transparent, it is possible to photograph the entire upper surface of the elastic body portion using an image sensor, making it possible to identify the contact object and grasp the approach state of the contact object, etc. [Effects of the Invention]

[0022] According to the present invention, it is possible to provide a visual-tactile sensor that has the functions of both a visual sensor and a tactile sensor, has a simple structure, and can be made thin. [Brief explanation of the drawings]

[0023] [Figure 1] 1 is a perspective view showing a visual-tactile sensor of the present invention. [Figure 2] 1 is a cross-sectional view showing a visual-tactile sensor of the present invention. [Figure 3] FIG. 2 is a block diagram showing a control unit of the visual-tactile sensor of the present invention. [Figure 4] (a) is a partially enlarged cross-sectional view of the visual-tactile sensor of the present invention before pressure is applied, and (b) is a partially enlarged view showing an image captured by the image sensor after pressure is applied (corresponding to a top view of the visual-tactile sensor). DETAILED DESCRIPTION OF THE INVENTION

[0024] Hereinafter, one embodiment of the visual-tactile sensor S will be described in detail with reference to the drawings. In the description of the drawings, the same elements are given the same reference numerals, and duplicated explanations will be omitted. Furthermore, each component can be modified as appropriate, and explanations of detailed structures that are not directly related to the present invention will be omitted.

[0025] [Configuration of this visual and tactile sensor] (1) Overall structure The visual-tactile sensor S includes a main body 10 and a control unit 20.

[0026] [Main body] The main body 10 includes a housing 11 with a convex opening at the top and an elastic body 12, and the main components are housed inside the housing 11 (FIGS. 1 and 2). A support plate 14 is provided on the upper surface of the housing 11. The support plate 14 is a component for fixing the position of the elastic body portion 12, and is required to have translucency and transparency, and an acrylic plate, a glass plate, or the like can be used.

[0027] A rectangular, transparent, gel-like elastic body having translucency is provided on the support plate 14, and this portion serves as the elastic body portion 12. A plurality of (four) markers 13 for detecting pressure are embedded at equal intervals inside the elastic body portion 12 (the number of markers 13 is determined for the convenience of explanation and is not limited). In this embodiment, the markers 13 are arranged in the elastic body portion 12 so as to be positioned at the center of a cube that is assumed to have the same shape. The arrangement of the markers 13 is not limited to the above, but may be arranged on the surface side of the elastic body portion 12, or in an irregular arrangement pattern, etc., so as to easily detect the contact status of the contact object F.

[0028] The interior of the housing 11 is divided into an upper chamber 11a and a lower chamber 11b by a step, and a CMOS sensor 16 (image sensor) is provided on the bottom surface of the hollow portion of the upper chamber 11a. A plate-shaped pinhole body 15 is provided with a predetermined gap between the support plate 14 and the CMOS sensor 16, and spaces are formed between the support plate 14 and the pinhole body 15, and between the pinhole body 15 and the CMOS sensor 16, to ensure a working distance (the distance from the small hole 15a to the contact object F), etc.

[0029] A small hole 15a is formed in the center of the pinhole body 15. The CMOS sensor 16 is capable of receiving light transmitted through the small hole 15a of the pinhole body 15, and is capable of capturing an image of the entire area of ​​the transparent elastic body part 12, including the marker 13, up to the surface part (the upper surface side of the elastic body part 12 (the surface side that is in contact with and pressurized by the contact object T)).

[0030] [Control Unit] The lower chamber 11b of the housing 11 accommodates an electronic circuit board, wiring, etc., and the control unit 20 is provided in this portion. The control unit 20 includes a position detection unit 21, a pressure calculation unit 22, and an image identification unit 23. The position detection unit 21 includes a horizontal position detection unit 21a and a vertical position detection unit 21b. The pressure calculation unit 22 includes a horizontal pressure calculation unit 22a and a vertical pressure calculation unit 22b (FIG. 3).

[0031] The horizontal position detection unit 21a is a means for detecting and outputting the relative horizontal displacement (displacement x1, y1 from the reference point) of a specific marker 13 on the plane coordinates (XY coordinates) from the data of the image actually captured by the CMOS sensor 16 (hereinafter referred to as "actual image data 5"), while the vertical position detection unit 21b is a means for detecting and outputting the vertical position coordinate (Z coordinate) from the change in size of the specific marker 13 in the actual image data 5 (the area ratio A1 / A0 between the reference area at the reference point and the area in the actual image data 5).

[0032] In addition, the horizontal pressure calculation unit 22a and the vertical pressure calculation unit 22b are means for calculating the horizontal pressure and the vertical pressure, respectively, for the elastic body used in the elastic body portion 12 based on a relational equation or a correspondence table between pressure force and displacement (which are different for the horizontal and vertical directions) that has been determined in advance based on a demonstration test or the like, and outputting the values. It should be noted that the control unit 20 may also use a method of calculating the horizontal pressure and the vertical pressure by performing machine learning using AI on the actual image data 5.

[0033] In addition, the image identification unit 23 can be configured to include a photographed image database that stores multiple photographed image data for comparison (hereinafter referred to as ``photographic image data for comparison''), and a contact object identification means (both not shown) that compares the actual image data 5 and the photographed image data for comparison to identify the contact object F. For example, assuming a case in which the elastic body part 12 is pressed by a human finger, which is the contact object F, it is possible to identify the subject who has come into contact with the elastic body part 12 by storing fingerprint image data associated with the subject's identification information in a database of photographed images for comparison.

[0034] The sense of slippage may be determined by continuously calculating the relative amount of movement in the horizontal direction in the control unit 20. With this configuration, when the visual-tactile sensor S is attached to a gripping device, it is possible to determine the amount of slippage of the contact object F (grasping object) and adjust the gripping force of the gripping device so that the amount of slippage becomes zero, thereby preventing the contact object from slipping off, etc.

[0035] [Operation of this visual and tactile sensor] Next, the operation of the visual and tactile sensor S will be described. When a contact object F (a finger is assumed in FIG. 4(a)) comes into contact with the surface of the elastic body portion 12 of the visual-tactile sensor S, pressure is applied to the surface of the elastic body portion 12, causing the elastic body portion 12 to deform. As a result, the position of the marker 13 embedded in the elastic body portion 12 moves depending on the magnitude and direction of the applied pressure (dotted lines in FIGS. 4(a) and 4(b)).

[0036] The position of the marker 13 at this time is photographed by the CMOS sensor 16, and the horizontal position detection unit 21a and vertical position detection unit 21b (position detection unit 21 in the control unit 20) detect the relative movement amount of the marker 13 from the actual image data 5. Then, based on the relative movement amount, the horizontal pressure calculation unit 22a and vertical pressure calculation unit 22b (pressure calculation unit 22 in the control unit 20) calculate the horizontal pressure and vertical pressure. Furthermore, the image identification unit 23 identifies the contact object F from the actual image data 5.

[0037] The output horizontal pressure and vertical pressure are output to a connected device (not shown) such as a gripping device, and are used to control the orientation and gripping force of the gripping device.

[0038] [Effects of this visual and tactile sensor] This visual-tactile sensor S detects position changes (changes in horizontal and vertical positions) based on actual image data 5 (photographed image data) of the marker 13, and calculates horizontal and vertical pressures corresponding to the pressure applied by the contact object F, by adopting a combination of a pinhole body 15 and a CMOS sensor 16. Therefore, the lens thickness can be made thinner than with conventional tactile sensors, and the entire device can be made thinner (for example, a cubic device with each side approximately 10 mm).

[0039] Furthermore, the elastic body portion 12 is translucent and transparent, and the CMOS sensor 16 can capture an image of the surface side of the elastic body portion 12, making it possible to identify the contact object F and grasp the approach state of the contact object F, etc.

[0040] In this way, with this visual and tactile sensor S, it is possible to simultaneously acquire image information of the contact object F and detect the applied pressure with a simple structure, thereby realizing a sensor that combines the functions of both conventional visual sensors and tactile sensors.

[0041] While one example of a preferred embodiment of the present invention has been described above, the present invention is not limited to this embodiment, and appropriate design modifications are possible within the scope of the spirit of the present invention. Furthermore, the above invention-specific matters define the minimum necessary components, and other components may be added as long as they do not impair the effects of the invention. In particular, with regard to each component of the audiovisual sensor described in the claims, various components can be used as long as they have the basic configuration and exhibit the same effects. [Explanation of symbols]

[0042] S Visual and tactile sensor F. Contact object 5 Actual image data 10 Main body 11. Housing 12 Elastic body part 13 Marker 14 Support plate 15 Pinhole body 15a small hole 16 CMOS sensor (image sensor) 20 Control Unit 21 Position detection unit 21a Horizontal position detection section 21b Vertical position detection unit 22 Pressure calculation unit 22a Horizontal pressure calculation section 22b Vertical pressure calculation section 23 Image Identification Unit

Claims

1. a transparent elastic portion that is deformed by pressure; one or more markers provided on the elastic body portion for detecting a pressure; a pinhole body having a small hole provided on the lower surface of the elastic body portion; an image sensor provided on the lower surface of the pinhole body and configured to receive light transmitted through the small hole; The image sensor is capable of capturing an image of the positional change of the marker, and is capable of calculating the vertical and horizontal pressures corresponding to the pressure force based on the positional change of the marker, and is capable of capturing an image of the entire upper surface of the elastic body portion.

2. 2. The visual-tactile sensor according to claim 1, wherein the elastic body portion is formed by laminating a plurality of layers having different elastic moduli.

Images