Tactile sensor system and program

Abstract

A tactile sensor is provided with: a transmission unit provided with a first surface capable of coming into contact with an object to be gripped, and a second surface which is the back surface of the first surface; an imaging unit capable of imaging, from the second surface side, an image of an object present on the first surface side of the transmissive unit; and a reflection unit that is disposed on the second surface side of the transmissive unit, reflects light from at least a partial region of the transmissive unit, and guides the light into an imaging angle of view of the imaging unit. According to the present invention, it is possible to provide a touch sensor system and a program that can be easily miniaturized.

Description

[0001] Related Application

[0002] This application is a divisional application of a Chinese patent application with application number 201980097957.3, filing date July 4, 2019, and invention title "Tactile Sensor, Tactile Sensor System, and Program". Technical Field

[0003] The present invention relates to a tactile sensor, a tactile sensor system, and a program. Background Art

[0004] Conventionally, as a method for realizing a tactile sensor, a method of detecting an object in contact with the outer surface of a rubber surface layer has been known (for example, refer to Patent Document 1).

[0005] Prior Art Documents

[0006] Patent Documents

[0007] Patent Document 1: Japanese Patent Application Laid-Open Publication No. 2000-288973 Summary of the Invention

[0008] Problems to be Solved by the Invention

[0009] In a conventional tactile sensor, the optical axis of the imaging unit is parallel to the normal direction of the rubber surface layer (hereinafter, referred to as the contact surface). The imaging unit images the contact surface and detects the holding state of the object by detecting the displacement of the contact surface when the object comes into contact with the contact surface.

[0010] Here, the size of the tactile sensor may sometimes be small, for example, when the object to be touched is small. When considering miniaturization of the tactile sensor, it becomes a problem to shorten the distance between the imaging unit for imaging the object and the contact surface and to miniaturize the imaging unit itself.

[0011] In a conventional tactile sensor, when it is necessary to shorten the distance between the imaging unit and the contact surface, a lens with a wide imaging field angle must be used. When using a lens with a wide imaging field angle, there are problems such as distortion and ensuring the amount of light. In addition, when it is necessary to miniaturize the imaging unit, since the miniaturization of the imaging unit is accompanied by a decrease in image sensitivity, there is also a limit, and it is impossible to make the tactile sensor smaller than the imaging unit. That is, when considering miniaturization of the tactile sensor, there are problems of difficulty in maintaining image quality by shortening the distance between the imaging unit and the contact surface and a limit in miniaturization of the imaging unit.

[0012] That is, according to the conventional method, there is a problem that miniaturization of the tactile sensor is not easy.

[0013] The present invention has been completed in view of such circumstances, and an object thereof is to provide a tactile sensor, a tactile sensor system, and a program that can be easily miniaturized.

[0014] Means for solving the problem

[0015] A tactile sensor according to one aspect of the present invention includes: a transmissive portion having a first surface that can come into contact with an object to be grasped and a second surface that is the back surface of the first surface; an imaging portion that can image an object existing on the first surface side of the transmissive portion from the second surface side; and a reflective portion that is disposed on the second surface side of the transmissive portion and reflects light from at least a part of the region of the transmissive portion and guides it into the imaging field angle of the imaging portion.

[0016] In addition, in the tactile sensor according to one aspect of the present invention, at least a part of the transmissive portion deforms along the shape of the object to be grasped that comes into contact with the first surface, and the imaging portion can image both an image of an object existing on the first surface side and an image of a mark indicating the deformation of the transmissive portion attached to the transmissive portion from the second surface side.

[0017] Further, in the tactile sensor according to one aspect of the present invention, the imaging portion is configured such that an imaging optical axis of the imaging portion intersects a normal line of the second surface of the transmissive portion.

[0018] Further, in the tactile sensor according to one aspect of the present invention, the reflective portion includes a plurality of reflecting surfaces having different angles of a normal line with respect to the imaging optical axis of the imaging portion.

[0019] Further, in the tactile sensor according to one aspect of the present invention, the imaging portion images both a first image and a second image as an image of the transmissive portion. The first image is an image of a photographed object region of the transmissive portion formed by light that enters without passing through the reflective portion, and the second image is an image of the photographed object region of the transmissive portion formed by light that is reflected by the reflective portion and enters.

[0020] Further, in the tactile according to one aspect of the present invention, the transmissive portion includes a plurality of regions having different angles of a normal line with respect to the imaging optical axis of the imaging portion, and the imaging portion can image an object existing on the first surface side formed by light that enters through the plurality of regions of the transmissive portion, respectively.

[0021] A tactile sensor system according to one aspect of the present invention includes: the above-described tactile sensor; and a detection portion that acquires an image photographed by the imaging portion and detects a contact state of an object with respect to the first surface based on the acquired image.

[0022] In addition, a program according to one aspect of the present invention executes, on a computer connected to a tactile sensor, an image acquisition step of acquiring an image captured by the imaging unit, and a detection step of detecting a contact state of an object with respect to the first surface based on the image acquired in the image acquisition step. The tactile sensor includes: a transmissive portion having a first surface that can contact a gripped object and a second surface that is the back surface of the first surface; an imaging unit that can image an object existing on the first surface side of the transmissive portion from the second surface side; and a reflection portion disposed on the second surface side of the transmissive portion that reflects light from at least a part of the area of the transmissive portion and guides it into the imaging field angle of the imaging unit.

[0023] Effects of the present invention

[0024] According to the present invention, it is possible to provide a tactile sensor, a tactile sensor system, and a program that can be easily miniaturized. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 FIG. is a diagram showing an example of a robot system in an embodiment.

[0026] Figure 2 FIG. is a diagram showing an example of a tactile sensor module in an embodiment.

[0027] Figure 3 FIG. is a diagram showing an example of a cross-sectional view of a tactile sensor in an embodiment.

[0028] Figure 4 FIG. is a diagram showing an example of the imaging range of an imaging unit in an embodiment.

[0029] Figure 5 FIG. is a diagram showing an example of an image captured by the imaging unit in an embodiment.

[0030] Figure 6 FIG. is a diagram showing an example of a cross-sectional view of a tactile sensor when a gripped object contacts a contact surface in an embodiment.

[0031] Figure 7 FIG. is a diagram showing an example of an image captured by a tactile sensor when a gripped object contacts a contact surface in an embodiment.

[0032] Figure 8 FIG. is a diagram showing an example of a control unit of a robot system in an embodiment.

[0033] Figure 9 FIG. is a diagram showing an example of the operation of a control unit of a robot system in an embodiment.

[0034] Figure 10This is a diagram showing an example of a cross-sectional view of the tactile sensor in the second embodiment.

[0035] Figure 11 This is a diagram showing an example of the shootable range of the shooting unit in the second embodiment.

[0036] Figure 12 This is a diagram showing an example of a cross-sectional view of the tactile sensor in the third embodiment.

[0037] Figure 13 This is a diagram showing an example of the shootable range of the shooting unit in the third embodiment.

[0038] Figure 14 This is a diagram showing an example of a cross-sectional view of the tactile sensor in the fourth embodiment.

[0039] Figure 15 This is a diagram showing an example of the shootable range of the shooting unit in the fourth embodiment.

[0040] Figure 16 This is a diagram showing an example of a cross-sectional view of the tactile sensor in the fifth embodiment.

[0041] Figure 17 This is a diagram showing an example of the shootable range of the shooting unit in the fifth embodiment. Detailed Embodiment

[0042] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0043] [Configuration of Robot System 100]

[0044] Figure 1 This is a diagram showing an example of the robot system 100 in the embodiment. The robot system 100 in this embodiment holds an object while detecting the holding state by contacting the object to be held.

[0045] In this embodiment, the robot system 100 includes a tactile sensor module 10, a robot system control unit 90, a front end portion 110, an upper arm portion 120, a joint portion 130, a lower arm portion 140, a main horizontal axis portion 150, a main vertical axis portion 160, and a base portion 170.

[0046] The base portion 170 is a portion connected to the main vertical axis portion 160.

[0047] The main vertical axis portion 160 is a portion connecting the main horizontal axis portion 150 and the base portion 170. The main vertical axis portion 160 is controlled by the robot system controller 90 to displace the main horizontal axis portion 150 around the main vertical axis portion 160.

[0048] The main horizontal axis portion 150 is the part that connects the lower arm portion 140 and the main vertical axis portion 160. The main horizontal axis portion 150 is controlled by the robot system control unit 90 to displace the lower arm portion 140 around the main horizontal axis portion 150.

[0049] The lower arm portion 140 is the part that connects the joint portion 130 and the main horizontal axis portion 150.

[0050] The joint portion 130 is the part that connects the upper arm portion 120 and the lower arm portion 140. The joint portion 130 is controlled by the robot system control unit 90 to displace the upper arm portion 120 around the joint portion 130.

[0051] The upper arm portion 120 is the part that connects the front end portion 110 and the joint portion 130.

[0052] The front end portion 110 is connected to the tactile sensor module 10. The posture (e.g., position and orientation) of the front end portion 110 is controlled by the robot system control unit 90. The posture of the tactile sensor module 10 changes as the posture of the front end portion 110 changes.

[0053] The tactile sensor module 10 detects the contact state of the object to be grasped and outputs information indicating the detected contact state of the object to be grasped to the robot system control unit 90.

[0054] The robot system control unit 90 acquires the information output by the tactile sensor module 10.

[0055] The robot system control unit 90 displaces each part (front end portion 110, upper arm portion 120, joint portion 130, lower arm portion 140, main horizontal axis portion 150, and main vertical axis portion 160) of the robot system 100 through a drive device (not shown), thereby moving the tactile sensor module 10. The robot system control unit 90 controls the robot system 100 based on the information acquired from the tactile sensor module 10.

[0056] Figure 2 This is a diagram showing an example of the tactile sensor module 10 in the embodiment. The tactile sensor module 10 in this embodiment includes a sensor connection portion 11, a first tactile sensor 1a, and a second tactile sensor 1b. In the following description, there are cases where the posture of the tactile sensor module 10 is represented by a three-dimensional rectangular coordinate system of the x-axis, y-axis, and z-axis.

[0057] The sensor connection portion 11 is the part that connects the front end portion 110 to the first tactile sensor 1a and the second tactile sensor 1b.

[0058] The first tactile sensor 1a is connected to the sensor connection portion 11. The first tactile sensor 1a includes a first transmissive portion contact surface 40a.

[0059] The second tactile sensor 1b is connected to the sensor connection part 11. The second tactile sensor 1b has a second transmissive part contact surface 40b.

[0060] The first tactile sensor 1a and the second tactile sensor 1b are arranged at positions where the first transmissive part contact surface 40a and the second transmissive part contact surface 40b face each other. The sensor connection part 11 has a drive device (not shown), and based on an instruction from the robot system control part 90, displaces the first tactile sensor 1a and the second tactile sensor 1b (or one of these sensors, the same in the following description) in the y-axis direction. The tactile sensor module 10 grips a gripped object located between the first tactile sensor 1a and the second tactile sensor 1b by driving the first tactile sensor 1a and the second tactile sensor 1b in the y-axis direction.

[0061] [Configuration of the tactile sensor 1]

[0062] Figure 3 This is a diagram showing an example of a cross-sectional view of the tactile sensor in the embodiment. In this diagram, it shows Figure 2 A cross-sectional view on the xy plane of the first tactile sensor 1a shown. The direction of the tactile sensor module 10 is represented by a three-dimensional rectangular coordinate system of the x-axis, y-axis, and z-axis.

[0063] In addition, since the first tactile sensor 1a and the second tactile sensor 1b have the same configuration, the first tactile sensor 1a will be described, and the description of the second tactile sensor 1b will be omitted.

[0064] The first tactile sensor 1a includes a first imaging unit 30a, a first reflection unit 20a, a first transmissive part 43a, a first marker 45a, a first transmissive part contact surface 40a, a first transmissive part non-contact surface 47a, a first hard layer 70a, and a first frame 50a.

[0065] In addition, hereinafter, in other words, the first imaging unit 30a will be described as the imaging unit 30, the first reflection unit 20a will be described as the reflection unit 20, the first transmissive part 43a will be described as the transmissive part 43, the first marker 45a will be described as the marker 45, the first transmissive part contact surface 40a will be described as the transmissive part contact surface 40, the first transmissive part non-contact surface 47a will be described as the transmissive part non-contact surface 47, the first hard layer 70a will be described as the hard layer 70, and the first frame 50a will be described as the frame 50.

[0066] The frame 50 holds the imaging unit 30, the reflection unit 20, the transmissive part 43, and the hard layer 70.

[0067] The transmissive part 43 is made of a transparent material that allows light to pass through, and has a transmissive part contact surface 40 and a transmissive part non-contact surface 47. As a specific material of the transmissive part 43, for example, there is a silicone material with a thickness of 2 mm and a transmittance of about 94%.

[0068] The transmissive part contact surface 40 refers to the surface and the back surface of the transmissive part 43 that can come into contact with the object to be held. The transmissive part non-contact surface 47 is the surface and the back surface of the transmissive part 43 that do not come into contact with the object to be held.

[0069] In addition, in the following description, the surface of the transmissive part 43 that can come into contact with the object to be held (i.e., the transmissive part contact surface 40) is also referred to as the front surface or the first surface, and the surface that does not come into contact with the object to be held (i.e., the transmissive part non-contact surface 47) is also referred to as the back surface or the second surface. That is, the transmissive part 43 has: the transmissive part contact surface 40 as the contact surface that can come into contact with the object to be held, and the transmissive part non-contact surface 47 as the non-contact surface that is the back surface of the contact surface and does not come into contact with the object to be held.

[0070] In addition, the transmissive part 43 is made of a transparent material. In this example, at least a part of the transmissive part 43 deforms along the shape of the object to be held that is in contact with the transmissive part contact surface 40 as the contact surface.

[0071] A plurality of markers 45 are arranged at predetermined positions of the transmissive part 43. In an example of the present embodiment, the marker 45 refers to an opaque member arranged at the positions of equally spaced grid points inside the transmissive part 43. In addition, the marker 45 is arranged inside the transmissive part 43, but is not limited thereto, and may also be provided on the transmissive part contact surface 40 or on the transmissive part non-contact surface 47. In addition, the case where the markers 45 are discretely arranged at the positions of the grid points is described, but is not limited thereto. The marker 45 may also be a grid pattern or other continuous pattern. In addition, the pattern of the marker 45 may also be an irregular pattern so as to easily detect the holding state of the object to be held. The marker 45 has been described as an opaque member, but is not limited thereto, and may also be a translucent member or a transparent member as long as the displacement when the object to be held comes into contact can be optically recognized.

[0072] The hard layer 70 is provided at a position in contact with the transmissive part non-contact surface 47 of the transmissive part 43. The hard layer 70 is made of a material such as transparent and hard acrylic resin. The hard layer 70 restricts the amount of deformation of the transmissive part 43 when the object to be held is held.

[0073] In addition, in the present embodiment, the transmissive part 43 and the hard layer 70 are described as independent components. As long as the amount of deformation of the transmissive part 43 when the object to be held is held is within a specified range, the hard layer 70 may be omitted.

[0074] The reflecting unit 20 includes a reflecting surface that reflects light, such as a mirror. This reflecting surface is disposed on the non-contact surface side of the transmissive unit 43. The reflecting unit 20 reflects the light transmitted through the transmissive unit 43 and guides the reflected light to the imaging unit 30.

[0075] The reflecting unit 20 reflects light from at least a part of the area of the transmissive unit 43. That is, the reflecting unit 20 is disposed on the non-contact surface (e.g., the non-contact surface 47 of the transmissive unit) side of the transmissive unit 43, reflects light from at least a part of the area of the transmissive unit 43, and guides it into the imaging field angle of the imaging unit 30.

[0076] The imaging unit 30 is disposed on the non-contact surface 47 side of the transmissive unit 43 among the front and back surfaces of the transmissive unit 43. More specifically, the imaging unit 30 is arranged such that the imaging optical axis OA of the imaging unit 30 and the normal line N41 of the non-contact surface 47 of the transmissive unit 43 have an intersection point (the imaging optical axis OA and the normal line N41 are not parallel).

[0077] The imaging unit 30 captures an image within the imaging field angle centered on the imaging optical axis OA and outputs the captured result as image information. The imaging unit 30 can capture an image of an object existing on the contact surface 40 side of the transmissive unit 43 from the non-contact surface 47 side of the transmissive unit 43.

[0078] Here, among the light transmitted through the transmissive unit 43, an image of an object existing on the contact surface 40 side of the transmissive unit 43 is included. In addition, among the light transmitted through the transmissive unit 43, an image of the mark 45 disposed on the transmissive unit 43 (i.e., an image of the transmissive unit 43 or an image of the transmissive part) is included. That is, the imaging unit 30 can capture both an image of an object existing on the contact surface 40 side of the transmissive unit 43, which is the contact surface side, and an image of a mark indicating the deformation of the transmissive unit 43 attached to the transmissive unit 43, from the non-contact surface 47 side of the transmissive unit 43, which is the non-contact surface side.

[0079] In addition, within the shootable range of the imaging unit 30, an image formed by the light transmitted through the transmissive unit 43 and reflected by the reflecting unit 20, and an image formed by the light that passes through the transmissive unit 43 and directly reaches the imaging unit 30 without passing through the reflecting unit 20 are included. In the following description, an image formed by the light that passes through the transmissive unit 43 and directly reaches the imaging unit 30 without passing through the reflecting unit 20 is also referred to as a direct image. In addition, an image formed by the light transmitted through the transmissive unit 43 and reflected by the reflecting unit 20 is also referred to as a reflected image. Refer to Figure 4 and Figure 5 A description will be given of the shootable range of the imaging unit 30.

[0080] [Shootable Range of Imaging Unit 30]

[0081] Figure 4This is a diagram showing an example of the shootable range of the shooting unit 30 in the embodiment. The shootable range of the shooting unit 30 included in the tactile sensor 1 will be described.

[0082] In this example, the shootable range of the shooting unit 30 is determined by the geometric relative relationship between the viewing angle A10 of the shooting unit 30 and the arrangement of the reflecting unit 20. The shootable range of the shooting unit 30 includes the area where a direct image can be shot and the area where a reflected image can be shot.

[0083] For example, when the incident angle of the light incident on the reflecting unit 20 from the transmissive part contact surface 40 side is the first incident angle IA10, the light incident on the reflecting unit 20 is emitted in the direction of the first reflection angle RA10.

[0084] When the incident angle of the light incident on the reflecting unit 20 from the transmissive part contact surface 40 side is the second incident angle IA20, the light incident on the reflecting unit 20 is emitted in the direction of the second reflection angle RA20.

[0085] In addition, when the incident angle of the light incident on the reflecting unit 20 from the transmissive part contact surface 40 side is the third incident angle IA30, the light incident on the reflecting unit 20 is emitted in the direction of the third reflection angle RA30 (in the case of this example, the shooting optical axis OA).

[0086] When the image (i.e., the reflected image) formed by the light emitted from the reflecting unit 20 is included within the viewing angle A10, the shooting unit 30 can shoot the light emitted from the reflecting unit 20.

[0087] The first shooting range AR1 is the range where the shooting unit 30 can shoot a direct image and is the range where the shooting unit 30 cannot shoot a reflected image.

[0088] The second shooting range AR2 is the range where the shooting unit 30 can shoot both a direct image and a reflected image.

[0089] The third shooting range AR3 is the range where the shooting unit 30 cannot shoot a direct image and is the range where the shooting unit 30 can shoot a reflected image.

[0090] Figure 5 This is a diagram showing an example of the captured image of the shooting unit 30 in the embodiment. The captured image P includes the directly-viewed captured image R and the reflected-viewed captured image M as the constituent elements of the image. The captured image P is represented by a two-dimensional orthogonal coordinate system of the x c axis and the y c axis. The x c y c plane represents the image plane of the yz plane captured in Figure 4 .

[0091] The direct-view photographing mark RM is an image obtained by photographing the direct image of the mark 45 by the photographing unit 30.

[0092] The reflection-view photographing mark MM is an image obtained by photographing the reflected image of the mark 45 by the photographing unit 30.

[0093] Hereinafter, with reference to Figure 4 and Figure 5 , the first object OB1 to the third object OB3 will be taken as an example for explanation.

[0094] In this example, the first object OB1 exists in the first photographing range AR1. In this case, the photographing unit 30 can photograph the direct image of the first object OB1, but cannot photograph the reflected image of the first object OB1.

[0095] The second object OB2 exists in the second photographing range AR2. In this case, the photographing unit 30 can photograph both the direct image and the reflected image of the second object OB2.

[0096] The third object OB3 exists in the third photographing range AR3. In this case, the photographing unit 30 cannot photograph the direct image of the third object OB3, but can photograph the reflected image of the third object OB3.

[0097] The photographing unit 30 photographs both the first image and the second image as images of the transmissive part 43, where the first image is an image of the photographing target area of the transmissive part 43 formed by light incident without passing through the reflecting part 20, and the second image is an image of the photographing target area of the transmissive part 43 formed by light reflected by the reflecting part 20 and then incident.

[0098] That is, the photographing unit 30 can photograph the direct-view photographing image R and the reflection-view photographing image M simultaneously.

[0099] [Case where the object to be gripped contacts the contact surface of the transmissive part]

[0100] Figure 6 This is a diagram showing an example of a cross-sectional view of the tactile sensor when the object to be gripped contacts the contact surface in the embodiment. As an example, the case where the object OB4 contacts the contact surface 40 of the transmissive part will be described.

[0101] In this example, the object OB4 as the object to be gripped contacts the contact surface 40 of the transmissive part. The range where the object OB4 contacts the contact surface 40 of the transmissive part is taken as the object detection range ODA. Before and after the object contacts the contact surface 40 of the transmissive part, the mark 45 located within the object detection range ODA is displaced. The photographing unit 30 photographs the mark 45 in the object detection range ODA in time series.

[0102] In addition, in this example, the object OB4 is located at a position straddling the second shooting range AR2 and the third shooting range AR3. Therefore, both direct vision and reflected vision are possible.

[0103] Figure 7 It is a diagram showing an example of a captured image of a tactile sensor when the object to be grasped in the embodiment comes into contact with the contact surface. The captured image P includes a direct vision captured image R and a reflected vision captured image M as constituent elements of the image. In this diagram, the captured image P when the object OB4 comes into contact with the contact surface 40 of the transmissive portion is shown.

[0104] In the direct vision captured image R, a direct vision capture mark RM is captured. Here, the direct vision object detection range RODA is the range within the object detection range ODA that is captured as the direct vision captured image R.

[0105] In the reflected vision captured image M, a reflected vision capture mark MM is captured. Here, the reflected vision object detection range MODA is the range within the object detection range ODA that is captured as the reflected vision captured image M.

[0106] Here, for the direct image of the mark 45, the position of the mark 45 before the object OB4 comes into contact and the position of the mark 45 after the object OB4 comes into contact are compared and described.

[0107] The pre-contact direct vision mark image RMB is the direct vision capture mark RM in the direct vision object detection range RODA before the object OB4 comes into contact with the contact surface 40 of the transmissive portion.

[0108] The post-contact direct vision mark image RMA is the direct vision capture mark RM in the direct vision object detection range RODA after the object OB4 comes into contact with the contact surface 40 of the transmissive portion.

[0109] Here, the difference in the position of the mark 45 within the image due to time change is represented by a mark vector. As shown in the figure, a difference in the position within the image occurs between the pre-contact direct vision mark image RMB and the post-contact direct vision mark image RMA. The direct vision mark vector RAR represents the difference between the pre-contact direct vision mark image RMB and the post-contact direct vision mark image RMA. The robot system 100 can detect the grasping state of the object OB4 by obtaining the direct vision mark vector RAR.

[0110] In addition, similar to the direct image, the reflected image of the marker 45 can also be represented by a marker vector to indicate the difference in the position of the marker 45 within the image due to temporal changes. That is, as shown in the figure, there is a difference in the position within the image between the pre-contact reflected-view marker image MMB and the post-contact reflected-view marker image MMA. The reflected-view marker vector MAR represents the difference between the pre-contact reflected-view marker image MMB and the post-contact reflected-view marker image MMA. The robot system 100 can detect the grasping state of the object OB4 by obtaining the reflected-view marker vector MAR.

[0111] [Case where the object to be grasped is in contact with the contact surface 40 of the transmissive portion]

[0112] Figure 8 It is a diagram showing an example of the robot system control unit 90 in the embodiment.

[0113] The robot system control unit 90 includes a robot control unit 91, an input unit 92, an output unit 93, and a grasping state detection unit 80.

[0114] The robot control unit 91 includes a microcomputer (not shown), a memory such as a RAM (Random Access Memory) and a ROM (Read Only Memory) (not shown), and a communication unit for communicating with external devices (not shown), etc.

[0115] The input unit 92 obtains information from sensors such as a pressure sensor, a position sensor, a temperature sensor, and an acceleration sensor, a camera, a microphone (all not shown), etc.

[0116] The output unit 93 outputs a drive signal to a motor for driving the robot (not shown) etc. that are not shown.

[0117] The grasping state detection unit 80 includes an image acquisition unit 81, an image processing unit 82, a control unit 83, and a reference state storage unit 84.

[0118] In the robot system 100, the grasping state detection unit 80, which is a detection unit, acquires the image captured by the imaging unit 30, and detects the contact state of the object with respect to the contact surface 40 of the transmissive portion based on the acquired image.

[0119] The grasping state detection unit 80 provides the detected grasping state to the robot control unit 91.

[0120] The image acquisition unit 81 acquires the image information captured by the imaging unit 30. The image acquisition unit 81 provides the image acquired by the imaging unit 30 to the image processing unit 82.

[0121] In addition, in this example, it is assumed that the image acquired by the imaging unit 30 is a still image for the description, but the information acquired by the imaging unit 30 may also be a moving image.

[0122] The image processing unit 82 acquires an image from the image acquisition unit 81. The image processing unit 82 processes the acquired image to detect the position of the detection marker 45.

[0123] The reference state storage unit 84 stores the position information of the marker 45 in the state where no object is detected, that is, the reference position information. That is, the reference state storage unit 84 stores the reference position information of the direct-view imaging marker RM in which the marker 45 is imaged in the direct-view imaging image R, and the reference position information of the reflected-view imaging marker MM in which the marker 45 is imaged in the reflected-view imaging image M.

[0124] The control unit 83 acquires the reference position information of the direct-view imaging marker RM, the reference position information of the reflected-view imaging marker MM, and the detection result of the position of the marker 45 from the image processing unit 82. The detection result of the position of the marker 45 includes the position information of the direct-view imaging marker RM and the position information of the reflected-view imaging marker MM.

[0125] In addition, the control unit 83 acquires the reference position information of the direct-view imaging marker RM and the reference position information of the reflected-view imaging marker MM from the reference state storage unit 84. The control unit 83 calculates the displacement of the direct-view imaging marker RM (for example, the direct-view marker vector RAR) based on the position information of the direct-view imaging marker RM represented by the captured image P and the reference position information of the direct-view imaging marker RM acquired from the reference state storage unit 84. In addition, the control unit 83 calculates the displacement of the reflected-view imaging marker MM (for example, the reflected-view marker vector MAR) based on the position information of the reflected-view imaging marker MM represented by the captured image P and the reference position information of the reflected-view imaging marker MM acquired from the reference state storage unit 84.

[0126] The control unit 83 outputs the displacement information of the marker 45 to the robot control unit 91. The displacement information of the marker 45 indicates the gripping state of the gripped object. That is, the control unit 83 detects the gripping state of the gripped object.

[0127] In addition, the control unit 83 may determine that the object is in contact with the contact surface 40 of the transmission unit when the displacement amount of the marker 45 exceeds a predetermined value.

[0128] In addition, the control unit 83 may also be configured such that the image acquisition unit 81 outputs the image captured by the imaging unit 30 to the robot control unit 91. In the image captured by the imaging unit 30, an image of an object existing on the contact surface 40 side of the transmissive portion 43 is included. That is, in the image captured by the imaging unit 30, an image of the external situation that can be observed through the transmissive portion 43 is included. According to the gripping state detection unit 80 configured in this way, the robot control unit 91 can grasp the situation around the contact surface 40 of the transmissive portion regardless of whether the object is in contact with the contact surface 40 of the transmissive portion.

[0129] Figure 9 FIG. is an example showing the operation of the robot system control unit 90 in the embodiment. Refer to Figure 9 An example of the operation of the robot system control unit 90 will be described.

[0130] (Step S10) The image acquisition unit 81 acquires the image information captured by the imaging unit 30. The image acquisition unit 81 provides the image acquired by the imaging unit 30 to the image processing unit 82.

[0131] (Step S20) The image processing unit 82 acquires the image from the image acquisition unit 81. The image processing unit 82 processes the acquired image. The image processing unit 82 identifies the range of the direct view captured image R and the range of the reflected view captured image M of the captured image P. The image processing unit 82 provides the position information of the direct view capture mark RM existing within the range of the direct view captured image R, the position information of the reflected view capture mark MM existing within the range of the reflected view captured image M, and the captured image P to the control unit 83. The control unit 83 acquires the position information of the direct view capture mark RM, the position information of the reflected view capture mark MM, and the captured image P from the image processing unit 82. In addition, the control unit 83 acquires the position information of the direct view capture mark RM and the position information of the reflected view capture mark MM in a state where no object is detected from the reference state storage unit 84. The control unit 83 compares the position information of the direct view capture mark RM and the position information of the reflected view capture mark MM acquired from the image processing unit 82 with the position information of the direct view capture mark RM and the position information of the reflected view capture mark MM acquired from the reference state storage unit 84.

[0132] (Step S30) When there is a difference in the comparison result (Step S30: Yes), the control unit 83 determines that the object is in contact with the contact surface 40 of the transmissive portion, and the process proceeds to Step S40. When there is no difference in the comparison result (Step S30: No), the control unit 83 determines that the object is not in contact with the contact surface 40 of the transmissive portion, and the process returns to Step S10.

[0133] (Step S40) The control unit 83 notifies the robot control unit 91 of the grasping state. Specifically, the control unit 83 notifies the robot control unit 91 of the displacement information of the direct-view shooting mark RM and the reflected-view shooting mark MM. In addition, the control unit 83 simultaneously provides the robot control unit 91 with the captured image P at the time when the displacement is detected.

[0134] [Second Embodiment]

[0135] Figure 10 It is a diagram showing an example of a cross-sectional view of the tactile sensor in the second embodiment.

[0136] In the above-described embodiment, it has been described that the reflecting portion 20 is a single plane. In the second embodiment, it is different from the above-described embodiment in that the reflecting portion 20 has a plurality of different angles.

[0137] In the second embodiment, the tactile sensor 1 includes a plurality of reflecting portions 20 having different angles. In this example, the tactile sensor 1 includes a first-angle reflecting portion 21 and a second-angle reflecting portion 22. (Hereinafter, in this embodiment, without distinguishing between the first-angle reflecting portion 21 and the second-angle reflecting portion 22, it is the reflecting portion 20)

[0138] The normal line N21 is the normal line of the first-angle reflecting portion 21.

[0139] The normal line N22 is the normal line of the second-angle reflecting portion 22.

[0140] Here, the normal line N21 and the normal line N22 intersect at the intersection point IP. That is, the tactile sensor 1 includes a plurality of reflecting surfaces having different angles of the normal line with respect to the shooting optical axis OA of the shooting unit 30.

[0141] In the second embodiment, since the tactile sensor 1 has a plurality of reflecting portions 20 having different angles, even when the shooting unit 30 has the same field angle A10 as in the first embodiment, a wider range can be observed.

[0142] In addition, in this example, the reflecting portion 20 is composed of a plurality of reflecting portions (the first-angle reflecting portion 21 and the second-angle reflecting portion 22) having different angles. These reflecting portions may be respectively composed of different reflecting members. Alternatively, a reflecting portion having the same effect may be formed by forming a plurality of reflecting portions having different angles on one reflecting member.

[0143] Figure 11This is a diagram showing an example of the photographable range of the photographing unit 30 in the second embodiment. In the second embodiment, the photographed image P includes a directly-viewed photographed image R as a constituent element of the image, a reflected-viewed photographed image M1 corresponding to the first-angle reflection unit 21, and a reflected-viewed photographed image M2 corresponding to the second-angle reflection unit 22. (Hereinafter, in this embodiment, without distinguishing between the reflected-viewed photographed image M1 corresponding to the first-angle reflection unit 21 and the reflected-viewed photographed image M2 corresponding to the second-angle reflection unit 22, it is referred to as the reflected-viewed photographed image M)

[0144] Similar to the first embodiment, a directly-viewed photographing mark RM is photographed in the directly-viewed photographed image R. In addition, a reflected-viewed photographing mark MM is photographed in the reflected-viewed photographed image M.

[0145] The photographing unit 30 detects the gripping state by observing the directly-viewed photographing mark RM and the reflected-viewed photographing mark MM.

[0146] [Third Embodiment]

[0147] Figure 12 This is a diagram showing an example of a cross-sectional view of the tactile sensor in Embodiment 3.

[0148] In the above embodiment, it has been described that the through-part contact surface 40 is a plane. In the third embodiment, it is different from the above embodiment in that the through-part contact surface 40 has a plurality of different angles (that is, has a curved surface).

[0149] In the third embodiment, the tactile sensor 1 includes a plurality of transparent part contact surfaces 40 having different angles. In this example, the tactile sensor 1 includes a first-angle through-part contact surface 41 and a second-angle through-part contact surface 42. (Hereinafter, in this embodiment, without distinguishing between the first-angle through-part contact surface 41 and the second-angle through-part contact surface 42, it is referred to as the through-part contact surface 40)

[0150] The normal line N41 is the normal line of the first-angle through-part contact surface 41.

[0151] The normal line N42 is the normal line of the second-angle through-part contact surface 42.

[0152] Here, the normal lines N41 and N42 intersect at the intersection point IP. That is, the through part 43 includes a plurality of regions where the angles of the normal lines with respect to the photographing optical axis OA of the photographing unit 30 are different. The photographing unit 30 can photograph the images of the objects existing on the side of the through-part contact surface 40 formed by the light incident through the plurality of regions of the through part 43 respectively.

[0153] In the third embodiment, the tactile sensor 1 has a plurality of transmissive part contact surfaces 40 with different angles from each other. Thus, even when the imaging unit 30 has the same field of view angle A10 as in the first embodiment, a wider range can be observed. In particular, in the third embodiment, by providing the second-angle transmissive part contact surface 42, a wider range of observation can be performed with respect to the x-axis direction. Therefore, when the robot system control unit 90 moves the position of the tactile sensor module 10 in the x-axis direction, collisions with objects located in the traveling direction can be detected in advance from the images captured by the imaging unit 30 provided in the tactile sensor 1.

[0154] That is, in the third embodiment, since the tactile sensor 1 is provided with the second-angle transmissive part contact surface 42, the tactile sensor 1 detects an object existing in the forward direction. Thus, the tactile sensor 1 can avoid collisions with objects existing in the traveling direction.

[0155] Figure 13 FIG. is an example showing the shootable range of the imaging unit 30 in the third embodiment. In the third embodiment, the captured image P includes a direct-view captured image R1 corresponding to the first-angle transmissive part contact surface 41, a direct-view captured image R2 corresponding to the second-angle transmissive part contact surface 42, and a reflected-view captured image M as constituent elements of the image. (Hereinafter, in the present embodiment, when not distinguishing between the direct-view captured image R1 corresponding to the first-angle transmissive part contact surface 41 and the direct-view captured image R2 corresponding to the second-angle transmissive part contact surface 42, it is the direct-view captured image R)

[0156] Similar to the first embodiment, a direct-view capture mark RM is captured in the direct-view captured image R. In addition, a reflected-view capture mark MM is captured in the reflected-view captured image M.

[0157] The imaging unit 30 detects the gripping state by observing the direct-view capture mark RM and the reflected-view capture mark MM.

[0158] [Fourth Embodiment]

[0159] Figure 14 FIG. is an example of a cross-sectional view showing the tactile sensor in Embodiment 4.

[0160] In the above-described second embodiment, an embodiment in which the reflection part 20 has a plurality of different angles has been described. In addition, in the above-described third embodiment, an embodiment in which the transmissive part contact surface 40 has a plurality of different angles has been described. In the fourth embodiment, it is different from the above-described embodiments in that the reflection part 20 has a plurality of different angles and the transmissive part contact surface 40 has a plurality of different angles.

[0161] In the fourth embodiment, the tactile sensor 1 includes a plurality of reflecting portions 20 having different angles. In addition, the tactile sensor 1 includes a plurality of transmissive portion contact surfaces 40 having different angles. In this example, the tactile sensor 1 includes a first-angle reflecting portion 21, a second-angle reflecting portion 22, a first-angle transmissive portion contact surface 41, and a second-angle transmissive portion contact surface 42. (Hereinafter, in this embodiment, when the first-angle reflecting portion 21 and the second-angle reflecting portion 22 are not distinguished, they are referred to as the reflecting portion 20. In addition, when the first-angle transmissive portion contact surface 41 and the second-angle transmissive portion contact surface 42 are not distinguished, they are referred to as the transmissive portion contact surface 40.)

[0162] In the fourth embodiment, the tactile sensor 1 has a plurality of reflecting portions 20 with different angles. Thus, even when the imaging unit 30 has the same field of view angle A10 as in the first embodiment, a wider range can be observed. In addition, in this example, the tactile sensor 1 has a plurality of transmissive portion contact surfaces 40 with different angles. Thus, even when the imaging unit 30 has the same field of view angle A10 as in the first embodiment, a wider range can be observed. Particularly in the fourth embodiment, by including the second-angle transmissive portion contact surface 42, observation can be made even in the x-axis direction. Therefore, when the robot system control unit 90 moves the position of the tactile sensor module 10 in the x-axis direction, a collision with an object located in the traveling direction can be detected in advance from the image captured by the imaging unit 30 included in the tactile sensor 1.

[0163] Figure 15 FIG. is an example showing the shootable range of the imaging unit 30 in the fourth embodiment. In the fourth embodiment, the captured image P includes a direct-view captured image R1 corresponding to the first-angle transmissive portion contact surface 41, a direct-view captured image R2 corresponding to the second-angle transmissive portion contact surface 42, a reflected-view captured image M1 corresponding to the first-angle reflecting portion 21, and a reflected-view captured image M2 corresponding to the second-angle reflecting portion 22 as the constituent elements of the image. (Hereinafter, in this embodiment, when the direct-view captured image R1 corresponding to the first-angle transmissive portion contact surface 41 and the direct-view captured image R2 corresponding to the second-angle transmissive portion contact surface 42 are not distinguished, they are referred to as the direct-view captured image R. In addition, when the reflected-view captured image M1 corresponding to the first-angle reflecting portion 21 and the reflected-view captured image M2 corresponding to the second-angle reflecting portion 22 are not distinguished, they are referred to as the reflected-view captured image M.)

[0164] Similar to the first embodiment, a direct-view capture mark RM is captured in the direct-view captured image R. In addition, a reflected-view capture mark MM is captured in the reflected-view captured image M.

[0165] The photographing unit 30 detects the gripping state by observing the direct-view photographing mark RM and the reflection-view photographing mark MM.

[0166] [Fifth Embodiment]

[0167] Figure 16 FIG. is an example of a cross-sectional view showing a tactile sensor in the fifth embodiment.

[0168] In the above-described embodiment, the through-section contact surface 40 is described as being planar. In addition, in the above-described third and fourth embodiments, embodiments in which the through-section contact surface 40 has a plurality of different angles are described. In particular, although an embodiment having a plurality of angles only in the x-axis direction is described, the fifth embodiment is different from the third and fourth embodiments in that it also has a plurality of angles in the z-axis direction.

[0169] In the fifth embodiment, the tactile sensor 1 includes a plurality of transparent-section contact surfaces 40 having different angles. Here, in the third and fourth embodiments, it is described that the through-section contact surface 40 has a plurality of different angles with respect to the x-axis direction. In the fifth embodiment, the through-section contact surface 40 also has a plurality of different angles with respect to the z-axis direction.

[0170] The tactile sensor 1 includes a plurality of reflecting sections 20. In this example, the tactile sensor 1 includes a first-angle reflecting section 21 and a second-angle reflecting section 22. (Hereinafter, in this embodiment, when not distinguishing between the first-angle reflecting section 21 and the second-angle reflecting section 22, they are referred to as the reflecting section 20)

[0171] The tactile sensor 1 includes a plurality of through-section contact surfaces 40 having different angles. In this example, the tactile sensor 1 includes a first-angle through-section contact surface 41 and a second-angle through-section contact surface 42.

[0172] In addition, the first-angle through-section contact surface 41 has a first-angle through-section contact surface 41S having different angles in the z-axis direction. The second-angle through-section contact surface 42 has a second-angle through-section contact surface 42S having different angles in the z-axis direction. (Hereinafter, in this embodiment, when not distinguishing between the first-angle through-section contact surface 41, the second-angle through-section contact surface 42, the first-angle through-section contact surface 41S, and the second-angle through-section contact surface 42S, they are referred to as the through-section contact surface 40)

[0173] In the fifth embodiment, the tactile sensor 1 can observe a wider range even with respect to the z-axis direction by having a transmissive contact surface 40S with a plurality of different angles with respect to the z-axis direction. When the robot system control unit 90 moves the position of the tactile sensor module 10 in the z-axis direction, collisions with objects located in the traveling direction can be detected in advance based on the images captured by the imaging unit 30 provided in the tactile sensor 1.

[0174] That is, in the fifth embodiment, the tactile sensor 1 can avoid collisions.

[0175] Figure 17 FIG. shows an example of the imaging range of the imaging unit 30 in the fifth embodiment. In the fifth embodiment, the captured image P includes a direct-view captured image R1 corresponding to the first-angle transmissive contact surface 41, a direct-view captured image RS1 corresponding to the first-angle transmissive contact surface 41S, a direct-view captured image R2 corresponding to the second-angle transmissive contact surface 42, a direct-view captured image RS2 corresponding to the second-angle transmissive contact surface 42S, a reflected-view captured image M1 corresponding to the first-angle reflecting portion 21, and a reflected-view captured image M2 corresponding to the second-angle reflecting portion 22. (Hereinafter, in this embodiment, when not distinguishing between the direct-view captured images R1, R2, RS1, and RS2, they are collectively referred to as the direct-view captured image R. Also, when not distinguishing between the reflected-view captured image M1 corresponding to the first-angle reflecting portion 21 and the reflected-view captured image M2 corresponding to the second-angle reflecting portion 22, they are collectively referred to as the reflected-view captured image M.)

[0176] The reflected-view captured image M1 has MS1 that reflects the transmissive contact surface 40S.

[0177] The reflected-view captured image M2 has MS2 that reflects the transmissive contact surface 40S.

[0178] Similar to the first embodiment, in the direct-view captured image R, a direct-view capture mark RM is captured. Also, in the reflected-view captured image M, a reflected-view capture mark MM is captured.

[0179] The imaging unit 30 detects the gripping state by observing the direct-view capture mark RM and the reflected-view capture mark MM.

[0180] In addition, the above-described tactile sensor module 10 and the gripping state detection unit 80 are collectively referred to as a tactile sensor system.

[0181] [Summary of the effects of the embodiment]

[0182] As described above, the tactile sensor 1 of the present embodiment can detect the gripping state by observing the transparent transmission part 43 that deforms along the shape of the gripped object in contact with the contact surface. The tactile sensor 1 includes a reflection part 20, and by observing the direct-view captured image R and the reflection-view captured image M, it is possible to observe the range outside the field of view angle A10 of the imaging part 30.

[0183] Here, according to an example of a conventional tactile sensor, the imaging part is arranged perpendicular to the transmission part, and in addition, there is no reflection part. Therefore, in order to miniaturize the tactile sensor, it is necessary to use an imaging part with a wide field of view to shorten the distance between the imaging part and the transmission part, or to miniaturize the imaging part itself. If the field of view of the imaging part is made wider or the imaging part is miniaturized, there are problems that the imaging quality deteriorates and the detection accuracy of the object gripping state decreases. That is, in the conventional method, there is a problem that it is not easy to miniaturize.

[0184] According to the tactile sensor 1 of the present embodiment, in addition to directly observing the transmission part 43, the transmission part 43 can also be observed through the reflection part 20. Therefore, it is not necessary to arrange the imaging part at a position where the transmission part 43 is imaged by direct view (a position where the optical axis of the imaging light of the imaging part is parallel to the normal direction of the transmission part). In addition, according to the tactile sensor 1 of the present embodiment, since, in addition to directly observing the transmission part 43, a method of observing the transmission part 43 through the reflection part 20 is also used, it is possible to perform imaging in a wider range. Therefore, the tactile sensor 1 can be configured without widening the field of view of the imaging part 30 or miniaturizing the imaging part 30.

[0185] That is, according to the tactile sensor 1 of the present embodiment, miniaturization can be easily achieved.

[0186] In addition, according to another example of a conventional tactile sensor, there is a tactile sensor that does not have a transmission part 43 and the imaging part cannot image the gripped object. According to this other example of the conventional tactile sensor, there is a problem that even if the gripping state of the gripped object slides, the imaging part cannot detect it.

[0187] Since the tactile sensor 1 of the present embodiment includes a transmission part 43, the imaging part 30 can detect that the gripping state of the gripped object has slid by directly observing the gripped object.

[0188] That is, according to the tactile sensor 1 of the present embodiment, the imaging part 30 can perform imaging in a wider range by observing the transmission part 43 through the reflection part 20 in addition to directly observing the transmission part 43. Furthermore, the imaging part 30 can detect that the gripping state of the gripped object has slid by directly observing the gripped object.

[0189] In addition, according to the above-described embodiments, the imaging unit 30 is configured such that the optical axis OA of the imaging unit 30 and the normal line of the non-contact surface 47 of the transmissive unit 43 have an intersection point.

[0190] In the prior art, the imaging unit 30 is disposed at a position where the optical axis OA of the imaging unit 30 is parallel to the normal line of the non-contact surface 47 of the transmissive unit 43. Therefore, the size of the tactile sensor 1 depends on the size of the imaging unit 30.

[0191] However, according to the above-described embodiments, the imaging unit 30 can be arranged such that the optical axis OA of the imaging unit 30 and the normal line of the non-contact surface 47 of the transmissive unit 43 have an intersection point.

[0192] That is, according to the tactile sensor 1 of the present embodiment, miniaturization can be easily achieved.

[0193] In addition, according to the above-described embodiments, the reflecting unit 20 includes a plurality of reflecting surfaces having different normal line angles. Therefore, the imaging unit 30 provides a field of view with a reduced range that can be intentionally observed. The tactile sensor 1 observes the grasped object through the plurality of reflecting surfaces, and can limit the range in which the grasped object exists in the three-dimensional space.

[0194] Therefore, the tactile sensor 1 can determine that the grasped object exists in a further limited three-dimensional space, and thus can detect a more accurate grasping state. In addition, by correctly grasping the three-dimensional space with the tactile sensor 1, when the robot system 100 drives the tactile sensor 1 to approach the grasped object, the grasped object can be detected more quickly.

[0195] In addition, according to the above-described embodiments, the transmissive unit 43 includes a plurality of regions having different angles of the normal line with respect to the optical axis OA of the imaging unit 30. The imaging unit 30 can respectively capture images of objects existing on the side of the contact surface 40 of the transmissive unit that are incident through the plurality of regions of the transmissive unit 43. The tactile sensor 1 can more accurately grasp the grasped object by observing the grasped object using direct vision and reflected vision, respectively.

[0196] That is, according to the above-described embodiments, the tactile sensor 1 can detect a more correct grasping state.

[0197] In addition, according to the above-described embodiments, the transmissive unit 43 includes a plurality of regions having different angles of the normal line with respect to the optical axis OA of the imaging unit 30. The imaging unit 30 can respectively capture images of objects existing on the side of the contact surface 40 of the transmissive unit that are incident through the plurality of regions of the transmissive unit 43. Therefore, compared with the case where the transmissive unit 43 is a plane, the imaging unit 30 can capture a wider range through direct vision.

[0198] That is, when the robot system control unit 90 moves the position of the tactile sensor module 10, the tactile sensor 1 can detect an image of an object. Therefore, the tactile sensor 1 in the above-described embodiment can avoid collisions.

[0199] In addition, according to the above-described embodiment, the grasping state detection unit 80 acquires an image captured by the imaging unit 30 and detects the contact state of the object with respect to the contact surface based on the acquired image.

[0200] That is, the robot system 100 can detect the grasping state of the object by including the grasping state detection unit 80.

[0201] By providing information to the robot control unit 91, the grasping state detection unit 80 enables the robot control unit 91 to control the robot system 100.

[0202] As described above, the embodiments have been used to illustrate the manner for implementing the present invention. However, the present invention is not limited to any of such embodiments, and various modifications and substitutions can be made without departing from the gist of the present invention.

[0203] Explanation of Reference Numerals

[0204] 1... Tactile sensor, 10... Tactile sensor module, 11... Sensor connection portion, 100... Robot system, 110... Front end portion, 120... Upper arm portion, 130... Joint portion, 140... Lower arm portion, 150... Main horizontal axis portion, 160... Main vertical axis portion, 170... Base portion, 90... Robot system control unit, 20... Reflecting portion, 30... Imaging unit, 40... Transmissive portion contact surface, 47... Transmissive portion non-contact surface, 43... Transmissive portion, 45... Mark, 50... Frame, 70... Hard layer, 91... Robot control unit, 92... Input unit, 93... Output unit, 80... Grasping state detection unit, 81... Image acquisition unit, 82... Image processing unit, 83... Control unit, 84... Reference state storage unit, A10... Field of view angle, OA... Imaging optical axis, IA10... First incident angle, RA10... First reflection angle, IA20... Second incident angle, RA20... Second reflection angle, IA30... Third incident angle, RA30... Third reflection angle, AR1... First imaging range, AR2... Second imaging range, AR3... Third imaging range, OB1... First object, OB2... Second object, OB3... Third object, RM... Direct view imaging mark, MM... Reflection view imaging mark, R... Direct view imaging image, M... Reflection view imaging image, P... Imaging image

Claims

1. A tactile sensor system, the tactile sensor system comprising: Two tactile sensors; and A detection unit that obtains a first image from an imaging unit included in the first of the two tactile sensors, obtains a second image from the imaging unit included in the second tactile sensor, and detects a contact state of an object with respect to a first surface of each of the two tactile sensors based on the obtained first image and the second image, The tactile sensor comprises: A transmissive part having a first surface that can come into contact with an object to be held and a second surface that is the back surface of the first surface; An imaging unit that can image an object existing on the first surface side of the transmissive part from the second surface side; and A reflecting part that is disposed on the second surface side of the transmissive part, reflects light from at least a part of the area of the transmissive part, and guides it into the imaging field angle of the imaging unit, Among them, The reflecting part has either a single flat surface or two flat surfaces with different angles, At least a part of the transmissive part deforms along the shape of the object to be held that contacts the first surface, The two tactile sensors are arranged at positions where the first surfaces of the two tactile sensors face each other.

2. The tactile sensor system according to claim 1, wherein, The imaging unit is arranged such that the imaging optical axis of the imaging unit intersects the normal line of the second surface of the transmissive part.

3. The tactile sensor system according to claim 1, wherein, The reflecting part includes a plurality of reflecting surfaces having different angles of the normal line with respect to the imaging optical axis of the imaging unit.

4. The tactile sensor system according to claim 2, wherein The reflecting part includes a plurality of reflecting surfaces having different angles of the normal line with respect to the imaging optical axis of the imaging unit.

5. The tactile sensor system according to any one of claims 1-4, wherein, The imaging unit images both a first image and a second image as images of the transmissive part. The first image is an image of the imaging object area of the transmissive part formed by light that enters without passing through the reflecting part, and the second image is an image of the imaging object area of the transmissive part formed by light that is reflected by the reflecting part and then enters.

6. The tactile sensor system according to any one of claims 1-4, wherein, The transmissive part includes a plurality of regions having different angles of the normal line with respect to the imaging optical axis of the imaging unit, The imaging unit can respectively image objects existing on the first surface side formed by light that enters through the plurality of regions of the transmissive part.

7. The tactile sensor system according to claim 5, wherein, The transmissive part includes a plurality of regions having different angles of the normal line with respect to the imaging optical axis of the imaging unit, The imaging unit can respectively image objects existing on the first surface side formed by light that enters through the plurality of regions of the transmissive part.

8. A program that executes, on a computer connected to two tactile sensors, an image acquisition step of acquiring images captured by the imaging units of the two tactile sensors respectively, and a detection step of detecting, on a detection unit, a contact state of an object with respect to a first surface of each of the two tactile sensors based on the images acquired in the image acquisition step, Among them, The tactile sensor comprises: A transmissive part having a first surface that can come into contact with an object to be held and a second surface that is the back surface of the first surface; An imaging unit that can image an object existing on the first surface side of the transmissive part from the second surface side; And A reflection unit, which is disposed on the second surface side of the transmission unit, reflects light from at least a part of the area of the transmission unit and guides it into the shooting field angle of the shooting unit. Wherein, the reflection unit has either a flat surface or two flat surfaces with different angles. At least a part of the transmission unit is deformed along the shape of the gripping object in contact with the first surface. The two tactile sensors are arranged at positions where the first surfaces of the two tactile sensors face each other. The detection unit obtains a first image from the shooting unit included in the first tactile sensor among the two tactile sensors, obtains a second image from the shooting unit included in the second tactile sensor, and detects the contact state of the object with respect to the first surfaces of the two tactile sensors based on the obtained first image and second image.

Images