Sensor Device

Abstract

[Problem] To perform haptic detection with a small and simple mechanism. [Solution] A sensor device according to the present invention comprises: a haptic action part which is exposed through an opening provided in an exterior part and which is attached inside the exterior part via a strain generator; a reflection space which is surrounded on two or more faces thereof by a first mirror that is provided on an inner bottom surface of the exterior part and a second mirror that is provided on a surface of the haptic action part or the strain generator which faces the first mirror; a light source unit which emits light to the reflection space; and an imaging unit which is provided on the inner bottom surface of the exterior part and which captures an image including a haptic detection area whereon reflection light of the light emitted from the light source unit appears.

Description

[Technical Field]

[0001] The present disclosure relates to a sensor device. [Background technology]

[0002] In recent years, sensor fusion technology has been attracting attention as it integrates information obtained from multiple sensors to extract new information that cannot be obtained from a single sensor. Therefore, there is a demand for more information to be obtained effectively from multiple sensors.

[0003] The application of sensor fusion technology to the control of a robot arm, for example, as disclosed in Patent Document 1 below, is being considered. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2020-43497 Summary of the Invention [Problem to be solved by the invention]

[0005] The robot arm disclosed in Patent Document 1 controls the operation of the robot arm by capturing an image of an object at the tip of the robot hand using an image sensor. The robot arm disclosed in Patent Document 1 also uses a force sensor to detect the force applied when the object is grasped by the robot hand, thereby enabling the object to be grasped with higher precision. However, installing a force sensor on the robot arm increases the size and mass of the entire system.

[0006] Therefore, the present disclosure proposes a new and improved sensor device that is capable of detecting force with a smaller and simpler mechanism. [Means for solving the problem]

[0007] According to the present disclosure, there is provided a sensor device comprising: a force sense acting section exposed from an opening provided in an exterior section and attached to the interior of the exterior section via a strain generating body; a reflection space surrounded on at least two sides by a first mirror provided on the interior bottom surface of the exterior section and a second mirror provided on a surface of the force sense acting section or the strain generating body facing the first mirror; a light source section that emits light into the reflection space; and an imaging section provided on the interior bottom surface of the exterior section that captures an image including a force sense detection area in which reflected light of the light emitted from the light source section is reflected. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a cross-sectional view schematically showing a configuration of a sensor device according to a first embodiment of the present disclosure. [Figure 2] FIG. 10 is an explanatory diagram showing a state in which light is emitted into a space surrounded by three mirrors facing each other. [Figure 3] FIG. 10 is an explanatory diagram showing a state in which light is emitted into a space surrounded by four mirrors facing each other. [Figure 4] 1 is an explanatory diagram showing an image captured by a sensor device and an external force acting on the sensor device; [Figure 5] 5 is an explanatory diagram showing an example of an image captured by the sensor device shown in FIG. 4. FIG. [Figure 6] FIG. 10 is a block diagram showing the flow of operations for spatial imaging and force calculation by the sensor device. [Figure 7] 10 is an explanatory diagram showing the relationship between the imaging timing of the sensor device according to the first modified example of the embodiment and the on / off timing of the light source unit. FIG. [Figure 8] FIG. 10 is a cross-sectional view showing a configuration in the vicinity of a light source unit of a sensor device according to a second modified example of the embodiment. [Figure 9] FIG. 10 is a cross-sectional view showing the configuration of a sensor device according to a third modified example of the embodiment. [Figure 10] 10A and 10B are explanatory diagrams showing an example of the configuration of a sensor device according to a fourth modified example of the embodiment from above and in cross section. [Figure 11]11 is a schematic diagram showing an example of an image captured by an imaging section of the sensor device shown in FIG. 10. FIG. [Figure 12] 11 is a schematic diagram showing another example of an image captured by the imaging section of the sensor device shown in FIG. 10. FIG. [Figure 13] 10A and 10B are explanatory diagrams showing another example of the configuration of the sensor device according to the fourth modified example of the embodiment from above and in cross section. [Figure 14] FIG. 10 is a cross-sectional view showing another example of the configuration of the sensor device according to the fourth modified example of the embodiment. [Figure 15] 15 is a schematic diagram showing an example of an image captured by an imaging section of the sensor device shown in FIG. 14. FIG. [Figure 16] FIG. 14 is a cross-sectional view showing still another example of the sensor device shown in FIG. [Figure 17] FIG. 10 is a perspective view schematically illustrating a configuration of a sensor device according to a second embodiment of the present disclosure. [Figure 18] FIG. 2 is an exploded view schematically illustrating the configuration of the sensor device according to the embodiment. [Figure 19] FIG. 10 is a perspective view showing the attachment of the force detection structure to the second exterior part. [Figure 20] FIG. 2 is a schematic diagram illustrating an example of an image captured by an imaging unit of the sensor device. [Figure 21] FIG. 4 is a cross-sectional view showing the configuration of a sensor device according to a first modified example of the embodiment. [Figure 22] FIG. 10 is a perspective view showing the configuration of a sensor device according to a second modified example of the embodiment, with a first exterior part removed. [Figure 23] FIG. 10 is a cross-sectional view schematically illustrating a first configuration example of a gripping device according to a third embodiment of the present disclosure. [Figure 24] FIG. 24 is a top view schematically showing the configuration of the gripping device shown in FIG. 23. [Figure 25] 24 is an explanatory diagram showing the configuration of an image captured by an imaging unit of the gripping device shown in FIG. 23. FIG. [Figure 26] FIG. 10 is a cross-sectional view schematically illustrating a second configuration example of a gripping device according to a third embodiment of the present disclosure. [Figure 27]FIG. 27 is a top view schematically showing the configuration of the gripping device shown in FIG. 26. [Figure 28] 27 is an explanatory diagram showing the configuration of an image captured by an imaging unit of the gripping device shown in FIG. 26. FIG. [Figure 29] 10 is a schematic diagram showing an example of an image captured by an imaging unit of a gripping device having four claws. FIG. [Figure 30] 10 is a schematic diagram showing an example of an image captured by an imaging unit of a gripping device having four claws. FIG. [Figure 31] FIG. 2 is a block diagram illustrating a functional configuration of the gripping device according to the embodiment. [Figure 32] FIG. 1 is an explanatory diagram showing an input device that is a first application example of the sensor device according to the first or second embodiment. [Figure 33] FIG. 10 is a perspective view showing a force detection device which is a second application example of the sensor device according to the first or second embodiment. [Figure 34] FIG. 34 is a longitudinal sectional view of the force detection device shown in FIG. 33. [Figure 35] 10A and 10B are diagrams illustrating a lens device that is a third application example of the sensor device according to the first or second embodiment. [Figure 36] FIG. 10 is an explanatory diagram showing a robot device that is a first application example of a gripping device according to a third embodiment. [Figure 37] FIG. 10 is an explanatory diagram showing an example of a robot arm device that is a second application example of the gripping device according to the third embodiment. [Figure 38] 10 is an explanatory view showing another example of a robot arm device which is a second application example of the gripping device according to the third embodiment. FIG. [Figure 39] FIG. 10 is an explanatory diagram showing a moving body that is a third application example of the gripping device according to the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Preferred embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant description will be omitted.

[0010] The explanation will be given in the following order. 1. First embodiment (sensor device) 1.1.Configuration 1.2.Operation 1.3. Variations 2. Second embodiment (sensor device) 3. Third embodiment (gripping device) Configuration Function 4. Application Examples

[0011] <1. First embodiment (sensor device)> (1.1. Configuration) First, the configuration of a sensor device according to a first embodiment of the present disclosure will be described with reference to Fig. 1. Fig. 1 is a cross-sectional view that schematically shows the configuration of a sensor device 10 according to this embodiment.

[0012] As shown in FIG. 1, the sensor device 10 includes a base section 110, a force sense acting section 120, strain generating bodies 130 and 180, an imaging section 140, a light source section 150, a first mirror 161, a second mirror 162, and a half mirror 170.

[0013] The base unit 110 is a rigid structural member in the shape of a flat plate with a light intake hole 110H provided in approximately the center. The base unit 110 allows light (also referred to as external light) incident from outside the sensor device 10 to be incident on the imaging unit 140 provided on the first surface S1 side of the base unit 110 (the side opposite to the incident side of the external light entering the sensor device 10) through the light intake hole 110H provided in approximately the center.

[0014] The force sense action unit 120 is a rigid structural member provided on the second surface S2 side of the base unit 110 (the side on which external light enters the sensor device 10) via a strain generating body 130. The force sense action unit 120 may be provided, for example, so as to face the second surface S2 of the base unit 110 around the light intake hole 110H.

[0015] The force sense action section 120 is a part of the sensor device 10 on which an external force (also referred to as an external force) acts. When an external force acts on the force sense action section 120, the strain generating body 130 between the force sense action section 120 and the base section 110 deforms, and the positional relationship between the first mirror 161 and the second mirror 162 changes. This causes a change in the position of each light spot of the reflected light that is emitted from the light source section 150 and reflected multiple times by the first mirror 161 and the second mirror 162. Therefore, the sensor device 10 can detect the external force acting on the force sense action section 120 by measuring the change in the position of each light spot of the reflected light that is reflected multiple times by the first mirror 161 and the second mirror 162.

[0016] There may be provided a plurality of force sense action units 120. In such a case, the sensor device 10 can detect external forces acting on a plurality of locations of the sensor device 10 by receiving the external forces acting on the plurality of locations of the sensor device 10 at each of the force sense action units 120.

[0017] Specifically, the multiple force-sense action units 120 may be arranged point-symmetrically or line-symmetrically with respect to the light intake hole 110H. For example, two force-sense action units 120 may be arranged at 180 degrees with the light intake hole 110H in between (i.e., facing each other with the light intake hole 110H in between). Three force-sense action units 120 may be arranged at 120 degrees from each other with the light intake hole 110H as the center, or four force-sense action units 120 may be arranged at 90 degrees from each other with the light intake hole 110H as the center. By arranging the multiple force-sense action units 120 point-symmetrically or line-symmetrically with respect to the light intake hole 110H, it is possible to isotropically detect external forces acting on the sensor device 10.

[0018] The half mirror 170 is provided on the external light incident side so as to cover the light intake hole 110H. Specifically, the half mirror 170 may be provided in the shape of a rectangular or circular flat plate, and may be provided so as to span between the multiple force sense action units 120 via the strain generating body 180.

[0019] The half mirror 170 is an optical element having a light transmittance of more than 20% but less than 90% and a light reflectance of more than 10% but less than 80%, which transmits and reflects a portion of incident light. For example, the half mirror 170 can be formed by depositing an ultra-thin film of a metal material such as chromium (Cr) on a transparent member made of glass, resin, or the like, so that the film has sufficient light transmittance and reflectance. Alternatively, the half mirror 170 can be formed by depositing a dielectric multilayer film on a transparent member made of glass, resin, or the like, so that the film has sufficient light transmittance and reflectance. The light transmittance and light reflectance of the half mirror 170 can be set to any value depending on the characteristics to be achieved by the sensor device 10.

[0020] The optically transparent half mirror 170 can, for example, transmit external light incident on the sensor device 10 into the interior of the sensor device 10, where the light intake hole 110H is provided. This allows the sensor device 10 to capture an image of the external space of the sensor device 10 with the imaging unit 140 using the external light taken in through the light intake hole 110H. Furthermore, the optically reflective half mirror 170 can, for example, reflect light emitted from the light source unit 150 in the same manner as the first mirror 161 and the second mirror 162. This allows the sensor device 10 to detect, with the imaging unit 140, the position of each light point of the reflected light that is emitted from the light source unit 150 and is multiple-reflected by the first mirror 161, the second mirror 162, and the half mirror 170. Therefore, the sensor device 10 can simultaneously capture an image of the external space of the sensor device 10 and detect a group of light points of reflected light that is multiple-reflected by the first mirror 161, the second mirror 162, and the half mirror 170 using the imaging unit 140.

[0021] The flexure bodies 130, 180 are structural members that deform in proportion to the applied stress. For example, the flexure bodies 130, 180 may be elastic members that are clearly easily deformed, such as rubber, elastomer, or springs. Alternatively, the flexure bodies 130, 180 may be structural members that are made of the same material as the other components but have lower rigidity so that they are more easily deformed than the other components. The flexure body 130 is provided between the base unit 110 and the force sense action unit 120, and the flexure body 180 is provided between the half mirror 170 and the force sense action unit 120. The flexure bodies 130, 180 deform in response to an external force acting on the force sense action unit 120, thereby displacing the positional relationship between the first mirror 161, the second mirror 162, and the half mirror 170.

[0022] The first mirror 161 is provided on the second surface S2 of the base unit 110, and the second mirror 162 is provided on the surface of the force sense action unit 120 facing the base unit 110. That is, the first mirror 161 and the second mirror 162 are provided on the surface facing the internal space of the sensor device 10 surrounded by the base unit 110 and the force sense action unit 120. The first mirror 161 and the second mirror 162 can be formed, for example, by depositing a metal material such as chromium (Cr) in a film thickness having sufficient light reflectivity on a transparent member made of glass, resin, or the like. The first mirror 161 and the second mirror 162, which face each other, can multiple-reflect light emitted from the light source unit 150 in the reflection space 121 between the first mirror 161 and the second mirror 162.

[0023] Here, multiple reflections by the opposing first mirror 161 and second mirror 162 will be described with reference to Figures 2 and 3. Figure 2 is an explanatory diagram showing a state in which light is emitted into a space surrounded by three opposing mirrors. Figure 3 is an explanatory diagram showing a state in which light is emitted into a space surrounded by four opposing mirrors.

[0024] 2, light L emitted from light source unit 1500 is multiple-reflected by first mirror 1610, second mirror 1620, and third mirror 1630, which are provided facing each other at positions corresponding to each side of the triangular prism. In such a case, light L emitted from light source unit 1500 is received by light-receiving unit 1400 after the number of light spots of the reflected light is amplified by multiple reflections by first mirror 1610, second mirror 1620, and third mirror 1630. Furthermore, when first mirror 1610, second mirror 1620, or third mirror 1630 is displaced, the positions of the light spots of the reflected light of light L emitted from light source unit 1500 are displaced by amplifying the displacement of first mirror 1610, second mirror 1620, or third mirror 1630.

[0025] The first mirror 1610, the second mirror 1620, and the third mirror 1630 may be arranged to correspond to each side of an equilateral triangle or an isosceles triangle, or may be arranged to correspond to each side of a triangle obtained by breaking an equilateral triangle or an isosceles triangle. By arranging the first mirror 1610, the second mirror 1620, and the third mirror 1630 to correspond to each side of a triangle with low symmetry (i.e., a triangle obtained by breaking an equilateral triangle or an isosceles triangle), it is possible to further increase the number of light spots of reflected light due to multiple reflections.

[0026] In the sensor device 10, the first mirror 161, the second mirror 162, and a third mirror (not shown) can form a structure corresponding to the side surfaces of a triangular prism as shown in Fig. 2. In this case, the sensor device 10 can perform multiple reflections of light emitted from the light source unit 150, using the interior of the triangular prism whose side surfaces are formed by the first mirror 161, the second mirror 162, and the third mirror as a reflection space 121.

[0027] The first mirror 161 and the second mirror 162 may form a structure corresponding to the side surfaces of a triangular pyramid between them and a third mirror (not shown). Even in this case, the sensor device 10 can perform multiple reflection of the light emitted from the light source unit 150 by using the inside of the triangular pyramid whose side surfaces are formed by the first mirror 161, the second mirror 162, and the third mirror as the reflection space 121.

[0028] 3, light L emitted from light source unit 1500 is multiple-reflected by first mirror 1610, second mirror 1620, third mirror 1630, and fourth mirror 1640, which are provided facing each other at positions corresponding to each side of the rectangular prism. In such a case, light L emitted from light source unit 1500 is multiple-reflected by first mirror 1610, second mirror 1620, third mirror 1630, and fourth mirror 1640, and the number of light spots of the reflected light is amplified and received by light-receiving unit 1400. Furthermore, when first mirror 1610, second mirror 1620, third mirror 1630, or fourth mirror 1640 is displaced, the positions of the light spots of the reflected light of light L emitted from light source unit 1500 are displaced by amplifying the displacement of first mirror 1610, second mirror 1620, third mirror 1630, or fourth mirror 1640.

[0029] In the sensor device 10, the first mirror 161 and the second mirror 162 can be configured with a third mirror and a fourth mirror (not shown) to form a structure corresponding to the side surfaces of a quadrangular prism as shown in Fig. 3. In this case, the sensor device 10 can perform multiple reflections of light emitted from the light source unit 150, using the interior of the quadrangular prism whose side surfaces are formed by the first mirror 161, the second mirror 162, the third mirror, and the fourth mirror as a reflection space 121.

[0030] The first mirror 161 and the second mirror 162 may form a structure corresponding to the side surfaces of a quadrangular pyramid between themselves and a third mirror and a fourth mirror (not shown). Even in this case, the sensor device 10 can multiple-reflect the light emitted from the light source unit 150 by using the inside of the quadrangular pyramid whose side surfaces are formed by the first mirror 161, the second mirror 162, the third mirror, and the fourth mirror as the reflection space 121.

[0031] The light source unit 150 emits light toward the second surface S2 side of the base unit 110. Specifically, the light source unit 150 emits light into a reflection space 121 surrounded on at least two sides by a first mirror 161 and a second mirror 162. The reflection space 121 is, for example, the space between the first mirror 161 and the second mirror 162 facing each other. By emitting light into the reflection space 121, the light source unit 150 can cause the emitted light to be multiply reflected in the reflection space 121 between the first mirror 161 and the second mirror 162. The light source unit 150 may emit light into the reflection space 121 from a bottom side of the reflection space 121 (i.e., the base unit 110 side) or from a side side of the reflection space 121 (i.e., the flexure body 130 side). The light source unit 150 may be, for example, an LED (Light Emitting Diode) light source capable of emitting light with high directional propagating properties.

[0032] For example, the light source unit 150 may be provided on the base unit 110 side. In this case, the sensor device 10 can form wiring to the light source unit 150 in the same way as wiring to the imaging unit 140, thereby reducing the cost and workload for forming wiring. Therefore, the sensor device 10 can further reduce production costs.

[0033] Furthermore, the light source unit 150 may be provided inside the base unit 110 so that the main body of the LED light source or the like and the wiring are not exposed to the reflection space 121. In such a case, the sensor device 10 can prevent the images of the main body and wiring of the light source unit 150 from being multiple-reflected by the first mirror 161 and the second mirror 162. Therefore, the sensor device 10 can prevent the multiple-reflected images of the main body and wiring of the light source unit 150 from becoming a noise source, and can therefore prevent a decrease in the detection sensitivity of the light spot group of reflected light that is multiple-reflected by the first mirror 161 and the second mirror 162.

[0034] Furthermore, the light source unit 150 may emit light into the reflection space 121 through a pinhole. The pinhole is, for example, a hole with a diameter of about several millimeters. By emitting light into the reflection space 121 through the pinhole, the light source unit 150 can further improve the convergence of the emitted light. This allows the light source unit 150 to make the shape of each light spot of the reflected light that is multiple-reflected by the first mirror 161 and the second mirror 162 smaller and more circular, thereby improving the detection sensitivity of each light spot. Furthermore, since the accuracy of pinhole processing is generally higher than the positioning accuracy during assembly of the light source unit 150, the light source unit 150 can further improve the accuracy of the light emission position by emitting light into the reflection space 121 through the pinhole. Therefore, the sensor device 10 can more easily control the position of each light spot of the reflected light that is multiple-reflected by the first mirror 161 and the second mirror 162.

[0035] Here, when a plurality of haptic sense units 120 are provided, a plurality of light source units 150 may be provided corresponding to each of the haptic sense units 120. In such a case, the light source unit 150 may emit light of a different color for each corresponding haptic sense unit 120. Furthermore, the light source unit 150 may emit light into a separate reflection space 121 surrounded on two sides by a second mirror 162 and a first mirror 161 provided in the corresponding haptic sense unit 120 so that the light spot groups of reflected light are separated for each corresponding haptic sense unit 120. In this manner, the sensor device 10 can detect an external force acting on each of the haptic sense units 120 by the displacement of the light spot groups of reflected light that can be separated from each other by color or position. Therefore, the sensor device 10 can detect external forces acting on each of the haptic sense units 120 separately from each other with high accuracy.

[0036] The imaging unit 140 is an image sensor that acquires a captured image by receiving light incident through the light intake hole 110H. The imaging unit 140 may be, for example, a CMOS image sensor or a CCD image sensor that can detect light within the wavelength band of visible light. The imaging unit 140 can receive external light that passes through the half mirror 170 and enters the sensor device 10, as well as a group of light spots of reflected light that is emitted from the light source unit 150 and is multiplexedly reflected by the first mirror 161, the second mirror 162, and the half mirror 170. That is, the imaging unit 140 can acquire an image of the external space of the sensor device 10 in which the group of light spots of the multiplexed reflected light are superimposed. This allows the sensor device 10 to detect an external force acting on the force sense action unit 120 from the displacement of the positions of the light spots of the reflected light that is multiplexedly reflected by the first mirror 161, the second mirror 162, and the half mirror 170. Therefore, the sensor device 10 can simultaneously capture an image of the external space and detect an external force acting on the force sense effect unit 120.

[0037] As another example, the image capturing unit 140 may be an image sensor capable of detecting light included in the wavelength band of visible light and light included in the wavelength band of infrared light (particularly, short-wavelength infrared light (near-infrared light)). In this case, the image capturing unit 140 can detect not only the appearance of the object but also the characteristics of the object resulting from the infrared absorption or reflection characteristics of the object, such as the object's temperature, moisture content, or material.

[0038] As another example, the image capturing unit 140 may be an image sensor capable of detecting light included in the wavelength band of visible light and light having a predetermined polarization. In this case, the image capturing unit 140 can detect stress or strain applied to the object in addition to the appearance of the object.

[0039] As another example, the image capturing unit 140 may include a ranging pixel capable of measuring the distance to an object using a time-of-flight (ToF) technique, in which case the sensor device 10 can further measure the distance to an object present in external space.

[0040] It should be noted that a cylindrical or rectangular tube-shaped exterior part that houses the force sense action part 120 and the reflection space 121 inside may be further provided on the outer edge of the base part 110. Specifically, the exterior part may be provided as the side surface of a cylinder or rectangular prism with the base part 110 as the bottom surface and the distance from the base part 110 to the half mirror 170 as the height, and may house each component of the sensor device 10 inside. The exterior part can prevent light from outside the sensor device 10 from entering the reflection space 121 from the side and becoming a source of noise, thereby improving the stability of force sense detection using the light point cloud of reflected light.

[0041] Furthermore, the inner surface of the exterior part may be a mirror surface or a non-mirror surface. When the inner surface of the exterior part is a mirror surface, the exterior part can reflect light that is emitted from the light source part 150 into the reflection space 121 and then travels outside the sensor device 10 back into the sensor device 10. Therefore, the exterior part can increase the number of light points of the reflected light that is multiple-reflected by the first mirror 161 and the second mirror 162, making it possible to detect an external force acting on the force sense action part 120 with higher accuracy.

[0042] (1.2.Operation) Next, functions of the sensor device 10 according to this embodiment will be described with reference to Fig. 4 to Fig. 6. Fig. 4 is an explanatory diagram showing an image captured by the sensor device 10 and an external force acting on the sensor device 10. Fig. 5 is an explanatory diagram showing an example of an image captured by the sensor device 10 shown in Fig. 4.

[0043] For example, as shown in Fig. 4, the sensor device 10 can capture an image of a subject 900 present in external space. Furthermore, when facing directly in Fig. 4, the sensor device 10 receives a force Fz1 in the Z-axis direction, a moment Mx1 about the X-axis, or a moment My1 about the Y-axis at the upper force-sense action section 120a, and can detect these force Fz1, moment Mx1, or moment My1. Furthermore, when facing directly in Fig. 4, the sensor device 10 receives a force Fz2 in the Z-axis direction, a moment Mx2 about the X-axis, or a moment My2 about the Y-axis at the lower force-sense action section 120b, and can detect these force Fz2, moment Mx2, or moment My2.

[0044] Specifically, as shown in FIG. 5, an image CI captured by the imaging section 140 includes a captured image of the subject 900 and light spot groups LC1 and LC2.

[0045] The light spot group LC1 is, for example, a light spot group of reflected light that is emitted from the upper light source unit 150a facing directly in FIG. 4 and is multiple-reflected by the first mirror 161a and the second mirror 162a. When a force Fz1, a moment Mx1, or a moment My1 acts on the force sense action unit 120a, the positions of the force sense action unit 120a and the second mirror 162a are displaced. As a result, the position of the light spot group LC1 on the image CI is displaced in a direction corresponding to the force Fz1, moment Mx1, or moment My1 acting on the force sense action unit 120a. Therefore, the sensor device 10 can calculate the direction and magnitude of the force Fz1, moment Mx1, and moment My1 acting on the force sense action unit 120a from the displacement of the position of the light spot group LC1.

[0046] Similarly, the light spot group LC2 is a light spot group formed by, for example, light emitted from the lower light source unit 150b facing directly in FIG. 4 and being multiple-reflected by the first mirror 161b and the second mirror 162b. When a force Fz2, a moment Mx2, or a moment My2 acts on the force sense action unit 120b, the positions of the force sense action unit 120b and the second mirror 162b are displaced. As a result, the position of the light spot group LC2 on the image CI is displaced in a direction corresponding to the force Fz2, the moment Mx2, or the moment My2 acting on the force sense action unit 120b. Therefore, the sensor device 10 can calculate the direction and magnitude of the force Fz2, the moment Mx2, and the moment My2 acting on the force sense action unit 120b from the displacement of the position of the light spot group LC2.

[0047] Therefore, the sensor device 10 can calculate the direction and magnitude of the external force acting on the force sense action unit 120a by previously associating the manner of displacement of the position of the light spot group LC1 of the reflected light with the actual measured values of the direction and magnitude of the external force acting on the force sense action unit 120a. Also, the sensor device 10 can calculate the direction and magnitude of the external force acting on the force sense action unit 120b by previously associating the manner of displacement of the position of the light spot group LC2 of the reflected light with the actual measured values of the direction and magnitude of the external force acting on the force sense action unit 120b. For example, the sensor device 10 may use machine learning to associate the manner of displacement of the position of each of the light spot groups LC1 and LC2 of the reflected light with the actual measured values of the direction and magnitude of the external force acting on each of the force sense action units 120a and 120b. Alternatively, the sensor device 10 may create a calibration curve to associate the state of displacement of each position of the light spot groups LC1 and LC2 of the reflected light with the actual measured values of the direction and magnitude of the external force acting on each of the force sense action units 120a and 120b.

[0048] According to this, the sensor device 10 can capture an image CI in which the light spot groups LC1 and LC2 are superimposed on the captured image of the subject 900, and therefore can simultaneously measure the forces and moments acting on the force sense acting sections 120a and 120b while capturing an image of the subject 900.

[0049] The flow of operations of the sensor device 10 will be described with reference to Fig. 6. Fig. 6 is a block diagram showing the flow of operations of the sensor device 10 for spatial imaging and force sense calculation.

[0050] 6, in the sensor device 10, first, external light incident on the sensor device 10 and reflected light 1 and reflected light 2 that are multiple-reflected inside the sensor device 10 are incident on the half mirror 170. Of the external light incident on the sensor device 10, light that has passed through the half mirror 170 and reflected light 1 and reflected light 2 that are reflected by the half mirror 170 are incident on the imaging unit 140 through the light intake hole 110H. As a result, the imaging unit 140 acquires an image in which a captured image generated from the external light, a light spot cloud LC1 generated from reflected light 1, and a light spot cloud LC2 generated from reflected light 2 are superimposed.

[0051] This allows the sensor device 10 to capture an image of the external space of the sensor device 10 from an image generated from external light (spatial imaging). The sensor device 10 can also calculate a force or moment acting on the force-sense action unit 120a based on the light spot cloud LC1 generated from reflected light 1 (force-sense calculation 1). The sensor device 10 can also calculate a force or moment acting on the force-sense action unit 120b based on the light spot cloud LC2 generated from reflected light 2 (force-sense calculation 2).

[0052] The sensor device 10 having the above configuration can simultaneously receive, at the imaging unit 140, external light incident on the sensor device 10 via the half mirror 170 and reflected light that is multiply reflected inside the sensor device 10. Therefore, the sensor device 10 can capture an image of the external space based on external light and detect a sense of force calculated from the multiply reflected reflected light, using a smaller and simpler mechanism. Furthermore, because the sensor device 10 can simultaneously acquire an image of the external space and the multiply reflected reflected light using the imaging unit 140, it is possible to synchronize the image of the external space with the detection result of an external force acting on the sensor device 10.

[0053] In the sensor device 10, light emitted from the light source unit 150 is multiple-reflected by a first mirror 161 and a second mirror 162 that face each other. This allows the sensor device 10 to amplify the number of light spots of the reflected light captured by the imaging unit 140 and the amount of positional displacement through multiple reflections, thereby improving the sensitivity of force detection and reducing noise. Furthermore, by increasing the number of light spots of the reflected light captured by the imaging unit 140, the sensor device 10 can further expand the dynamic range of force detection.

[0054] (1.3. Variations) Next, first to fourth modified examples of the sensor device 10 according to this embodiment will be described with reference to FIGS.

[0055] (First Modification) FIG. 7 is an explanatory diagram showing the relationship between the imaging timing of the sensor device 10 according to the first modified example and the on / off timing of the light source unit 150. In FIG.

[0056] 7 , in the sensor device 10 according to the first modification, the emission of light by the light source unit 150 may be controlled in synchronization with the frame rate of image capture by the image capture unit 140. Specifically, the on / off of the emission of light by the light source unit 150 may be switched every few frames captured by the image capture unit 140. For example, the on / off of the emission of light by the light source unit 150 may be switched every five frames in synchronization with the frame rate of image capture by the image capture unit 140. In this manner, the image capture unit 140 can alternately capture an image CI on which a light spot cloud LC of reflected light of light emitted from the light source unit 150 is superimposed and an image CIX on which the light spot cloud LC is not superimposed every few frames.

[0057] Therefore, the imaging section 140 can separately capture an image CI on which the light spot cloud LC is superimposed for detecting an external force acting on the force sense action section 120, and an image CIX on which the light spot cloud LC is not superimposed for capturing an image of the external space. In such a case, the sensor device 10 can prevent the captured image of the external space and the light spot cloud LC for detecting an external force acting on the force sense action section 120 from becoming noise sources for each other. Therefore, the sensor device 10 can perform both the imaging of the external space and the detection of the force sense with lower noise.

[0058] In the sensor device 10 according to the first modification, the light source unit 150 may be an LED light source capable of switching light emission on and off at high speed, and the image capture unit 140 may be an EVS (Event Based Vision Sensor). The EVS is an image capture device that outputs only data of pixels whose luminance has changed. Because the EVS outputs only data of pixels whose luminance has changed, it is possible to capture images at an extremely high frame rate (e.g., approximately 10 kFPS) compared to a normal RGB camera. Therefore, when the image capture unit 140 is an EVS, the sensor device 10 can capture images by switching at high speed between an image CI on which the light spot cloud LC is superimposed and an image CIX on which the light spot cloud LC is not superimposed. This allows the sensor device 10 to more smoothly detect an external force acting on the haptic effect unit 120 while separately capturing an image CIX for capturing an image of the external space and an image CI for detecting an external force acting on the haptic effect unit 120.

[0059] (Second Modification) FIG. 8 is a cross-sectional view showing the configuration of the vicinity of the light source unit 150 of the sensor device 10 according to the second modification.

[0060] As shown in FIG. 8 , in the sensor device 10 according to the second modification, the light source unit 150 may emit light into the reflection space 121 through a conical pinhole 150H. Specifically, the pinhole 150H is a conical hole that tapers forward toward the reflection space 121. By emitting light into the reflection space 121 through the conical pinhole 150H, the light source unit 150 can diffuse the light over a wider area than when light is emitted into the reflection space 121 through a linear pinhole. This allows the light source unit 150 to further increase the number of light spots of the reflected light that is multiple-reflected by the first mirror 161 and the second mirror 162. Therefore, the sensor device 10 can detect an external force acting on the force sense action unit 120 with higher accuracy.

[0061] (Third Modification) FIG. 9 is a cross-sectional view showing the configuration of a sensor device 10 according to a third modified example.

[0062] 9, in the sensor device 10 according to the third modification, an infrared cut filter 171 may be provided on either surface of the half mirror 170. Specifically, the infrared cut filter 171 is a filter that selectively cuts (absorbs) infrared rays and transmits visible light. For example, the infrared cut filter 171 may be provided on the surface of the half mirror 170 on which external light is incident.

[0063] In such a case, the sensor device 10 according to the third modification can prevent infrared rays contained in external light from entering the sensor device 10. Therefore, by using infrared light as the light emitted from the light source unit 150, the sensor device 10 according to the third modification can capture an image of the external space using visible light and detect an external force acting on the force sense unit 120 using a light spot cloud of reflected infrared light. That is, the sensor device 10 according to the third modification can separate the wavelength band of light used to detect an external force acting on the force sense unit 120 from the wavelength band of light used to capture an image of the external space. This makes it possible for the sensor device 10 according to the third modification to prevent mutual interference between the light used to detect an external force acting on the force sense unit 120 and the light used to capture an image of the external space.

[0064] (Fourth Modification) Fig. 10 is an explanatory diagram showing an example of the configuration of a sensor device 11 according to a fourth modified example from above and in cross section. Fig. 11 is a schematic diagram showing an example of an image captured by the imaging unit 140 of the sensor device 11 shown in Fig. 10. Fig. 12 is a schematic diagram showing another example of an image captured by the imaging unit 140 of the sensor device 11 shown in Fig. 10.

[0065] 10, in the sensor device 11 according to the fourth modification, the half mirror 170 may be provided on the incident side of external light at an oblique angle with respect to the light intake hole 110H. Specifically, in the half mirror 170 using a dielectric multilayer film, light loss can be reduced by allowing light to be incident obliquely (for example, by allowing light to be incident at an incident angle of about 45 degrees). Therefore, in the sensor device 11, by providing the half mirror 170 so that external light incident on the sensor device 11 and reflected light multiplexed by the first mirror 161 and the second mirror 162 are incident obliquely, the incident light can be more efficiently received by the imaging unit 140.

[0066] 10 , a pair of force sense action sections 120 are provided on the base section 110 via strain bodies 130 on both sides of a light intake hole 110H as the center. In addition, a pair of protrusions 111 are provided on the base section 110 on the inside of the pair of force sense action sections 120 so as to face the force sense action sections 120. A first mirror 161 and a second mirror 162 are provided on the opposing surfaces of the force sense action sections 120 and the protrusions 111, and the light source section 150 emits light into a reflection space 121 between the first mirror 161 and the second mirror 162.

[0067] 10, the half mirror 170 is provided so as to be able to reflect the light reflected from the second mirrors 162 provided in the pair of force sense action units 120 to the image capturing unit 140. For example, the half mirror 170 may be provided in a valley fold shape above the light intake hole 110H and may reflect the light reflected from the second mirrors 162 provided in each of the force sense action units 120 to the image capturing unit 140.

[0068] As a result, the reflected light that has been multiple-reflected between the first mirror 161 and the second mirror 162 is incident at an angle on the half mirror 170 provided above the light intake hole 110H and is reflected toward the imaging unit 140. Furthermore, external light that has been incident from outside the sensor device 11 is incident at an angle on the half mirror 170 and is transmitted toward the imaging unit 140. Therefore, the sensor device 11 can receive external light from the external space and reflected light that has been multiple-reflected by the first mirror 161 and the second mirror 162 at the imaging unit 140.

[0069] 10, the half mirror 170 may be replaced with a normal mirror. Specifically, the sensor device 11 shown in FIG. 10 may include a mirror that covers only the center of the angle of view of the image capturing unit 140 so as not to completely block light that enters the image capturing unit 140 from the external space. In this case, the image capturing unit 140 can simultaneously capture the reflected light of the light from the light source unit 150 that is reflected by the mirror and the external space that enters from the angle of view of the outer edge that is not covered by the mirror.

[0070] In such a case, the imaging unit 140 of the sensor device 11 shown in FIG. 10 can capture an image 901A including a force detection area 903 located in the center of the angle of view and imaging areas 902 located on the outer edges of both sides of the angle of view, as shown in FIG. 11, for example.

[0071] The force sense detection area 903 is an area in which a light spot cloud LC of reflected light of light emitted from the light source unit 150 is reflected, and is arranged in the center of the angle of view corresponding to the area in which the mirror is provided. The imaging area 902 is an area in which the external space is reflected, and is arranged on the outer edges on both sides of the angle of view corresponding to the area other than the area in which the mirror is provided. That is, in the image 901A, the force sense detection area 903 that detects an external force acting on the force sense application unit 120 and the imaging area 902 that captures an image of the external space are arranged separately from each other.

[0072] 12, the imaging section 140 of the sensor device 11 shown in FIG. 10 may capture an image 901B including only a force sense detection region 903 in which a light spot cloud LC of reflected light of light emitted from the light source section 150 is captured. In such a case, the sensor device 11 shown in FIG. 10 is configured as a small sensor capable of detecting force sense at multiple locations. This allows the sensor device 11 shown in FIG. 10 to detect external forces acting on multiple force sense action sections 120 in synchronization with each other. Furthermore, the sensor device 11 shown in FIG. 10 can expand the region in which the light spot cloud LC of reflected light of light emitted from the light source section 150 is captured, thereby further widening the dynamic range of force sense detection.

[0073] 13 is a schematic diagram showing a top view and a cross section of another example of the configuration of sensor device 11 according to the fourth modification. As shown in FIG. 13, in sensor device 11 according to the fourth modification, half mirror 170 is provided on the incident side of external light at an oblique angle with respect to light intake hole 110H. Furthermore, in FIG. 13, half mirrors 170 are separated from each other and provided near protrusion 111 at an angle that allows reflected light from second mirror 162 to be reflected to imaging unit 140. Half mirror 170 may be provided so as not to cover light intake hole 110H and not to block external light entering from outside sensor device 11.

[0074] As a result, the reflected light that is multiple-reflected between the first mirror 161 and the second mirror 162 is obliquely incident on the half mirror 170 provided above the light intake hole 110H and is reflected toward the imaging unit 140. Furthermore, external light that enters the sensor device 11 from outside enters the imaging unit 140 in part of the field of view without passing through the half mirror 170, and in part of the field of view passes through the half mirror 170 and enters the imaging unit 140. In such a case, the imaging unit 140 can receive external light that is not reflected or absorbed by the half mirror 170 in part of the field of view, thereby enabling a clearer captured image of the external space to be obtained. Therefore, the sensor device 11 can receive external light from the external space and reflected light that is multiple-reflected by the first mirror 161 and the second mirror 162 at the imaging unit 140.

[0075] Fig. 14 is a cross-sectional view showing yet another example of the configuration of sensor device 11 according to the fourth modified example. Fig. 15 is a schematic diagram showing an example of an image captured by imaging section 140 of sensor device 11 shown in Fig. 14.

[0076] As shown in Fig. 14, the sensor device 11 according to the fourth modification may have a structure including a force sense action section 120, a strain generating body 130, a light source section 150, a first mirror 161, a second mirror 162, and a half mirror 170 on only one side of the imaging section 140. The sensor device 11 shown in Fig. 14 can detect an external force acting on the force sense action section 120 provided on one side. The half mirror 170 may be replaced with a normal mirror that does not transmit light.

[0077] In such a case, the imaging unit 140 of the sensor device 11 shown in FIG. 14 can capture an image 901C including a force detection area 903 that occupies most of the angle of view and an imaging area 902 that occupies only a small portion of the angle of view, as shown in FIG. 15, for example.

[0078] The force detection area 903 is an area in which at least the light spot group LC of reflected light of light emitted from the light source unit 150 is reflected, and is arranged in an area that occupies more than half of the angle of view, corresponding to the area in which the half mirror 170 is provided. Note that the force detection area 903 may reflect only the light spot group LC of reflected light of light emitted from the light source unit 150, or may reflect a superimposed image of the light spot group LC and the external space. The imaging area 902 is an area in which the external space is reflected, and is arranged in an area that occupies less than half of the angle of view, corresponding to the area other than the area in which the half mirror 170 is provided.

[0079] 14, most of the angle of view of the captured image 901C becomes the force sense detection area 903 for detecting an external force acting on the force sense action unit 120. This allows the sensor device 11 shown in Fig. 14 to further increase the number of light points of reflected light included in the captured image 901C, thereby further improving noise resistance in force sense detection.

[0080] 16 is a cross-sectional view showing yet another example of the sensor device 11 shown in FIG. 13. As shown in FIG. 16, when external light incident from outside the sensor device 11 enters the imaging unit 140 without passing through the half mirror 170, the half mirror 170 may have an end surface 170E cut parallel to the ray direction of the light incident from the external space to the imaging unit 140. In this case, the sensor device 11 can prevent the external light incident from outside the sensor device 11 from being reflected by the end surface 170E of the half mirror 170. Therefore, the sensor device 11 can prevent the image of the external space captured by the imaging unit 140 from being distorted by the end surface 170E of the half mirror 170.

[0081] Furthermore, the end surface 170E of the half mirror 170 may be painted in a dark color so that the light emitted from the light source unit 150 does not generate a bright spot on the end surface 170E of the half mirror 170. The dark color is, for example, an achromatic color with a brightness of 30% or less in the HSV color space. This prevents the light emitted from the light source unit 150 from passing through the half mirror 170 and generating a bright spot on the end surface 170E of the half mirror 170, thereby preventing the bright spot from obstructing the field of view of the external space.

[0082] (Fifth Modification) The sensor device 10 according to the fifth modification may further be provided with various sensors.

[0083] For example, the sensor device 10 according to the fifth modification may be provided with a microphone on a member other than the force sense action unit 120. In such a case, the sensor device 10 can use the microphone to pick up contact sounds made when an external force acts on the force sense action unit 120, thereby pseudo-detecting the tactile sensation at the time of contact of the external force acting on the force sense action unit 120 through the picked up high-frequency band sound wave vibrations. Furthermore, the sensor device 10 can also detect or predict a failure or the like of the sensor device 10 by using the microphone to pick up abnormal sounds generated from the imaging unit 140 or light source unit 150 inside the sensor device 10, or from a drive unit outside the sensor device 10.

[0084] For example, the sensor device 10 according to the fifth modified example may further be provided with a temperature sensor. In such a case, the sensor device 10 can detect an external force by detecting the temperature of the sensor device 10 with the temperature sensor and correcting for expansion or contraction of the base unit 110 and the force sense unit 120 due to temperature changes. Specifically, structural members such as the base unit 110 and the force sense unit 120 expand or contract due to temperature changes. Therefore, the positions of the light points of the reflected light that are multiple-reflected by the first mirror 161 and the second mirror 162 may be displaced due to temperature changes accompanying the expansion or contraction of the base unit 110 and the force sense unit 120. Therefore, the sensor device 10 can detect an external force with higher accuracy by correcting for expansion or contraction of the base unit 110 and the force sense unit 120 based on the temperature detected by the temperature sensor. In addition, the sensor device 10 can also detect malfunctions of the sensor device 10 by using a temperature sensor to detect abnormal heat generation in the imaging unit 140 or light source unit 150 inside the sensor device 10, or in a driving unit outside the sensor device 10.

[0085] For example, the sensor device 10 according to the fifth modification may further be provided with an illuminance sensor that detects the brightness of the external space. In this case, the sensor device 10 can further improve the detection sensitivity of the light spot group of the reflected light that has been multiple-reflected by the first mirror 161 and the second mirror 162 by changing the brightness of the light emitted from the light source unit 150 according to the brightness of the external space.

[0086] For example, the sensor device 10 according to the fifth modification may further be provided with an IMU (Inertial Measurement Unit) that detects three-dimensional acceleration and angular velocity. In such a case, the sensor device 10 can calculate the attitude or motion state of the sensor device 10 based on the three-dimensional acceleration and angular velocity detected by the IMU. Therefore, the sensor device 10 can detect an external force acting on the force sense effect unit 120, taking into account the influence of the attitude or motion state of the sensor device 10.

[0087] For example, the sensor device 10 according to the fifth modification may further be provided with an illumination unit (not shown) that illuminates the external space. In this case, the sensor device 10 can illuminate the external space with the illumination unit, thereby enabling a clearer captured image of the external space to be acquired. The illumination unit that illuminates the external space may emit light of a wavelength band different from the wavelength band of light emitted by the light source unit 150. For example, the illumination unit and the light source unit 150 may emit light of different colors. In this way, the sensor device 10 can prevent interference between the light emitted from the illumination unit to illuminate the external space and the reflected light of the light emitted from the light source unit 150, which would otherwise cause noise.

[0088] <Second embodiment (sensor device)> The configuration of a sensor device according to a second embodiment of the present disclosure will be described with reference to Figs. 17 to 20. Fig. 17 is a perspective view schematically showing the configuration of a sensor device 100 according to this embodiment. Fig. 18 is an exploded view schematically showing the configuration of a sensor device 100 according to this embodiment. Fig. 19 is a perspective view showing attachment of a force detection structure 131 to a second exterior part 192. Fig. 20 is a schematic diagram showing an example of an image captured by an imaging unit 140 of the sensor device 100.

[0089] As shown in Figures 17 and 18, the sensor device 100 includes a first exterior part 191, a second exterior part 192, a force detection structure 131, a first mirror 161, a second mirror 162, a base part 110, an imaging part 140, a light source part 150, and a third exterior part 193.

[0090] The first exterior part 191 is part of the housing of the sensor device 100. Specifically, the first exterior part 191 is a structural member that covers the top surface of the sensor device 100 and two opposing side surfaces of the sensor device 100. The top surface of the first exterior part 191 is provided with an opening 191H for exposing the force sense action part 120 of the force sense detection structure 131 to the outside, and an external light intake port 100H for taking in external light incident on the imaging part 140.

[0091] The second exterior part 192 is part of the housing of the sensor device 100. Specifically, the second exterior part 192 is a structural member that covers the bottom surface of the sensor device 100 and two opposing side surfaces of the sensor device 100. The second exterior part 192 can form a housing having a rectangular parallelepiped shape by fitting with the first exterior part 191. An opening is provided on the bottom surface of the second exterior part 192 to guide external light that has entered through the external light inlet 100H to the imaging unit 140.

[0092] The force sense detection structure 131 is a structure including the force sense action section 120 and a strain body 130. As shown in Fig. 19, the force sense detection structure 131 is attached to two opposing side surfaces of the second exterior part 192 via the strain body 130. By attaching the force sense detection structure 131 to the two opposing side surfaces of the second exterior part 192, it becomes easier to balance deformation in the X, Y and Z axial directions compared to when the force sense detection structure 131 is attached to the underside of the second exterior part 192. This allows the force sense detection structure 131 to have a greater degree of freedom in structural design.

[0093] The force sense acting section 120 is provided as a convex section and is exposed to the outside through an opening 191H provided in the first exterior section 191. The strain body 130 is provided extending outwards from the lower part of the force sense acting section 120 to both sides, and the force sense acting section 120 is attached to two opposing side surfaces of the second exterior section 192. The strain body 130 may be configured with a low rigidity structure so that it is more easily deformed than other structures. The force sense detection structure 131 can deform the strain body 130 below the force sense acting section 120 when an external force acts on the force sense acting section 120.

[0094] Furthermore, the force-sense action unit 120 may be provided so that the gap between the force-sense action unit 120 and the outer edge of the opening 191H of the first exterior part 191 is smaller than the allowable deformation amount of the flexure body 130. In this way, the force-sense detection structure 131 can limit the deformation amount of the flexure body 130 by the size of the opening 191H of the first exterior part 191, thereby preventing excessive deformation load from being applied to the flexure body 130. Note that an elastic member such as rubber or elastomer may be sandwiched in the gap between the force-sense action unit 120 and the outer edge of the opening 191H of the first exterior part 191 to improve waterproofing.

[0095] The first mirror 161 is provided on the underside of the second exterior part 192, and the second mirror 162 is provided on the strain body 130 of the force detection structure 131 so as to face the first mirror 161. As shown in FIG. 19 , the first mirror 161 and the second mirror 162 are provided at positions corresponding to the respective side surfaces of the triangular prism, so that the light emitted from the light source part 150 can be multiple-reflected in the reflection space 121 between the first mirror 161 and the second mirror 162.

[0096] The base unit 110 is provided on the back surface of the second exterior unit 192, opposite to the bottom surface on which the first mirror 161 is provided. The base unit 110 is provided with a plurality of light source units 150 that emit light into the reflection space 121. The base unit 110 may also be provided with various sensors that sense information about the space surrounded by the first exterior unit 191 and the second exterior unit 192. For example, the base unit 110 may be provided with various sensors such as a temperature sensor, an illuminance sensor, or a microphone. These various sensors can sense information about the space surrounded by the first exterior unit 191 and the second exterior unit 192 through openings separately provided in the second exterior unit 192.

[0097] The light source unit 150 emits light into a space surrounded by a first exterior unit 191 and a second exterior unit 192. Specifically, a plurality of light source units 150 are provided at positions corresponding to the force detection structures 131, and emit light into a reflection space 121 surrounded by a first mirror 161 and a second mirror 162. By emitting light into the reflection space 121, the light source unit 150 can cause the emitted light to be multiple-reflected in the reflection space 121 between the first mirror 161 and the second mirror 162.

[0098] The imaging unit 140 is provided on the surface of the base unit 110 opposite to the surface on which the light source unit 150 is provided, and captures an image of the space surrounded by the first exterior unit 191 and the second exterior unit 192. As a result, the imaging unit 140 can capture an image of the external space of the sensor device 100 through the external light inlet 100H, and can also capture an image of a light spot group of reflected light that is emitted from the light source unit 150 and is multiply reflected by the first mirror 161 and the second mirror 162. The image captured by the imaging unit 140 is output to the outside via a connection unit 145 that is configured, for example, by a flexible printed circuit board or a flexible cable.

[0099] The third exterior part 193 is a part of the housing of the sensor device 100. Specifically, the third exterior part 193 can hold the base part 110 and the imaging part 140 by sandwiching the base part 110 and the imaging part 140 between the third exterior part 193 and the second exterior part 192.

[0100] In the sensor device 100 having the above configuration, when an external force acts on the force sense action unit 120, the strain element 130 of the force sense detection structure 131 is deformed, and the position of the second mirror 162 attached to the strain element 130 changes. This causes a displacement of the positions of each light spot of the reflected light that is multiple-reflected by the first mirror 161 and the second mirror 162. Therefore, the sensor device 100 can detect the external force acting on the force sense action unit 120 by detecting the displacement of the position of each light spot of the reflected light with the imaging unit 140. Furthermore, the sensor device 100 can capture an image of the external space visible through the external light inlet 100H with the imaging unit 140. Therefore, the sensor device 100 can simultaneously capture an image of the external space and detect an external force acting on the force sense action unit 120.

[0101] In the sensor device 100, a reflection space 121 where light emitted from the light source unit 150 is multiple-reflected is separated from a space where light from external space enters via the external light inlet 100H. Therefore, the imaging unit 140 captures an image 910 including a force-sense detection region 912 in which a light spot cloud LC of reflected light of light emitted from the light source unit 150 is captured, and an imaging region 911 in which the external space of the sensor device 100 is captured, as shown in Fig. 20. That is, in the image 910 captured by the imaging unit 140, the force-sense detection region 912 that detects an external force acting on the force-sense action unit 120 is separated from the imaging region 911 that captures the external space. Therefore, the sensor device 100 can suppress interference between the reflected light of light emitted from the light source unit 150 and the light that enters from external space via the external light inlet 100H.

[0102] (First Modification) 21 is a cross-sectional view showing the configuration of a sensor device 100 according to a first modification. As shown in FIG. 21, the sensor device 100 may further include a sub-mirror 162S in addition to the first mirror 161 and the second mirror 162.

[0103] The sub-mirrors 162S are provided, for example, on both sides of the external light inlet 100H. As an example, the sub-mirrors 162S may be provided on both sides of the external light inlet 100H so that the end faces do not enter the area where the external space is imaged.

[0104] As another example, the sub-mirror 162S may have an end surface cut parallel to the ray direction of light incident on the imaging unit 140 from the external space, and may be provided on both sides of the external light inlet 100H. In such a case, the sub-mirror 162S can prevent the image of the external space captured by the imaging unit 140 from being distorted by the end surface of the sub-mirror 162S. Furthermore, the cut end surface of the sub-mirror 162S may be painted a dark color to prevent reflection of external light incident from the external space. As described above, the dark color is, for example, an achromatic color with a brightness of 30% or less in the HSV color space.

[0105] The sub-mirror 162S reflects the light that has been multiply reflected by the first mirror 161 and the second mirror 162, and can guide the multiply reflected light to the imaging unit 140. At this time, the area where the imaging unit 140 images the sub-mirror 162S becomes the force detection area 912 in Fig. 20, and the area where the imaging unit 140 images the external space between the sub-mirrors 162S becomes the imaging area 911 in Fig. 20.

[0106] Therefore, the sensor device 100 according to the first modification can more clearly separate the force sense detection area 912 that detects an external force acting on the force sense action unit 120 from the imaging area 911 that images the external space. This makes it possible for the sensor device 100 according to the first modification to prevent interference between the reflected light of the light emitted from the light source unit 150 and the light that enters from the external space via the external light inlet 100H.

[0107] (Second Modification) 22 is a perspective view showing the configuration of sensor device 100 according to the second modification, excluding first exterior part 191. As shown in FIG. 22, sensor device 100 may further include various sensors such as microphone 113 and close-up light source 112.

[0108] The close-up photography light source 112 is a light source that illuminates the space surrounded by the first exterior part 191 and the second exterior part 192. The close-up photography light source 112 is provided on both sides of the base part 110 across the imaging part 140, and may illuminate the space surrounded by the first exterior part 191 and the second exterior part 192 through an opening 192H on the bottom surface of the second exterior part 192.

[0109] With this, for example, when an object is close to the sensor device 100 and external light is unlikely to enter the space enclosed by the first exterior part 191 and the second exterior part 192, the close-up photography light source 112 can illuminate the space enclosed by the first exterior part 191 and the second exterior part 192. Therefore, even when external light is unlikely to enter the space enclosed by the first exterior part 191 and the second exterior part 192, the sensor device 100 can obtain a clear image of the object with the imaging part 140.

[0110] Furthermore, the close-up photography light source 112 may emit light in a wavelength band different from the wavelength band of the light emitted by the light source unit 150. In other words, the close-up photography light source 112 and the light source unit 150 may emit light of different colors. For example, if the light source unit 150 emits green or blue laser light, the close-up photography light source 112 may emit white light. This allows the sensor device 100 to prevent the illumination light from the close-up photography light source 112 from becoming noise in the reflected light of the light emitted from the light source unit 150.

[0111] The microphone 113 detects sound in the space surrounded by the first exterior part 191 and the second exterior part 192. The microphone 113 may be provided on the base part 110, and may detect sound in the space surrounded by the first exterior part 191 and the second exterior part 192 through an opening 192H on the bottom surface of the second exterior part 192. In this way, the sensor device 100 can detect or predict a failure or the like of the sensor device 100 by detecting abnormal sound from the imaging part 140, the light source part 150, the force detection structure 131, or the like with the microphone 113.

[0112] 3. Third embodiment (gripping device) (3.1. Configuration) Next, the configuration of a gripping device according to a third embodiment of the present disclosure will be described with reference to FIGS.

[0113] The gripping device according to the third embodiment is a so-called gripper in which a swingable or parallel-movable claw is provided on the force sense application unit 120 of the sensor device 10 according to the first embodiment. The gripping device according to the third embodiment can simultaneously capture an image of an object to be gripped and detect the force sense when the object is gripped by the sensor device 10 according to the first embodiment.

[0114] (First configuration example) Fig. 23 is a cross-sectional view schematically showing a first configuration example of the gripping device 21. Fig. 24 is a top view schematically showing the configuration of the gripping device 21 shown in Fig. 23. Fig. 25 is an explanatory diagram showing the configuration of an image captured by the imaging unit 140 of the gripping device 21 shown in Fig. 23.

[0115] 23 and 24, in the gripping device 21, a drive unit 210a and a claw unit 220a are provided on the first force sense action unit 120a of the sensor device 10, and a drive unit 210b and a claw unit 220b are provided on the second force sense action unit 120b. The gripping device 21 functions as a gripper that grips an object with the two claw units 220a and 220b.

[0116] The two force sense action units 120a and 120b are arranged at an angle of 180 degrees around the light intake hole 110H. The driving units 210a and 210b are configured by, for example, motors. The driving unit 210a is provided on the force sense action unit 120a, and the driving unit 210b is provided on the force sense action unit 120b. b The two drive units 210a and 210b can swing the claw units 220a and 220b left and right independently of each other or in conjunction with each other.

[0117] The claws 220a and 220b are rigid, elongated structural members that are swung by the drive units 210a and 210b to clamp an object. When the claws 220a and 220b grip an object, an external force is applied to the force-sense operation units 120a and 120b of the sensor device 10. For example, an external force applied to the force-sense operation unit 120a is detected by the sensor device 10 as a displacement of the position of each light spot of the reflected light that is emitted from the light source unit 150a into the reflection space 121a and multiple-reflected by the first mirror 161a and the second mirror 162a. Similarly, an external force applied to the force-sense operation unit 120b is detected by the sensor device 10 as a displacement of the position of each light spot of the reflected light that is emitted from the light source unit 150b into the reflection space 121b and multiple-reflected by the first mirror 161b and the second mirror 162b. This allows the gripping device 21 to detect the force sense applied to each of the claws 220a and 220b in a multi-axial and independent manner. Also, the gripping device 21 can capture an image of the object held between the claws 220a and 220b with the imaging unit 140 via the half mirror 170 and the light intake hole 110H.

[0118] For example, the imaging unit 140 of the gripping device 21 acquires an image CI including an imaging area IM, a first force sense detection area FD1, and a second force sense detection area FD2, as shown in Fig. 25. The first force sense detection area FD1 and the second force sense detection area FD2 are provided spaced apart from each other around the imaging area IM at approximately the center of the image CI, corresponding to the positions where the force sense action units 120a and 120b are provided.

[0119] The first force-sense detection region FD1 is a region where a group of light spots of reflected light whose positions are displaced based on an external force acting on the force-sense action unit 120a is captured. The second force-sense detection region FD2 is a region where a group of light spots of reflected light whose positions are displaced based on an external force acting on the force-sense action unit 120b is captured. The imaging region IM is a region where only the image of the object that has passed through the half mirror 170 is captured.

[0120] According to this, the imaging section 140 of the gripping device 21 can acquire an image CI that is structurally divided into a first force sense detection region FD1 where a light spot group for detecting an external force acting on the force sense action section 120a is imaged, and a second force sense detection region FD2 where a light spot group for detecting an external force acting on the force sense action section 120b is imaged. Also, the imaging section 140 of the gripping device 21 can acquire an image CI that includes an imaging region IM where only the image of the target is imaged, and where the light spot group for detecting the external force acting on each of the force sense action sections 120a, 120b is not imaged.

[0121] Therefore, the gripping device 21 according to the first configuration example can grip an object while capturing an image of the object, and can detect the force sensation when gripping the object, thereby enabling the object to be gripped with greater precision.

[0122] (Second configuration example) Fig. 26 is a cross-sectional view schematically showing a second configuration example of the gripping device 22. Fig. 27 is a top view schematically showing the configuration of the gripping device 22 shown in Fig. 26. Fig. 28 is an explanatory diagram showing the configuration of an image captured by the imaging unit 140 of the gripping device 22 shown in Fig. 26.

[0123] 26 and 27, in the gripping device 22, a drive unit 210a and a claw unit 220a are provided on the first force sense action unit 120a of the sensor device 10, a drive unit 210b and a claw unit 220b are provided on the second force sense action unit 120b, and a drive unit 210c and a claw unit 220c are provided on the third force sense action unit 120c (not shown). The gripping device 22 functions as a gripper that grips an object with the three claw units 220a, 220b, and 220c.

[0124] The three force sense effect units 120a, 120b, and 120c are arranged at 120-degree angles around the light intake hole 110H. The drive units 210a, 210b, and 210c are formed, for example, by motors. The drive unit 210a is provided on the force sense effect unit 120a, the drive unit 210b is provided on the force sense effect unit 120b, and the drive unit 210c is provided on the force sense effect unit 120c. The three drive units 210a, 210b, and 210c can swing the claws 220a, 220b, and 220c back and forth independently of each other or in conjunction with each other.

[0125] The claws 220a, 220b, and 220c are rigid, elongated structural members that are swung by the drive units 210a, 210b, and 210c to clamp an object. When the claws 220a, 220b, and 220c grip an object, an external force is applied to each of the force-sense application units 120a, 120b, and 120c of the sensor device 10. For example, the external force applied to the force-sense application unit 120a is detected by the sensor device 10 as a displacement of the position of each light spot of the reflected light that is emitted from the light source unit 150a into the reflection space 121a and multiple-reflected by the first mirror 161a and the second mirror 162a. Furthermore, an external force acting on the force sense acting unit 120b is emitted from the light source unit 150b into the reflection space 121b, and is detected by the sensor device 10 as a displacement of the position of each light point of the reflected light that is multiple-reflected by the first mirror 161b and the second mirror 162b. Furthermore, an external force acting on the force sense acting unit 120c is emitted from the light source unit (not shown) into the reflection space, and is detected by the sensor device 10 as a displacement of the position of each light point of the reflected light that is multiple-reflected by the first mirror 161b and the second mirror 162b. This allows the gripping device 22 to detect the force sense acting on each of the claws 220a, 220b, 220c in a multi-axial and independent manner. Furthermore, the gripping device 22 detects ... light An image of an object held by the claws 220a, 220b, and 220c can be captured by the imaging unit 140 through the intake hole 110H.

[0126] For example, the imaging section 140 of the gripping device 22 acquires an image CI including an imaging area IM, a first force sense detection area FD1, a second force sense detection area FD2, and a third force sense detection area FD3, as shown in Fig. 28. The first force sense detection area FD1, the second force sense detection area FD2, and the third force sense detection area FD3 are provided spaced apart from one another around the imaging area IM in the approximate center of the image CI, corresponding to the positions where the force sense action units 120a, 120b, and 120c are provided.

[0127] The first force sense detection region FD1 is a region where an image of a group of light spots of reflected light whose position changes based on an external force acting on the force sense action unit 120a is captured. The second force sense detection region FD2 is a region where an image of a group of light spots of reflected light whose position changes based on an external force acting on the force sense action unit 120b is captured. The third force sense detection region FD3 is a region where an image of a group of light spots of reflected light whose position changes based on an external force acting on the force sense action unit 120c is captured. The imaging region IM is a region where only the image of the object that has passed through the half mirror 170 is captured.

[0128] According to this, the imaging section 140 of the gripping device 22 can acquire an image CI that is structurally divided into a first force sense detection region FD1 where a light spot group for detecting an external force acting on the force sense action unit 120a is imaged, a second force sense detection region FD2 where a light spot group for detecting an external force acting on the force sense action unit 120b is imaged, and a third force sense detection region FD3 where a light spot group for detecting an external force acting on the force sense action unit 120c is imaged. Also, the imaging section 140 of the gripping device 22 can acquire an image CI that includes an imaging region IM where only the image of the object is imaged, without superimposing the light spot groups for detecting the external forces acting on each of the force sense action units 120a, 120b, and 120c.

[0129] Therefore, the gripping device 22 according to the second configuration example can grip an object while capturing an image of the object, and can detect the force sensation when gripping the object, thereby enabling the object to be gripped with greater precision.

[0130] (Addendum) The gripping device according to the third embodiment may have four or more claws. For example, the gripping device may be configured with a sensor device 10 having four force sense action units 120 provided at positions corresponding to the vertices of a quadrangle, and a drive unit and a claw unit provided on each of the force sense action units 120.

[0131] The imaging unit 140 of such a gripping device can capture, for example, images CI shown in Figures 29 and 30. Figures 29 and 30 are schematic diagrams showing an example of an image captured by the imaging unit 140 of a gripping device having four claws.

[0132] For example, the imaging unit 140 of a gripping device having four claws can capture an image CI including an imaging area IM, a first force detection area FD1, a second force detection area FD2, a third force detection area FD3, and a fourth force detection area FD4, as shown in Figures 29 and 30.

[0133] The imaging area IM is an area in which the external space is captured, and is located at the center of the angle of view. The first force sense detection area FD1, the second force sense detection area FD2, the third force sense detection area FD3, and the fourth force sense detection area FD4 are areas in which light spot groups of reflected light whose positions are displaced based on external forces acting on each of the four force sense application units 120 are captured. The first force sense detection area FD1, the second force sense detection area FD2, the third force sense detection area FD3, and the fourth force sense detection area FD4 are located at positions corresponding to the vertices of the quadrangular image CI, corresponding to the positions where each of the four force sense application units 120 is provided.

[0134] As shown in Fig. 29, the first force detection region FD1, the second force detection region FD2, the third force detection region FD3, and the fourth force detection region FD4 may be provided as triangular regions at positions corresponding to the vertices of a quadrangular image CI. Also, as shown in Fig. 30, the first force detection region FD1, the second force detection region FD2, the third force detection region FD3, and the fourth force detection region FD4 may be provided as quadrangular regions at positions corresponding to the vertices of the quadrangular image CI.

[0135] The imaging unit 140 of a gripping device having four claws can capture an image CI that is structurally divided into a first force-sense detection area FD1, a second force-sense detection area FD2, a third force-sense detection area FD3, a fourth force-sense detection area FD4, and an imaging area IM. This allows the imaging unit 140 to acquire information about the external forces acting on each of the four force-sense application units 120 and an image of the external space with high accuracy without mutual interference. Therefore, a gripping device having four claws can grip an object while capturing an image and detecting the force of the object with lower noise, thereby enabling the object to be gripped with higher accuracy.

[0136] (3.2. Function) Next, the functional configuration of the gripping device according to this embodiment will be described with reference to Fig. 31. Fig. 31 is a block diagram illustrating the functional configuration of the gripping device according to this embodiment. Note that, hereinafter, the gripping device 21 according to the first configuration example and the gripping device 22 according to the second configuration example will be referred to as gripping device 20 without any distinction.

[0137] As shown in FIG. 31, the gripping device 20 includes an imaging unit 140, an image processing unit 141, a force sense calculation unit 142, a light source control unit 151, a light source unit 150, a recognition unit 230, a memory unit 240, a drive control unit 211, and a drive unit 210.

[0138] As described above, the imaging unit 140 acquires a captured image by receiving external light that has passed through the half mirror 170 and a group of light spots of reflected light that has been multiple-reflected by the first mirror 161 and the second mirror 162.

[0139] The image processing unit 141 performs image processing on the captured image acquired by the imaging unit 140. For example, the image processing unit 141 may process the captured image acquired by the imaging unit 140 to generate an image including an object generated from external light transmitted through the half mirror 170. Furthermore, if a temperature sensor is further provided, the image processing unit 141 may perform temperature compensation on the captured image acquired by the imaging unit 140 based on the sensing result of the temperature sensor. If an IMU is further provided, the image processing unit 141 may perform gravity compensation on the captured image acquired by the imaging unit 140 based on the sensing result of the IMU.

[0140] The force sense calculation unit 142 calculates the magnitude and direction of an external force acting on the force sense action unit 120 based on the displacement of the position of each light point of the reflected light that has been multiple-reflected by the first mirror 161 and the second mirror 162. Specifically, the force sense calculation unit 142 may calculate the magnitude and direction of the external force acting on the force sense action unit 120 based on the correspondence relationship between the displacement of the position of each light point of the reflected light and the magnitude and direction of the external force acting on the force sense action unit 120. The correspondence relationship between the displacement of the position of each light point of the reflected light and the magnitude and direction of the external force acting on the force sense action unit 120 can be constructed in advance using, for example, a mathematical model or machine learning.

[0141] As described above, the light source unit 150 emits light into the reflection space 121 surrounded by the first mirror 161 and the second mirror 162, and the light source control unit 151 controls the brightness and light emission timing of the light emitted from the light source unit 150. For example, the light source control unit 151 may control the light emission timing of the light emitted from the light source unit 150 in accordance with the frame rate of image capture by the image capture unit 140. Furthermore, the light source control unit 151 may control the brightness of the light emitted from the light source unit 150 based on the brightness of the external space sensed by an illuminance sensor.

[0142] The recognition unit 230 recognizes the external space of the gripping device 20 by performing image recognition on the image captured by the imaging unit 140. The recognition unit 230 may recognize the external space of the gripping device 20 by performing image recognition on the image captured by the imaging unit 140 using machine learning.

[0143] The storage unit 240 stores various information used for control by the drive control unit 211. Specifically, the storage unit 240 may store images captured by the imaging unit 140, information about the external space of the gripping device 20 recognized by the recognition unit 230, information about the external force acting on the force sense application unit 120 calculated by the force sense calculation unit 142, and the like.

[0144] As described above, the drive unit 210 grips an object by swinging the claws 220 of the gripping device 20, and the drive control unit 211 controls the drive of the drive unit 210. Specifically, the drive control unit 211 may control the drive of the drive unit 210 using information about the external space of the gripping device 20 recognized by the recognition unit 230, information about the external force acting on the force sense application unit 120 calculated by the force sense calculation unit 142, and the like.

[0145] The gripping device 20 having the above functional configuration can simultaneously sense the external space of the gripping device 20 and the external force acting on the force sense application unit 120, and can grip an object based on the sensing results. Therefore, the gripping device 20 according to this embodiment can capture an image of the external space and detect the force sense with a smaller and simpler mechanism, and can therefore grip an object with higher precision.

[0146] <4. Application Examples> Next, application examples of the sensor device 10 according to the first embodiment and the gripping device 20 according to the third embodiment will be described with reference to FIGS.

[0147] (First Application Example of Sensor Device 10) 32 is an explanatory diagram showing an input device 31 which is a first application example of the sensor device 10 according to the first embodiment. As shown in Fig. 32, the input device 31 is an input device which allows input of a rotation angle with three degrees of freedom by manipulating a spherical structure with a finger. The input device 31 includes an outer shell 311 provided with an opening 312, and the sensor device 10.

[0148] Outer shell 311 has a hollow spherical structure. At least one of an acceleration sensor, an angular velocity sensor, and a magnetic sensor is mounted near the center of outer shell 311 to detect the rotation angle of the spherical structure of outer shell 311. A user can input a rotation angle with three degrees of freedom to input device 31 by inserting a finger into opening 312 and rotating outer shell 311.

[0149] The sensor device 10 supports the outer shell part 311 with the force sense acting part 120, and captures an image of the surface of the outer shell part 311 with the imaging part 140. A pattern is provided on the surface of the outer shell part 311, and the sensor device 10 can derive the absolute angle of the outer shell part 311 based on the pattern provided on the surface of the outer shell part 311.

[0150] According to this, the input device 31 can detect the rotation angle of three degrees of freedom using the sensor mounted on the outer shell part 311, and can also detect the force applied to the outer shell part 311 and the absolute angle of the outer shell part 311 using the sensor device 10. Therefore, the input device 31 can receive input of more detailed information from the user.

[0151] (Second Application Example of Sensor Device 10) Fig. 33 is a perspective view showing a force detection device 32 which is a second application example of the sensor device 10 according to the first embodiment. Fig. 34 is a longitudinal sectional view of the force detection device 32 shown in Fig. 33. As shown in Figs. 33 and 34, the force detection device 32 includes a touch panel 321, a lens 322, and the sensor device 10.

[0152] The sensor device 10 is embedded inside the touch panel 321. The touch panel 321 is a flat board, and is fixed to the force sense action unit 120 of the sensor device 10. This allows the sensor device 10 to detect contact with the touch panel 321 as an external force acting on the force sense action unit 120. The lens 322 is provided on the half mirror 170 of the sensor device 10, and focuses external light incident on the sensor device 10 onto the imaging unit 140. The lens 322 may be a fisheye lens.

[0153] This allows the force detection device 32 to detect contact with the touch panel 321 in advance by capturing an image of the external space using the sensor device 10. Furthermore, the force detection device 32 can detect contact with the touch panel 321 with low noise by force detection using the sensor device 10.

[0154] (Third Application Example of Sensor Device 10) Fig. 35 is a diagram illustrating lens devices 33A, 33B, and 33C, which are a third application example of the sensor device 10 according to the first embodiment. As shown in Fig. 35, the lens devices 33A, 33B, and 33C are lens mount mechanisms that can configure the sensor device 10 according to this embodiment by combining them with any of the image sensors 40A, 40B, and 40C.

[0155] For example, the lens device 33A includes a housing 331A including the base portion 110 and the force sense action portion 120, and a fisheye lens 332A provided convexly relative to the housing 331A. The lens device 33B includes a housing 331B including the base portion 110 and the force sense action portion 120, and a normal lens 332B provided concavely relative to the housing 331B. The lens device 33C includes a housing 331C including the base portion 110 and the force sense action portion 120, a lens 332C provided in the housing 331C, and a claw portion 333C constituting a gripper provided on the housing 331C.

[0156] The lens devices 33A, 33B, and 33C may be provided so as to be attachable to various image sensors by sharing a common connection mechanism with the image sensors 40A, 40B, and 40C. In this way, the lens devices 33A, 33B, and 33C can be retrofitted to existing image sensors, thereby enabling the existing image sensors to be used to capture images of the external space and detect forces.

[0157] (First application example of the gripping device 20) 36 is an explanatory diagram showing a robot device 34 which is a first application example of the gripping device 20 according to the third embodiment. As shown in FIG. 36, the robot device 34 is a humanoid robot device. The robot device 34 includes a main body unit 341, an arm unit 342, and an effector unit 343.

[0158] The main body 341 is the torso and head of the humanoid robot device, and includes, for example, a power supply, a control device, and an imaging device that captures images of the external space. The arm 342 is the arm of the humanoid robot device, and is configured with a link mechanism in which links are connected to each other by rotatable joints. The effector 343 is provided at the tip of the arm 342, and is configured to include the gripping device 20 according to this embodiment.

[0159] By applying the gripping device 20 according to this embodiment to the effector unit 343, the robot device 34 can simultaneously capture images of the object to be gripped using the imaging device provided in the main body unit 341 and the gripping device 20. Therefore, the robot device 34 can check with higher accuracy using the gripping device 20 whether the object is being manipulated as intended. For example, when rotating an object such as a doorknob using the effector unit 343, the robot device 34 can check, using the captured image, whether the object is following the rotation of the effector unit 343. This allows the robot device 34 to grip the object while checking the image captured by the gripping device 20, thereby making it possible to grip various objects in a more stable manner.

[0160] (Second Application Example of the Grip Device 20) 37 and 38 are explanatory diagrams showing robot arm devices 35A and 35B which are a second application example of the gripping device 20 according to the third embodiment.

[0161] As shown in Fig. 37, robot arm device 35A is a single-arm robot arm device equipped with a two-jaw gripper. Robot arm device 35A includes arm unit 351 and effector unit 352A. As shown in Fig. 38, robot arm device 35B is a single-arm robot arm device equipped with a three-jaw gripper. Robot arm device 35B includes arm unit 351 and effector unit 352B.

[0162] Arm 351 is configured with a link mechanism in which links are connected to each other by rotatable joints. Effector unit 352A is provided at the tip of arm 351 and is configured to include gripping device 21 according to this embodiment. Effector unit 352B is provided at the tip of arm 351 and is configured to include gripping device 22 according to this embodiment.

[0163] By applying the gripping device 20 according to this embodiment to the effector units 352A and 352B, the robot arm device 35 can grip an object while capturing an image with the gripping device 20. Furthermore, the robot arm device 35 can detect the gripping force applied to the object with the gripping device 20, so it can grip the object in a more stable state.

[0164] (Third Application Example of the Grip Device 20) 39 is an explanatory diagram showing a moving body 36 which is a third application example of the gripping device 20 according to the third embodiment. As shown in Fig. 39, the moving body 36 is a drone that can move and stand still in the air by remote control. The moving body 36 includes a main body 361, an arm 362, and an effector 363.

[0165] The main body 361 is a main part of the moving body 36 and includes a power supply, a control device, an imaging device that captures images of the external space, and a flight mechanism. The main body 361 may be, for example, a rotary-wing aircraft equipped with three or more rotors (a so-called multicopter). The arm 362 is provided on the main body 361 and is configured with a link mechanism in which links are connected to each other by rotatable joints. The effector 363 is provided at the tip of the arm 362 and is configured to include the gripping device 20 according to this embodiment.

[0166] By applying the gripping device 20 according to this embodiment to the effector unit 363, the moving body 36 can capture an image of the object and detect the force sense using the gripping device 20, and therefore can accurately grip the object even under conditions of large vibrations. Furthermore, even under conditions where it is difficult for the imaging device mounted on the main body unit 361 to capture an image of the object because it is blocked by the arm unit 362, the moving body 36 can accurately grip the object because the gripping device 20 can capture an image of the object.

[0167] Although the preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, the technical scope of the present disclosure is not limited to such examples. It is clear that a person skilled in the art of the present disclosure can conceive of various modified or altered examples within the scope of the technical idea described in the claims, and it is understood that these also naturally fall within the technical scope of the present disclosure.

[0168] For example, in the above embodiment, the sensor device 10 and the gripping device 20 are described as including the imaging unit 140, but the technology according to the present disclosure is not limited to the above example. The sensor device 10 and the gripping device 20 may receive light incident on the light intake hole 110H using an image sensor that is attached later or separately. Even in such a case, it is possible to capture an image of the external space and detect a force from the image captured by the image sensor.

[0169] Furthermore, the effects described herein are merely descriptive or exemplary and are not limiting. In other words, the technology according to the present disclosure may achieve other effects that will be apparent to those skilled in the art from the description of this specification, in addition to or in place of the above-described effects.

[0170] The following configurations also fall within the technical scope of the present disclosure. (1) a force sense acting unit that is exposed through an opening provided in the exterior part and is attached to the interior of the exterior part via a strain generating body; a reflection space surrounded on at least two sides by a first mirror provided on the inner bottom surface of the exterior part and a second mirror provided on a surface of the force sense action part or the strain generating body facing the first mirror; a light source unit that emits light into the reflection space; an imaging unit provided on the inner bottom surface of the exterior unit that captures an image including a force detection area in which reflected light of light emitted from the light source unit is reflected; A sensor device comprising: (2) The sensor device according to (1), wherein the reflection space is surrounded by the first mirror and the second mirror on at least three sides. (3) The sensor device according to (2), wherein the first mirror and the second mirror are provided opposite to each other so as to form side surfaces of a triangular prism, a quadrangular prism, a triangular pyramid, or a quadrangular pyramid. (4) The sensor device according to any one of (1) to (3), wherein a group of light spots of the reflected light reflected in the reflection space is projected onto the force detection area, and an external force acting on the force sense acting portion is calculated from the displacement of the group of light spots. (5) The sensor device according to (4), wherein an external force acting on the force sense action unit is calculated from the displacement of the light point cloud using machine learning. (6) The sensor device according to any one of (1) to (5), wherein the force sense action portion is attached to an inner side surface of the exterior portion via the strain generating body. (7) The sensor device according to any one of (1) to (6), wherein the emission of the light from the light source section is controlled in accordance with an imaging frame rate of the imaging section. (8) The sensor device according to any one of (1) to (7), wherein a plurality of the light source sections are provided corresponding to the force sense action sections. (9) The sensor device according to any one of (1) to (8), wherein the image captured by the imaging unit further includes an imaging area in which the external space of the exterior unit is captured. (10) The sensor device according to (9), wherein the force sense detection areas are provided at positions corresponding to the force sense action portions with respect to the imaging area. (11) The sensor device according to (9) or (10), wherein the first mirror has an end face cut parallel to a ray direction of light incident on the imaging unit from the external space. (12) The sensor device according to (11), wherein the end face of the first mirror is a black-based color. (13) The sensor device according to any one of (1) to (12), wherein the light source unit emits the light into the reflection space through a pinhole. (14) The sensor device according to any one of (1) to (13), wherein the light source unit is provided on the inner bottom surface. (15) The sensor device according to any one of (1) to (14), wherein the imaging unit detects light included in a wavelength band of visible light and light included in a wavelength band of infrared light. (16) the light source unit emits light included in the infrared wavelength band, The sensor device described in (15), wherein the imaging unit captures an image including a force detection area in which the reflected light of light included in the infrared wavelength band is captured, and an imaging area in which the external space in which light of the infrared wavelength band is reduced is captured. (17) The sensor device according to any one of (1) to (16), wherein the imaging unit includes a ranging pixel. (18) The sensor device according to any one of (1) to (17), wherein a gap between the outer edge of the opening provided in the exterior part and the force-sense acting part is smaller than an allowable deformation amount of the strain body. (19) The sensor device according to any one of (1) to (18), further comprising a close-up photography light source provided on the inner bottom surface. (20) The sensor device according to (19), wherein the close-up photography light source and the light source unit emit the light in different wavelength bands.

[0171] Furthermore, the following configurations also fall within the technical scope of the present disclosure. (1) a base portion provided with a light intake hole for taking in external light; a force-sense acting portion provided on the base portion around the light intake hole via a strain generating body; a reflection space surrounded on at least two sides by mirrors provided on opposing surfaces of the base portion and the force sense action portion; a light source unit that emits light into the reflection space; a half mirror provided on the incident surface side of the external light, which transmits a part of the external light and reflects a part of the light emitted from the light source unit or a reflected light of the light; An optical device comprising: (2) The optical device according to (1) above, wherein the reflection space is surrounded by the mirror on at least three sides. (3) The optical device according to (2) above, wherein the mirrors are arranged opposite each other to form the sides of a triangular prism, a quadrangular prism, a triangular pyramid, or a quadrangular pyramid. (4) An optical device described in any one of (1) to (3) above, wherein in an image captured through the light intake hole, a group of light points of the light reflected in the reflection space are superimposed on an image of the external space captured by the external light. (5) The optical device according to (4) above, wherein an external force acting on the force sense application unit is calculated from the displacement of the light spot group. (6) The optical device according to (5) above, wherein the external force acting on the force sense application unit is calculated from the displacement of the light point cloud using machine learning. (7) The optical device according to any one of (4) to (6), wherein in an image captured through the light capture hole, the light spot group is superimposed on the captured image of the area corresponding to the force sense action portion. (8) The optical device according to any one of (4) to (7) above, wherein the emission of the light from the light source section is controlled in accordance with a frame rate of an image captured via the light intake hole. (9) The optical device according to any one of (1) to (8) above, wherein two or more of the force-sense action portions are provided around the light intake hole in point symmetry or line symmetry. (10) The optical device according to (9) above, wherein the half mirror is connected to each of the force-sense acting portions via a strain generating body so as to cover the light intake hole. (11) The optical device according to any one of (1) to (10) above, wherein the half mirror has a rectangular flat plate shape. (12) The optical device according to (9) or (10) above, wherein each of the force sense acting sections is connected to a corresponding one of the claws of a gripper. (13) The optical device according to any one of (1) to (12) above, wherein the light source unit emits the light into the reflection space through a pinhole. (14) The optical device according to (13) above, wherein the pinhole is a conical hole tapered forward toward the reflection space. (15) The optical device according to any one of (1) to (14) above, wherein the light source section is provided on the base section. (16) The light source unit is provided in plurality, The optical device according to any one of (1) to (15) above, wherein the plurality of light source sections emit the light of the same color or different colors. (17) The force sense action unit is provided in plurality, The optical device according to (16) above, wherein each of the light source units is provided corresponding to each of the force sense action units. (18) The optical device according to any one of (1) to (17) above, further comprising an exterior part that is cylindrical or rectangular tubular and is provided on the outer edge of the base part, and that houses the force sense acting part and the reflection space inside. (19) a base portion provided with a light intake hole for taking in external light; a force-sense acting portion provided on the base portion around the light intake hole via a strain generating body; a reflection space surrounded on at least two sides by mirrors provided on opposing surfaces of the base portion and the force sense action portion; a light source unit that emits light into the reflection space; a half mirror provided on the incident surface side of the external light, which transmits a part of the external light and reflects a part of the light emitted from the light source unit or a reflected light of the light; an imaging unit that captures an image in which a light spot group of the light reflected in the reflection space and an image captured by the external light are superimposed via the light intake hole; A sensor device comprising: [Explanation of symbols]

[0172] 10, 11, 100 Sensor device 110 Base 110H Light intake hole 111 Protrusion 120 Force sensation acting part 121 Reflection space 130,180 Strain body 131 Force detection structure 140 Imaging unit 141 Image processing section 142 Force Sense Calculation Unit 150 Light source section 150H Pinhole 151 Light source control unit 161 1st Mirror 162 Second Mirror 170 Half Mirror 191 First Exterior Section 192 Second Exterior Section 192 Third Exterior Section 20,21,22 Gripping device 210 Drive unit 211 Drive control unit 220 Claw 230 Recognition part 240 Storage section CI,CIX images LC1,LC2 light point group

Claims

1. a force sense acting unit that is exposed through an opening provided in the exterior part and is attached to the interior of the exterior part via a strain generating body; a reflection space surrounded on at least two sides by a first mirror provided on the inner bottom surface of the exterior part and a second mirror provided on a surface of the force sense action part or the strain generating body facing the first mirror; a light source unit that emits light into the reflection space; an imaging unit provided on the inner bottom surface of the exterior unit that captures an image including a force detection area in which reflected light of light emitted from the light source unit is reflected; A sensor device comprising:

2. The sensor device according to claim 1 , wherein the reflection space is surrounded by the first mirror and the second mirror on at least three sides.

3. The sensor device according to claim 2 , wherein the first mirror and the second mirror are provided opposite each other so as to form side surfaces of a triangular prism, a quadrangular prism, a triangular pyramid, or a quadrangular pyramid.

4. 2. The sensor device according to claim 1, wherein a group of light points of the reflected light reflected in the reflection space is projected onto the force detection area, and an external force acting on the force sense acting portion is calculated from the displacement of the group of light points.

5. The sensor device according to claim 4 , wherein the external force acting on the force sense application unit is calculated from the displacement of the light point cloud using machine learning.

6. The sensor device according to claim 1 , wherein the force sense acting portion is attached to an inner side surface of the exterior portion via the strain generating element.

7. The sensor device according to claim 1 , wherein the emission of the light from the light source unit is controlled in accordance with an imaging frame rate of the imaging unit.

8. The force sense acting portion is provided in plurality, The sensor device according to claim 1 , wherein the light source unit is provided corresponding to each of the plurality of force sense application units.

9. The sensor device according to claim 1 , wherein the image captured by the imaging unit further includes an imaging area in which an external space of the exterior unit is captured.

10. The force sense acting portion is provided in plurality, The sensor device according to claim 9 , wherein the force sense detection area is provided at a position corresponding to each of the plurality of force sense application portions with respect to the imaging area.

11. The sensor device according to claim 9 , wherein the first mirror has an end surface cut parallel to a ray direction of light incident on the imaging unit from the external space.

12. The sensor device according to claim 11 , wherein the end surface of the first mirror is a black-based color.

13. The sensor device according to claim 1 , wherein the light source unit emits the light into the reflection space through a pinhole.

14. The sensor device according to claim 1 , wherein the light source unit is provided on the inner bottom surface.

15. The sensor device according to claim 1 , wherein the imaging unit detects light included in a wavelength band of visible light and light included in a wavelength band of infrared light.

16. the light source unit emits light included in the infrared wavelength band, The sensor device according to claim 15, wherein the imaging unit captures an image including a force detection region in which the reflected light of light included in the infrared wavelength band is captured, and an imaging region in which an external space in which light of the infrared wavelength band is reduced is captured.

17. The sensor device according to claim 1 , wherein the imaging unit includes a ranging pixel.

18. 2. The sensor device according to claim 1, wherein a gap between an outer edge of the opening provided in the exterior portion and the force-sense acting portion is smaller than an allowable deformation amount of the strain-generating body.

19. The sensor device according to claim 1 , further comprising a close-up photography light source on the inner bottom surface.

20. The sensor device according to claim 19 , wherein the close-up imaging light source and the light source unit emit the light in different wavelength bands.

Images