Imaging device and imaging method

Abstract

To accurately measure a distance to an object using laser diode light having different wavelengths.SOLUTION: An imaging apparatus includes: a first light source that radiates first irradiation light, which is light having a first wavelength; a second light source that radiates second irradiation light, which is light having a second wavelength different from the first wavelength; a first detector that detects first reflected light, which is light reflected by irradiating an object with the first irradiation light; and a second detector that detects second reflected light, which is light reflected by irradiating the object with the second irradiation light. The second light source is arranged at a position closer to an optical axis than the first light source.SELECTED DRAWING: Figure 17

Description

Technical Field

[0001] The present invention relates to an imaging device and an imaging method.

Background Art

[0002] Conventionally, there has been a technique for measuring the distance to an object by irradiating the object with laser diode light having a predetermined wavelength, receiving the light reflected by the object, and analyzing the received light (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, a specific wavelength of the laser diode light used for distance measurement may be attenuated by the solar spectrum reaching the ground surface. Further, by using other wavelengths that are not the specific wavelength, attenuation by the solar spectrum can be avoided, but there are problems such as a decrease in transmittance and spectral sensitivity of the image sensor indoors where it is not affected by the solar spectrum. That is, according to the conventional technology, a suitable wavelength of the laser diode light used for distance measurement is different indoors and outdoors, so there are problems such as the inability to accurately measure the distance to an object if the measurement environment changes. To solve these problems, it is conceivable to accurately measure the distance to an object using laser diode light of different wavelengths. Here, for measurements at short distances, it is preferable to place the light source near the optical axis to reduce distance errors due to angular differences. On the other hand, when the light source is placed near the optical axis, there is a problem in that the illumination casts a shadow on the side of the object. In other words, when multiple light sources are used, depending on the arrangement, there is a problem in that the distance to the object cannot be measured accurately.

[0005] This invention has been made in view of the above circumstances, and aims to provide a technology that can accurately measure the distance to an object using laser diode light of different wavelengths. [Means for solving the problem]

[0006] An imaging device according to one aspect of the present invention comprises a plurality of first light sources that emit first irradiation light having a first wavelength, a plurality of second light sources that emit second irradiation light having a second wavelength different from the first wavelength, a first detection unit that detects first reflected light which is light reflected when the first irradiation light is irradiated onto an object, and a second detection unit that detects second reflected light which is light reflected when the second irradiation light is irradiated onto the object, wherein the plurality of first light sources are arranged on the circumference of a first circle centered on the optical axis, a portion of the plurality of second light sources are arranged on the circumference of a second circle which is a circle with a different radius from the first circle and is concentric with respect to the optical axis, and another portion of the plurality of second light sources are further arranged on the circumference of a third circle which is a circle with a different radius from both the first circle and the second circle and is concentric with respect to the optical axis. multiple The second light source some of It is positioned closer to the optical axis than the first light source. In addition, some of the other second light sources are positioned further from the optical axis than the first light source. .

[0007] Furthermore, in an imaging device according to one aspect of the present invention, both the first light source and the second light source are arranged on a plane that intersects the optical axis.

[0010] Furthermore, an imaging method according to one aspect of the present invention comprises: a first irradiation step of irradiating an object with a first irradiation light having a first wavelength using a plurality of first light sources; a second irradiation step of irradiating an object with a second irradiation light having a second wavelength different from the first wavelength using a plurality of second light sources; a first detection step of detecting a plurality of first reflected light, which is light reflected after the plurality of first irradiation lights have been irradiated onto an object; and a second detection step of detecting a plurality of second reflected light, which is light reflected after the plurality of second irradiation lights have been irradiated onto the object, wherein the plurality of first light sources are arranged on the circumference of a first circle centered on the optical axis; some of the plurality of second light sources are arranged on the circumference of a second circle, which is a circle with a different radius from the first circle and is concentric with respect to the optical axis; and other parts of the plurality of second light sources are further arranged on the circumference of a third circle, which is a circle with a different radius from both the first and second circles and is concentric with respect to the optical axis. multiple The second light source some of It is positioned closer to the optical axis than the first light source. In addition, some of the other second light sources are positioned further from the optical axis than the first light source. . [Effects of the Invention]

[0011] According to the present invention, the distance to an object can be measured with high accuracy using laser diode light of different wavelengths. [Brief explanation of the drawing]

[0012] [Figure 1] This is a diagram illustrating the schematic of the imaging device according to Embodiment 1. [Figure 2] This is a schematic diagram showing an example of a cross-section of the imaging device according to Embodiment 1. [Figure 3] This is a schematic diagram showing an example of a cross-section of an imaging device according to a modified example 1 of Embodiment 1. [Figure 4] This is a schematic diagram showing an example of a cross-section of an imaging device according to a modified example 2 of Embodiment 1. [Figure 5] This is a diagram illustrating an imaging device according to a modified example 3 of Embodiment 1. [Figure 6]This is a diagram for explaining the imaging device according to Modification Example 4 of Embodiment 1. [Figure 7] This is a diagram for explaining the effects when the number of pixels and the angle of view of the RGB sensor and the ToF sensor are the same in the imaging device according to Embodiment 1. [Figure 8] This is a diagram for explaining the effects when the number of pixels and the angle-of-view adjustment parameter of the RGB sensor and the ToF sensor are known in the imaging device according to Embodiment 1. [Figure 9] This is a diagram for explaining the sharing of distortion correction values in the imaging device according to Embodiment 1. [Figure 10] This is a diagram for explaining the sharing of peripheral light amount drop correction data in the imaging device according to Embodiment 1. [Figure 11] This is a diagram for explaining the sharing of chromatic aberration correction data in the imaging device according to Embodiment 1. [Figure 12] This is a diagram for explaining the interference of near-infrared light of two different wavelengths according to Embodiment 2. [Figure 13] This is a timing chart showing an example of the operation periods of laser light irradiation and exposure according to Embodiment 2. [Figure 14] This is a diagram for explaining the problems to be solved by the imaging device in Embodiment 3. [Figure 15] This is a diagram showing an example of distance measurement indoors according to Embodiment 3. [Figure 16] This is a diagram showing an example of distance measurement outdoors according to Embodiment 3. [Figure 17] This is a schematic diagram showing an example of the arrangement of light sources according to Embodiment 3. [Figure 18] This is a schematic diagram showing an example of the arrangement of light sources according to the modification example of Embodiment 3.

Mode for Carrying Out the Invention

[0013] The embodiments of the present invention will be described below with reference to the drawings. The embodiments described below are merely examples, and the embodiments to which the present invention is applied are not limited to the embodiments described below. Furthermore, in this application, "based on XX" means "based on at least XX," and includes cases where it is based on another element in addition to XX. Also, "based on XX" is not limited to cases where XX is used directly, but also includes cases where it is based on something that has been calculated or processed from XX. "XX" is any element (for example, any information). Furthermore, in the following explanation, the orientation of the imaging device 10 may be shown using a three-dimensional Cartesian coordinate system with x, y, and z axes.

[0014] [Embodiment 1] Figure 1 is a diagram illustrating the general outline of the imaging device according to Embodiment 1. The general outline of the imaging device 10 will be described with reference to this figure. The imaging device 10 measures the distance L1 to an object T that exists in three-dimensional space. The imaging device 10 may measure the distance L1 to the object T outdoors, where it is affected by the solar spectrum, or it may measure the distance L1 to the object T indoors, where it is not affected by the solar spectrum.

[0015] The imaging device 10 comprises a lens 110, a laser diode 120, and a sensor (not shown). The lens 110 may be, for example, an objective lens. The laser diode 120 is a light source that irradiates an object T with illumination light having a predetermined wavelength. The illumination light irradiated by the laser diode 120 is reflected by the object T and incident on the lens 110. The sensor receives the reflected light reflected by the object T through the lens 110. The imaging device 10 measures the distance from the imaging device 10 to the object T by analyzing the received reflected light.

[0016] The imaging device 10 may be equipped with a plurality of laser diodes 120. Furthermore, the illumination light emitted by the plurality of laser diodes 120 equipped with the imaging device 10 may each have a different wavelength. When the imaging device 10 is equipped with a plurality of laser diodes 120 that emit illumination light having different wavelengths, the laser diode 120 that emits the first illumination light BM1-1 having a first wavelength will be described as the first laser diode 121, and the laser diode 120 that emits the second illumination light BM2-1 having a second wavelength will be described as the second laser diode 122. The reflected light obtained when the first illumination light BM1-1 is reflected by the object T will be described as the first reflected light BM1-2, and the reflected light obtained when the second illumination light BM2-1 is reflected by the object T will be described as the second reflected light BM2-2.

[0017] The imaging device 10 may also include a plurality of laser diodes 120 that emit illumination light having the same wavelength. That is, the imaging device 10 may include a plurality of first laser diodes 121 and a plurality of second laser diodes 122. The plurality of laser diodes 120 may be arranged on a circumference centered on the optical axis from which the lens 110 receives reflected light. Furthermore, the imaging device 10 may include a number of sensors corresponding to the types of wavelengths of illumination light emitted by the plurality of laser diodes 120.

[0018] Furthermore, the imaging device 10 may also include an image sensor (not shown). When the imaging device 10 includes an image sensor, it images the object T at an angle of view α. Specifically, multiple pixels of the image sensor receive visible light formed by the lens 110, and form image information based on the received information.

[0019] Figure 2 is a schematic diagram showing an example of a cross-section of the imaging device according to Embodiment 1. An example of the configuration of the imaging device 10 will be described with reference to this figure. The imaging device 10 comprises a lens 110, a laser diode 120, an image acquisition unit 140, and a distance measuring unit 150. The image acquisition unit 140 captures images using visible light, and the distance measuring unit 150 performs distance measurement using infrared light. In other words, the imaging device 10 may be a ToF (Time Of Flight) camera that measures the three-dimensional shape of an object.

[0020] It is known that sunlight is absorbed and attenuated by the Earth's atmosphere before reaching the Earth's surface. In particular, absorption by water vapor molecules in the atmosphere has a significant impact on wavelength characteristics. Specifically, wavelength attenuation due to absorption by water vapor molecules is pronounced at wavelengths such as 850 nm, 940 nm, and 1110 nm. Also, at 730 nm, wavelength attenuation due to absorption by oxygen molecules is pronounced.

[0021] In infrared-based distance measurement technology, three elements are particularly important: sunlight attenuation, lens transmittance, and image sensor spectral sensitivity. In this embodiment, an example is described in which a laser diode that emits near-infrared light, with a balanced combination of these elements, uses laser diode light in the 850 nm and 940 nm wavelength bands. The 850 nm wavelength laser diode light is used for distance measurement indoors, and the 940 nm wavelength laser diode light is used for distance measurement outdoors.

[0022] The imaging device 10 includes a first laser diode 121 as a light source that emits laser diode light in the 940 nm wavelength band. The imaging device 10 also includes a second laser diode 122 as a light source that emits laser diode light in the 850 nm wavelength band.

[0023] 940 nm is also referred to as the first wavelength. Furthermore, the irradiated light having a wavelength of 940 nm is also referred to as the first irradiated light. In other words, the first laser diode 121 is the first light source that emits the first irradiated light, which has the first wavelength.

[0024] 850 nm is also referred to as the second wavelength. Furthermore, the irradiated light having a wavelength of 850 nm is also referred to as the second irradiated light. In other words, the second laser diode 122 is a second light source that emits the second irradiated light, which has a second wavelength. The first and second wavelengths are different wavelengths. It is also desirable that the wavelength band of the first wavelength exhibits a more pronounced attenuation of sunlight compared to the wavelength band of the second wavelength.

[0025] The image acquisition unit 140 captures an image using visible light from the light incident on the lens 110. The image acquisition unit 140 comprises a visible light reflective dichroic film 141, an infrared cut filter 142, a sensor 143, and a reflective surface 145.

[0026] The visible light reflective dichroic film 141 reflects visible light and transmits light with wavelengths in the near-infrared region and above (i.e., infrared light). Light L incident on lens 110 is reflected as visible light VL and transmitted as infrared light IL by the visible light reflective dichroic film 141. The optical axis of lens 110 is denoted as optical axis OA. The visible light VL reflected by the visible light reflective dichroic film 141 is reflected by the reflective surface 145 and incident on sensor 143 via infrared cut filter 142.

[0027] Visible light (VL) and infrared light (i.e., the first reflected light and the second reflected light) pass through substantially the same optical axis between the lens 110 and the visible light reflective dichroic film 141. This substantially same range may be, for example, the range in which an optical path is formed by a common lens.

[0028] The infrared cut filter 142 blocks infrared light from the visible light spectrum (VL). Sensor 143 detects visible light VL incident through an infrared cut filter 142. Sensor 143 comprises a plurality of pixels 144. Specifically, sensor 143 may be an image sensor in which each RGB color pixel is arranged in a Bayer array.

[0029] The sensor 143 will also be referred to as the third detection unit, and the visible light reflective dichroic film 141 will also be referred to as the visible light reflective film. The third detection unit detects visible light. The visible light reflective film guides the visible light VL incident on the lens 110 to the sensor 143 by reflecting it. The visible light reflective film also transmits the first reflected light and the second reflected light, which are infrared light, from the light L incident on the lens 110. By transmitting the infrared light from the light L incident on the lens 110, the visible light reflective film guides the first reflected light to the sensor 153 and the second reflected light to the sensor 163. Furthermore, the visible light reflective film is provided on the optical path between the lens 110 and the half mirror 130.

[0030] The distance measuring unit 150 comprises a half mirror 130, a bandpass filter 152, a sensor 153, a bandpass filter 162, and a sensor 163. Sensors 153 and 163 are also referred to as ToF sensors.

[0031] Infrared light transmitted through the visible light reflective dichroic film 141 is spectrally separated into two optical paths, transmitted light and reflected light, by the half mirror 130 in the distance measuring unit 150. The half mirror 130 may be, for example, a dielectric half mirror.

[0032] The half-mirror 130 is located on the optical path between the lens 110 and the sensor 153, and is positioned on the optical path between the lens 110 and the sensor 163. Furthermore, the first reflected light and the second reflected light pass through substantially the same optical axis between the lens 110 and the half-mirror 130. This substantially same range may, for example, be the range in which the optical path is formed by a common lens.

[0033] The half-mirror 130 can be any optical element that transmits some of the incident light and reflects some of the other light. The half-mirror 130 transmits some of the light that is reflected from the object T by the light emitted from the first laser diode 121, thereby guiding the first reflected light to the sensor 153. The half-mirror 130 also reflects some of the light that is reflected from the object T by the light emitted from the second laser diode 122, thereby guiding the second reflected light to the sensor 163.

[0034] The light, split into two optical paths by the half-mirror 130—transmitted light and reflected light—is received by ToF sensors positioned in each optical path. Specifically, the light transmitted through the half-mirror 130 is received by sensor 153, and the light reflected by the half-mirror 130 is received by sensor 163.

[0035] An optical bandpass filter is placed in front of each ToF sensor (i.e., in the optical path between the half-mirror 130 and each ToF sensor) to allow only light with a predetermined narrow bandwidth of wavelengths to pass through. Specifically, a bandpass filter 152 is placed in front of sensor 153. The bandpass filter 152 allows only the 940 nm narrow bandwidth to pass through. Similarly, a bandpass filter 162 is placed in front of sensor 163. The bandpass filter 162 allows only the 850 nm narrow bandwidth to pass through.

[0036] The sensor 153 will also be referred to as the first detection unit. The first detection unit detects the first reflected light, which is the light reflected after the first irradiation light is irradiated onto the object T by the first laser diode 121. Furthermore, the sensor 163 will also be referred to as the second detection unit. The second detection unit detects the second reflected light, which is the light reflected after the second irradiation light is irradiated onto the object T by the second laser diode 122.

[0037] Furthermore, by changing the spectral ratio of transmitted and reflected light from the half-mirror 130 according to the operating conditions, optimal signal detection can be achieved in both the 850 nm and 940 nm bands.

[0038] [Modification 1 of Embodiment 1] Figure 3 is a schematic diagram showing an example of a cross-section of an imaging device according to Modification 1 of Embodiment 1. Referring to this figure, an example of the configuration of imaging device 10A, which is Modification 1 of imaging device 10, will be described. Imaging device 10A differs from imaging device 10 in that it does not have a half mirror 130 and further includes a switchable bandpass filter 172. By including the switchable bandpass filter 172, imaging device 10A does not require two ToF sensors, and both the first reflected light and the second reflected light can be detected by a single ToF sensor. In the description of imaging device 10A, components similar to those of imaging device 10 may be denoted by the same reference numerals, and their description may be omitted.

[0039] Visible light VL and infrared light IL are incident on lens 110. Here, visible light VL and infrared light IL are incident on lens 110 along a common optical axis OA. Light L incident on lens 110 is incident on visible light reflective dichroic film 141. The reflective dichroic film 141 reflects the incident visible light VL and guides it to sensor 143. The reflective dichroic film 141 also transmits incident infrared light IL and guides it to switchable bandpass filter 172.

[0040] The switchable bandpass filter 172 combines the functions of the bandpass filter 152 and the bandpass filter 162. The switchable bandpass filter 172 switches to either one of the two functions by time division. That is, the switchable bandpass filter 172 exclusively has a period that allows only the 940 nm narrowband to pass through and a period that allows only the 850 nm narrowband to pass through.

[0041] Specifically, the switchable bandpass filter 172 may have a rotating structure that rotates the filter. In this case, the filter may have a disc shape, with one semicircular portion having a filter that passes only the 940 nm narrowband and the other semicircular portion having a filter that passes only the 850 nm narrowband. The switchable bandpass filter 172 may exclusively switch between a period in which only the 940 nm narrowband passes and a period in which only the 850 nm narrowband passes by rotating the disc and aligning the optical axis with either filter.

[0042] Furthermore, the switchable bandpass filter 172 may have a sliding structure for sliding the filter. In this case, the filter may have a rectangular shape, with one side having a filter that passes only the 940 nm narrowband and the other side having a filter that passes only the 850 nm narrowband. The switchable bandpass filter 172 may exclusively switch between a period in which only the 940 nm narrowband passes and a period in which only the 850 nm narrowband passes by sliding the rectangular filter and aligning the optical axis with either filter.

[0043] [Modification 2 of Embodiment 1] Figure 4 is a schematic diagram showing an example of a cross-section of an imaging device according to a modified example 2 of Embodiment 1. Referring to this figure, an example of the configuration of imaging device 10B, which is a modified example 2 of imaging device 10, will be described. Imaging device 10B differs from imaging device 10 in that it does not have an image acquisition unit 140. That is, imaging device 10B is a distance measuring sensor that does not have an image sensor. In the description of imaging device 10B, components similar to those of imaging device 10 are denoted by the same reference numerals, and their description may be omitted.

[0044] Light L incident on lens 110 is split into two optical paths, transmitted light and reflected light, by half mirror 130. The transmitted light is incident on sensor 153, and the reflected light is incident on sensor 163. A bandpass filter 152 that allows only the 940 nm narrow band to pass through is provided in the optical path between half mirror 130 and sensor 153. Additionally, a bandpass filter 162 that allows only the 850 nm narrow band to pass through is provided in the optical path between half mirror 130 and sensor 163.

[0045] [Modification 3 of Embodiment 1] Figure 5 is a diagram illustrating an imaging device according to a modification 3 of Embodiment 1. Referring to this figure, an example of the configuration of imaging device 10C, which is a modification 3 of imaging device 10, will be described. Imaging device 10C has an image sensor and one ToF sensor. Imaging device 10C differs from imaging device 10 in that it does not have a visible light reflective dichroic film 141 and a half mirror 130. In the description of imaging device 10C, components similar to those of imaging device 10 may be denoted by the same reference numerals, and their description may be omitted.

[0046] Figure 5(A) is a front view of the imaging device 10C. The imaging device 10C includes a substrate 180. The substrate 180 includes an infrared cut filter section 181 and a bandpass filter section 182. The infrared cut filter section 181 blocks infrared light and transmits visible light from the light incident on the lens 110. The bandpass filter section 182 transmits light having a predetermined wavelength and blocks light other than light having a predetermined wavelength from the light incident on the lens 110.

[0047] The imaging device 10C has a sliding mechanism (not shown) that allows the relative position of the lens 110 and housing 112 and the substrate 180 to be varied in the y-axis direction (sliding direction DIR). The imaging device 10C, by having a sliding mechanism, directs the light incident on the lens 110 to either the infrared cut filter section 181 or the bandpass filter section 182. The light incident on the infrared cut filter section 181 is directed to the RGB sensor, and the light incident on the bandpass filter section 182 is directed to the ToF sensor.

[0048] Figure 5(B) is a plan view of the imaging device 10C. In the example shown in the figure, the sliding mechanism is positioned to direct the light incident on the lens 110 to the infrared cut filter section 181. As shown in the figure, the relative positions of the lens 110 and housing 112 and the substrate 180 are varied along the sliding direction DIR, thereby changing the optical axis of the light incident on the lens 110 from the infrared cut filter section 181 to the bandpass filter section 182.

[0049] Figure 5(C) is a side view of the imaging device 10C. The figure shows a cross-section in the xz plane that crosses the infrared cut filter section 181. As shown in the figure, the optical axes of the lens 110 and housing 112 and the infrared cut filter section 181 provided on the substrate 180 coincide. Although a cross-section crossing the bandpass filter section 182 is not shown, similarly, the optical axes of the lens 110 and housing 112 and the bandpass filter section 182 provided on the substrate 180 coincide.

[0050] [Modification 4 of Embodiment 1] Figure 6 is a diagram illustrating an imaging device according to modification 4 of Embodiment 1. Referring to this figure, an example of the configuration of imaging device 10D, which is modification 4 of imaging device 10, will be described. Imaging device 10D has an image sensor and one ToF sensor. Imaging device 10D differs from imaging device 10 in that it does not have a visible light reflective dichroic film 141 and a half mirror 130. Also, imaging device 10D is the same as imaging device 10C in that it does not have a visible light reflective dichroic film 141 and a half mirror 130. On the other hand, imaging device 10D differs from imaging device 10C in that it has a rotation mechanism instead of the slide mechanism that imaging device 10C has. In the description of imaging device 10D, components that are the same as those of imaging device 10C may be denoted by the same reference numerals and their descriptions may be omitted.

[0051] Figures 6(A) to 6(C) are front views of the imaging device 10D. The imaging device 10D includes a substrate 190. The substrate 190 includes an infrared cut filter section 191 and a bandpass filter section 192. The infrared cut filter section 191 blocks infrared light and transmits visible light from the light incident on the lens 110. The bandpass filter section 192 transmits light with a predetermined wavelength and blocks light with other wavelengths from the light incident on the lens 110.

[0052] The imaging device 10D has a rotation mechanism (not shown) that rotates the substrate 190 around a rotation center C. By rotating the substrate 190 clockwise (CW) or counterclockwise (CCW) (not shown), the imaging device 10D changes the optical axis of the light incident on the lens 110 to either the infrared cut filter section 191 or the bandpass filter section 192.

[0053] Figure 6(A) shows an example where light incident on lens 110 is positioned to enter the infrared cut filter section 191, and Figure 6(B) shows an example where the substrate 190 is rotated 90 degrees clockwise (CW) by the imaging device 10D. At the position shown in Figure 6(B), light incident on lens 110 does not enter either the infrared cut filter section 191 or the bandpass filter section 192. Figure 6(C) shows an example where the substrate 190 is rotated another 90 degrees clockwise (CW) by the imaging device 10D from the position shown in Figure 6(B). At the position shown in Figure 6(C), light incident on lens 110 enters the bandpass filter section 192.

[0054] Furthermore, by designating one half of the substrate 190 as the infrared cut filter section 191 and the other half as the bandpass filter section 192, it is possible to prevent the situation shown in Figure 6(B) where light incident on the lens 110 does not enter either the infrared cut filter section 191 or the bandpass filter section 192.

[0055] [Summary of Embodiment 1] According to the embodiment described above, the imaging device 10 includes a first laser diode (first light source) 121 to irradiate the object T with first irradiation light having a first wavelength, a second laser diode (second light source) 122 to irradiate the object T with second irradiation light having a second wavelength, a sensor (first detection unit) 153 to detect first reflected light, which is light reflected after the first irradiation light is irradiated onto the object T, and a sensor (second detection unit) 163 to detect second reflected light, which is light reflected after the second irradiation light is irradiated onto the object T. Furthermore, the imaging device 10 includes a half mirror (optical element) 130 to spectrally separate the light incident on the lens 110 and send it to sensors 153 and 163. Furthermore, the imaging device 10 includes a bandpass filter 152 in the optical path between the half mirror 130 and the sensor 153, allowing only the 940 nm narrow band to pass through to the sensor 153, and a bandpass filter 162 in the optical path between the half mirror 130 and the sensor 163, allowing only the 850 nm narrow band to pass through to the sensor 163.

[0056] Therefore, according to this embodiment, the imaging device 10 can simultaneously obtain distance measurement data from an 850 nm ToF camera with features suitable for indoor use and a 940 nm ToF camera with features suitable for outdoor use, thus complementing the data obtained under each other's weaker conditions. Thus, the imaging device 10 can accurately measure the distance to an object even in multiple different environments.

[0057] Furthermore, according to the embodiment described above, the half-mirror 130 of the imaging device 10 is provided on the optical path between the lens 110 and the sensors 153 and 163, and the first reflected light and the second reflected light pass through substantially the same optical axis between the lens 110 and the half-mirror 130. Therefore, according to this embodiment, it is not necessary to provide separate optical paths for the first reflected light and the second reflected light, and the imaging device 10 can be miniaturized.

[0058] Furthermore, according to the embodiment described above, the imaging device 10 detects visible light by including a sensor (third detection unit) 143, and guides infrared light to sensors 153 and 163 and visible light to sensor 143 by including a visible light reflective dichroic film (visible light reflective film) 141. Therefore, the imaging device 10 can obtain an RGB image and distance measurement information. Thus, the imaging device 10 can obtain a highly accurate 3D image by combining the acquired RGB image and distance measurement information.

[0059] Furthermore, according to the embodiment described above, the visible light reflective dichroic film 141 of the imaging device 10 is provided on the optical path between the lens 110 and the half mirror 130. That is, the imaging device 10 first separates the incident light into visible light and infrared light, and then further spectrally separates the infrared light into two infrared light streams. Therefore, according to this embodiment, an RGB image and distance measurement information can be easily obtained.

[0060] Furthermore, according to the embodiments described above, in the imaging device 10, visible light passes through substantially the same optical axis as the first reflected light and the second reflected light between the lens 110 and the visible light reflective dichroic film 141. Therefore, with the imaging device 10, a highly accurate 3D image can be obtained in real time by combining the RGB image obtained on the same optical axis with distance measurement information.

[0061] [Effects of using the same optical axis] Next, with reference to Figures 7 through 11, we will explain in detail the effects of aligning the optical axes.

[0062] First, we will explain the effects when the number of pixels and field of view are the same or known, referring to Figures 7 and 8. Figure 7 is a diagram illustrating the effect of the imaging device according to Embodiment 1 when the number of pixels and field of view of the RGB sensor and the ToF sensor are the same. In the example shown in the figure, the imaging device 10 includes an RGB sensor and a ToF sensor with the same number of pixels and the same field of view (image size). Specifically, both the RGB sensor and the ToF sensor have 640 pixels × 480 pixels. Furthermore, the RGB sensor and the ToF sensor are on the same optical axis.

[0063] Figure 7(A) shows an example of RGB data acquired by an RGB sensor. Figure 7(B) shows an example of Depth data acquired by a ToF sensor. Depth data includes, for example, distance information from the imaging device 10 to the object T. Specifically, Depth data may include distance information corresponding to each of the multiple pixels included in the 2D image information. Figure 7(C) shows an example of 3D point cloud data generated based on the acquired RGB data and Depth data. In this embodiment, since the RGB sensor and the ToF sensor have the same number of pixels and field of view, the imaging device 10 does not need to perform pixel count matching and field of view matching processing for the acquired RGB data and depth data. Furthermore, since the RGB sensor and the ToF sensor have the same optical axis, there is no parallax or FoV difference between the RGB data and the depth data. Therefore, the imaging device 10 does not need to perform parallax correction for field of view matching or peripheral field of view limitation processing due to FoV difference. Thus, according to this embodiment, the imaging device 10 can generate 3D point cloud data without correcting (i.e., without processing) the RGB data and depth data.

[0064] Figure 8 is a diagram illustrating the effect of the imaging device according to Embodiment 1 when the number of pixels and field of view adjustment parameters of the RGB sensor and ToF sensor are known. In the example shown in the figure, the pixel count and field of view of the RGB sensor and the ToF sensor are different. Specifically, the RGB sensor has 1280 pixels x 960 pixels, while the ToF sensor has 640 pixels x 480 pixels. Furthermore, the RGB sensor and the ToF sensor share the same optical axis.

[0065] Figure 8(A) shows an example of RGB data acquired by an RGB sensor. Figure 8(B) shows an example of RGB data cropped according to the field of view from which depth data was acquired. Figure 8(C) shows an example of RGB data resized according to the number of pixels in the depth data. Figure 8(D) shows an example of depth data acquired by a ToF sensor. Figure 8(E) shows an example of processed RGB data and 3D point cloud data generated based on the acquired depth data.

[0066] As shown in Figures 8(A) to 8(C), when the number of pixels and field of view of the RGB sensor and the ToF sensor are different, the data with the larger number of pixels is combined with the data with the smaller number of pixels. Specifically, in this embodiment, since the number of pixels of the RGB sensor is larger than that of the ToF sensor, the RGB data is first trimmed and then resized.

[0067] In other words, even if the RGB sensor and the ToF sensor have different pixel counts and field of view, if the ratio parameter for cropping the data from the wider field of view sensor to the narrower field of view sensor, and the resize ratio parameter for matching the pixel count, are known in advance, the imaging device 10 can easily match the pixel count and field of view of the acquired RGB data and Depth data. Furthermore, since the RGB sensor and the ToF sensor are on the same optical axis, there is no parallax or FoV difference. Therefore, the imaging device 10 does not need to perform parallax correction for field of view matching or peripheral field of view limitation processing due to FoV difference. Thus, according to this embodiment, the imaging device 10 can easily generate 3D point cloud data from the RGB data and Depth data.

[0068] Furthermore, since the RGB sensor and the ToF sensor share the same optical axis, even if processing such as distortion correction, peripheral light falloff correction, and chromatic aberration correction are necessary due to the lens characteristics of the lens 110, the imaging device 10 can apply the same correction data to both the RGB data and the Depth data. In other words, since it is not necessary to apply different correction data to the RGB data and the Depth data, the imaging device 10 can easily correct the data. While there is a correlation between the characteristics of visible light and infrared light, if there are differences, it may be necessary to make corrections according to the correlation.

[0069] Furthermore, since the RGB sensor and the ToF sensor are on the same optical axis, the imaging device 10 can integrate the image frequency information obtained from the RGB sensor with the subject distance information obtained from the ToF sensor to perform more accurate focusing and edge detection.

[0070] Furthermore, according to this embodiment, even in dark conditions such as at night or in shadowed areas where sufficient information cannot be obtained from the RGB sensor, the imaging device 10 can generate 3D data based on distance information obtained from the ToF sensor.

[0071] Next, with reference to Figures 9 to 11, an example of applying various corrections due to lens characteristics to a ToF camera will be described. According to this embodiment, since the RGB sensor and the ToF sensor are on the same optical axis, various corrections due to lens characteristics can be applied to the ToF camera. Furthermore, by using distance information obtained from the ToF sensor in combination, the accuracy of focusing by edge detection in conventional RGB cameras can be improved. In addition, in the example shown in Figures 9 to 11, the imaging device 10 is equipped with an RGB sensor and a ToF sensor having the same number of pixels and the same field of view (image size), and an example is described in which the RGB sensor and the ToF sensor are on the same optical axis.

[0072] Figure 9 is a diagram illustrating the sharing of distortion correction values ​​in the imaging device according to Embodiment 1. The sharing of distortion correction values ​​will be explained with reference to this figure. Figure 9(A) shows an example of RGB data acquired by an RGB sensor. The RGB data shown in this figure is barrel-distorted. Figure 9(B) shows an example of RGB data acquired by an RGB sensor after barrel distortion correction. Figure 9(C) shows an example of ToF data acquired by a ToF sensor. The ToF data shown in this figure is barrel-distorted, similar to the RGB data. Figure 9(D) shows an example of ToF data acquired by a ToF sensor after barrel distortion correction. Note that ToF data is an example of depth data acquired by a ToF sensor.

[0073] Since the RGB sensor and the ToF sensor share the same optical axis, as shown in the example in Figure 9, both the RGB data and the ToF data exhibit similar barrel distortion. Therefore, the distortion correction data calculated from the lens distortion information obtained by the RGB sensor can be directly applied to the distortion correction of the ToF data. In other words, according to this embodiment, the correction values ​​for the RGB data and the ToF data can be shared. Thus, according to this embodiment, correction can be easily performed.

[0074] Figure 10 is a diagram illustrating the sharing of peripheral light falloff correction data in the imaging device according to Embodiment 1. The sharing of peripheral light falloff correction data will be explained with reference to this figure. Figure 10(A) shows an example of RGB data acquired by an RGB sensor. The RGB data shown in this figure has reduced ambient light (insufficient). Figure 10(B) shows an example of RGB data acquired by an RGB sensor after correction for reduced ambient light. The RGB data shown in this figure has been corrected for reduced ambient light. Figure 10(C) shows the light intensity of each RGB color in the A-A' cross-section of the data shown in Figure 10(A) on the vertical axis and the horizontal coordinates (pixels) of the image on the horizontal axis. Figure 10(D) shows the light intensity of each RGB color in the A-A' cross-section of the data shown in Figure 10(B) on the vertical axis and the horizontal coordinates (pixels) of the image on the horizontal axis.

[0075] As shown in the example in Figure 10, according to this embodiment, since the RGB sensor and the ToF sensor are on the same optical axis, the imaging device 10 can use the peripheral light falloff correction data calculated from the peripheral light falloff information of the lens obtained by the RGB sensor to directly correct the peripheral light falloff of the ToF data. However, if there is a correlation between the peripheral light falloff characteristics of visible light and infrared light, but also differences, it may be necessary to make corrections according to the correlation.

[0076] In conventional dual-lens cameras, the peripheral light falloff correction data differs between RGB data and ToF data, requiring the correction amount to be adjusted according to the characteristics of each lens. According to this embodiment, the same or corresponding correction data can be applied to both RGB data and ToF data, making it easy to correct both RGB data and ToF data.

[0077] Figure 11 is a diagram illustrating the sharing of chromatic aberration correction data in the imaging device according to Embodiment 1. The sharing of chromatic aberration correction data will be explained with reference to this figure. Figure 11(A) is an example of RGB data where chromatic aberration occurs with red on the left edge and cyan on the right edge. Figure 11(B) is an example of RGB data where chromatic aberration occurs with cyan on the left edge and red on the right edge. The upper part of Figure 11(C) shows an example of the left and right edge waveforms of Figure 11(A), and the lower part of Figure 11(C) shows an example of the left and right edge waveforms of Figure 11(B). Figure 11(D) is an example of RGB data after magnification chromatic aberration correction processing. Figure 11(E) is an example of ToF data after magnification difference correction processing.

[0078] The imaging device 10 calculates chromatic aberration correction data based on the chromatic aberration information of the lens 110 obtained from the RGB data. The imaging device 10 also applies the obtained chromatic aberration correction data to correct the magnification difference of the image in the ToF data. In particular, the imaging device 10 directly uses the chromatic aberration correction data of the Rch, which is close to the near-infrared region used in the ToF camera, to correct the magnification difference of the ToF data. Alternatively, the imaging device 10 may estimate the amount of magnification difference correction in the correlated near-infrared region from the chromatic aberration correction data of the Rch and apply it.

[0079] Furthermore, the imaging device 10 detects the edges of subjects that are at a distance from the background based on distance information obtained from ToF data. The imaging device 10 also detects the edges of subjects with differences in brightness or frequency based on signals obtained from RGB data. By using these methods in combination, the imaging device 10 can further improve the accuracy of edge detection and use it for focusing the camera.

[0080] Furthermore, the imaging device 10 corrects the magnification difference between the RGB data and the ToF data by also using the Rch magnification chromatic aberration correction data for the ToF data. By correcting the magnification difference between the RGB data and the ToF data, the imaging device 10 can accurately superimpose the images during 3D data generation and suppress the occurrence of distance shifts at the edges.

[0081] [Embodiment 2] Next, Embodiment 2 will be described. The imaging device 10 according to this embodiment includes a first laser diode (first light source) 121 to emit a first irradiation light having a first wavelength, and includes a sensor (first detection unit) 153 to detect a first reflected light, which is the light reflected by the object from the first irradiation light. The imaging device 10 also includes a second laser diode (second light source) 122 to emit a second irradiation light having a second wavelength different from the first wavelength, and includes a sensor (second detection unit) 163 to detect a second reflected light, which is the light reflected by the object from the second irradiation light. Here, the first irradiated light and the first reflected light, and the second irradiated light and the second reflected light, have different wavelengths and may interfere with each other. Embodiment 2 aims to suppress interference between lights of different wavelengths.

[0082] In the description of Embodiment 2, an example of using the imaging device 10 described with reference to Figure 2 will be explained. However, the control method according to Embodiment 2 is not limited to the example applied to the imaging device 10, but can also be similarly applied to imaging devices 10A to 10D.

[0083] Figure 12 is a diagram illustrating the interference of two different near-infrared light wavelengths according to Embodiment 2. The interference of two different near-infrared light wavelengths will be explained with reference to this figure. The figure shows the output of visible light (Rch, Gch, Bch) received by the RGB camera and infrared light (850nm, 940nm) received by the ToF camera, with wavelength [nm] on the horizontal axis and relative output on the vertical axis. The figure also similarly shows the wavelengths that can be blocked by the IR cut filter (infrared cut filter 142), the 940nm bandpass filter (bandpass filter 152), and the 850nm bandpass filter (bandpass filter 162).

[0084] The 850nm and 940nm bandpass filters commonly used in ToF cameras often have a bandwidth of around 150nm. By applying the 940nm and 850nm bandpass filters shown in Figure 12, the influence of the visible light region can be eliminated.

[0085] However, when near-infrared light of 850 nm and 940 nm is emitted simultaneously, interference from infrared light of other wavelengths cannot be completely eliminated. Specifically, in range A, near-infrared light of 850 nm and 940 nm interfere with each other.

[0086] To prevent such interference, it is effective to use a narrowband bandpass filter that can abruptly block wavelengths within 100 nm, rather than a general-purpose bandpass filter used in ToF cameras. However, narrowband bandpass filters that can abruptly block wavelengths within 100 nm are expensive. Therefore, in the second embodiment, the imaging device 10 suppresses interference between the wavelengths of the laser diode and the sensor by controlling the emission timing of the laser diode and the exposure timing of the sensor.

[0087] Figure 13 is a timing chart showing an example of the operation period for laser light irradiation and exposure according to Embodiment 2. Referring to this figure, an example of the period for the laser diode's light emission operation and the period for the sensor's exposure operation will be explained. In the figure, the "940nm LD light emission period" indicates the period for the first laser diode 121's light emission operation. The "940nm ToF exposure period" indicates the period for the sensor 153's exposure operation. The "850nm LD light emission period" indicates the period for the second laser diode 122's light emission operation. The "850nm ToF exposure period" indicates the period for the sensor 163's exposure operation. The horizontal axis represents time, and the vertical axis represents whether the light emission or exposure is on or off. High levels indicate on, and low levels indicate off. Similarly, the horizontal axis represents time to show the frame pulse timing. Note that the on or off periods shown in the figure represent the period during which the laser diode emits light or the sensor exposes light, and the actual control signal may repeat multiple switching operations within that period. Specifically, in the actual on period, the LD emission period and the ToF exposure period are each composed of multiple fine control pulses, and the LD emission period and the ToF exposure period may not necessarily be in phase.

[0088] Figure 13(A) is a timing chart illustrating an example of the first interference prevention measure. First, the details of the first interference prevention measure will be explained. In the first interference prevention measure, the emission and exposure of the 940 nm ToF sensor and the 850 nm ToF sensor are controlled based on a common frame pulse timing VD. The frame pulse timing VD has a period t11.

[0089] The period t12 during which the first laser diode (first light source) 121 emits the first irradiation light and the sensor (first detection unit) 153 detects the first reflected light is referred to as the first period. The period t13 during which the second laser diode (second light source) 122 emits the second irradiation light and the sensor (second detection unit) 163 detects the second reflected light is referred to as the second period. The processing performed in the first period and the processing performed in the second period are carried out in different frames. In other words, the first and second periods do not overlap.

[0090] Specifically, the timing of laser diode emission and ToF sensor exposure is controlled alternately by performing 850 nm laser diode emission and 850 nm ToF sensor exposure on even frames, and 940 nm laser diode emission and 940 nm ToF sensor exposure on odd frames. In other words, the first period and the second period alternate at predetermined intervals. More specifically, the first period is the period within the odd-numbered periods of the predetermined period t11. The second period is the period within the even-numbered periods of the predetermined period t11. The first and second periods may be swapped. Specifically, 850 nm laser diode emission and 850 nm ToF sensor exposure may be performed on odd-numbered frames, while 940 nm laser diode emission and 940 nm ToF sensor exposure may be performed on even-numbered frames.

[0091] As described above, the first interference prevention measure prevents interference of near-infrared light by alternately controlling the operating timing of laser diode emission and ToF sensor exposure. The first interference prevention measure has the advantage that the laser diode emission timing control between the two wavelengths and the ToF sensor exposure timing control can be managed with a common synchronization system. On the other hand, the first interference prevention measure has the drawback that the ranging frame rate is halved. The second interference prevention measure solves the problems of the first interference prevention measure.

[0092] Figure 13(B) is a timing chart illustrating an example of the second interference prevention measure. Next, the details of the second interference prevention measure will be explained. In the second interference prevention measure, control is performed based on the frame pulse timing VD1 for the 940 nm ToF sensor and the frame pulse timing VD2 for the 850 nm ToF sensor. The frame pulse timing VD1 has a period t21, and the frame pulse timing VD2 has a period t24. Period t21 will be referred to as the first period, and period t24 as the second period.

[0093] The period t22 during which the first laser diode (first light source) 121 emits the first irradiation light and the sensor (first detection unit) 153 detects the first reflected light is referred to as the first period. The period t25 during which the second laser diode (second light source) 122 emits the second irradiation light and the sensor (second detection unit) 163 detects the second reflected light is referred to as the second period. In the second interference prevention measure, the first period is a period within the first cycle, and the second period is a period within the second cycle. In other words, the first period is a period based on the frame pulse timing VD1, and the second period is a period based on the frame pulse timing VD2. If the processing performed in the first period and the processing performed in the second period overlap at the same timing, light with different wavelengths will interfere. Therefore, the first and second periods are controlled so as not to overlap.

[0094] Furthermore, in the second interference prevention measure, the first and second periods are in different phases. Specifically, the frame pulse timing VD2 may be half a period behind the frame pulse timing VD1. In other words, the phase difference between the first and second periods may be half a period (180 degrees). Also, the first period, period t21, and the second period, period t24, may be the same period.

[0095] In the second interference prevention measure, the period during which the laser diode irradiation light is applied and the sensor detects the reflected light may be less than or equal to half the frame pulse period. Specifically, the first period within the first period may be less than or equal to half the length of the first period, and the second period within the second period may be less than or equal to half the length of the second period.

[0096] As described above, the second interference prevention measure involves shifting the emission of the 850 nm laser diode and exposure to the 850 nm ToF sensor by half a frame, and the emission of the 940 nm laser diode and exposure to the 940 nm ToF sensor, thereby preventing interference of near-infrared light of each other's wavelengths. The second interference prevention measure maintains the ranging frame rate, unlike the first measure which halves it. However, the second measure has the disadvantage of making the synchronization circuits for both control and signal processing more complex. Also, with the second measure, extending the laser diode emission period for long-distance ranging shortens the interval period, making interference between the two wavelengths unavoidable. If interference occurs due to extending the emission period, adjustments such as reducing the frame rate are effective.

[0097] In another embodiment, by using narrow-band bandpass filters that sharply attenuate at approximately ±40 nm of the center wavelengths of 850 nm and 940 nm, interference between near-infrared light of each wavelength can be virtually eliminated without the control measures described above.

[0098] [Summary of Embodiment 2] According to the embodiment described above, the imaging device 10 includes a first laser diode (first light source) 121 to irradiate the object T with first irradiation light having a first wavelength, a second laser diode (second light source) 122 to irradiate the object T with second irradiation light having a second wavelength, a sensor (first detection unit) 153 to detect first reflected light, which is light reflected after the first irradiation light is irradiated onto the object T, and a sensor (second detection unit) 163 to detect second reflected light, which is light reflected after the second irradiation light is irradiated onto the object T. Furthermore, in the imaging device 10, the first period in which the first laser diode 121 irradiates the first irradiation light and the sensor 153 detects the first reflected light does not overlap with the second period in which the second laser diode 122 irradiates the second irradiation light and the sensor 163 detects the second reflected light. Therefore, according to this embodiment, the imaging device 10 can irradiate and receive reflected light of laser light of different wavelengths without interfering with each other.

[0099] Furthermore, according to the embodiments described above, in the imaging device 10, the first period is a period within an odd-numbered period of a predetermined cycle, and the second period is a period within an even-numbered period of a predetermined cycle. Therefore, the imaging device 10 can easily prevent interference between near-infrared light of different wavelengths by controlling the operation timing of laser diode emission and ToF sensor exposure alternately for each frame. In addition, because the imaging device 10 controls the operation timing of laser diode emission and ToF sensor exposure alternately for each frame, they do not interfere with each other even if the emission period and exposure period become long.

[0100] Furthermore, according to the embodiment described above, in the imaging device 10, the first period is a period within the first cycle, and the second period is a period within the second cycle that has a different phase from the first cycle. In other words, the imaging device 10 prevents interference by timing the emission and exposure of light of different wavelengths based on the respective frame pulse timings. Therefore, according to this embodiment, the imaging device 10 can prevent interference of near-infrared light of different wavelengths without reducing the frame rate.

[0101] Furthermore, according to the embodiment described above, in the imaging device 10, the first period and the second period are the same period, and the phase difference between the first period and the second period is half a period. Therefore, according to this embodiment, the imaging device 10 can easily prevent interference between near-infrared light of different wavelengths by emitting and exposing light of different wavelengths at half-frame shifted timings. In addition, by emitting and exposing light of different wavelengths at half-frame shifted timings, the imaging device 10 can make interference less likely even if the emission period and exposure period of light having each wavelength are extended.

[0102] Furthermore, according to the embodiment described above, in the imaging device 10, the first period within the first cycle is less than half the length of the first cycle, and the second period within the second cycle is less than half the length of the second cycle. Therefore, according to this embodiment, the first period and the second period do not overlap, thus preventing interference of light with different wavelengths.

[0103] While the imaging device 10A, as described with reference to Figure 3, cannot simultaneously measure distances at 850 nm and 940 nm, it does not require the synchronized control of the two laser diode emission and the synchronized control of the two ToF sensor exposures described in Embodiment 2. Therefore, the imaging device 10A can obtain the advantages of both wavelengths without causing laser diode optical interference between the two wavelengths.

[0104] [Embodiment 3] Next, Embodiment 3 will be described. The imaging device 10 according to this embodiment includes a first laser diode (first light source) 121 to emit a first irradiation light having a first wavelength, and includes a sensor (first detection unit) 153 to detect a first reflected light, which is the light reflected from the first irradiation light by the object. The first wavelength is, for example, 940 [nm] and is used for distance measurement outdoors. Furthermore, the imaging device 10 is equipped with a second laser diode (second light source) 122 to emit a second irradiation light, which has a second wavelength different from the first wavelength, and is equipped with a sensor (second detection unit) 163 to detect a second reflected light, which is the light reflected from the second irradiation light by the object. The second wavelength is, for example, 850 nm and is used for distance measurement indoors.

[0105] Figure 14 is a diagram illustrating the problems that the imaging device aims to solve in Embodiment 3. First, the problems that the device aims to solve in Embodiment 3 will be explained with reference to this figure. Multiple first laser diodes 121 and second laser diodes 122 are arranged around the front surface of the lens 110, surrounding the lens 110. In one example shown in the figure, the first laser diode 121 consists of a first laser diode 121-1 and a first laser diode 121-2, and the second laser diode 122 consists of a second laser diode 122-1 and a second laser diode 122-2, both arranged to surround the lens 110.

[0106] The first laser diode 121, which emits light with a wavelength of 940 nm, is used for long-distance outdoor applications. The second laser diode 122, which emits light with a wavelength of 850 nm, is used for short-distance indoor applications. Here, the distance error due to the angular difference between the optical axis OA of lens 110 and the laser diode irradiation axis (laser beams BM11-2 and BM12-2) becomes a problem. In order to suppress the effects of distance errors during short-range measurement, it is preferable to place the light source as close as possible to the optical axis OA.

[0107] The effect of distance errors is particularly pronounced when the object T is at a short distance. Furthermore, indoors, distance measurements are often taken to objects T that are at a short distance, compared to outdoor settings. Therefore, for the second laser diode 122 used for short-range measurements, it is preferable to position the light source as close as possible to the optical axis OA to mitigate the effects of distance errors. Therefore, it is preferable that the second laser diode (second light source) 122 used for indoor use be positioned closer to the optical axis OA (near the outer circle of the lens) than the first laser diode (first light source) 121 used for outdoor use.

[0108] However, when the subject is large, the shadow of the laser diode illumination is cast on the sides of the subject, resulting in a large area where distance measurement is impossible. In Figure 14, areas AR1 and AR2 are affected by the shadow of the laser diode illumination, resulting in the problem of being unable to measure distance. Embodiment 3 aims to suppress the occurrence of such areas where distance measurement is impossible.

[0109] Figure 15 shows an example of indoor distance measurement according to Embodiment 3. The functional configuration of the imaging device 10E according to this embodiment and its effects will be explained with reference to this figure. In the description of the imaging device 10E, components similar to those of the imaging device 10 may be denoted by the same reference numerals, and their explanation may be omitted.

[0110] As shown in the figure, the imaging device 10E has the first laser diode 121 and the second laser diode 122 on a plane S that intersects the optical axis OA. That is, both the first laser diode (first light source) 121 and the second laser diode (second light source) 122 are arranged on a plane that intersects the optical axis OA. Also, plane S is perpendicular to the optical axis OA. That is, both the first laser diode (first light source) 121 and the second laser diode (second light source) 122 are arranged on a plane perpendicular to the optical axis OA.

[0111] The imaging device 10E differs from the imaging device 10 in that, in addition to the 850 nm laser diode located near the optical axis OA, it also has another 850 nm laser diode located further away from the optical axis. Specifically, the imaging device 10E further includes a second laser diode 122-5 and a second laser diode 122-6.

[0112] The imaging device 10E can reduce the generation of illumination shadows by providing 850 nm laser diodes not only near the optical axis but also at positions further from the optical axis. Specifically, the second laser diode 122-5 suppresses the generation of illumination shadows in range AR1 by irradiating with laser light BM15, and the second laser diode 122-6 suppresses the generation of illumination shadows in range AR2 by irradiating with laser light BM16. Because the imaging device 10E can reduce the generation of illumination shadows, it can also measure distances to the sides of the subject.

[0113] The imaging device 10E controls the emission of light from each of the laser diodes located at two locations (i.e., near and far from the optical axis). The imaging device 10E can generate optimized distance data by combining two types of distance data received by the ToF sensor (distance data obtained from the second laser diode 122-1 and the second laser diode 122-2, and distance data obtained from the second laser diode 122-5 and the second laser diode 122-6). Furthermore, to prevent signal saturation in the ToF sensor, it is preferable for the imaging device 10E to reduce the emission intensity of the light emitted from each laser diode.

[0114] Figure 16 shows an example of outdoor distance measurement according to Embodiment 3. An example of outdoor distance measurement using the imaging device 10E according to this embodiment will be described with reference to the same figure. In outdoor distance measurement, the imaging device 10E uses the first laser diode 121. The first laser diode 121 is positioned outside the second laser diode 122. Since the first laser diode 121 is used for medium to long distance measurement, the distance to the target object T is long, and the influence of distance error due to the angle difference between the lens optical axis OA and the laser diode irradiation axis (laser beam BM21-2 and laser beam BM22-2) is small. Also, because the distance to the target object T is long, the shadow of the laser diode irradiation is less than at short distances.

[0115] Furthermore, since the signal intensity received by the ToF sensor decreases as the distance to the subject increases, it is preferable to increase the light emission intensity of the first laser diode 121.

[0116] Figure 17 is a schematic diagram showing an example of the arrangement of the light source according to Embodiment 3. An example of the arrangement of the first laser diode 121 and the second laser diode 122 will be described with reference to this figure. The figure shows the positional relationship between the lens 110 and the multiple laser diodes 120 when the imaging device 10E is viewed from the front.

[0117] The imaging device 10E comprises a plurality of first laser diodes 121 and a plurality of second laser diodes 122. In one example shown in Figure 17, it comprises four first laser diodes 121 and eight second laser diodes 122. Multiple first laser diodes 121 are arranged on the circumference of a first circle C1 centered on the optical axis OA. Multiple second laser diodes 122 are arranged on the circumferences of a second circle C2 and a third circle C3. The first circle C1 is a circle with different radii from the second circle C2 and the third circle C3. Furthermore, the first circle C1, the second circle C2, and the third circle C3 are all concentric circles centered on the common optical axis OA.

[0118] Some of the multiple second laser diodes 122, specifically second laser diodes 122-1 to 122-4, are arranged on the circumference of the second circle C2. Other parts of the multiple second laser diodes 122, specifically second laser diodes 122-5 to 122-8, are arranged on the circumference of the third circle C3. The third circle C3 is a circle with a radius different from both the first circle C1 and the second circle C2, and is a concentric circle with the optical axis OA as its center.

[0119] Figure 18 is a schematic diagram showing an example of the arrangement of a light source according to a modified embodiment of Embodiment 3. A modified arrangement of the first laser diode 121 and the second laser diode 122 will be described with reference to this figure. In the modified configuration, the multiple first laser diodes 121 and some of the second laser diodes 122 are arranged on the circumference of the same circle, which is different from the example described with reference to Figure 17. In the example shown in the figure, the second laser diode 122 is the same as in the example explained with reference to Figure 17, so the explanation may be omitted by using the same reference numeral. On the other hand, the first laser diode 121 has a different arrangement, so it is described as first laser diode 121-nA (where n is a natural number from 1 to 4).

[0120] The first laser diodes 121-1A to 121-4A are arranged on the same circumference as the second laser diodes 122-1 to 122-4. In other words, in the modified example, the first circle C1 and the second circle C2 are the same circle.

[0121] The first laser diodes 121-1A to 121-4A are arranged at angles A1, where A1 is 90 degrees. The second laser diodes 122-1 to 122-4 are arranged at angles A2, where A2 is 90 degrees. Furthermore, the first laser diodes 121-1A to 121-4A are positioned between the second laser diodes 122-1 and 122-4. The first laser diodes 121-1A to 121-4A and the second laser diodes 122-1 to 122-4 are positioned at angles A3, respectively. Angle A2 is 45 degrees.

[0122] [Summary of Embodiment 3] According to the embodiment described above, the imaging device 10E includes a first laser diode (first light source) 121 to irradiate the object T with first irradiation light having a first wavelength, a second laser diode (second light source) 122 to irradiate the object T with second irradiation light having a second wavelength, a sensor (first detection unit) 153 to detect first reflected light, which is light reflected after the first irradiation light is irradiated onto the object T, and a sensor (second detection unit) 163 to detect second reflected light, which is light reflected after the second irradiation light is irradiated onto the object T. Therefore, according to this embodiment, when the imaging device 10E measures the distance to an object T placed at close range indoors, the angular difference between the optical axis OA of the lens 110 and the laser diode irradiation axis can be reduced. Thus, the imaging device 10E can reduce distance errors caused by the angular difference between the optical axis OA of the lens 110 and the laser diode irradiation axis.

[0123] Furthermore, according to the embodiments described above, in the imaging device 10, both the first laser diode 121 and the second laser diode 122 are arranged on a surface that intersects the optical axis OA of the lens 110. Also, the light emitted from the first laser diode 121 and the second laser diode 122, respectively, both enter the lens 110 along the same optical axis. Therefore, according to this embodiment, sensors 153 and 163 can share the same lens 110.

[0124] Furthermore, according to the embodiment described above, in the imaging device 10, both the first laser diode 121 and the second laser diode 122 are arranged on a plane perpendicular to the optical axis OA of the lens 110. Therefore, when the object T is located on the optical axis OA, the distance from the first laser diode 121 to the object T and the distance from the second laser diode 122 to the object T are the same. Therefore, according to this embodiment, the imaging device 10E can accurately measure the distance to the object T.

[0125] Furthermore, according to the embodiment described above, in the imaging device 10, the plurality of first laser diodes 121 are arranged on the circumference of a first circle C1 centered on the optical axis OA of the lens 110, and the plurality of second laser diodes 122 are arranged on the circumference of a second circle C2, which is a circle with a different radius from the first circle C1 and is a concentric circle centered on the optical axis OA of the lens 110. Therefore, when the object T is located on the optical axis OA, the distance from each of the plurality of first laser diodes 121 to the object T is the same, and the distance from each of the plurality of second laser diodes 122 to the object T is the same. Therefore, according to this embodiment, the imaging device 10E can accurately measure the distance to the object T.

[0126] Furthermore, according to the embodiment described above, in the imaging device 10, some of the plurality of second laser diodes 122 are arranged on the circumference of the second circle C2, and other parts of the plurality of second laser diodes 122 are arranged on the circumference of a third circle C3, which has a radius different from both the first circle C1 and the second circle C2, and is a concentric circle centered on the optical axis OA of the lens 110. In other words, according to this embodiment, the second laser diodes 122 that measure the distance to an object T that is at close range indoors are arranged at a position close to the optical axis OA of the lens 110 and at a position far from it, respectively. Therefore, according to this embodiment, the imaging device 10E can accurately measure distance by suppressing the effect of the shadow cast by the laser light on the object T.

[0127] Although embodiments of the present invention have been described above, the present invention is not limited to the above embodiments, and various modifications can be made without departing from the spirit of the invention, and the above embodiments may be combined as appropriate. [Explanation of symbols]

[0128] 10…Imaging device, 110…Lens, 120…Laser diode, 121…First laser diode, 122…Second laser diode, 130…Half mirror, 140…Image acquisition unit, 141…Visible light reflective dichroic film, 142…Infrared cut filter, 143…Sensor, 144…Pixel, 145…Reflective surface, 150…Distance measuring unit, 152…Bandpass filter, 153…Sensor, 162…Bandpass filter, 163…Sensor, 172…Switchable bandpass filter, 173…Sensor, T…Object, BM…Laser light, L…Light, VL…Visible light, IL…Infrared light

Claims

1. Multiple first light sources that emit a first irradiation light having a first wavelength, A plurality of second light sources that emit a second irradiation light having a second wavelength different from the first wavelength, A first detection unit detects the first reflected light, which is the light reflected when the first irradiation light is irradiated onto an object, The system includes a second detection unit that detects a second reflected light, which is light reflected when the second irradiation light is irradiated onto the object, Multiple first light sources are arranged on the circumference of a first circle centered on the optical axis. Some of the multiple second light sources are arranged on the circumference of a second circle which is a circle having a different radius from the first circle and is a concentric circle centered on the optical axis. Some of the multiple second light sources are further arranged on the circumference of a third circle, which is a concentric circle centered on the optical axis and has a radius different from both the first and second circles. Some of the multiple second light sources are positioned closer to the optical axis than the first light source, and other parts of the multiple second light sources are positioned further from the optical axis than the first light source. Imaging device.

2. Both the first and second light sources are arranged on a plane that intersects the optical axis. The imaging apparatus according to claim 1.

3. A first irradiation step involves irradiating with a first irradiation light having a first wavelength using a plurality of first light sources, A second irradiation step involves irradiating with a second irradiation light having a second wavelength different from the first wavelength using a plurality of second light sources, A first detection step involves detecting multiple first reflected light beams, which are light beams that have been reflected from an object after multiple first irradiation beams have been irradiated onto the object. The process includes a second detection step of detecting a plurality of second reflected light beams, which are light beams that have been reflected after a plurality of the second irradiation light beams have been irradiated onto the object, Multiple first light sources are arranged on the circumference of a first circle centered on the optical axis. Some of the multiple second light sources are arranged on the circumference of a second circle which is a circle having a different radius from the first circle and is a concentric circle centered on the optical axis. Some of the multiple second light sources are further arranged on the circumference of a third circle, which is a concentric circle centered on the optical axis and has a radius different from both the first and second circles. Some of the multiple second light sources are positioned closer to the optical axis than the first light source, and other parts of the multiple second light sources are positioned further from the optical axis than the first light source. Imaging method.

Images