Distance detector

Abstract

To provide a distance detector which has high distance detection accuracy and azimuth resolution irrelevantly to the distance to an object, is immune to interference and disturbance between distance detectors, is small-sized and inexpensive, and is integrated with an image pickup device.SOLUTION: A distance detector includes a light source configured to generate illumination light obtained by performing chirp modulation on amplitude modulation of carrier light, a light source configured to generate reference light obtained by performing the same chirp modulation as that of the illumination light on amplitude modulation of the carrier light, a coherence reduction mechanism configured to modulate the chirp modulation by chirp modulation signal and to reduce temporal coherence and spatial coherence of both or one of the carrier lights while maintaining temporal coherence and spatial coherence of a chirp modulation signal, an imaging optical system configured to image reflected light from an object illuminated with the illumination light on a photodetector, and a multiplexer configured to multiplex the reflected light and the reference light. The photodetector receives the combined reflected light and reference light, and detects an interference fringe signal generated by heterodyne between the chirp modulation signals of the illumination light and the reference light.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a distance detection device used in autonomous driving, robots, drones, etc. [Background technology]

[0002] It is desirable for a distance detection device for autonomous driving to meet the following requirements: High distance detection accuracy and azimuth resolution ranging from a few centimeters (cm) to several hundred meters (m). - Resistant to interference and disturbance between distance detection devices. - It is small, low-cost, and can be integrated with an imaging device.

[0003] Known conventional distance detection devices include stereo cameras, ToF cameras, LiDAR, millimeter-wave radar, and ultrasonic sensors (see, for example, Non-Patent Document 1). The advantages and disadvantages of each distance detection device will be described later. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Nobuyoshi Fugeno et al., "Millimeter Wave Propagation and Radar Rain Gauges," Quarterly Report of the Radio Research Laboratory, December 1976, pp. 407-426, Vol. 22, No. 121 Summary of the Invention [Problem to be solved by the invention]

[0005] It is difficult for conventional distance detection devices to simultaneously satisfy all of the above requirements, so currently, three or four types of distance detection devices are used in addition to an imaging device, and the disadvantages of each are compensated for.

[0006] The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a distance detection device that has high distance detection accuracy and azimuth resolution regardless of the distance to an object, is resistant to interference and disturbance between distance detection devices, is small and low-cost, and can be integrated with an imaging device. [Means for solving the problem]

[0007] In order to solve the above-mentioned problems and achieve the object, a first aspect of the distance detection device of the present invention is: a light source that generates illumination light in which chirp modulation is applied to amplitude modulation of carrier light; a light source for generating reference light by subjecting the amplitude modulation of carrier light to the same chirp modulation as that of the illumination light; Chirp modulation is modulated by a chirp modulation signal, a coherence reduction mechanism that reduces the temporal coherence and spatial coherence of both or one of the carrier lights while maintaining the temporal coherence and spatial coherence of the chirp modulation signal; an imaging optical system that forms an image of reflected light from an object illuminated with illumination light on a light receiving element; a combiner that combines the reflected light and the reference light, The light receiving element receives the combined reflected light and reference light, and detects an interference fringe signal generated by heterodyning between the linear chirp modulation signals of the illumination light and the reference light.

[0008] Furthermore, in order to solve the above-mentioned problems and achieve the object, a second aspect of the distance detection device of the present invention is a light source that generates illumination light in which chirp modulation is applied to amplitude modulation of carrier light; an imaging optical system that forms an image of reflected light from an object illuminated with illumination light on a light receiving element; Further, the optical fiber includes a mixer that multiplies the chirp-modulated electrical signal by the same chirp-modulated signal as the illumination light and detects an interference fringe signal, or a light-receiving element that also functions as a mixer by direct switching using the same chirp-modulated signal as the illumination light, The light receiving element detects the reflected light and converts the chirp modulation of the reflected light into an electrical signal.

[0009] A third aspect of the distance detection device of the present invention is the distance detection device of the first or second aspect, wherein the light receiving elements are two-dimensionally arranged, Alternatively, the one-dimensionally arranged light receiving elements are scanned one-dimensionally across the imaging surface of the imaging optical system in a direction intersecting the one-dimensional arrangement direction, Alternatively, by two-dimensionally scanning the imaging surface with a single light receiving element, Alternatively, the light receiving elements are arranged one-dimensionally on the imaging plane, and the directivity of the illumination light in a direction intersecting the one-dimensional array is increased to scan the object one-dimensionally in the direction. Alternatively, a single light receiving element is placed on the imaging plane, and the illumination light is increased in directivity to a beam to two-dimensionally scan the object. The apparatus is provided with a two-dimensional detection mechanism that detects the interference fringe signals at two-dimensional positions while capturing an image of the object.

[0010] A fourth aspect of the distance detection device of the present invention is the distance detection device of the third aspect, further comprising a determination mechanism that detects a position of interference light that interferes with detection of the distance to the object from an image captured by the two-dimensional detection mechanism and determines a local position where the distance to the object is detected, The distance to the local position of the object is detected while avoiding the interfering light.

[0011] A fifth aspect of the distance detection device of the present invention is the distance detection device of the third or fourth aspect, further comprising a determination mechanism that determines a position of interfering light that interferes with reception of optical wireless communication from an image captured by the two-dimensional detection mechanism, and determines a position of a transmitter of an optical wireless communication device that receives the optical wireless communication, The optical wireless communication is received while avoiding the interfering light. [Effects of the Invention]

[0012] By using chirp modulation of the light amplitude, high lateral resolution and distance detection accuracy can be achieved over a wide range of several centimeters to several hundred meters. In addition, by using an image sensor, color images and distance images can be detected simultaneously, enabling the realization of a small, low-cost visual sensor.

[0013] Additionally, the imaging device and distance detection device are visual sensors essential for autonomous driving. Using the information from these two sensors, artificial intelligence (hereinafter referred to as "AI" or "determination mechanism" as appropriate) performs object detection in the external world and makes autonomous driving decisions. Object detection for autonomous driving identifies objects around the vehicle, detects the size and position or area of objects relevant to autonomous driving, and then detects the relative position, speed, and direction of those objects and the vehicle. The higher the level of autonomous driving, the more important these two sensors become. [Brief explanation of the drawings]

[0014] [Figure 1] The graph shows the change in the amplitude of the carrier light over time. [Figure 2] Indicates the frequency of the chirp modulation. [Figure 3] 1 shows a distance detection device according to a first embodiment. [Figure 4] 10 shows the configuration of a distance detection device according to a second embodiment. [Figure 5] 1 shows a modification of the first embodiment. [Figure 6] 4 shows the flow of processing of the electrical signal 43. [Figure 7] Harmonics are shown. [Figure 8] The frequency of the interference fringe signal is shown. [Figure 9] This shows the relationship between the coherence of the carrier light and the size of the light source. [Figure 10] The front configuration of the image sensor is shown. [Figure 11] Indicates a situation where multiple objects exist. [Figure 12] 1 is a timing chart showing the reading of pixels in a horizontal line of an image sensor; [Figure 13] The front configuration of the filter is shown. [Figure 14] 10 shows a schematic configuration of a distance detection device according to a modified example of the second embodiment. [Figure 15] 10 shows another schematic configuration of the distance detection device according to the modified example of the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0015] The following describes the effects of the embodiments and modifications of the present invention. When specifically describing the effects of the embodiments and modifications, specific examples will be used. However, these exemplified embodiments and modifications are merely a portion of the examples included in the present invention, and there are numerous variations in its aspects. Therefore, the present invention is not limited to the exemplified embodiments and modifications.

[0016] Before describing the embodiments and modifications, the advantages and disadvantages of distance detection devices such as a stereo camera, a ToF camera, a LiDAR, a millimeter-wave radar, and an ultrasonic sensor will be described below.

[0017] (stereo camera) Distance detection using a stereo camera is the same as distance perception using binocular vision, and is a triangulation method. The amount of misalignment between images captured by the two cameras is detected locally to obtain a distance image.

[0018] The advantage of a stereo camera is that it can detect color images and distance images simultaneously. Also, because it is a passive system, there is no interference between the stereo cameras.

[0019] The disadvantages of stereo cameras are explained below. Stereo cameras use correlation calculations to detect the amount of deviation between each local area of the images captured by the two cameras. This is how a distance image is obtained. As a result, the distance detection accuracy and azimuth resolution vary based on the frequency components of each local area of the image.

[0020] For example, if there is little change in brightness in a local area, the distance detection accuracy and azimuth resolution will decrease. If there is a periodic change in brightness in a local area, false detection will occur. Also, because stereo cameras are passive, their accuracy is low in dark places such as at night.

[0021] Furthermore, in principle, stereo cameras require high accuracy in the six-axis spatial positioning of the two cameras, which means that distance detection accuracy is prone to degradation due to changes over time and temperature.

[0022] Furthermore, the stereo angle of a stereo camera decreases as the object moves farther away from the camera. As the stereo angle decreases, the amount of deviation between the two images decreases exponentially. This causes a rapid decrease in distance detection accuracy.

[0023] Increasing the stereo angle to improve distance detection accuracy for distant objects requires larger cameras. This makes the positional accuracy of the two cameras even more stringent. For this reason, the distance detection accuracy and azimuth resolution of stereo cameras are generally lower than those of other distance detection devices.

[0024] To mitigate the above disadvantages, there is an active stereo camera that projects highly autocorrelated patterns. However, the accuracy of detecting the distance to distant objects remains low, and interference between the stereo cameras occurs.

[0025] Another method has been reported that uses AI to detect distance from a single camera image. The AI learns a large amount of data on various image information of an object that changes depending on the relative position of the object and the vehicle, such as changes in the object's size, shape, texture, out-of-focus, and overlap due to motion parallax. This is how the system detects the distance to an object.

[0026] This method is similar to the spatial perception processing that takes place in the dorsal pathway of the visual cortex, and is intended to compensate for the fact that distance perception in human binocular vision rapidly declines for distant objects. However, most spatial perception processing is based on triangulation of monocular disparity. Therefore, for the same reasons as mentioned above, highly accurate distance detection is difficult. Even with one eye closed, humans can roughly determine the distance to an object based on past experience. However, driving a car with one eye (monocular) is extremely dangerous, even with a lot of experience.

[0027] If it were possible to obtain the necessary distance information from monocular images with experience, humans would not have evolved to use binoculars. Similarly, no matter how much AI learns, it is difficult to detect high-precision distance images from information from a single camera.

[0028] To operate a vehicle that moves much faster than a human, faster, more accurately, and safely than a human, requires highly accurate distance detection that far exceeds human vision. Due to the disadvantages mentioned above, it is said that there are limits to the application of stereo cameras to autonomous driving. Furthermore, it can be said that a method of detecting distance images using AI from a single camera image is even more difficult.

[0029] ToF cameras use the ToF (time of flight) method for distance detection. In the ToF method, the intensity of illumination light is modulated at a constant frequency to illuminate the measurement space. The light reflected from the illuminated object is imaged using an optical system. When the light is received by the image sensor, switching detection is performed at the same frequency for each pixel of the image sensor. From the amount of charge obtained, the phase of the signal modulating the reflected light (the reflection delay time) is detected for each pixel. A distance image is then obtained.

[0030] The advantages of ToF cameras are that they can form images using an optical system, which allows them to obtain distance images with high lateral resolution, their distance detection accuracy remains constant regardless of distance, and they are small and inexpensive.

[0031] The disadvantage of ToF cameras is that there is a limit to the detection speed of the light receiving part of the image sensor, making it difficult to measure close and long distances in terms of the signal-to-noise ratio.In addition, the ToF method is, in principle, strongly affected by background light (sunlight, etc.), which causes significant interference between ToF cameras.

[0032] LiDAR mainly uses the ToF method for distance detection, which measures distance while scanning an infrared laser beam with a wavelength of 1550 nm in two dimensions to obtain a distance image.

[0033] The advantages of LiDAR are its high lateral resolution, and the fact that some single light receiving sensors have fast detection speeds, high sensitivity and high signal-to-noise ratios, and because infrared light with a wavelength of 1550 nm is rare in external light, it outperforms ToF cameras in both short and long-distance measurements.

[0034] In addition, because the infrared laser beam is highly directional, there is a low probability of beam overlap between LiDARs, making it less susceptible to interference between LiDARs.

[0035] The disadvantages of LiDAR are that it requires a two-dimensional scanning mechanism and the light-receiving sensor is expensive, which poses challenges in terms of durability and cost. Furthermore, for reasons that will be explained later regarding millimeter-wave radar, ToF LiDAR is less accurate than chirp modulation millimeter-wave radar in long-distance detection.

[0036] Distance detection by millimeter-wave radar uses a linear chirp modulation method. The millimeter wave frequency is linearly chirp modulated and transmitted and received via an antenna. The received wave is multiplied by a reference signal that has the same chirp modulation as the transmitted wave, and heterodyne detection is performed. This is converted into an interference fringe signal (IF signal) with a frequency proportional to the distance to the object. The interference fringe signal is then converted into frequency components using an FFT (Fast Fourier Transform). The distance to the object is measured by detecting the peak value of the frequency component.

[0037] Although millimeter-wave radar has low distance resolution for distance detection, its high signal-to-noise ratio allows it to achieve distance detection accuracy comparable to that of LiDAR by detecting peak values.

[0038] The advantage of millimeter-wave radar is its highly autocorrelated encoding technique called chirp modulation. The chirp modulation method improves the power signal-to-noise ratio of distance detection in proportion to the modulation bandwidth and chirp time, and spreads the spectrum of ambient light when detecting distance. This makes it resistant to jamming.

[0039] Furthermore, differences in reflectivity and reflection distance make it possible to detect the distance to an object while excluding unwanted reflections from rain, fog, etc. Therefore, even though the attenuation of millimeter waves and infrared rays with a wavelength of 1550 nm in rain is about the same (Figure 1 of Non-Patent Document 1), millimeter-wave radar far outperforms LiDAR in long-distance detection. Another major advantage is that it can detect the relative speed of an object using simple means, and the device is relatively inexpensive.

[0040] The greatest advantage of millimeter-wave radar is its ability to apply various jamming suppression technologies developed in radar and communications. Chirp modulation, a highly autocorrelated coding technique, spreads and suppresses the frequency components of jamming signals. Furthermore, by applying special double coding, jamming can be further suppressed, making it possible to avoid intentional jamming (deceptive jamming) in addition to denial jamming.

[0041] The disadvantage of millimeter-wave radar is that, due to the realistic size of the antenna and the wavelength of the radio waves, the azimuth resolution (directivity) is two or more orders of magnitude lower than that of LiDAR, making it difficult to identify small or nearby objects. In particular, at close range, it is difficult to avoid clutter from the ground or guardrails due to side lobes.

[0042] Furthermore, there are objects with reflectivity different from that of optical images. Also, obtaining a distance image requires mechanical two-dimensional scanning or electronic scanning using an array antenna. Furthermore, when applying millimeter-wave radar to various applications, there is a major disadvantage in that the degree of freedom in design is limited by radio wave protection regulations.

[0043] To solve the above disadvantages (issues), it is known to perform distance detection by chirp-modulating the frequency of LiDAR laser light. This provides lateral resolution equivalent to that of optical images, and similar to millimeter-wave radar, it offers various benefits of the chirp modulation method. However, a laser light source that has a coherence length corresponding to distance detection of several hundred meters and enables highly linear chirp modulation is very large and expensive. However, the disadvantage of requiring an expensive, highly sensitive sensor and a two-dimensional scanning mechanism remains.

[0044] Ultrasonic sensors use a pulse radar method for distance detection. They detect the distance to an object from the time it takes for an ultrasonic pulse to be transmitted and received. The advantages are their small size and low cost. However, because ultrasonic waves are highly attenuated in the air and have low lateral resolution, they are used for very close range detection using multiple sensors.

[0045] (First embodiment) Next, an embodiment of the present invention will be described.

[0046] The distance detection device (imaging device) according to the first embodiment includes a light source that generates illumination light in which chirp modulation is performed on the amplitude modulation of carrier light; another light source that generates reference light by subjecting the amplitude modulation of carrier light to the same chirp modulation as that of the illumination light; The chirp modulation is modulated by a linear chirp modulation signal, a coherence reduction mechanism that reduces the temporal coherence and spatial coherence of both or one of the carrier lights while maintaining the temporal coherence and spatial coherence of the chirp modulation signal; an imaging optical system that forms an image of reflected light from an object illuminated with illumination light on a light receiving element; a combiner that combines the reflected light and the reference light, The light receiving element receives the combined reflected light and reference light, and detects an interference fringe signal generated by heterodyning between the chirp modulation signals of the illumination light and the reference light.

[0047] In this embodiment, distance detection is performed using a linear chirp modulation method that has excellent sensitivity and interference suppression. However, unlike millimeter-wave radar, chirp-modulated signals are transmitted and received using light as a carrier, rather than radio waves. Instead of chirp-modulating the frequency of the carrier light, chirp modulation is performed on the frequency that modulates the amplitude of the carrier light.

[0048] The temporal change in the amplitude of the carrier light is shown in Figure 1. Specifically, as shown in Figure 1, a small, low-cost, high-speed switching light-emitting diode is used as the light source of the carrier light, and the light from the light-emitting diode is turned on and off, with linear chirp modulation performed on the on-off frequency.

[0049] Fig. 2 shows the frequency of chirp modulation. Using Fig. 2, we will explain the distance detection method using chirp modulation.

[0050] Line 1 represents a reference (standard) chirp signal whose frequency changes linearly with time. Line 2 represents the chirp signal reflected from the object whose distance is being measured. The reflected chirp signal 2 generates a time delay 3 relative to line 1, the reference chirp signal, that corresponds to the round-trip distance to the object. As a result, heterodyning between the chirp signals generates a low-frequency interference fringe signal that corresponds to the frequency difference 4 between the chirp signals.

[0051] When chirp modulation has high linearity, the frequency of the interference fringe signal remains constant over the chirp time. As can be seen from Figure 2, as the optical path difference increases, the time delay 3 increases. The frequency difference 4 between the chirp signals increases, and the frequency of the interference fringe signal becomes higher. In this way, the frequency of the interference fringe signal is proportional to the distance to the object. Figure 2 also shows that the frequency of the interference fringe signal increases as the chirp modulation bandwidth 5 widens and decreases as the chirp time 6 becomes longer.

[0052] 3 shows the configuration of a distance detection device 10A according to the first embodiment. An amplitude modulator 8 applies linear chirp modulation to the amplitude of a carrier light 7 using a chirp modulation signal 9. The carrier light 7 is then used as illumination light 10 to illuminate an object 11, the distance of which is to be detected. An optical system 13 forms an image of reflected light 12 from the object 11 on a light receiving element 14. A reference light 15 is combined with the reflected light 12 by a combiner 16. The light receiving element 14 receives the combined light.

[0053] When the light is received by the light receiving element 14, heterodyning between the carrier lights and heterodyning between the chirp signals occurs. The reference light 15 is generated by using an amplitude modulator 21 to linearly chirp modulate the amplitude of the carrier light 17 with the chirp modulation signal 9.

[0054] When the coherence between carrier light 7 and carrier light 17 is low, the heterodyne amplitude between the carrier lights approaches a constant value of the product of their respective power values due to the time averaging of the photodetector. This constant value becomes the coefficient of the interference fringe signal generated by the heterodyne between the chirp signals, and the interference fringe signal 18 can be stably detected.

[0055] The frequency of the interference fringe signal 18 is proportional to the distance to the object 11. If the coherence between the carrier light 7 and the carrier light 17 is high, the heterodyne amplitude between the carrier lights will produce a specific frequency, making it impossible to detect the interference fringe signal, or will produce a frequency component equivalent to the interference fringe signal, causing noise in the interference fringe signal.

[0056] When a high-coherence light source such as a laser diode (LD), which is good at high-speed amplitude modulation, is used for generating and modulating the carrier light, the temporal coherence and spatial coherence of both or one of the carrier lights 7 and 17 are reduced by coherence reduction mechanisms 19 and 20. Then, the heterodyne amplitude between the carrier lights is brought close to a constant value which is the product of the respective power values, thereby ensuring the signal-to-noise ratio of the interference fringe signal 18. The specific configuration of the coherence reduction mechanisms 19 and 20 will be described later.

[0057] The interference fringe signal 18 generated by heterodyning between chirp signals is converted into frequency components by a frequency converter such as an FFT (not shown), and the peak values of the frequency components are detected by a correlator and interpolation circuit (not shown) and converted into the distance to the object. The correlator processing will be described later.

[0058] In this embodiment, a color image and a distance image can be detected simultaneously using an imaging element, and a small, low-cost visual sensor integrated with an imaging device can be realized.

[0059] (Second embodiment) The distance detection device 20A according to the second embodiment is a light source that generates illumination light in which chirp modulation is applied to amplitude modulation of carrier light; an imaging optical system that forms an image of reflected light from an object illuminated with the illumination light on a light receiving element, The light receiving element further includes a mixer that multiplies the chirp-modulated electrical signal by the same chirp-modulated signal as the illumination light to detect an interference fringe signal, or the light receiving element also functions as the mixer by direct switching using the same chirp-modulated signal as the illumination light, The light receiving element detects the reflected light and converts the chirp modulation of the reflected light into an electrical signal.

[0060] FIG. 4 shows the configuration of a distance detection device 20A according to the second embodiment. A modulator 23 applies linear chirp modulation to the amplitude of carrier light 22 using a chirp modulation signal 24. Then, an object 25, the distance of which is to be detected, is illuminated with the illumination light. Reflected light 26 from object 25 is imaged on a light receiving element 28 by an optical system 27. Direct detection is performed by the light receiving element 28, which is capable of high-speed detection, and the chirp-modulated signal of reflected light 26 is converted into an electrical signal.

[0061] Thereafter, a mixer 29 multiplies the chirp modulation signal 24 as a reference signal and converts it into an interference fringe signal 30. Then, as in the first embodiment, the signal is converted into the distance to the object through processing by a frequency converter, a correlator, and an interpolation circuit (not shown).

[0062] In this case, instead of the light receiving element 28 and the mixer 29, a light receiving element capable of direct switching may be used, and heterodyne detection may be performed by switching using the chirp modulation signal 24, thereby detecting the interference fringe signal 30.

[0063] The higher the coherence of a light-emitting diode light source, the more possible it is to perform high-speed amplitude modulation by direct switching or by using an external element such as a Mach-Zehnder element. In the second embodiment, the carrier light may be high-coherence light. Therefore, by using such a high-coherence light-emitting diode or light source, the bandwidth of the chirp modulation can be widened. This allows for highly accurate distance detection.

[0064] The effects of the distance detection device according to the first embodiment and the distance detection device according to the second embodiment will be described. The distance detection device according to the first embodiment and the distance detection device according to the second embodiment can satisfy all the requirements for a distance detection device and can solve the problems (disadvantages) that conventional distance detection devices have.

[0065] For example, when infrared light with a wavelength of 1550 nm is used as the carrier light, long-distance detection is possible at the same level as millimeter-wave radar. In addition, because an imaging optical system can be used, much higher azimuth resolution can be achieved than with millimeter-wave radar.

[0066] High azimuth resolution can be achieved from close to long distances using autofocus and pan focus, etc. This means that there is no ghost reflection (clutter) at close range caused by side lobes like millimeter wave radar.

[0067] Furthermore, the chirp modulation method spreads (distributes) the power used to emit the illumination light within the chirp modulation time in accordance with the chirp modulation and transmits it. As a result, even if the illumination light intensity is the same, the total illumination power, including the time axis, is two to three orders of magnitude higher than that of the ToF method, resulting in a correspondingly higher power-to-noise ratio for distance detection. If the illumination light intensity (power) of a ToF LiDAR or camera is increased to achieve the same SN ratio, the reflected light from close-range objects becomes too strong, quickly saturating the light-receiving circuit. As such, the ToF method has a limited dynamic range (SN ratio), making it impossible to measure distances over a wide range, such as from a few centimeters to several hundred meters.

[0068] In this way, the chirp modulation method spreads the power of the illumination light over the chirp modulation time, which provides ample dynamic range for the light-receiving circuit system and avoids circuit saturation caused by strong reflections at close range, as occurs with the ToF method, making it possible to detect distances with high accuracy over a wide range from a few centimeters to several hundred meters.

[0069] Furthermore, because the system uses chirp modulation of the carrier light amplitude, it is possible to dually apply radar jamming suppression technology and optical communication interference avoidance technology. Furthermore, by utilizing its high azimuth resolution and linking it with a judgment mechanism such as AI, it is possible to detect distance while avoiding interference from the illumination light of distance detection sensors on other vehicles or strong external light.

[0070] Furthermore, according to the first embodiment, it is possible to simultaneously detect a color image and a range image using an imaging element, and it is possible to realize a small, low-cost sensor integrated with an imaging device.

[0071] In addition, since there are no regulations such as radio wave protection regulations, there is a high degree of freedom in design, making it easy to apply to a variety of uses, such as robots and drones.

[0072] In addition, by taking advantage of its high lateral resolution and linking it with a judgment mechanism such as AI, it is possible to receive optical wireless communications.

[0073] FIG. 5 shows a modification of the first embodiment. A linear chirp modulation signal 32 of, for example, 0.1 to 1.1 GHz generated by a synthesizer oscillator 31 or the like turns on and off the output light of an illumination light source 33, and the output light is used as illumination light 34 to illuminate an object 35 for which distance is to be detected.

[0074] The illumination light source 33 uses a light emitting diode capable of high speed switching, such as an infrared LD with a central wavelength of 1550 nm or an SLD (super luminescent diode).

[0075] Infrared LDs are often low-priced due to their large distribution in optical communications, etc. In addition, infrared light with a wavelength of 1550 nm is highly safe for human eyes, has little external light, is little attenuated in the atmosphere, and has almost the same attenuation in rain as millimeter-wave radar of 76 to 77 GHz.

[0076] The output light of the illumination light source 33 is turned on and off by direct switching of the light emitting diode or by using an external junction element such as an EA (Electro-Absorption) modulator or a Mach-Zehnder (MZ) modulator.

[0077] Reflected light 36 from an object 35 is imaged on an image sensor 38 by an imaging optical system 37. At this time, reference light 40, which is the output light of a reference light source 39 that is turned on and off by a linear chirp modulation signal 32, is combined with the reflected light 36 by a combiner 41.

[0078] When the light is received by the image sensor 38, heterodyning between the chirp signals generates an interference fringe signal at each light-receiving pixel of the image sensor 38, the frequency of which corresponds to the distance image of the object 35. The reference light source 39 is an infrared LD or SLD with a central wavelength of 1550 nm.

[0079] At this time, the position of the reference light source 39 is set by the optical system 42 so that it is optically conjugate with the position of the exit pupil of the imaging optical system 37. This allows the traveling directions of the wavefronts of the reference light and the chief ray that form an image on each pixel of the imaging element 38 to coincide, improving the efficiency of heterodyne and suppressing unevenness in intensity for each pixel.

[0080] Then, by repeating the imaging of the imaging element 38 at high speed, the interference fringe signals are sampled in parallel for all pixels and converted into an electrical signal (interference fringe signal) 43.

[0081] 6 shows the flow of processing the electrical signal 43. A frequency converter 44 such as an FFT converts the signal into frequency components for each pixel. A correlator 48 and a distance converter 49 detect the peak values of the frequency components and convert them into a distance image. The processing by the correlator 48 will be described later.

[0082] When the temporal and spatial coherence of the carrier light in the reflected light 36 and the carrier light in the reference light 40 are sufficiently low, the heterodyne amplitude between the carrier lights approaches the product of their respective powers, which is determined by the average exposure time of the sampling image capture by the image sensor 38 and the surface integral on the light-receiving pixels. This makes it possible to detect interference fringe signals with a high S / N ratio.

[0083] At this time, the heterodyne output I of the reflected light 36 and the reference light 40 is expressed by the following equation: I=K·Ps PL {1 / 2 cos[φ(t+d)-φ(t)]+1} (A) This becomes:

[0084] The reason why the heterodyne output I is expressed by equation (A) will be explained later. In the above formula, Ps is the power of the carrier light of the reflected light 36, PL is the power of the carrier light of the reference beam 40, φ(t) is the phase of the linear chirp (modulation) signal 32; t is time, d is the time delay due to the optical path difference between the reflected light 36 and the reference light 40; K represents a constant determined by the characteristics of the light-receiving pixel.

[0085] According to equation (A), heterodyning between chirp signals generates interference fringe signals at each pixel that are proportional to the distance to the object (proportional to d). The amplitude of these signals is proportional to the power Ps of the reflected light 36 at each pixel, and is amplified by the power PL of the reference light.

[0086] Here is a concrete example based on formula (A). When a rapidly rotating object is illuminated with a stroboscope, if the blinking rate of the strobe light matches the rotation period, the object will appear to be stationary. If the blinking rate of the strobe light is slightly slower than the rotation period, the object will rotate more slowly. If the blinking rate of the strobe light is slightly faster than the rotation period, the object will appear to rotate in the opposite direction.

[0087] At this time, the light that is imaged at a point on the observer's retina is a mixture of the light reflected from the object due to the indoor lighting, which has been brightness-modulated by the object's rotation, and the light reflected from the object, which has been brightness-modulated by the flashing of the strobe.

[0088] When the retina receives the reflected light, a heterodyne occurs between the two modulated signals, which are then time-averaged by the eye's persistence of vision, leaving behind a low-frequency intensity modulation that is the difference between the rotation of the object and the flashing of the strobe light, resulting in the observed phenomenon described above.

[0089] Room lighting and strobe light are incoherent lights and therefore have no coherence. The brightness of the object image changes in proportion to the product of the brightness (light intensity) of the room lighting and the strobe light. Whether the room light or the strobe light is made brighter, the object image will become brighter. In this case, the object image and rotational movement do not change. However, if the room light and strobe light are intentionally made coherent, the brightness of the object image will fluctuate, interference fringes will appear, and the rotational movement will appear to fluctuate.

[0090] A one-dimensional array of light-receiving sensors may be used instead of the imaging element 38. A distance image can also be obtained by sampling interference fringe signals while scanning the imaging surface of the imaging optical system 37 one-dimensionally in a direction intersecting the array direction with the one-dimensional array of light-receiving sensors, and performing the processing described above.

[0091] Furthermore, any one of the following configurations (a), (b), and (c) may be used. (a) A distance image can also be obtained by using a single light receiving element and scanning the imaging plane of the imaging optical system 37 two-dimensionally. (b) A distance image can also be obtained by placing a one-dimensional array of sensors on the imaging plane, increasing the directivity of the illumination light in a direction intersecting the sensor array direction to illuminate object 35, and scanning it one-dimensionally in a direction intersecting the array direction. (c) A distance image can also be obtained by placing a single light receiving element on the imaging plane of the imaging optical system 37 and scanning the object 35 two-dimensionally with illumination light converged into a beam.

[0092] In this way, with two-dimensionally arranged light receiving elements, Alternatively, the one-dimensionally arranged light receiving elements are scanned one-dimensionally in a direction intersecting the one-dimensional array direction across the imaging surface of the imaging optical system, Alternatively, by two-dimensionally scanning the image plane with a single light receiving element, Alternatively, by placing a one-dimensionally arranged light receiving element on the imaging plane and increasing the directivity of the illumination light in the direction intersecting the one-dimensional array, the object can be scanned one-dimensionally in the direction. Alternatively, a single light receiving element is placed on the imaging plane, and the illumination light is directed into a beam to scan the object in two dimensions. It is desirable to provide a two-dimensional detection mechanism that detects interference fringe signals at two-dimensional positions while simultaneously capturing an image of the object.

[0093] The processing of the correlator 48 shown in FIG. 6 will be described. Interference fringe signal 43 is converted into frequency components 45 by frequency converter 44. On the frequency axis, correlator 48 performs a correlation calculation between frequency components 45 of the interference fringe signal and a reference signal (standard signal) 47 that is a complex conjugate.

[0094] The output of the correlator 48 is subjected to data interpolation in a distance conversion unit 49. Then, a peak value is detected and converted into a distance L to the object 35 according to equation (8) described later.

[0095] Fig. 7 shows harmonics. We will now describe the reference signal generator 46 that generates the reference signal 47. The frequency components of the square wave that performs chirp modulation by switching the carrier light include, as shown in Fig. 7, a fundamental wave 53 as well as harmonics that are 2n+1 times (n: natural number including 0) times the fundamental wave.

[0096] The symbol 54 (first harmonic) in Figure 7 is the first harmonic when n = 1. The amplitude of harmonics for n = 2 and above is small and can be ignored because they are suppressed by the response time of the light source switching. Furthermore, if the rise and fall times of the light source switching are different, a harmonic will also appear at position 2n, but this too has a small amplitude and can be ignored.

[0097] 8 shows the frequencies of the interference fringe signal. As described above, frequency component 45 of the interference fringe signal has frequency 55 of the interference fringe signal generated by heterodyning the chirp signal of fundamental wave 53 and frequency (component) 56 of the interference fringe signal generated by heterodyning the chirp signal of first harmonic 54.

[0098] The frequencies (components) 55, 56 of these two interference fringe signals change according to the distance to the object 35, as shown in Fig. 8. When the distance to the object 35 increases and the frequency 56 of the interference fringe signal generated by the chirp signal of the first harmonic 54 exceeds the Nyquist frequency 57 of sampling, an aliasing component 58 is generated. When the aliasing component 58 falls below zero frequency, another aliasing component 59 is generated.

[0099] As described above, the frequency component 56 of the interference fringe signal changes depending on the distance to the object 35. For this reason, the reference signal 47 used in the correlation calculation by the correlator 48 is changed in accordance with the change in the frequency component 56 of the interference fringe signal.

[0100] In this way, a correlation calculation is performed between the frequency components 45 of the interference fringe signal, which change depending on the distance to the object 35, and the reference signal 47, which takes this into consideration. Then, by performing data interpolation in the distance conversion unit 49 to detect the peak value, the harmonic components can be concentrated at the peak value. As a result, the S / N ratio of the interference fringe signal can be increased. As will be described later, the distance detection accuracy is proportional to the distance resolution and the S / N ratio; for example, when the S / N ratio is 40 dB, the distance detection accuracy is about 1 / 40 of the distance resolution.

[0101] Furthermore, the frequency 55 and initial phase of the interference fringe signal can be calculated using equation (6), which will be described later. The frequency 56 of the interference fringe signal can also be calculated using equation (7), since a and f0 in equation (6) can be tripled.

[0102] The aliasing components can be calculated from the results of these calculations, and the reference signal 47 can be calculated by normalizing the amplitudes of these components and taking the phase conjugate.

[0103] Alternatively, a reflector can be placed at an appropriate distance, the frequency components of the interference fringe signal can be measured in advance, and reference signals at other distances can be calculated by interpolation. If the rise and fall times of the LD switching waveform change due to temperature changes, the amplitude of the harmonics may change slightly, or harmonics with twice the frequency of the fundamental wave may be generated. Here, these amplitudes are small and can be ignored.

[0104] We explain why heterodyning between chirp signals generates an interference fringe signal, and why the heterodyne amplitude between carrier lights approaches a constant value when the coherence of the carrier lights is low.

[0105] When the carrier light has low coherence and a spatially random phase, the heterodyne outputs of the reflected light 36 and the reference light 40 differ depending on the spatial coherence on the light-receiving pixels.

[0106] Therefore, the electric field component of the carrier light of the reflected light 36 and the electric field component of the carrier light of the reference light 40 (see FIG. 5) are assumed to be functions of time t and position r on the light receiving surface of the light receiving element. In the case of a chirp signal, the wavelength of the fundamental wave is 10 times the size of the light receiving element. 4 Therefore, the phase change relative to the change in position r is extremely small, and the explanation is given as a function of time t only.

[0107] where: The carrier light of the reflected light 36 is As(r,t), The carrier light of the reference light 40 is AL(r,t), If the square wave functions with amplitude 1 that chirp modulate the amplitudes of the carrier light As(r,t) and AL(r,t) are S(t+d) and S(t), respectively, then The electric field components of the reflected light 36 and the reference light 40 imaged at position r on the light-receiving pixel are As(r, t)S(t+d) and AL(r, t)S(t), respectively. d indicates the delay time according to the optical path difference between the reflected light 36 and the reference light 40 .

[0108] In this case, the heterodyne amplitude H(r, t) when the reflected light 36 and the reference light 40 are combined and received is given by H(r,t)=As(r,t) S(t+d)·AL(r,t) S(t) (B) This becomes:

[0109] (B) By swapping S(t+d) and AL(r,t), H(r,t)=As(r,t) AL(r,t)·S(t+d) S(t) (C) Let's say.

[0110] As(r,t) AL(r,t) represents the heterodyne amplitude between the carrier light of the reflected light 36 and the reference light 40, and S(t+d) S(t) represents the heterodyne amplitude between the chirp signals.

[0111] In this case, when the correlation distance on the photosensitive pixel of both or one of the carrier lights As(r,t) AL(r,t) is extremely short in space and time, as described below, the heterodyne amplitude As(r,t) AL(r,t) between the carrier lights approaches a constant value regardless of the position on the photosensitive pixel due to the exposure time averaging of the sampling image. Then, due to the surface integral on the photosensitive pixel, it converges to the product of the respective power values, and the interference fringe signal generated by the heterodyne of S(t+d) S(t) can be detected with a good S / N ratio.

[0112] where: The two positions on the light-receiving pixel are r1, r2, When the exposure time of sampling imaging is τ, The correlation function C(r1,r2) between H(r1,t) and H(r2,t) is

[0113]

number

[0114] The heterodyne output I of the reflected light 36 and the reference light 40 is obtained by integrating the correlation function C(r1, r2) with respect to r1 and r2 on the light-receiving pixel using the following equation: K is a constant determined by the characteristics of the light-receiving pixel of the image sensor 38.

[0115]

number

[0116] Square wave functions S(t+d) and S(t) with amplitude 1 become the same square wave function even when squared. Therefore, H(r1,t) and H(r2,t) in equation (1) can be written as follows: H(r1,t) H(r2,t)=As(r1,t) AL(r1,t)·As(r2,t) AL(r2,t)·S(t+d) S(t) (D) Let's say.

[0117] Furthermore, the explanation will be given by replacing S(t+d) and S(t) with the expressions for the fundamental wave and DC component of those frequency components, cosφ(t+d)+1 and cosφ(t)+1.

[0118] The same idea can be used to explain the interference fringe signals generated by the harmonic components of S(t+d) and S(t). Here, the amplitude coefficients are omitted because they are meaningless for the purpose of explanation.

[0119] (D) Replace S(t+d) and S(t) with cosφ(t+d)+1 and cosφ(t)+1, and swap the positions of AL(r1,t) and As(r2,t), H(r1,t) H(r2,t)=As(r1,t) As(r2,t)·AL(r1,t) AL(r2,t)·[cosφ(t+d)+1] [cosφ(t)+1] (E) Let's say.

[0120] Expand [cosφ(t+d)+1] [cosφ(t)+1] in equation (E) using a product-sum formula for trigonometric functions, then substitute into equation (2) via equation (1). If the temporal and spatial coherence of the carrier light, As(r,t) and AL(r,t), is sufficiently low, the spatial correlation distance between the carrier light, As(r1,t) and As(r2,t), and between AL(r1,t) and AL(r2,t), on the photodetector is extremely short, and the heterodyne amplitudes of As(r1,t) As(r2,t) and AL(r1,t) AL(r2,t) in the above equation are detected as square-law detection on the time axis regardless of the position of r. Therefore, they approach a constant value due to the time averaging of the sampling image.

[0121] Then we are left with the following term, which represents the low frequency interference fringe signal: As(r1,t) As(r2,t) AL(r1,t) AL(r2,t) {1 / 2 cos[φ(t+d)-φ(t)]+1} The other terms converge to 0 by time averaging of the sampling images.

[0122] And 1 / 2·cos[φ(t+d)-φ(t)]+1 does not have the variable r. Therefore, it is excluded from the integral of equation (2) and becomes a coefficient term. As(r1,t) As(r2,t)·AL(r1,t) AL(r2,t) in the above equation is calculated by the time average of equation (1) and the integration on the photosensitive pixel of equation (2), and is the product of the total power of the carrier light on the photosensitive pixel, K Ps Converge to PL.

[0123] As a result, the heterodyne output I of the reflected light 36 and the reference light 40 is expressed as follows: I=K Ps PL {1 / 2·cos[φ(t+d)-φ(t)]+1} (3) This becomes:

[0124] From equation (3), we can see that heterodyning between chirp signals generates an interference fringe signal of low frequency (symbol 2 in Figure 2) corresponding to the delay time d (symbol 3 in Figure 2), and its amplitude is proportional to the power Ps of the reflected light and is amplified by the power PL of the reference light.

[0125] In this example, a method for ensuring the signal-to-noise ratio of the interference fringe signal will be described. When the reflected light 36 and the reference light 40 are combined and received to detect the interference fringe signal, if the coherence of the carrier light is not sufficiently low, low frequency components equivalent to the interference fringe signal will be generated by heterodyning between the carrier lights. These components will not be averaged over the exposure time of the sampling image, and will remain as noise in the interference fringe signal (beat noise, speckle noise).

[0126] When the bandwidth of the spatial frequency components of the carrier light of the reflected light 36 and the reference light 40 on the light-receiving pixel is widened, the spatial frequency components of the heterodyne amplitude between the carrier lights are further widened by their convolution integral. This increases the coefficient K Ps PL of the interference fringe signal in equation (3). On the other hand, the bandwidth of the noise power remains unchanged because the sampling exposure time is constant, and the power SNR of the interference fringe signal increases.

[0127] Here, the temporal coherence of light is proportional to the inverse of the bandwidth of the spatial frequency components in the direction of light propagation, and the spatial coherence is proportional to the inverse of the bandwidth of the spatial frequency components in a plane normal to the direction of light propagation.

[0128] Therefore, the above statement that "increasing the bandwidth of the spatial frequency components of the carrier light of the reflected light 36 and the reference light 40 on the light-receiving pixel increases the power S / N ratio of the interference fringe signal" can be rephrased as "reducing the temporal coherence and spatial coherence of the carrier light on the light-receiving pixel increases the S / N ratio of the interference fringe signal."

[0129] Infrared LDs with a wavelength of 1550 nm are capable of direct switching in the GHz range, and are small and inexpensive, making them suitable as light sources for chirp modulation by on-off switching.

[0130] Because the temporal and spatial coherence of the output light is relatively high, the frequency bandwidth is narrow at 15 GHz. If the maximum detection distance is 150 m, the chirp time is 15 ms, and the bandwidth is 1 GHz, the frequency of the interference fringe signal will be 67 kHz according to equation (7) described below.

[0131] The sampling frequency must be more than twice that, at about 150 kHz. At this point, the ratio of the frequency bandwidth of the carrier light to the bandwidth of the low-pass cutoff frequency due to sampling exposure, i.e., the power SNR, is 94 dB.

[0132] The amplitude SNR of the interference fringe signal is half that, at 47 dB. When considering the attenuation of the interference fringe signal over long distances, if an SNR of 60 dB is required, this means that there is a shortfall of about 12 dB. Therefore, the following method is used to broaden the bandwidth of the carrier light and ensure the SNR.

[0133] Next, a method for reducing the temporal and spatial coherence of the carrier light will be described. Coherence can be reduced by arranging multiple LD light sources two-dimensionally on a plane normal to the main axis of the light from the light source and combining the output light, while also ensuring the light intensity of the light source.

[0134] The coherence of the combined light can be reduced linearly in proportion to the diameter of the light source beams irradiated from the two-dimensionally arranged LD light sources. In this case, the reduction of spatial coherence is much more efficient than the reduction of temporal coherence.

[0135] In order to make the spatial frequency bandwidth of the multiple LD lights combined on the light-receiving pixel surface wide and uniform, the spacing between the LD light sources arranged two-dimensionally is appropriately selected in consideration of the frequency bandwidth (wavelength bandwidth) of the LD light source and the waveform of its envelope.

[0136] In this case, similar to staggered tuning, by shifting the center wavelengths of multiple LDs and overlapping parts of the frequency bands to combine them into a wide, uniform signal, it is possible to reduce spatial coherence, including temporal coherence, even more efficiently.

[0137] When the light source assembly structure requires a large spacing between light sources, the light sources can be divided into a finer arrangement using a collimator optical system, a fly's eye optical system, and a Kohler illumination optical system. Furthermore, by combining this with the above method, coherence can be reduced efficiently. At this time, the length of the fly's eye lens can be randomized to reduce temporal coherence as well.

[0138] Furthermore, if the switching speed allows, low-coherence light-emitting diodes such as surface-emitting SLDs and LEDs may be used instead of LDs. In recent years, efforts have been made to further increase the output power and switching speed of these low-coherence light-emitting diodes in anticipation of visible light communications, optical wireless, Li-Fi (Light Fidelity), and other applications.

[0139] Instead of direct switching, high-speed switching can also be achieved by using an external switching element such as an EA element. However, since these low-coherence light sources often have low temporal coherence but high spatial coherence (point light source properties), they must be combined with the above-mentioned means for reducing spatial coherence.

[0140] Furthermore, by using separate light sources for the carrier light of the illumination light and the reference light, the phase and polarization plane between the carrier lights can be made random, and beat noise can be spread over a wide band.

[0141] Additionally, the illumination light can be split using a splitter and guided through a fiber bundle to be used as reference light. The coherence length of an LD is approximately 10 cm. Therefore, if the round-trip distance to the object is 1 m or more, the temporal coherence of the reflected light is already sufficiently lost. Furthermore, by using a multimode fiber bundle or the like and adjusting the length and diameter of the fiber bundle, the temporal and spatial coherence can be further reduced.

[0142] In addition to the methods mentioned above, there are many other ways to reduce the coherence of the carrier light, but no single method can achieve nearly incoherent light. Depending on the application, several methods can be combined to achieve the necessary and sufficient reduction in coherence of the carrier light and ensure the signal-to-noise ratio of the interference fringe signal to the beat noise mentioned above.

[0143] Figure 9 shows the relationship between the coherence of the carrier light and the size of the light source. In either case, the size of the light-emitting section of the illumination light and reference light is set within the range 61 shown in Figure 9. Then, by applying the same chirp modulation to the illumination light and reference light, it is necessary to sufficiently reduce the temporal and spatial coherence of the carrier light while maintaining the temporal and spatial coherence of the chirp signal.

[0144] In Fig. 9, graph 62 shows the decrease in spatial coherence of the carrier light relative to the size of the light-emitting section, and graph 63 shows the decrease in spatial coherence of the chirp signal. In Fig. 9, due to space limitations, range 61 is depicted narrowly, but since the ratio of the wavelengths of the carrier light and the chirp signal is about five orders of magnitude, the actual range 61 is quite wide, and it is easy to set the above.

[0145] The graph 62 of the reflected light 36 changes depending on the distance to the object 35, the magnification (angle of view), and the size of the light-receiving pixel. For this reason, the graph 62 uses the graph of the coherence reduction under the worst conditions, and the graph 63 of the chirp signal uses the graph under the best conditions for coherence reduction.

[0146] In the case of the reference light 40, the distance from the light source to the light receiving element is short and constant, so the settable range 61 is wide and it is easy to reduce the coherence.

[0147] Next, the image sensor 38 that simultaneously detects an RGB image and a distance image will be described. An image sensor with 1.34 million pixels and wavelength characteristics of 400nm to 1700nm is commercially available. The light receiving section is made of InGaAs semiconductor, and the circuit system such as the readout mechanism is made of Si, making it a two-layer image sensor.

[0148] Figure 10 shows the front configuration of the image sensor. Using this type of image sensor, RGB color filters and IR wavelength filters are arranged in a mosaic pattern, as shown in Figure 10. The wavelength characteristics of the IR filter have a center wavelength of 1550 nm, and are matched to the wavelength bands of the illumination light 34 and reference light 40 to prevent noise caused by unnecessary external light.

[0149] By driving such an image sensor 38 at high speed, the interference fringe signals generated by the IR pixels are sampled in parallel for all pixels, and a distance image is detected after processing by the frequency converter 44, correlator 48, and distance conversion unit 49 shown in Figure 6.

[0150] In parallel, a large number of RGB images and IR images obtained by high-speed imaging are cumulatively added by a cumulative adder (not shown) to obtain RGB images and IR images with a wide dynamic range.

[0151] From these RGB and IR images, AI51 can detect external objects, as well as determine the position of interfering illumination light from distance detection devices on other vehicles and external light, and can eliminate pixels of interference fringe signals corresponding to those positions. AI51 is an essential judgment mechanism for object detection in autonomous driving and robots. AI51 object detection will be described later.

[0152] We will now describe how to apply the above-mentioned method of obtaining a distance image to a robot's visual sensor. The RGB image, IR image, and distance image are merged, and the AI51 is trained as an image with five channels of information per pixel.

[0153] Many of the objects handled by robots cannot be identified by their shape, texture, or color alone in an RGB image. For this reason, the accuracy of classification of work objects can be improved by using 3D shape information (distance image) obtained by distance detection in addition to the image. This also makes it possible to detect the 3D position of the robot's handpiece.

[0154] For example, the range of distance detection for a work object is divided into two stages: 0 to 0.3 m and 0.3 to 3 m. When the distance is 3 m, the chirp modulation width is set to 3 GHz, and the distance detection accuracy is 1.25 mm. This results in an interference fringe signal frequency of 4 kHz. The frame rate required for the image sensor is 8000 frames per second.

[0155] When the robot handpiece approaches within 0.3 m, the chirp modulation bandwidth is switched to 30 GHz to set the distance detection accuracy to 125 μm, and the frequency of the interference fringe signal becomes 4 kHz.

[0156] The frame rate required for the image sensor is also 8000 frames per second, which is a level that can be achieved with current image sensors with high-speed imaging. Switching exceeding 30 GHz can be achieved by switching the output light of the LD using a Mach-Zehnder element or similar.

[0157] As described above, from the five-channel images, AI51 can determine interference from multiple distance sensors and external light used in the robot's workspace and avoid their effects.

[0158] Next, the driving of the image sensor 38 when detecting an RGB image and detecting a long distance at the same time, as in a vehicle, will be described.

[0159] In the case of a vehicle, the detection distance is as long as 150 m, so even if the chirp modulation bandwidth is set to 1 GHz, the frequency of the interference fringe signal will be as high as 67 kHz. If this interference fringe signal is sampled in parallel across all pixels using the high-speed imaging of the image sensor 38, the sampling theorem dictates that high-speed imaging of 134,000 frames per second or more will be required, which exceeds the high-speed imaging limit of current image sensors.

[0160] However, object detection in a vehicle simply involves detecting an object 35 related to autonomous driving and detecting its distance. For this purpose, AI object detection techniques are used, such as bounding boxes that detect the size and position of objects related to autonomous driving, and semantic segmentation that detects the area and position. Then, only the interference fringe signals of IR pixels corresponding to the local parts of the object 35 where distance detection is required are read out. This reduces the number of pixels to be read out, allowing the sampling rate for the interference fringe signals to be sufficiently high.

[0161] 11 shows a situation where multiple objects exist. As shown in FIG. 11, when there are multiple automobiles 70A and 70B, which are objects related to safe driving, the AI 51 determines multiple local areas 71, 72, 73, and 74 from the RGB image and the IR image to detect the distances of the objects 70A and 70B while avoiding interfering lights 75 and 76.

[0162] Reference numeral 52 in FIG. 6 denotes a signal transmitted by AI51 to the readout mechanism of the image sensor 38 in FIG. 5, including the pixel address of the local position of the object where the interference fringe signal is detected, determined from the RGB image and the IR1 image, and the pixel address of the position where the optical wireless communication is received, determined from the IR2 image.

[0163] In response to the result (reference numeral 52 in FIG. 6), the readout mechanism of the image sensor 38 repeats sequentially reading out the interference fringe signals of the plurality of IR pixels determined by the AI 51 by random access so as to have the same sampling rate.

[0164] This makes it possible to ensure the required sampling rate by simply increasing the frame rate of the image sensor 38 by about one digit. Interfering light 75 and interfering light 76 indicate the positions of the illumination light sources of the distance detection devices of other vehicles that cause interference.

[0165] 12 is a timing chart showing the readout of pixels on a horizontal line of the image sensor. A method for high-speed sampling of interference fringe signals of IR pixels determined by AI 51 while simultaneously reading out the RGB image and IR image using the image sensor 38 will be described.

[0166] By sequentially repeating the reading of pixels in horizontal lines of the image sensor 38 and the reading of horizontal lines containing IR pixels determined by AI51, it is possible to rapidly sample interference fringe signals of multiple IR pixels determined by AI51 while simultaneously reading the RGB image and IR image.

[0167] When a CMOS image sensor is used for the image sensor 38, if the horizontal lines including the IR pixels (corresponding to reference numerals 71 to 74 in FIG. 11) for which distance detection is required as determined by the AI 51 are designated m, n, o, and p, then, as shown in FIG. 12, first, the signals of the R and G pixels on horizontal line 1 are transferred to the horizontal shift register of the image sensor 38 at the timing indicated by reference numeral 81. Then, at the timing indicated by reference numeral 82, the accumulated charges are reset, and horizontal readout of the R and G pixels and exposure for a frame period 83 begins.

[0168] Next, horizontal lines m, n, o, and p are selected in sequence. At timings 84 to 87, the B pixel and IR pixel signals are sequentially shifted to the horizontal shift register, while being sequentially reset at timings 88 to 91. Horizontal readout of the B pixel and IR pixel signals and exposure at sampling periods 92 to 95 are then started in sequence.

[0169] Next, moving to horizontal line 2, the B and IR pixels are shifted to the horizontal shift register at timing 96 and reset at timing 97, and horizontal readout of the B and IR pixels and exposure for a frame period 98 begins.

[0170] Next, horizontal lines m, n, o, and p are selected in sequence again, and horizontal readout of the B and IR pixels and exposure for the sampling period begin. By repeating the above process for the number of horizontal lines, the RGB and IR images are detected and at the same time, the interference fringe signals of the IR pixels determined by AI51 are sampled.

[0171] At this time, the exposure times of the B pixels on horizontal lines m, n, o, and p are the sampling periods 92 to 95 of the interference fringe signal. Therefore, cumulative addition is performed within the frame period to obtain the B images on horizontal lines m, n, o, and p.

[0172] However, even with cumulative addition, there is a slight lack of exposure time due to the time it takes to transfer the signal to the horizontal shift register and reset, so the gain is adjusted to compensate. This allows for high-speed sampling of the interference fringe signals of the IR pixels corresponding to the four positions 71 to 74 where AI51 has determined that distance detection is necessary, while also obtaining the RGB and IR images. The information on the detected RGB and IR images is used in AI51's next determination.

[0173] Now, let's say the chirp modulation bandwidth is 1GHz. Chirp time is 15ms, When the maximum detection distance to object 35 is 150 m, The maximum frequency of the interference fringe signal is 67 kHz.

[0174] If the AI51 determines that there are four pixels, a horizontal scanning frequency of about 750 kHz, five times higher, is required. Converting this to a frame rate of about 600 frames per second for HD image sensors, this is a level that can be easily achieved with the high-speed imaging technology of existing HD image sensors.

[0175] Thus, in a preferred aspect of this embodiment, a determination mechanism that detects the position of interference light that interferes with the detection of the distance to the object from the image captured by the two-dimensional detection mechanism and determines the local position where the distance to the object is detected; It is desirable to detect the distance to the local location of an object while avoiding interfering light. Furthermore, according to a preferred aspect of this embodiment, a determination mechanism that determines the position of interference light that interferes with reception of optical wireless communication from an image captured by the two-dimensional detection mechanism, and determines the position of a transmitter of an optical wireless communication device that receives the optical wireless communication, It is desirable to receive optical wireless communications while avoiding interfering light.

[0176] A method for receiving optical wireless communications using the image sensor 38 is described. Fig. 13 shows the front configuration of the filters. When the image sensor 38 also has a receiving function for optical wireless communication, RGB color filters for color image detection, an IR1 filter for distance detection, and an IR2 filter for optical wireless communication are arranged in a mosaic pattern on the pixels of the image sensor 38, as shown in Fig. 13.

[0177] The readout timing for these pixels is the same as that of the timing chart in Figure 12, with the addition of the readout and reset timing for the IR2 pixel for optical wireless communication. If there are four IR1 pixels and four IR2 pixels determined by AI51, the horizontal scanning repetition frequency will be 1200 kHz, nine times that of the timing chart in Figure 12. Converted to an image sensor frame rate, this is 1150 frames per second, a level that can be achieved with existing HD image sensor high-speed imaging technology. In this case, the communication capacity of the optical wireless communication will be 67 kbps.

[0178] Alternatively, a multi-plate prism may be used, and a dichroic mirror may be used to separate the wavelength bands for RGB images, the IR1 wavelength band for distance measurement, and the IR2 wavelength band for optical wireless communication, and dedicated imaging elements suitable for each detection may be provided.

[0179] We have described above a method that enables random selection of horizontal lines without changing the basic configuration of the light-receiving section of a CMOS image sensor, thereby enabling high-speed sampling of interference fringe signals of IR pixels determined by AI51 and reception of optical wireless communication at the same time as detecting RGB images and IR images.

[0180] By changing the semiconductor configuration and readout mechanism of the CMOS image sensor's pixels, it is possible to sample the IR pixel interference fringe signal at even higher speeds and receive optical wireless communication.

[0181] For example, first, the horizontal and vertical scanning circuits of the CMOS image sensor are changed to a circuit configuration (decoder circuit, etc.) that allows random pixel selection. In addition, a transistor that resets the vertical line is connected in series (cascode connection) to the transistor in each pixel that resets the charge of the pixels in the horizontal line. This allows for a configuration in which resetting each pixel is performed by an XY matrix switch.

[0182] Using the same procedure as described above, it is possible to read out the light-receiving pixels of the image sensor 38 and the IR pixels determined by the AI 51, repeating this process pixel by pixel. In parallel with the detection of RGB and IR images, the sampling rate of the IR pixel interference fringe signals and the receiving capacity of the optical wireless communication can be increased by about three orders of magnitude. However, this requires an additional transistor for resetting the vertical line per pixel, and an additional signal wiring for driving it per vertical line.

[0183] Furthermore, if a semiconductor configuration such as an InGaAs PIN semiconductor is used for the light receiving section or a single photon avalanche diode (SPAD) is used for the IR light receiving pixel, high frequency interference fringe signals can be detected with high sensitivity, and high bit rate optical wireless communication can be received with high sensitivity.

[0184] 14 and 15 each show a schematic configuration of a distance measuring device according to a modification of the second embodiment.

[0185] The light from the illumination light source 102 is turned on and off by a linear chirp signal 119A of, for example, 0.1 to 1.1 GHz, generated by a synthesizer oscillator 101. After passing through a circulator 103, the light is narrowed down into a beam by an optical system 104.

[0186] An object 107 is two-dimensionally scanned with illumination light 105 by a two-dimensional scanning mechanism 106. The illumination light source 102 is a light-emitting diode capable of high-speed switching, such as an LD or SLD with a central wavelength of 1550 nm.

[0187] Reflected light 108 from object 107 passes through optical system 104 and circulator 103, and is then separated into wavelength bands IR1, IR2, R, G, and B by splitter 109. The light is then input to light receiving elements 110, 111, 112, 113, and 114 via optical switches 115 and 116, and converted into electrical signals.

[0188] Symbol IR1 denotes infrared light with a center wavelength of 1550 nm generated by illumination light source 102. Symbol IR2 denotes light for external infrared communication. Here, circulator 103, demultiplexer 109, optical switch 115, optical switch 116, and light receiving elements 110, 111, 112, 113, and 114 are optical devices connected by optical fiber. Similar optical devices are often used in optical fiber communication.

[0189] The AI 120 shown in Fig. 15 determines the position (71-74 in Fig. 11) where distance detection is required while avoiding interference. Then, according to the instruction 121, the optical switch 115 switches the connection to the light receiving element 117 when the scanning of the beam of the illumination light 105 overlaps that position (71-74 in Fig. 11).

[0190] Any position other than those (71-74 in Figure 11) is connected to the photodetector 110 to detect the image signal of code IRI. The photodetector 117 is made of an InGaAs PIN type semiconductor or the like, has a current-voltage conversion type amplifier, and is capable of high-speed direct detection with a high S / N ratio.

[0191] The chirp signal of the reflected light 108 is directly detected and converted into an electrical signal, and the detected chirp signal is multiplied by a chirp signal 119A as a reference signal by a mixer 118 to detect an interference fringe signal.

[0192] 6, the interference fringe signal is processed to detect the distance to the object 107. If the distances of all pixels are detected, a distance image can be obtained.

[0193] Alternatively, instead of the light receiving element 117 and mixer 118, a light receiving element that can receive single photons (Geiger mode) by the avalanche effect and can perform heterodyne detection by high-speed direct switching, such as a single-photon avalanche diode (SPAD), can be used. Heterodyne detection can be performed directly by switching with the chirp signal 119A, and interference fringe signals can be detected.

[0194] Similarly, the AI 120 determines the position (reference numerals 77 and 78 in FIG. 11) for receiving the optical wireless communication while avoiding interfering light (not shown) such as strong reflection of sunlight. In accordance with the instruction 122, when the scanning of the beam of the illumination light 105 overlaps with the position (reference numerals 77 and 78 in FIG. 11), the optical switch 116 switches to the signal line of the optical fiber connected to the input terminal of the optical wireless receiver (not shown).

[0195] Reference numerals 77 and 78 in Figure 11 indicate the positions of optical wireless transmitters installed on other vehicles. Alternatively, they may be optical wireless transmitters installed on side roads that provide position information. In this case, the current position can be detected with centimeter-level accuracy instead of GPS from the position information provided by such transmitters and the relative positions of the transmitters and vehicles detected by the distance detection device of the present invention, and can be electronically compared with a map.

[0196] The positions other than those (reference numerals 77 and 78 in FIG. 11) are connected to the light receiving element 111, and the image signal of the IR2 pixel is detected.

[0197] The separated R, G, and B light are also converted into electrical signals as a color image by light receiving elements 112 to 114. Then, the IR1, IR2, R, G, and B signals are input to AI 120 via buffer memory 119 as a 5-channel / pixel image.

[0198] The next timing for object detection, the position for avoiding interfering light, and the position for distance detection and reception of optical wireless communication are determined.

[0199] As described above, variant example 21B of the second embodiment in Figure 14 is an example in which a distance image is detected using a single light receiving element 117 by scanning an object two-dimensionally with illumination light 105 narrowed into a beam shape.

[0200] Furthermore, as a new modified example of the second embodiment, a semiconductor that is highly sensitive and capable of high-speed switching, such as an InGaAs PIN-type semiconductor or a SPAD, may be used for the light receiving portion of the pixel of the image sensor 38 of modified example 11A of the first embodiment shown in FIG.

[0201] A distance image can be obtained by directly performing heterodyne detection for each light-receiving pixel, detecting interference fringe signals, and performing switching using the chirp-modulated reference signal 32 shown in Fig. 5 while receiving the chirp signal of the reflected light 36 with high sensitivity. In this case, a high-coherence light source can be used for the illumination light source 33 in Fig. 5, making the reference light source 39 and multiplexer 41 in Fig. 5 unnecessary.

[0202] We will now discuss AI51 and AI120 shown in Figures 6 and 15. AI51 and AI120, which perform object detection, use a convolutional neural network (CNN). CNNs excel at pattern recognition such as image recognition, and are suited to object detection based on pattern recognition. CNN learning is primarily performed by supervised learning.

[0203] In specific areas where there are few decision-making situations, it is easier to collect learning data, and there are increasing cases where AI outperforms humans in competitions due to the difference in the amount of learning. In general areas where there are many decision-making situations, AI cannot possibly compete with humans.

[0204] When AI divides the feature space and makes judgments, if the distances between the feature vectors of the images used for training are uneven and biased, over-training or under-training will occur, resulting in a drop in judgment accuracy. For this reason, know-how for setting up training data is important.

[0205] By randomly increasing the amount of data, bias can be reduced probabilistically, and object detection accuracy can be improved in proportion to the logarithm of the amount of training data. However, this requires setting up a huge amount of data. Several proposals have been made to make the setup of training data more efficient based on feature space, but there is currently no definitive solution.

[0206] AI never tires or forgets, and can complete huge amounts of learning at the speed of light, day or night. Given the current situation, where AI is increasingly superior to humans in terms of the amount of learning it can do, the world is turning toward acquiring infrastructure that can set up huge amounts of learning data.

[0207] As methods for collecting training data evolve in the future, the way we think about and set up the system will change, and there will undoubtedly be a sharp increase in cases where AI outperforms humans in certain areas.

[0208] The basic configuration of a CNN processing circuit is a feedforward processing circuit consisting of an input layer, a feature extraction layer, a decision layer, and an output layer.

[0209] The input layer has input terminals whose number is the number of information channels per pixel multiplied by the number of pixels. For RGB images, the number of information channels is 3 channels / pixel, and for images that combine RGB images, range images, and velocity images, the number is 5 channels / pixel.

[0210] The feature extraction layer is composed of a set of convolutional layers, activation functions, and pooling layers stacked in multiple stages. When the number of information channels and tasks is large, increasing the number of feature extraction layers can ensure object detection accuracy.

[0211] The convolution layer is a multidimensional filter whose dimension is the number of information channels, and performs convolution operations on kernels (weighting coefficients of the neural network) acquired through learning, and then extracts a feature map.

[0212] In shallower layers where the correlation between neighboring pixels is high, a feature map close to principal component analysis is extracted, while the deeper the layer, the closer the feature map to independent component analysis is extracted. The kernel size is often 3x3x3 for three-channel images such as RGB images, in order to strike a balance between the number of layers and the number of operations.

[0213] If the number of input channels to the convolution layer is n, then the kernel size is 3. nIf the number of kernel types acquired through learning is m, then 3 n The number of input channels n of the next convolutional layer is the number m of kernel types in the previous stage.

[0214] The activation function has a simple structure in which the output of m types of convolution operations is multiplied by weighting coefficients obtained through learning, a bias is added, and a nonlinear function transformation is performed. By performing processing similar to fuzzy logic operations, it plays an important role in connecting to higher-level feature extraction in the next layer and beyond. There are several types of activation functions, and in object detection, rectified linear functions are often used to balance the scale of operations.

[0215] The pooling layer performs a process similar to data compression, which subsamples a certain range of the output (feature map) of the convolution layer, for example, a 2x2 range, into one. The convergence is achieved by selecting the maximum value (max pooling) that is the first principal pole of independent component analysis (similar to principal component analysis in shallow layers) within the 2x2 range of signals.

[0216] The decision layer and output layer are configured according to the task. The decision layer for category classification is configured by stacking one or two layers of fully connected layers and sets of activation functions depending on the application. If the category has multiple classes, the output layer is provided with the same number of output terminals as the number of classes. The activation function placed between the decision layer and the output layer uses a softmax function to express the decision result as a probability.

[0217] In object detection for autonomous driving, a decision layer and output layer for detecting the size and position of an object (bounding box) and classifying the object's region (semantic segmentation) are provided in parallel to the decision layer and output layer for object category classification.

[0218] If the input layer and feature extraction layer are encoding functions that extract feature vectors, the decision layer and output layer are decoding functions that generate output according to the purpose from the feature vectors, and can be flexibly configured for each task. For example, in the configuration of a U-Net to detect object regions, the output of each stage of the feature extraction layer is extracted using a shortcut, and unpooling and deconvolution operations are performed to decode the object region.

[0219] In AI51 shown in Figure 6 and AI120 shown in Figure 15, in addition to the judgment layer and output layer for object detection necessary for autonomous driving, judgment layers and output layers for determining the local position of an object for which distance measurement is required and the position of the transmitter of the other party receiving the optical wireless communication are provided in parallel.

[0220] Then, AI51 and AI120 are trained on a 5-channel / pixel image obtained by merging the IR1 image and the IR2 image into an RGB image as training data, and the 5-channel / pixel image detected by the distance detection device of this embodiment is input to AI51 and AI120, whereby AI51 and AI120 determine the IR1 pixels and IR2 pixels that need to be detected and transmit the addresses of those pixels to the image sensor 38 and the optical switches 115 and 116 in FIG. 14.

[0221] We will discuss the frequency and initial phase of the interference fringe signal, the distance to the object, the distance resolution, and the distance measurement accuracy.

[0222] where: f0 is the starting frequency of the linear chirp modulation. fw is the initial phase φ0, the bandwidth of the chirp modulation, T is the chirp time. a=fw / T is the rate of change of the frequency of the chirp modulation, If t is time, the instantaneous frequency f(t) of the chirp signal is f(t)= at + f0 This becomes:

[0223] If the phase of the chirp signal modulating the reference light is φ(t) and the initial phase is φ0, then φ(t) can be obtained by integrating the above equation with respect to t: φ(t)=2πf(t) t =2π(1 / 2·a t+f0) t+φ0(4) This becomes:

[0224] If the time delay corresponding to the optical path difference between the reflected light and the reference light is d, the phase φ(t+d) of the chirp signal of the reflected light is φ(t+d)=2π[1 / 2·a (t+d)+f0](t+d)+φ0(5) This can be expressed as:

[0225] Substituting equations (4) and (5) into equation (3), we get I=K Ps PL cos [2π(a·d)t +πa·d 2 +2πf0·d] (6) This becomes:

[0226] If the speed of light is C and the distance to the object is L, then L=C·d / 2, so From equation (6), the frequency fi of the interference fringe signal is fi=a·d =(fw / T) d=2 fw L / CT (7) This becomes:

[0227] It can be seen that the frequency fi of the interference fringe signal increases in proportion to the distance L to the object (delay time d) and the chirp modulation bandwidth fw, and decreases in proportion to the chirp time T.

[0228] The initial phase φ0 of the interference fringe signal is φ0=πa d 2 +2πf0·d=π / a·fi 2 +2πf0 / a·fi This becomes: The initial phase of the interference fringes varies parabolically in proportion to the frequency fi of the interference fringes (object distance).

[0229] In addition, the distance L to the object is L=C·d / 2. Therefore, from equation (7), L=C·d / 2=C·fi / 2a=C·T fi / 2fw (8) This becomes: The distance L can be calculated using the frequency fi of the interference fringes, the chirp modulation bandwidth fw, and the chirp time T.

[0230] The distance resolution ρ is the ability to resolve two objects aligned in the distance direction. If the envelope of the chirp signal is a square wave, the envelope of the interference fringe signal will also be a square wave, and when such an interference fringe signal is converted into frequency components, the waveform of the envelope will be a sinc function.

[0231] The full width at half maximum (FWHM) of the sinc function, 1 / T, is the spectral resolution on the frequency axis. The spectral resolution, 1 / T, is the change in fi when light travels a distance corresponding to the distance resolution, ρ, back and forth.

[0232] Therefore, if fi and L in equation (7) are replaced with 1 / T and ρ, and rearranged, the distance resolution ρ becomes ρ=C / 2fw (9) This becomes:

[0233] The waveform of the envelope is a sinc function with a full width at half maximum (FWHM) of 1 / T = 1 / 2fw. This shows that the distance resolution ρ is uniquely determined by the bandwidth (fw) of the chirp modulation.

[0234] Distance detection accuracy is the accuracy in detecting the distance L to an object, and is the width by which the peak position of the distance resolution waveform fluctuates due to noise. Therefore, distance detection accuracy increases in proportion to both the distance resolution and the S / N ratio.

[0235] Furthermore, the higher the S / N ratio, the smaller the distance detection accuracy becomes. As mentioned above, when the correlator 48 performs a correlation calculation with the reference signal 47 and detects the peak value, the waveform of the distance resolution (sinc function) becomes close to a Mexican hat function.

[0236] For example, if the S / N ratio is 40 dB, the distance detection accuracy value will be about 1 / 40 to 1 / 50 of the distance resolution. Increasing the chirp time T narrows the spectral width 1 / T of the interference fringe signal, and the S / N ratio increases in proportion to the square root of the ratio of the spectral width 1 / T to the chirp modulation bandwidth fw.

[0237] Increasing the chirp modulation bandwidth fw increases the distance resolution ρ, and so the distance detection accuracy increases in proportion to both the chirp time T and the chirp modulation bandwidth fw. This is why the chirp modulation method is superior to other methods in terms of sensitivity and distance detection accuracy. [Industrial Applicability]

[0238] As described above, the present invention is suitable for a distance detection device that has high distance detection accuracy and azimuth resolution regardless of the distance to an object, is resistant to interference and disturbance between distance detection devices, is small and low-cost, and is integrated with an imaging device. [Explanation of symbols]

[0239] 10A, 20A Distance detection device; 10 Illumination light; 11 Object; 12 Reflected light; 13 Optical system; 14 Light receiving element; 15 Reference light; 16 Combiner; 17 Carrier light; 18 Interference fringe signal; 19, 20 Coherence reduction mechanism; 21 Amplitude modulator; 22 Carrier light; 23 Modulator; 24 Chirp modulation signal; 25 Object; 26 Reflected light; 27 Optical system; 28 Light receiving element; 29 Combiner; 30 Interference fringe signal; 31 Synthesizer oscillator; 32 Linear chirp modulation signal; 33 Illumination light source; 34 Illumination light; 35 Object; 36 Reflected light; 37 Imaging optical system; 38 Imaging element; 39 Reference light source; 40 Reference light; 41 Combiner; 42 Optical system; 43 Interference fringe signal; 44 Frequency converter; 45 Frequency components; 46 Reference signal generator; 47 Reference signal (reference signal); 48 Correlator; 49 Distance converter; 51 AI; 52 Reference symbol; 53 Fundamental wave; 54 First harmonic; 70A, 70B Object (automobile); 71, 72, 73, 74 Local area; 75, 76 Interference light; 101 Synthesizer oscillator; 102 Illumination light source; 103 Circulator; 104 Optical system; 105 Illumination light; 106 Two-dimensional scanning mechanism; 107 Object; 108 Reflected light; 109 Demultiplexer; 110-114, 117 Light receiving element; 115, 116 Optical switch; 118 Mixer (mixer); 119 Buffer memory; 119A Linear chirp signal

Claims

1. a light source that generates illumination light in which chirp modulation is applied to amplitude modulation of carrier light; another light source that generates reference light by subjecting the amplitude modulation of carrier light to the same chirp modulation as that of the illumination light; the chirp modulation is modulated by a linear chirp modulation signal; a coherence reduction mechanism that reduces the temporal coherence and the spatial coherence of both or one of the carrier lights while maintaining the temporal coherence and the spatial coherence of the linear chirp modulated signal; an imaging optical system that forms an image of reflected light from an object illuminated with the illumination light on a light receiving element; a combiner that combines the reflected light and the reference light, The distance detection device is characterized in that the light receiving element receives the combined reflected light and the reference light, and detects an interference fringe signal generated by heterodyning between the linear chirp modulation signals of the illumination light and the reference light.

2. a light source that generates illumination light in which chirp modulation is applied to amplitude modulation of carrier light; an imaging optical system that forms an image of reflected light from an object illuminated with the illumination light on a light receiving element, death, The light receiving element further includes a mixer that multiplies the chirp-modulated electrical signal by the same chirp-modulated signal as the illumination light to detect an interference fringe signal, or the light receiving element also functions as the mixer by direct switching using the same chirp-modulated signal as the illumination light, The light receiving element detects the reflected light and converts chirp modulation of the reflected light into an electrical signal.

3. The two-dimensionally arranged light receiving elements Alternatively, the one-dimensionally arranged light receiving elements are scanned one-dimensionally across the imaging surface of the imaging optical system in a direction intersecting the one-dimensional arrangement direction, Alternatively, by two-dimensionally scanning the image plane with a single light receiving element, Alternatively, the light receiving elements are arranged one-dimensionally on the imaging plane, and the directivity of the illumination light in a direction intersecting the one-dimensional arrangement is increased, thereby scanning the object one-dimensionally in the direction. Alternatively, a single light receiving element is placed on the imaging plane, and the illumination light is increased in directivity to a beam to two-dimensionally scan the object.

3. The distance detection device according to claim 1, further comprising a two-dimensional detection mechanism for detecting the interference fringe signals at two-dimensional positions while simultaneously capturing an image of the object.

4. a determination mechanism that detects a position of interference light that interferes with detection of the distance to the object from the image captured by the two-dimensional detection mechanism and determines a local position where the distance to the object is detected, 4. The distance detection device according to claim 3, wherein the distance to the local position of the object is detected while avoiding the interfering light.

5. a determination mechanism that determines a position of interfering light that interferes with reception of optical wireless communication from an image captured by the two-dimensional detection mechanism, and determines a position of a transmitter of an optical wireless communication device that receives the optical wireless communication, 4. The distance detection device according to claim 3, wherein the optical wireless communication is received while avoiding the interfering light.

Images