Light projecting and receiving device, distance measuring device, distance measuring system, and program

Abstract

To enable flexible photographing according to a scene that is a distance measurement target.SOLUTION: A light projecting and receiving device comprises: a light projecting unit that projects first light and second light; a light receiving unit that receives first reflected light that is the first light reflected on an object, and second reflected light that is the second light reflected on the object; and a control unit that controls the light projecting unit and the light receiving unit. Each of the first light and the second light is pattern light having a plurality of distributions of light intensities having peak regions, and at least part of a region on which light is projected includes a portion where the peak region of one pattern light and the peak region of the other pattern light do not overlap each other. The control unit projects and receives light in a mode selected from a plurality of light projecting modes including a first mode for projecting and receiving the first light and the second light at different timings, and a second mode for projecting and receiving the first light and the second light at the same time.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a light transmitting and receiving device, a distance measuring device, a distance measuring system, and a program.

Background Art

[0002] Conventionally, as one of the methods for measuring the distance to an object, a method called the ToF (Time of Flight) method is known. A ToF camera, which is a distance measuring device using the ToF method, irradiates an object with distance measuring light that is infrared light intensity-modulated by a predetermined irradiation pattern, and then receives the distance measuring light reflected by the object with an image sensor for infrared rays. Then, the ToF camera detects the time difference from irradiation to reception for each pixel according to the irradiation pattern and calculates the distance. The ToF camera collects the calculated distance values in a bitmap form for each pixel and stores them as a "distance image".

[0003] Patent Document 1 discloses an optical distance measurement system including at least one light source and a diffuser, a light projecting illuminator configured to project a first illumination pattern, a dot pattern projector configured to project a second illumination pattern, and an image capturing device configured to capture an image of the illumination pattern reflected from an object.

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, according to the conventional optical distance measurement system, there is a problem that flexible imaging according to the scene of the distance measurement target cannot be performed.

[0005] The present invention has been made in view of the above, and an object thereof is to enable flexible imaging according to the scene of the distance measurement target.

Means for Solving the Problems

[0006] A light projecting unit that projects first light and second light, a light receiving unit that receives first reflected light obtained by reflecting the first light from an object and second reflected light obtained by reflecting the second light from the object, and a control unit that controls the light projecting unit and the light receiving unit, wherein the first light and the second light are both pattern lights each having a plurality of light intensity distributions with peak regions, and at least a part of the region where the lights are projected includes a portion where the peak region of one pattern light does not overlap with the peak region of the other pattern light, and the control unit performs light projection and reception in a mode selected from a plurality of light projection modes including a first mode in which the first light and the second light are projected at different timings and received respectively, and a second mode in which the first light and the second light are projected and received simultaneously.

Advantages of the Invention

[0007] According to the present invention, there is an effect that flexible imaging according to the scene of the distance measurement target can be enabled.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Figure 28

Figure 29

Figure 30

Figure 31

Figure 32

Figure 33

Figure 34

Figure 35

[0009] Hereinafter, embodiments of the light projecting and receiving device, the distance measuring device, the distance measuring system, and the program will be described in detail with reference to the accompanying drawings.

[0010] (First Embodiment) FIG. 1 is a block diagram showing the hardware configuration of a distance measuring device 100 including a light projecting and receiving device according to the first embodiment. The distance measuring device 100 that also functions as an imaging device measures the distance from the distance measuring device 100 to an object. The distance measuring device 100 is a ToF (Time of Flight) camera that calculates the distance to the object based on the time from when light is irradiated until the reflected light is received.

[0011] As shown in FIG. 1, the distance measuring device 100 includes two light projecting units 1-1 and 1-2, a light receiving unit 2, an analog-to-digital converter (ADC) 3, and a distance measuring control unit 4 which is a control unit. In this embodiment, the two light projecting units 1-1 and 1-2 constitute the light projecting unit, and the light receiving unit 2 constitutes the light receiving unit.

[0012] As the light projecting unit 1-1 which is the first light projecting unit, for example, a light source 1-1a such as a VCSEL (Vertical Cavity Surface Emitting Laser) and a light projecting optical system 1-1b such as a lens, a DOE (Diffractive Optical Element), a collimator, and an MLA (Micro Lens Array) can be used. The light source 1-1a is an example of the first light source, and the light projecting optical system 1-1b is an example of the first light projecting optical system. With such a configuration, the light projecting unit 1-1 projects a plurality of lights in a dot shape. That is, the light composed of a plurality of dot-shaped lights (spot lights) projected by the light projecting unit 1-1 is an example of pattern light, and is light patterned in a dot shape so as to have an intensity distribution in the projected range (space).

[0013] The light projecting unit 1-2 which is the second light projecting unit, similar to the light projecting unit 1-1, projects a plurality of lights in a dot shape by using a configuration including a light source 1-2a and a light projecting optical system 1-2b. That is, the light projected by the light projecting unit 1-2 is also pattern light. The light source 1-2a is an example of the second light source, and the light projecting optical system 1-2b is an example of the second light projecting optical system.

[0014] Here, the dot-shaped light will be described. The dot-shaped light does not necessarily have a contrast ratio of "100:0" between the point of the reflected light of the dot-shaped light received by the light receiving unit 2 and the other regions due to a certain degree of light spread or a small amount of multipath interference in reality. Taking one of the general methods for determining the beam diameter of light as an example, the dot-shaped light is defined below.

[0015] FIG. 2 is a diagram exemplarily showing pattern light composed of a plurality of dot-like lights. FIG. 2(a) is a diagram showing the light projection state of the pattern light, and FIG. 2(b) is a diagram showing the luminance values of a part of the pattern light shown in FIG. 2(a). Today, there is an idea that the range where the luminance value becomes 1 / e 2 times that of the peak luminance value is defined as the beam diameter of the light. In order to distinguish between the beam region and the other region, as shown by the luminance values of a part of the cross-section of two dot-like lights shown by the straight line a in FIG. 2(a) (shown in FIG. 2(b)), generally, the luminance value of the valley is 1 / e 2 times, that is, the light is projected so that it is about 13.5% or less. Generally, if the luminance value of the valley is 1 / e 2 times, that is, about 13.5% or less with respect to the luminance value of the peak, the dot-like light and the other region can be clearly distinguished.

[0016] The above is the definition of dot-like light taking the beam diameter determination method as an example, but the numbers themselves have no inventive meaning. It is only necessary to obtain a contrast difference such that the point and the other region can be distinguished by some method. The wavelengths of the light emitted from the light sources 1-1a and 1-2a are, for example, 850 nm or 940 nm. The wavelength of the light source 1-1a and the wavelength of the light source 1-2a may be the same or different.

[0017] In addition, in this embodiment, the light projecting units 1-1 and 1-2 that project pattern light having a regular dot arrangement of a plurality of dot-like lights have been taken as an example, but the present invention is not limited to this. The light projecting units 1-1 and 1-2 that irradiate pattern light having an arbitrary pattern such as a random dot pattern or a stripe pattern with an irregular dot arrangement may also be used. That is, the pattern light used in this embodiment has a plurality of distributions of light intensity having peak regions, and the arrangement of each distribution may be regular or irregular. Further, the shape of the peak region is not limited to a circle or an ellipse, and may be a stripe shape or other shapes. In this embodiment, when the light projecting units 1-1 and 1-2 that irradiate such pattern light project two pattern lights at the same time, they are set so that the peak regions of each other do not overlap at least in part of the projected region.

[0018] As the light receiving unit 2, an image sensor 2a and a light receiving optical system 2b such as a lens can be used. The image sensor 2a is a so-called ToF sensor. The image sensor 2a receives the light irradiated from the light source 1-1a or the light source 1-2a to the object and reflected by the object or the like. The light receiving unit 2 receives the reflected light of the pattern light irradiated from the light projecting units 1-1 and 1-2 to the object. Although it will be described in detail later, the image sensor 2a acquires, for each pixel, an electrical signal corresponding to the intensity of the received reflected light and divides it into a plurality of phase signals.

[0019] The ADC 3 converts the phase signal acquired for each pixel from an analog signal into digital data and supplies it to the distance measurement control unit 4.

[0020] The distance measurement control unit 4 includes a sensor I / F (Interface) 41, light source drive circuits 42-1 and 42-2, an input / output interface (input / output I / F) 43, a CPU (Central Processing Unit) 44, a ROM (Read Only Memory) 45, a RAM (Random Access Memory) 46, and an SSD (Solid State Drive) 47. The sensor I / F 41, the light source drive circuits 42-1 and 42-2, the input / output I / F 43, the CPU 44, the ROM 45, the RAM 46, and the SSD 47 are electrically connected to each other via a system bus 48.

[0021] The sensor I / F 41 is an interface for acquiring the phase signal from the image sensor 2a.

[0022] The input / output I / F 43 is an interface for connecting to an external device such as a main controller device or a personal computer (PC) device.

[0023] The light source driving circuit 42-1 supplies a driving signal to the light projecting unit 1-1 based on a control signal supplied from the CPU 44, at a predetermined voltage waveform and a predetermined light emission frequency, to time-modulate (temporally control) the light emission by the light projecting unit 1-1. The light source driving circuit 42-2 supplies a driving signal to the light source 1-2a based on a control signal supplied from the CPU 44, at a predetermined voltage waveform and a predetermined light emission frequency, to time-modulate (temporally control) the light emission by the light source 1-2a. Note that, as the driving signal supplied to the light sources 1-1a and 1-2a, a rectangular wave, a sine wave, or a voltage waveform of a predetermined waveform shape can be used. The light source driving circuits 42-1 and 42-2 change the frequency of the voltage waveform to modulation-control the frequency of the driving signal. Further, the light source driving circuits 42-1 and 42-2 can also simultaneously control the light emission of some of the plurality of light emitting units of the light sources 1-1a and 1-2a, or change the light emitting units to be made to emit light.

[0024] The ROM 45 is a non-volatile semiconductor memory (storage device) capable of retaining programs or data even when the power is turned off. In the ROM 45, programs or data such as the BIOS (Basic Input / Output System) executed at the startup of the CPU 44 and OS (Operating System) settings are stored. The RAM 46 is a volatile semiconductor memory (storage device) that temporarily retains programs or data.

[0025] The SSD 47 is a non-volatile memory in which programs for executing processing by the distance measurement control unit 4 or various data are stored. As an example, a distance measurement imaging program is stored in the SSD 47. Specifically, as will be described later, the CPU 44 controls the image sensor 2a so as to acquire, for each pixel, an electrical signal corresponding to the intensity of the received reflected light, divided into a plurality of phase signals, by executing this distance measurement imaging program. Note that, instead of the SSD 47, other storage devices such as an HDD (Hard Disk Drive) may be used.

[0026] The CPU 44 reads a program or data from a storage device such as the ROM 45 or the SSD 47 onto the RAM 46 and executes processing to control the entire distance measurement control unit 4. Note that part or all of the functions of the CPU 44 may be realized by an electronic circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field-Programmable Gate Array).

[0027] Here, the distance measurement principle using a general ToF camera will be described.

[0028] (Operation of obtaining phase signals) The image sensor 2a has, for example, two charge storage units (a first charge storage unit and a second charge storage unit) for one light receiving element, and can rapidly switch the charge storage unit that stores charge. Therefore, two phase signals that are exactly opposite can be simultaneously detected for one rectangular wave. As an example, a phase signal of 0 degrees and a phase signal of 180 degrees can be simultaneously detected. Also, a phase signal of 90 degrees and a phase signal of 270 degrees can be simultaneously detected. This means that distance measurement is possible through two light projection and reception processes.

[0029] Figure 3 is a timing chart for explaining the distance measurement principle. Among them, Figure 3(a) shows the timing of light projection, and Figure 3(b) shows the timing of the reflected light obtained by the light projection. Also, Figure 3(c) shows the timing at which a phase signal with a phase of 0 degrees is accumulated in the first charge storage unit among the two charge storage units provided in the image sensor 2a, and Figure 3(d) shows the timing at which a phase signal with a phase of 180 degrees is accumulated in the second charge storage unit. Also, Figure 3(e) shows the timing at which a phase signal with a phase of 90 degrees is accumulated in the first charge storage unit among the two charge storage units provided in the image sensor 2a, and Figure 3(f) shows the timing at which a phase signal with a phase of 270 degrees is accumulated in the second charge storage unit.

[0030] While the charges of the phase signals of each phase are being accumulated in the first charge storage section or the second charge storage section, as indicated by the hatching in FIGS. 3(c) to 3(f). Specifically, as shown in FIG. 3(c), as the charge of the phase signal of 0 degrees, the charge between the pulse edge of the end of light projection and the pulse edge of the start of reception of the reflected light is accumulated in the first charge storage section. As the charge of the phase signal of 180 degrees, as shown in FIG. 3(d), the charge between the completion of the charge accumulation of the phase signal of 0 degrees and the pulse edge of the end of reception of the reflected light is accumulated in the second charge storage section.

[0031] Similarly, as the charge of the phase signal of 90 degrees, as shown in FIG. 3(e), the charge between the pulse edge of the start of reception of the reflected light and the pulse edge of the end of charge accumulation of the pulse for performing charge accumulation control is accumulated in the first charge storage section. As the charge of the phase signal of 270 degrees, as shown in FIG. 3(f), the charge between the completion of the charge accumulation of the phase signal of 90 degrees and the pulse edge of the end of reception of the reflected light is accumulated in the second charge storage section.

[0032] Note that, actually, in order to increase the amount of charge to be accumulated, the light projection is not a single rectangular wave but a repeating pattern of rectangular waves, and the switching control to the first and second charge storage sections according to the timing of projecting the light of this repeating pattern is also repeatedly performed.

[0033] (Calculation of distance value) The signals (N0, N90, N180, and N270) obtained at the four phases of 0 degrees, 90 degrees, 180 degrees, and 270 degrees are received by being temporally divided into four phases of 0 degrees, 90 degrees, 180 degrees, and 270 degrees with respect to the pulse period of the light (irradiation light) to be projected, and are phase signals based on the amount of accumulated charge. Therefore, the phase difference angle φ can be obtained using the following mathematical formula.

[0034] φ = Arctan{(N90 - N270) / (N0 - N180)}

[0035] Also, from this phase difference angle φ, the delay time Td can be obtained using the following mathematical formula. Note that fm is the modulation frequency.

[0036] Td = φ / (2πfm)

[0037] Also, from this delay time Td, the distance value D to the object can be obtained using the following mathematical formula.

[0038] D = Td × c ÷ 2 (c: speed of light)

[0039] The example in Figure 3 is an example of acquiring the phase signals at 0 degrees and 180 degrees in the first measurement. However, a phase signal obtained by subtracting the charge amount of the second charge accumulation unit from the charge amount of the first charge accumulation unit acquired in the first measurement may be generated. In such a measurement, one phase signal is acquired by one light emission and exposure. Therefore, to acquire the phase signals for four phases, four light emissions and exposures are required, and the imaging time becomes twice as long as that in the case of imaging without external light. However, there is an advantage that the influence of external light can be reduced by the subtraction process.

[0040] By the above method, a signal obtained by subtracting the charge amount of the second charge accumulation unit (tap B) from the charge amount of the first charge accumulation unit (tap A) acquired in the first measurement is called a DCS (Differential Correlation Sample) signal. From this DCS signal, it is converted into a distance image. The conversion method is the same as the method shown above, and the phase difference angle φ is calculated from the following formula.

[0041] φ = Arctan{(DCS90 - DCS270) / (DCS0 - DCS180)} φ = Arctan[{(A90 - B90) - (A270 - B270)} / {(A0 - B0) - (A180 - B180)}]

[0042] Here, for example, DCS90 is the DCS signal at 90 degrees, A90 is the charge amount of tap A at 90 degrees, and B90 is the charge amount of tap B at 90 degrees. From this phase difference angle φ, in the same way as the method shown above, the delay time Td and the distance value D to the object can be obtained.

[0043] In the following description, it is assumed that the phase signal obtained by the exposure of one emission and the reflected light is a phase signal (DCS signal) obtained by subtracting the charge amount of the second charge storage unit from the charge amount of the first charge storage unit.

[0044] Subsequently, the principle of noise generation due to multipath interference in a general ToF camera will be described.

[0045] FIG. 4 is a diagram showing the principle of noise generation due to multipath interference in a ToF camera. As shown in FIG. 4, the light in the τ0 optical path of the first reflection (direct reflection component) is received with the light in the multiple reflections (for example, τ1 optical path) (multipath component) mixed in. Since the distance information (distance measurement value) of the solid line is mixed with respect to the distance information (distance measurement value) of the dotted line shown in FIG. 4, a value far from the actual distance is calculated. In particular, it is likely to occur in a scene where light in a plurality of optical paths is reflected and received, such as a corner of a room. That is, although the diffused light can acquire the depth value with high spatial resolution, the error due to multipath interference becomes large.

[0046] Therefore, the distance measurement device 100 including the light transmitting and receiving device of the present embodiment is provided with an imaging mode for reducing the error due to multipath interference by performing distance measurement using the light projecting units 1-1 and 1-2 that project pattern light.

[0047] Subsequently, the function of the distance measurement control unit 4 will be described.

[0048] FIG. 5 is a functional block diagram showing the functional configuration of the distance measurement control unit 4. As shown in FIG. 5, the CPU 44 of the distance measurement control unit 4 realizes the respective functions of the imaging control unit 401, the image storage unit 402, the distance calculation unit 403, and the output unit 404 by executing the distance measurement imaging program stored in the SSD 47.

[0049] The imaging control unit 401 controls the image sensor 2a to capture phase images of multiple phases and accumulate the charges of each phase image in the charge accumulation unit for each phase image. The imaging control unit 401 controls the light emission of the light source 1-1a via the light source drive circuit 42-1. The imaging control unit 401 controls the light emission of the light source 1-2a via the light source drive circuit 42-2.

[0050] The image storage unit 402 stores or controls the reading of the phase signals (phase images) of each phase from the image sensor 2a output from the ADC 3 in a storage unit such as the RAM 46.

[0051] The distance calculation unit 403 calculates distance information indicating the distance to the object based on a plurality of phase images stored by the image storage unit 402.

[0052] The output unit 404 outputs the distance information indicating the distance to the object calculated by the distance calculation unit 403 to an external device via the input / output I / F 43.

[0053] Note that the functions of the imaging control unit 401, the image storage unit 402, the distance calculation unit 403, and the output unit 404 shown in FIG. 5 are each realized by software using a distance measurement imaging program. However, all or part of these may be realized by hardware such as an IC (Integrated Circuit).

[0054] Subsequently, the features of the distance measurement device 100 including the light transmitting and receiving device of the present embodiment will be described.

[0055] FIG. 6 is a diagram showing the light projecting system and the light receiving system of the distance measuring device 100. As shown in FIG. 6, the distance measuring device 100 includes light projecting units 1-1 and 1-2 that project pattern light, and a light receiving unit 2 that receives the pattern light projected by the light projecting units 1-1 and 1-2. That is, the distance measuring device 100 is a ToF camera having a light receiving unit 2 including ToF sensors corresponding to the light projecting units 1-1 and 1-2. The light projecting range of the light projecting units 1-1 and 1-2 is set to include the measurement target area in the distance measuring device 100, and the light receiving range of the light receiving unit 2 is set to receive light from the measurement target area. It is preferable that the light projecting ranges of the light projecting unit 1-1 and the light projecting unit 1-2 are set to substantially overlap.

[0056] FIG. 7 is a diagram showing an installation example of the distance measuring device 100. As shown in FIG. 7, the measurement target area of the distance measuring device 100 includes the corners of the room. Hereinafter, the details will be described with an example of photographing the corners of the room with the distance measuring device 100.

[0057] Here, the influence of multipath interference will be described. FIG. 8 is a diagram exemplarily showing the influence of multipath interference. When the distance measurement values (distance measurement points) of the corners of the room are shown in a bird's-eye view, it is as shown in FIG. 8. As shown in FIG. 8, the distance measurement values of the diffused light obtained by the conventional diffused illumination can, in principle, measure all pixels as distance measurement points, so they can be continuously acquired with high spatial resolution. However, the distance measurement values of the diffused light obtained by diffused illumination have large errors due to the influence of multipath interference. Specifically, as described above, the distance measurement values (distance measurement points) of the diffused light obtained by diffused illumination are calculated with the distance being farther than the original position. On the other hand, as shown in FIG. 8, the distance measurement values of the spot light obtained by the light projecting units 1-1 and 1-2 that project pattern light are discontinuous because there are few distance measurement points, but they are relatively accurate.

[0058] Here, the distance image will be described. FIG. 9 is an example of a distance image obtained by emitting pattern light. As shown in FIG. 9, since the spot light is focused, the distance can be calculated only at the position where the light hits. That is, as shown in FIG. 9, in the case of pattern light, a distance image is obtained only at the point portions of the spot light.

[0059] Figure 10 is a cross-sectional view of the periphery of the light projecting system of the light transmitting and receiving device. As shown in Figure 10, the light projecting unit 1-1 including the light source 1-1a, the light projecting optical system 1-1b and the lens barrel member, and the light projecting unit 1-2 including the light source 1-2a, the light projecting optical system 1-2b and the lens barrel member are fastened and fixed to the structural member with screws. The structural member has a shape that gives a predetermined distance and relative angle between the light projecting unit 1-1 and the light projecting unit 1-2. Thereby, the light projecting unit 1-1 and the light projecting unit 1-2 irradiate the pattern light at positions where the spot lights thereof do not overlap with respect to substantially the same irradiation range.

[0060] Figure 11 is a diagram showing the relationship of the pattern light irradiated from the light projecting units 1-1 and 1-2. Here, the solid line indicates the pattern light irradiated from the light projecting unit 1-1, and the broken line indicates the pattern light irradiated from the light projecting unit 1-2. In this way, the light projecting units 1-1 and 1-2 are set to be irradiated at positions where the spot lights thereof do not overlap. Note that the light projecting units 1-1 and 1-2 may be set so that their spot lights do not overlap at least in part of the irradiation region. For example, in an irradiation region where the importance is low in performing distance measurement, a region where the spot lights of the two pattern lights overlap may occur.

[0061] In the example of Figure 10, the light projecting unit is used as two light sources 1-1a and 1-2a and two light projecting optical systems 1-1b and 1-2b, but the combination of the light source and the light projecting optical system is not limited to this. For example, the light projecting units 1-1 and 1-2 may be irradiated at positions where their spot lights do not overlap by the following combinations.

[0062] The light projecting unit that projects two pattern lights may be realized by a combination of two light sources 1-1a and 1-2a and one light projecting optical system 1b. In this case, the two light sources may be configured to emit two different pattern lights, and one light projecting optical system may be configured to irradiate the spot lights of each pattern light at positions where they do not overlap with each other.

[0063] The light projecting unit that projects two pattern lights may be realized by a combination of a light source 1a and a single light projecting optical system 1b. In this case, the light projecting optical system 1b is driven so that the pattern lights are irradiated to positions where the dot-like lights of the first and second lights do not overlap with each other.

[0064] The light projecting unit that projects two pattern lights may be realized by a combination of a single light source 1a that can emit two types of pattern lights and a single light projecting optical system 1b. In this case, by changing the pattern lights emitted by the first and second lights, the dot-like lights of each pattern light can be configured to be irradiated to positions where they do not overlap with each other.

[0065] In the present embodiment, there are provided two types of light projecting and receiving modes: a first mode in which the light projecting units 1-1 and 1-2 are irradiated at different timings and received by the light receiving unit 2, and a second mode in which the light projecting units 1-1 and 1-2 are irradiated simultaneously and received by the light receiving unit 2. The user can select the first mode and the second mode by operating a switching switch or the like. As will be described later, when the first mode is selected, a distance image with high accuracy and high spatial resolution can be obtained, and when the second mode is selected, a distance image with high temporal resolution can be obtained by high-speed imaging.

[0066] FIG. 12 is a diagram showing an example of the pattern light irradiated from the light projecting unit. FIGS. 12(a) to (c) are examples in which the light projecting unit 1-2 irradiates the pattern light at a position shifted by half of the dot interval in the horizontal direction with respect to the light projecting unit 1-1. In the first mode, the light projecting unit 1-1 projects the pattern light shown in FIG. 12(a) for the first light, and the light projecting unit 1-2 projects the pattern light shown in FIG. 12(b) for the second light. In the second mode, the light projecting units 1-1 and 1-2 project their respective pattern lights simultaneously, so that the pattern light as shown in FIG. 12(c) is irradiated. The dot-like light density of the pattern light in the second mode is larger than the dot-like light density of each individual pattern light in the first mode.

[0067] Figures 12(d) to 12(f) show an example in which the light projecting unit 1-2 irradiates pattern light at a position shifted by half of the dot interval in the vertical direction with respect to the light projecting unit 1-1. In the first mode, the pattern light in Fig. 12(d) is projected from the first light projecting unit 1-1, and the pattern light in Fig. 12(e) is projected from the second light projecting unit 1-2. In the second mode, the pattern light in Fig. 12(f) is irradiated by simultaneously projecting the pattern light from the light projecting units 1-1 and 1-2.

[0068] According to the first mode, since the interval between the dot lights of the irradiated pattern light is sufficiently wide, the multipath component measured at the dot (distance measurement point) of the dot light is minimized. Furthermore, high-precision distance measurement with reduced multipath interference can be performed using the intensity of the multipath light measured at a point (nearby point) located near the dot light and not incident with direct reflected light. Specifically, the multipath intensity at the position of the dot light is estimated based on the intensity of the multipath light measured at a nearby point other than the position of the dot light, and correction is performed by subtracting the multipath component from the light measured at the position of the dot light, thereby using a technique for reducing multipath interference. In the first mode, by integrating the distance image obtained with high precision by the pattern light of the light projecting unit 1-1 and the distance image obtained by the pattern light of the light projecting unit 1-2, a distance image with high spatial resolution (resolution) and high precision can be obtained.

[0069] According to the second mode, the interval between the spot lights of the patterned light irradiated is narrower than that in the first mode. When the spot lights interfere with each other and the neighboring points disappear, it is not possible to improve the accuracy of the distance image by subtracting the multipath components using the characteristics of the patterned light as in the first mode. However, since the time required for one shot can be shortened to about half of that in the first mode, high-speed shooting becomes possible. This is similar to shortening the shutter speed in a camera, and it is possible to reduce the blurring of the image of the distance image even when shooting a moving object. Thus, in the second mode, a distance image with high time resolution can be obtained. Also, in the second mode, although it is larger than the multipath component measured in the first mode, the multipath component measured at the distance measurement point is smaller compared to the case where diffused light that irradiates the entire area around the distance measurement point is used. That is, in the second mode, it is possible to obtain a distance image with a balance between distance measurement accuracy and time resolution.

[0070] FIG. 13 is a flowchart showing the procedure for acquiring a distance image in the first mode. First, the imaging control unit 401 controls the light projecting unit 1-1 and the light receiving unit 2 so that the light projecting unit 1-1 projects the first patterned light and the light receiving unit 2 receives the reflected light (step S10). As described above, FIG. 12(a) is an example of the first received image. Also, the image storage unit 402 stores the first received image received by the light receiving unit 2 (step S11).

[0071] Next, the imaging control unit 401 controls the light projecting unit 1-2 and the light receiving unit 2 so that the light projecting unit 1-2 projects the second patterned light and the light receiving unit 2 receives the reflected light (step S12). Also, the image storage unit 402 stores the second received image received by the light receiving unit 2 (step S13). As described above, FIG. 12(b) is an example of the second received image. Steps S10 to S13 are the shooting procedure according to the first mode.

[0072] Next, the distance calculation unit 403 reads out the first received image and the second received image from the image storage unit 402 (step S14). Subsequently, the distance calculation unit 403 acquires a first distance image using the first received image (step S15), and acquires a second distance image using the second received image (step S16).

[0073] Subsequently, the distance calculation unit 403 integrates the acquired first distance image and second distance image to acquire a distance image with high spatial resolution (step S17). Finally, the output unit 404 outputs the integrated distance image to an external device or the like (step S18). Steps S14 to S18 are post-processing procedures in the first mode. As described above, in the first mode, a distance image with high spatial resolution and high accuracy can be acquired.

[0074] FIG. 14 is a flowchart showing a procedure for acquiring a distance image in the second mode. First, the imaging control unit 401 controls the light projecting unit 1-1, the light projecting unit 1-2, and the light receiving unit 2 so that the light receiving unit 2 simultaneously receives the first pattern light irradiated by the light projecting unit 1-1 and the second pattern light irradiated by the light projecting unit 1-2 (step S20). Also, the image storage unit 402 stores the received image (third received image) received by the light receiving unit 2 (step S21). Here, the third received image is a pattern light in which the first pattern light and the second pattern light are superimposed as shown in FIGS. 12(c) and (f). Steps S20 to S21 are imaging procedures in the second mode.

[0075] Next, the distance calculation unit 403 reads out the third received image from the image storage unit 402 (step S22). Subsequently, the distance calculation unit 403 acquires a third distance image (step S23).

[0076] Finally, the output unit 404 outputs the acquired third distance image to an external device or the like (step S24). Steps S22 to S24 are post-processing procedures in the second mode. As described above, in the second mode, high-speed imaging is possible, and a distance image with high time resolution can be acquired.

[0077] FIG. 15 is a diagram showing an example of a distance image obtained by photographing a person moving in the first mode. Here, the broken line in the upper part of FIG. 15 indicates the silhouette of the person. In FIGS. 15(a) and 15(b), it is assumed that a second distance image is taken after the first distance image. In FIG. 15(a), there is little movement of the person, and in the distance image obtained by integrating the first distance image and the second distance image, the shape of the person is not distorted. On the other hand, in FIG. 15(b), the movement of the person is large, and it can be seen that the shape of the person is greatly distorted in the distance image obtained by integrating the first distance image and the second distance image.

[0078] Therefore, it is preferable to use the first mode capable of high-resolution photographing for a subject with little movement, and the second mode capable of high-speed photographing for a subject with large movement. For example, the user can select the first mode when the movement of the subject to be photographed is small and the second mode when the movement of the subject to be photographed is large by using the main controller device connected via the input / output I / F 43.

[0079] As described above, in this embodiment, by using the first mode for a subject with little movement, a high-precision and high-resolution distance image can be obtained, and by using the second mode for a subject with large movement, a high-quality distance image with little blur due to movement can be obtained.

[0080] In this embodiment, the recommended light projection / reception mode (recommended mode) may be presented to the user among the first mode and the second mode. FIG. 16 is a block diagram showing the hardware configuration of a modification of the first embodiment. The distance measuring device 100 in FIG. 16 is obtained by adding an RGB camera 8, an ADC 9, and an image input circuit 50 to the distance measuring device 100 in FIG. 1. Hereinafter, the description of the same parts as those in FIG. 1 will be omitted, and the parts different from those in FIG. 1 will be described.

[0081] The RGB camera 8 is an example of a luminance light-receiving unit, and includes a CMOS (Complementary Metal Oxide Semiconductor) sensor for capturing a luminance image (RGB image) and a luminance light-receiving optical system including an optical element that guides incident light to the CMOS sensor. The ADC 9 converts the captured luminance image into digital data. The image input circuit 50 inputs the digitized luminance image and stores it in a storage unit such as the RAM 46, or controls the reading of the stored luminance image. The distance measurement control unit 4 calculates a recommended mode according to the functional configuration described later and presents it to the user.

[0082] FIG. 17 is a functional block diagram showing the functional configuration of presenting a recommended mode in the distance measurement control unit 4. As shown in FIG. 17, the CPU 44 of the distance measurement control unit 4 realizes the functions of the imaging control unit 501, the RGB image storage unit 502, the recommended mode calculation unit 503, the output unit 504, and the user operation unit 510 by executing the light transmission / reception mode recommendation program stored in the SSD 47.

[0083] The imaging control unit 501 controls the RGB camera 8, the ADC 9, and the image input circuit 50 so as to capture and input a luminance image (RGB image). The RGB image storage unit 502 stores the input luminance image in a storage unit such as the RAM 46.

[0084] The recommended mode calculation unit 503 reads out the luminance image, calculates a recommended mode based on the read luminance image, and inputs it to the output unit 504. The output unit 504 outputs the recommended mode and inputs it to the user operation unit 510.

[0085] The user operation unit 510 includes an information display unit 511 and a light transmission / reception mode operation unit 512. The information display unit 511 displays the recommended mode input from the output unit 504 on an external device via the input / output I / F 43. The user determines the light transmission / reception mode with reference to the recommended mode and inputs the determined light transmission / reception mode to the external device. Note that as the external device for displaying the recommended mode, a PC, a tablet, etc. that are externally connected wirelessly or by wire can be used. Further, by providing the light transmission / reception device with a display panel and an operation panel, the recommended mode may be displayed and the light transmission / reception mode input by the user may be accepted. These display panel and operation panel may each have a touch panel or physical buttons.

[0086] The light transmission / reception mode operation unit 512 acquires the light transmission / reception mode input by the user from the external device via the input / output I / F 43 and inputs it to the imaging control unit 401. The imaging control unit 401 controls the light projection unit and the light reception unit to perform imaging in the first mode or the second mode according to the above-described procedure.

[0087] FIG. 18 is a flowchart showing a procedure for calculating the recommended mode. First, the imaging control unit 501 performs imaging twice with the RGB camera and stores two luminance images that are temporally close to each other in a storage unit such as the RAM 46. In the first imaging, the first luminance image is stored (steps S30, S31), and in the second imaging, the second luminance image is stored (steps S32, S33).

[0088] Next, the recommended mode calculation unit 503 reads out the first luminance image and the second luminance image, and calculates the sum of absolute differences (SAD) of the pixel values for the corresponding pixels of both (step S34). The SAD may be obtained by calculating the difference between the R, G, B values of each pixel in the first luminance image and the R, G, B values of the corresponding pixel in the second luminance image. For example, the SAD may be obtained using only the G value of each pixel. Alternatively, the SAD may be obtained using the luminance value converted from the R, G, B values of each pixel.

[0089] Next, the recommended mode calculation unit 503 determines whether the calculated SAD is greater than a predetermined value (step S35). Here, the predetermined value is for determining whether the movement of the distance measurement target is large, and is assumed to be set in advance by a preliminary experiment or the like. In step S35, if it is determined that SAD is equal to or less than the predetermined value (SAD is not large), the first mode is set as the recommended mode (step S36). If it is determined that SAD is greater than the predetermined value (SAD is large), the second mode is set as the recommended mode (step S37).

[0090] As described above, in this modification, among the first mode and the second mode, the recommended light emission / reception mode (recommended mode) can be presented to the user. Therefore, the user can easily select which of the first mode and the second mode to use in consideration of the presented recommended mode.

[0091] Note that the presentation of the recommended mode may be performed at the stage of preliminary shooting before starting distance measurement, or may be performed during shooting after starting distance measurement. When presenting the recommended mode after starting distance measurement, for example, when the distance measurement device 100 starts distance measurement using the first mode and the output unit 504 outputs the second mode as the recommended mode, the information display unit 511 may display a recommendation of the second mode as an alert to the user. By doing so, it is possible to notify the user that there is a possibility that the distance measurement target moves during distance measurement in the first mode and the distance image is blurred. In addition, by viewing this alert, the user can quickly switch the light emission / reception mode and perform distance measurement in an appropriate mode. Further, when the output unit 504 outputs the first mode as the recommended mode during distance measurement in the second mode by the distance measurement device 100, the information display unit 511 may display a recommendation of the first mode.

[0092] FIG. 19 is a functional block diagram showing another functional configuration of the presentation of the recommended mode. In the above-described modification, the configuration is such that the light transmission / reception mode determined and input by the user in consideration of the recommended mode is used. However, as shown in FIG. 19, the recommended mode calculated by the recommended mode calculation unit 503 may be used as the light transmission / reception mode as it is. The output unit 504 in FIG. 19 inputs the input recommended mode as it is to the imaging control unit 401 as the light transmission / reception mode. By doing so, when the distance measurement target starts to move during distance measurement in the first mode and there is a possibility that the distance image blurs, the light transmission / reception mode can be automatically switched to the second mode to prevent the distance image from blurring. Also, when the movement of the distance measurement target becomes small during distance measurement in the second mode, the light transmission / reception mode can be automatically switched to the first mode, and a high-precision distance image with reduced multipath interference can be acquired at a high resolution.

[0093] In the above determination of the recommended mode, a luminance image (RGB image) is used. However, the same determination may be made using a distance image acquired by a ToF camera instead. For example, the recommended mode calculation unit 503 may calculate the SAD using the difference between corresponding pixels from the two distance images output from the output unit 404 in FIG. 5 and determine the recommended mode in the same manner as described above.

[0094] At this time, as the two distance images, two images that are temporally close are used. For example, when acquiring distance images in the first mode, if the distance image obtained by the irradiation of the light projecting unit 1-1 is the first image, the distance image obtained by the irradiation of the light projecting unit 1-2 is the second image, and subsequently, the distance image obtained by the irradiation of the light projecting unit 1-1 is the third image. When using the first image and the third image, these images are acquired at relatively close time intervals, and the positions of the dot lights also coincide with each other in each pattern light. Therefore, the SAD may be calculated from the difference in pixel values (distance data) at the positions of the corresponding dot lights in each image. When using the first image and the second image, although these images are acquired at the closest time intervals, since the positions of the dot lights are shifted from each other by half of the dot light interval, interpolation processing is required. Specifically, from the distance data at the dot light positions in the second image, the distance data at the positions corresponding to the dot light positions in the first image is obtained by interpolation, and the SAD may be obtained using the interpolated data in the second image and the data in the first image.

[0095] By using the distance images acquired by the ToF camera in this way, the recommended mode calculation unit 503 can more directly determine the presence or absence of blurring of the distance images due to the movement of the measurement target.

[0096] In this modified example, the SAD obtained from the images by the RGB camera or the ToF camera is used as an index (movement index) of the magnitude of the movement of the measurement target. However, other values may also be used as the movement index. For example, from two temporally close images, a motion vector known in video compression technology or the like is obtained, and whether the movement of the measurement target is large may be determined based on whether the magnitude of the motion vector in the measurement target is greater than a predetermined value.

[0097] In the calculation of the above recommended mode, the determination of whether the movement of the object to be measured is large is used, but it may be possible to use the determination of whether the multipath interference is large. Fig. 20 is a plan view of the scene of the object to be measured as seen from above. As shown in Fig. 20(a), scenes with a large amount of multipath interference have characteristics such as "surrounded by high-reflectivity walls", "many irregularities in shape", and "many nearby target objects". Also, as shown in Fig. 20(b), scenes with little multipath interference have characteristics such as "open space or surrounded by low-reflectivity walls", "few irregularities in shape", and "few nearby target objects".

[0098] In order to determine such differences in scenes, for example, it is possible to measure the light amount of points other than the dot light in the irradiation area of the pattern light. At points other than the dot light, mainly noise due to multipath interference is measured. Therefore, if the measured value is larger than a predetermined value, it is determined that the scene has a large amount of multipath interference. Conversely, if the measured value is not larger than the predetermined value, it is determined that the scene has little multipath interference. Then, the above-described recommended mode calculation unit 503 outputs the first mode when it is determined that the scene has a large amount of multipath interference, and the second mode when it is determined that the scene has little multipath interference, as the recommended mode. Note that the predetermined value used for the determination is set in advance by a preliminary experiment or the like.

[0099] By doing so, it is possible to perform light transmission and reception using an appropriate recommended mode according to the scene of the object to be measured, based on the determination of whether the multipath interference is large.

[0100] Thus, according to this embodiment, by taking a picture in the light transmission and reception mode selected from the first mode and the second mode, it becomes possible to take a picture according to the situation and scene of the object to be photographed.

[0101] Further, according to the present embodiment, since one light receiving unit 2 is common, there is no optical difference at the location where the distance can be measured. Therefore, the distance images obtained by receiving light from each light projecting unit can be integrated by simple image processing.

[0102] Furthermore, according to the distance measuring device 100 of the present embodiment, there is an effect that the housing can be made smaller than arranging two ToF cameras side by side.

[0103] Note that, according to the present embodiment, the distance measuring device 100 is configured to perform distance calculation, correction, etc., but the present invention is not limited thereto. Here, FIG. 21 is a block diagram showing a system configuration of a distance measuring system 300 including a distance measuring device (imaging device) 500. As shown in FIG. 21, the distance measuring system 300 has a system configuration in which a distance measuring device (imaging device) 500 is connected to an external system (external device) 400 such as a PC or the cloud. According to the distance measuring system 300 shown in FIG. 21, the distance measuring device (imaging device) 500 may be an imaging device that performs ToF imaging (imaging procedure in each mode), and processing such as distance calculation and correction may be performed by an external system 400 such as a PC or the cloud. That is, the external system 400 functions as a control unit that controls the acquisition of a distance image (post-processing in each mode) based on the received light images captured and stored in each mode.

[0104] Note that, according to the present embodiment, the distance measuring device 100 is configured to include two pattern light projecting units (light projecting units 1-1 and 1-2), but the number of pattern light projecting units is not limited to two, and may be three or more. That is, there may be three or more pattern light projecting units that project pattern light toward the same measurement target area. At this time, it is preferable to arrange the positions of the dot lights of each pattern light so that the dot lights of each pattern light are irradiated at different positions from the dot lights of other pattern lights. Note that, by using one pattern light projecting unit and controlling the light emission positions of the light emitting element array as a light source, multiple irradiations of a sparse pattern light with shifted dot light positions (first mode) and irradiation of a dense pattern light (second mode) may be configured. Further, by changing the projection optical system, the position and density of the dot light may be changed.

[0105] FIG. 22 shows an example of pattern light having different positions of dot lights. FIG. 22(a) shows an example where the number of types of pattern light is three, and FIG. 22(b) shows an example where the number of types of pattern light is four. In FIG. 22, white circles represent dot lights belonging to the first pattern light, hatched circles represent dot lights belonging to the second pattern light, dotted circles represent dot lights belonging to the third pattern light, and black circles represent dot lights belonging to the fourth pattern light. Thus, even when three or more light projecting units are provided, it is preferable to arrange the positions of the dot lights of each light projecting unit so that the dot lights are irradiated at different positions from each other. Further, the pattern light may be an array in which each dot light is arranged at the vertices of a quadrilateral, an array in which each dot light is arranged at the vertices of a polygon such as a triangle, or a random array.

[0106] Note that, as an example, the distance measuring device 100 of the present embodiment is configured such that each optical element (one light receiving unit 2 and two different light projecting units 1-1, 1-2) is fixedly provided on one side as shown in FIG. However, the present invention is not limited to this. Here, FIG. 23 is a diagram showing another configuration example of the distance measuring device 100. As shown in FIG. 23, the distance measuring device 100 may be provided with a device 200 such as a tripod that functions as a support unit, and a rotating means 220 that is a rotating table rotated by, for example, an electric motor or the like in the front stage of the device 200. In this case, the distance measuring device 100 rotates the device itself by the rotating means 220 to change the imaging direction and acquires distance images of the entire circumference.

[0107] When the distance measurement control unit 4 of the distance measuring device 100 controls the rotation of the rotating means 220, it has a predetermined rotation pattern. For example, the distance measurement control unit 4 of the distance measuring device 100 controls according to a rotation pattern of "rotate by a predetermined distance" → "stop rotating and take a picture" → "rotate by a predetermined distance" → "stop rotating and take a picture".

[0108] (Second Embodiment) Next, the second embodiment will be described.

[0109] The second embodiment is different from the first embodiment in that each light projecting unit is installed such that the distance from the light receiving unit is equal. Hereinafter, in the description of the second embodiment, the description of the same parts as those of the first embodiment will be omitted, and the parts different from those of the first embodiment will be described.

[0110] Here, FIG. 24 is a diagram showing the light projecting system and the light receiving system of the distance measuring device 100 according to the second embodiment. As shown in FIG. 24, in the distance measuring device 100 of the present embodiment, each light projecting unit 1-1, 1-2 as the light projecting system is installed such that the distance from the light receiving unit 2 as the light receiving system is equal. When a box-shaped object to be measured is placed as shown in FIG. 24, the light of the light projecting unit 1-1 does not hit the right side surface of the object, but the light of the light projecting unit 1-2 hits it, and the reflected light is received by the light receiving unit 2.

[0111] FIG. 25 is a diagram showing an example of a distance image. FIG. 25(a) is an example of a distance image obtained by emitting the pattern light of the light projecting unit 1-1, and FIG. 25(b) is an example of a distance image obtained by emitting the pattern light of the light projecting unit 1-2. The distance measurement result with the pattern light of the light projecting unit 1-1 in the arrangement shown in FIG. 24 is obtained only on the left side surface of the object to be measured where the light hits, as shown in FIG. 25(a). On the other hand, the distance measurement result of the pattern light of the light projecting unit 1-2 in the arrangement shown in FIG. 24 is obtained on the left and right side surfaces of the object to be measured, as shown in FIG. 25(b).

[0112] When performing light emission and reception in the above-described first mode, in the location where the results of the pattern lights of the light projecting units 1-1 and 1-2 are both obtained (the left side surface in FIG. 25), a high-precision and high-resolution distance image can be acquired by the multi-path interference correction process and the integration of the two distance images described in the first embodiment. Also, in the location where the distance measurement value is obtained only with the pattern light of the light projecting unit 1-2 (the black region shown in FIG. 25(a)), a high-precision distance image with multi-path interference corrected by the pattern light of the light projecting unit 1-2 can be acquired.

[0113] On the other hand, when light is projected and received in the second mode, a distance image with high time resolution is obtained in locations where the results of both the pattern light from light projecting units 1-1 and 1-2 are obtained, and at locations where only the results of the pattern light from light projecting unit 1-2 are obtained, although the number of ranging points is reduced by about half, a high-precision distance image with multipath interference corrected can be obtained.

[0114] 24 and 25 show examples in which light from light-projecting unit 1-2 strikes the left and right sides of the target. However, depending on the angle of the target, only light from light-projecting unit 1-1 may strike the left side and only light from light-projecting unit 1-2 strike the right side. In such a case, in the first mode, distance images for both the left and right sides can be acquired by integrating the distance image acquired by light projection and reception from light-projecting unit 1-1 and the distance image acquired by light projection and reception from light-projecting unit 1-2. In addition, in the second mode, light-projecting units 1-1 and 1-2 simultaneously project light, so distance images for both the left and right sides can be acquired in approximately half the time required in the first mode. As described above, the distance measuring device 100 of this embodiment includes one light-receiving unit 2 and two different light-projecting units 1-1 and 1-2. Information on the areas struck by the respective light projections is used to complement each other to acquire a distance image, enabling distance measurements to be obtained over a wider area.

[0115] Furthermore, according to the distance measuring device 100 of this embodiment, since one light receiving unit 2 is shared, there is no optical difference in the points where distance can be measured, and therefore when processing two types of images, correction values for each pixel can be calculated by simple image processing.

[0116] Furthermore, the distance measuring device 100 of this embodiment has the advantage that the housing can be made smaller than when two ToF cameras are arranged side by side.

[0117] (Third embodiment) Next, a third embodiment will be described.

[0118] The third embodiment is different from the first and second embodiments in that an omnidirectional imaging device is applied as the distance measuring device 100. Hereinafter, in the description of the third embodiment, the description of the same parts as those in the first and second embodiments will be omitted, and the parts different from those in the first and second embodiments will be described.

[0119] FIG. 26 is an external perspective view showing the configuration of the distance measuring device 100 according to the third embodiment, FIG. 27 is a diagram showing an example of the schematic configuration of the distance measuring device 100, and FIG. 28 is a diagram showing an example of the arrangement of the optical system. In the present embodiment, the distance measuring device 100 has a function as a distance measuring device (ToF camera) using the ToF (Time of Flight) method and a function as a luminance camera (RGB camera), and performs imaging with the ToF camera and the luminance camera for the entire sphere.

[0120] As shown in FIGS. 26 to 28, the distance measuring device 100 includes first light projecting units 20A and 20B, second light projecting units 30A and 30B, a ToF light receiving unit 60 as a light receiving unit, a luminance light receiving unit 80, and a control unit 120. The first light projecting units 20A and 20B and the ToF light receiving unit 60, and the second light projecting units 30A and 30B and the ToF light receiving unit 60 each function as a distance measuring device using the ToF method, that is, a ToF camera, and the luminance light receiving unit 80 has a function as a luminance camera.

[0121] The first light projection units 20A and 20B and the second light projection units 30A and 30B each irradiate the measurement target area with ranging light (such as infrared light). The first light projection units 20A and 20B include light sources 210A and 210B that emit infrared light, and ToF light projection systems 211A and 211B (light projection optical systems) composed of optical elements that expand the divergence angle, and emit the light of the light sources 210A and 210B at a wide angle. The second light projection units 30A and 30B include light sources 310A and 310B that emit infrared light, and ToF light projection systems 311A and 311B (light projection optical systems) composed of optical elements that expand the divergence angle, and emit the light of the light sources 310A and 310B at a wide angle. The optical elements of the ToF light projection systems 211A, 211B, 311A, and 311B include, for example, lenses, DOE (diffractive optical elements), diffusion plates, etc. The light sources 210A, 210B, 310A, and 310B are, for example, 2D array VCSELs. In the distance measurement device 100 of the present embodiment, the two first light projection units 20A and 20B are arranged facing opposite directions, and the two second light projection units 30A and 30B are arranged facing opposite directions.

[0122] The two first light projection units 20A and 20B and the two second light projection units 30A and 30B are structured illumination that irradiates patterned light (dot pattern in this embodiment), which is an example of structured light, into space.

[0123] The ranging light emitted from the first light projection units 20A and 20B and the second light projection units 30A and 30B is reflected by an object existing in the measurement target area. The ToF light receiving unit 60 receives the reflected light from the object in the measurement target area. The ToF light receiving unit 60 includes a ToF sensor 110 that is sensitive to the ranging light, and a ToF light receiving optical system 112 (first light receiving optical system) composed of an optical element that guides the incident light to the ToF sensor 110. The optical element of the ToF light receiving optical system 112 includes, for example, a lens. The ToF sensor 110 is a light receiving element in which the light receiving pixels are two-dimensionally arranged, and each pixel corresponds to each position in the measurement target area. Therefore, the ToF light receiving unit 60 can individually receive the light from each position in the measurement target area. The distance measurement device 100 of the present embodiment has four ToF light receiving units 60 arranged facing different directions.

[0124] The luminance light receiving unit 80 acquires a two-dimensional image by means of the CMOS sensor 33. The luminance light receiving unit 80 includes a CMOS sensor 33 for photographing a luminance image (RGB image) and a luminance light receiving optical system 113 (second light receiving optical system) composed of optical elements that guide incident light to the CMOS sensor 33. The optical elements of the luminance light receiving optical system 113 include, for example, lenses.

[0125] The distance measuring device 100 of the present embodiment maps the luminance image (RGB image) obtained by the luminance light receiving unit 80 to the coordinate point cloud obtained from the distance image. Thereby, the distance measuring device 100 can digitize the distance and shape information of the surrounding space with color information.

[0126] The control unit 120 drives or controls the first light projecting units 20A and 20B, the ToF light receiving unit 60, the luminance light receiving unit 80, and the second light projecting units 30A and 30B. The control unit 120 is connected to each of the light sources 210A and 210B, 310A and 310B, the ToF sensor 110, and the CMOS sensor 33 by cables, FPCs, FFCs, etc.

[0127] Here, the first light-projecting units 20A and 20B are an example of a first light-projecting unit that projects patterned light (structured light). The second light-projecting units 30A and 30B are an example of a second light-projecting unit that projects patterned light (structured light). The first light-projecting unit 20A and the second light-projecting unit 30A, which are arranged on the +X side in FIG. 27, irradiate their respective patterned lights toward the +X side of the distance measuring device 100. The first light-projecting unit 20A and the second light-projecting unit 30A are arranged so that their point-like lights do not overlap. Similarly, the first light-projecting unit 20B and the second light-projecting unit 30B, which are arranged on the -X side in FIG. 27, are arranged so that the point-like lights of their respective patterned lights irradiated toward the -X side of the distance measuring device 100 do not overlap. For example, such an arrangement can be achieved by tilting the first light-projecting units 20A and 20B in the -Z direction and the second light-projecting units 30A and 30B in the +Z direction. The combination of inclinations of the light projecting units may be other than those described above. Furthermore, the patterned lights from different light projecting units may include areas where the point lights of each unit overlap. The ToF light receiving unit 60 is an example of a light receiving section to which incident light including projected light is incident. The luminance light receiving unit 80 outputs information including at least luminance.

[0128] In this embodiment, as shown in Figures 26 to 28, the distance measuring device 100 has an elongated shape that is long in the Z-axis direction. In the first stage of the distance measuring device 100, which is closest to the +Z direction, four ToF light receiving optical systems 112, each with a field angle of 120 degrees or more, are arranged so as to face three directions in the XY plane and one direction, the +Z direction. In the second stage, which is on the -Z side of the first stage of the distance measuring device 100, two ToF light projecting systems 211A and 211B, each with a field angle of 180 degrees or more, and two luminance light receiving optical systems 113, each with a field angle of 180 degrees, are arranged. The two ToF light projecting systems 211A and 211B face in opposite directions (+X direction and -X direction), and the two luminance light receiving optical systems 113 also face in opposite directions (+Y direction and -Y direction). Two ToF light projecting systems 311A and 311B, each with an angle of view of 180 degrees or more, are arranged in the third stage, which is on the -Z side of the second stage of distance measuring device 100. The two ToF light projecting systems 311A and 311B face in opposite directions (+X and -X directions). A control unit 120 and a battery 130 are arranged in the lower stage on the -Z side of distance measuring device 100. This allows for a compact arrangement of optical systems that cover the entire celestial sphere, making it possible to miniaturize the distance measuring device.

[0129] The control unit 120 controls the timing at which the first light-projecting units (light-projecting units) 20A and 20B and the second light-projecting units (light-projecting units) 30A and 30B project light, and detects light reception by the ToF light-receiving unit (light-receiving unit) 60. First, the control unit 120 controls the timing at which the light sources 210A and 210B are driven, causing them to emit light toward the measurement target area. The control unit 120 then photoelectrically converts the light received by the ToF sensor 110 and outputs the converted light as a distance image. At the same time, the control unit 120 causes the CMOS sensor 33 to capture an image and output a brightness image.

[0130] When a direct ToF sensor is used as the ToF sensor 110, the control unit 120 outputs a distance image based on the timing of light reception at each pixel. On the other hand, when an indirect ToF sensor is used as the ToF sensor 110, the control unit 120 outputs a phase image based on the amount of light received at each pixel at four different phases. The control unit 120 can generate a distance image from the four phase images.

[0131] In this embodiment, such an omnidirectional imaging device has the two light projection modes (first mode and second mode) described in the first and second embodiments. In the first mode, the control unit 120 controls the first light-projecting unit 20A and the second light-projecting unit 30A to project pattern light at different timings, and the first light-projecting unit 20B and the second light-projecting unit 30B to project pattern light at different timings. In the second mode, the control unit 120 controls the first light-projecting unit 20A and the second light-projecting unit 30B to simultaneously project pattern light, and the first light-projecting unit 20B and the second light-projecting unit 30B to simultaneously project pattern light.

[0132] In the first and second modes of this embodiment, the timings at which the first light-projecting unit 20A and the first light-projecting unit 20B project light may be simultaneous or different. The timings at which the second light-projecting unit 30A and the second light-projecting unit 30B project light may be simultaneous or different. For example, if there is an overlap in the projection ranges of the pattern lights, they may be projected at different times.

[0133] FIG. 29 illustrates the illumination range of the pattern light projected by the second light-projecting units 30A and 30B. Here, the first light-projecting units 20A and 20B are tilted in the −Z direction, and the second light-projecting units 30A and 30B are tilted in the +Z direction. When the second light-projecting units 30A and 30B project light simultaneously, as shown in FIG. 29 , overlapping of the pattern light occurs above the distance measuring device 100. In this overlapping area, the spacing between the point lights becomes narrow, making it difficult to reduce the effects of multipath interference. To avoid this, the second light-projecting units 30A and 30B can project light at different times. Note that overlapping of the pattern light may also occur within the illumination range of the first light-projecting units 20A and 20B. However, when the first light-projecting units 20A and 20B are tilted in the −Z direction as described above, the pattern light is blocked by the housing of the distance measuring device 100, so overlapping does not occur below the distance measuring device 100.

[0134] 30 is a flowchart showing the procedure for acquiring a distance image in the first mode. First, the ToF light receiving unit 60 receives the pattern emitted by the first light projecting units 20A and 20B (step S40), and the distance measuring device 100 stores the acquired first received light image (step S41).

[0135] Next, the ToF light receiving unit 60 receives the pattern emitted by the second light projecting unit 30A (step S42), and the distance measuring device 100 saves the acquired second received light image (step S43). After that, the ToF light receiving unit 60 receives the pattern emitted by the second light projecting unit 30B (step S44), and the distance measuring device 100 saves the acquired third received light image (step S45). Because the light projection timings of the second light projecting unit 30A and the second light projecting unit 30B are different, overlapping of the pattern lights as described above does not occur. Steps S40 to S45 are the shooting procedure in the first mode.

[0136] Next, the distance measuring device 100 reads out the first, second, and third received light images, and uses the respective received light images to obtain first, second, and third distance images (steps S46 to S49). Since sufficient spacing between point lights is maintained in each received light image, the influence of multipath interference is reduced when obtaining the distance images, resulting in high accuracy.

[0137] Next, distance measuring device 100 integrates the acquired first, second, and third distance images to acquire a spherical distance image (step S50). Since the pattern light positions in the second and third distance images are shifted by half the spacing between the point lights relative to the first distance image, integrating these images allows for a high-resolution, high-precision distance image to be acquired. Finally, distance measuring device 100 outputs the integrated distance image to an external device or the like (step S51). Steps S46 to S51 are post-processing procedures in the first mode.

[0138] Note that, when overlapping of the pattern lights occurs in the irradiation ranges of the first light-projecting units 20A and 20B by changing the combination of the inclinations of the respective light-projecting units, the first light-projecting units 20A and 20B may be configured to project light at different timings. Furthermore, when the irradiation ranges of the first light-projecting units 20A and 20B overlap and the irradiation ranges of the second light-projecting units 30A and 30B overlap, the first light-projecting units 20A and 20B may be configured to project light at different timings, and the second light-projecting units 30A and 30B may be configured to project light at different timings.

[0139] As described above, according to this embodiment, even when an omnidirectional imaging device is applied as distance measuring device 100, by using the first mode for a target with little movement, it is possible to acquire a distance image with high accuracy and high resolution, and by using the second mode for a target with much movement, it is possible to acquire a high-quality distance image with little blur due to movement.

[0140] In this embodiment, the first light-projecting units 20A, 20B and the second light-projecting units 30A, 30B that project a dot pattern as an irradiation pattern have been given as examples, but this is not limited to this, and the first light-projecting units 20A, 20B and the second light-projecting units 30A, 30B may project any pattern of light, such as a random dot pattern with an irregular dot arrangement or a stripe pattern.

[0141] Application examples in which the distance measuring device 100 described above in the first to third embodiments is used in various detection systems will be described with reference to FIGS. 31 to 35. The detection systems in these application examples have the respective functional blocks described below in addition to the distance measuring device 100. In FIGS. 31 to 35, functional blocks such as a determination unit provided in the detection system are depicted outside the detection system 50X for convenience of drawing. The various detection systems shown in FIGS. 31 to 35 have a system control unit that receives information from the distance measuring device 100 and controls the various detection systems and devices equipped with the various detection systems based on the information from the distance measuring device 100.

[0142] Fig. 31 is a diagram showing an example in which the distance measuring device is applied to an electronic information terminal that authenticates a user. Fig. 31 shows an example of a shape measurement system as a detection system, and is an application example in which the distance measuring device 100 is used for user authentication of an electronic device.

[0143] The portable information terminal 60X, which is an electronic device, has a function for authenticating a user. The authentication function may be realized by dedicated hardware, or may be realized by a CPU that controls the portable information terminal 60X executing a program stored in a ROM or the like. A detection system 50X mounted on the portable information terminal 60X uses a distance measuring device 100.

[0144] When authenticating a user, light is projected from a light source device of a distance measuring device 100 mounted on the portable information terminal 60X toward a user 61X using the portable information terminal 60X.

[0145] Light reflected by the user 61X and its surroundings is received by the light receiving element of the distance measuring device 100, and image data is generated (image is captured) by the image processing unit 62X. The judgment unit 63X judges the degree of match between the image information of the user 61X captured by the distance measuring device 100 and pre-registered user information, and determines whether the user is a registered user.

[0146] Specifically, the shapes (contours and irregularities) of the face, ears, head, etc. of the user 61X can be measured and used as user information.

[0147] In the application example of FIG. 31, the detection system 50X can detect the user 61X with high accuracy, similar to the distance measuring device 100 in the first to third embodiments, and improvement in recognition accuracy can be achieved.

[0148] Figure 31 shows an example in which the distance measuring device 100 is installed in a mobile information terminal 60X, but user authentication using the distance measuring device 100 can also be used in office automation equipment such as stationary PCs and printers, building security systems, etc.

[0149] In terms of functionality, the device is not limited to personal authentication functions and can also be used for scanning three-dimensional shapes such as faces. In this case, too, highly accurate scanning can be achieved by incorporating the distance measuring device 100.

[0150] Fig. 32 is a diagram showing an example in which the distance measuring device is applied to an autonomous traveling system. Fig. 32 shows an example in which the distance measuring device 100 is used in an autonomous traveling system in a moving body equipped with a detection system.

[0151] 32, the distance measuring device 100 is used to sense an object outside the moving body 70X. The moving body 70X is an autonomous moving body that can move automatically while recognizing the external situation.

[0152] A distance measuring device 100 is mounted on a moving object 70X, and the distance measuring device 100 emits light in the direction of travel of the moving object 70X and in the surrounding area. In a room 71X which is the moving area of the moving object 70X, a desk 72X is installed in the direction of travel of the moving object 70X.

[0153] Of the light projected from the light source device of the distance measuring device 100 mounted on the moving body 70X, the light reflected by the desk 72X and its surroundings is received by the light receiving element of the distance measuring device 100, and the photoelectrically converted electrical signal is sent to the signal processing unit 73X.

[0154] Based on the electrical signals sent from the light receiving elements, the signal processing unit 73X calculates information about the layout of the room 71X, such as the distance to the desk 72X, the position of the desk 72X, and the surrounding conditions other than the desk 72X.

[0155] Based on the calculated information, a determination unit 74X determines the movement route, movement speed, etc. of the moving object 70X, and based on the determination result of the determination unit 74X, a driving control unit 75X controls the traveling of the moving object 70X. Here, the driving control unit 75X is an example of a system control unit included in the detection system 50X. Furthermore, the traveling of the moving object 70X is controlled by controlling the operation of a motor or the like, which is a driving source.

[0156] In the application example of FIG. 32, regarding the layout detection of the interior 71X by the detection system 50X, it is possible to perform ranging with high accuracy similar to the ranging device 100 in the first to third embodiments, and it is possible to improve the accuracy of autonomous driving of the moving body 70X.

[0157] FIG. 32 shows an example in which the ranging device 100 is mounted on an autonomous driving type moving body 70X that travels in the interior 71X, but it can also be applied to an autonomous driving type vehicle (so-called self-driving vehicle) that travels outdoors.

[0158] In addition, it can also be applied to a driving support system in a moving body such as an automobile driven by a driver, rather than an autonomous driving type. In this case, the detection system 50X can be used to detect the surrounding situation of the moving body, and the driving of the driver can be supported according to the detected surrounding situation.

[0159] FIG. 33 is a diagram showing an example in which the ranging device is applied to an article inspection system. As shown in FIG. 33, the detection system 50X may be applied to an article inspection system in a factory or the like. Specifically, based on the information acquired by the ranging device 100, the determination unit 52X of the article inspection system (detection system 50X) determines the state of each article 51X.

[0160] FIG. 34 is a diagram showing an example in which the ranging device is applied to an operating device. As shown in FIG. 34, the detection system 50X may be applied to the operation control of a movable device.

[0161] The articulated arm 54X as a movable device has a plurality of arms connected by bendable joints and is provided with a hand part 55X at the tip. The articulated arm 54X is used, for example, in an assembly line in a factory, and grips the object 56X with the hand part 55X when inspecting, transporting, or assembling the object 56X.

[0162] The detection system 50X detects the object 56X and its surrounding area using the distance measuring device 100, and the determination unit 58X of the detection system 50X determines various information about the object 56X, such as the distance to the object 56X, the shape of the object 56X, the position of the object 56X, and the relative positions of multiple objects 56X if multiple objects 56X are present, based on the information acquired by the distance measuring device 100. Then, the drive control unit 59X controls the operation of the articulated arm 54X based on the determination result by the determination unit 58X. Here, the drive control unit 59X is an example of a system control unit included in the detection system 50X.

[0163] Fig. 35 is a diagram showing an example in which the distance measuring device is applied to a driving assistance system. As shown in Fig. 35, a detection system 50X may be applied to a driving assistance system for a moving body such as an automobile.

[0164] A detection system 50X mounted inside an automobile 64X uses a distance measuring device 100 to detect a driver 65X driving the automobile 64X and the surrounding area, and a determination unit 67X of the detection system 50X determines information such as the face (expression) and posture of the driver 65X based on the information acquired by the distance measuring device 100. Then, based on the determination result of the determination unit 67X, a driving control unit 68X provides appropriate driving assistance according to the situation of the driver 65X. Here, the driving control unit 68X is an example of a system control unit included in the detection system 50X.

[0165] The shape measurement system, autonomous driving system, article inspection system, mobile equipment, and driving assistance system described above are all examples of applications of the detection system. The detection system of this embodiment detects information based on high-precision, high-spatial-resolution distance information with little error due to multipath interference obtained by the distance measuring device 100, and high-temporal-resolution distance images obtained by high-speed imaging, thereby enabling highly accurate detection.

[0166] Although various embodiments of the present invention have been described above, the above-described embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These novel embodiments and modifications thereof are included within the scope and spirit of the invention, as well as within the scope of the invention and its equivalents as set forth in the claims. Furthermore, components from different embodiments and modifications may be combined as appropriate.

[0167] For example, aspects of the present invention are as follows. <1> a light-projecting / receiving device comprising: a light-projecting unit that projects first light and second light; a light-receiving unit that receives first reflected light obtained by reflecting the first light off an object and second reflected light obtained by reflecting the second light off an object; and a control unit that controls the light-projecting unit and the light-receiving unit, wherein the first light and the second light are patterned lights each having a plurality of light intensity distributions with peak regions, and at least a portion of the projected region includes a portion where the peak region of one patterned light does not overlap with the peak region of the other patterned light, and the control unit performs light projection and reception in a mode selected from a plurality of light-projecting modes including a first mode in which the first light and the second light are projected at different times and received respectively, and a second mode in which the first light and the second light are projected and received simultaneously. <2> the control unit selects the second mode when a motion index indicating the motion of the object is greater than a predetermined value; <1> 1 is a light projecting and receiving device according to the first embodiment. <3> the control unit determines whether the motion index of the object is greater than the predetermined value, and when the motion index is greater than the predetermined value, presents the second mode to the user as a recommended mode; <2> 1 is a light projecting and receiving device according to the first embodiment. <4> the control unit determines whether the movement index of the object is greater than a predetermined value, and when the movement index is equal to or less than the predetermined value, selects the first mode to project and receive light, and when the movement index is greater than the predetermined value, selects the second mode to project and receive light; <2> or <3> 1 is a light projecting and receiving device according to the first embodiment. <5> The camera further includes a luminance light receiving unit, and before emitting and receiving light in the selected mode, the camera performs multiple image captures using the luminance light receiving unit. <1> ~ <4> 1. The light projecting and receiving device according to claim 1, wherein: <6> selecting the mode from the plurality of light projection modes based on the plurality of photographs taken by the luminance light receiving unit; <5> 1 is a light projecting and receiving device according to the first embodiment. <7> the light projecting unit includes a first light source that emits the first light, and a second light source that is driven by a circuit different from that of the first light source and emits the second light, <1> ~ <6> 1. The light projecting and receiving device according to claim 1, wherein: <8> the light-projecting unit includes a first light-projecting optical system that projects the first light, and a second light-projecting optical system that projects the second light from a position different from that of the first light-projecting optical system; <1> ~ <6> 1. The light projecting and receiving device according to claim 1, wherein: <9> The aforementioned <1> ~ <8> and a distance calculation unit that calculates distance information from the light projecting and receiving device to the object, wherein when the first mode is selected, the distance calculation unit calculates distance information by integrating first distance information calculated based on the projection and reception of the first light and second distance information calculated based on the projection and reception of the second light, and when the second mode is selected, calculates distance information based on the projection and reception of the first light and the second light. <10>A distance measurement system for measuring the distance to an object, comprising: a light projecting unit that projects first light and second light; a light receiving unit that receives first reflected light obtained by reflecting the first light from the object and second reflected light obtained by reflecting the second light from the object; a control unit that controls the light projecting unit and the light receiving unit; and a distance calculation unit that calculates distance information from the imaging device to the object. The first light and the second light are each pattern light having a plurality of light intensity distributions with peak regions, and at least a part of the region where the light is projected includes a portion where the peak region of one pattern light does not overlap with the peak region of the other pattern light. The control unit performs light projection and reception in a mode selected from a plurality of light projection modes including a first mode in which the first light and the second light are projected at different timings and received respectively, and a second mode in which the first light and the second light are projected and received simultaneously. When the first mode is selected, the distance calculation unit integrates first distance information calculated based on the light projection and reception of the first light and second distance information calculated based on the light projection and reception of the second light to calculate distance information. When the second mode is selected, the distance calculation unit calculates distance information based on the light projection and reception of the first light and the second light. <11>A program that causes a computer to function as: a light projecting means for projecting first light and second light; a light receiving means for receiving first reflected light obtained by reflecting the first light from an object and second reflected light obtained by reflecting the second light from the object; and a control means for controlling the light projecting means and the light receiving means. The first light and the second light are each pattern light having a plurality of light intensity distributions with peak regions, and at least a part of the region where the light is projected includes a portion where the peak region of one pattern light does not overlap with the peak region of the other pattern light. The control means performs light projection and reception in a mode selected from a plurality of light projection modes including a first mode in which the first light and the second light are projected at different timings and received respectively, and a second mode in which the first light and the second light are projected and received simultaneously.

Explanation of Reference Numerals

[0168] 1-1 First light projecting unit 1-2 Second light projecting unit 2 Light receiving unit 4 Distance measurement control unit 60 Light receiving unit 100 Distance measurement device 300 Distance measurement system 400 External system 401 Imaging control unit 402 Image storage unit 403 Distance calculation unit 404 Output unit 500 Distance measurement device (imaging device)

Prior art documents

Patent documents

[0169]

Patent Document 1

Claims

1. A light projecting unit that projects first light and second light; A light receiving unit that receives a first reflected light obtained by reflecting the first light from an object and a second reflected light obtained by reflecting the second light from the object; A control unit that controls the light projecting unit and the light receiving unit, wherein the first light and the second light are each pattern light having a plurality of light intensity distributions each having a peak region, and at least a part of the region to which the light is projected includes a portion where the peak region of one pattern light does not overlap with the peak region of the other pattern light; The control unit performs light projection and reception in a mode selected from a plurality of light projection modes including a first mode in which the first light and the second light are projected at different timings and received respectively, and a second mode in which the first light and the second light are projected and received simultaneously. A light projection and reception device.

2. The light projection and reception device according to claim 1, wherein the control unit selects the second mode when a motion index indicating the motion of the object is greater than a predetermined value.

3. The control unit determines whether the motion index is greater than the predetermined value, The light projection and reception device according to claim 2, wherein when the motion index is greater than the predetermined value, the second mode is presented to the user.

4. The control unit determines whether the motion index is greater than the predetermined value, When the motion index is less than or equal to the predetermined value, the first mode is selected for light projection and reception, The light projection and reception device according to claim 2, wherein when the motion index is greater than the predetermined value, the second mode is selected for light projection and reception.

5. Further comprising a luminance light receiving unit, The light projection and reception device according to claim 1 or claim 2, wherein a plurality of shootings are performed by the luminance light receiving unit before performing light projection and reception in the selected mode.

6. The light projection and reception device according to claim 5, wherein the mode is selected from the plurality of light projection modes based on the plurality of shootings by the luminance light receiving unit.

7. The light projecting unit, A first light source that emits the first light, The light projection and reception device according to claim 1, including a second light source that is driven by a circuit different from the first light source and emits the second light.

8. The light projecting unit, A first light projection optical system that projects the first light, The light projection and reception device according to claim 1, including a second light projection optical system that projects the second light from a position different from the first light projection optical system.

9. The light projection and reception device according to any one of claims 1 to 4, A distance calculation unit that calculates distance information from the light transmitting and receiving device to the object; comprising; the distance calculation unit; when the first mode is selected, integrates first distance information calculated based on the light transmission and reception of the first light and second distance information calculated based on the light transmission and reception of the second light to calculate distance information; a distance measuring device that calculates distance information based on the light transmission and reception of the first light and the second light when the second mode is selected.

10. A distance measuring system for measuring the distance to an object, a light projecting unit that projects first light and second light, a light receiving unit that receives first reflected light in which the first light is reflected by the object and second reflected light in which the second light is reflected by the object, a control unit that controls the light projecting unit and the light receiving unit, a distance calculation unit that calculates distance information from the distance measuring system to the object, comprising; the first light and the second light are each pattern light having a plurality of light intensity distributions with peak regions, and at least a part of the region where they are projected includes a portion where the peak region of one pattern light does not overlap with the peak region of the other pattern light, the control unit performs light transmission and reception in a mode selected from a plurality of light projection modes including a first mode in which the first light and the second light are projected at different timings and received respectively, and a second mode in which the first light and the second light are projected and received simultaneously, the distance calculation unit; when the first mode is selected, integrates first distance information calculated based on the light transmission and reception of the first light and second distance information calculated based on the light transmission and reception of the second light to calculate distance information; a distance measuring system that calculates distance information based on the light transmission and reception of the first light and the second light when the second mode is selected.

11. A computer, a light projecting means for projecting first light and second light, a light receiving means for receiving first reflected light in which the first light is reflected by an object and second reflected light in which the second light is reflected by the object, a control means for controlling the light projecting means and the light receiving means, functioning as, the first light and the second light are each pattern light having a plurality of light intensity distributions with peak regions, and at least a part of the region where they are projected includes a portion where the peak region of one pattern light does not overlap with the peak region of the other pattern light, The control means performs light transmission and reception in a mode selected from a plurality of light projection modes including a first mode in which the first light and the second light are projected and received at different timings, and a second mode in which the first light and the second light are projected and received simultaneously, program.

Images