Image-capturing apparatus, distance-measuring apparatus, distance-measuring system, and distance-measuring method

Abstract

An image-capturing apparatus includes a first light emitter (1-1) to emit first light, such as diffused light, to an object in an area, the first light having a first light intensity distribution; a second light emitter (1-2) to emit second light, such as structured light, to the object, the second light having a second light intensity distribution larger than the first light intensity distribution in the area; and a light receiver (2-1) to receive first reflected light that is a reflection of the first light reflected by the object; and second reflected light that is a reflection of the second light reflected by the object. A distance measurement value of diffused light can be acquired continuously with high spatial resolution. A distance measurement value of spot light is relatively correct although the distance measurement value is discontinuous because the number of ranging points is small. A distance calculation unit calculates distance information indicating the distance to the object based on multiple phase images captured by an image sensor (2-1a). A correction-value is calculated to correct the distance information using a distance image obtained by the distance calculation unit through emission of light from the first light source (1-1a) and a distance image obtained by the distance calculation unit through emission of light from the second light source (1-2a). Distance information is corrected using the correction value by the distance measuring apparatus (100) and a distance image with further high accuracy is generated.

Description

FN202304898[DESCRIPTION][Title of Invention]IMAGE-CAPTURING APPARATUS, DISTANCE-MEASURING APPARATUS, DISTANCE-MEASURING SYSTEM, AND DISTANCE-MEASURING METHOD [Technical Field]

[0001] The present disclosure relates to an image-capturing apparatus, a distance-measuring apparatus, a distance-measuring system, and a distance-measuring method. [Background Art]

[0002] In related art, a technique called a time of flight (ToF) system is known as one of techniques of measuring the distance to an object. A ToF camera is a distance-measuring apparatus using the ToF system. The ToF camera emits distance measurement light to an object. The distance measurement light is infrared light having an intensity modulated with a predetermined emission pattern. Then the ToF camera receives the distance measurement light reflected by the object using an image-capturing element for infrared light. The ToF camera detects the difference in time from emission to reception of light per pixel in accordance with the emission pattern, and calculates a distance. The ToF camera collects calculated distance values on the pixel basis in a bit map form and stores the distance values as a "distance image".

[0003] Patent Literature (PTL) 1 and PTL 2 each disclose a distance-measuring apparatus using a ToF system that emits spot-shaped light.[Citation List][Patent Literature]

[0004] [PTL 1]Japanese Unexamined Patent Application Publication No. 2021-110626[PTL 2]Japanese Translation of PCT International Application Publication No. JP-T-2017-517737 [Summary of Invention] [Technical Problem]

[0005] However, with the distance-measuring apparatus using the ToF system of the related art, there is a disadvantage that a phenomenon called multipath interference, in which the distance is calculated to be long, occurs because emitted light enters a sensor in a light path in which the emitted light is reflected multiple times, and the accuracy of distance measurement decreases.

[0006] The present disclosure is made in light of the above-described situations, and an object of the disclosure is to generate a distance image with further high accuracy.FN202304898[Solution to Problem]

[0007] According to an embodiment of the present disclosure, an image-capturing apparatus includes a first light emitter to emit first light to an object in an area, the first light having a first light intensity distribution in the area; a second light emitter to emit second light to the object in the area, the second light having a second light intensity distribution larger than the first light intensity distribution in the area; and a light receiver to receive first reflected light that is a reflection of the first light reflected by the object in the area; and second reflected light that is a reflection of the second light reflected by the object in the area.According to an embodiment of the present disclosure, a distance-measuring apparatus that detects distance information using a time of flight (ToF) system. The distance-measuring apparatus includes: the image-capturing apparatus described above; and a controller to output distance information based on a detection signal of first reflected light of the first light emitted from the first light emitter to the object and a detection signal of second reflected light of the second light emitted from the second light emitter to the object.According to an embodiment of the present disclosure, a distance-measuring system includes: a first light emitter to emit diffused light to an object at a first timing; a second light emitter to emit structured light to the object at a second timing different from the first timing of the first light emitter; a light receiver to: receive first reflected light that is a reflection of the diffused light emitted from the first light emitter to the object; and receive second reflected light that is a reflection of the structured light emitted from the second light emitter to the object; and a controller configured to: obtain distance information based a detection signal of the first reflected light; and correct the distance information using a detection signal of the second reflected light, to detect distance information using a ToF system.According to an embodiment of the present disclosure, a distance-measuring method performed by a distance-measuring apparatus, includes: emitting diffused light to an object at a first timing; emitting structured light to the object at a second timing different from the first timing of the diffused light; receiving first reflected light that is a reflection of the diffused light emitted to the object in the emitting of the diffused light; receiving second reflected light that is a reflection of the structured light emitted to the object in the emitting of the structured light; calculating distance information from a detection signal of the first reflected light; and correcting the distance information using a detection signal of the second reflected light, to detect distance information using a ToF system.[Advantageous Effects of Invention]

[0008] With the present disclosure, an advantageous effect is attained such that a distance image with further high accuracy can be generated.[Brief Description of Drawings]

[0009] FN202304898A more complete appreciation of embodiments of the present disclosure and many of the attendant advantages and features thereof can be readily obtained and understood from the following detailed description with reference to the accompanying drawings.[FIG. 1]FIG. l is a block diagram illustrating a hardware configuration of a distance-measuring apparatus according to a first embodiment.[FIG. 2A]FIG. 2A is a diagram exemplarily illustrating spot light.[FIG. 2B]FIG. 2B is a diagram exemplarily illustrating the spot light.[FIG. 3]FIG. 3 is a timing chart presenting the principle of distance measurement.[FIG. 4]FIG. 4 is a diagram illustrating the principle of generation of noise due to multipath interference with a ToF camera.[FIG. 5]FIG. 5 is a functional block diagram illustrating a functional configuration of a distancemeasuring controller according to the first embodiment.[FIG. 6]FIG. 6 is a view illustrating a light-emitting system and a light-receiving system of the distance-measuring apparatus according to the first embodiment.[FIG. 7]FIG. 7 is a view illustrating an installation example of the distance-measuring apparatus.[FIG. 8]FIG. 8 is a view exemplarily illustrating an influence of multipath interference.[FIG. 9]FIG. 9 is a view exemplarily illustrating a technique of calculating a correction value.[FIG. 10]FIG. 10 is a flowchart presenting a flow of a distance-measuring process performed by the distance-measuring apparatus according to the first embodiment.[FIG. 11 A]FIG. 11 A is a view illustrating an example of a distance image.[FIG. 11B]FIG. 1 IB is a view illustrating an example of a distance image.[FIG. 12]FIG. 12 is a block diagram illustrating a system configuration of a distance-measuring system according to the first embodiment.[FIG. 13]FIG. 13 is a block diagram illustrating a hardware configuration of a distance-measuring apparatus according to a second embodiment.FN202304898[FIG. 14]FIG. 14 is a view illustrating a light-emitting system and a light-receiving system of the distance-measuring apparatus according to the second embodiment.[FIG. 15]FIG. 15 is a view illustrating another example of arrangement of optical elements of the distance-measuring apparatus.[FIG. 16]FIG. 16 is a view illustrating a light-emitting system and a light-receiving system of a distance-measuring apparatus according to a third embodiment.[FIG. 17 A]FIG. 17A is a view illustrating an example of a distance image.[FIG. 17B]FIG. 17B is a view illustrating an example of a distance image.[FIG. 18 A]FIG. 18A is a view exemplarily illustrating saturated pixels in a ToF camera according to a fourth embodiment.[FIG. 18B]FIG. 18B is a view exemplarily illustrating saturated pixels in the ToF camera according to the fourth embodiment.[FIG. 19]FIG. 19 is a flowchart presenting a flow of an image-capturing process to reduce saturated pixels according to the fourth embodiment.[FIG. 20A]FIG. 20A is a view illustrating an example of an image acquired when control on spot light is switched in accordance with the result with diffused light.[FIG. 20B]FIG. 20B is a view illustrating an example of an image acquired when the control on spot light is switched in accordance with the result with diffused light.[FIG. 21]FIG. 21 is an external perspective view illustrating a configuration of a distance-measuring apparatus according to a fifth embodiment.[FIG. 22A]FIG. 22A is a view illustrating an example of a schematic configuration of the distancemeasuring apparatus.[FIG. 22B]FIG. 22B is a view illustrating the example of the schematic configuration of the distancemeasuring apparatus.[FIG. 23]FIG. 23 is a view illustrating an example of arrangement of an optical system.[FIG. 24]FN202304898FIG. 24 is a diagram exemplarily illustrating multipath interference specific to a case where a spherical image-capturing apparatus is applied.[FIG. 25]FIG. 25 is a chart presenting an image capturing sequence to reduce multipath interference when the spherical image-capturing apparatus is applied according to the fifth embodiment. [FIG. 26]FIG. 26 is a diagram of a hardware configuration a distance-measuring apparatus device according to a sixth embodiment.[FIG. 27]FIG. 27 is a diagram of a hardware configuration of a light-source unit according to the sixth embodiment.[FIG. 28]FIG. 28 is a diagram of a hardware configuration of a light-source unit according to a first modification of the sixth embodiment.[FIG. 29]FIG. 29 is a diagram of a hardware configuration of a light-source unit according to the sixth embodiment.[FIG. 30]FIG. 30 is a diagram of a hardware configuration of a light-source unit according to a second modification of the sixth embodiment.[FIG. 31]FIG. 31 is a functional block diagram according to the sixth embodiment.[FIG. 32]FIG. 32 is a view illustrating spot light and diffused light according to the sixth embodiment. [FIG. 33]FIG. 33 is a diagram illustrating a correction range with spot light according to the sixth embodiment.[FIG. 34]FIG. 34 is a flowchart according to the sixth embodiment.[FIG. 35]FIG. 35 is a functional block diagram according to the sixth embodiment.[FIG. 36]FIG. 36 is a diagram illustrating a converter circuit according to the seventh embodiment.[FIG. 37]FIG. 37 is a flowchart according to the seventh embodiment.[FIG. 38]FIG. 38 is a functional block diagram according to the eighth embodiment.[FIG. 39]FIG. 39 is a flowchart according to the eighth embodiment.[FIG. 40]FN202304898FIG. 40 is a functional block diagram according to a ninth embodiment. [FIG. 41]FIG. 41 is a flowchart according to the ninth embodiment.[FIG. 42]FIG. 42 is a graph presenting the relationship between the emission distance of emitted light and the light level of light incident on a sensor according to the ninth embodiment.[FIG. 43]FIG. 43 is a flowchart according to a tenth embodiment.[FIG. 44]FIG. 44 is a functional block diagram according to an eleventh embodiment.[FIG. 45]FIG. 45 is a sequence diagram according to the eleventh embodiment.The accompanying drawings are intended to depict embodiments of the present disclosure and should not be interpreted to limit the scope thereof. The accompanying drawings are not to be considered as drawn to scale unless explicitly noted. Also, identical or similar reference numerals designate identical or similar components throughout the several views. [Description of Embodiments]

[0010] In describing embodiments illustrated in the drawings, specific terminology is employed for the sake of clarity. However, the disclosure of this specification is not intended to be limited to the specific terminology so selected and it is to be understood that each specific element includes all technical equivalents that have a similar function, operate in a similar manner, and achieve a similar result.Referring now to the drawings, embodiments of the present disclosure are described below.As used herein, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.Hereinafter, embodiments of an image-capturing apparatus, a distance-measuring apparatus, a distance-measuring system, and a distance-measuring method are described in detail referring to the accompanying drawings.

[0011] First EmbodimentFIG. l is a block diagram illustrating a hardware configuration of a distance-measuring apparatus 100 according to a first embodiment. The distance-measuring apparatus 100 that also functions as an image-capturing apparatus measures the distance from the distancemeasuring apparatus 100 to an object. The distance-measuring apparatus 100 is a time of flight (ToF) camera that calculates the distance to an object based on the time from when light is emitted to when reflected light is received.As illustrated in FIG. 1, the distance-measuring apparatus 100 includes two light emitters 1-1 and 1-2, two light receivers 2-1 and 2-2, two analog-digital converters (ADCs) 3-1 and 3-2, and a distance-measuring controller 4 serving as a controller.FN202304898

[0012] The light emitter 1-1 serving as a first light emitter may include, for example, a light source 1- la such as a vertical cavity surface emitting laser (VCSEL), and a light-emitting optical system 1 - lb such as a lens. The light emitter 1-1 emits a laser beam from the light source 1- la such as a VCSEL over a wide range via the light-emitting optical system 1 - lb such as a wide-angle lens or a fisheye lens. The light emitter 1-1 is not limited to the combination of the laser and the wide-angle lens, and may use a light emitting diode (LED) or the like as the light source 1-la, and a diffraction optical element (DOE), a diffusion plate, or the like as the light-emitting optical system 1 -lb as long as the light emitter 1-1 can emit light to an object. The light emitter 1-1 diffuses light to obtain substantially uniform luminance within an emission range. That is, the light emitted from the light emitter 1-1 is diffused light. The diffused light is an example of first light.

[0013] The light emitter 1-2 serving as a second light emitter may include, for example, a light source l-2a such as a VCSEL, and a light-emitting optical system l-2b such as a lens, a DOE, a collimator, or a microlens array (MLA). With such a configuration, the light emitter 1-2 emits multiple spots of light (spot light). In other words, the spotlights emitted from the light emitter 1-2 are pattern light in a spot shape. The spot light is an example of second light. The second light has an intensity distribution in a range (space) where the second light is projected. The second light includes light such as structured light or focused light.

[0014] Spot light is described below. In an actual situation, spot light does not necessarily have a contrast difference of "100:0" between spots of reflected light of spot light received by the light receiver 2-1 or 2-2 and an area other than the spots due to divergence of light by a certain degree or a small amount of multipath interference. In the following description, spot light is defined using an example of a general method of determining the beam diameter of light.

[0015] FIGs. 2A and 2B are diagrams exemplarily presenting spot light. FIG. 2A is a diagram illustrating an emission state of spot light. FIG. 2B is a diagram illustrating a luminance value of part of the spot light illustrated in FIG. 2A.There is a way of thinking to define a range of l / e2 times the luminance value at the peak as the beam diameter of light. To distinguish the beam area from the other area, light is emitted such that the luminance value at the valley is substantially l / e2 times the luminance value at the peak or less, that is, about 13.5% or less of the luminance value at the peak, as indicated by the luminance value (FIG. 2B) of the section of part of the spot light indicated by a line "a" in FIG. 2A. Spot light and the other area can be clearly distinguished from each other when the luminance value at the valley is substantially l / e2 times the luminance value at the peak or less, that is, about 13.5% or less of the luminance value at the peak.

[0016] FN202304898The definition of spot light has been described above using the method of determining the beam diameter as an example; however, the numerals do not have the inventive meaning, and any method may be used as long as a contrast difference to the extent that a spot can be distinguished from an area other than the spot is obtained. The light emitted from the light sources 1-la and l-2a has a wavelength of, for example, 850 nm or 940 nm. The wavelength of the light source 1-la and the wavelength of the light source l-2a may be the same as or different from each other. The first light preferably has substantially uniform brightness in the projection range, but may be light having a smaller intensity distribution in the projection range than the second light projected from the second light emitter.

[0017] In the present embodiment, the light emitter 1-2 that emits multiple spots of light has been described as an example; however, the light emitter 1-2 is not limited thereto, and may be a light emitter 1-2 that emits any patterned light, such as random dot pattern light with an irregular dot arrangement or stripe pattern light.

[0018] The light receivers 2-1 and 2-2 may include image sensors 2- la and 2-2a, and light-receiving optical systems 2-lb and 2-2b such as lenses. The image sensors 2-1 a and 2-2a are so-called ToF sensors. The image sensors 2-la and 2-2a receive light that is emitted from the light source 1-la or the light source l-2a to an object and that is reflected by the object or the like. More specifically, the light receiver 2-1 is a first light receiver that receives reflected light of diffused light emitted from the light emitter 1-1 to the object. The light receiver 2-2 is a second light receiver that receives reflected light of spot light emitted from the light emitter 1- 2 to the object. Although details will be described later, the image sensors 2-la and 2-2a each divide an electric signal corresponding to the light intensity of the received reflected light into multiple phase signals and acquire the phase signals on the pixel basis.

[0019] The ADCs 3-1 and 3-2 each convert the phase signals acquired on the pixel basis from an analog signal into digital data, and supplies the digital data to the distance-measuring controller 4.

[0020] The distance-measuring controller 4 includes sensor interfaces (I / Fs) 41-1 and 41-2, lightsource drive circuits 42-1 and 42-2, an input-output I / F 43, a central processing unit (CPU) 44, a read only memory (ROM) 45, a random access memory (RAM) 46, and a solid state drive (SSD) 47. The sensor I / Fs 41-1 and 41-2, the light-source drive circuits 42-1 and 42-2, the input-output I / F 43, the CPU 44, the ROM 45, the RAM 46, and the SSD 47 are electrically connected to one another via a system bus 48.

[0021] The sensor I / F 41-1 is an interface that acquires the phase signals from the image sensor 2-la. The sensor I / F 41-2 is an interface that acquires the phase signals from the image sensor 2 -2a.

[0022] FN202304898The input-output I / F 43 is an interface for connection with an external device, such as a main controller device or a personal computer device.

[0023] The light-source drive circuit 42-1 supplies a drive signal with a predetermined voltage waveform and a predetermined emission frequency to the light source 1-la based on a control signal supplied from the CPU 44 to temporally modulate (temporally control) emission of light from the light source 1-la. The light-source drive circuit 42-2 supplies a drive signal with a predetermined voltage waveform and a predetermined emission frequency to the light source l-2a based on a control signal supplied from the CPU 44 to temporally modulate (temporally control) emission of light from the light source l-2a. The drive signals supplied to the light sources 1-la and l-2a may use rectangular waves, sine waves, or voltage waveforms having a predetermined waveform shape. The light-source drive circuits 42-1 and 42-2 change the frequencies of the voltage waveforms to control modulation of the frequencies of the drive signals. The light-source drive circuits 42-1 and 42-2 can simultaneously control emission of some of multiple light-emitting units included in each of the light sources 1-la and l-2a, or can change the light-emitting unit that emits light.

[0024] The ROM 45 is a non-volatile semiconductor memory (storage device) that can hold a program or data although the power is turned off. The ROM 45 stores programs or data of, for example, settings of a basic input / output system (BIOS) and an operating system (OS) that are executed when the CPU 44 is activated. The RAM 46 is a volatile semiconductor memory (storage device) that temporarily holds a program or data.

[0025] The SSD 47 is a non-volatile memory that stores a program to execute processing by the distance-measuring controller 4 or various data. For example, the SSD 47 stores a distancemeasuring and image-capturing program. Although details will be described later, the CPU 44 executes the distance-measuring and image-capturing program to control the image sensors 2- la and 2-2a to divide an electric signal corresponding to the light intensity of received reflected light into multiple phase signals and acquire the phase signals on the pixel basis. In one example, another storage device such as a hard disk drive (HDD) may be used instead of the SSD 47.

[0026] The CPU 44 loads a program or data from a storage device, such as the ROM 45 or the SSD 47, into the RAM 46 and executes processing to control the entire distance-measuring controller 4. In one example, part or all of the functions of the CPU 44 may be implemented by an electronic circuit, such as an application specific integrated circuit (ASIC) or a field- programmable gate array (FPGA).

[0027] The principle of distance measurement using a typical ToF camera is described below.

[0028] FN202304898Operation of Acquiring Phase SignalEach of the image sensors 2- la and 2-2a includes, for example, two charge storage units (a first storage unit and a second storage unit) for one light-receiving element. The storage unit that stores a charge can be switched at high speed. Thus, two inverted phase signals can be simultaneously detected for one rectangular wave. For example, a phase signal at 0 degrees and a phase signal at 180 degrees can be simultaneously detected. A phase signal at 90 degrees and a phase signal at 270 degrees can be simultaneously detected. Thus, distance measurement can be performed by performing a light-emitting and light-receiving process two times.

[0029] FIG. 3 is a timing chart presenting the principle of distance measurement. FIG. 3(a) presents a timing of emission of light. FIG. 3(b) presents a timing of reception of reflected light obtained through emission of light. FIG. 3(c) presents a timing at which a phase signal in a phase at 0 degrees is stored in the first storage unit, and FIG. 3(d) presents a timing at which a phase signal in a phase at 180 degrees is stored in the second storage unit among the two storage units included in the image sensor 2-1 a or 2 -2a. FIG. 3(e) presents a timing at which a phase signal in a phase at 90 degrees is stored in the first storage unit, and FIG. 3(f) presents a timing at which a phase signal in a phase at 270 degrees is stored in the second storage unit among the two storage units included in the image sensor 2- la or 2 -2a.

[0030] In a period indicated by oblique lines in FIG. 3(c) to FIG. 3(f), a charge of the phase signal in each phase is stored in the first storage unit or the second storage unit. Specifically, as the charge of the phase signal in the phase at 0 degrees, a charge between a pulse edge when the emission of light is ended and a pulse edge when the reception of reflected light is started is stored in the first storage unit as presented in FIG. 3(c). As the charge of the phase signal in the phase at 180 degrees, a charge between a pulse edge when the storage of the charge of the phase signal in the phase at 0 degrees is completed and a pulse edge when the reception of reflected light is ended is stored in the second storage unit as presented in FIG. 3(d).

[0031] Similarly, as the charge of the phase signal in the phase at 90 degrees, a charge between a pulse edge when the reception of reflected light is started and a pulse edge when storage of a charge of a pulse to perform charge-storing control is ended is stored in the first storage unit as presented in FIG. 3(e). As the charge of the phase signal in the phase at 270 degrees, a charge between a pulse edge when the storage of the charge of the phase signal in the phase at 90 degrees is completed and a pulse edge when the reception of reflected light is ended is stored in the second storage unit as presented in FIG. 3(f).

[0032] In an actual situation, to increase the amount of charge to be stored, light is emitted with a repetition pattern of rectangular waves instead of a one-time rectangular wave, and switchingFN202304898 control between the first and second storage units is repeatedly performed in accordance with the timing of emitting light with the repetition pattern.

[0033] Calculation of Distance ValueSignals (NO, N90, N180, N270) obtained in four phases (0 degrees, 90 degrees, 180 degrees, 270 degrees) are received and accumulated in four temporal phases of 0 degrees, 90 degrees, 180 degrees, and 270 degrees, respectively, with respect to the pulse period of the light to be emitted (emission light), and are phase signals based on the accumulated amount of charge. Thus, a phase difference angle (p can be obtained using the following equation.

[0034] (p = Arctan {(N90 - N270) / (N0 - N180)}

[0035] A delay time Td can be obtained from the phase difference angle (p using the following equation. In the following equations, fm indicates a modulation frequency.

[0036] Td = (p / 27tfm

[0037] A distance value D to an object can be obtained from the delay time Td using the following equation. light speed)The example in FIG. 3 is an example in which the phase signals at 0 degrees and 180 degrees are acquired in the measurement at the first time. In some embodiments, the charge amount of the second storage unit is subtracted from the charge amount of the first storage unit acquired in the measurement at the first time to generate a phase signal with the influence of external light reduced. With such measurement, one phase signal is acquired through one time of emission of light and exposure with light. Thus, four times of emission of light and exposure with light are performed to acquire phase signals for the four phases, and the imagecapturing time is twice that in a case without external light. However, the influence of external light can be reduced by the subtraction process.The signal processed by subtracting the charge amount of the second storage unit (Tap B) from the charge amount of the first storage unit (Tap A) acquired in the measurement at the first time by the above-described technique is referred to as a differential correlation sample (DCS) signal. The DCS signal is converted into a distance image. The conversion method is similar to the above-described method, and the phase difference angle (p is calculated using the following equation.(p = Arctan{(DCS90 - DCS270) / (DCS0 - DCS 180)}(p = Arctan[{(A90 - B90) - (A270 - B270)} / {(A0 - B0) - (Al 80 - Bl 80)}]FN202304898For example, DCS90 is a DCS signal at 90 degrees, A90 is the charge amount of the Tap A at 90 degrees, and B90 is the charge amount of the Tap B at 90 degrees. The delay time Td and the distance value D to the object can be obtained from the phase difference angle (p as in the method described above.

[0040] In the following description, it is assumed that the phase signal obtained through one time of emission of light and exposure with reflected light is a phase signal (the DCS signal) that is obtained by subtracting the charge amounts of the second storage unit from the charge amounts of the first storage unit.

[0041] The principle of generation of noise due to multipath interference with a typical ToF camera is described next.

[0042] FIG. 4 is a diagram illustrating the principle of generation of noise due to multipath interference with a ToF camera. As illustrated in FIG. 4, light from a light path TO, which includes one time of reflection (or direct reflection component), is received mixed with light from, for example, a light path TI that includes multiple reflections (or multipath component). The illumination is actually uniform. Light reflected many times is received in a manner mixed in the light path TO from which light is expected to be received. Since distance information (distance measurement value) indicated by a solid line in FIG. 4 is mixed with distance information (distance measurement value) indicated by a dotted line in FIG. 4, a value farther than the actual distance is calculated. For example, such a phenomenon is likely to occur in a scene in which light in multiple light paths is reflected and received at a comer of the room. That is, although a depth value with high spatial resolution can be acquired with diffused light, an error due to multipath interference increases.

[0043] Thus, the distance-measuring apparatus 100 according to the present embodiment performs distance correction using distance measurement results obtained using the multiple different light emitters 1-1 and 1-2 to reduce the error due to multipath interference.

[0044] Functions of the distance-measuring controller 4 are described below.

[0045] FIG. 5 is a functional block diagram illustrating a functional configuration of the distancemeasuring controller 4. As illustrated in FIG. 5, the CPU 44 of the distance-measuring controller 4 executes the distance-measuring and image-capturing program stored in the SSD 47 to implement functions of an imaging controller 401, an image storage unit 402, a distance calculation unit 403, a correction- value calculation unit 404, a distance correction unit 405, and an output unit 406.

[0046] FN202304898The imaging controller 401 captures phase images of multiple phases, and controls the image sensors 2- la and 2-2a to store the charges of the phase images in the storage units for the phase images. The imaging controller 401 controls emission of light from the light source 1- la via the light-source drive circuit 42-1. The imaging controller 401 controls emission of light from the light source l-2a via the light-source drive circuit 42-2.

[0047] The image storage unit 402 stores phase signals (phase images) in respective phases acquired by the image sensors 2- la and 2-2a and output from the ADCs 3-1 and 3-2 in a storage unit such as the RAM 46 or controls reading of phase signals from the storage unit.

[0048] The distance calculation unit 403 calculates distance information indicating the distance to an object based on the multiple phase images stored by the image storage unit 402.

[0049] The correction-value calculation unit 404 calculates a correction value to correct the distance information using a distance image obtained by the distance calculation unit 403 through emission of light from the light source 1-la and a distance image obtained by the distance calculation unit 403 through emission of light from the light source l-2a.

[0050] The distance correction unit 405 corrects the distance information obtained through emission of light from the light source 1-la using the correction value calculated by the correctionvalue calculation unit 404.

[0051] The output unit 406 outputs distance information indicating the distance to the object and corrected by the distance correction unit 405 to an external device via the input-output I / F 43.

[0052] The functions of the imaging controller 401, the image storage unit 402, the distance calculation unit 403, the correction-value calculation unit 404, the distance correction unit 405, and the output unit 406 illustrated in FIG. 5 each are implemented by software using the distance-measuring and image-capturing program. However, all or part of the functions may be implemented by hardware such as an integrated circuit (IC).

[0053] Features of the distance-measuring apparatus 100 according to the present embodiment are described below.

[0054] FIG. 6 is a view illustrating a light-emitting system and a light-receiving system of the distance-measuring apparatus 100. As illustrated in FIG. 6, the distance-measuring apparatus 100 includes the light emitter 1-1 that serves as diffused illumination to emit diffused light, the light receiver 2-1 that receives the diffused light emitted from the light emitter 1-1, the light emitter 1-2 that serves as spot illumination to emit spot light, and the light receiver 2-2 that receives the spot light emitted from the light emitter 1-2. That is, the distance-measuringFN202304898 apparatus 100 is a ToF camera including the light receivers 2-1 and 2-2 including respective ToF sensors for the light emitters 1-1 and 1-2. Emission ranges of the light emitters 1-1 and 1-2 are set to include a measurement object area of the distance-measuring apparatus 100. Reception ranges of the light receivers 2-1 and 2-2 are set to receive light from the measurement object area. The correction value is not calculated even when the distance measurement value is obtained in an area in which light is emitted from one of the light emitters. Thus, the emission range of the light emitter 1-1 and the emission range of the light emitter 1-2 are desirably set to substantially overlap each other.

[0055] FIG. 7 is a view illustrating an installation example of the distance-measuring apparatus 100. As illustrated in FIG. 7, the measurement object area of the distance-measuring apparatus 100 includes a comer of the room. Hereinafter, details are described using an example in which the distance-measuring apparatus 100 captures an image of a comer of the room.

[0056] An influence of multipath interference is described below.

[0057] FIG. 8 is a view exemplarily illustrating an influence of multipath interference. FIG. 8 illustrates, in a view from above, distance measurement values (ranging points) at the corner of the room. As illustrated in FIG. 8, a distance measurement value of diffused light obtained from the light source 1-la that serves as diffused illumination can be acquired continuously with high spatial resolution because, in principle, all pixels can serve as ranging points. However, an error in a distance measurement value of diffused light obtained from the light source 1-la that serves as diffused illumination increases because of the influence of multipath interference. Specifically, as described above, the distance measurement value (ranging point) of diffused light obtained from the light source 1-la that serves as diffused illumination is calculated to be farther than the actual position. In contrast, as illustrated in FIG. 8, a distance measurement value of spot light obtained from the light source l-2a that serves as spot illumination is relatively correct although the distance measurement value is discontinuous because the number of ranging points is small.

[0058] The correction-value calculation unit 404 of the distance-measuring apparatus 100 according to the present embodiment corrects distance information obtained by emitting light from the light emitter 1-1 that serves as diffused illumination, by a technique described below. Thus, distance information with further high accuracy based on the distance measurement value using spot illumination can be output for a larger number of points than the number of ranging points obtained using spot illumination.

[0059] FIG. 9 is a view exemplarily illustrating a technique of calculating a correction value. As illustrated in FIG. 9, the correction-value calculation unit 404 calculates a correction value (correction amount) by subtracting a distance measurement value of spot light from a distanceFN202304898 measurement value of diffused light. The correction value is not calculated for a pixel between spots by the above-described technique. Thus, the correction- value calculation unit 404 performs an interpolation process for the pixel between the spots to calculate a correction value (correction amount) for the pixel in which a distance measurement value (distance information) that is a distance measurement result has not been obtained in a distance image of spot light. The calculation of the correction value (correction amount) using image processing performed by the correction-value calculation unit 404 is described below.

[0060] For example, it is assumed that ddiffosion represents a distance measurement value in a distance image of diffused light, and dspot represents a distance measurement value in a distance image of spot light.

[0061] The correction-value calculation unit 404 calculates a correction value c(x, y) of a pixel at a position (x, y) on an image using Equation (1) below. The correction-value calculation unit 404 performs calculation for x and y in Equation (1) using a pixel value for which a distance measurement value of spot light has been obtained.

[0062] [Math. 1]

[0063] The correction-value calculation unit 404 interpolates a correction value at a pixel position (x1, y') at which a distance measurement value of spot light has not been obtained, using Equation (2) below. Values XN and yN are the coordinates of a pixel for which a correction value closest to the pixel position (x1, y') has been calculated (the coordinates of a pixel in which a distance measurement value of spot light has been acquired).

[0064] [Math. 2]

[0065] Then, the distance correction unit 405 corrects the distance measurement value obtained through emission of light from the light emitter 1-1, using the correction value calculated by the correction- value calculation unit 404. Specifically, the distance correction unit 405 subtracts the correction value from the distance measurement value ddiffusion of diffused light to calculate a distance value (distance information) dcorrection after correction as indicated in Equation (3) below. Since the correction value of the pixel in which the distance measurement value of spot light has not been obtained is interpolated by the interpolation process, the distance correction unit 405 can execute correction on all pixel positions in the image for x and y in Equation (3).FN202304898

[0066] [Math. 3]

[0067] The above-described interpolation method is simple nearest-neighbor interpolation; however, the interpolation method may be any method. For example, linear interpolation may be performed using multiple correction values.

[0068] The correction process may be performed on all pixels in an image, or may be performed on some pixels in an image. For example, in an area in which the reliability of a distance measurement value of spot light is low such as an area in which the light intensity of reflected light is low, or in an area in which there is no large difference between a distance measurement value of diffused light and a distance measurement value of spot light, the distance measurement value of diffused light may be directly used without performing the correction. In one example, the interpolation process may be omitted in an area not subjected to correction to increase the speed of the processing.

[0069] FIG. 10 is a flowchart presenting a flow of a distance-measuring process performed by the distance-measuring apparatus 100. As presented in FIG. 10, the imaging controller 401 controls the light source 1-1 a and the image sensor 2- la to cause the image sensor 2- la to receive diffused light and capture an image (step SI). The image storage unit 402 stores a phase signal (phase image) in each phase from the image sensor 2- la in a storage unit such as the RAM 46 (step S2).

[0070] The distance calculation unit 403 calculates distance information indicating the distance to an object based on multiple phase images stored by the image storage unit 402 (step S3).

[0071] The imaging controller 401 controls the light source l-2a and the image sensor 2-2a to cause the image sensor 2-2a to receive spot light and capture an image (step S4). The image storage unit 402 stores a phase signal (phase image) in each phase from the image sensor 2-2a in a storage unit such as the RAM 46 (step S5).

[0072] That is, the distance-measuring apparatus 100 emits diffused light and spot light at different timings to capture images, and acquires two distance images.

[0073] In the present embodiment, the distance images are acquired in the order of the diffused light and the spot light; however, the order is not limited thereto, and the order may be any order as long as the image-capturing timings are different.

[0074] FN202304898The distance calculation unit 403 calculates distance information indicating the distance to the object based on the multiple phase images stored by the image storage unit 402 (step S6).

[0075] FIGs. 11 A and 1 IB are views illustrating examples of distance images. FIG. 11 A illustrates an example of a distance image obtained through emission of diffused light. FIG. 11B illustrates an example of a distance image obtained through emission of spot light. As illustrated in FIG. 11 A, since the diffused light is uniformly emitted on the entire surface of the area, an image to be acquired is substantially filled with pixels to which the light returns. The image refers to an image that corresponds to the area where the object measurable for distance is present. In contrast, as illustrated in FIG. 11B, since spot light is focused, only the pixels corresponding to the illuminated position detect the direct reflection component, allowing for distance calculation. That is, as illustrated in FIG. 11B, distance information is obtained only at a spot portion of spot light.

[0076] The correction-value calculation unit 404 calculates a correction value to correct the distance information using the distance image obtained through emission of light from the light source1-la and the distance image obtained through emission of light from the light source l-2a (step S7). Specifically, the correction- value calculation unit 404 uses a residual (the difference between a distance measurement value of diffused light and a distance measurement value of spot light) at a position at which a spot has been obtained in the distance image obtained through emission of light from the light source l-2a as a correction value, and interpolates a position at which a spot is not obtained by general image processing to calculate a correction value of the entire image.

[0077] As illustrated in FIG. 6, in the light-emitting system and the light-receiving system of the distance-measuring apparatus 100 according to the present embodiment, the light receiver 2-1 and the light receiver 2-2 are disposed to be spaced apart from each other. When the light receiver 2-1 and the light receiver 2-2 are disposed to be spaced apart from each other, position alignment is performed on distance information on one of the light receivers 2-1 and2-2 based on calibration data acquired in advance, and hence correction can be performed by processes similar to Equations (1) to (3) described above. The calibration data is obtained in advance using a calibration chart or the like to obtain camera parameters of the image sensors 2- la and 2-2a of the light receivers 2-1 and 2-2 and rotational and translational amounts between the image sensors 2- la and 2-2a. Using such calibration data can calculate where the ranging point reflected in the image sensor 2-la is reflected in the image sensor 2 -2a. *DEL*

[0078] The distance correction unit 405 corrects the distance information obtained through emission of light from the light source 1-la using the correction value calculated by the correctionvalue calculation unit 404 (step S8). Specifically, the distance correction unit 405 subtracts the correction value calculated by the correction- value calculation unit 404 from the distanceFN202304898 measurement value of diffused light obtained through emission of light from the light source 1-la to perform correction.

[0079] In one example, the distance correction unit 405 may omit the correction when the difference or ratio between the distance measurement value of diffused light and the distance measurement value of spot light calculated by the correction-value calculation unit 404 is within a predetermined range.

[0080] The output unit 406 outputs the distance information indicating the distance to the object and corrected by the distance correction unit 405 to an external device via the input-output I / F 43 (step S9).

[0081] As described above, according to the present embodiment, since the distance correction is performed using the distance measurement results obtained using the multiple different light emitters, the error due to multipath interference can be reduced, and hence a distance image with further high accuracy can be generated. That is, according to the present embodiment, since a multipath error component of an image with high spatial resolution is eliminated, distance measurement with higher accuracy and higher resolution than those of normal ToF distance measurement can be performed. Thus, it is possible to provide an image-capturing apparatus, a distance-measuring apparatus, a distance-measuring system, and a distancemeasuring method that can capture an image from which a distance image with further high accuracy can be generated.

[0082] According to the present embodiment, when distance measurement is performed using spot light, a correct distance measurement value can be obtained as compared to a typical ToF camera although the pixel density is low. When light with a light intensity for distance measurement is emitted at a spot position, the total light intensity can be lower than that of diffused illumination when light is emitted to obtain the same light intensity at the same spot position. Since the total light intensity of emitted light is less than that of a typical ToF camera, the influence of multipath interference can be reduced.

[0083] According to the present embodiment, since the correction value in the pixel in which the distance measurement value of spot light has not been obtained is interpolated based on the correction value calculated from the distance measurement value obtained using diffused light and the distance measurement value obtained using spot light, it is possible to perform correction with further high accuracy.

[0084] While the distance-measuring apparatus 100 performs the distance calculation and the distance correction according to the present embodiment, what performs the distance calculation and the distance correction is not limited to the distance-measuring apparatus 100.FN202304898FIG. 12 is a block diagram illustrating a system configuration of a distance-measuring system 300 including a distance-measuring apparatus (image-capturing apparatus) 500. As illustrated in FIG. 12, the distance-measuring system 300 has a system configuration in which the distance-measuring apparatus (image-capturing apparatus) 500 is connected to an external system 400 such as a personal computer (PC) or a cloud. With the distance-measuring system 300 illustrated in FIG. 12, the distance-measuring apparatus (image-capturing apparatus) 500 may be an image-capturing apparatus that performs ToF image-capturing, and processes such as the distance calculation and correction that are performed by the distance-measuring controller 4 of the distance-measuring apparatus (image-capturing apparatus) 500 may be performed by the external system 400 such as a PC or a cloud. That is, the external system 400 functions as a controller that corrects distance information calculated from a detection signal of first reflected light of light emitted from a first light emitter to an object, using a detection signal of second reflected light of light emitted from a second light emitter to the object.

[0085] While the distance-measuring apparatus 100 includes one spot light emitter (light emitter 1-2) for one diffused light emitter (light emitter 1-1) according to the present embodiment, the number of spot light emitters is not limited to one and may be plural. That is, multiple spot light emitters that emit spot light to an equivalent measurement object area may be provided. In this case, spot positions of the multiple spot light emitters may be disposed to be shifted from one another so that spot light is emitted at positions different from one another. In one example, one spot light emitter may be provided and the emission position of the light source or the position of the light-emitting optical system may be moved to shift the spot position during emission of light.

[0086] Second EmbodimentNext, a second embodiment will be described.

[0087] The second embodiment differs from the first embodiment in that the second embodiment includes one light receiver, one ADC, and one sensor I / F. Description of part of the second embodiment similar to that of the first embodiment will be omitted, and part of the second embodiment different from that of the first embodiment will be described.

[0088] In the distance-measuring apparatus 100 according to the first embodiment, calibration data is obtained in advance. In the distance-measuring apparatus 100 according to the first embodiment, since the light receiver 2-1 and the light receiver 2-2 are spaced apart from each other, the influence of occlusion may remain even when position alignment is performed (when one of the image sensors 2- la and 2-2a can view an object but the other one cannot view the object, disadvantages such as omission may occur or a distance of an invisible areaFN202304898 may remain when the position alignment is performed). The housing of the distancemeasuring apparatus 100 according to the first embodiment increases in size.

[0089] FIG. 13 is a block diagram illustrating a hardware configuration of a distance-measuring apparatus 100 according to the second embodiment. FIG. 14 is a view illustrating a lightemitting system and a light-receiving system of the distance-measuring apparatus 100. As illustrated in FIGs. 13 and 14, the distance-measuring apparatus 100 according to the present embodiment differs from the distance-measuring apparatus 100 according to the first embodiment in that the distance-measuring apparatus 100 includes one light receiver 2 (an image sensor 2a and a light-receiving optical system 2bb), one ADC 3, and one sensor I / F 41.

[0090] Also in the distance-measuring apparatus 100 according to the second embodiment, distance information obtained through emission of light from the light emitter 1-1 that serves as diffused illumination is corrected, as in the first embodiment.

[0091] As described above, the distance-measuring apparatus 100 according to the present embodiment includes the one light receiver 2 and the two different light emitters 1-1 and 1-2. The different light emitters 1-1 and 1-2 emit light at different timings to measure a distance. The two light emitters 1-1 and 1-2 respectively emit diffused light and spot light, and hence distance images having two different characteristics are acquired. A correction value to correct the result with diffused light is calculated from the two kinds of distance images, and correction is performed. By performing the distance correction using the distance measurement results obtained using the multiple different light emitters 1-1 and 1-2 in this way, an error due to multipath interference can be reduced, thereby providing an imagecapturing apparatus, a distance-measuring apparatus, and a distance-measuring method that can capture an image from which a distance image with further high accuracy can be generated.

[0092] With the distance-measuring apparatus 100 according to the present embodiment, the one light receiver 2 is used in common and hence there is no optical difference at a position at which the distance can be measured. Thus, when two kinds of images are processed, a correction value of each pixel can be calculated by simple image processing.

[0093] The distance-measuring apparatus 100 according to the present embodiment has an advantageous effect that the housing can be smaller than the housing in a case where two ToF cameras are arranged side by side.

[0094] In the distance-measuring apparatus 100 according to the present embodiment, as illustrated in FIG. 14, for example, the optical elements (the one light receiver 2, and the two different light emitters 1-1 and 1-2) are fixedly provided on one surface; however, the arrangement isFN202304898 not limited thereto. FIG. 15 illustrates another arrangement example of the distancemeasuring apparatus 100. As illustrated in FIG. 15, the distance-measuring apparatus 100 may include a tool 200a such as a tripod that functions as a support unit, and rotation means 220 between the distance-measuring apparatus 100 and the tool 200a. The rotation means 220 is a rotation table that is rotated by, for example, an electric motor. In this case, the distancemeasuring apparatus 100 acquires a distance image of the entire circumference while rotating itself using the rotation means 220 to change the image-capturing direction.

[0095] A distance-measuring controller 4 of the distance-measuring apparatus 100 has a predetermined rotation pattern when controlling the rotation of the rotation means 220. For example, the distance-measuring controller 4 of the distance-measuring apparatus 100 performs control in a rotation pattern of "rotate by a predetermined distance", "stop rotation and capture an image", "rotate by a predetermined distance", and "stop rotation and capture an image" in that order.

[0096] Third EmbodimentNext, a third embodiment will be described.

[0097] The third embodiment differs from the second embodiment in that light emitters are disposed symmetrically with respect to a light receiver. Description of part of the third embodiment similar to that of the second embodiment will be omitted, and part of the third embodiment different from that of the second embodiment will be described.

[0098] FIG. 16 is a view illustrating a light-emitting system and a light-receiving system of a distance-measuring apparatus 100 according to the third embodiment. As illustrated in FIG. 16, in the distance-measuring apparatus 100 according to the present embodiment, light emitters 1-1 and 1-2 serving as the light-emitting system are disposed symmetrically with respect to a light receiver 2 serving as the light-receiving system.

[0099] FIGs. 17A and 17B are views illustrating examples of distance images. FIG. 17A illustrates an example of a distance image obtained through emission of diffused light. FIG. 17B illustrates an example of a distance image obtained through emission of spot light. The distance measurement result with diffused light in the arrangement illustrated in FIG. 16 is obtained from the surface that receives light as illustrated in FIG. 17 A. In contrast, the distance measurement result with spot light in the arrangement illustrated in FIG. 16 is obtained from the left and right surfaces as illustrated in FIG. 17B.

[0100] As described above, when both the result with diffused light and the result with spot light are obtained, the correction process for multipath interference described in the second embodiment is performed. For a position at which a distance measurement value is obtainedFN202304898 with spot light (a black area illustrated in FIG. 17 A), interpolation is performed by image processing using the distance measurement result with spot light to obtain distance measurement values for all pixels.

[0101] As described above, the distance-measuring apparatus 100 according to the present embodiment includes the one light receiver 2 and the two different light emitters 1-1 and 1-2. The different light emitters 1-1 and 1-2 emit light at different timings to measure a distance. The two light emitters 1-1 and 1-2 respectively emit diffused light and spot light, and hence distance images having two different characteristics are acquired. A correction value to correct the result with diffused light is calculated from the two kinds of distance images, and correction is performed. By performing the distance correction using the distance measurement results obtained using the multiple different light emitters 1-1 and 1-2 in this way, an error due to multipath interference can be reduced, thereby providing an imagecapturing apparatus, a distance-measuring apparatus, and a distance-measuring method that can capture an image from which a distance image with further high accuracy can be generated.

[0102] With the distance-measuring apparatus 100 according to the present embodiment, the one light receiver 2 is used in common and hence there is no optical difference at a position at which the distance can be measured. Thus, when two kinds of images are processed, a correction value of each pixel can be calculated by simple image processing.

[0103] The distance-measuring apparatus 100 according to the present embodiment has an advantageous effect that the housing can be smaller than the housing in a case where two ToF cameras are arranged side by side.

[0104] Fourth EmbodimentNext, a fourth embodiment will be described.

[0105] The fourth embodiment differs from any one of the first embodiment to the third embodiment in that control information for capturing an image with spot light is changed based on an image-capturing result with diffused light. Description of part of the fourth embodiment similar to that of any one of the first embodiment to the third embodiment will be omitted, and part of the fourth embodiment different from that of any one of the first embodiment to the third embodiment will be described.

[0106] When the light intensity exceeds the maximum light-receiving capacity of the image sensor during image-capturing, the pixel that receives light with the light intensity exceeding the maximum light-receiving capacity becomes a saturated pixel, and data is not obtained. In theFN202304898 case of a ToF camera, a saturated pixel is generated when the distance to an object of imagecapturing is short or when the light intensity of light to be emitted is too high.

[0107] FIGs. 18A and 18B are views exemplarily illustrating saturated pixels in a ToF camera according to the fourth embodiment. FIG. 18A illustrates an example of saturated pixels in a distance image obtained through emission of diffused light. FIG. 18B illustrates an example of saturated pixels in a distance image obtained through emission of spot light.As illustrated in FIG. 18 A, when the ToF camera receives diffused light, a saturated pixel area b may be generated. When the saturated pixel area b is generated in this way, the result at a position included in the saturated pixel area b and receiving spot light is no longer obtained as illustrated in FIG. 18B.Thus, an image-capturing method to reduce saturated pixels is desired.

[0108] FIG. 19 is a flowchart presenting a flow of an image-capturing process to reduce saturated pixels. In this case, description is given based on that the distance-measuring apparatus 100 includes one light receiver 2 and two different light emitters 1-1 and 1-2.

[0109] As presented in FIG. 19, an imaging controller 401 of the distance-measuring apparatus 100 captures an image with diffused light first (step S101).

[0110] The imaging controller 401 of the distance-measuring apparatus 100 counts the number of pixels (saturated pixels) that receive light with intensities exceeding the maximum lightreceiving capacity of the light receiver 2 based on the image-capturing result with diffused light emitted first (step SI 02).[OHl]The imaging controller 401 of the distance-measuring apparatus 100 determines whether the counted number of saturated pixels is larger than a threshold value (step SI 03).

[0112] When the imaging controller 401 of the distance-measuring apparatus 100 determines that the counted number of saturated pixels is larger than the threshold value (Yes in step SI 03), the imaging controller 401 lowers light intensity setting during image-capturing with spot light (step SI 04), and then captures an image with spot light (step SI 05).

[0113] In contrast, when the imaging controller 401 of the distance-measuring apparatus 100 determines that the counted number of saturated pixels is not larger than the threshold value (No in step SI 03), the imaging controller 401 captures an image with spot light without a change in the light intensity setting (step SI 05).

[0114] As described above, the imaging controller 401 of the distance-measuring apparatus 100 fixes the image-capturing order and changes the control information on image-capturing with spotFN202304898 light based on the image-capturing result with diffused light emitted first. Thus, the result with spot light is obtained although saturation occurs with diffused light.

[0115] FIGs. 20A and 20B are views illustrating examples of images acquired when control on spot light is switched in accordance with the result with diffused light. As illustrated in FIGs. 20A and 20B, a distance value with high accuracy can be acquired by reducing multipath interference using Equations (1) to (3) as described above in an area in which both the results with diffused light and spot light are obtained. In contrast, as illustrated in FIGs. 20A and 20B, distance information with a less omission can be acquired by performing interpolation on an image based on the result with spot light in an area in which the saturated pixel area b is generated and the result with diffused light has not been obtained.

[0116] In step SI 04, the light intensity setting during image-capturing with spot light is lowered; however, the setting is not limited thereto. The exposure time of the light receiver 2 may be decreased. A similar advantageous effect can be obtained by decreasing the exposure time of the light receiver 2.

[0117] As described above, according to the present embodiment, even when the saturated pixel area is generated in the image sensor, distance information with a less omission can be acquired.

[0118] The distance-measuring apparatus 100 has been described as including the one light receiver 2; however, the configuration is not limited thereto, and may include two different light receivers 2-1 and 2-2 corresponding to the two different light emitters 1-1 and 1-2.

[0119] Fifth EmbodimentNext, a fifth embodiment will be described.

[0120] The fifth embodiment differs from any one of the first embodiment to the fourth embodiment in that a spherical image-capturing apparatus is applied as a distance-measuring apparatus 100. Description of part of the fifth embodiment similar to that of any one of the first embodiment to the fourth embodiment will be omitted, and part of the fifth embodiment different from that of any one of the first embodiment to the fourth embodiment will be described.

[0121] FIG. 21 is an external perspective view illustrating a configuration of the distance-measuring apparatus 100 according to the fifth embodiment. FIGs. 22 A and 22B are views illustrating an example of a schematic configuration of the distance-measuring apparatus 100. FIG. 23 is a view illustrating an example of arrangement of an optical system. In the present embodiment, the distance-measuring apparatus 100 has a function as a distance-measuring apparatus using a ToF system (ToF camera) and a function as a luminance camera (red-green-FN202304898 blue (RGB) camera), and captures an image using the ToF camera and the luminance camera for a spherical area.

[0122] As illustrated in FIGs. 21 to 23, the distance-measuring apparatus 100 includes a first lightemitting unit 20, a second light-emitting unit 30, a ToF light-receiving unit 60, a luminance light-receiving unit 80, and a controller 120. Each of a combination of the first light-emitting unit 20 and the ToF light-receiving unit 60 and a combination of the second light-emitting unit 30 and the ToF light-receiving unit 60 functions as a distance-measuring apparatus using a ToF system, that is, a ToF camera. The luminance light-receiving unit 80 functions as a luminance camera.

[0123] The first light-emitting unit 20 and the second light-emitting unit 30 each emit distance measurement light (infrared light or the like) to a measurement object area. The first lightemitting unit 20 includes a light source 210 that emits infrared light and a ToF light-emitting system 211 (light-emitting optical system) including an optical element that widens the angle of divergence, and emits the light of the light source 210 at a wide angle. The second lightemitting unit 30 includes a light source 310 that emits infrared light and a ToF light-emitting system 311 (light-emitting optical system) including an optical element that widens the angle of divergence, and emits the light of the light source 310 at a wide angle. The optical element of each of the ToF light-emitting system 211 and the ToF light-emitting system 311 includes, for example, a lens, a DOE, or a diffusion plate. The light source 210 and the light source 310 are, for example, two-dimensional arrays of VCSELs. The distance-measuring apparatus 100 according to the present embodiment includes two first light-emitting units 20 disposed to face in directions opposite to each other, and two second light-emitting units 30 disposed to face in directions opposite to each other.

[0124] The two first light-emitting units 20 serve as structured illumination that emits patterned light (in the present embodiment, dot pattern) that is an example of structured light in a space. The two second light-emitting units 30 serve as diffused illumination that emits diffused light with uniform illuminance in a space.

[0125] The distance measurement light emitted from the first light-emitting units 20 and the distance measurement light emitted from the second light-emitting units 30 are reflected by an object existing in a measurement object area. The ToF light-receiving unit 60 receives reflected light from the object in the measurement object area. The ToF light-receiving unit 60 includes a ToF sensor 110a having sensitivity to distance measurement light, and a ToF light-receiving optical system 112 (first light-receiving optical system) including an optical element that guides incident light to the ToF sensor 110a. The optical element of the ToF light-receiving optical system 112 includes, for example, a lens.FN202304898The ToF sensor 110a is a light-receiving element in which light-receiving pixels are two- dimensionally arranged, and each pixel corresponds to a position in the measurement object area, so that the ToF light-receiving unit 60 can individually receive light from each position in the measurement object area. The distance-measuring apparatus 100 according to the present embodiment includes four ToF light-receiving units 60 disposed to face in directions different from one another.

[0126] The distance-measuring apparatus 100 according to the present embodiment further includes the two second light-emitting units 30 disposed to face in directions opposite to each other. The two second light-emitting units 30 emit diffused light with uniform illuminance to the entire surrounding area of a housing 11. While the distance-measuring apparatus 100 emits diffused light to the entire surrounding area using the second light-emitting units 30, in distance images obtained by the second light-emitting units 30, the illuminance of illumination is lower than that of the first light-emitting units 20 that focuses light, and hence the accuracy of distance measurement in a long range decreases. However, since the second light-emitting units 30 emit diffused light with uniform illuminance to the entire surrounding area of the housing 11, the distance-measuring apparatus 100 can use information on the distance and shape of a blank portion between dots of the dot pattern of light emitted from the first light-emitting units 20 as information for interpolation.

[0127] The luminance light-receiving unit 80 acquires a two-dimensional image using a complementary metal oxide semiconductor (CMOS) sensor 33. The luminance lightreceiving unit 80 includes the CMOS sensor 33 for capturing a luminance image (RGB image), and a luminance light-receiving optical system 113 (second light-receiving optical system) including an optical element that guides incident light to the CMOS sensor 33. The optical element of the luminance light-receiving optical system 113 includes, for example, a lens.

[0128] The distance-measuring apparatus 100 according to the present embodiment maps a luminance image (RGB image) obtained by the luminance light-receiving unit 80 onto a coordinate point group obtained from a distance image. Accordingly, the distance-measuring apparatus 100 can convert distance and shape information of a surrounding space into digital data with color information.

[0129] The controller 120 drives or controls the first light-emitting units 20, the ToF light-receiving units 60, the luminance light-receiving unit 80, and the second light-emitting units 30. The controller 120 is connected to each of the light sources 210, the ToF sensors 110a, the CMOS sensor 33, and the second light-emitting units 30 via a cable, a flexible printed circuit (FPC), a flexible flat cable (FFC), or the like.

[0130] FN202304898In this case, the first light-emitting units 20 are an example of a first light emitter that emits spot light (structured light). The ToF light-receiving units 60 are an example of a light receiver that receives incident light including emitted light. The luminance light-receiving unit 80 outputs information including at least luminance. The second light-emitting units 30 are an example of a second light emitter that emits diffused light.

[0131] In the present embodiment, as illustrated in FIGs. 21 to 23, the distance-measuring apparatus 100 has a long shape that is long in the Z-axis direction. In a first stage on the most +Z side of the distance-measuring apparatus 100, four ToF light-receiving optical systems 112 each having an angle of view of 120 degrees or more are disposed to face in three directions in an XY plane and in one direction that is the +Z side. In a second stage on the -Z side of the first stage of the distance-measuring apparatus 100, two ToF light-emitting systems 211 each having an angle of view of 180 degrees or more, and two luminance light-receiving optical systems 113 each having an angle of view of 180 degrees are disposed. The two ToF lightemitting systems 211 face in directions opposite to each other (+X side and -X side), and the two luminance light-receiving optical systems 113 face in directions opposite to each other (+Y side and -Y side). In a lower stage on the -Z side of the distance-measuring apparatus 100, the controller 120 and a battery 130 are disposed. Thus, the optical systems that cover the spherical area can be compactly disposed, and the distance-measuring apparatus can be downsized.

[0132] The controller 120 controls the timing at which the first light-emitting units (light emitters) 20 emit light, and detects the reception of light by the ToF light-receiving units (light receivers) 60. The controller 120 controls the timing of driving the light sources 210 to emit light to the measurement object area. The controller 120 photoelectrically converts the light received by the ToF sensors 110a and outputs the converted result as a distance image. Simultaneously, the controller 120 causes the CMOS sensor 33 to capture an image, and outputs a luminance image.

[0133] When a direct ToF sensor is used as each of the ToF sensors 110a, the controller 120 outputs a distance image based on the light-receiving timing in each pixel. In contrast, when an indirect ToF sensor is used as each of the ToF sensors 110a, the controller 120 outputs phase images based on the light reception level in each pixel in four different phases. The controller 120 can generate a distance image from the four phase images.

[0134] In a case where a spherical image-capturing apparatus is applied as the distance-measuring apparatus 100, when the second light-emitting units 30 emit diffused light and the first lightemitting units 20 are turned on, an image can be captured in a short time; however, the light emitted from the second light-emitting units 30 may enter the ToF light-receiving units 60 asFN202304898 light in multipath. Even in the case of spot light, a similar disadvantage arises when multiple first light-emitting units 20 simultaneously emit light.

[0135] Multipath interference specific to the case where the spherical image-capturing apparatus is applied as the distance-measuring apparatus 100 is described below.

[0136] FIG. 24 is a diagram exemplarily illustrating multipath interference specific to the case where the spherical image-capturing apparatus is applied. As illustrated in FIG. 24, in the ToF lightreceiving units 60, a one-dot chain line indicating direct reflection at a ranging point, a multipath (solid line) from one first light-emitting unit 20, and a multipath (dotted line) from another first light-emitting unit 20 are mixed. Thus, the distance-measuring apparatus 100 according to the present embodiment executes an image capturing sequence to reduce multipath interference.

[0137] FIG. 25 is a chart presenting an image capturing sequence to reduce multipath interference when the spherical image-capturing apparatus is applied. As illustrated in FIG. 25, the distance-measuring apparatus 100 according to the present embodiment performs emission of light and image-capturing by each of the first light-emitting units 20 and each of the ToF light-receiving units 60, and combines captured image data in post processing to obtain a distance measurement value in the entire area of the ToF light-receiving units 60.

[0138] Thus, the luminous output per image-capturing can be decreased, and the influence of multipath interference is reduced as compared to a case where multiple light sources are simultaneously turned on. By performing the processes using Equations (1) to (3) on images captured by the ToF light-receiving units 60, multipath interference can be reduced even when the spherical image-capturing apparatus is applied as the distance-measuring apparatus 100.

[0139] When the number of first light-emitting units 20 increases, the number of times of imagecapturing and the number of images to be combined increase by the amount by which the number of first light-emitting units 20 increases. When the number of the ToF light-receiving units 60 increases, the number of images to be combined increases similarly.

[0140] As described above, according to the present embodiment, multipath interference can be reduced even when the spherical image-capturing apparatus is applied as the distancemeasuring apparatus 100.

[0141] In the present embodiment, the first light-emitting units 20 that emit light with the dot pattern as an emission pattern have been described as an example; however, the first light-emitting units 20 are not limited thereto, and may be first light-emitting units 20 that emit anyFN202304898 patterned light, such as random dot pattern light with an irregular dot arrangement or stripe pattern light.

[0142] Sixth EmbodimentA sixth embodiment is described below.The sixth embodiment differs from the first embodiment in that the measurement range of spot light and the measurement range of diffused light are adjusted. Description of part of the sixth embodiment similar to that of the first embodiment will be omitted, and part of the sixth embodiment different from that of the first embodiment will be described.The multipath interference can be corrected as described above in a limited area in which the measurement distance measurable with spot light and the measurement distance measurable with diffused light match each other. When the luminous output of the light source that emits spot light and the luminous output of the light source that emits diffused light are the same, the spot light is focused in a spot shape, whereas the diffused light is expanded to the entire emission area. Thus, when the luminous outputs of both light sources are the same, the light intensity at each spot when the spot light is emitted is higher than the light intensity of the diffused light when the diffused light is emitted on the same area as that of the spot. Thus, when the spot light is emitted, the measurement range may be a longer distance than that in the case of emission of the diffused light, and a short distance is no longer detectable due to saturation of the light intensity of detected light. That is, there is a disadvantage that the detectable distance ranges with the spot light and the diffused light differ from each other.

[0143] Hereinafter, how much the light intensity of spot light differs from the light intensity of diffused light is specifically calculated.FIG. 13 is a model view for comparing the light intensity ratio between spot light 20 and diffused light 21. It is considered that spot light 20 is a circle with a radius of R, the pitch is 100R, and the light intensity of diffused illumination that is the light intensity per unit area is a dotted-line rectangle. The emission area with spot light 20 is 7tR2, whereas the emission area with diffused light 21 is lOORx lOOR = 10000R2.That is, the emission area ratio is spot light / diffused light ~ 3200. When the same light source emits light with the same light intensity and respective optical systems generate spot light and diffused light, such a large difference is generated. When the same sensor detects these beams of light, a dynamic range large enough to accommodate the large difference is to be used. When spots are spaced farther apart than the spots in this example, the difference in light intensity increases, and a larger dynamic range is to be used.Thus, spot light covers a long range and diffused light covers a short range as the range of distance measurement, and hence the correction range for multipath interference is narrowed.

[0144] FIG. 14 illustrates a correction range for multipath interference (MPI). The vertical axis indicates the distance, and a distance measurable with diffused illumination and a distanceFN202304898 measurable with spot illumination are indicated by bar graphs. The range from the upper limit distance to the lower limit distance of the bar graph is a measurable range.The upper limit distance, that is, the farthest measurable distance is determined based on the SN ratio between light (signal S) returning from a distant place and light (noise N) of the outside or the like. The noise includes noise generated from an amplifier or the like included in an electronic circuit. The major noise of the electronic circuit is environmental noise from an external electron beam or an external electronic device.The noise component also varies depending on, for example, the ambient temperature. Noise increases as the temperature increases. Thermal noise refers to noise that is also referred to as Johnson noise or Nyquist noise. The noise is generated due to thermal agitation of charge carriers in an electronic circuit such as a resistor. The noise appears almost as white noise and exhibits a distribution close to a Gaussian distribution.The lower limit distance, that is, the shortest measurable distance may be limited due to saturation of the detection value. The saturated state of the detection value can be controlled based on the integrated time and the gain of the detector. The saturated state can also be controlled based on the integrated luminous output of the light source.

[0145] The measurement range with diffused illumination and the measurement range with spot illumination differ from each other. This is because the emission area of diffused illumination on an object and the emission area of spot illumination on the object differ from each other. Although described above, in the embodiment of the present disclosure, the correction range for multipath interference can be increased by adjusting the measurement range.Specifically, the measurement range can be adjusted by changing any one of the number of light sources, the luminous output, the integrated time, and the sensor gain between imagecapturing with diffused illumination and image-capturing with spot illumination. Thus, distance measurement data with a small distance error due to multipath interference can be acquired in a wide distance range.The integrated luminous output is obtained by integrating the luminous output for one-time image-capturing. For example, in the case of pulsed light that is intermittently emitted, the integrated luminous output can be determined as an integrated value based on the pulse width and the luminous output at a peak of the pulsed light. The unit is power (W).

[0146] FIG. 26 is a block diagram illustrating a hardware configuration of a distance-measuring apparatus 100 according to a sixth embodiment.In the present embodiment, a light emitter 1-1 includes a light-source unit 10-1, a light-source unit 10-3, a light-emitting optical system 1 - lb, and a light-emitting optical system 1 -3b. The light-emitting optical systems 1-lb and l-3b are, for example, lenses. A light emitter 1-2 includes a light-source unit 10-2 and a light-emitting optical system l-2b such as a lens. In some embodiments, the light emitter 1-1 include a light-source unit 10-1 alone as a lightsource unit. In some other embodiments, the light emitter 1-1 includes a light-source unit 10-FN2023048981 and a light-source unit 10-3. The light emitter 1-1 is a diffused -light source that emits diffused light 27 to obtain substantially uniform luminance within an emission range. The light emitter 1-2 is a spot-light source that emits spot light 29 that is pattern light.The light receiver 2 receives the reflected light of the diffused light emitted from the light emitter 1-1 to the object. Reflected light 25 of the spot light 29 emitted from the light emitter 1-2 to the object 26 is also received at a different timing. The number of image sensors 2a of the light receiver 2 is not limited. The same sensor may receive the reflected light of diffused light and the reflected light of spot light emitted from the respective light emitters, or a sensor that receives the reflected light of diffused light and a sensor that receives the reflected light of spot light may be provided separately.

[0147] FIG. 27 is a diagram of a hardware configuration of the light-source unit 10 (e.g., the lightsource units 10-1, 10-2, 10-3). The light-source unit 10 includes a direct current-direct current (DCDC) converter 101, a light-emission controller 102, a temperature detector 103, and a light-emission circuit 105. The light-emission circuit 105 includes a light source 110 that is a VCSEL or laser diode (LD) (VCSEL / LD), and a drive switch 104. The light-source units 10-1, 10-2, and 10-3 include light sources 1-la, l-2a, and 1-3 a, respectively, as the light source 110. Each component is mounted on an electric substrate or the like to define the light-source unit 10.The light-emission circuit 105 and the light-emission controller 102 in the light-source unit 10 supply drive signals with a predetermined voltage waveform and a predetermined lightemission frequency based on control signals supplied from the CPU 44 to temporally modulate (temporally control) emission of light from the light source 110. The drive signal supplied to the light source 110 may use a rectangular wave, a sine wave, or a voltage waveform having a predetermined waveform shape. Drive signals input to the light source 110 is modulated by varying the frequency of the voltage waveform.The light-source unit 10 includes the DCDC converter 101 and the light-emission controller 102, and controls the pulse width of light output from the light-emission controller 102 and the drive switch 104, and the voltage applied to the DCDC converter 101.The DCDC converter 101 is an example of a voltage applier that converts a voltage input from an external power supply into a voltage of a predetermined value to generate a voltage to be applied to the light-emission circuit 105. For example, the DCDC converter 101 can convert a voltage, such as 5 [V], 12 [V], or 24 [V], output from the external power supply into a voltage in a range from 5 [V] to 100 [V] to generate a voltage to be applied. The external power supply may be, for example, a battery of a car on which a laser imaging detection and ranging (LiDAR) device is mounted.The light-emission controller 102 includes, for example, a FPGA or a system on a chip (SoC). The light-emission controller 102 is an example of a pulse-width controller that controls the pulse width of light output from the light-emission controller 102 by switching the drive of the drive switch 104 at a desirable timing.FN202304898The light-emission circuit 105 includes the light source 110, and the drive switch 104, and is a portion of the light-source unit 10 that emits a laser beam.The light source 110 emits a laser beam. Instead of the LD, for example, a VCSEL or a LED may be used.The drive switch 104 is a switching circuit that switches the state between emission (on) and non-emission (off) of the laser beam that is emitted from the light source 110. In the present embodiment, the drive switch 104 includes a gallium nitride (GaN) field-effect transistor (FET). In one example, the drive switch 104 may include a metal oxide semiconductor field effect transistor (MOSFET) or another transistor.The temperature detector 103 includes a temperature sensor such as a thermocouple that detects at least one of the temperature of the light source 110 and the temperature of the area around the light source 110. Since the current-voltage (I-V) characteristics and the current- luminous output (I-L) characteristics of the light source 110 may vary depending on the temperature, the light source 110 can be controlled based on the detection value obtained by the temperature detector 103.

[0148] In one example, a light emitter 1 may include, in addition to the above-described configuration, an optical element, such as a lens or a mirror; a spectroscope that separates the emitted laser beam; or a light intensity measuring device that measures the light intensity of the emitted laser beam.

[0149] FIG. 28 illustrates, as a first modification, a system in which two light sources 110 and 111 are connected in series. When multiple light sources 110 and 111 are connected in series, the voltage may exceed 20 V, and the voltage is boosted by the DCDC converter 101. The light sources 110 and 111 are controlled to be turned on or off (on / off control) by the drive switch 104. The timing of the on / off control is determined by the light-emission controller 102. For example, a rectangular signal of 100 MHz is input from the light-emission controller 102 for about 1 ms. The drive switch 104 repeats the on / off control with 100 MHz in accordance with the rectangular signal. Consequently, an optical signal with 100 MHz is generated.An element for driving the drive switch 104 is provided for light-emission control. The element is an IC typically referred to as a FET. The multiple light sources 110 and 111 may be connected to the DCDC converter 101 in parallel or in series. A light-emitting unit often uses a high-power light source, and the temperature detector 103 may be provided. For example, a temperature IC or a thermistor may be used.

[0150] FIG. 29 is a diagram of a hardware configuration of the light-emission circuit 105 in detail. The voltage supplied from the DCDC converter 101 is connected to a resistor 106, the light source 110, and a FET 104a, and is connected to the GND. In the present embodiment, the drive switch 104 uses the FET 104a. The resistor 106 is connected to the light source 110 to set an optimal current value for the light source 110. For example, the current value may be 1FN202304898 or 0.56 . The luminous output can be changed by making the resistance of the resistor 106 variable, or providing multiple resistors 106 and making one of the multiple resistors 106 selective. In one example, the luminous output can be changed by changing the voltage of the DCDC converter 101. The FET 104a may use a MOSFET or a GaN FET.

[0151] FIG. 30 is a diagram of a hardware configuration of a light-source unit 10 according to a second modification. In this example, a laser driver (LDDR) 107 including a resistor and a FET incorporated in one chip is used. For example, the configuration of the LDDR 107 may include a variable resistor and a FET, or a current mirror circuit.Hereinafter, a technique of specifically expanding the correction range for multipath interference will be described.

[0152] FIG. 31 is the functional block diagram of the distance-measuring apparatus 100 according to the present embodiment. The light emitter includes three light-source units (10-1, 10-2, 10-3). The light-source units (10-1, 10-2, 10-3) are controlled by a drive-condition setting unit 150. Two light-source units (10-1, 10-3) are used for diffused illumination. The light-source units (10-1, 10-3) include respective optical elements, and are optically designed such that the entire area that receives diffused light can uniformly receive the light from two light sources. A light-source unit (10-2) is used for spot illumination.

[0153] A method of setting the light-source unit 10 is described next.A first light source of diffused light indicates the light-source unit 10-1. A second light source of diffused light indicates the light-source unit 10-3. A first light source of spot light indicates the light-source unit 10-2. No second light source of spot light is present. In the following examples, the second light source of diffused light is used in one case whereas the second light source of diffused light is not used in another case.Table 1 presents an example of setting conditions. In the example of Table 1, the second light source of diffused light is not used, but the voltage value of the first source of diffused light is set higher than that of the first source of spot light.Table 1]Table 1 presents luminous outputs in terms of voltage values. The unit is voltage (V). A specific method of adjusting the luminous output is a method of changing the voltage of the DCDC converter 101 illustrated in FIG. 27. In this case, a set voltage is designated from the drive-condition setting unit 150.

[0154] In one example, the luminous output can be set using a current value. In this case, in the circuit in FIG. 30, the current flowing to the LDDR 107 and the light source 110 (e.g., theFN202304898 light source 1-la, the light source l-3a) is changed, and changing the flowing current can change the luminous output. Table 2 presents luminous outputs different from those in Table 1, and presents an example of settings in terms of current. In the example of Table 2, the current value of the first source of diffused light is set higher than that of the first source of spot light.[Table 2]Table 2 presents luminous outputs in terms of current values. The unit is ampere (A).

[0155] Methods for setting the luminous output include a method of representing the luminous output in terms of power (unit of watt (W)). In this case, light emitted via an optical element is detected by a large detector and is converted into watts per unit time to be quantified. As a setting method, voltage values or current values are recorded in a table for multiple luminous outputs (unit of power), and a recorded voltage value or current value is set for a set power value to be used.When the luminous output (power value) is not recorded, the luminous output (power value) is derived by linearly interpolating a recorded close luminous output (power value). In the case of a light source such as a VCSEL, the luminous output, and the input current and the input voltage have a substantially linear relationship, and are expected to be reproduced under the same temperature. With this unit system, the luminous output can be quantified with an optical element or the like included.Table 3 presents an example of the case described above. Table 3 presents luminous outputs different from those in Table 1 and Table 2, and presents an example of settings in terms of power. In the example of Table 2, the power value of the first source of diffused light is set higher than that of the first source of spot light.[Table 3]Table 3 presents luminous outputs in terms of power. The unit is watt (W).In the above examples, the luminous output of the second light source of diffused light is set zero (0). In other words, the second light source of diffused light is not used in the above examples. However, in the following example, two light sources of diffused light (e.g., the light-source unit 10-1, the light-source unit 10-3) are used. In this case, settings are as presented in Table 4. The number of light sources can be changed by setting as described above.FN202304898Table 4 presents an example in which the number of light sources is plural. In the example of Table 4, the number of light-source units of diffused light (i.e., two light-source unit 10-1 and light-source unit 10-3) is more than the number of light-source units of spot light (i.e., one light-source unit 10-2).Table 4]Table 4 presents luminous outputs in terms of power. The unit is watt (W).

[0156] A distance-measuring apparatus 100 according to the present embodiment includes a first light emitter 1-1 that emits first light (e.g., diffused light), a second light emitter 1-2 that emits second light (e.g., spot light), and a light receiver 2 that detects the first light and the second light, which are reflected from an object 26 within an area 26. The first light emitter 1-1 includes a first light-source unit (e.g., the light-source unit 10-1, the light-source unit 10-3). The second light emitter 1-2 includes a second light-source unit (e.g., the light-source unit 10- 2). An integrated luminous output of the first light-source unit is larger than an integrated luminous output of the second light-source unit. The integrated luminous output of diffused light is made larger than the integrated luminous output of spot light, matching the measurement ranges of the spot light and the diffused light.

[0157] In the first example, the integrated luminous output of the first light-source unit is made larger than the integrated luminous output of the second light-source unit by controlling the current and voltage to be applied to the first light-source unit and the second light-source unit.

[0158] In the second example, each of the first light-source unit and the second light-source unit includes multiple light-emitting units. The number of light-emitting units in the first lightsource unit is more than the number of light-emitting units in the second light-source unit. Further, the integrated luminous output of the first light-source unit is larger than the integrated luminous output of the second light-source unit. The light-emitting unit represents a light-emitting element having a light-emitting area. Each of The light sources 1-la to 1-3 a represents an array light source in which multiple light-emitting units are arranged in one dimension or two dimensions. For example, when the light source is a VCSEL, the lightemitting unit refers to an individual VCSEL element having a light-emitting area determined by a current confinement layer and an active layer. The multiple light-emitting units have respective independent light-emitting areas. The light-emitting units have respective limits of light-emission intensities of substantially the same degree. Thus, a light source including as many light-emitting units as possible can obtain a stronger integrated luminous output.

[0159] FN202304898In the second example, for example, the first light emitter, which emits diffused light, includes multiple first light-source units (e.g., the light-source unit 10-1 and the light-source unit 10-3). The second light emitter, which emits spot light, includes one or more second light-source units (e.g., light-source unit 10-2), which are fewer in number than the first light-source units. This configuration allows for a greater integrated luminous output from the first light-source units than from the second light-source units. In this case, the light-source units (e.g., 10-1 to 10-3) represent a portion including a light-source element circuit including a light-emitting element such as a VCSEL, a resistor, and a switching element such as a FET. Thus, in the first light-source unit that involves a larger integrated luminous output, the emission can be distributed among the multiple first light-source units. This distribution helps maintain the required integrated luminous output, while preventing the light-source unit 10 from overheating due to the heat generated by emitting intense light.

[0160] Further, a configuration with a higher number of light-emitting units can be achieved by increasing the number of chips in light sources (e.g., a light-emitting element such as a VCSELS, an LD, an LED) included in the light-source unit 10-1, or increasing the number of light-emitting units included in one chip while maintaining the number of light-source units 10 of diffused light (i.e., without using the light-source unit 10-3). For example, the lightemission controller 102 reduces the number of light-emitting units in the light source l-2a of spot light to control the integrated luminous output The number of light emitters 1, including the light-source unit 10 and the optical element, may be increased. In this configuration, the integrated luminous output of diffused light can be increased.

[0161] It is known that the efficiency of a light-emitting element such as a VCSEL varies depending on the luminous output. Thus, even though all the integrated luminous outputs are the same, the number of VCSELs serving as light-emitting elements is increased and the luminous output of each VCSEL is decreased, thereby reducing the amount of generated heat. Thus, for example, the frame rate is increased, a variation in temperature due to heat generation is reduced, and measurement with high accuracy can be performed.

[0162] The light-emitting unit may be electrically independent, or may be electrically connected in parallel to another light-emitting unit. Since the light-emitting units are discretely disposed by the oxidized confinement layer, heat generated by the respective light-emitting units is likely to be dissipated to a material with high thermal conductivity, such as the back surface. A light source including a larger number of thermally discretely independent light-emitting units can have a higher integrated luminous output.

[0163] FIG. 34 is a flowchart presenting a flow of a distance-measuring process performed by the distance-measuring apparatus 100. As presented in FIG. 34, the drive-condition setting unit 150 sets emission conditions of the light-source unit (10-2) of spot light. Simultaneously, theFN202304898 integrated time and the gain for spot light are set in the image sensor 2a and a converter circuit 2b (step S10). Subsequently, spot light is emitted and the image sensor 2a receives the reflected light of the spot light to capture an image (step Sil). The image storage unit 402 stores a phase signal (phase image) in each phase from the image sensor 2a in a storage unit such as the RAM 46 (step S12).The distance calculation unit 403 calculates distance information indicating the distance to the object based on the multiple phase images stored by the image storage unit 402 (step SI 3). The drive-condition setting unit 150 sets emission conditions of the light-source unit of diffused light (e.g., 10-1, 10-3). Simultaneously, the integrated time and the gain for diffused light are set in the image sensor 2a and a converter circuit 2b (step S14). Subsequently, diffused light is emitted, and the image sensor 2a receives the reflected light of the diffused light to capture an image (step SI 5).The image storage unit 402 stores a phase signal (phase image) in each phase from the image sensor 2a in a storage unit such as the RAM 46 (step SI 6). The distance calculation unit 403 calculates distance information indicating the distance to the object based on the multiple phase images stored by the image storage unit 402 (step SI 7).That is, the distance-measuring apparatus 100 emits diffused light and spot light at different timings to capture images, and acquires two distance images.In the present embodiment, the distance images are acquired in the order of the spot light and diffused light; however, the order is not limited thereto, and the order may be any order as long as the image-capturing timings are different.

[0164] A distance-measuring apparatus 100 according to the present embodiment includes a first light emitter 1-1 that emits first light; a second light emitter 1-2 that emits second light; and a light receiver 2 that detects the first light and the second light reflected by an object 26 in an area 28. The light receiver performs first light reception in a case where the first light 27 is emitted and the light receiver performs second light reception in a case where the second light 29 is emitted. The second light is light (spot light) having a light intensity distribution in the area. The first light is light (diffused light) having a light intensity distribution smaller than the light intensity distribution of the first light emitted from the first light emitter in the area. An integrated luminous output of the first light is larger than an integrated luminous output of the second light.

[0165] The area 28 indicates a space within a distance that the diffused light or the spot light reaches. It is expected that the diffused light and the spot light are emitted on a desirable curved surface that can be set in the space. In the light intensity distribution of light emitted on the curved surface, the difference between the maximum value and the minimum value of the light intensities of the light is compared. In the comparison of the difference, smaller one is referred to as diffused light (first light), and larger one is referred to as spot light (second light).FN202304898While the light emitted from the light-source unit 10 is deflected by an optical element, it is assumed that all light emitted from the housing of the distance-measuring apparatus 100 is integrated in all directions. The integrated luminous output is proportional to the amount of power input to the light source, and the magnitude relationship between the spot light and the diffused light does not change. That is, the integrated luminous output can be also referred to as an input power amount. In this case, it is assumed that absorption of light by an optical system in the distance-measuring apparatus 100, such as an optical aperture included in the optical element, does not have a large influence in comparison between the luminous outputs of the diffused light and the spot light. In an irregular case such as a case where light absorption in the distance-measuring apparatus 100 is large, the luminous output of light obtained by integrating the light emitted from a more correct distance-measuring apparatus 100 in all directions is used as an integrated luminous output.The drive-condition setting unit 150 serves as a controller that controls the integrated luminous outputs of the first light and the second light emitted from the first light emitter 1-1 and the second light emitter 1-2, respectively. This allows for appropriate adjustment of the integrated luminous outputs, enabling high-accuracy measurement over a wide range of distances.

[0166] In the distance-measuring apparatus 100 according to the present embodiment, the light receiver includes a photodetector that detects light, and the photodetector performs the first light reception and the second light reception.This eliminates the use of multiple light receivers. Since the same ToF sensor can be used, a common circuit can be used, thereby attaining decreases in cost and size.

[0167] Seventh EmbodimentFIG. 35 is a functional block diagram according to the present embodiment. In the present embodiment, a light-receiving system controller 160 can set conditions of reception of light. Specifically, at least one of the integrated time and the gain (amlification factor) of the reception of light controlled by the image sensor 2a and the converter circuit 2b are controlled. In the present embodiment, at least the light emitter 1-1 as a diffused-light source and the light emitter 1-2 as a spot-light source are used.For example, even when the peak intensity and the pulse width of spot light are substantially equal to those of diffused light, when image-capturing is performed with one pulse of spot light and two pulses of diffused light, the integrated luminous output of diffused light can be substantially twice the integrated luminous output of spot light. By performing integration multiple times, for example, by capturing and integrating two images, the multiple images are combined together to ensure the integrated luminous output and further to average the luminous outputs, thereby providing tolerance to shot noise or the like. In the present embodiment, since the modulation frequencies of the spot light and the diffused light are theFN202304898 same, lengthening the integrated time is synonymous with increasing the number of integrations.

[0168] In some cases, it is desirable to perform distance measurement at a high frame rate. An example of the case is capturing a moving image. In this case, the luminous output of the light source 110 and the integrated time of emission of light are minimized to prevent heat, which is disadvantageous for a high frame rate, from being generated. For example, the luminous output of the VCSEL and the integrated time of emission of light are set such that the luminous outputs of both spot light and diffused light are minimized (to about a light emission threshold value of the VCSEL), and the gain of the ToF sensor is changed, thereby matching the measurement ranges of the spot light and the diffused light while being advantageous for a high frame rate.Table 5 presents a setting example. In this case, the luminous output is expressed in units of power (W). As illustrated in FIG. 35, the integrated time is controlled by the light-receiving system controller 160 and is set by the image sensor 2a and the converter circuit 2b. The gain is changed by the converter circuit 2b.Table 5]

[0169] FIG. 36 illustrates a specific configuration for changing the gain. The light-receiving system controller 160 controls multiplexers 60a and 64 to select one gain 62. In FIG. 36, the multiplexers 60a and 64 are provided in the converter circuit 2b; however, a similar circuit may be provided in the image sensor 2a. The multiplexers 60a and 64 are an example of a converter circuit.

[0170] FIG. 37 is a flowchart presenting a flow of a distance-measuring process according to the present embodiment. As presented in FIG. 37, the light-receiving system controller 160 controls the image sensor 2a and the converter circuit 2b to set an integrated time and a gain for spot light (step S21). Simultaneously, the drive-condition setting unit 150 sets emission conditions of the light-source unit of spot light (10-2). Subsequently, spot light is emitted and the image sensor 2a receives the reflected light of the spot light to capture an image (step S22). The image storage unit 402 stores a phase signal (phase image) in each phase from the image sensor 2a in a storage unit such as the RAM 46.The light-receiving system controller 160 controls the image sensor 2a and the converter circuit 2b to set an integrated time and a gain for diffused light (step S23). Simultaneously, the drive-condition setting unit 150 sets emission conditions of the light-source unit of diffused light (10-1). Subsequently, diffused light is emitted, and the image sensor 2a receives the reflected light of the diffused light to capture an image (step S24). The imageFN202304898 storage unit 402 stores a phase signal (phase image) in each phase from the image sensor 2a in a storage unit such as the RAM 46.That is, the distance-measuring apparatus 100 emits diffused light and spot light at different timings to capture images, and acquires two distance images.In the present embodiment, the distance images are acquired in the order of the spot light and diffused light; however, the order is not limited thereto, and the order may be any order as long as the image-capturing timings are different.The distance calculation unit 403 calculates distance information indicating the distance to an object based on multiple phase images stored by the image storage unit 402.The correction-value calculation unit 404 calculates a correction value to correct the distance information using the distance image obtained through emission of light from the light-source unit 10-1 (diffused light) and the distance image obtained through emission of light from the light-source unit 10-2 (spot light). Specifically, the correction- value calculation unit 404 uses a residual (the difference between a distance measurement value of diffused light and a distance measurement value of spot light) at a position at which spot light is obtained in the distance image obtained through emission of light from the light-source unit 10-1 (diffused light) as a correction value, and interpolates a position at which spot light is not obtained by general image processing to calculate a correction value of the entire image.The distance correction unit 405 corrects the distance information obtained through emission of light from the light-source unit 10-3 (spot light) using the correction value calculated by the correction- value calculation unit 404. Specifically, the distance correction unit 405 subtracts the correction value calculated by the correction-value calculation unit 404 from the distance measurement value of diffused light obtained through emission of light from the light-source unit 10-1 to perform correction (S25).

[0171] In one example, the distance correction unit 405 may omit the correction when the difference or ratio between the distance measurement value of diffused light and the distance measurement value of spot light calculated by the correction-value calculation unit 404 is within a predetermined range.The output unit 406 outputs the distance information indicating the distance to the object and corrected by the distance correction unit 405 to an external device via the input-output I / F 43.

[0172] A distance-measuring apparatus 100 according to the present embodiment includes a first light emitter 1-1 that emits first light (diffused light); a second light emitter 1-2 that emits second light (spot light); and a light receiver 2 that detects the first light and the second light reflected by an object 26 in an area 28. The light receiver performs first light reception in a case where the first light 27 is emitted and the light receiver performs second light reception in a case where the second light 29 is emitted. The first light reception includes at least one of either a greater number of integrations or a higher amplification factor compared to the second light reception.FN202304898The light-receiving system controller 160 serves as a controller that controls at least one of either the number of integrations or the amplification factor of the light receiver. At least one of either the integrated time or the gain of reception of light is set by the light-receiving system controller 160. The received light is integrated as a signal using the image sensor 2a, the converter circuit 2b, and the ADC 3. When the integrated time is set by the light-receiving system controller 160, the drive-condition setting unit 150 sets the driving time of the light source according to the integrated time.The setting of the image sensor 2a that has received the reflected light of diffused light is an example of setting of first light reception when the first light is emitted. The setting of the image sensor 2a that has received the reflected light of spot light is an example of setting of second light reception when the second light is emitted. The gain is an example of an amplification factor. With the amplification factor, the signal is amplified and digitized by the ADC 3.

[0173] In the present embodiment, at least one of either the integrated time or the gain of the light receiver 2 with spot light and at least one of either the integrated time or the gain of the light receiver with diffused light can be individually controlled. Thus, the intensity of the received signal can be controlled by controlling the numerical value. This allows the measurement ranges of the spot light and the diffused light to be matched with each other. Further, according to the present embodiment, since the luminous output can also be decreased as much as possible and the amount of heat generated from the light source can be reduced, an image can be captured at a high frame rate, and ToF distance measurement for, for example, a moving image can be performed. By performing integration multiple times, the multiple images are combined together to ensure the integrated luminous output and further to average the luminous outputs, thereby providing tolerance to, for example, shot noise.

[0174] Eighth EmbodimentFIG. 38 is a functional block diagram according to the present embodiment. The present embodiment has a feature in which a light-receiving system and a light-emitting system include temperature measurement units 171 to 173. A temperature determination unit 170 compares the results of the temperature measurement with respective reference values to make a determination. Based on the determination results, the light-receiving system controller 160 and the drive-condition setting unit 150 set respective settings. Emission of light and reception of light controlled to the settings are performed to capture images.The temperature measurement unit 171 of the light receiver 2 includes a built-in temperature sensor of the image sensor 2a or a temperature sensor mounted on a substrate on which the image sensor 2a is mounted. A temperature sensor is mounted on a light source substrate of the light emitter 1-1 to serve as the temperature measurement unit 172. A temperature sensor is mounted on a light source substrate of the light emitter 1-2 to serve as the temperature measurement unit 173. Thus, optimal control can be performed based on temperatureFN202304898 information. For example, the value of each temperature sensor is acquired and compared with a reference temperature. The reference temperature may be a value with a margin from the rated value of a part (for example, 10°C), or an upper limit or a lower limit of a temperature at which performance of distance measurement can be ensured. When the value of each temperature sensor is the reference temperature or higher and a permissible temperature or higher (for example, during abnormal heat generation), the operation is ended without capturing an image. In this case, the permissible temperature is set to a temperature having a value with a margin from the rated value of each part.

[0175] When one part has a temperature of the permissible temperature or lower and the reference temperature or higher, the measurement ranges of diffused illumination (the light emitter 1-1) and spot illumination (the light emitter 1-2) can be matched with each other by setting the light-emitting system and the light-receiving system to the design presented in the following table. When one or more parts each have a temperature of the permissible temperature or lower and the reference temperature or higher, it is effective to change the setting of the part having a temperature closest to the reference temperature, the part having the highest temperature, or the like as follows.Table 6]Table 6 presents reference settings. [Table 7]Table 7 presents settings when the temperature of the light source (VCSEL) substrate for diffused illumination is high. For example, by changing the luminous output of the VCSEL and changing the gain, distance measurement can be performed with both diffused illumination and spot illumination, and correction for multipath interference can be performed while heat generation of the VCSEL substrate is reduced.[Table 8]Table 8 presents settings when the temperature of the VCSEL substrate for spot illumination is high. For example, by changing the integrated time and the gain, distance measurement can be performed with both diffused illumination and spot illumination, and correction forFN202304898 multipath interference can be performed while heat generation of the VCSEL substrate is reduced.[Table 9]Table 9 presents settings when the temperature of the image sensor 2a is high. Since the image sensor 2a operates and generates heat as the integrated time increases, decreasing the integrated time is effective for decreasing the amount of generated heat. Thus, by decreasing the integrated time and increasing the gain or the luminous output, distance measurement can be performed with both diffused illumination and spot illumination, and correction for multipath interference can be performed.

[0176] FIG. 39 is a flowchart presenting a flow of a distance-measuring process according to the present embodiment. As presented in FIG. 39, the temperature of a substrate is measured (step S31). The temperature of the substrate in this case indicates temperatures measured by the temperature measurement unit 171 relating to the image sensor 2a, the temperature measurement unit 172 relating to the light-source unit 10-1, and the temperature measurement unit 173 relating to the light-source unit 10-3. Each of the temperatures has a determined reference value in the temperature determination unit 170.At this time, the obtained temperature is compared with the reference value (step S32). When the temperature is higher than the reference value, it is determined whether the temperature is within a permissible range (step S33). When the temperature is lower than the reference value, an image is captured with standard settings set in advance (Table 6 is an example thereof).

[0177] When the temperature is within the permissible range, a mode is selected, and the settings are determined. The settings are settings as presented in Tables 6 to 9. The values are updated to the settings (step S34). When the temperature is higher than the permissible range, an alert is displayed (step S42) and the measurement is ended.The drive-condition setting unit 150 sets emission conditions of the light-source unit of spot light (10-3) (step S35). The light-receiving system controller 160 controls the image sensor 2a and the converter circuit 2b to set an integrated time and a gain for spot light (step S36). Subsequently, spot light is emitted and the image sensor 2a receives the reflected light of the spot light to capture an image (step S37). The image storage unit 402 stores a phase signal (phase image) in each phase from the image sensor 2a in a storage unit such as the RAM 46. The drive-condition setting unit 150 sets emission conditions of the light-source unit (10-1) of diffused light (step S38). The light-receiving system controller 160 controls the image sensor 2a and the converter circuit 2b to set an integrated time and a gain for diffused light (step S39). Subsequently, diffused light is emitted, and the image sensor 2a receives the reflectedFN202304898 light of the diffused light to capture an image (step S40). The image storage unit 402 stores a phase signal (phase image) in each phase from the image sensor 2a in a storage unit such as the RAM 46.That is, the distance-measuring apparatus 100 emits diffused light and spot light at different timings to capture images, and acquires two distance images.In the present embodiment, the distance images are acquired in the order of the spot light and diffused light; however, the order is not limited thereto, and the order may be any order as long as the image-capturing timings are different.The distance calculation unit 403 calculates distance information indicating the distance to an object based on multiple phase images stored by the image storage unit 402.The correction-value calculation unit 404 calculates a correction value to correct the distance information using the distance image obtained through emission of light from the light-source unit 10-1 and the distance image obtained through emission of light from the light-source unit 10-2. Specifically, the correction-value calculation unit 404 uses a residual (the difference between a distance measurement value of diffused light and a distance measurement value of spot light) at a position at which spot light from the light-source unit 10-2 (spot light) has been obtained in the distance image obtained through emission of light from the light-source unit 10-1 (diffused light) as a correction value, and interpolates a position at which spot light is not obtained by general image processing to calculate a correction value of the entire image.The distance correction unit 405 corrects the distance information obtained through emission of light from the light-source unit 10-3 (spot light) using the correction value calculated by the correction- value calculation unit 404. Specifically, the distance correction unit 405 subtracts the correction value calculated by the correction-value calculation unit 404 from the distance measurement value of diffused light obtained through emission of light from the light-source unit 10-1 to perform correction (S41).

[0178] In one example, the distance correction unit 405 may omit the correction when the difference or ratio between the distance measurement value of diffused light and the distance measurement value of spot light calculated by the correction-value calculation unit 404 is within a predetermined range.The output unit 406 outputs the distance information indicating the distance to the object and corrected by the distance correction unit 405 to an external device via the input-output I / F 43. According to the present embodiment, when one part has a temperature of the permissible temperature or less and the reference temperature or higher, the measurement ranges of spot illumination and diffused illumination can be matched with each other by setting the lightemitting system and the light-receiving system. When one or more parts each have a temperature of the permissible temperature or less and the reference temperature or higher, it is effective to change the setting of the part having a temperature closest to the reference temperature, the part having the highest temperature, or the like. Thus, a stable operation canFN202304898 be ensured even when heat is generated by continuous operation or capturing of a moving image and the distance-measuring apparatus is in a high temperature state. A distancemeasuring apparatus that can provide a stable operation even when the ambient temperature rises can be provided.

[0179] In the distance-measuring apparatus 100 according to the present embodiment, a temperature of the light emitting and receiving device is measured, at least one of the integrated luminous output of the first light and the integrated luminous output of the second light is changed based on the measured temperature, and at least one of a combination of the integrated time and the amplification factor of the first light reception and a combination of the integrated time (the number of integrations) and the amplification factor of the second light reception is changed. A decrease in luminous output due to heat generation can be avoided. Thus, measurement with further high accuracy can be performed.

[0180] Ninth EmbodimentFIG. 40 is a functional block diagram according to the present embodiment. An external light determination unit 180 determines a measurement value obtained by measuring external light and stored in the image storage unit 402. At this time, as a detection method, the image sensor 2a may be used, or an external light acquisition unit 181 installed to detect external light may be used.The determination result is transmitted to the light-receiving system controller 160 and the drive-condition setting unit 150. In response to the result, the light-receiving system controller 160 and the drive-condition setting unit 150 set the settings of the light emitters 1-1 and 1-3 and the light receiver 2. Emission of light and reception of light controlled to the settings are performed to capture images. By acquiring external light before measurement, conditions of spot illumination and diffused illumination advantageous for external light can be set. Examples of the method of acquiring external light include a method of acquiring an image without emitting a laser beam. With this method, a value proportional to the light level of external light can be acquired.

[0181] The settings to maximize the multipath interference correction range differ depending on whether external light is strong or weak. When external light is strong, the SN ratio in a long range decreases due to external light, and hence it is difficult to perform measurement in a long range. The probability that the image sensor 2a is saturated in a short range increases. Thus, it is also difficult to perform measurement in a short range. Thus, after the light level of external light is detected, the luminous outputs of spot light and diffused light are increased to enhance the measurement in a long range. The gains and integrated times of the image sensor 2a and the converter circuit 2b are decreased to avoid occurrence of a saturated state.

[0182] FN202304898Tables 10 and 11 present examples of setting conditions for reception and emission of light. [Table 10]Table 10 presents reference settings when external light is weak.[Table 11]Table 11 presents an example of settings when external light is strong. By increasing the luminous output and adjusting at least one of the integrated time (the number of integrations) and the gain by that increased amount, distance measurement can be performed even when external light is strong.

[0183] FIG. 41 is a flowchart presenting a flow of a distance-measuring process according to the present embodiment. As presented in FIG. 41, external light is measured (step S51). In the external light measurement, the image sensor 2a is used to illuminate an object with sunlight or indoor light while the distance-measuring apparatus 100 is not emitting light, and quantify the reflected light. The measurement is not limited to using the image sensor 2a, and may use the external light acquisition unit 181. The external light is compared with a reference value (step S52). When the external light is the reference value or less, setting is performed using standard settings set in advance (Table 10 presents an example thereof). When the external light is more than the reference value, the settings are changed (Table 11 presents an example thereof) (step S60).The drive-condition setting unit 150 sets emission conditions of the light-source unit of spot light (10-3) (step S53). The light-receiving system controller 160 controls the image sensor 2a and the converter circuit 2b to set an integrated time and a gain for spot light (step S54). Subsequently, spot light is emitted and the image sensor 2a receives the reflected light of the spot light to capture an image (step S55). The image storage unit 402 stores a phase signal (phase image) in each phase from the image sensor 2a in a storage unit such as the RAM 46. The drive-condition setting unit 150 sets emission conditions of the light-source unit (10-1) of diffused light (step S56). The light-receiving system controller 160 controls the image sensor 2a and the converter circuit 2b to set an integrated time and a gain for diffused light (step S57). Diffused light is emitted and the diffused light is received by the image sensor 2a to capture an image (step S58). The image storage unit 402 stores a phase signal (phase image) in each phase from the image sensor 2a in a storage unit such as the RAM 46.That is, the distance-measuring apparatus 100 emits diffused light and spot light at different timings to capture images, and acquires two distance images.FN202304898In the present embodiment, the distance images are acquired in the order of the spot light and diffused light; however, the order is not limited thereto, and the order may be any order as long as the image-capturing timings are different.The distance calculation unit 403 calculates distance information indicating the distance to an object based on multiple phase images stored by the image storage unit 402.The correction-value calculation unit 404 calculates a correction value to correct the distance information using the distance image obtained through emission of light from the light-source unit 10-1 and the distance image obtained through emission of light from the light-source unit 10-2. Specifically, the correction-value calculation unit 404 uses a residual (the difference between a distance measurement value of diffused light and a distance measurement value of spot light) at a position at which spot light from the light-source unit 10-2 (spot light) has been obtained in the distance image obtained through emission of light from the light-source unit 10-1 (diffused light) as a correction value, and interpolates a position at which spot light is not obtained by general image processing to calculate a correction value of the entire image.The distance correction unit 405 corrects the distance information obtained through emission of light from the light-source unit 10-2 (spot light) using the correction value calculated by the correction- value calculation unit 404. Specifically, the distance correction unit 405 subtracts the correction value calculated by the correction-value calculation unit 404 from the distance measurement value of diffused light obtained through emission of light from the light-source unit 10-1 to perform correction (S59).

[0184] In one example, the distance correction unit 405 may omit the correction when the difference or ratio between the distance measurement value of diffused light and the distance measurement value of spot light calculated by the correction-value calculation unit 404 is within a predetermined range.The output unit 406 outputs the distance information indicating the distance to the object and corrected by the distance correction unit 405 to an external device via the input-output I / F 43.

[0185] In the distance-measuring apparatus 100 according to the present embodiment, the light receiver 2 (or an external light acquisition unit 181) detects a light level of external light other than the first light and the second light emitted from the first light emitter and the second light emitter, and the integrated luminous output of the first light and the integrated luminous output of the second light are changed based on the detected light level of the external light. Thus, noise due to external light can be reduced. For example, the light intensity can be adjusted to a light intensity with which a significant difference can be obtained even in measurement in a long range. The S / N ratio at a long distance is improved, and measurement with further high accuracy can be performed.

[0186] FN202304898In the distance-measuring apparatus 100 according to the present embodiment, the light receiver 2 (or the external light acquisition unit 181) detects a light level of external light other than the first light and the second light emitted from the first light emitter and the second light emitter, and at least one of the number of integrations and the amplification factor of the first light reception and at least one of the number of integrations and the amplification factor of the second light reception are changed based on the detected light level of the external light. Thus, noise due to external light can be reduced. For example, a disadvantage that the image sensor is saturated for an object in a short range with high reflectivity can be reduced. This can be implemented by decreasing the amplification factor or decreasing the integrated time. Thus, the disadvantage due to saturation in a short range can be reduced, measurement in a short range is improved, and hence measurement with further high accuracy can be performed.

[0187] Tenth EmbodimentSince the reception light level decreases in inverse proportion to the square of the distance, the reception light level range increases as the measurement range increases. Since the dynamic range of the image sensor is narrow, it is difficult to cover all the range at one time. There is a high dynamic range (HDR) system of HDR image-capturing that combines multiple captured images to measure a distance.

[0188] Even when spot illumination and diffused illumination are used in combination, distance measurement can be performed in a further wide range through HDR image-capturing. In FIG. 42, the horizontal axis indicates the distance, and the vertical axis indicates the light level of light input to the image sensor. For example, when measurement is performed in a wide range from 0.25 m to 10 m, the light level of light reflected by an object at 10 m to be used is about 1000 times the light level of light reflected by the object at 25 cm.

[0189] In the present embodiment, emission of light with a luminous output suitable for each distance has been studied. For example, a technique may be used in which the distances are set to a short range (1 m or less), a medium range (1 m to 3 m), and a long range (3 m to 10 m), and respective light levels are set. For example, the gain, the luminous output, and the integrated time are adjusted as presented in Tables 12 to 14. Thus, distance measurement with both spot illumination and diffused illumination can be performed for the set three kinds of ranges. Thus, correction for multipath interference suitable for each range can be performed. Even in this case, distance measurement with further high accuracy can be performed by updating the values in accordance with, for example, the external light or the substrate temperature based on other flowcharts.FN202304898Table 12 presents setting conditions for a short range.[Table 13]Table 13 presents setting conditions for a medium range. [Table 14]Table 14 presents setting conditions for a long range.

[0190] FIG. 43 is a flowchart presenting a flow of a distance-measuring process according to the present embodiment. As presented in FIG. 43, the drive-condition setting unit 150 sets emission conditions of the light-source unit (10-2) of spot light (step S71). At this time, the emission conditions are set to three kinds of luminous outputs corresponding to the short range (Table 12), the medium range (Table 13), and the long range (Table 14). The lightreceiving system controller 160 controls the image sensor 2a and the converter circuit 2b to set an integrated time and a gain for spot light (step S72). At this time, the reception conditions are set to three kinds of conditions corresponding to the short range (Table 12), the medium range (Table 13), and the long range (Table 14). Spot light is emitted and reflected light of the spot light is received by the image sensor 2a to capture an image. At this time, image-capturing is performed three times while the conditions are changed to the short range, the medium range, and the long range (step S73). The image storage unit 402 stores a phase signal (phase image) in each phase from the image sensor 2a in a storage unit such as the RAM 46.The drive-condition setting unit 150 sets emission conditions of the light-source unit (10-1) of diffused light (step S74). At this time, the emission conditions are set to three kinds of luminous outputs corresponding to the short range (Table 12), the medium range (Table 13), and the long range (Table 14). The light-receiving system controller 160 controls the image sensor 2a and the converter circuit 2b to set an integrated time and a gain for diffused light (step S75). At this time, the reception conditions are set to three kinds of conditions corresponding to the short range (Table 12), the medium range (Table 13), and the long range (Table 14). Diffused light is emitted, and the image sensor 2a receives the reflected light of the diffused light to capture an image (step S76). The image storage unit 402 stores a phase signal (phase image) in each phase from the image sensor 2a in a storage unit such as the RAM 46.FN202304898That is, the distance-measuring apparatus 100 emits diffused light and spot light at different timings for the three kinds of ranges to capture images, and acquires six distance images in total. In the present embodiment, the distance images are acquired in the order of the spot light and diffused light; however, the order is not limited thereto, and the order may be any order as long as the image-capturing timings are different. The distance calculation unit 403 calculates distance information indicating the distance to an object based on multiple phase images stored by the image storage unit 402. The correction-value calculation unit 404 calculates a correction value to correct the distance information using the distance image obtained through emission of light from the light-source unit 10-1 and the distance image obtained through emission of light from the light-source unit 10-3. Specifically, the correction-value calculation unit 404 uses a residual (the difference between a distance measurement value of diffused light and a distance measurement value of spot light) at a position at which spot light has been obtained in the distance image obtained through emission of light from the light-source unit 10-1 (diffused light) as a correction value, and interpolates a position at which light from the light-source unit 10-3 (spot light) is not obtained by general image processing to calculate a correction value of the entire image. The distance correction unit 405 corrects the distance information obtained through emission of light from the light-source unit 10-2 (spot light) using the correction value calculated by the correction- value calculation unit 404. Specifically, the distance correction unit 405 subtracts the correction value calculated by the correction-value calculation unit 404 from the distance measurement value of diffused light obtained through emission of light from the light-source unit 10-1 to perform correction (S59).

[0191] In one example, the distance correction unit 405 may omit the correction when the difference or ratio between the distance measurement value of diffused light and the distance measurement value of spot light calculated by the correction-value calculation unit 404 is within a predetermined range.The output unit 406 outputs the distance information indicating the distance to the object and corrected by the distance correction unit 405 to an external device via the input-output I / F 43.

[0192] Eleventh EmbodimentFIG. 44 is a functional block diagram according to the present embodiment. The present embodiment provides a distance-measuring system 300 including a light emitting and receiving device 200 that emits light to an object and detects the reflected light, and a computer 310a that converts a phase signal (DCS signal) into distance information. The light emitting and receiving device 200 outputs the detection result as a phase signal or the like. The output is transmitted to a cloud 216 and is sent to the computer 310a. In the present embodiment, since a signal is transmitted to an external cloud, a phase signal with a small communication volume is appropriate; however, in a case with strong external light orFN202304898 the like, a DCS signal can be selectively used as appropriate. Hereinafter, an example of transmitting a phase signal will be described.

[0193] A light emitter 1 causes a light-source unit 10 to emit light under conditions set in a drivecondition setting unit 150. In the present embodiment, one light-source unit is provided, and the light path is mechanically switched by a conversion element 102a. An example thereof is a switching device that includes a mechanical drive unit in an optical element such as a mirror. The conversion element 102a switches light between spot light and diffused light. At a destination after the switching, an optical element 11-2 to form spot light or an optical element 11-1 to form diffused light is disposed.

[0194] A light receiver 2 converts light through an optical element 2c into an electric signal by an image sensor 2a, and amplifies and integrates the signal. The electric signal is converted into a digital value by an ADC 3 based on the integrated time and the gain set by a light-receiving system controller 160. The converted signal is transmitted to an arithmetic unit 212 via a sensor I / F. The arithmetic unit 212 calculates the above-described phase signal. The calculated phase signal is output. The output is transmitted to the computer 310a via a transmission unit 215 and the cloud 216. The phase signal is converted into distance information by the computer 310a.

[0195] FIG. 45 is a sequence diagram illustrating a flow of a distance-measuring process according to the present embodiment. As illustrated in FIG. 45, the drive-condition setting unit 150 sets emission conditions of the light source 110, and the light-receiving system controller 160 sets the integrated time and gain for the image sensor 2a and the converter circuit 2b (step S81). The light source 110 emits light, which is then projected as spot light through the optical element 11-2. The image sensor 2a receives the reflected spot light for image capturing (step S82). The signal from the image sensor 2a is stored in a storage unit 213. The arithmetic unit 212 calculates a phase signal from the stored signal of the image sensor 2a. The calculation result is stored in the storage unit 213 (step S83).The conversion element 102a that converts the light path of light emitted from the light source 110 is switched (step S84). Thus, the light path emits diffused light via the optical element 11-1 for diffused light. The drive-condition setting unit 150 sets emission conditions of the light source 110, and the light-receiving system controller 160 sets the integrated time and gain for the image sensor 2a and the converter circuit 2b (step S85). At this time, the emission conditions, the integrated time, and the gain may be set by the method described in the sixth to tenth embodiments. The light source 110 emits light, which is then projected as diffused light through the optical element 11-1. The image sensor 2a receives the reflected diffused light for image capturing (step S86). The signal from the image sensor 2a is stored in a storage unit 213. The arithmetic unit 212 calculates a phase signal from the stored signal of the image sensor 2a. The calculation result is stored in the storage unit 213 (step S87). TheFN202304898 phase signal obtained with spot light and the phase signal obtained with diffused light are output from an output unit 214 (step S88).The output phase signal is received by the computer 310a via the cloud 216 (step S89). The phase signals are received via an I / F 301 and is once stored in a storage unit 304 in the computer 310a (step S90). The phase signal captured with diffused light is converted into a distance image (step S91). The phase signal captured with spot light is converted into a distance image (step S92). Correction for multipath interference is performed using each of the distance images (step S93). The corrected distance image is stored in the storage unit 304 (step S94). The distance image is output and transmitted to the cloud 216 (step S95).

[0196] The light emitting and receiving device 200 according to the present embodiment can easily calculate an output phase image using a computer or the like having a large calculation capacity via a communication network such as a cloud. For example, when distance measurement information is generated from a phase signal, it is appropriate to use a large- scale integrated device such as a graphics processing unit (GPU). An object that moves at high speed is captured as an image per short period, and distance measurement data of each image can be created. The motion state of the object that moves at high speed can be analyzed.

[0197] The embodiments according to the present disclosure have been described above; however, the above-described embodiments have been provided as examples, and are not intended to limit the scope of the present disclosure. The new embodiments may be implemented in a variety of other forms; furthermore, various omissions, substitutions, and changes in the forms may be made without departing from the gist and scope of the disclosure. The embodiments and modifications thereof are included in the scope and gist of the disclosure, and are included in the disclosure described in the claims and the equivalents of the disclosure. Components according to different embodiments and modifications may be combined as appropriate.

[0198] As described above, examples of aspects of the present disclosure are as follows.

[0199] <Aspect 1>An image-capturing apparatus 500 includes a first light emitter to emit first light to an object in an area, the first light having a first light intensity distribution in the area; a second light emitter to emit second light to the object in the area, the second light having a second light intensity distribution larger than the first light intensity distribution in the area; and a light receiver to receive first reflected light that is a reflection of the first light reflected by the object in the area; and second reflected light that is a reflection of the second light reflected by the object in the area.<Aspect 2>FN202304898In the image-capturing apparatus according to Aspect 1, the light receiver receives both the first reflected light and the second reflected light.< Aspect 3>In the image-capturing apparatus according to Aspect 1, the light receiver receives: a first light receiver to receive the first reflected light; and a second light receiver to receive the second reflected light.<Aspect 4>In the image-capturing apparatus according to Aspect 1, the light receiver is capturable an image covering 360 degrees around the image-capturing apparatus.< Aspect 5>In the image-capturing apparatus according to Aspect 1, an integrated luminous output of the second light is larger than an integrated luminous output of the first light.<Aspect 6>In the image-capturing apparatus according to Aspect 1, the light receiver performs: a first light reception in response to an emission of the first light by the first light emitter; and a second light reception in response to an emission of the second light by the second light emitter. The second light reception has at least one of an integrated time longer than an integrated time of the first light reception or an amplification factor larger than an amplification factor of the first light reception.<Aspect 7>A distance-measuring apparatus includes: the image-capturing apparatus according to any one of Aspects 1 to 6; and a controller configured to output distance information based on: a detection signal of first reflected light that is the reflection of the first light emitted from the first light emitter to the object; and a detection signal of second reflected light that is the reflection of the second light emitted from the second light emitter to the object, to detect the distance information using a time of flight (ToF) system.<Aspect 8>In the distance-measuring apparatus according to Aspect 7, the controller outputs the distance information at multiple points. A number of the multiple points is larger than a number of points at which distance measurement values are calculated from the detection signal of the second reflected light.<Aspect 9>In the distance-measuring apparatus according to Aspect 7, the controller includes: a distance calculation unit configured to calculate the distance information indicating a distance to the object based on the detection signal of the first reflected light; a correction- value calculation unit configured to calculate a correction value to correct the distance information using a distance image calculated from the detection signal of the first reflected light and a distance image calculated from the detection signal of the second reflected light; and a distance correction unit configured to correct the distance information calculated from the detectionFN202304898 signal of the first reflected light using the correction value calculated by the correction- value calculation unit.<Aspect 10>In the distance-measuring apparatus according to Aspect 7, the controller performs interpolation using the distance information calculated from the detection signal of the second reflected light in a case where only the second reflected light of the structured light emitted from the second light emitter to the object is obtained.<Aspect 11>A distance-measuring system that detects distance information using a time of flight (ToF) system, the distance-measuring system includes: a first light emitter to emit diffused light to an object; a second light emitter to emit structured light to the object at a timing different from a timing of the first light emitter; a light receiver to receive first reflected light of the diffused light emitted from the first light emitter to the object and second reflected light of the structured light emitted from the second light emitter to the object; and a controller to: obtain distance information based a detection signal of the first reflected light; and correct the distance information using a detection signal of the second reflected light.In other words, a distance-measuring system includes: a first light emitter to emit diffused light to an object at a first timing; a second light emitter to emit structured light to the object at a second timing different from the first timing of the first light emitter; a light receiver to: receive first reflected light that is a reflection of the diffused light emitted from the first light emitter to the object; and receive second reflected light that is a reflection of the structured light emitted from the second light emitter to the object; and a controller configured to: obtain distance information based a detection signal of the first reflected light; and correct the distance information using a detection signal of the second reflected light, to detect distance information using a ToF system.< Aspect 12>In a distance-measuring method, performed by a distance-measuring apparatus, the method includes: emitting diffused light to an object; emitting structured light to the object at a timing different from a timing of the emitting of the diffused light; receiving first reflected light of the diffused light emitted to the object in the emitting of the diffused light and second reflected light of the structured light emitted to the object in the emitting of the structured light; and performing control to correct distance information calculated from a detection signal of the first reflected light using a detection signal of the second reflected light, to detect distance information using a ToF system.In other words, a distance-measuring method, performed by a distance-measuring apparatus, includes: emitting diffused light to an object at a first timing; emitting structured light to the object at a second timing different from the first timing of the diffused light; receiving first reflected light that is a reflection of the diffused light emitted to the object in the emitting of the diffused light; receiving second reflected light that is a reflection of the structured light emitted to the object in the emitting of the structured light; calculating distance informationFN202304898 from a detection signal of the first reflected light; and correcting the distance information using a detection signal of the second reflected light, to detect distance information using a ToF system.

[0200] The embodiments according to the present disclosure have been described above; however, the above-described embodiments have been provided as examples, and are not intended to limit the scope of the present disclosure. The new embodiments may be implemented in a variety of other forms; furthermore, various omissions, substitutions, and changes in the forms may be made without departing from the gist and scope of the disclosure. The embodiments and modifications thereof are included in the scope and gist of the disclosure, and are included in the disclosure described in the claims and the equivalents of the disclosure. Components according to different embodiments and modifications may be combined as appropriate.

[0201] The present invention can be implemented in any convenient form, for example using dedicated hardware, or a mixture of dedicated hardware and software. The present invention may be implemented as computer software implemented by one or more networked processing apparatuses. The processing apparatuses include any suitably programmed apparatuses such as a general purpose computer, a personal digital assistant, a Wireless Application Protocol (WAP) or third-generation (3G)-compliant mobile telephone, and so on. Since the present invention can be implemented as software, each and every aspect of the present invention thus encompasses computer software implementable on a programmable device. The computer software can be provided to the programmable device using any conventional carrier medium (carrier means). The carrier medium includes a transient carrier medium such as an electrical, optical, microwave, acoustic or radio frequency signal carrying the computer code. An example of such a transient medium is a Transmission Control Protocol / Internet Protocol (TCP / IP) signal carrying computer code over an IP network, such as the Internet. The carrier medium may also include a storage medium for storing processor readable code such as a floppy disk, a hard disk, a compact disc read-only memory (CD- ROM), a magnetic tape device, or a solid state memory device.The functionality of the elements disclosed herein may be implemented using circuitry or processing circuitry which includes general purpose processors, special purpose processors, integrated circuits, application specific integrated circuits (ASICs), digital signal processors (DSPs), field programmable gate arrays (FPGAs), conventional circuitry and / or combinations thereof which are configured or programmed to perform the disclosed functionality. Processors are considered processing circuitry or circuitry as they include transistors and other circuitry therein. In the disclosure, the circuitry, units, or means are hardware that carry out or are programmed to perform the recited functionality. The hardware may be any hardware disclosed herein or otherwise known which is programmed or configured to carry out the recited functionality. When the hardware is a processor which may be considered aFN202304898 type of circuitry, the circuitry, means, or units are a combination of hardware and software, the software being used to configure the hardware and / or processor.This patent application is based on and claims priority to Japanese Patent Application No. 2023-009736, filed on January 25, 2023 and Japanese Patent Application No. 2023-045879, filed on March 22, 2023, in the Japan Patent Office, the entire disclosure of which is hereby incorporated by reference herein.[Reference Signs List]

[0202] 1-1 first light emitter1-2 second light emitter2 light receiver2-1 light receiver, first light receiver2-2 light receiver, second light receiver4 controller20 first light emitter30 second light emitter60 light receiver100 image-capturing apparatus, distance-measuring apparatus300 distance-measuring system400 external system (controller)403 distance calculation unit404 correction- value calculation unit405 distance correction unit

Claims

FN202304898[CLAIMS]

1. An image-capturing apparatus comprising: a first light emitter to emit first light to an object in an area, the first light having a first light intensity distribution in the area; a second light emitter to emit second light to the object in the area, the second light having a second light intensity distribution larger than the first light intensity distribution in the area; and a light receiver to receive: first reflected light that is a reflection of the first light reflected by the object in the area; and second reflected light that is a reflection of the second light reflected by the object in the area.

2. The image-capturing apparatus according to claim 1, wherein the light receiver receives both the first reflected light and the second reflected light.

3. The image-capturing apparatus according to claim 1, wherein the light receiver includes: a first light receiver to receive the first reflected light; and a second light receiver to receive the second reflected light.

4. The image-capturing apparatus according to claim 1, wherein the light receiver is capturable an image covering 360 degrees around the imagecapturing apparatus.

5. The image-capturing apparatus according to claim 1, wherein an integrated luminous output of the second light is larger than an integrated luminous output of the first light.

6. The image-capturing apparatus according to claim 1, wherein the light receiver performs: a first light reception in response to an emission of the first light by the first light emitter; and a second light reception in response to an emission of the second light by the second light emitter, the second light reception having at least one of an integrated time longer than an integrated time of the first light reception or an amplification factor larger than an amplification factor of the first light reception.

7. A distance-measuring apparatus comprising: the image-capturing apparatus according to any one of claims 1 to 6; and a controller configured to output distance information based on:FN202304898 a detection signal of first reflected light that is the reflection of the first light emitted from the first light emitter to the object; and a detection signal of second reflected light that is the reflection of the second light emitted from the second light emitter to the object, to detect the distance information using a time of flight (ToF) system.

8. The distance-measuring apparatus according to claim 7, wherein the controller outputs the distance information at multiple points, and a number of the multiple points is larger than a number of points at which distance measurement values are calculated from the detection signal of the second reflected light.

9. The distance-measuring apparatus according to claim 7, wherein the controller includes: a distance calculation unit configured to calculate the distance information indicating a distance to the object based on the detection signal of the first reflected light; a correction- value calculation unit configured to calculate a correction value to correct the distance information using a distance image calculated from the detection signal of the first reflected light and a distance image calculated from the detection signal of the second reflected light; and a distance correction unit configured to correct the distance information calculated from the detection signal of the first reflected light using the correction value calculated by the correction-value calculation unit.

10. The distance-measuring apparatus according to claim 7, wherein the controller performs interpolation using the distance information calculated from the detection signal of the second reflected light in a case where only the second reflected light of the structured light emitted from the second light emitter to the object is obtained.

11. A distance-measuring system comprising: a first light emitter to emit diffused light to an object at a first timing; a second light emitter to emit structured light to the object at a second timing different from the first timing of the first light emitter; a light receiver to: receive first reflected light that is a reflection of the diffused light emitted from the first light emitter to the object; and receive second reflected light that is a reflection of the structured light emitted from the second light emitter to the object; and a controller configured to: obtain distance information based a detection signal of the first reflected light; and correct the distance information using a detection signal of the second reflected light,FN202304898 to detect distance information using a ToF system.

12. A distance-measuring method performed by a distance-measuring apparatus, the distancemeasuring method comprising: emitting diffused light to an object at a first timing; emitting structured light to the object at a second timing different from the first timing of the diffused light; receiving first reflected light that is a reflection of the diffused light emitted to the object in the emitting of the diffused light; receiving second reflected light that is a reflection of the structured light emitted to the object in the emitting of the structured light; calculating distance information from a detection signal of the first reflected light; and correcting the distance information using a detection signal of the second reflected light, to detect distance information using a ToF system.

Images