Distance measuring device and distance measuring method

Abstract

A distance measuring device (100) comprises: a light source unit (10) for emitting emitted light; a light receiving unit (20) including a pixel for generating a pixel signal on the basis of incident light; a drive control unit (30) for controlling the drive of the light source unit (10) and the light receiving unit (20); and a signal processing unit (40) for deriving a distance to a target object on the basis of the pixel signal. The drive control unit (30) drives the light source unit (10) and the pixel using a CW-ToF sequence and a pulse ToF sequence for measuring the distance using mutually different types of indirect ToF schemes, and switches between the CW-ToF sequence and the pulse ToF sequence between frames. The signal processing unit (40) derives the distance to the target object on the basis of a first pixel signal generated by the pixel using the CW-ToF sequence, and a second pixel signal generated by the pixel using the pulse ToF sequence. A distance measurement range of the pulse ToF sequence is greater than the distance measurement range of the CW-ToF sequence.

Description

Distance measuring device and distance measuring method

[0001] The present disclosure relates to a distance measuring device and a distance measuring method.

[0002] 2. Description of the Related Art Distance measuring devices that employ an indirect ToF (Time of Flight) method have been known in the past.

[0003] One known indirect ToF method is the continuous wave ToF (hereinafter also referred to as CW-ToF) method, which irradiates an object with modulated light of a predetermined light emission frequency and calculates the distance from the phase difference between the irradiated light and the reflected light from the object (see, for example, Patent Document 1).

[0004] Special Publication No. 2013-538342

[0005] In the CW-ToF method, the distance measurement range is limited by the frequency of the irradiated light, and the lower the frequency, the larger the distance measurement range, while the higher the frequency of the irradiated light, the higher the distance measurement accuracy.

[0006] Furthermore, in the CW-ToF method, when the distance is calculated based on the reflected light from an object located farther than the distance measurement range, the calculated distance is folded over because the phase difference repeats from 0° to 360°. As a result, a false distance that is shorter than the actual distance is calculated, making it impossible to measure the distance accurately.

[0007] Patent Document 1 discloses a technology for expanding the ranging range in a CW-ToF method by irradiating light at multiple frequencies and calculating the distance based on signals acquired based on the irradiated light at each frequency.

[0008] In the technology described in Patent Literature 1, the ranging range is extended based on the relationship between multiple frequencies. However, even when the technology described in Patent Literature 1 is used, if the distance is calculated based on reflected light from an object located farther than the extended ranging range, a false distance that is shorter than the actual distance is calculated, just as in the case where irradiation light of one frequency is used.

[0009] Therefore, the present disclosure provides a distance measuring device and a distance measuring method that can achieve both an expansion of the distance measurement range and an improvement in distance measurement accuracy.

[0010] A distance measuring device according to one aspect of the present disclosure is a distance measuring device that measures the distance to an object using an indirect ToF (Time of Flight) method, and includes: a light source unit that emits irradiation light; a light receiving unit having pixels that generate pixel signals based on incident light; a drive control unit that controls the drive of the light source unit and the light receiving unit; and a signal processing unit that derives the distance to the object based on the pixel signals. The drive control unit drives the light source unit and the pixels with a continuous wave ToF sequence and a pulsed ToF sequence for measuring distance using different types of indirect ToF methods, and switches between the continuous wave ToF sequence and the pulsed ToF sequence between frames. The distance measurement range in the pulsed ToF sequence is longer than the distance measurement range in the continuous wave ToF sequence. The signal processing unit derives the distance to the object based on a first pixel signal generated by the pixel in the continuous wave ToF sequence and a second pixel signal generated by the pixel in the pulsed ToF sequence.

[0011] A ranging method according to one aspect of the present disclosure is a ranging method using a ranging device that measures the distance to an object using an indirect ToF (Time of Flight) method, the ranging device comprising a light source unit that irradiates irradiation light and a light receiving unit having pixels that generate pixel signals based on incident light, the ranging method including: a drive control step that drives the light source unit and the pixels with a continuous wave ToF sequence and a pulse ToF sequence for measuring distance using different types of indirect ToF methods; and a signal processing step that derives the distance to the object based on a first pixel signal generated by the pixel in the continuous wave ToF sequence and a second pixel signal generated by the pixel in the pulse ToF sequence, wherein the drive control step switches between the continuous wave ToF sequence and the pulse ToF sequence between frames, and the ranging range in the pulse ToF sequence is longer than the ranging range in the continuous wave ToF sequence.

[0012] According to the present disclosure, it is possible to achieve both an expansion of the distance measurement range and an improvement in distance measurement accuracy.

[0013] FIG. 1 is a functional block diagram showing an example of the configuration of a distance measuring device according to an embodiment. FIG. 2 is a schematic diagram of a pixel array included in a light receiving unit according to an embodiment. FIG. 3 is a plan view showing an example of the configuration of a pixel according to an embodiment. FIG. 4 is a diagram showing an example of a drive sequence of a distance measuring device according to an embodiment. FIG. 5 is a time chart showing an example of a first CW light emission exposure period and a second CW light emission exposure period in a CW-ToF sequence according to an embodiment. FIG. 6 is a time chart showing an example of a first pulse light emission exposure period and a second pulse light emission exposure period in a pulse ToF sequence according to an embodiment. FIG. 7 is a diagram for explaining estimated distances calculated using the CW-ToF method and the pulse ToF method. FIG. 8 is a diagram for explaining multipath. FIG. 9 is a first diagram for explaining the influence of multipath on the estimated distance. FIG. 10 is a second diagram for explaining the influence of multipath on the estimated distance. FIG. 11 is a third diagram for explaining the influence of multipath on the estimated distance. FIG. 12 is a diagram showing another first example of a drive sequence of a distance measuring device according to an embodiment. FIG. 13 is a diagram showing a second example of a drive sequence of a distance measuring device according to an embodiment. FIG. 14 is a time chart showing an example of a first CW light emission exposure period and a second CW light emission exposure period in another CW-ToF sequence according to an embodiment. FIG. 15 is a plan view showing an example of a pixel configuration according to a first modification of an embodiment. FIG. 16 is a diagram showing an example of a drive sequence of a distance measuring device according to a first modification of an embodiment. FIG. 17 is a time chart showing an example of a first CW light emission exposure period in a CW-ToF sequence according to a first modification of an embodiment. FIG. 18 is a time chart showing an example of a first CW light emission exposure period in another CW-ToF sequence according to a first modification of an embodiment. FIG. 19 is a time chart showing an example of a first pulse light emission exposure period in a pulse ToF sequence according to a first modification of an embodiment. FIG. 20 is a plan view showing an example of a pixel configuration according to a second modification of an embodiment. FIG. 21 is a diagram showing an example of a drive sequence of a distance measuring device according to a second modification of an embodiment. FIG. 22 is a time chart showing an example of a first CW light emission exposure period in a CW-ToF sequence according to the second modification of the embodiment.FIG. 23 is a time chart showing an example of a first pulse light emission exposure period in a pulse ToF sequence according to the second modification of the embodiment.

[0014] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings.

[0015] The embodiments described below are all comprehensive or specific examples. The numerical values, shapes, components, component placement and connection configurations, steps, and step sequences shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, among the components in the following embodiments, components not recited in independent claims are described as optional components. Each figure is a schematic diagram and is not necessarily an exact illustration. In each figure, substantially identical components are designated by the same reference numerals, and redundant descriptions may be omitted or simplified.

[0016] Furthermore, in this specification, terms indicating the relationship between elements, such as perpendicular, parallel, or coincident, terms indicating the shape of elements, such as circular or rectangular, and numerical ranges are not expressions that only express a strict meaning, but are expressions that also include a substantially equivalent range, for example, a difference of about a few percent.

[0017] Furthermore, in this specification, unless otherwise specified, ordinal numbers such as "first" and "second" do not refer to the number or order of components, etc., but are used for the purpose of avoiding confusion and distinguishing between components of the same type.

[0018] (Embodiment) [Configuration] First, the configuration of a distance measuring device according to this embodiment will be described. Fig. 1 is a functional block diagram showing an example of the configuration of a distance measuring device 100 according to this embodiment. Fig. 2 is a schematic diagram of a pixel array included in a light receiving unit 20 according to this embodiment. Fig. 3 is a plan view showing an example of the configuration of a pixel 21 according to this embodiment.

[0019] The distance measuring device 100 is a distance measuring device that measures the distance to an object using an indirect ToF method. The distance measuring device 100 generates, for example, a distance image that indicates the distance to a subject, which is an example of an object.

[0020] The distance measuring device 100 includes a light source unit 10 , a light receiving unit 20 , a drive control unit 30 , and a signal processing unit 40 .

[0021] The light source unit 10 is a light irradiator that irradiates an object with irradiation light in accordance with, for example, an input light emission control signal. The light source unit 10 irradiates, as irradiation light, a plurality of pulsed lights that are repeated at a predetermined duty ratio in accordance with, for example, the timing indicated by a light emission control pulse included in the input light emission control signal.

[0022] In the example shown in FIG. 1 , the light source unit 10 has two light sources: a first light source 11 and a second light source 12. The first light source 11 emits irradiation light in a CW-ToF sequence, which will be described later. The second light source 12 emits irradiation light in a pulsed ToF sequence, which will be described later. The first light source 11 and the second light source 12 each include, for example, a light-emitting element such as a light-emitting diode or a laser element that emits infrared light, and an optical system that receives light from the light-emitting element and controls the light distribution from the light-emitting element. Note that the light source unit 10 is not limited to the example configured with two light sources, and may also be configured with a single light source that emits irradiation light in both the CW-ToF sequence and the pulsed ToF sequence.

[0023] The light receiving unit 20 is configured with an imaging element such as a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor. As shown in Fig. 2, the light receiving unit 20 has a pixel array configured with a plurality of pixels 21 arranged two-dimensionally. For the sake of explanation, Fig. 2 shows a configuration of 4 pixels horizontally and 4 pixels vertically, for a total of 16 pixels, but the number of pixels 21 included in the light receiving unit 20 is not particularly limited. The plurality of pixels 21 have, for example, the same configuration as each other.

[0024] The pixel 21 generates a pixel signal based on incident light. Specifically, the pixel 21 converts the incident light into a signal charge and generates a pixel signal based on the converted signal charge.

[0025] 3, pixel 21 includes a photoelectric conversion unit 22, a plurality of charge accumulation units 23a and 23b, a plurality of charge transfer units 24a and 24b, a charge discharge unit 25, and a discharge control unit 26. The photoelectric conversion unit 22, the plurality of charge accumulation units 23a and 23b, the plurality of charge transfer units 24a and 24b, the charge discharge unit 25, and the discharge control unit 26 are provided on, for example, a semiconductor substrate.

[0026] The photoelectric conversion unit 22 generates signal charges by converting incident light incident on the pixel 21 into signal charges. The incident light incident on the pixel 21 includes, for example, light irradiated from the light source unit 10 and reflected by an object. The photoelectric conversion unit 22 is formed of, for example, a photoelectric conversion element such as a photodiode.

[0027] The plurality of charge accumulation units 23a, 23b each accumulate signal charges converted by the photoelectric conversion unit 22. The plurality of charge transfer units 24a, 24b are provided in one-to-one correspondence with the plurality of charge accumulation units 23a, 23b. In the pixel 21, there are two charge accumulation units and two charge transfer units. The plurality of charge transfer units 24a, 24b are electrically connected to the photoelectric conversion unit 22 and transfer signal charges converted by the photoelectric conversion unit 22 from the photoelectric conversion unit 22 to the plurality of charge accumulation units 23a, 23b. Specifically, the charge transfer unit 24a transfers signal charges to the charge accumulation unit 23a, and the charge transfer unit 24b transfers signal charges to the charge accumulation unit 23b. The plurality of charge transfer units 24a, 24b are, for example, field effect transistors (FETs) formed on a semiconductor substrate. The charge storage portions 23a and 23b are, for example, impurity regions that function as the source or drain of the FET.

[0028] The signal charges accumulated in the charge accumulation units 23 a, 23 b are read out as pixel signals by a signal detection circuit (not shown). The signal detection circuit reads out pixel signals corresponding to, for example, the potentials of the charge accumulation units 23 a, 23 b. The pixel signals read out from each pixel 21 include signals indicating signal values ​​based on the amounts of signal charges accumulated in the charge accumulation units 23 a, 23 b, respectively.

[0029] The charge discharge unit 25 discharges the signal charge converted by the photoelectric conversion unit 22. For example, a predetermined reset voltage is applied to the charge discharge unit 25. The reset voltage may be a ground voltage. The discharge control unit 26 is electrically connected to the photoelectric conversion unit 22 and controls the discharge of the signal charge converted by the photoelectric conversion unit 22 by the charge discharge unit 25. The discharge control unit 26 resets the charge of the photoelectric conversion unit 22 by causing the charge discharge unit 25 to discharge the signal charge. The discharge control unit 26 is, for example, an FET formed on a semiconductor substrate. The charge discharge unit 25 is, for example, an impurity region that functions as the source or drain of the FET.

[0030] 1 again, the drive control unit 30 controls the drive of the light source unit 10 and the light receiving unit 20. The drive control unit 30 outputs, for example, a light emission control pulse that instructs the light source unit 10 to emit irradiation light with a predetermined pulse width as a light emission control signal for controlling the drive of the light source unit 10. The light emission control pulse includes a plurality of pulses that cause the light source unit 10 to repeatedly emit pulsed light. The light source unit 10 emits a plurality of pulsed light as irradiation light at a timing according to the light emission control pulse.

[0031] The drive control unit 30 also outputs, for example, an exposure control pulse that instructs each pixel 21 of the light receiving unit 20 to expose itself as a control signal for controlling the driving of the light receiving unit 20. Each pixel 21 exposes itself to light at a timing according to the exposure control pulse and accumulates signal charge. In this specification, the exposure period refers to a period during which signal charge used for reading out pixel signals is accumulated. Therefore, even if light is incident on a pixel 21 and signal charge is generated, if the signal charge is not used for reading out pixel signals because it is discharged, etc., it is considered non-exposed. For example, exposure refers to a state in which the charge transfer unit 24a or 24b is turned on and signal charge is transferred from the photoelectric conversion unit 22 to the charge accumulation unit 23a or 23b.

[0032] The drive control unit 30 drives the light source unit 10 and each pixel 21 with a CW-ToF sequence and a pulsed ToF sequence for measuring distances at different distance measurement ranges using different types of indirect ToF methods. The CW-ToF sequence is a sequence for performing distance measurement using the CW-ToF method, and the pulsed ToF sequence is a sequence for performing distance measurement using the pulsed ToF method. The CW-ToF method is an indirect ToF method in which the light source unit 10 irradiates a continuous wave whose intensity is modulated at a predetermined period as irradiation light, and performs distance measurement based on the phase difference between the irradiation light irradiated by the light source unit 10 and the reflected light of the irradiation light received by the light receiving unit 20 from an object. The pulsed ToF method is an indirect ToF method in which the light source unit 10 irradiates pulsed light with a predetermined pulse width as irradiation light, and performs distance measurement based on the time difference between the time when the light source unit 10 irradiates the irradiation light and the time when the light receiving unit 20 receives the reflected light of the irradiation light from the object. The ranging range in the pulsed ToF sequence is longer than that in the CW-ToF sequence, and details of the CW-ToF sequence and the pulsed ToF sequence will be described later.

[0033] The signal processing unit 40 performs signal processing on the pixel signals output from the light receiving unit 20. The signal processing unit 40 derives the distance to the object for each pixel 21 based on the pixel signals generated by each pixel 21. Specifically, the signal processing unit 40 derives the distance to the object for each pixel 21 based on the first pixel signals generated by each pixel 21 in the CW-ToF sequence and the second pixel signals generated by each pixel 21 in the pulse ToF sequence. Details of the derivation of the distance by the signal processing unit 40 will be described later.

[0034] The drive control unit 30 and the signal processing unit 40 are processing circuits realized by, for example, a memory that stores a program and a processor that executes the program. Although shown as separate blocks, all or part of the drive control unit 30 and the signal processing unit 40 may be configured with the same memory and processor. The drive control unit 30 and the signal processing unit 40 may also be dedicated logic circuits that perform predetermined processing.

[0035] [Drive Sequence] Next, a description will be given of a drive sequence of the distance measuring device 100 according to the present embodiment. Fig. 4 is a diagram showing an example of a drive sequence of the distance measuring device 100 according to the embodiment.

[0036] The "Sequence" in FIG. 4 shows a drive sequence in which the drive control unit 30 drives the light source unit 10 and the light receiving unit 20 (specifically, each pixel 21). In FIG. 4, the sequences indicated by rectangles with dotted patterns are CW-ToF sequences for performing distance measurement using the CW-ToF method, and the sequences indicated by rectangles without dotted patterns are pulse ToF sequences for performing distance measurement using the pulse ToF method. Furthermore, the "Light Emission Control" in FIG. 4 schematically shows the period in which the drive control unit 30 causes the light source unit 10 to emit irradiation light. Furthermore, the same items as in FIG. 4 are shown in FIGS. 12, 13, 16, and 21, which will be described later.

[0037] As shown in FIG. 4 , the drive control unit 30 drives the light source unit 10 and each pixel 21 using a time-division CW-ToF sequence and a pulse ToF sequence. The drive control unit 30 switches between the CW-ToF sequence and the pulse ToF sequence between frames. The CW-ToF sequence and the pulse ToF sequence each consist of one or more frames (multiple frames in FIG. 4 ). Each frame includes an emission exposure period during which the light source unit 10 irradiates irradiation light and exposes each pixel 21, and a readout period during which pixel signals based on signal charges generated during the emission exposure period are readout. During the readout period in the CW-ToF sequence, first pixel signals generated by each pixel 21 are readout. During the readout period in the pulse ToF sequence, second pixel signals generated by each pixel 21 are readout. During the readout period, each pixel 21 is reset after the pixel signals are readout. Note that when the CW-ToF sequence and the pulse ToF sequence are composed of multiple frames, the multiple frames in each of the CW-ToF sequence and the pulse ToF sequence may or may not be consecutive. For example, multiple frames constituting one of the CW-ToF sequence and the pulse ToF sequence may be consecutive, followed by multiple frames constituting the other. Also, for example, a frame included in one of the CW-ToF sequence and the pulse ToF sequence may be followed by a frame included in the other. In other words, switching between frames of the CW-ToF sequence and the pulse ToF sequence does not have to be performed on a frame-by-frame basis, but may be performed on a frame-by-frame basis.

[0038] 4, the CW-ToF sequence is made up of two types of frames, a first CW-ToF frame Fa1 and a second CW-ToF frame Fa2, in which the timing at which each pixel 21 is exposed differs from the timing at which the light source unit 10 emits irradiation light (specifically, a plurality of pulsed lights as described below). The first CW-ToF frame Fa1 includes a first CW light emission exposure period Sa1 and a readout period following the first CW light emission exposure period Sa1. The second CW-ToF frame Fa2 includes a second CW light emission exposure period Sa2 and a readout period following the second CW light emission exposure period Sa2.

[0039] 4 , the pulse ToF sequence includes two types of frames: a first pulse ToF frame Fb1 and a second pulse ToF frame Fb2, which have different timings for exposing each pixel 21 relative to the timing for irradiating the light source unit 10 with irradiation light (specifically, a plurality of pulsed lights as described below). The first pulse ToF frame Fb1 includes a first pulse emission exposure period Sb1 and a readout period following the first pulse emission exposure period Sb1. The second pulse ToF frame Fb2 includes a second pulse emission exposure period Sb2 and a readout period following the second pulse emission exposure period Sb2.

[0040] The drive control unit 30 repeats a set of one frame each of a first CW-ToF frame Fa1, a second CW-ToF frame Fa2, a first pulse ToF frame Fb1, and a second pulse ToF frame Fb2 a predetermined number of times as a repetition unit. In the example shown in FIG. 4, the first CW-ToF frame Fa1, the second CW-ToF frame Fa2, the first pulse ToF frame Fb1, and the second pulse ToF frame Fb2 are repeated in this order, but the order of the frames within the repetition unit is not particularly limited. In the example shown in FIG. 4, the CW-ToF sequence and the pulse ToF sequence are divided into two types of frames, which reduces the number of charge accumulation units that accumulate signal charges with different exposure timings, making it easier to allocate the signal charges to the charge accumulation units.

[0041] In the example shown in FIG. 4, the drive control unit 30 causes the light source unit 10 to emit irradiation light during each light emission exposure period, but may also cause the light source unit 10 to emit irradiation light during the readout period.

[0042] Next, the first CW light emission exposure period Sa1, the second CW light emission exposure period Sa2, the first pulse light emission exposure period Sb1, and the second pulse light emission exposure period Sb2 will be described in detail.

[0043] First, the first CW light emission exposure period Sa1 and the second CW light emission exposure period Sa2 included in the CW-ToF sequence will be described. Fig. 5 is a time chart showing an example of the first CW light emission exposure period Sa1 and the second CW light emission exposure period Sa2.

[0044] 5 shows an example of a first light emission control pulse (i.e., a waveform of the light emitted by the light source unit 10) that the drive control unit 30 outputs to the light source unit 10 during the first CW light emission exposure period Sa1 and the second CW light emission exposure period Sa2. During the first CW light emission exposure period Sa1 and the second CW light emission exposure period Sa2, a first light emission control pulse of the same waveform is output.

[0045] The drive control unit 30 outputs a first light-emitting control pulse with a first duty ratio to the light source unit 10 during the first CW light-emitting exposure period Sa1 and the second CW light-emitting exposure period Sa2, causing the light source unit 10 to emit pulsed light as irradiation light. In the example shown in FIG. 5 , the frequency of the first light-emitting control pulse is f1. The pulse width of the first light-emitting control pulse is Tp1, and the period of the first light-emitting control pulse is T1, so the first duty ratio of the first light-emitting control pulse is Tp1 / T1. In the example shown in FIG. 5 , the first duty ratio is 50%, but distance measurement using the CW-ToF method is possible as long as the duty ratio is between 25% and 75%.

[0046] 5 shows exposure periods C0, C90, C180, and C270 during which the drive control unit 30 exposes each pixel 21 to light and accumulates signal charge. Specifically, the drive control unit 30 outputs an exposure control pulse to each pixel 21 during the exposure periods C0, C90, C180, and C270 to turn on the charge transfer unit 24a or 24b and transfer the signal charge from the photoelectric conversion unit 22 to the charge accumulation unit 23a or 23b. The exposure periods C0, C90, C180, and C270 are set to periods associated with the first light-emission control pulse.

[0047] The first CW light-emitting exposure period Sa1 includes an exposure period C0 and an exposure period C180, and the exposure periods C0 and C180 are alternately and continuously repeated until the readout period. In Fig. 5, a rectangle with a low-density dot pattern indicates the exposure period C0, and a rectangle with a high-density diagonal line pattern indicates the exposure period C180. The exposure period C0 starts when the phase difference with the first light-emitting control pulse is 0°, and the exposure period C180 starts when the phase difference with the first light-emitting control pulse is 180°. The total length of the exposure period C0 and the exposure period C180 is the same as the period T1 of the first light-emitting control pulse.

[0048] During exposure period C0 and exposure period C180, signal charge is accumulated in different charge accumulation units. For example, during exposure period C0, charge transfer unit 24b and discharge control unit 26 are off, and charge transfer unit 24a is turned on, causing signal charge to accumulate in charge accumulation unit 23a. During exposure period C180, charge transfer unit 24a and discharge control unit 26 are off, and charge transfer unit 24b is turned on, causing signal charge to accumulate in charge accumulation unit 23b.

[0049] In the readout period following the first CW light emission exposure period Sa1, a signal based on the signal charge accumulated in the exposure period C0 and a signal based on the signal charge accumulated in the exposure period C180 are read out as the first pixel signal. Hereinafter, the signal value of the signal corresponding to the exposure period C0 is set to C0, and the signal value of the signal corresponding to the exposure period C180 is set to C180.

[0050] The second CW light-emitting exposure period Sa2 includes an exposure period C90 and an exposure period C270, and the exposure periods C90 and C270 are alternately and continuously repeated until the readout period. In Fig. 5, a rectangle with a dense dot pattern indicates the exposure period C90, and a rectangle with a sparse diagonal line pattern indicates the exposure period C270. The exposure period C90 starts when the phase difference with the first light-emitting control pulse is 90°, and the exposure period C270 starts when the phase difference with the first light-emitting control pulse is 270°. The total length of the exposure period C90 and the exposure period C270 is the same as the period T1 of the first light-emitting control pulse.

[0051] During exposure period C90 and exposure period C270, signal charge is accumulated in different charge accumulation units. For example, during exposure period C90, charge transfer unit 24b and discharge control unit 26 are off, and charge transfer unit 24a is on, causing signal charge to accumulate in charge accumulation unit 23a. During exposure period C270, charge transfer unit 24a and discharge control unit 26 are off, and charge transfer unit 24b is on, causing signal charge to accumulate in charge accumulation unit 23b.

[0052] In the readout period following the second CW light emission exposure period Sa2, a signal based on the signal charge accumulated in the exposure period C90 and a signal based on the signal charge accumulated in the exposure period C270 are read out as parts of the first pixel signal. Hereinafter, the signal value of the signal corresponding to the exposure period C90 is referred to as C90, and the signal value of the signal corresponding to the exposure period C270 is referred to as C270.

[0053] The exposure periods C0, C180, C90, and C270 have the same length, T1 / 2, which is the length obtained by dividing the period of the first light-emitting control pulse by two. During each of the first CW light-emitting exposure period Sa1 and the second CW light-emitting exposure period Sa2, the drive control unit 30 continuously exposes each pixel 21 from the start of exposure of the pixel 21 to the readout of the first pixel signal. Note that there may be an interval between the end of exposure of the pixel 21 and the readout of the first pixel signal. Continuously exposing each pixel 21 means that the exposure is not interrupted midway between the start of exposure of the pixel 21 and the readout of the first pixel signal to drain the signal charge from the photoelectric conversion unit 22. Therefore, the first CW light-emitting exposure period Sa1 and the second CW light-emitting exposure period Sa2 do not include a charge drain period, which will be described later.

[0054] Next, the first pulse light emission exposure period Sb1 and the second pulse light emission exposure period Sb2 included in the pulse ToF sequence will be described. Fig. 6 is a time chart showing an example of the first pulse light emission exposure period Sb1 and the second pulse light emission exposure period Sb2.

[0055] 6 shows an example of a second light-emission control pulse (i.e., a waveform of the light emitted by the light source unit 10) that the drive control unit 30 outputs to the light source unit 10 during the first pulse light-emission exposure period Sb1 and the second pulse light-emission exposure period Sb2. During the first pulse light-emission exposure period Sb1 and the second pulse light-emission exposure period Sb2, second light-emission control pulses of the same waveform are output.

[0056] The drive control unit 30 outputs a second light-emitting control pulse having a second duty ratio to the light source unit 10 during the first pulse light-emitting exposure period Sb1 and the second pulse light-emitting exposure period Sb2, causing the light source unit 10 to emit pulsed light as irradiation light. In the example shown in FIG. 6 , the pulse width of the second light-emitting control pulse is Tp2, and the period of the second light-emitting control pulse is T2, so the second duty ratio of the second light-emitting control pulse is Tp2 / T2. The second duty ratio is, for example, smaller than the first duty ratio. The second duty ratio may be, for example, less than 50% or less than 25%. The pulse width Tp2 of the second light-emitting control pulse is, for example, equal to or greater than the pulse width Tp1 of the first light-emitting control pulse. The pulse width Tp2 of the second light-emitting control pulse may be longer than the pulse width Tp1 of the first light-emitting control pulse. This allows the light source unit 10 to emit pulsed light stably while widening the distance measurement range in the pulse ToF sequence.

[0057] The "charge accumulation" section in FIG. 6 shows exposure periods P0, P1, P2, and P3 during which the drive control unit 30 exposes each pixel 21 to light to accumulate signal charge, and a charge discharge period during which the drive control unit 30 discharges the signal charge from the photoelectric conversion unit 22 in each pixel 21. Specifically, the drive control unit 30 outputs an exposure control pulse to each pixel 21 during the exposure period to turn on the charge transfer unit 24a or 24b, causing the signal charge to be transferred from the photoelectric conversion unit 22 to the charge accumulation unit 23a or 23b. The exposure periods P0, P1, P2, and P3 are set to periods associated with the second light-emission control pulse. Furthermore, the drive control unit 30 outputs a charge discharge pulse to each pixel 21 during the charge discharge period to turn on the discharge control unit 26, causing the signal charge from the photoelectric conversion unit 22 to be discharged to the charge discharge unit 25. The charge discharge period can also be considered a non-exposure period during which the pixels 21 are not exposed to light.

[0058] The first pulse light emission exposure period Sb1 includes an exposure period P0, an exposure period P1, and a charge discharge period, and the exposure period P0, the exposure period P1, and the charge discharge period are repeated in this order until the readout period. The exposure period P0 starts simultaneously with the start of each pulse of the second light emission control pulse. The exposure period P1 starts with a delay of 1×Tp2 from the start of each pulse of the second light emission control pulse, which is the end timing of the exposure period P0. The charge discharge period starts at the end timing of the exposure period P1. The total length of the exposure period P0, the exposure period P1, and the charge discharge period is the same as the period T2 of the second light emission control pulse.

[0059] During exposure period P0 and exposure period P1, signal charge is accumulated in different charge accumulation units. For example, during exposure period P0, charge transfer unit 24b and discharge control unit 26 are off, and charge transfer unit 24a is turned on, causing signal charge to accumulate in charge accumulation unit 23a. During exposure period P1, charge transfer unit 24a and discharge control unit 26 are off, and charge transfer unit 24b is turned on, causing signal charge to accumulate in charge accumulation unit 23b.

[0060] During the charge discharge period following the exposure period P1, the signal charge generated by the photoelectric conversion unit 22 is discharged to the charge discharge unit 25, and no signal charge is accumulated in the charge storage units 23a and 23b. Also, during the charge discharge period, the charge transfer units 24a and 24b are off, and the signal charge accumulated in the charge storage units 23a and 23b is retained in the charge storage units 23a and 23b. The existence of a charge discharge period between the end of one exposure period P0 or P1 and the next exposure period P0 or P1 makes it less likely that aliasing will occur, even if the distance to the target object is long. Note that, during the charge discharge period, as long as the discharge control unit 26 is on for a predetermined period until the end of the charge discharge period, the discharge control unit 26 does not need to be on from the beginning of the charge discharge period.

[0061] In the readout period following the first pulse light emission exposure period Sb1, a signal based on the signal charge accumulated in the exposure period P0 and a signal based on the signal charge accumulated in the exposure period P1 are read out as second pixel signals. Hereinafter, the signal value of the signal corresponding to the exposure period P1 is denoted as P0, and the signal value of the signal corresponding to the exposure period P1 is denoted as P1.

[0062] The second pulse light emission exposure period Sb2 includes an exposure period P2, an exposure period P3, and a charge discharge period, and the exposure period P2, the exposure period P3, and the charge discharge period are repeated in this order until the readout period. The exposure period P2 starts with a delay of 2×Tp2 from the start of each pulse of the second light emission control pulse. The exposure period P3 starts with a delay of 3×Tp2 from the start of each pulse of the second light emission control pulse, which is the end of the exposure period P2. The charge discharge period starts at the end of the exposure period P3. The total length of the exposure period P2, the exposure period P3, and the charge discharge period is the same as the period T2 of the second light emission control pulse.

[0063] During exposure period P2 and exposure period P3, signal charges are accumulated in different charge accumulation units. For example, during exposure period P2, charge transfer unit 24b and discharge control unit 26 are off, and charge transfer unit 24a is turned on, causing signal charges to accumulate in charge accumulation unit 23a. During exposure period P3, charge transfer unit 24a and discharge control unit 26 are off, and charge transfer unit 24b is turned on, causing signal charges to accumulate in charge accumulation unit 23b.

[0064] In the charge discharging period following the exposure period P3, the same operation as in the charge discharging period following the exposure period P1 described above is performed.

[0065] In the readout period following the second pulse light emission exposure period Sb2, a signal based on the signal charge accumulated in the exposure period P2 and a signal based on the signal charge accumulated in the exposure period P3 are read out as second pixel signals. Hereinafter, the signal value of the signal corresponding to the exposure period P2 is denoted as P2, and the signal value of the signal corresponding to the exposure period P3 is denoted as P3.

[0066] The lengths of the exposure periods P0, P1, P2, and P3 are all the same, and are the same as the pulse width Tp2 of the second light-emitting control pulse. The exposure period P0 starts at a timing based on the start of each pulse of the second light-emitting control pulse, and the start of the exposure periods P1, P2, and P3 is delayed by Tp2 in this order from the start of the exposure period P0. Note that the exposure period P0 may start with a predetermined offset from the start of each pulse of the second light-emitting control pulse.

[0067] The drive control unit 30 intermittently exposes each pixel 21 to light from the start of exposure of the pixel 21 until the readout of the second pixel signal during each of the first pulse light emission exposure period Sb1 and the second pulse light emission exposure period Sb2. In other words, a charge discharge period, which is a non-exposure period, exists between the exposure periods. By having such a charge discharge period, even if the pixel 21 receives reflected light during the charge discharge period, the signal charge is discharged and is not read out as a pixel signal, thereby suppressing the generation of a pixel signal that causes distance folding.

[0068] [Deriving Distance to Object] Next, a method in which the signal processing unit 40 derives the distance to the object based on the pixel signals generated by the above drive sequence will be described.

[0069] In deriving the distance to the object, the signal processing unit 40 first calculates a first estimated distance based on the first pixel signal using distance calculation by the CW-ToF method, and then calculates a second estimated distance based on the second pixel signal using distance calculation by the pulse ToF method. The signal processing unit 40 derives the distance to the object based on the first estimated distance and the second estimated distance. The first pixel signal and the second pixel signal are output from each pixel 21 by the above-mentioned drive sequence.

[0070] First, the calculation of the first estimated distance using the CW-ToF method will be described. The signal processing unit 40 calculates the first estimated distance based on the first pixel signals output from each pixel 21 in a CW-ToF sequence including the first CW light emission exposure period Sa1 and the second CW light emission exposure period Sa2 as described above. If the first estimated distance is d1, d1 is calculated using the following equation (1):

[0071]

[0072] Here, c is the speed of light. The above formula (1) is a formula for calculating distance by utilizing the fact that the phase of light reflected by an object shifts relative to the irradiated light depending on the distance to the object. Furthermore, if the ranging range in the CW-ToF sequence is df1, then df1 = c / f1.

[0073] Next, calculation of the second estimated distance using the pulse ToF method will be described. The signal processing unit 40 calculates the second estimated distance based on the second pixel signal output from each pixel 21 in a pulse ToF sequence including the first pulse emission exposure period Sb1 and the second pulse emission exposure period Sb2 as described above. If the second estimated distance is d2, d2 is calculated using the following equation (2):

[0074]

[0075] The above equation (2) is an equation for the case where the pixel 21 receives, during exposure periods P0 and P1, reflected light from an object of pulsed light that returns to the pixel 21 with a delay of time Δt after irradiation of pulsed light by the light source unit 10. In this case, P1 is a signal value corresponding to the reflected light for the time Δt, and P0 is a signal value corresponding to the reflected light for the time obtained by subtracting time Δt from Tp2. Since the ratio of time Δt to the pulse width Tp2 of the pulsed light is P1 / (P0+P1), Δt can be expressed as Δt = Tp2×P1 / (P0+P1). Since the round-trip flight time of the pulsed light to the distance to the object is 2×Δt, d2 can be calculated using the above equation (2).

[0076] When the pixel 21 receives the reflected light during the exposure period P1 and the exposure period P2, d2 is calculated by the following equation (3).

[0077]

[0078] Furthermore, when the pixel 21 receives reflected light during the exposure periods P2 and P3, d2 is calculated by the following equation (4).

[0079]

[0080] For example, the signal processing unit 40 compares P0, P1, P2, and P3, and applies a formula using the signal values ​​corresponding to the two exposure periods with the largest signal values ​​among the signal values ​​corresponding to the two exposure periods whose start timings differ from each other by Tp2. Therefore, if the ranging range in the pulse ToF sequence in the above example is df2, then df2 = (c × 3 × Tp2) / 2.

[0081] Furthermore, in the above formula, the signal values ​​of P0, P1, P2, and P3 that were not used to calculate d2 are signal values ​​corresponding to background light that is incident on pixel 21 during the exposure period, regardless of reflected light, and therefore, in the above formula, the signal values ​​that were not used to calculate d2 may be subtracted from the signal values ​​corresponding to reflected light.

[0082] Next, the signal processing unit 40 determines the smallest value of n among the values ​​of n that minimizes the difference between n×df1+d1 and the second estimated distance d2, where df1 is the ranging range in the CW-ToF sequence and n is an integer greater than or equal to 0.

[0083] FIG. 7 is a diagram illustrating estimated distances calculated using the CW-ToF method and the pulse ToF method. The upper part of FIG. 7 shows a solid-line graph illustrating the relationship between the estimated distance calculated using the CW-ToF method and the actual distance. That is, the upper graph of FIG. 7 shows the relationship between the first estimated distance calculated based on the first pixel signal output from the pixel 21 in the CW-ToF sequence and the actual distance. The lower part of FIG. 7 shows a solid-line graph illustrating the relationship between the estimated distance calculated using the pulse ToF method and the actual distance. That is, the lower graph of FIG. 7 shows the relationship between the second estimated distance calculated based on the second pixel signal output from the pixel 21 in the pulse ToF sequence and the actual distance. The vertical axis of these two graphs represents the calculated estimated distance, and the horizontal axis represents the actual distance. Note that the vertical and horizontal axes of these graphs do not have the same scale, with the horizontal axis having a larger scale.

[0084] As shown in FIG. 7 , the first estimated distance d1 calculated using the CW-ToF method is never greater than the ranging range df1 in the CW-ToF sequence. In the CW-ToF method, the phase difference repeats from 0° to 360°, so the distance folds over each ranging range df1. As a result, the calculated first estimated distance d1 repeats within the ranging range df1 even if the actual distance is longer. Therefore, even if the actual position of the target is farther than the ranging range df1, the first estimated distance d1 is calculated as a value less than the ranging range df1. The value of n above corresponds to the number of times the distance folds over each ranging range df1, and one of the virtual positions where the distance from the ranging device 100 is expressed as n × df1 + d1 is the actual position of the target. Virtual positions other than the actual position are positions corresponding to pseudo-distances.

[0085] On the other hand, in the pulse ToF method, it is possible to set conditions under which distance aliasing does not occur substantially, and distance aliasing is unlikely to occur, so the second estimated distance d2 calculated by the pulse ToF method usually corresponds to the actual position of the target. Therefore, the first estimated distance after addition, after adding the ranging range calculated by n×df1+d1 using the smallest value of n among those that minimize the difference between n×df1+d1 and d2, becomes the distance corresponding to the actual position of the target in the CW-ToF method.

[0086] From the above, the signal processing unit 40 determines, as the distance to the object, either (i) the first estimated distance after addition calculated by n×df1+d1 using the smallest value of n among those that minimize the difference between n×df1+d1 and d2, or (ii) the second estimated distance d2. The signal processing unit 40 determines, for example, as the distance to the object, the first estimated distance after addition calculated by a CW-ToF system, which has a shorter ranging range and higher ranging accuracy than a pulsed ToF system. In particular, continuous wave illumination light such as that of the CW-ToF system can easily be increased in frequency, making it easy to improve ranging accuracy.

[0087] [Effect of Multipath] Next, we will explain how the signal processing unit 40 determines the distance to the target object, taking into account the effect of multipath of the irradiated light. Multipath of the irradiated light is one factor that reduces the distance measurement accuracy in the indirect ToF method. FIG. 8 is a diagram for explaining multipath. The first estimated distance and the second estimated distance after addition calculated above are unlikely to produce an error from the true distance when the pulsed light irradiated by the light source unit 10 travels along a direct path, as shown in FIG. 8 , in which the pulsed light directly strikes the target object OBJ1 and is reflected therefrom. However, in reality, there may be multipaths in which part of the pulsed light irradiated by the light source unit 10 is reflected by another object OBJ2 and then strikes the target object OBJ1 and is reflected therefrom. In the example shown in FIG. 8 , there is only one multipath, but multiple multipaths may exist. The pulsed light traveling along this multipath returns to the distance measuring device 100 later than the pulsed light traveling along the direct path. Therefore, the distance corresponding to the time of flight of the pulsed light traveling along the multipath is longer than the true distance. In reality, the reflected light from the object OBJ returning to the distance measuring device 100 contains a direct path component that travels along the direct path and a multipath component that travels along the multipath, so the first estimated distance and the second estimated distance after addition calculated above will contain an error that makes them longer than the true distance.

[0088] Fig. 9 is a first diagram for explaining the influence of multipath on estimated distance. Fig. 9 shows an example in which the distance range dm of multipath components is relatively narrow. The distance range dm of multipath components is a virtual distance range corresponding to the flight time of each multipath component when multiple multipaths exist. Also, in Fig. 9, the relationship between the calculated estimated distance and the actual distance is shown by a solid line graph, similar to Fig. 7.

[0089] As shown in Figure 9, the distance range dm of the multipath components is longer than the true distance dt. As a result, the first estimated distance de1 and the second estimated distance de2 after addition, which are calculated under the influence of multipath, show values ​​longer than the true distance dt. When the distance range dm of the multipath components is relatively narrow, in the CW-ToF method, the distance range dm of the multipath components and the true distance dt often fall within the ranging range df1. As shown in Figure 9, when the distance range dm of the multipath components and the true distance dt fall within the ranging range df1, the effect of multipath on the estimated distance is the same in the CW-ToF method and the pulsed ToF method, and the first estimated distance de1 and the second estimated distance de2 after addition show the same value.

[0090] Next, a case where the distance range dm of the multipath components is wider than that in Fig. 9 will be described. Fig. 10 is a second diagram for explaining the influence of multipath on the estimated distance. Fig. 10 shows an example where the distance range dm of the multipath components is relatively wide. In Fig. 10, the relationship between the calculated estimated distance and the actual distance is shown by a solid line graph, similar to Fig. 7.

[0091] When the distance range dm of the multipath components is relatively wide, the distance range dm of the multipath components is likely to not fall within the ranging range df1 in the CW-ToF system. As shown in Figure 10, when the distance range dm of the multipath components does not fall within the ranging range df1, part of the distance range dm of the multipath components is folded back to the short-distance side, and this part affects the first estimated distance de1 after addition to be shorter. As a result, the influence of multipath on the estimated distance differs between the CW-ToF system and the pulsed ToF system, and the first estimated distance de1 after addition is likely to be shorter than the second estimated distance de2. Therefore, by using the first estimated distance de1 after addition, a distance value closer to the true distance dt to the target object is derived. Therefore, when the signal processing unit 40 determines either the first estimated distance de1 after addition or the second estimated distance de2 as the distance to the object, taking into account the effects of multipath, determining the first estimated distance de1 after addition as the distance to the object usually tends to reduce the error with the true distance dt.

[0092] Next, a case will be described in which, in the CW-ToF method, more than half of the distance range dm of the multipath components is folded back beyond the ranging range df1. Fig. 11 is a third diagram for explaining the influence of multipath on the estimated distance. Fig. 11 shows an extreme example of a case in which more than half of the distance range dm of the multipath components is folded back beyond the ranging range df1, in which the distance range dm of the multipath components is relatively wide and the entire distance range dm of the multipath components is a folded back component. Also, in Fig. 11, the relationship between the calculated estimated distance and the actual distance is shown by a solid line graph, similar to Fig. 7.

[0093] As shown in FIG. 11 , in the CW-ToF method, when the entire distance range dm of the multipath component becomes an aliased component, the difference between the true distance dt and the distance range dm of the multipath component becomes large within the repeated ranging range df1, and the distance range becomes shorter than the true distance dt. Therefore, the first estimated distance de1 after addition becomes a value significantly shorter than the true distance dt. Therefore, even though the true distance dt exists within the ranging range df1 when n = 2, the difference between the first estimated distance de1 after addition and the second estimated distance de2 when n = 3 is the smallest. In such a case, if the signal processing unit 40 determines the first estimated distance de1 after addition as the distance to the target, the true distance dt will be erroneously detected, resulting in a large error from the true distance dt. Therefore, when the second estimated distance de2 is shorter than the first estimated distance de1 after addition, the signal processing unit 40 determines, for example, the second estimated distance de2 as the distance to the target. This reduces the error between the determined distance and the true distance dt.

[0094] From the above, when the influence of multipath signals described in the examples shown in Figures 9 to 11 is taken into consideration, the signal processing unit 40 can improve the distance measurement accuracy by determining the shorter of the second estimated distance de2 and the first estimated distance de1 after addition as the distance to the object. Furthermore, the signal processing unit 40 may determine that abnormal distance measurement has been performed when the second estimated distance de2 is shorter than the first estimated distance de1 after addition. In this case, the signal processing unit 40 outputs information indicating that abnormal distance measurement has been performed.

[0095] [Effects, etc.] As described above, the distance measuring device 100 according to this embodiment includes a light source unit 10 that emits illumination light, a light receiving unit 20 having pixels 21 that generate pixel signals based on incident light, a drive control unit 30 that controls the driving of the light source unit 10 and the light receiving unit 20, and a signal processing unit 40 that derives the distance to an object based on the pixel signals. The drive control unit 30 drives the light source unit 10 and the pixels 21 with a CW-ToF sequence and a pulsed ToF sequence for measuring distance using different types of indirect ToF methods, and switches between the CW-ToF sequence and the pulsed ToF sequence between frames. The signal processing unit 40 derives the distance to the object based on a first pixel signal generated by the pixel 21 in the CW-ToF sequence and a second pixel signal generated by the pixel 21 in the pulsed ToF sequence. Here, the distance measurement range in the pulsed ToF sequence is longer than the distance measurement range in the CW-ToF sequence.

[0096] As a result, the ranging device 100 can derive the distance to the target object using a first pixel signal generated by a CW-ToF sequence using a CW-ToF system, which has a shorter ranging range than a pulse ToF sequence and is easy to improve ranging accuracy, and a second pixel signal generated by a pulse ToF sequence using a pulse ToF system, which is less likely to cause distance aliasing. Because the second pixel signal is used to derive the distance in addition to the first pixel signal, the ranging range of the ranging device 100 is expanded and the detection of false distances is also suppressed. Therefore, the ranging device 100 can achieve both an expanded ranging range and improved ranging accuracy.

[0097] The distance measurement method by the distance measuring device 100 includes a drive control step of driving the light source unit 10 and the pixels 21 with a CW-ToF sequence and a pulsed ToF sequence for measuring distance using different types of indirect ToF methods, and a signal processing step of deriving the distance to the object based on a first pixel signal generated by the pixel 21 in the CW-ToF sequence and a second pixel signal generated by the pixel 21 in the pulsed ToF sequence. Here, the distance measurement range in the pulsed ToF sequence is longer than the distance measurement range in the CW-ToF sequence.

[0098] This makes it possible to obtain the same effect as that of the distance measuring device 100 described above.

[0099] Also, for example, in the CW-ToF sequence, the drive control unit 30 continuously exposes the pixel 21 from the start of exposure of the pixel 21 until the readout of the first pixel signal, and in the pulse ToF sequence, the drive control unit 30 intermittently exposes the pixel 21 from the start of exposure of the pixel 21 until the readout of the second pixel signal.

[0100] As a result, in the pulse ToF sequence, there is a period during which the pixel is not exposed from the start of exposure to the readout of the second pixel signal, which prevents reflected light from entering the pixel during the exposure period corresponding to the next pulse of the light-emitting control pulse, thereby further suppressing the occurrence of distance folding.

[0101] Also, for example, the light source unit 10 irradiates pulsed light as irradiation light in accordance with an emission control pulse output from the drive control unit 30, and the drive control unit 30 outputs a first emission control pulse with a first duty ratio to the light source unit 10 in the CW-ToF sequence to cause the light source unit 10 to irradiate pulsed light as irradiation light, and outputs a second emission control pulse with a second duty ratio to the light source unit 10 in the pulse ToF sequence to cause the light source unit 10 to irradiate pulsed light as irradiation light.

[0102] This makes it easy for the drive control unit 30 to generate a control signal for the light source unit 10 because the light emission control pulse causes the light source unit 10 to emit pulsed light in both the CW-ToF sequence and the pulse ToF sequence.

[0103] For example, the second duty ratio is smaller than the first duty ratio. For example, the second duty ratio is less than 50%. For example, the second duty ratio is less than 25%. For example, the pulse width of the second light-emitting control pulse is longer than the pulse width of the first light-emitting control pulse.

[0104] These features make it possible to further expand the distance measurement range in the pulse ToF sequence.

[0105] Furthermore, for example, the signal processing unit 40 calculates a first estimated distance based on the first pixel signal, calculates a second estimated distance based on the second pixel signal, and determines either (i) the first estimated distance after addition calculated by n×df1+d1 using the smallest value of n among n's that minimizes the difference between n×df1+d1 and d2 when the first estimated distance is d1, the second estimated distance is d2, the ranging range in the CW-ToF sequence is df1, and n is an integer greater than or equal to 0, and (ii) the second estimated distance as the distance to the object.

[0106] This allows the distance to the target object to be selected from either the first estimated distance after addition calculated using the CW-ToF method or the second estimated distance calculated using the pulse ToF method, making it possible to select a more appropriate estimated distance depending on the situation.

[0107] Also, for example, the signal processing unit 40 determines the shorter of the second estimated distance and the first estimated distance after addition as the distance to the object.

[0108] As a result, even if the calculated estimated distance is longer than the true distance due to the influence of multipath, an estimated distance that is less affected by multipath is selected.

[0109] Furthermore, for example, if the second estimated distance is shorter than the first estimated distance after addition, the signal processing unit 40 determines that abnormal distance measurement has been performed.

[0110] This makes it possible to determine that the distance measurement is abnormal and output the determination result when the second estimated distance becomes shorter than the first estimated distance after addition because the effects of multipath are not correctly reflected.

[0111] Furthermore, for example, pixel 21 includes a photoelectric conversion unit 22 that converts incident light into signal charges, a plurality of charge accumulation units 23a, 23b that accumulate the signal charges converted by photoelectric conversion unit 22, a plurality of charge transfer units 24a, 24b that correspond one-to-one to the plurality of charge accumulation units 23a, 23b and transfer the signal charges converted by photoelectric conversion unit 22 to the plurality of charge accumulation units 23a, 23b, a charge discharge unit 25 that discharges the signal charges converted by photoelectric conversion unit 22, and a discharge control unit 26 that controls the discharge of signal charges by charge discharge unit 25.

[0112] This allows the signal charge of the photoelectric conversion section 22 to be discharged by the charge discharge section 25 and the discharge control section 26, so that the non-exposure period of the pixel 21 can be easily realized.

[0113] [Another Example of Drive Sequence] Next, a description will be given of another example of the drive sequence of the distance measuring device 100 according to the present embodiment. In the following description of the other example of the drive sequence, differences from the description of the drive sequence described above will be mainly described, and descriptions of commonalities will be omitted or simplified.

[0114] First, a first example of another driving sequence will be described. Fig. 12 is a diagram showing a first example of another driving sequence of the distance measuring device 100 according to the embodiment.

[0115] In the example shown in FIG. 12, the CW-ToF sequence includes a first CW-ToF frame Fa1 and a second CW-ToF frame Fa2, similar to the example shown in FIG. 4, and is made up of these two types of frames.

[0116] In addition, in the example shown in Figure 12, the pulse ToF sequence includes a first pulse ToF frame Fb1 that exposes each pixel 21 at a predetermined timing relative to the timing at which the light source unit 10 irradiates the irradiation light, and consists of this one type of frame.

[0117] 12, the drive control unit 30 repeats a set of one frame each of the first CW-ToF frame Fa1, the second CW-ToF frame Fa2, and the first pulse ToF frame Fb1 a predetermined number of times as a repetition unit. In the example shown in FIG. 12, the first CW-ToF frame Fa1, the second CW-ToF frame Fa2, and the first pulse ToF frame Fb1 are repeated in this order, but the order of the frames within the repetition unit is not particularly limited.

[0118] As described above, the drive sequence shown in FIG. 12 has a configuration in which the second pulse ToF frame Fb2 is removed from the drive sequence shown in FIG. 4 . Therefore, the drive sequence time required for distance derivation can be shortened, enabling a ranging image with reduced motion blur to be captured. Furthermore, because the pulse ToF sequence shown in FIG. 12 is composed of the first pulse ToF frame Fb1, the second estimated distance is calculated using only equation (2) among equations (2), (3), and (4) above. Therefore, the ranging range in the pulse ToF sequence shown in FIG. 12 is (c×Tp2) / 2. However, by making the pulse width Tp2 longer than in the pulse ToF sequence shown in FIG. 4 , it is possible to make the ranging range equivalent to that of the pulse ToF sequence shown in FIG. 4 . Note that the method for deriving the distance to the target object using the first and second estimated distances in the drive sequence shown in FIG. 12 is the same as in the drive sequence shown in FIG. 4 .

[0119] Next, a second example of the driving sequence will be described with reference to Fig. 13. Fig. 13 is a diagram showing a second example of the driving sequence of the distance measuring device 100 according to the embodiment.

[0120] The drive sequence shown in Fig. 13 has a configuration in which another CW-ToF sequence is added to the drive sequence shown in Fig. 4. In Fig. 13, the another CW-ToF sequence is indicated by a rectangle with a dot pattern that is less dense than the rectangle with the dot pattern indicating the CW-ToF sequence. In the example shown in Fig. 13, the drive control unit 30 drives the light source unit 10 and each pixel 21 with a time-divided CW-ToF sequence, another CW-ToF sequence, and a pulsed ToF sequence. The another CW-ToF sequence has a longer ranging range than the CW-ToF sequence.

[0121] 13 , the other CW-ToF sequence includes a first CW-ToF frame Fc1 and a second CW-ToF frame Fc2, in which the timing at which each pixel 21 is exposed to light differs from the timing at which the light source unit 10 emits irradiation light. The first CW-ToF frame Fc1 includes a first CW light emission exposure period Sc1 and a readout period following the first CW light emission exposure period Sc1. The second CW-ToF frame Fc2 includes a second CW light emission exposure period Sc2 and a readout period following the second CW light emission exposure period Sc2. During the readout period in the other CW-ToF sequence, a third pixel signal generated by each pixel 21 is read out.

[0122] 13, the drive control unit 30 repeats a set of one frame each of a first CW-ToF frame Fa1, a second CW-ToF frame Fa2, a first CW-ToF frame Fc1, a second CW-ToF frame Fc2, a first pulse ToF frame Fb1, and a second pulse ToF frame Fb2 a predetermined number of times as a repetition unit. In the example shown in FIG. 13, the first CW-ToF frame Fa1, the second CW-ToF frame Fa2, the first CW-ToF frame Fc1, the second CW-ToF frame Fc2, the first pulse ToF frame Fb1, and the second pulse ToF frame Fb2 are repeated in this order, but the order of the frames within the repetition unit is not particularly limited.

[0123] FIG. 14 is a time chart showing an example of the first CW light emission exposure period Sc1 and the second CW light emission exposure period Sc2.

[0124] 14 shows an example of a third light-emission control pulse (i.e., a waveform of the light irradiated by the light source unit 10) that the drive control unit 30 outputs to the light source unit 10 during the first CW light-emission exposure period Sc1 and the second CW light-emission exposure period Sc2. During the first CW light-emission exposure period Sc1 and the second CW light-emission exposure period Sc2, a third light-emission control pulse of the same waveform is output.

[0125] During the first CW light emission exposure period Sc1 and the second CW light emission exposure period Sc2, the drive control unit 30 outputs a third light emission control pulse with a third duty ratio to the light source unit 10, causing the light source unit 10 to emit pulsed light as irradiation light. In the example shown in FIG. 14 , the frequency of the third light emission control pulse is f3. The pulse width of the third light emission control pulse is Tp3, and the period of the third light emission control pulse is T3, resulting in a third duty ratio of Tp3 / T3. In the example shown in FIG. 14 , the third duty ratio is 50%, but distance measurement using the CW-ToF method is possible if the duty ratio is between 25% and 75%. The frequency f3 of the third light emission control pulse is lower than the frequency f1 of the first light emission control pulse. In other words, f1 > f3. Furthermore, when a1 and a3 are different natural numbers, a1 × f1 = a3 × f3 holds.

[0126] 14 shows exposure periods C0, C90, C180, and C270 in which the drive control unit 30 exposes each pixel 21 to light and accumulates signal charge. Except that the exposure periods C0, C90, C180, and C270 in the first CW light emission exposure period Sc1 and the second CW light emission exposure period Sc2 are set to periods associated with the third light emission control pulse, the same driving is performed as for the exposure periods C0, C90, C180, and C270 in the first CW light emission exposure period Sa1 and the second CW light emission exposure period Sa2 described using FIG.

[0127] The first CW light-emitting exposure period Sc1 includes an exposure period C0 and an exposure period C180, and the exposure periods C0 and C180 are alternately repeated continuously until the readout period. The exposure period C0 starts when the phase difference with the third light-emitting control pulse is 0°, and the exposure period C180 starts when the phase difference with the third light-emitting control pulse is 180°. The total length of the exposure periods C0 and C180 is the same as the period T3 of the third light-emitting control pulse.

[0128] The second CW light-emitting exposure period Sc2 includes an exposure period C90 and an exposure period C270, and the exposure periods C90 and C270 are alternately repeated continuously until the readout period. The exposure period C90 starts when the phase difference with the third light-emitting control pulse is 90°, and the exposure period C270 starts when the phase difference with the third light-emitting control pulse is 270°. The total length of the exposure period C90 and the exposure period C270 is the same as the period T3 of the third light-emitting control pulse.

[0129] The exposure periods C0, C180, C90, and C270 have the same length, T3 / 2, which is the length obtained by dividing the period of the third light-emitting control pulse in half. During each of the first CW light-emitting exposure period Sc1 and the second CW light-emitting exposure period Sc2, the drive control unit 30 continuously exposes each pixel 21 from the start of exposure of the pixel 21 to the readout of the third pixel signal. Note that there may be an interval between the end of exposure of the pixel 21 and the readout of the third pixel signal.

[0130] In the drive sequence shown in FIG. 13 , the signal processing unit 40 derives the distance to the object based on the first pixel signal, the second pixel signal, and the third pixel signal. Specifically, in deriving the distance to the object, the signal processing unit 40 first calculates the first estimated distance and the second estimated distance, and then calculates the third estimated distance based on the third pixel signal using distance calculation by the CW-ToF method. The signal processing unit 40 derives the distance to the object based on the first estimated distance, the second estimated distance, and the third estimated distance. For example, the signal processing unit 40 calculates the third estimated distance based on the third pixel signal output from each pixel 21 in another CW-ToF sequence including the first CW light emission exposure period Sc1 and the second CW light emission exposure period Sc2 as described above. If the third estimated distance is d3, d3 is calculated using the following equation (5):

[0131]

[0132] Similarly to equation (1), equation (5) calculates distance by utilizing the fact that the phase of the reflected light from an object shifts relative to the emitted light depending on the distance to the object. Furthermore, if the ranging range in another CW-ToF sequence is df3, then df3 = c / f3.

[0133] Next, when the ranging range in a CW-ToF sequence is df1 and m1 is an integer greater than or equal to 0, the ranging range in another CW-ToF sequence is df3 and m3 is an integer greater than or equal to 0. Among the pairs of m1 and m3 that minimize the difference between m1×df1+d1 and m3×df3+d3, the signal processing unit 40 determines the pair of m1 and m3 that minimizes the difference between m1×df1+d1 and m3×df3+d3. Then, the signal processing unit 40 determines the smallest value of n among the pairs of n that minimize the difference between n×m1×df1+d1 and the second estimated distance d2. In the drive sequence shown in FIG. 13, the unit of return of the estimated distance calculated using the CW-ToF method is m1×df1, which is wider, and therefore erroneous determination of the value of n can be suppressed compared to the drive sequence shown in FIG. 4.

[0134] Finally, the signal processing unit 40 determines, as the distance to the object, either (i) the first estimated distance after addition calculated by n×m1×df1+d1 using the smallest value of n among those that minimize the difference between n×m1×df1+d1 and d2, or (ii) the second estimated distance d2. The selection of the first estimated distance after addition or the second estimated distance by the signal processing unit 40 is the same as in the case of the drive sequence shown in FIG.

[0135] 13, a further CW-ToF sequence may be added in which light emission control pulses are output at frequencies different from frequencies f1 and f3. This further widens the unit of return for the estimated distance calculated using the CW-ToF method, making it possible to further reduce erroneous determinations in determining the value of n.

[0136] The drive sequence of the distance measuring device 100 may further include another pulse ToF sequence having a different distance measurement range from the pulse ToF sequence. In the light emission exposure period in the other pulse ToF sequence, for example, the drive control unit 30 outputs a light emission control pulse that differs from the second light emission control pulse in at least one of the duty ratio, pulse width, and period, and exposes the pixel 21 in the exposure period at the timing corresponding to the light emission control pulse.

[0137] Furthermore, the distance measuring device 100 according to this embodiment may operate with one fixed drive sequence, or may be capable of switching between a plurality of drive sequences. For example, the distance measuring device 100 may be a device that performs only one of the drive sequences shown in Figures 4, 12, and 13, or may be a device that performs two or more of these drive sequences by switching between them.

[0138] [Modification 1] Next, a description will be given of Modification 1 of the embodiment. In the following description of Modification 1, differences from the embodiment will be mainly described, and descriptions of commonalities will be omitted or simplified.

[0139] 15 is a plan view showing an example of the configuration of a pixel 21A according to this modification. The distance measuring device according to this modification has a configuration in which the pixel 21 of the distance measuring device 100 according to the embodiment is replaced with a pixel 21A.

[0140] Pixel 21A differs from pixel 21 in that the number of charge accumulation sections and charge transfer sections in pixel 21A is four. Specifically, pixel 21A includes a photoelectric conversion section 22, a plurality of charge accumulation sections 23a, 23b, 23c, and 23d, a plurality of charge transfer sections 24a, 24b, 24c, and 24d, a charge discharge section 25, and a discharge control section 26. Charge transfer section 24c transfers signal charge to charge accumulation section 23c, and charge transfer section 24d transfers signal charge to charge accumulation section 23d.

[0141] Next, a driving sequence of the distance measuring device according to this modification will be described below with reference to Fig. 16, which is a diagram showing an example of a driving sequence of the distance measuring device according to this modification.

[0142] In the example shown in Fig. 16, the drive control unit 30 drives the light source unit 10 and each pixel 21A using a time-divided CW-ToF sequence, another CW-ToF sequence, and a pulsed ToF sequence. In Fig. 16, the another CW-ToF sequence is indicated by a rectangle with a dot pattern that is less dense than the rectangle with the dot pattern indicating the CW-ToF sequence. The another CW-ToF sequence has a longer ranging range than the CW-ToF sequence.

[0143] 16, the CW-ToF sequence is made up of one type of frame, and includes a first CW-ToF frame Fa3 that exposes each pixel 21A at a predetermined timing relative to the timing of emitting irradiation light from the light source unit 10. The first CW-ToF frame Fa3 includes a first CW light emission exposure period Sa3 and a readout period following the first CW light emission exposure period Sa3.

[0144] 16, another CW-ToF sequence is made up of one type of frame, and includes a first CW-ToF frame Fc3 that exposes each pixel 21A at a predetermined timing relative to the timing of irradiating light from the light source unit 10. The first CW-ToF frame Fc3 includes a first CW light emission exposure period Sc3 and a readout period following the first CW light emission exposure period Sc3.

[0145] 16, the pulse ToF sequence is made up of one type of frame, and includes a first pulse ToF frame Fb3 that exposes each pixel 21A at a predetermined timing relative to the timing of emitting irradiation light from the light source unit 10. The first pulse ToF frame Fb3 includes a first pulse emission exposure period Sb3 and a readout period following the first pulse emission exposure period Sb3.

[0146] The drive control unit 30 repeats a set of a first CW-ToF frame Fa3, a first CW-ToF frame Fc3, and a first pulse-ToF frame Fb3, each of which corresponds to only one frame, a predetermined number of times as a repetition unit. Therefore, compared to the drive sequence shown in FIG. 13 , the time required for the drive sequence to derive the distance can be shortened, and a ranging image with reduced blurring due to motion can be captured. In the example shown in FIG. 16 , the first CW-ToF frame Fa3, the first CW-ToF frame Fc3, and the first pulse-ToF frame Fb3 are repeated in this order, but the order of the frames within the repetition unit is not particularly limited.

[0147] Next, the first CW light emission exposure period Sa3, the first CW light emission exposure period Sc3, and the first pulse light emission exposure period Sb3 will be described in detail.

[0148] First, the first CW light emission exposure period Sa3 included in the CW-ToF sequence will be described. Fig. 17 is a time chart showing an example of the first CW light emission exposure period Sa3.

[0149] 17 shows an example of a first light emission control pulse that the drive control unit 30 outputs to the light source unit 10 during the first CW light emission exposure period Sa3. During the first CW light emission exposure period Sa3, a first light emission control pulse similar to that during the first CW light emission exposure period Sa1 and the second CW light emission exposure period Sa2 shown in FIG.

[0150] The "charge accumulation" section in FIG. 17 shows exposure periods C0, C90, C180, and C270 during which the drive control unit 30 exposes each pixel 21A to light and accumulates signal charge. Specifically, the drive control unit 30 outputs an exposure control pulse to each pixel 21A during the exposure periods C0, C90, C180, and C270 to turn on one of the charge transfer units 24a, 24b, 24c, and 24d, thereby transferring signal charge from the photoelectric conversion unit 22 to one of the charge accumulation units 23a, 23b, 23c, and 23d. The exposure periods C0, C90, C180, and C270 are set to periods associated with the first light-emission control pulse. The phase difference between the start timing of the first light-emission control pulse and the exposure periods C0, C90, C180, and C270 is the same as that of the first CW light-emission exposure period Sa1 and the second CW light-emission exposure period Sa2 shown in FIG. 5.

[0151] The first CW light emission exposure period Sa3 includes exposure periods C0, C90, C180, and C270, which are continuously repeated in this order until the readout period. In the first CW light emission exposure period Sa3, the total length of the exposure periods C0, C90, C180, and C270 is the same as the period T1 of the first light emission control pulse.

[0152] During the first CW light emission exposure period Sa3, signal charge is accumulated in different charge accumulation units during exposure periods C0, C90, C180, and C270. For example, during exposure period C0, charge transfer unit 24a is turned on and signal charge is accumulated in charge accumulation unit 23a. During exposure period C90, charge transfer unit 24b is turned on and signal charge is accumulated in charge accumulation unit 23b. During exposure period C180, charge transfer unit 24c is turned on and signal charge is accumulated in charge accumulation unit 23c. During exposure period C270, charge transfer unit 24d is turned on and signal charge is accumulated in charge accumulation unit 23d.

[0153] In the readout period following the first CW light emitting exposure period Sa3, a signal based on the signal charges accumulated in the exposure period C0, a signal based on the signal charges accumulated in the exposure period C90, a signal based on the signal charges accumulated in the exposure period C180, and a signal based on the signal charges accumulated in the exposure period C270 are read out as the first pixel signals.

[0154] In the first CW light emission exposure period Sa3, the exposure periods C0, C180, C90, and C270 have the same length, which is T1 / 4, the length of the period of the first light emission control pulse divided by four. In the first CW light emission exposure period Sa3, the drive control unit 30 continuously exposes each pixel 21A from the start of exposure of each pixel 21A to readout of the first pixel signal.

[0155] 17, for example, the first CW light emission exposure period Sa3 may alternately include exposure periods C0 and C180 in the first half of the first CW light emission exposure period Sa3, and may alternately include exposure periods C90 and C270 in the second half of the first CW light emission exposure period Sa3. In this case, the lengths of the exposure periods C0, C180, C90, and C270 are T1 / 2, which is the length obtained by dividing the period of the first light emission control pulse by two.

[0156] Next, a first CW light emission exposure period Sc3 included in another CW-ToF sequence will be described. Fig. 18 is a time chart showing an example of the first CW light emission exposure period Sc3.

[0157] 18 shows an example of a third light emission control pulse that the drive control unit 30 outputs to the light source unit 10 during the first CW light emission exposure period Sc3. During the first CW light emission exposure period Sc3, a third light emission control pulse similar to that during the first CW light emission exposure period Sc1 and the second CW light emission exposure period Sc2 shown in FIG.

[0158] 18 shows exposure periods C0, C90, C180, and C270 in which the drive control unit 30 exposes each pixel 21A to light and accumulates signal charge. Except that the exposure periods C0, C90, C180, and C270 in the first CW light emission exposure period Sc3 are set to periods associated with the third light emission control pulse, the exposure periods C0, C90, C180, and C270 are driven in the same manner as the exposure periods C0, C90, C180, and C270 in the first CW light emission exposure period Sa3 described using FIG.

[0159] In the readout period following the first CW light emitting exposure period Sc3, a signal based on the signal charge accumulated in the exposure period C0, a signal based on the signal charge accumulated in the exposure period C90, a signal based on the signal charge accumulated in the exposure period C180, and a signal based on the signal charge accumulated in the exposure period C270 are read out as the third pixel signals.

[0160] In the first CW light emission exposure period Sc3, the exposure periods C0, C180, C90, and C270 have the same length, which is T3 / 4, which is the length obtained by dividing the period of the third light emission control pulse by four. In the first CW light emission exposure period Sc3, the drive control unit 30 continuously exposes each pixel 21A from the start of exposure of each pixel 21A until the readout of the third pixel signal.

[0161] 18, for example, the exposure period C0 and the exposure period C180 may be alternately repeated in the first half of the first CW light emission exposure period Sc3, and the exposure period C90 and the exposure period C270 may be alternately repeated in the second half of the first CW light emission exposure period Sc3. In this case, the lengths of the exposure periods C0, C180, C90, and C270 are T3 / 2, which is the length obtained by dividing the period of the third light emission control pulse by two.

[0162] Next, the first pulse light emission exposure period Sb3 included in the pulse ToF sequence will be described. Fig. 19 is a time chart showing an example of the first pulse light emission exposure period Sb3.

[0163] 19 shows an example of a second light emission control pulse that the drive control unit 30 outputs to the light source unit 10 during the first pulse light emission exposure period Sb3. During the first pulse light emission exposure period Sb3, a second light emission control pulse similar to that during the first pulse light emission exposure period Sb1 and the second pulse light emission exposure period Sb2 shown in FIG.

[0164] The "charge accumulation" section in FIG. 19 shows exposure periods P0, P1, P2, and P3 during which the drive control unit 30 exposes each pixel 21A to light to accumulate signal charge, and charge discharge periods during which the drive control unit 30 discharges signal charge from the photoelectric conversion unit 22 in each pixel 21A. Specifically, the drive control unit 30 outputs an exposure control pulse to each pixel 21A during the exposure period to turn on one of the charge transfer units 24a, 24b, 24c, and 24d, thereby transferring signal charge from the photoelectric conversion unit 22 to one of the charge accumulation units 23a, 23b, 23c, and 23d. The exposure periods P0, P1, P2, and P3 are set to periods associated with the second light-emission control pulse. The relationship between the start timing of the second light-emission control pulse and the exposure periods P0, P1, P2, and P3 is the same as that of the first pulse light-emission exposure period Sb1 and the second pulse light-emission exposure period Sb2 shown in FIG. 6.

[0165] The first pulse light emission exposure period Sb3 includes an exposure period P0, an exposure period P1, an exposure period P2, an exposure period P3, and a charge discharge period, which are repeated in this order until the readout period. The total length of the exposure periods P0, P1, P2, P3, and the charge discharge period is the same as the period T2 of the second light emission control pulse.

[0166] During the first pulse emission exposure period Sb3, signal charge is accumulated in different charge accumulation units during exposure periods P0, P1, P2, and P3. For example, during exposure period P0, charge transfer unit 24a is turned on and signal charge is accumulated in charge accumulation unit 23a. During exposure period P1, charge transfer unit 24b is turned on and signal charge is accumulated in charge accumulation unit 23b. During exposure period P2, charge transfer unit 24c is turned on and signal charge is accumulated in charge accumulation unit 23c. During exposure period P3, charge transfer unit 24d is turned on and signal charge is accumulated in charge accumulation unit 23d.

[0167] In the readout period following the first pulse light emission exposure period Sb3, a signal based on the signal charges accumulated in the exposure period P0, a signal based on the signal charges accumulated in the exposure period P1, a signal based on the signal charges accumulated in the exposure period P2, and a signal based on the signal charges accumulated in the exposure period P3 are read out as second pixel signals.

[0168] In the first pulse light emission exposure period Sb3, the drive control unit 30 causes each pixel 21A to be exposed intermittently from the start of exposure of each pixel 21A until the second pixel signal is read out.

[0169] The signal processing unit 40 in this modification derives the distance to the object in the same manner as in the drive sequence shown in FIG.

[0170] It should be noted that the drive sequence shown in FIG. 17 does not necessarily include another CW-ToF sequence configured from the first CW-ToF frame Fc3.

[0171] The distance measuring device according to this modification can also perform a drive sequence similar to the drive sequence performed by the distance measuring device 100 according to the embodiment.

[0172] [Modification 2] Next, a description will be given of Modification 2 of the embodiment. In the following description of Modification 2, differences from the embodiment and Modification 1 will be mainly described, and descriptions of commonalities will be omitted or simplified.

[0173] 20 is a plan view showing an example of the configuration of a pixel 21B according to this modification. The distance measuring device according to this modification has a configuration in which the pixel 21 of the distance measuring device 100 according to the embodiment is replaced with a pixel 21B.

[0174] Pixel 21B differs from pixel 21 in that the number of charge accumulation sections and charge transfer sections in pixel 21B is different: 3. Specifically, pixel 21B includes a photoelectric conversion section 22, a plurality of charge accumulation sections 23a, 23b, and 23c, a plurality of charge transfer sections 24a, 24b, and 24c, a charge discharge section 25, and a discharge control section 26.

[0175] Next, a driving sequence of the distance measuring device according to this modification will be described below with reference to Fig. 21, which is a diagram showing an example of the driving sequence of the distance measuring device according to this modification.

[0176] In the example shown in FIG. 21, the drive control unit 30 drives the light source unit 10 and each pixel 21B in a time-divided CW-ToF sequence and a pulse ToF sequence.

[0177] 21, the CW-ToF sequence is made up of one type of frame, and includes a first CW-ToF frame Fa4 that exposes each pixel 21B at a predetermined timing relative to the timing of emitting irradiation light from the light source unit 10. The first CW-ToF frame Fa4 includes a first CW light emission exposure period Sa4 and a readout period following the first CW light emission exposure period Sa4.

[0178] 21 , the pulse ToF sequence is made up of one type of frame, and includes a first pulse ToF frame Fb4 that exposes each pixel 21B at a predetermined timing relative to the timing of emitting irradiation light from the light source unit 10. The first pulse ToF frame Fb4 includes a first pulse emission exposure period Sb4 and a readout period following the first pulse emission exposure period Sb4.

[0179] The drive control unit 30 repeats a set including only one first CW-ToF frame Fa4 and one first pulse ToF frame Fb4 a predetermined number of times as a repetition unit. Therefore, compared to the drive sequence shown in FIG. 4, the time required for the drive sequence to derive the distance can be shortened, and a ranging image with reduced blurring due to motion can be captured. In the example shown in FIG. 21, the first CW-ToF frame Fa4 and the first pulse ToF frame Fb4 are repeated in this order, but the order of the frames within the repetition unit is not particularly limited.

[0180] Next, the first CW light emission exposure period Sa4 and the first pulse light emission exposure period Sb4 will be described in detail.

[0181] First, the first CW light emission exposure period Sa4 included in the CW-ToF sequence will be described. Fig. 22 is a time chart showing an example of the first CW light emission exposure period Sa4.

[0182] "Light emission control pulse" in Figure 22 shows an example of a first light emission control pulse that the drive control unit 30 outputs to the light source unit 10 during the first CW light emission exposure period Sa4. During the first CW light emission exposure period Sa4, a first light emission control pulse similar to that during the first CW light emission exposure period Sa1 and the second CW light emission exposure period Sa2 shown in Figure 5 is output. In the example shown in Figure 22, the first duty ratio of the first light emission control pulse is 50%, but distance measurement using the CW-ToF method in this modified example is possible as long as it is between (100 / 3)% and (200 / 3)%.

[0183] 22 shows exposure periods C0, C120, and C240 ​​during which the drive control unit 30 exposes each pixel 21B to light and accumulates signal charge. Specifically, the drive control unit 30 outputs an exposure control pulse to each pixel 21B during the exposure periods C0, C120, and C240 ​​to turn on one of the charge transfer units 24a, 24b, and 24c, thereby transferring the signal charge from the photoelectric conversion unit 22 to one of the charge accumulation units 23a, 23b, and 23c. The exposure periods C0, C120, and C240 ​​are set to periods associated with the first light-emission control pulse.

[0184] The first CW light-emitting exposure period Sa3 includes an exposure period C0, an exposure period C120, and an exposure period C240, which are continuously repeated in this order until the readout period. In FIG. 22 , a rectangle with a sparse dot pattern indicates the exposure period C0, a rectangle with a dense dot pattern indicates the exposure period C120, and a rectangle with a diagonal line pattern indicates the exposure period C240. The exposure period C0 starts when the phase difference with the first light-emitting control pulse is 0°, the exposure period C120 starts when the phase difference with the first light-emitting control pulse is 120°, and the exposure period C240 ​​starts when the phase difference with the first light-emitting control pulse is 240°. The total length of the exposure periods C0, C120, and C240 ​​is the same as the period T1 of the first light-emitting control pulse.

[0185] During exposure period C0, exposure period C120, and exposure period C240, signal charges are accumulated in different charge accumulation units. For example, during exposure period C0, charge transfer unit 24a is turned on and signal charges are accumulated in charge accumulation unit 23a. During exposure period C120, charge transfer unit 24b is turned on and signal charges are accumulated in charge accumulation unit 23b. During exposure period C240, charge transfer unit 24c is turned on and signal charges are accumulated in charge accumulation unit 23c.

[0186] In the readout period following the first CW light emission exposure period Sa4, a signal based on the signal charge accumulated in the exposure period C0, a signal based on the signal charge accumulated in the exposure period C120, and a signal based on the signal charge accumulated in the exposure period C240 ​​are read out as first pixel signals. Hereinafter, the signal value of the signal corresponding to the exposure period C0 is denoted as C0, the signal value of the signal corresponding to the exposure period C120 is denoted as C120, and the signal value of the signal corresponding to the exposure period C240 ​​is denoted as C240.

[0187] The exposure periods C0, C120, and C240 ​​have the same length, which is T1 / 3, which is the length obtained by dividing the period of the first light-emitting control pulse by three. During the first CW light-emitting exposure period Sa4, the drive control unit 30 continuously exposes each pixel 21B from the start of exposure of each pixel 21B until the readout of the first pixel signal.

[0188] Next, the first pulse light emission exposure period Sb4 included in the pulse ToF sequence will be described. Fig. 23 is a time chart showing an example of the first pulse light emission exposure period Sb4.

[0189] 23 shows an example of a second light emission control pulse that the drive control unit 30 outputs to the light source unit 10 during the first pulse light emission exposure period Sb4. During the first pulse light emission exposure period Sb4, a second light emission control pulse similar to that during the first pulse light emission exposure period Sb1 and the second pulse light emission exposure period Sb2 shown in FIG.

[0190] The "charge accumulation" section in Figure 23 shows exposure periods P0, P1, and P2 during which the drive control unit 30 exposes each pixel 21B to light and accumulates signal charge, as well as charge discharge periods during which the signal charge in the photoelectric conversion unit 22 in each pixel 21B is discharged. Specifically, the drive control unit 30 outputs an exposure control pulse to each pixel 21B during the exposure period to turn on one of the charge transfer units 24a, 24b, and 24c, thereby transferring the signal charge from the photoelectric conversion unit 22 to one of the charge accumulation units 23a, 23b, and 23c. The exposure periods P0, P1, and P2 are set to periods associated with the second light-emission control pulse. The relationship between the start timing of the second light-emission control pulse and the exposure periods P0, P1, and P2 is the same as that of the first pulse light-emission exposure period Sb1 and the second pulse light-emission exposure period Sb2 shown in Figure 6.

[0191] The first pulse light emission exposure period Sb4 includes an exposure period P0, an exposure period P1, an exposure period P2, and a charge discharge period, which are repeated in this order until the readout period. The total length of the exposure periods P0, P1, P2, and the charge discharge period is the same as the period T2 of the second light emission control pulse.

[0192] During the first pulse emission exposure period Sb3, signal charges are accumulated in different charge accumulation units during exposure periods P0, P1, and P2. For example, during exposure period P0, charge transfer unit 24a is turned on and signal charges are accumulated in charge accumulation unit 23a. During exposure period P1, charge transfer unit 24b is turned on and signal charges are accumulated in charge accumulation unit 23b. During exposure period P2, charge transfer unit 24c is turned on and signal charges are accumulated in charge accumulation unit 23c.

[0193] In the readout period following the first pulse light emission exposure period Sb4, a signal based on the signal charge accumulated in the exposure period P0, a signal based on the signal charge accumulated in the exposure period P1, and a signal based on the signal charge accumulated in the exposure period P2 are read out as the second pixel signal.

[0194] In the first pulse light emission exposure period Sb4, the drive control unit 30 causes each pixel 21B to be exposed intermittently from the start of exposure of each pixel 21B until the second pixel signal is read out.

[0195] In the drive sequence shown in FIG. 21 , the signal processing unit 40 derives the distance to the object based on the first pixel signal and the second pixel signal. Specifically, in deriving the distance to the object, the signal processing unit 40 first calculates a first estimated distance and a second estimated distance. The signal processing unit 40 derives the distance to the object based on the first estimated distance and the second estimated distance. For example, the signal processing unit 40 calculates the first estimated distance based on the first pixel signal output from each pixel 21B in a CW-ToF sequence including the first CW light emission exposure period Sa4 as described above. The first estimated distance d1 in this modification is calculated using the following equation (6):

[0196]

[0197] Similar to equation (1), equation (6) above is an equation for calculating distance by utilizing the fact that the phase of the light reflected by an object shifts relative to the irradiated light depending on the distance to the object.

[0198] The second estimated distance is calculated using equations (2) and (3) above.

[0199] In the drive sequence shown in FIG. 21, the method of deriving the distance to the object using the first estimated distance and the second estimated distance is the same as in the drive sequence shown in FIG. 4 above.

[0200] The driving sequence shown in Fig. 21 may include one or more other CW-ToF sequences. The driving sequence shown in Fig. 21 may also be performed by the distance measuring device according to the first modification.

[0201] (Others) While the distance measuring device according to one or more aspects of the present disclosure has been described above based on the embodiments, the present disclosure is not limited to the embodiments. As long as it does not deviate from the spirit of the present disclosure, various modifications conceivable by a person skilled in the art to each embodiment and configurations constructed by combining components of different embodiments may also be included within the scope of one or more aspects of the present disclosure.

[0202] Furthermore, the distance measuring device according to the present disclosure does not need to include all of the components described in the above embodiments, and may be configured with only the components required to perform the intended operation.

[0203] In the above-described embodiments, each component may be realized by executing a software program suitable for that component, or by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0204] Furthermore, each component may be realized by hardware. Each component may be a circuit (or integrated circuit). These circuits may form a single circuit as a whole, or each may be a separate circuit. Furthermore, each of these circuits may be a general-purpose circuit or a dedicated circuit.

[0205] Furthermore, the general or specific aspects of the present disclosure may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, or as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium.

[0206] For example, the present disclosure may be realized as a ranging device according to the above-described embodiments, as a control device that controls the ranging device, as a ranging method including steps (processing) performed by components that make up the ranging device, as a program for causing a computer to execute such a ranging method, or as a computer-readable non-transitory recording medium on which such a program is recorded.

[0207] Examples of the distance measuring device and distance measuring method according to the present disclosure that have been described based on the above embodiment are shown below. The distance measuring device and distance measuring method according to the present disclosure are not limited to the following examples.

[0208] For example, a distance measuring device according to a first aspect of the present disclosure is a distance measuring device that measures the distance to an object using an indirect ToF (Time of Flight) method, and includes: a light source unit that irradiates irradiation light; a light receiving unit having pixels that generate pixel signals based on incident light; a drive control unit that controls driving of the light source unit and the light receiving unit; and a signal processing unit that derives the distance to the object based on the pixel signals. The drive control unit drives the light source unit and the pixels with a continuous wave ToF sequence and a pulsed ToF sequence for measuring distance using different types of indirect ToF methods, and switches between the continuous wave ToF sequence and the pulsed ToF sequence between frames. A distance measurement range in the pulsed ToF sequence is longer than a distance measurement range in the continuous wave ToF sequence. The signal processing unit derives the distance to the object based on a first pixel signal generated by the pixel in the continuous wave ToF sequence and a second pixel signal generated by the pixel in the pulsed ToF sequence.

[0209] Also, for example, a distance measuring device according to a second aspect of the present disclosure is a distance measuring device according to the first aspect, wherein the drive control unit continuously exposes the pixel from the start of exposure of the pixel to the readout of the first pixel signal in the continuous wave ToF sequence, and intermittently exposes the pixel from the start of exposure of the pixel to the readout of the second pixel signal in the pulse ToF sequence.

[0210] Furthermore, for example, a distance measuring device according to a third aspect of the present disclosure is a distance measuring device according to the first or second aspect, wherein the light source unit irradiates pulsed light as the irradiation light in accordance with an emission control pulse output from the drive control unit, and the drive control unit outputs a first emission control pulse of a first duty ratio to the light source unit in the continuous wave ToF sequence to cause the light source unit to irradiate pulsed light as the irradiation light, and outputs a second emission control pulse of a second duty ratio to the light source unit in the pulse ToF sequence to cause the light source unit to irradiate pulsed light as the irradiation light.

[0211] Furthermore, for example, a distance measuring device according to a fourth aspect of the present disclosure is the distance measuring device according to the third aspect, wherein the second duty ratio is smaller than the first duty ratio.

[0212] Furthermore, for example, a distance measuring device according to a fifth aspect of the present disclosure is the distance measuring device according to the fourth aspect, in which the second duty ratio is less than 50%.

[0213] Furthermore, for example, a distance measuring device according to a sixth aspect of the present disclosure is the distance measuring device according to the fifth aspect, in which the second duty ratio is less than 25%.

[0214] Also, for example, a distance measuring device according to a seventh aspect of the present disclosure is a distance measuring device according to any one of the third to sixth aspects, in which the pulse width of the second light emission control pulse is longer than the pulse width of the first light emission control pulse.

[0215] Also, for example, a ranging device according to an eighth aspect of the present disclosure is a ranging device according to any one of the first to seventh aspects, wherein the signal processing unit calculates a first estimated distance based on the first pixel signal and calculates a second estimated distance based on the second pixel signal, and determines either (i) the first estimated distance after addition calculated by n×df1+d1 using the smallest value of n among n's that minimizes the difference between n×df1+d1 and d2, where d1 is the first estimated distance, d2 is the second estimated distance, and n is an integer greater than or equal to 0, and (ii) the second estimated distance as the distance to the object.

[0216] Also, for example, a ranging device according to a ninth aspect of the present disclosure is a ranging device according to the eighth aspect, in which the signal processing unit determines the shorter of the second estimated distance and the first estimated distance after addition as the distance to the target object.

[0217] Also, for example, a ranging device according to a tenth aspect of the present disclosure is a ranging device according to the eighth aspect, in which the signal processing unit determines that abnormal ranging has been performed when the second estimated distance is shorter than the first estimated distance after addition.

[0218] Also, for example, a distance measuring device according to an eleventh aspect of the present disclosure is a distance measuring device according to any one of the first to tenth aspects, wherein the pixel includes a photoelectric conversion unit that converts the incident light into a signal charge, a plurality of charge accumulation units that accumulate the signal charge converted by the photoelectric conversion unit, a plurality of charge transfer units that correspond one-to-one to the plurality of charge accumulation units and transfer the signal charge converted by the photoelectric conversion unit to the plurality of charge accumulation units, a charge discharge unit that discharges the signal charge converted by the photoelectric conversion unit, and a discharge control unit that controls the discharge of the signal charge by the charge discharge unit.

[0219] Furthermore, for example, a distance measuring device according to a twelfth aspect of the present disclosure is a distance measuring device according to any one of the first to eleventh aspects, wherein the continuous wave ToF sequence includes a first continuous wave ToF frame and a second continuous wave ToF frame in which the timing for exposing the pixels is different from each other relative to the timing for irradiating the irradiation light from the light source unit, the pulse ToF sequence includes a first pulse ToF frame and a second pulse ToF frame in which the timing for exposing the pixels is different from each other relative to the timing for irradiating the irradiation light from the light source unit, and the drive control unit repeats a set including the first continuous wave ToF frame, the second continuous wave ToF frame, the first pulse ToF frame, and the second pulse ToF frame a predetermined number of times.

[0220] Furthermore, for example, a distance measuring device according to a thirteenth aspect of the present disclosure is a distance measuring device according to any one of the first to eleventh aspects, wherein the continuous wave ToF sequence includes a first continuous wave ToF frame and a second continuous wave ToF frame in which the timing for exposing the pixels relative to the timing for irradiating the irradiation light from the light source unit is different from each other, the pulse ToF sequence includes a first pulse ToF frame that exposes the pixels at a predetermined timing relative to the timing for irradiating the irradiation light from the light source unit, and the drive control unit repeats a set including the first continuous wave ToF frame, the second continuous wave ToF frame, and the first pulse ToF frame a predetermined number of times.

[0221] Furthermore, for example, a distance measuring device according to a fourteenth aspect of the present disclosure is a distance measuring device according to any one of the first to eleventh aspects, wherein the continuous wave ToF sequence includes a first continuous wave ToF frame that exposes the pixel at a predetermined timing relative to the timing at which the light source unit irradiates the irradiation light, the pulse ToF sequence includes a first pulse ToF frame that exposes the pixel at a predetermined timing relative to the timing at which the light source unit irradiates the irradiation light, and the drive control unit repeats a set including the first continuous wave ToF frame and the first pulse ToF frame a predetermined number of times.

[0222] Furthermore, for example, a ranging device according to a fifteenth aspect of the present disclosure is a ranging device according to any one of the first to fourteenth aspects, wherein the drive control unit drives the light source unit and the pixel with the continuous wave ToF sequence, the pulsed ToF sequence, and another continuous wave ToF sequence having a longer ranging range than the continuous wave ToF sequence, and the signal processing unit derives the distance to the object based on the first pixel signal, the second pixel signal, and a third pixel signal generated by the pixel in the other continuous wave ToF sequence.

[0223] Furthermore, for example, a ranging method according to a sixteenth aspect of the present disclosure is a ranging method using a ranging device that measures the distance to an object using an indirect ToF (Time of Flight) method, the ranging device including a light source unit that irradiates irradiation light and a light receiving unit having pixels that generate pixel signals based on incident light, the ranging method including a drive control step of driving the light source unit and the pixels with a continuous wave ToF sequence and a pulsed ToF sequence for measuring distance using different types of indirect ToF methods, and a signal processing step of deriving the distance to the object based on a first pixel signal generated by the pixel in the continuous wave ToF sequence and a second pixel signal generated by the pixel in the pulsed ToF sequence, and the ranging range in the pulsed ToF sequence is longer than the ranging range in the continuous wave ToF sequence.

[0224] The distance measuring device and the like according to the present disclosure can be applied to a variety of applications, such as distance measurement systems, sensing systems using distance images, and recognition systems.

[0225] REFERENCE SIGNS LIST 10 Light source unit 11 First light source 12 Second light source 20 Light receiving unit 21, 21A, 21B Pixel 22 Photoelectric conversion unit 23a, 23b, 23c, 23d Charge storage unit 24a, 24b, 24c, 24d Charge transfer unit 25 Charge discharge unit 26 Discharge control unit 30 Drive control unit 40 Signal processing unit 100 Distance measuring device

Claims

1. A distance measuring device that measures a distance to an object by an indirect ToF (Time of Flight) method, A light source unit that emits irradiation light; a light receiving section having pixels for generating pixel signals based on incident light; A drive control unit that controls driving of the light source unit and the light receiving unit; A signal processing unit that derives a distance to the object based on the pixel signal, The drive control unit drives the light source unit and the pixel with a continuous wave ToF sequence and a pulse ToF sequence for measuring distance using different types of indirect ToF methods; switching between the continuous wave ToF sequence and the pulsed ToF sequence between frames; a ranging range in the pulsed ToF sequence is longer than a ranging range in the continuous wave ToF sequence; The signal processing unit derives a distance to the object based on a first pixel signal generated by the pixel in the continuous wave ToF sequence and a second pixel signal generated by the pixel in the pulse ToF sequence. Ranging device.

2. The drive control unit is In the continuous wave ToF sequence, the pixel is continuously exposed from a start of exposure of the pixel to a readout of the first pixel signal; In the pulse ToF sequence, the pixel is intermittently exposed from a start of exposure of the pixel to a readout of the second pixel signal.

2. A distance measuring device according to claim 1.

3. the light source unit irradiates pulsed light as the irradiation light in accordance with a light emission control pulse output from the drive control unit; The drive control unit is In the continuous wave ToF sequence, a first light emission control pulse having a first duty ratio is output to the light source unit to cause the light source unit to irradiate pulsed light as the irradiation light; In the pulse ToF sequence, a second light emission control pulse having a second duty ratio is output to the light source unit to cause the light source unit to irradiate pulsed light as the irradiation light.

2. A distance measuring device according to claim 1.

4. The second duty ratio is smaller than the first duty ratio.

4. A distance measuring device according to claim 3.

5. The second duty ratio is less than 50%.

5. A distance measuring device according to claim 4.

6. The second duty ratio is less than 25%.

6. A distance measuring device according to claim 5.

7. The pulse width of the second light emission control pulse is longer than the pulse width of the first light emission control pulse.

4. A distance measuring device according to claim 3.

8. The signal processing unit includes: Calculating a first estimated distance based on the first pixel signal; Calculating a second estimated distance based on the second pixel signal; (i) determining, as the distance to the object, either one of (i) the first estimated distance being d1, the second estimated distance being d2, the ranging range in the continuous wave ToF sequence being df1, and n being an integer equal to or greater than 0, a first estimated distance after addition calculated by n×df1+d1 using the smallest value of n among n that minimizes the difference between n×df1+d1 and d2, and (ii) the second estimated distance; A distance measuring device according to any one of claims 1 to 7.

9. The signal processing unit determines the shorter of the second estimated distance and the first estimated distance after addition as the distance to the object.

9. A distance measuring device according to claim 8.

10. The signal processing unit determines that an abnormal distance measurement has been performed when the second estimated distance is shorter than the first estimated distance after the addition.

9. A distance measuring device according to claim 8.

11. The pixel is a photoelectric conversion unit that converts the incident light into a signal charge; a plurality of charge accumulation units that accumulate the signal charges converted by the photoelectric conversion unit; a plurality of charge transfer units each corresponding to one of the plurality of charge accumulation units and configured to transfer the signal charge converted by the photoelectric conversion unit to the plurality of charge accumulation units; a charge discharging section that discharges the signal charge converted by the photoelectric conversion section; a discharge control unit that controls the discharge of the signal charges by the charge discharge unit, A distance measuring device according to any one of claims 1 to 7.

12. The continuous wave ToF sequence includes a first continuous wave ToF frame and a second continuous wave ToF frame in which a timing for exposing the pixel with respect to a timing for irradiating the irradiation light from the light source unit is different from each other, the pulse ToF sequence includes a first pulse ToF frame and a second pulse ToF frame in which a timing at which the pixel is exposed to light with respect to a timing at which the light source unit is caused to irradiate the irradiation light is different from each other, the drive control unit repeats a set including the first continuous wave ToF frame, the second continuous wave ToF frame, the first pulse ToF frame, and the second pulse ToF frame a predetermined number of times. A distance measuring device according to any one of claims 1 to 7.

13. The continuous wave ToF sequence includes a first continuous wave ToF frame and a second continuous wave ToF frame in which a timing for exposing the pixel with respect to a timing for irradiating the irradiation light from the light source unit is different from each other, the pulse ToF sequence includes a first pulse ToF frame for exposing the pixel at a predetermined timing with respect to a timing for irradiating the light source unit with the irradiation light, the drive control unit repeats a set including the first continuous wave ToF frame, the second continuous wave ToF frame, and the first pulse ToF frame a predetermined number of times. A distance measuring device according to any one of claims 1 to 7.

14. the continuous wave ToF sequence includes a first continuous wave ToF frame for exposing the pixel at a predetermined timing with respect to a timing for irradiating the irradiation light from the light source unit, the pulse ToF sequence includes a first pulse ToF frame for exposing the pixel at a predetermined timing with respect to a timing for irradiating the light source unit with the irradiation light, the drive control unit repeats a set including the first continuous wave ToF frame and the first pulse ToF frame a predetermined number of times. A distance measuring device according to any one of claims 1 to 7.

15. the drive control unit drives the light source unit and the pixels with the continuous wave ToF sequence, the pulsed ToF sequence, and another continuous wave ToF sequence having a longer distance measurement range than the continuous wave ToF sequence; The signal processing unit derives a distance to the object based on the first pixel signal, the second pixel signal, and a third pixel signal generated by the pixel in the different continuous wave ToF sequence. A distance measuring device according to any one of claims 1 to 7.

16. A distance measuring method using a distance measuring device that measures a distance to an object by an indirect ToF (Time of Flight) method, comprising: The distance measuring device is A light source unit that emits irradiation light; a light receiving unit having pixels that generate pixel signals based on incident light, The distance measuring method includes: a drive control step of driving the light source unit and the pixels with a continuous wave ToF sequence and a pulse ToF sequence for measuring distance using different types of indirect ToF methods; A signal processing step of deriving a distance to the object based on a first pixel signal generated by the pixel in the continuous wave ToF sequence and a second pixel signal generated by the pixel in the pulse ToF sequence, In the drive control step, the continuous wave ToF sequence and the pulse ToF sequence are switched between frames, A ranging range in the pulsed ToF sequence is longer than a ranging range in the continuous wave ToF sequence. Distance measurement method.