Ranging device and ranging method

Abstract

A ranging device (100) comprises: a light source (10) that emits irradiation light; a light reception unit (20) that has a plurality of pixels; a drive control unit (30) that controls the driving of the light source (10) and the plurality of pixels; and a distance calculation unit (50) that calculates the distance to an object (OBJ) on the basis of signals outputted by the plurality of pixels. The drive control unit (30): classifies the plurality of pixels into a plurality of groups each for exposing pixels therein at the same timing, the plurality of groups including a first group to which first pixels among the plurality of pixels belong and a second group to which second pixels adjacent to the first pixels among the plurality of pixels belong; drives the plurality of pixels such that, in the same frame, the timing of exposure based on emission of irradiation light differs between the plurality of groups; and drives the plurality of pixels such that, across consecutive frames, the timing of exposure based on emission of irradiation light alternates between the plurality of groups.

Description

Distance Measuring Device and Distance Measuring Method

[0001] The present disclosure relates to a distance measuring device and a distance measuring method.

[0002] Conventionally, a distance measuring device that employs an indirect TOF (Time Of Flight) method has been known. A distance measuring device that employs the indirect TOF method includes, for example, a light source and a light receiving unit. Such a distance measuring device receives, with the light receiving unit, reflected light of irradiation light emitted from the light source by an object, and calculates the distance to the object based on a signal based on the reflected light output from the light receiving unit.

[0003] Patent Document 1 discloses a driving method for a solid-state imaging device that performs distance measurement imaging, the driving method including performing distance measurement imaging in which two unit pixels perform an exposure sequence in which exposure starts and exposure ends at mutually different timings based on the start of pulsed light.

[0004] International Publication No. 2017 / 022219

[0005] In the prior art, further improvement in distance measurement accuracy has been demanded.

[0006] The present disclosure provides a distance measuring device and a distance measuring method capable of improving distance measurement accuracy.

[0007] A distance measuring device according to an aspect of the present disclosure includes a light source that emits irradiation light, a light receiving unit having a plurality of pixels that receive reflected light reflected by an object of the irradiation light emitted from the light source, a drive control unit that controls driving of the light source and the plurality of pixels, and a distance calculation unit that calculates the distance to the object based on signals output from the plurality of pixels. The drive control unit classifies the plurality of pixels into a plurality of groups that are exposed at the same timing, the plurality of groups including a first group to which a first pixel among the plurality of pixels belongs and a second group to which a second pixel adjacent to the first pixel among the plurality of pixels belongs, drives the plurality of pixels such that, in the same frame, the timing of exposure based on the emission of the irradiation light is different between the plurality of groups, and drives the plurality of pixels such that, in a plurality of consecutive frames, the timing of exposure based on the emission of the irradiation light is switched between the plurality of groups.

[0008] A distance measuring method according to one aspect of the present disclosure is a distance measuring method using a distance measuring device, the distance measuring device comprising a light source that emits illumination light, and a light receiving unit having a plurality of pixels that receive reflected light reflected by an object from the illumination light emitted from the light source, the distance measuring method includes a drive control step that controls the driving of the light source and the plurality of pixels, and a distance calculation step that calculates the distance to the object based on signals output by the plurality of pixels, the drive control step classifies the plurality of pixels into a plurality of groups to be exposed at the same timing, the plurality of pixels including a first group to which a first pixel belongs and a second group to which a second pixel adjacent to the first pixel belongs, the plurality of pixels are driven so that in the same frame the exposure timing based on the emission of the illumination light differs among the plurality of groups, and the plurality of pixels are driven so that in a plurality of consecutive frames the exposure timing based on the emission of the illumination light is reversed among the plurality of groups.

[0009] According to this disclosure, the accuracy of distance measurement can be improved.

[0010] Figure 1 is a diagram illustrating the phenomenon in which accurate distance values ​​cannot be calculated. Figure 2 is a functional block diagram showing an example of the configuration of a distance measuring device according to the embodiment. Figure 3 is a schematic diagram of a light receiving unit provided in the distance measuring device according to the embodiment. Figure 4 is a plan view showing a schematic configuration of pixels according to the embodiment. Figure 5 is a diagram showing an example of the drive sequence of the distance measuring device according to the embodiment. Figure 6A is a diagram showing an example of the arrangement of the first and second pixels. Figure 6B is a diagram showing an example of the arrangement of the first and second pixels. Figure 6C is a diagram showing an example of the arrangement of the first and second pixels. Figure 6D is a diagram showing an example of the arrangement of the first and second pixels. Figure 7 is a time chart showing an example of the timing of the emission control pulse and exposure control pulse during the A0 / A1 emission exposure period and the A2 / A3 emission exposure period. Figure 8 is a diagram illustrating the signal output by the drive sequence of the distance measuring device according to the embodiment. Figure 9 is a flowchart showing a first example of the operation of the distance measuring device according to the embodiment. Figure 10 is a flowchart showing a second example of the operation of the distance measuring device according to the embodiment. Figure 11 is a flowchart showing a third example of the operation of the distance measuring device according to the embodiment. Figure 12 is a flowchart showing a fourth example of the operation of the distance measuring device according to the embodiment. Figure 13 is a flowchart showing a fifth example of the operation of the distance measuring device according to the embodiment. Figure 14 is a flowchart showing a sixth example of the operation of the distance measuring device according to the embodiment. Figure 15 is a diagram showing another example of the drive sequence of the distance measuring device according to the embodiment. Figure 16 is a time chart showing an example of the timing of the emission control pulse and exposure control pulse during the S0 / S2 emission exposure period and the S1 / S3 emission exposure period. Figure 17 is a functional block diagram showing an example of the configuration of the distance measuring device according to a modified example of the embodiment. Figure 18 is a diagram for explaining the estimation of noise amount and determination of threshold by the threshold determination unit according to a modified example of the embodiment.

[0011] (Background leading to one aspect of this disclosure) Before specifically describing the embodiments of this disclosure, the inventors of this application will explain the problems they have encountered.

[0012] In the indirect TOF (Time-of-Flight) distance measurement method, the pixels of the light-receiving unit store the charge generated by exposure in a charge storage unit, and a signal corresponding to the signal charge stored in the charge storage unit is read out from the pixel. In this distance measurement method, pixels are exposed at different timings based on the start of the illumination light, and the distance to the object is calculated using the signal corresponding to each exposure.

[0013] As described in Patent Document 1, by exposing multiple pixels in the same frame at different timings relative to the start of the illumination light, the number of signals used to calculate the distance to the object can be increased without increasing the number of charge storage units in the pixels. Alternatively, instead of using multiple pixels, the number of signals used to calculate the distance to the object can also be increased without increasing the number of charge storage units in the pixels by changing the exposure timing relative to the illumination light for the same pixel across multiple frames.

[0014] However, when acquiring signals used to calculate the distance to an object using multiple pixels in the same frame or the same pixels in multiple frames, it may become impossible to calculate the accurate distance due to spatial or temporal changes in the distance to the object. Figure 1 illustrates the phenomenon in which an accurate distance value cannot be calculated. Figure 1 shows an example of a time chart when a pixel is exposed during the period from exposure control pulses A0 to A3, which start at a timing based on the emission control pulse that instructs the light source to emit illumination light.

[0015] Exposure control pulses A0 to A3 start with a delay of Tp, the pulse width of the emission control pulse, relative to the start of the emission control pulse. Exposure pulses A0 and A1 instruct exposure to the same frame and the same pixel. Exposure pulses A2 and A3 instruct exposure to the same frame and the same pixel. The pair of exposure control pulses A0 and A1 and the pair of exposure control pulses A2 and A3 instruct exposure to different pixels in the same frame, or to the same pixel in different frames. Reflected light X is the reflected light from the object of the incident light on the pixel corresponding to exposure control pulses A0 and A1 in the frame corresponding to exposure control pulses A0 and A1. Reflected light X is incident on the pixel with a time delay Δtx from the start of the emission control pulse, depending on the distance to the object. Reflected light Y is the reflected light from the object of the incident light on the pixel corresponding to exposure control pulses A2 and A3 in the frame corresponding to exposure control pulses A2 and A3. The reflected light Y is incident on the pixel with a time delay Δty from the start of the light emission control pulse, depending on the distance to the object.

[0016] When there is no temporal or spatial change in the distance to the object, as shown in the upper part of Figure 1, time Δtx and time Δty are the same, and the pixel does not generate a signal based on reflected light X, but generates a signal based on reflected light Y by exposure control pulses A2 and A3. Also, because background light is incident on the pixel, the pixel generates a signal based on background light by exposure control pulses A0 to A3. As a result, the distance to the object can be calculated from the ratio of the signals after subtracting the background light component from the signals generated by exposure control pulses A2 and A3.

[0017] On the other hand, as shown in the lower part of Figure 1, if time Δty becomes shorter than time Δtx due to temporal or spatial changes in the distance to the object, the pixel does not generate a signal based on reflected light X, as described above, but a signal based on reflected light Y is generated only in exposure control pulse A2. Therefore, the signal necessary for calculating the distance to the object cannot be obtained, and the distance to the object cannot be calculated accurately.

[0018] The inventors of the present invention have focused on the fact that such problems arise when acquiring signals used to calculate the distance to an object using multiple pixels or multiple frames, and have come to obtain one aspect of the present disclosure.

[0019] The embodiments of this disclosure will be described in detail below with reference to the drawings.

[0020] The embodiments described below are all comprehensive or specific examples. The numerical values, shapes, components, arrangement and connection configurations of components, steps, and the order of steps shown in the following embodiments are examples only and are not intended to limit the scope of this disclosure. Furthermore, components in the following embodiments that are not described in an independent claim are described as optional components. In addition, each figure is a schematic diagram and is not necessarily a strict illustration. Furthermore, substantially identical components in each figure are denoted by the same reference numerals, and redundant explanations may be omitted or simplified.

[0021] Furthermore, in this specification, terms indicating relationships between elements, such as perpendicular, parallel, or coincident, and terms indicating the shape of elements, such as circular or rectangular, as well as numerical ranges, are not expressions that represent only strict meanings, but also expressions that include substantially equivalent ranges, such as differences of a few percent.

[0022] Furthermore, in this specification, ordinal numbers such as "first," "second," etc., unless otherwise specified, do not refer to the number or order of constituent elements, etc., but are used for the purpose of avoiding confusion and distinguishing similar constituent elements, etc.

[0023] (Embodiment) [Configuration] First, the configuration of the distance measuring device according to this embodiment will be described. Figure 2 is a functional block diagram showing an example of the configuration of the distance measuring device 100 according to this embodiment. Figure 3 is a schematic diagram of the light receiving unit 20 provided in the distance measuring device 100 according to this embodiment. Figure 4 is a plan view showing the schematic configuration of the pixel 21 according to this embodiment.

[0024] The distance measuring device 100 is a distance measuring device that measures distance using an indirect TOF method. The distance measuring device 100 generates a distance image that shows the distance to the object OBJ, for example.

[0025] As shown in Figure 2, the distance measuring device 100 comprises a light source 10, a light receiving unit 20, a drive control unit 30, a change detection unit 40, and a distance calculation unit 50.

[0026] The light source 10 emits illumination light that is irradiated onto the target object OBJ according to the input control signal. The light source 10 emits pulsed light as illumination light according to the timing indicated by the input light emission control pulse, for example. The light source 10 includes, for example, a light-emitting element such as a light-emitting diode or laser element that emits infrared light (IR), and an optical system that receives light from the light-emitting element and controls the light distribution of the light from the light-emitting element.

[0027] The light-receiving unit 20 is composed of an image sensor such as a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor. As shown in Figure 3, the light-receiving unit 20 has a plurality of pixels 21 arranged in a matrix. In Figure 3, for illustrative purposes, it is shown as having a configuration of 4 pixels horizontally and 4 pixels vertically, for a total of 16 pixels, but the number of pixels 21 that the light-receiving unit 20 has is not particularly limited. The plurality of pixels 21 have, for example, the same configuration as each other.

[0028] Each of the multiple pixels 21 generates a signal based on the incident light. More specifically, each of the multiple pixels 21 converts the incident light into a signal charge and generates a signal based on the converted signal charge. The pixels 21 receive the reflected light that is reflected off the object OBJ of the light emitted from the light source 10.

[0029] As shown in Figure 4, the pixel 21 includes a photoelectric conversion unit 22, a plurality of charge storage units 23a and 23b, a plurality of charge transfer units 24a and 24b, a charge discharge unit 25, and a discharge control unit 26. The photoelectric conversion unit 22, the plurality of charge storage units 23a and 23b, the plurality of charge transfer units 24a and 24b, the charge discharge unit 25, and the discharge control unit 26 are provided, for example, on a semiconductor substrate. Note that the pixel 21 does not necessarily have to include the charge discharge unit 25 and the discharge control unit 26.

[0030] The photoelectric conversion unit 22 generates signal charges by converting incident light incident on the pixel 21 into signal charges. The incident light incident on the pixel 21 may include reflected light from the object OBJ of the light emitted from the light source 10. The photoelectric conversion unit 22 is composed of a photoelectric conversion element such as a photodiode.

[0031] Multiple charge storage units 23a and 23b each store the signal charge converted by the photoelectric conversion unit 22. Multiple charge transfer units 24a and 24b are provided in a one-to-one correspondence with the multiple charge storage units 23a and 23b. In the pixel 21, there are two charge storage units and two charge transfer units. Multiple charge transfer units 24a and 24b are electrically connected to the photoelectric conversion unit 22 and transfer the signal charge converted by the photoelectric conversion unit 22 from the photoelectric conversion unit 22 to the multiple charge storage units 23a and 23b. Specifically, charge transfer unit 24a transfers signal charge to charge storage unit 23a, and charge transfer unit 24b transfers signal charge to charge storage unit 23b. Multiple charge transfer units 24a and 24b are, for example, FETs (Field Effect Transistors) formed on a semiconductor substrate. Furthermore, the multiple charge storage areas 23a and 23b are, for example, impurity regions that function as the source or drain of the FET.

[0032] The signal charges accumulated in the multiple charge storage units 23a and 23b are read out as signals by a signal detection circuit (not shown). The signal detection circuit reads out signals corresponding to the potentials of the multiple charge storage units 23a and 23b, for example. The light receiving unit 20 performs an Analog-to-Digital (AD) conversion after reading out the signals and outputs the AD-converted digital signal.

[0033] The charge discharge unit 25 discharges the signal charge converted by the photoelectric conversion unit 22. A predetermined reset voltage is applied to the charge discharge unit 25, for example. The reset voltage may be the power supply voltage or the ground voltage. The discharge control unit 26 is electrically connected to the photoelectric conversion unit 22 and controls the discharge of the signal charge converted by the photoelectric conversion unit 22 by the charge discharge unit 25. The discharge control unit 26 resets the charge of the photoelectric conversion unit 22 by causing the charge discharge unit 25 to discharge the signal charge. The discharge control unit 26 is, for example, an FET formed on a semiconductor substrate. The charge discharge unit 25 is, for example, an impurity region that functions as the source or drain of the FET.

[0034] Referring again to Figure 2, the drive control unit 30 outputs various control signals for controlling the driving of the light source 10 and the light receiving unit 20. As a control signal for controlling the driving of the light source 10, the drive control unit 30 outputs, for example, a light emission control pulse that instructs the light source 10 to emit irradiation light with a predetermined pulse width. The drive control unit 30 also outputs an exposure control pulse that instructs each pixel 21 to be exposed as a control signal for controlling the driving of the plurality of pixels 21 of the light receiving unit 20. Each pixel 21 is exposed for a period of time according to the exposure control pulse, for example, and accumulates signal charge. In this specification, the exposure period means the period for generating signal charge used for reading out the signal. Therefore, even if light is incident on a pixel 21 and signal charge is generated, if the signal charge is discharged or otherwise not used for reading out the pixel signal, it is considered unexposed.

[0035] The change detection unit 40 detects whether there is a temporal change or a spatial change in the distance to the object OBJ at multiple pixels 21. The change detection unit 40 detects whether there is a temporal change or a spatial change in the distance to the object OBJ at multiple pixels 21, for example, based on an evaluation value calculated based on the signals output by the multiple pixels 21. The evaluation value is, for example, a brightness value or a distance value. A temporal change in the distance to the object OBJ means that the distance to the object OBJ at the same pixel 21 changes over time. A spatial change in the distance to the object OBJ means that at the same point in time, the distance to the object OBJ changes due to differences in the position of the pixel 21.

[0036] The distance calculation unit 50 calculates the distance to the object OBJ for each pixel 21 by performing predetermined calculations based on the signals generated by the multiple pixels 21. The distance calculation unit 50 outputs the calculated distance value for each pixel 21 as a pixel value. If signals generated by two or more pixels 21 are used to calculate the distance, the distance to the object OBJ is calculated for each of the two or more pixels 21.

[0037] Furthermore, the distance calculation unit 50 determines, based on the detection result of the change detection unit 40, which of the signals output from the multiple pixels 21 will be used to calculate and output the distance to the target object OBJ.

[0038] The drive control unit 30, change detection unit 40, and distance calculation unit 50 are processing circuits implemented, for example, by a memory for storing a program and a processor for executing the program. Although they are shown separately in the block diagram, all or part of the drive control unit 30, change detection unit 40, and distance calculation unit 50 may be composed of the same memory and processor. Furthermore, at least one of the drive control unit 30, change detection unit 40, and distance calculation unit 50 may be a dedicated logic circuit for performing predetermined processing. Details of the processing performed by the drive control unit 30, change detection unit 40, and distance calculation unit 50 will be described later.

[0039] [Operation] Next, the operation of the distance measuring device 100 according to this embodiment will be described. Specifically, the distance measuring method using the distance measuring device 100 will be described as the operation of the distance measuring device 100.

[0040] The distance measurement method by the distance measuring device 100 generally includes a drive control step that controls the driving of the light source 10 and a plurality of pixels 21, and a distance calculation step that calculates the distance to the target object OBJ based on the signals output by the plurality of pixels 21. The distance measurement method by the distance measuring device 100 may further include a change detection step that detects whether there is a temporal change or a spatial change in the distance to the target object OBJ at the plurality of pixels 21. In this embodiment, the drive control step is performed by the drive control unit 30, the distance calculation step is performed by the distance calculation unit 50, and the change detection step is performed by the change detection unit 40. The details of the distance measurement method by the distance measuring device 100 will be described below.

[0041] First, we will explain the drive sequence of the distance measuring device 100 and the calculation of the distance to the target object OBJ.

[0042] Figure 5 shows an example of the drive sequence of the distance measuring device 100 according to this embodiment. Figure 5 shows the drive sequence when a plurality of pixels 21 output a signal for the distance calculation unit 50 to calculate the distance to the object OBJ using the pulse TOF method. The pulse TOF method is an indirect TOF method in which the light source 10 emits pulsed light of a predetermined pulse width as illumination light, and the distance to the object OBJ is calculated based on the time difference (delay time) between the time the light source 10 emits the illumination light and the time the plurality of pixels 21 receive the reflected light of the illumination light from the object OBJ.

[0043] In the driving sequence when the distance measuring device 100 performs distance measurement, each frame includes a light emission exposure period and a readout period. First, in the light emission exposure period, the emission of the irradiation light by the light source 10 and the exposure of the plurality of pixels 21 are performed. Then, in the readout period, a signal based on the signal charges generated in the plurality of pixels 21 during the light emission exposure period is read out. In the example shown in FIG. 5, each frame includes an A0 / A1 light emission exposure period or an A2 / A3 light emission exposure period and a readout period.

[0044] As shown in FIG. 5, the drive control unit 30 classifies the plurality of pixels 21 into a plurality of groups that expose the plurality of pixels 21 at the same timing, including a first group to which a plurality of first pixels among the plurality of pixels 21 belong and a second group to which a plurality of second pixels among the plurality of pixels 21 belong. Hereinafter, the case where the plurality of pixels 21 are classified into two groups consisting of a first group and a second group will be described, but the plurality of groups may further include another group.

[0045] The drive control unit 30 drives the plurality of pixels 21 so that the exposure timing based on the emission of the irradiation light by the light source 10 is different between the plurality of groups in the same frame. Also, the drive control unit 30 drives the plurality of pixels 21 so that the exposure timing based on the emission of the irradiation light by the light source 10 is switched between the plurality of groups in a plurality of consecutive frames.

[0046] FIGS. 6A to 6D are diagrams showing examples of the arrangements of the first pixel 21A and the second pixel 21B. In FIGS. 6A to 6D, the first pixel 21A is shown as a rectangle without a pattern, and the second pixel 21B is shown as a rectangle with a pattern. As shown in FIGS. 6A to 6D, each first pixel 21A belonging to the first group is arranged so as to be adjacent to at least one second pixel 21B. Also, each second pixel 21B belonging to the second group is arranged so as to be adjacent to at least one first pixel 21A. In the examples shown in FIGS. 6A to 6C, the first pixel 21A and the second pixel 21B are arranged in the column direction or the row direction, respectively. Also, in the example shown in FIG. 6D, in the column direction and the row direction, the first pixel 21A and the second pixel 21B are arranged alternately.

[0047] The drive control unit 30 controls the driving of the light source 10 and the plurality of pixels 21 so as to repeat a plurality of consecutive frames including, for example, the first frame shown in FIG. 5 and the second frame following the first frame. The first frame and the second frame are frames in which the exposure timing based on the emission of the irradiation light by the light source 10 is switched between a plurality of groups.

[0048] In the example shown in FIG. 5, the drive control unit 30 drives the first pixel 21A in the exposure sequence of the A0 / A1 emission exposure period and drives the second pixel 21B in the exposure sequence of the A2 / A3 emission exposure period in the first frame. Also, the drive control unit 30 drives the first pixel 21A in the exposure sequence of the A2 / A3 emission exposure period and drives the second pixel 21B in the sequence of the A0 / A1 emission exposure period in the second frame. That is, in the first pixel 21A and the second pixel 21B, the exposure timing based on the emission of the irradiation light by the light source 10 is switched between the first frame and the second frame. The exposure sequence of the A0 / A1 emission exposure period is an example of the first exposure sequence, and the exposure sequence of the A2 / A3 emission exposure period is an example of the second exposure sequence.

[0049] Here, we will explain the details of each emission exposure period. Figure 7 is a time chart showing examples of the timing of emission control pulses and exposure control pulses during the A0 / A1 emission exposure period and the A2 / A3 emission exposure period. During each emission exposure period, for example, the emission control pulses and exposure control pulses shown in Figure 7 are repeatedly output from the drive control unit 30, and during the readout period described above, signals based on multiple emission of light and multiple exposures are read out from the pixel 21.

[0050] As shown in Figure 7, the drive control unit 30 drives the light source 10 with light emission control pulses and drives the multiple pixels 21 with exposure control pulses A0 to A3. The light emission control pulse is a control pulse that instructs the light source 10 to emit illumination light, and in the example shown in Figure 7, the light source 10 emits illumination light during the high-level period. The exposure control pulses A0 to A3 are control pulses that instruct the exposure of the pixels 21, and in the example shown in Figure 7, the pixels 21 are exposed during the low-level period. Note that as long as the emission period of illumination light from the light source 10 and the exposure period of the pixels 21 can be defined, the control signals for controlling the light source 10 and the multiple pixels 21 are not particularly limited.

[0051] As shown in Figure 7, the drive control unit 30 outputs a light emission control pulse with a pulse width Tp. The drive control unit 30 also outputs exposure control pulses A0 to A3 at timings relative to the light emission control pulse. In the example shown in Figure 7, the pulse widths of exposure control pulses A0 to A3 are equal to the pulse width Tp of the light emission control pulse. Furthermore, exposure control pulses A0 to A3 instruct the pixels 21 to expose the pixels 21 such that the exposure timings relative to the emission of light from the light source 10 are different. Exposure control pulses A0 to A3 are delayed by a pulse width Tp in the order of exposure control pulses A0 to A3. Specifically, exposure control pulse A0 starts simultaneously with the start of the light emission control pulse. Exposure control pulse A1 starts with a pulse width Tp delay from the start of exposure control pulse A0. Exposure control pulse A2 starts with a pulse width Tp delay from the start of exposure control pulse A1. Exposure control pulse A3 starts with a pulse width Tp delay from the start of exposure control pulse A2. Furthermore, if the distance to the object's oblique joint (OBJ) can be calculated using the pulsed TOF method, there are no particular restrictions on the timing of each exposure control pulse or the number of exposure control pulses with different timings. For example, exposure control pulse A0 does not have to start simultaneously with the start of the light emission control pulse. Exposure control pulse A0 may start, for example, with a predetermined offset amount after the start of the light emission control pulse.

[0052] As shown in Figure 7, during the A0 / A1 emission exposure period, the drive control unit 30 outputs emission control pulses and exposure control pulses A0 and A1. The low-level periods of exposure control pulse A0 and exposure control pulse A1 do not overlap. The signal charge generated by exposure control pulse A0 is stored in one of the charge storage units 23a and 23b, and the signal charge generated by exposure control pulse A1 is stored in the other of the charge storage units 23a and 23b. Furthermore, during the A2 / A3 emission exposure period, the drive control unit 30 outputs emission control pulses and exposure control pulses A2 and A3. The low-level periods of exposure control pulse A2 and exposure control pulse A3 do not overlap. The signal charge generated by exposure control pulse A2 is stored in one of the charge storage units 23a and 23b, and the signal charge generated by exposure control pulse A3 is stored in the other of the charge storage units 23a and 23b. In Figure 7, light emission control pulses are shown for both the A0 / A1 and A2 / A3 light emission exposure periods for clarity. As shown in Figure 5, in the same frame, the A0 / A1 and A2 / A3 light emission exposure periods are performed in parallel for the first and second groups. Therefore, the drive control unit 30 outputs common light emission control pulses for both the A0 / A1 and A2 / A3 light emission exposure periods in the same frame. Note that the two of the four exposure control pulses A0 to A3 that are output in each light emission exposure period are not limited to the example shown in Figure 7.

[0053] The distance calculation unit 50 calculates the distance to the object OBJ based on the signals generated by each of the multiple pixels 21 by exposure control pulses A0 to A3. Specifically, the distance calculation unit 50 calculates the distance Z to the object OBJ using one of the following equations (1) to (3), which is based on the delay time from the emission of the irradiated light to the reception of the reflected light.

[0054]

[0055]

[0056]

[0057] Here, c is the speed of light, and Tp is the pulse width of the light emission control pulse. A0 to A3 are the signal values ​​of the signals generated by the pixel 21 by the exposure control pulses A0 to A3, respectively.

[0058] The distance calculation unit 50 uses equation (1) to calculate distance Z when the pixel 21 receives reflected light during the exposure control pulses A0 and A1 (in other words, when A0 and A1 in the above equations are greater than A2 and A3). The distance calculation unit 50 also uses equation (2) to calculate distance Z when the pixel 21 receives reflected light during the exposure control pulses A1 and A2 (in other words, when A1 and A2 in the above equations are greater than A0 and A3). The distance calculation unit 50 also uses equation (3) to calculate distance Z when the pixel 21 receives reflected light during the exposure control pulses A2 and A3 (in other words, when A2 and A3 in the above equations are greater than A0 and A1).

[0059] In equations (1) to (3) above, the signal value of the signal generated by the exposure control pulse during the period when the pixel 21 is not receiving reflected light is subtracted from the signal value of the signal generated by the exposure control pulse during the period when the pixel 21 is receiving reflected light. Therefore, it is possible to eliminate the influence of background light that the pixel 21 is constantly receiving when calculating the distance Z. However, depending on the environment in which the distance measuring device 100 is used, the distance Z may be calculated using an equation that does not subtract the signal due to background light.

[0060] In the drive sequence of the distance measuring device 100, the first frame and the second frame are repeated alternately. Figure 8 is a diagram illustrating the signals output by the drive sequence of the distance measuring device 100 according to this embodiment. Figure 8 shows what kind of signals are output from the pixel 21 depending on the pixel position and time. In Figure 8, "A0 / A1" indicates that the signals generated by exposure control pulses A0 and A1, respectively, are output from the pixel 21. Also, "A2 / A3" indicates that the signals generated by exposure control pulses A2 and A3, respectively, are output from the pixel 21.

[0061] As shown in Figure 8, in the first and second frames, signals generated by exposure control pulses A0 to A3 are output from the same pixel 21. Also, in the same frame, signals generated by exposure control pulses A0 to A3 are output from the first pixel 21A and the second pixel 21B, respectively. As a result, the distance calculation unit 50 can calculate the distance to the object OBJ based on the signals output from the same pixel 21 in the first and second frames, and the distance to the object OBJ based on the signals output from the first pixel 21A and the second pixel 21B in the same frame. Here, the signals output from the same pixel 21 in the first and second frames are signals output when there is no spatial change in the distance to the object OBJ. Also, the signals output from the first pixel 21A and the second pixel 21B in the same frame are signals output when there is no temporal change in the distance to the object OBJ. Therefore, by causing the drive control unit 30 to drive the light source 10 and the plurality of pixels 21 as shown in Figures 5 to 8, it is possible to use a signal that is output in a state where there is no spatial or temporal change in the distance to the object OBJ when calculating the distance to the object OBJ.

[0062] Next, an example of the operation of the distance measuring device 100 will be described when the distance calculation unit 50 determines, based on the detection result of the change detection unit 40, which signal from among the signals output from multiple pixels 21 will be used to calculate the distance to the object OBJ. The example of operation described below is an example of the operation of the distance measuring device 100 when calculating the distance to the object OBJ at the first pixel 21A after the signal readout in the second second frame (i.e., the latest second frame) shown in Figure 8 has been completed. The distance measuring device 100 can also perform the operation to calculate the distance to the object OBJ at the second pixel 21B by swapping the first pixel 21A and the second pixel 21B in the following example of operation. For example, the distance measuring device 100 performs the following operations to calculate the distance to the object OBJ at each of the multiple pixels 21. The distance measuring device 100 may also have two or more operating modes corresponding to two or more of the multiple examples of operation described below.

[0063] First, a first example of the operation of the distance measuring device 100 will be described. Figure 9 is a flowchart showing a first example of the operation of the distance measuring device 100 according to this embodiment.

[0064] As shown in Figure 9, first, the change detection unit 40 detects whether there is a temporal change in the distance from the target pixel 21 to the target object OBJ at one or more target pixels 21, including the first pixel 21A (step S11). The one or more target pixels 21 are, for example, the first pixel 21A. In step S11, the change detection unit 40 detects, for example, that there is a temporal change in the distance from the first pixel 21A to the target object OBJ when the temporal difference in the brightness value calculated based on the signal output by the first pixel 21A in each of the consecutive first and second frames is greater than a preset threshold.

[0065] The luminance value is the sum of the signal values ​​of the signals generated by exposure control pulses A0 to A3, each with different timings, specifically A0 + A1 + A2 + A3. In step S11, the luminance value is the sum of the signal values ​​of the signals output by the first pixel 21A in each of the consecutive first and second frames (A0 + A1 + A2 + A3). The temporal difference in the luminance value is, for example, the difference between the current luminance value based on the signal enclosed by the dashed line t1 output by the first pixel 21A in each of the most recent first and second frames shown in Figure 8, and the past luminance value based on the signal enclosed by the dashed line t0 output by the first pixel 21A in each of the first and second frames preceding the most recent. As the distance to the object OBJ changes over time, the amount of reflected light received by the first pixel 21A also changes. Therefore, the temporal difference in the luminance value allows detection of whether or not there has been a temporal change in the distance to the object OBJ. For example, the greater the distance to the object OBJ, the more the reflected light is attenuated, and the amount of reflected light received by the first pixel 21A decreases. Therefore, if the distance to the object OBJ changes over time to increase, the calculated brightness value decreases. Note that some of the signals used to calculate the difference in brightness values ​​over time may be duplicated. For example, the signal output by the first pixel 21A in the latest first frame within the dashed line t1 shown in Figure 8 may be used to calculate both the current brightness value and the past brightness value.

[0066] In step S11, the one or more target pixels 21 may be two pixels 21, a first pixel 21A and a second pixel 21B adjacent to the first pixel 21A. More specifically, the change detection unit 40 may detect whether or not there is a temporal change in the distance to the target object OBJ at the first pixel 21A and the distance to the target object OBJ at the second pixel 21B. In this case, the change detection unit 40 detects not only whether or not there is a temporal change in the distance to the target object OBJ at the first pixel 21A, but also whether or not there is a temporal change in the distance to the target object OBJ at the second pixel 21B, and if it detects that there is a temporal change in either one or both of the distances, it determines that there is a temporal change in the distance to the target object OBJ.

[0067] The change detection unit 40 detects, for example, that there is a temporal change in the distance to the object OBJ at the second pixel 21B when the temporal difference in the brightness value calculated based on the signal output by the second pixel 21B in each of the consecutive first and second frames is greater than a preset threshold. The temporal difference in the brightness value at the second pixel 21B is calculated in the same way as the calculation of the temporal difference in the brightness value at the first pixel 21A.

[0068] If the change detection unit 40 detects that there is no temporal change in the distance to the object OBJ at each of the one or more target pixels 21 (No in step S11), the distance calculation unit 50 calculates and outputs a first distance as the distance to the object OBJ at the first pixel 21A (step S12). The first distance is the distance to the object OBJ calculated based on the signals output by the same pixel 21 in each of the consecutive first and second frames. In step S12, the distance calculation unit 50 calculates the first distance at the first pixel 21A using any of the above formulas (1) to (3). At this time, the distance calculation unit 50 uses the signals generated by the first pixel 21A in the latest first frame by exposure control pulses A0 and A1, respectively, and the signals generated by the first pixel 21A in the latest second frame by exposure control pulses A2 and A3, respectively, which are enclosed by the dashed line t1 in Figure 8, to calculate the first distance.

[0069] On the other hand, if the change detection unit 40 detects that there is a temporal change in the distance to the object OBJ in at least one of the one or more target pixels 21 (Yes in step S11), the distance calculation unit 50 calculates and outputs a second distance as the distance to the object OBJ at the first pixel 21A (step S13). The second distance is the distance to the object OBJ calculated based on the signals output by the first pixel 21A and the second pixel 21B in the same frame. The distance calculation unit 50 calculates the second distance using any of the above equations (1) to (3). In this case, the distance calculation unit 50 uses the signals generated by the first pixel 21A by exposure control pulses A2 and A3, respectively, and the signals generated by the second pixel 21B by exposure control pulses A0 and A1, respectively, in the latest second frame, which are enclosed by the dashed line p1 in Figure 8, to calculate the second distance. The distance calculation unit 50 may also use the signal enclosed by the dashed line p0 in Figure 8 to calculate the second distance.

[0070] In calculating the second distance, the distance calculation unit 50 uses, for example, the signal output by a predetermined second pixel 21B. Furthermore, if the first pixel 21A and the second pixel 21B are arranged as shown in Figure 6A, Figure 6B, or Figure 6D, the distance calculation unit 50 may select a second pixel 21B that outputs a signal used for calculating the second distance from among a plurality of second pixels 21B that are adjacent to the first pixel 21A in different directions. For example, the distance calculation unit 50 selects from the two or more second pixels 21B that minimizes the difference between the distance to the object OBJ at the second pixel 21B and the distance to the object OBJ at the first pixel 21A. The difference between the distance to the object OBJ at the second pixel 21B and the distance to the object OBJ at the first pixel 21A is compared, for example, by an evaluation value used to detect spatial changes in the distance to the object OBJ by the change detection unit 40. Details regarding the detection of spatial changes in the distance to the object OBJ by the change detection unit 40 will be described later.

[0071] The first distance is calculated using signals output from the same pixel 21 in multiple consecutive frames, so it becomes an inaccurate distance if there is a temporal change in the distance to the object OBJ. When the first example of the operation of the distance measuring device 100 shown in Figure 9 is performed, the first distance at the first pixel 21A is output when it is detected that there is no temporal change in the distance to the object OBJ at each of the one or more target pixels 21 including the first pixel 21A. Furthermore, if it is detected that there is a temporal change in the distance to the object OBJ at at least one of the one or more target pixels 21 including the first pixel 21A, the second distance is output instead of the first distance. Therefore, if the first distance becomes an inaccurate distance due to a temporal change in the distance to the object OBJ, the first distance is not output. Thus, the distance measuring accuracy of the distance measuring device 100 can be improved.

[0072] Next, a second example of the operation of the distance measuring device 100 will be described. Figure 10 is a flowchart showing a second example of the operation of the distance measuring device 100 according to this embodiment.

[0073] As shown in Figure 10, first, the change detection unit 40 detects whether there is a spatial change in the distance from the first pixel 21A to the object OBJ between the first pixel 21A and the second pixel 21B (step S21). In step S21, the change detection unit 40 detects, for example, that there is a spatial change in the distance from the first pixel 21A to the object OBJ between the first pixel 21A and the second pixel 21B if the difference between the first frame and the second frame in the brightness value calculated based on the signals output by the first pixel 21A and the second pixel 21B in the same frame is greater than a preset threshold.

[0074] In step S21, the luminance value is the sum of the signal values ​​(A0 + A1 + A2 + A3) of the signals output by the first pixel 21A and the second pixel 21B in the same frame. The difference in luminance value between the first and second frames is, for example, the difference between the luminance value based on the signals enclosed by the dashed line p1 output by the first pixel 21A and the second pixel 21B in the latest second frame shown in Figure 8, and the luminance value based on the signals enclosed by the dashed line p0 output by the first pixel 21A and the second pixel 21B in the latest first frame. When the distance to the object OBJ changes spatially, the timing of receiving reflected light by the first pixel 21A and the second pixel 21B also changes according to the difference in distance. As a result, if the exposure timing of the first pixel 21A and the second pixel 21B is swapped, the signal values ​​of the signals generated based on the reflected light will be different. Therefore, the difference in luminance values ​​between the first and second frames, where the exposure timings are reversed, allows for the detection of whether or not there is a spatial change in the distance to the object OBJ. For example, consider a case where there is a spatial change in the distance to the object OBJ such that the first pixel 21A receives reflected light during the exposure control pulses A0 and A1, and the second pixel 21B receives reflected light during the exposure control pulses A2 and A3. In this case, in the first frame, signals based on reflected light are generated in both the first pixel 21A and the second pixel 21B, whereas in the second frame, no signals based on reflected light are generated in either the first pixel 21A or the second pixel 21B, resulting in a difference in luminance values ​​between the first and second frames. Therefore, the difference in luminance values ​​between the first and second frames allows for the detection of whether or not there is a temporal change in the distance to the object OBJ.

[0075] If the change detection unit 40 detects that there is no spatial change in the distance to the object OBJ between the first pixel 21A and the second pixel 21B (No in step S21), the distance calculation unit 50 calculates and outputs a second distance as the distance to the object OBJ at the first pixel 21A (step S13). Step S13 is as described in the first example above.

[0076] On the other hand, if the change detection unit 40 detects that there is a spatial change in the distance from the first pixel 21A to the object OBJ between the first pixel 21A and the second pixel 21B (Yes in step S21), the distance calculation unit 50 calculates and outputs a first distance as the distance from the first pixel 21A to the object OBJ (step S12). Step S12 is as described in the first example above.

[0077] The second distance is calculated using signals output from the first pixel 21A and the second pixel 21B, which are located at different positions within the same frame. Therefore, if there is a spatial change in the distance to the object OBJ, the second distance will be inaccurate. In the second example of the operation of the distance measuring device 100 shown in Figure 10, the second distance is output when it is detected that there is no spatial change in the distance to the object OBJ. If a spatial change in the distance to the object OBJ is detected, the first distance is output instead of the second distance. Therefore, if the second distance becomes inaccurate due to a spatial change in the distance to the object OBJ, the second distance is not output. Thus, the distance measuring accuracy of the distance measuring device 100 can be improved.

[0078] Next, a third example of the operation of the distance measuring device 100 will be described. Figure 11 is a flowchart showing the third example of the operation of the distance measuring device 100 according to this embodiment.

[0079] As shown in Figure 11, first, the change detection unit 40 detects whether or not there is a temporal change in the distance from one or more target pixels 21, including the first pixel 21A, to the target object OBJ (step S11). Step S11 is as described in the first example above.

[0080] If the change detection unit 40 detects that there is no temporal change in the distance to the object OBJ at each of the one or more target pixels 21 (No in step S11), the distance calculation unit 50 calculates and outputs a first distance as the distance to the object OBJ at the first pixel 21A (step S12). Step S12 is as described in the first example above.

[0081] On the other hand, if the change detection unit 40 detects that there is a temporal change in the distance to the object OBJ in at least one of the one or more target pixels 21 (Yes in step S11), the change detection unit 40 detects whether or not there is a spatial change in the distance to the object OBJ in the first pixel 21A and the second pixel 21B (step S21). Step S21 is as described in the second example above.

[0082] If the change detection unit 40 detects that there is no spatial change in the distance to the object OBJ between the first pixel 21A and the second pixel 21B (No in step S21), the distance calculation unit 50 calculates and outputs a second distance as the distance to the object OBJ at the first pixel 21A (step S13). Step S13 is as described in the first example above.

[0083] On the other hand, if the change detection unit 40 detects that there is a spatial change in the distance to the object OBJ between the first pixel 21A and the second pixel 21B (Yes in step S21), the distance calculation unit 50 outputs an invalid signal indicating that the distance to the object OBJ at the first pixel 21A is invalid (step S14). The invalid signal is, for example, a signal indicating a fixed value such as the minimum or maximum value of the distance value output by the distance calculation unit 50. The invalid signal may also include a flag indicating that the distance to the object OBJ is invalid.

[0084] In the third example of the operation of the distance measuring device 100 shown in Figure 11, if it is detected that there is no temporal change in the distance to the object OBJ at each of the one or more target pixels 21 including the first pixel 21A, the first distance at the first pixel 21A is output. Furthermore, if it is detected that there is a temporal change in the distance to the object OBJ at at least one of the one or more target pixels 21 including the first pixel 21A, and that there is no spatial change in the distance to the object OBJ, the second distance is output. Furthermore, if it is detected that there is a temporal change in the distance to the object OBJ at at least one of the one or more target pixels 21 including the first pixel 21A, and that there is a spatial change in the distance to the object OBJ, an invalid signal is output. Therefore, if the first distance becomes inaccurate due to a temporal change in the distance to the object OBJ, the first distance is not output, and if the second distance becomes inaccurate due to a spatial change in the distance to the object OBJ, the second distance is not output. Therefore, the distance measurement accuracy of the distance measuring device 100 can be improved.

[0085] Next, a fourth example of the operation of the distance measuring device 100 will be described. Figure 12 is a flowchart showing the fourth example of the operation of the distance measuring device 100 according to this embodiment.

[0086] As shown in Figure 12, first, the distance calculation unit 50 calculates a first distance as the distance from the first pixel 21A to the object OBJ (step S31). The calculation of the first distance is performed in the same manner as in step S12 above. In step S31, the distance calculation unit 50 temporarily stores the calculated first distance in memory, for example.

[0087] Next, the distance calculation unit 50 calculates a second distance as the distance from the first pixel 21A to the object OBJ (step S32). The calculation of the second distance is performed in the same manner as in step S13 above. In step S32, the distance calculation unit 50 temporarily stores the calculated second distance in memory, for example. Note that the order of steps S31 and S32 may be reversed, or they may be performed in parallel.

[0088] Next, the change detection unit 40 detects whether or not there is a temporal change in the distance from the first pixel 21A to the object OBJ in one or more target pixels 21 (step S33). The one or more target pixels 21 are, for example, the first pixel 21A. In step S33, the change detection unit 40 detects that there is a temporal change in the distance from the first pixel 21A to the object OBJ if the temporal difference of the first distance in the first pixel 21A is greater than a threshold.

[0089] The temporal difference of the first distance is, for example, the difference between the current first distance (the first distance calculated in step S31) based on the signal enclosed by the dashed line t1 output by the first pixel 21A in the most recent first and second frames shown in Figure 8, and the past first distance based on the signal enclosed by the dashed line t0 output by the first pixel 21A in the first and second frames preceding the most recent. As the distance to the object OBJ changes over time, the timing at which the first pixel 21A receives reflected light also changes. Therefore, the temporal difference of the first distance allows for the detection of whether or not there has been a temporal change in the distance to the object OBJ.

[0090] In step S33, as described in step S11, the one or more target pixels 21 may be two pixels 21: a first pixel 21A and a second pixel 21B adjacent to the first pixel 21A.

[0091] In this case, the change detection unit 40 detects that there is a temporal change in the distance to the object OBJ at the second pixel 21B when the temporal difference of the first distance at the second pixel 21B is greater than a threshold. The temporal difference of the first distance at the second pixel 21B is calculated in the same way as the calculation of the temporal difference of the first distance at the first pixel 21A.

[0092] If the change detection unit 40 detects that there is no temporal change in the distance to the object OBJ at each of the one or more target pixels 21 (No in step S33), the distance calculation unit 50 selects the first distance from the first and second distances calculated in steps S31 and S32 and outputs it (step S34).

[0093] On the other hand, if the change detection unit 40 detects that there is a temporal change in the distance to the object OBJ in at least one of the one or more target pixels 21 (Yes in step S33), the distance calculation unit 50 selects the second distance from the first and second distances calculated in steps S31 and S32 and outputs it (step S35).

[0094] As shown in Figure 12, the fourth example of the operation of the distance measuring device 100 is performed, and similar to the first example of the operation of the distance measuring device 100 described above, if the first distance becomes an inaccurate distance due to the temporal change in the distance to the object OBJ, the first distance is not output. Therefore, the distance measuring accuracy of the distance measuring device 100 can be improved.

[0095] In the fourth example of the operation of the distance measuring device 100, step S32 may be omitted. In this case, in step S34, the distance calculation unit 50 outputs the first distance calculated in step S31 without selecting the first distance from the first distance and the second distance. Also in this case, step S13 described above is performed instead of step S35.

[0096] Next, a fifth example of the operation of the distance measuring device 100 will be described. Figure 13 is a flowchart showing the fifth example of the operation of the distance measuring device 100 according to this embodiment.

[0097] As shown in Figure 13, first, the distance calculation unit 50 calculates a first distance as the distance from the first pixel 21A to the object OBJ (step S31). Step S31 is as described in the fourth example above.

[0098] Next, the distance calculation unit 50 calculates a second distance as the distance from the first pixel 21A to the object OBJ (step S32). Step S32 is as described in the fourth example above.

[0099] Next, the change detection unit 40 detects whether there is a spatial change in the distance to the object OBJ between the first pixel 21A and the second pixel 21B (step S41). In step S41, the change detection unit 40 detects that there is a spatial change in the distance to the object OBJ between the first pixel 21A and the second pixel 21B if the difference in the second distance between the first frame and the second frame is greater than a threshold, or if the temporal difference is greater than a threshold.

[0100] The difference in the second distance between the first and second frames is, for example, the difference between the second distance based on the signals enclosed by the dashed line p1 output by the first pixel 21A and the second pixel 21B in the latest second frame shown in Figure 8, and the second distance based on the signals enclosed by the dashed line p0 output by the first pixel 21A and the second pixel 21B in the latest first frame. When the distance to the object OBJ changes spatially, the timing at which the first pixel 21A and the second pixel 21B receive reflected light also differs. Therefore, if the exposure timing of the first pixel 21A and the second pixel 21B is swapped, the second distance changes. Thus, by the difference in the second distance between the first and second frames, it is possible to detect whether or not there is a spatial change in the distance to the object OBJ.

[0101] If the change detection unit 40 detects that there is no spatial change in the distance from the first pixel 21A to the object OBJ between the first pixel 21A and the second pixel 21B (No in step S41), the distance calculation unit 50 selects the second distance from the first and second distances calculated in steps S31 and S32 and outputs it (step S35).

[0102] On the other hand, if the change detection unit 40 detects that there is a spatial change in the distance from the first pixel 21A to the object OBJ between the first pixel 21A and the second pixel 21B (Yes in step S41), the distance calculation unit 50 selects the first distance from the first and second distances calculated in steps S31 and S32 and outputs it (step S34).

[0103] As shown in Figure 13, the fifth example of operation of the distance measuring device 100 is performed, and similar to the second example of operation of the distance measuring device 100 described above, if the second distance becomes an inaccurate distance due to a spatial change in the distance to the object OBJ, the second distance will not be output. Therefore, the distance measuring accuracy of the distance measuring device 100 can be improved.

[0104] In the fifth example of the operation of the distance measuring device 100, step S31 may be omitted. In this case, in step S35, the distance calculation unit 50 outputs the second distance calculated in step S32 without selecting the second distance from the first distance and the second distance. Also in this case, step S12 described above is performed instead of step S34.

[0105] Next, a sixth example of the operation of the distance measuring device 100 will be described. Figure 14 is a flowchart showing the sixth example of the operation of the distance measuring device 100 according to this embodiment.

[0106] As shown in Figure 14, first, the distance calculation unit 50 calculates a first distance as the distance from the first pixel 21A to the object OBJ (step S31). Step S31 is as described in the fourth example above.

[0107] Next, the distance calculation unit 50 calculates a second distance as the distance from the first pixel 21A to the object OBJ (step S32). Step S32 is as described in the fourth example above.

[0108] Next, the change detection unit 40 detects whether or not there is a temporal change in the distance from one or more target pixels 21, including the first pixel 21A, to the target object OBJ (step S33). Step S33 is as described in the fourth example above.

[0109] If the change detection unit 40 detects that there is no temporal change in the distance to the object OBJ at each of the one or more target pixels 21 (No in step S33), the distance calculation unit 50 selects the first distance from the first and second distances calculated in steps S31 and S32 and outputs it (step S34).

[0110] On the other hand, if the change detection unit 40 detects that there is a temporal change in the distance to the object OBJ in at least one of the one or more target pixels 21 (Yes in step S33), the change detection unit 40 detects whether or not there is a spatial change in the distance to the object OBJ in the first pixel 21A and the second pixel 21B (step S41). Step S41 is as described in the fifth example above.

[0111] If the change detection unit 40 detects that there is no spatial change in the distance from the first pixel 21A to the object OBJ between the first pixel 21A and the second pixel 21B (No in step S41), the distance calculation unit 50 selects the second distance from the first and second distances calculated in steps S31 and S32 and outputs it (step S35).

[0112] On the other hand, if the change detection unit 40 detects that there is a spatial change in the distance to the object OBJ between the first pixel 21A and the second pixel 21B (Yes in step S41), the distance calculation unit 50 outputs an invalid signal indicating that the distance to the object OBJ at the first pixel 21A is invalid (step S14). Step S14 is as described in the third example above.

[0113] By performing the sixth example of operation of the distance measuring device 100 shown in Figure 14, similar to the third example of operation of the distance measuring device 100 described above, if the first distance becomes inaccurate due to a temporal change in the distance to the object OBJ, the first distance is not output, and if the second distance becomes inaccurate due to a spatial change in the distance to the object OBJ, the second distance is not output. Therefore, the distance measuring accuracy of the distance measuring device 100 can be improved.

[0114] [Another example of a drive sequence] The above describes an example where the drive sequence of the distance measuring device 100 is a drive sequence for calculating distance using the pulse TOF method. However, the drive sequence of the distance measuring device 100 may also be a drive sequence for calculating distance using the CW (Continuous Wave) TOF method. The CW TOF method is an indirect TOF method in which the light source 10 emits a continuous wave whose intensity is modulated at a predetermined frequency as illumination light, and the distance to the object OBJ is calculated based on the phase difference between the illumination light emitted by the light source 10 and the reflected light of the illumination light by the object OBJ received by a plurality of pixels 21.

[0115] Figure 15 shows another example of the drive sequence of the distance measuring device 100 according to this embodiment. Figure 15 shows the drive sequence when multiple pixels 21 output a signal for the distance calculation unit 50 to calculate the distance to the target object OBJ using the CWTOF method.

[0116] The drive sequence shown in Figure 15 is the same as the drive sequence shown in Figure 5, except that the A0 / A1 emission exposure period is changed to the S0 / S2 emission exposure period, and the A2 / A3 emission exposure period is changed to the S1 / S3 emission exposure period. Even when the distance measuring device 100 is driven with the drive sequence shown in Figure 15, the first frame and the second frame are repeated alternately.

[0117] In the example shown in Figure 15, the drive control unit 30 drives the first pixel 21A in the exposure sequence of the S0 / S2 emission exposure period in the first frame, and drives the second pixel 21B in the exposure sequence of the S1 / S3 emission exposure period. In the second frame, the drive control unit 30 drives the first pixel 21A in the exposure sequence of the S1 / S3 emission exposure period, and drives the second pixel 21B in the sequence of the S0 / S2 emission exposure period. In other words, for the first pixel 21A and the second pixel 21B, the exposure timing based on the emission of light from the light source 10 is reversed between the first frame and the second frame. The exposure sequence of the S0 / S2 emission exposure period is an example of the first exposure sequence, and the exposure sequence of the S1 / S3 emission exposure period is an example of the second exposure sequence.

[0118] Figure 16 is a time chart showing examples of the timing of emission control pulses and exposure control pulses during the S0 / S2 emission exposure period and the S1 / S3 emission exposure period.

[0119] As shown in Figure 16, the drive control unit 30 continuously outputs light emission control pulses at frequency f. In the example shown in Figure 16, the light emission control pulse is a sine wave, but it may be a square wave or a triangular wave or other waveform. The drive control unit 30 also outputs exposure control pulses S0 to S3 at timings relative to the light emission control pulse. Exposure control pulses S0 to S3 are control pulses that are output continuously at the same frequency f as the light emission control pulse. In the example shown in Figure 16, the pulse width of exposure control pulses S0 to S3 is equal to half the period T (= 1 / f) of the light emission control pulse. Exposure control pulses S0 to S3 instruct the pixels 21 to expose so that the exposure timings relative to the emission of light from the light source 10 are different. The exposure control pulses S0 to S3 have different phase differences from the light emission control pulse, relative to the light emission control pulse. The exposure control pulses S0 to S3 are 90° behind the light emission control pulse in the order of exposure control pulses S0 to S3. Specifically, the exposure control pulse S0 has a phase difference of 0° from the light emission control pulse. The exposure control pulse S1 has a phase difference of 90° from the light emission control pulse. The exposure control pulse S2 has a phase difference of 180° from the light emission control pulse. The exposure control pulse S3 has a phase difference of 270° from the light emission control pulse. Note that if the distance to the object OBJ can be calculated using the CWTOF method, there is no particular limit on the number of exposure control pulses with different phase differences from the light emission control pulse and from each other.

[0120] As shown in Figure 16, during the S0 / S2 light emission exposure period, the drive control unit 30 outputs a light emission control pulse and exposure control pulses S0 and S2. The phase difference between exposure control pulse S0 and exposure control pulse S2 is 180°. The low-level periods of exposure control pulse S0 and exposure control pulse S2 do not overlap. The signal charge generated by exposure control pulse S0 is stored in one of the charge storage units 23a and 23b, and the signal charge generated by exposure control pulse S2 is stored in the other of the charge storage units 23a and 23b. Also, during the S1 / S3 light emission exposure period, the drive control unit 30 outputs a light emission control pulse and exposure control pulses S1 and S3. The phase difference between exposure control pulse S1 and exposure control pulse S3 is 180°. The low-level periods of exposure control pulse S1 and exposure control pulse S3 do not overlap. The signal charge generated by the exposure control pulse S1 is stored in one of the charge storage units 23a and 23b, and the signal charge generated by the exposure control pulse S3 is stored in the other of the charge storage units 23a and 23b.

[0121] The distance calculation unit 50 calculates the distance to the object OBJ based on the signals generated by each of the multiple pixels 21 by the exposure control pulses S0 to S3. Specifically, the distance calculation unit 50 calculates the distance Z to the object OBJ using the following equation (4), which is based on the phase delay of the reflected light relative to the irradiated light.

[0122]

[0123] Here, c is the speed of light, and f is the frequency of the light emission control pulse. Also, S0 to S3 are the signal values ​​of the signals generated by the pixel 21 by the exposure control pulses S0 to S3, respectively.

[0124] In the drive sequence shown in Figure 15, signals generated by exposure control pulses S0 to S3 are output from the same pixel 21 in both the first and second frames. Also, in the same frame, signals generated by exposure control pulses S0 to S3 are output from the first pixel 21A and the second pixel 21B, respectively. As a result, the distance calculation unit 50 can calculate the distance to the object OBJ based on the signals output from the same pixel 21 in both the first and second frames, and the distance to the object OBJ based on the signals output from the first pixel 21A and the second pixel 21B in the same frame. Therefore, in the drive sequence shown in Figure 15, similar to the drive sequence shown in Figure 5, the signals output in a situation where there is no spatial or temporal change in the distance to the object OBJ can be used in calculating the distance to the object OBJ.

[0125] Furthermore, even when the distance measuring device 100 is driven by the drive sequence shown in Figure 15, the first to sixth examples of the operation of the distance measuring device 100 shown in Figures 9 to 14 can be performed.

[0126] When the distance measuring device 100 is driven in the drive sequence shown in Figure 15, in steps S12, S13, S31, and S32, the distance calculation unit 50 uses the above formula (4) to calculate the first distance and the second distance.

[0127] Furthermore, when the distance measuring device 100 is driven in the drive sequence shown in Figure 15, in step S11, the change detection unit 40 can use, for example, the difference between S0 + S2 and S1 + S3 in the signals output by one or more target pixels 21, including the first pixel 21A, in each of the most recent first and second frames as an evaluation value for detecting whether or not there is a temporal change in the distance to the target object OBJ. For example, the change detection unit 40 detects that there is a temporal change in the distance to the target object OBJ at the first pixel 21A if the difference between S0 + S2 and S1 + S3 in the signal output by the first pixel 21A is greater than a threshold. Also, the change detection unit 40 detects that there is a temporal change in the distance to the target object OBJ at the second pixel 21B if, for example, the difference between S0 + S2 and S1 + S3 in the signal output by the second pixel 21B is greater than a threshold.

[0128] Furthermore, when the distance measuring device 100 is driven in the drive sequence shown in Figure 15, in step S21, the change detection unit 40 can use, for example, the difference between S0 + S2 and S1 + S3 in the signals output by the first pixel 21A and the second pixel 21B in the latest second frame as an evaluation value for detecting whether or not there is a spatial change in the distance to the object OBJ. The change detection unit 40 detects, for example, that there is a spatial change in the distance to the object OBJ between the first pixel 21A and the second pixel 21B if the difference between S0 + S2 and S1 + S3 is greater than a threshold.

[0129] [Modified Versions] Next, a distance measuring device according to a modified version of the embodiment will be described. In the following description of the modified version, the differences from Embodiment 1 will be the main focus, and the explanation of the common points will be omitted or simplified.

[0130] Figure 17 is a functional block diagram showing an example of the configuration of the distance measuring device 100A according to this modified example.

[0131] As shown in Figure 17, the distance measuring device 100A according to this modified example differs from the distance measuring device 100 according to the embodiment in that it further includes a threshold determination unit 60.

[0132] The threshold determination unit 60 is a processing circuit implemented, for example, by a memory for storing the program and a processor for executing the program. Although they are shown separately in the block diagram, all or part of the drive control unit 30, change detection unit 40, distance calculation unit 50, and threshold determination unit 60 may be composed of the same memory and processor. Furthermore, the threshold determination unit 60 may be a dedicated logic circuit that performs predetermined processing.

[0133] The threshold determination unit 60 determines a threshold used by the change detection unit 40 in at least one of the above steps S11, S21, S31, and S41, based on the signals output by the multiple pixels 21. Specifically, the threshold determination unit 60 estimates the amount of noise based on the signals output by the multiple pixels 21 and determines a threshold based on the estimated amount of noise. The calculation for determining the threshold is not particularly limited, but the threshold determination unit 60 determines the threshold by multiplying the estimated amount of noise by a predetermined numerical value, for example. Because the threshold is determined by the threshold determination unit 60, it becomes a threshold corresponding to the signals output by the multiple pixels 21, thus suppressing false detections in the change detection unit 40. For example, the more light incident on the pixel 21, the greater the amount of noise in the signal output by the pixel 21. Therefore, by the threshold determination unit 60 determining the threshold based on the estimated amount of noise, even when the noise is large, it is possible to suppress the change detection unit 40 from detecting the noise as a temporal or spatial change in the distance to the object OBJ.

[0134] Figure 18 is a diagram illustrating the estimation of noise amount and determination of the threshold by the threshold determination unit 60 in this modified example. Figure 18 shows the case in which the threshold of the difference in luminance values ​​used in steps S11 and S21 is determined. The threshold determination unit 60 calculates A0 + A1 + A2 + A3 as luminance values ​​and estimates the noise amount from the relationship between the luminance values ​​and the noise amount shown in Figure 18. In the example shown in Figure 18, the threshold determination unit 60 determines the threshold by adding the estimated noise amount to the threshold reference value.

[0135] In determining the threshold in step S11, for example, A0 + A1 + A2 + A3 in the signal output by the first pixel 21A in each of the most recent first and second frames shown in Figure 8 is used as the luminance value. Also, when detecting whether or not there is a temporal change in the distance to the object OBJ at the second pixel 21B, A0 + A1 + A2 + A3 in the signal output by the second pixel 21B in each of the most recent first and second frames is used as the luminance value.

[0136] Furthermore, in determining the threshold in step S21, for example, A0 + A1 + A2 + A3 in the signals output by the first pixel 21A and the second pixel 21B, respectively, in the latest second frame shown in Figure 8, are used as the luminance values.

[0137] The relationship between luminance value and noise level shown in Figure 18 is stored, for example, in a memory device (not shown). Generally, as the luminance value increases, the noise level also increases; therefore, the relationship between luminance value and noise level can be calculated theoretically or determined experimentally.

[0138] Furthermore, the threshold determination unit 60 estimates the amount of noise for determining the threshold used in step S33 based on the first distance and the reflectance of the object OBJ. Typically, the amount of noise increases as the first distance increases and as the reflectance of the object OBJ decreases, so the amount of noise can be estimated based on the first distance and the reflectance of the object OBJ. The reflectance of the object OBJ can be calculated from the distance to the object OBJ and the luminance value. Similarly, the threshold determination unit 60 estimates the amount of noise for determining the threshold used in step S41 based on the second distance and the reflectance of the object OBJ. Typically, the amount of noise increases as the second distance increases and as the reflectance of the object OBJ decreases, so the amount of noise can be estimated based on the second distance and the reflectance of the object OBJ.

[0139] (Other) The distance measuring devices relating to one or more embodiments of this disclosure have been described above based on embodiments (including modifications), but this disclosure is not limited to embodiments. Without departing from the spirit of this disclosure, various modifications that a person skilled in the art can conceive of may be applied to each embodiment, and forms constructed by combining components from different embodiments may also be included within the scope of one or more embodiments of this disclosure.

[0140] Furthermore, the distance measuring device according to this disclosure does not need to include all of the components described in the above embodiments, and may consist only of components necessary to perform the desired operation.

[0141] For example, the distance measuring device 100 does not necessarily have to include a change detection unit 40. For example, the distance calculation unit 50 may output both a first distance and a second distance, and a subsequent processing circuit may select the distance to be used for generating a distance image from the output first distance and second distance.

[0142] Furthermore, in the above embodiment, for example, the change detection unit 40 detects at least one of whether there is a temporal change or a spatial change in the distance to the object OBJ at the multiple pixels 21, based on an evaluation value calculated based on the signals output by the multiple pixels 21, but is not limited to this. The change detection unit 40 may, for example, detect at least one of whether there is a temporal change or a spatial change in the distance to the object OBJ at the multiple pixels 21, based on information output from an external sensor that detects the distance to the object OBJ or the brightness of the object OBJ, etc. In this case, the distance calculation unit 50 may uniformly apply the detection result by the change detection unit 40 to the multiple pixels 21.

[0143] Furthermore, for example, in the above embodiment, the pixel 21 includes a plurality of charge storage units 23a and 23b, but is not limited to this. The number of charge storage units included in the pixel 21 may be one. In this case, in one frame, the pixel 21 is exposed for one exposure period that starts with reference to the emission of the illumination light. For example, the drive control unit 30 may output exposure control pulses A0 to A3 to the same pixel 21 in four consecutive frames, and may also output exposure control pulses A0 to A3 to four pixels 21 in the same frame.

[0144] Furthermore, in the above embodiment, each component may be realized by executing a software program suitable for each component. Each component may also be realized by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0145] Furthermore, each component may be implemented by hardware. Each component may also be a circuit (or integrated circuit). These circuits may form a single circuit as a whole, or they may be separate circuits. Also, each of these circuits may be a general-purpose circuit or a dedicated circuit.

[0146] Furthermore, the general or specific embodiments of this disclosure may be implemented as systems, apparatus, methods, integrated circuits, computer programs, computer program products, or recording media such as computer-readable CD-ROMs. They may also be implemented as any combination of systems, apparatus, methods, integrated circuits, computer programs, computer program products, and recording media.

[0147] For example, this disclosure may be implemented as a distance measuring device according to the above embodiment, as a control device for controlling a distance measuring device, as a distance measuring method including steps (processes) performed by components constituting a distance measuring device, as a program for causing a computer to execute such a distance measuring method, or as a computer-readable non-temporary recording medium on which such a program is recorded.

[0148] Examples of the distance measuring device and distance measuring method according to the present disclosure, as described based on the above embodiments, are shown below. The distance measuring device and distance measuring method according to the present disclosure are not limited to the following examples.

[0149] For example, a distance measuring device according to a first aspect of the present disclosure includes a light source that emits illumination light, a light receiving unit having a plurality of pixels that receive reflected light reflected by an object from the illumination light emitted from the light source, a drive control unit that controls the driving of the light source and the plurality of pixels, and a distance calculation unit that calculates the distance to the object based on signals output by the plurality of pixels. The drive control unit classifies the plurality of pixels into a plurality of groups to be exposed at the same timing, each including a first group to which a first pixel belongs and a second group to which a second pixel adjacent to the first pixel belongs. The drive control unit drives the plurality of pixels such that the exposure timing based on the emission of the illumination light differs among the plurality of groups in the same frame, and drives the plurality of pixels such that the exposure timing based on the emission of the illumination light is reversed among the plurality of groups in a plurality of consecutive frames.

[0150] As a result, the distance calculation unit can calculate the distance to the object based on the signals output from the first and second pixels in the same frame, and the distance to the object based on the signals output from the same pixel in each of multiple consecutive frames. Here, the signals output from the first and second pixels in the same frame and the signals output from the same pixel in each of multiple consecutive frames are signals whose exposure timing is the same, based on the emission of the irradiated light. Furthermore, the signals output from the first and second pixels in the same frame are signals output when there is no temporal change in the distance to the object. Also, the signals output from the same pixel 21 in multiple consecutive frames are signals output when there is no spatial change in the distance to the object. Therefore, in calculating the distance to the object, it is possible to use signals output when there is no spatial or temporal change in the distance to the object, thereby improving the distance measurement accuracy.

[0151] Furthermore, for example, a distance measuring device according to a second aspect of the present disclosure is a distance measuring device according to a first aspect, wherein the drive control unit further comprises a change detection unit that detects at least one of whether there is a temporal change or a spatial change in the distance to the object at the plurality of pixels, and the distance calculation unit determines, based on the detection result of the change detection unit, which of the signals output from the plurality of pixels to use to calculate the distance to the object and output the result.

[0152] As a result, the distance calculation unit can output a distance to the object in which the influence of either temporal or spatial changes in the distance to the object is suppressed, based on the detection results of the change detection unit.

[0153] Furthermore, for example, a distance measuring device according to a third aspect of the present disclosure is a distance measuring device according to a second aspect, wherein the change detection unit detects whether or not there is a temporal change in the distance to the object in one or more pixels including the first pixel; the distance calculation unit outputs a first distance to the object calculated based on the signal output by the first pixel in each of a plurality of consecutive frames if the change detection unit detects that there is no temporal change in the distance to the object in each of the one or more pixels; and outputs a second distance to the object calculated based on the signals output by the first pixel and the second pixel in the same frame if the change detection unit detects that there is a temporal change in the distance to the object in at least one of the one or more pixels.

[0154] As a result, the first distance is output when it is detected that there is no temporal change in the distance to the object for each of the one or more pixels, including the first pixel. However, if it is detected that there is a temporal change in the distance to the object for at least one of the one or more pixels, including the first pixel, the second distance is output instead of the first distance. Therefore, if the first distance becomes inaccurate due to the temporal change in the distance to the object, the first distance is not output. Thus, the distance measurement accuracy of the distance measuring device can be improved.

[0155] Furthermore, for example, a distance measuring device according to a fourth aspect of the present disclosure is a distance measuring device according to a second aspect, wherein the change detection unit detects whether there is a temporal change in the distance to the object in one or more pixels including the first pixel and whether there is a spatial change in the distance to the object between the first pixel and the second pixel, and the distance calculation unit, when the change detection unit detects that there is no temporal change in the distance to the object in each of the one or more pixels, outputs a first distance to the object calculated based on the signal output by the first pixel in each of a plurality of consecutive frames, and the change detection unit detects that there is no temporal change in the distance to the object in at least one of the one or more pixels If the change detection unit detects that there is a temporal change in the distance to the object and that there is no spatial change in the distance to the object between the first pixel and the second pixel, it outputs a second distance to the object calculated based on the signals output by the first pixel and the second pixel in the same frame. If the change detection unit detects that there is a temporal change in the distance to the object in at least one of the one or more pixels and that there is a spatial change in the distance to the object between the first pixel and the second pixel, it outputs a signal indicating that the distance to the object is invalid.

[0156] As a result, a first distance is output when it is detected that there is no temporal change in the distance to the object for each of the one or more pixels, including the first pixel. A second distance is output when it is detected that there is a temporal change in the distance to the object for at least one of the one or more pixels, including the first pixel, and that there is no spatial change in the distance to the object. An invalid signal is output when it is detected that there is a temporal change in the distance to the object for at least one of the one or more pixels, including the first pixel, and that there is a spatial change in the distance to the object. Therefore, if the first distance becomes inaccurate due to a temporal change in the distance to the object, the first distance is not output, and if the second distance becomes inaccurate due to a spatial change in the distance to the object, the second distance is not output. Thus, the distance measurement accuracy of the distance measuring device can be improved.

[0157] Furthermore, for example, a distance measuring device according to a fifth aspect of the present disclosure is a distance measuring device according to a second aspect, wherein the change detection unit detects whether or not there is a spatial change in the distance to the object between the first pixel and the second pixel, and the distance calculation unit outputs a second distance to the object calculated based on the signals output by the first pixel and the second pixel in the same frame if the change detection unit detects that there is no spatial change in the distance to the object between the first pixel and the second pixel, and outputs a first distance to the object calculated based on the signals output by the first pixel in each of a plurality of consecutive frames if the change detection unit detects that there is a spatial change in the distance to the object between the first pixel and the second pixel.

[0158] This means that if no spatial change in the distance to the object is detected, the second distance is output. Conversely, if a spatial change in the distance to the object is detected, the first distance is output instead of the second distance. Therefore, if the second distance becomes inaccurate due to a spatial change in the distance to the object, the second distance is not output. Thus, the distance measurement accuracy of the distance measuring device can be improved.

[0159] Furthermore, for example, a distance measuring device according to a sixth aspect of the present disclosure is a distance measuring device according to a third or fourth aspect, wherein the change detection unit detects that there is a temporal change in the distance to the object at the first pixel when the temporal difference in the brightness value calculated based on the signal output by the first pixel in each of a plurality of consecutive frames is greater than a threshold.

[0160] This allows for the detection of temporal changes in the distance to the object at the first pixel by utilizing the signal output by the first pixel in each of multiple consecutive frames.

[0161] Furthermore, for example, a distance measuring device according to the seventh aspect of this disclosure is a distance measuring device according to the third or fourth aspect, wherein the change detection unit detects that there is a temporal change in the distance to the object at the first pixel when the temporal difference of the first distance is greater than a threshold.

[0162] This allows the temporal change in the distance to the object in the first pixel to be detected using the first distance.

[0163] Furthermore, for example, the distance measuring device according to the eighth aspect of the present disclosure is a distance measuring device according to the third or fourth aspect, wherein the distance calculation unit calculates the first distance and the second distance, the change detection unit detects that there is a temporal change in the distance to the object at the first pixel when the temporal difference of the first distance calculated by the distance calculation unit is greater than a threshold, and the distance calculation unit selects and outputs the calculated first distance and second distance when outputting the first distance or the second distance.

[0164] This allows the change detection unit to use the first distance calculated by the distance calculation unit to detect temporal changes in the distance to the object.

[0165] Furthermore, for example, a distance measuring device according to the ninth aspect of the present disclosure is a distance measuring device according to any one of the sixth to eighth aspects, further comprising a threshold determination unit that determines the threshold based on the signal output by the first pixel in each of a plurality of consecutive frames.

[0166] This allows a threshold value to be determined for each of the consecutive frames, corresponding to the signal output by the first pixel, thereby suppressing false detections in the change detection unit.

[0167] Furthermore, for example, a distance measuring device according to the tenth aspect of the present disclosure is a distance measuring device according to the fourth or fifth aspect, wherein the change detection unit detects a spatial change in the distance to the object between the first pixel and the second pixel when the difference between frames in which the exposure timing based on the emission of the illumination light is swapped among the plurality of groups of luminance values ​​calculated based on the signals output by the first pixel and the second pixel in the same frame is greater than a threshold.

[0168] This allows for the detection of spatial changes in the distance to the object between the first and second pixels, using the signals output by the first and second pixels respectively within the same frame.

[0169] Furthermore, for example, a distance measuring device according to the 11th aspect of the present disclosure is a distance measuring device according to the 4th or 5th aspect, wherein the change detection unit detects that there is a spatial change in the distance to the object between the first pixel and the second pixel when the difference in the second distance between frames in which the exposure timing based on the emission of the irradiation light is reversed among the plurality of groups is greater than a threshold.

[0170] This allows the spatial change in the distance to the object between the first and second pixels to be detected using the second distance.

[0171] Furthermore, for example, a distance measuring device according to the twelfth aspect of the present disclosure is a distance measuring device according to the fourth or fifth aspect, wherein the distance calculation unit calculates the first distance and the second distance, and the change detection unit detects that there is a spatial change in the distance to the object between the first and second pixels when the difference between the second distance calculated by the distance calculation unit and between frames in which the exposure timing based on the emission of the illumination light changes among the plurality of groups is greater than a threshold, and when the distance calculation unit outputs the first distance or the second distance, it selects and outputs from the calculated first distance and the second distance.

[0172] This allows the change detection unit to use the second distance calculated by the distance calculation unit to detect spatial changes in the distance to the object.

[0173] Furthermore, for example, a distance measuring device according to a 13th aspect of the present disclosure is a distance measuring device according to any one of the 10th to 12th aspects, further comprising a threshold determination unit that determines the threshold based on signals output by the first pixel and the second pixel, respectively, in the same frame.

[0174] This allows thresholds to be determined according to the signals output by the first and second pixels in the same frame, thereby suppressing false detections in the change detection unit.

[0175] Furthermore, for example, a distance measuring device according to a 14th aspect of the present disclosure is a distance measuring device according to any one of the 3rd to 13th aspects, wherein the second group includes two or more second pixels adjacent to the first pixel in different directions, and the distance calculation unit selects the second pixel from the two or more second pixels to output a signal used for calculating the second distance.

[0176] This makes it possible to select a second pixel that outputs a signal capable of calculating a second distance with minimal error.

[0177] Furthermore, for example, a distance measuring device according to a 15th aspect of the present disclosure is a distance measuring device according to any one of the first to 14th aspects, wherein the drive control unit controls the driving of the light source and the plurality of pixels to repeat a plurality of consecutive frames including a first frame and a second frame following the first frame, and in the first frame, drives the first pixel in a first exposure sequence which includes exposing it for a first period that starts at a timing based on the emission of the illumination light, drives the second pixel in a second exposure sequence which includes exposing it for a second period that starts at a timing different from the first period based on the emission of the illumination light, and in the second frame, exposes the first pixel in the second exposure sequence and exposes the second pixel in the first exposure sequence.

[0178] This allows the first and second frames to acquire signals output from pixels due to exposure during the first and second periods in the same frame, as well as signals output from the same pixels due to exposure during the first and second periods in multiple consecutive frames.

[0179] Furthermore, for example, a distance measuring method according to a 16th aspect of the present disclosure is a distance measuring method using a distance measuring device, the distance measuring device comprising a light source that emits illumination light, and a light receiving unit having a plurality of pixels that receive reflected light reflected by an object from the illumination light emitted from the light source, the distance measuring method includes a drive control step that controls the driving of the light source and the plurality of pixels, and a distance calculation step that calculates the distance to the object based on signals output by the plurality of pixels, the drive control step classifies the plurality of pixels into a plurality of groups that are exposed at the same timing, the plurality of pixels including a first group to which a first pixel belongs and a second group to which a second pixel adjacent to the first pixel belongs, the plurality of pixels are driven so that in the same frame the exposure timing based on the emission of the illumination light differs among the plurality of groups, and the plurality of pixels are driven so that in a plurality of consecutive frames the exposure timing based on the emission of the illumination light is reversed among the plurality of groups.

[0180] This makes it possible to improve the distance measurement accuracy, similar to the distance measuring device according to the first embodiment described above.

[0181] The distance measuring device and other related equipment described herein can be applied to a variety of applications, including distance measurement systems, sensing systems using distance images, and authentication systems.

[0182] 10 Light source 20 Light receiving unit 21 Pixel 21A First pixel 21B Second pixel 22 Photoelectric conversion unit 23a, 23b Charge storage unit 24a, 24b Charge transfer unit 25 Charge discharge unit 26 Discharge control unit 30 Drive control unit 40 Change detection unit 50 Distance calculation unit 60 Threshold determination unit 100, 100A Distance measuring device OBJ Target object

Claims

1. A distance measuring device comprising: a light source that emits illumination light; a light receiving unit having a plurality of pixels that receive reflected light reflected by an object from the illumination light emitted from the light source; a drive control unit that controls the driving of the light source and the plurality of pixels; and a distance calculation unit that calculates the distance to the object based on signals output by the plurality of pixels, wherein the drive control unit classifies the plurality of pixels into a plurality of groups to be exposed at the same timing, the plurality of pixels including a first group to which a first pixel belongs and a second group to which a second pixel adjacent to the first pixel belongs; drives the plurality of pixels such that the exposure timing based on the emission of the illumination light differs among the plurality of groups in the same frame; and drives the plurality of pixels such that the exposure timing based on the emission of the illumination light is reversed among the plurality of groups in a plurality of consecutive frames.

2. The distance measuring device according to claim 1, further comprising a change detection unit that detects whether or not there is a temporal change and or a spatial change in the distance to the object at the plurality of pixels, wherein the distance calculation unit determines, based on the detection result of the change detection unit, which of the signals output from the plurality of pixels to use to calculate the distance to the object.

3. The distance measuring device according to claim 2, wherein the change detection unit detects whether there is a temporal change in the distance to the object in one or more pixels including the first pixel; the distance calculation unit outputs a first distance to the object calculated based on the signal output by the first pixel in each of a plurality of consecutive frames when the change detection unit detects that there is no temporal change in the distance to the object in each of the one or more pixels; and the change detection unit outputs a second distance to the object calculated based on the signals output by the first pixel and the second pixel in the same frame when the change detection unit detects that there is a temporal change in the distance to the object in at least one of the one or more pixels.

4. The change detection unit detects whether there is a temporal change in the distance to the object in one or more pixels including the first pixel, and whether there is a spatial change in the distance to the object between the first pixel and the second pixel. The distance calculation unit, if the change detection unit detects that there is no temporal change in the distance to the object in each of the one or more pixels, outputs a first distance to the object calculated based on the signal output by the first pixel in each of a plurality of consecutive frames. If the change detection unit detects that there is a temporal change in the distance to the object in at least one of the one or more pixels, and the change detection unit detects that there is no spatial change in the distance to the object between the first pixel and the second pixel, it outputs a second distance to the object calculated based on the signals output by the first pixel and the second pixel in the same frame. The distance measuring device according to claim 2, wherein the change detection unit detects that there is a temporal change in the distance to the object in at least one of the one or more pixels, and the change detection unit detects that there is a spatial change in the distance to the object in the first pixel and the second pixel, and outputs a signal indicating that the distance to the object is invalid.

5. The distance measuring device according to claim 2, wherein the change detection unit detects whether or not there is a spatial change in the distance to the object between the first pixel and the second pixel, the distance calculation unit outputs a second distance to the object calculated based on the signals output by the first pixel and the second pixel in the same frame if the change detection unit detects that there is no spatial change in the distance to the object between the first pixel and the second pixel, and outputs a first distance to the object calculated based on the signals output by the first pixel in each of a plurality of consecutive frames.

6. The distance measuring device according to claim 3 or 4, wherein the change detection unit detects that there is a temporal change in the distance to the object at the first pixel when the temporal difference in the brightness value calculated based on the signal output by the first pixel in each of a plurality of consecutive frames is greater than a threshold.

7. The distance measuring device according to claim 3 or 4, wherein the change detection unit detects that there is a temporal change in the distance to the object at the first pixel when the temporal difference of the first distance is greater than a threshold.

8. The distance measuring device according to claim 3 or 4, wherein the distance calculation unit calculates the first distance and the second distance, the change detection unit detects that there is a temporal change in the distance to the object at the first pixel when the temporal difference of the first distance calculated by the distance calculation unit is greater than a threshold, and the distance calculation unit, when outputting the first distance or the second distance, selects and outputs from the calculated first distance and the second distance.

9. The distance measuring device according to any one of claims 6 to 8, further comprising a threshold determination unit that determines the threshold based on the signal output by the first pixel in each of a plurality of consecutive frames.

10. The distance measuring device according to claim 4 or 5, wherein the change detection unit detects a spatial change in the distance to the object between the first pixel and the second pixel when the difference between frames in which the exposure timing relative to the emission of the illumination light is reversed among the plurality of groups of luminance values ​​calculated based on the signals output by the first pixel and the second pixel in the same frame is greater than a threshold.

11. The distance measuring device according to claim 4 or 5, wherein the change detection unit detects that there is a spatial change in the distance to the object between the first pixel and the second pixel when the difference in the second distance between frames in which the exposure timing based on the emission of the irradiation light is reversed among the plurality of groups is greater than a threshold.

12. The distance calculation unit calculates the first distance and the second distance; the change detection unit detects a spatial change in the distance to the object between the first pixel and the second pixel when the difference between the second distance calculated by the distance calculation unit and between frames in which the exposure timing based on the emission of the illumination light changes among the plurality of groups is greater than a threshold; and the distance calculation unit, when outputting the first distance or the second distance, selects and outputs from the calculated first distance and the second distance, the distance measuring device according to claim 4 or 5.

13. The distance measuring device according to any one of claims 10 to 12, further comprising a threshold determination unit that determines the threshold based on signals output by the first pixel and the second pixel, respectively, in the same frame.

14. The distance measuring device according to any one of claims 3 to 13, wherein the second group includes two or more second pixels adjacent to the first pixel in different directions, and the distance calculation unit selects a second pixel from the two or more second pixels to output a signal used for calculating the second distance.

15. The distance measuring device according to any one of claims 1 to 14, wherein the drive control unit controls the driving of the light source and the plurality of pixels to repeat a plurality of consecutive frames including a first frame and a second frame following the first frame, drives the first pixel in a first exposure sequence which includes exposing it for a first period that starts at a timing reference to the emission of the illumination light in the first frame, drives the second pixel in a second exposure sequence which includes exposing it for a second period that starts at a timing different from the first period that starts at a timing different from the emission of the illumination light in the second frame, exposes the first pixel in the second exposure sequence and exposes the second pixel in the first exposure sequence.

16. A distance measuring method using a distance measuring device, the distance measuring device comprising: a light source that emits illumination light; and a light receiving unit having a plurality of pixels that receive reflected light reflected by an object from the illumination light emitted from the light source; the distance measuring method comprising: a drive control step that controls the driving of the light source and the plurality of pixels; and a distance calculation step that calculates the distance to the object based on signals output by the plurality of pixels; the drive control step classifies the plurality of pixels into a plurality of groups to be exposed at the same timing, the plurality of pixels including a first group to which a first pixel belongs and a second group to which a second pixel adjacent to the first pixel belongs; drives the plurality of pixels such that, in the same frame, the exposure timing based on the emission of the illumination light differs among the plurality of groups; and drives the plurality of pixels such that, in a plurality of consecutive frames, the exposure timing based on the emission of the illumination light is reversed among the plurality of groups.

Images