Distance measurement imaging device and method for driving distance measurement imaging device

Abstract

A distance measurement imaging device (10) is for generating a distance image by using a TOF method, and comprises: a light source unit (11); an imaging unit (13); a drive circuit (14) that drives the imaging unit (13) to perform exposure in each of a plurality of continuous exposure periods; and a signal processing unit (15) that calculates and outputs the distance from the distance measurement imaging device (10) to a subject (5) on the basis of a signal generated by the imaging unit (13). The plurality of continuous exposure periods correspond to the distance measurement range of the distance measurement imaging device (10). In addition to the plurality of continuous exposure periods, the drive circuit (14) causes the imaging unit (13) to perform exposure in an additional exposure period in at least one of a timing that is earlier than the plurality of continuous exposure periods and a timing that is later than the plurality of continuous exposure periods, and a signal generated in the additional exposure period is added to or subtracted from a signal generated in any of the plurality of continuous exposure periods.

Description

Distance measuring imaging device and driving method for distance measuring imaging device

[0001] The present disclosure relates to a distance measuring image capturing device that generates a distance image using a TOF (Time of Flight) method, and a method for driving the distance measuring image capturing device.

[0002] A distance image measurement device has been proposed that prevents the generation of a distance image containing false distance information when a subject is located outside the distance measurement range (see, for example, Patent Document 1). Note that "distance information" is also simply called "distance" or "distance value."

[0003] The distance image measurement device of Patent Document 1 comprises a light source, a light source control means for controlling the light source, a pixel circuit, a charge transfer control means, and a distance calculation means, and the distance calculation means calculates a total value SA of the signal components of the charges generated from the pulsed light excluding background light from the detection signals S1 to S3, and if the total value SA exceeds a first threshold value Th1, calculates the distance from the detection signals S1 to S3 using a predetermined calculation formula, and if the total value SA does not exceed the first threshold value Th1, invalidates the distance calculation.

[0004] International Publication No. 2019 / 031510

[0005] However, the distance image measurement device of Patent Document 1 has the problem that, depending on the conditions such as the brightness of the subject (or the presence of a subject with high reflectivity), it is not possible to sufficiently prevent the generation of distance images containing false distance information.

[0006] Therefore, an object of the present disclosure is to provide a distance measuring image capturing device and a method for driving a distance measuring image capturing device that can more reliably prevent the generation of distance images containing false distance information than conventional devices.

[0007] In order to achieve the above object, a distance measuring and imaging device according to an embodiment of the present disclosure includes a Time of Flight (TOF) a signal processing unit that calculates and outputs a distance from the distance measuring device to the subject from the signals generated in each of the exposure periods, for each of the plurality of pixels; and a signal processing unit that calculates and outputs a distance from the distance measuring device to the subject from the signals generated in each of the plurality of exposure periods, for each of the plurality of pixels, wherein the plurality of consecutive exposure periods correspond to a distance measurement range of the distance measuring device. In addition to the plurality of consecutive exposure periods, the drive circuit exposes the imaging unit in at least one additional exposure period that is earlier than the plurality of consecutive exposure periods and / or later than the plurality of consecutive exposure periods, and the signal generated in the additional exposure period is added to or subtracted from a signal generated in any of the plurality of consecutive exposure periods.

[0008] In order to achieve the above object, a driving method of a distance measuring imaging device according to one embodiment of the present disclosure is a driving method of a distance measuring imaging device that generates a distance image using a TOF (Time of Flight) system, the distance measuring imaging device including: a light source unit that emits irradiation light toward an object; an imaging unit having a plurality of pixels that receive reflected light from the object and background light; a driving circuit that drives the light source unit to emit the irradiation light and drives the imaging unit to expose in each of a plurality of consecutive exposure periods, thereby generating, for each of the plurality of pixels, a signal indicating an amount of charge generated by exposure in each of the plurality of exposure periods; and a driving circuit that drives the light source unit to emit the irradiation light and drives the imaging unit to expose in each of the plurality of consecutive exposure periods, thereby generating, for each of the plurality of pixels, a signal indicating an amount of charge generated by exposure in each of the plurality of exposure periods. and a signal processing unit that calculates and outputs a distance from the distance measuring imaging device to the subject, wherein the plurality of consecutive exposure periods corresponds to a distance measuring range of the distance measuring imaging device, and a driving method of the distance measuring imaging device includes a step in which the driving circuit exposes the imaging unit in an additional exposure period that is earlier than the plurality of consecutive exposure periods and / or later than the plurality of consecutive exposure periods in addition to the plurality of consecutive exposure periods, and a signal generated in the additional exposure period is added to or subtracted from a signal generated in any of the plurality of consecutive exposure periods.

[0009] The present disclosure provides a distance measuring image capturing device and a method for driving a distance measuring image capturing device that can more reliably prevent the generation of distance images containing false distance information than conventional devices.

[0010] FIG. 1 is a block diagram showing the configuration of a distance measuring imaging device according to an embodiment. FIG. 2 is a layout diagram showing an example of the structure of one of the multiple pixels included in the imaging unit shown in FIG. 1. FIG. 3 is a timing chart illustrating the basic functions of a distance measuring imaging device according to an embodiment when calculating distance using a TOF method. FIG. 4 is a timing chart illustrating a problem solved by a distance measuring imaging device according to an embodiment. FIG. 5 is a timing chart illustrating exposure driving method A according to a conventional technique. FIG. 6A is a timing chart illustrating an example of operation (i.e., example of operation of Example A-1) of a distance measuring imaging device according to an embodiment that solves the problem with exposure driving method A according to the conventional technique shown in FIG. 5. FIG. 6B is a timing chart illustrating another example of operation (i.e., example of operation of Example A-2) of a distance measuring imaging device according to an embodiment that solves the problem with exposure driving method A according to the conventional technique shown in FIG. 5. FIG. 7 is a timing chart illustrating exposure driving method B according to a conventional technique. FIG. 8 is a timing chart illustrating an example of operation (i.e., example of operation of Example B-1) of a distance measuring imaging device according to an embodiment that solves the problem with exposure driving method B according to the conventional technique shown in FIG. 7. Fig. 9 is a timing chart illustrating exposure driving method C according to the conventional technology. Fig. 10 is a timing chart illustrating an example of operation (i.e., an example of operation of Example C-1) of a distance measuring imaging device according to an embodiment that solves the problem with exposure driving method C according to the conventional technology shown in Fig. 9. Fig. 11 is a flowchart illustrating an example of operation of the distance measuring imaging device according to the embodiment. Fig. 12 is a diagram illustrating experimental results regarding the operation of the distance measuring imaging device according to the embodiment.

[0011] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. Note that each embodiment described below represents a specific example of the present disclosure. The numerical values, shapes, materials, components, the arrangement and connection of the components, steps, the order of steps, and the like shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, each drawing is not necessarily an exact illustration. In each drawing, substantially identical components are assigned the same reference numerals, and duplicate explanations are omitted or simplified.

[0012] 1 is a block diagram showing the configuration of a distance measuring image capturing device 10 according to an embodiment. The distance measuring image capturing device 10 is a camera that generates a distance image using a TOF method (more specifically, pulse TOF), and includes a light source unit 11, a lens 12, an image capturing unit 13, a drive circuit 14, and a signal processing unit 15. The light source unit 11, the image capturing unit 13, the drive circuit 14, and the signal processing unit 15 are realized as a single semiconductor integrated circuit.

[0013] The light source unit 11 is a light source, such as a laser light source or an LED light source, that emits irradiation light (i.e., pulsed light) toward the subject 5 under the control of a drive circuit 14. Infrared light with a wavelength of, for example, 940 nm is used.

[0014] The lens 12 is an optical lens that collects reflected light from a subject and makes it incident on the imaging unit 13. Although not shown, a bandpass filter that transmits only light around a specific wavelength (for example, 940 nm) is provided between the lens 12 and the imaging unit 13.

[0015] The imaging unit 13 is a pixel array having a plurality of images that receives reflected light from a subject and background light that has passed through the lens 12 and is controlled by the drive circuit 14, and is, for example, a pixel array including photodiodes arranged two-dimensionally. Note that background light refers to external light that is incident even when no reflected light is incident. In addition to the pixel array, the imaging unit 13 also includes a vertical scanning circuit that controls the vertical sequential reading of signals obtained by the pixel array, an A / D converter that converts the vertically sequentially read analog signals into digital signals, and the like.

[0016] The drive circuit 14 is a circuit that drives the light source unit 11 to emit irradiation light and drives the imaging unit 13 to expose in each of a plurality of consecutive exposure periods (i.e., exposure pulses corresponding to a plurality of consecutive phases) under the control of the signal processing unit 15, thereby generating a signal indicating the amount of charge generated by exposure in each of the plurality of exposure periods in each of the plurality of pixels. Here, the plurality of consecutive exposure periods corresponds to the ranging range of the ranging imaging device 10. In other words, the timing of the start points of the plurality of consecutive exposure periods determines the shortest distance in the ranging range, while the timing of the end points of the plurality of consecutive exposure periods determines the longest distance in the ranging range.

[0017] Furthermore, as a characteristic operation, in order to detect that the subject is located outside the ranging range, the drive circuit 14 exposes the imaging unit 13 for an additional exposure period that is earlier than the multiple consecutive exposure periods and / or later than the multiple consecutive exposure periods, in addition to the multiple consecutive exposure periods. Note that a signal generated in the additional exposure period is added to or subtracted from a signal generated in one of the multiple consecutive exposure periods. The additional exposure period includes two types: an exposure period with irradiation light and an exposure period without irradiation light.

[0018] The signal processing unit 15 is a processing unit that controls the drive circuit 14, calculates the distance from the distance measuring imaging device 10 to the subject 5 (i.e., distance information) from the signals generated in each of the multiple exposure periods after addition or subtraction of the signals generated in the additional exposure periods for each of the multiple pixels, and outputs it as a distance image, and is composed of, for example, a frame memory for holding signals read out from the multiple pixels, a processor that performs signal processing according to an internal program, etc.

[0019] Here, if the value expressed by a predetermined formula (for example, the delay rate described below) using the signals generated in each of the multiple exposure periods after addition or subtraction of the signals generated in the additional exposure periods exceeds a predetermined range, the signal processing unit 15 determines that the distance to the subject is outside the ranging range (hereinafter, this will also be simply referred to as "outside the ranging range") and invalidates the distance information corresponding to that value.

[0020] Of the additions and subtractions of signals generated in the additional exposure periods, the additions are performed in the charge storage units of the multiple pixels under the control of the drive circuit 14. On the other hand, of the additions and subtractions of signals generated in the additional exposure periods, the subtractions are performed by the signal processing unit 15 that reads out the signals generated in the multiple pixels from the multiple pixels. Also, the number of exposures in the additional exposure period is less than the number of exposures in the multiple consecutive exposure periods.

[0021] Fig. 2 is a layout diagram showing an example structure of one pixel 20 among the multiple pixels included in the imaging unit 13 shown in Fig. 1. The pixel 20 includes a photodiode PD, readout gates TG1 and TG2 provided in the region of the photodiode PD, discharge gates ODG1 and ODG2, control gates VG1 to VG5 of a charge storage unit that stores charges read out from the photodiode PD during each exposure period via the readout gates TG1 and TG2, and overflow drains OD1 and OD2 that receive charges discharged from the photodiode PD via the discharge gates ODG1 and ODG2.

[0022] The control gate of the charge storage unit is controlled in the manner of a CCD (charge coupled device), and for example, by setting VG1 / 3 / 5 to a LOW potential and VG2 / 4 to a HIGH potential, charge can be stored in two regions, VG2 and VG4. The charge storage unit has control gates formed continuously further above and below in Figure 2, and by transferring and evacuating the stored signal above or below, it is also possible to store four signals per pixel.

[0023] 2, two read gates TG1 and TG2 that enable simultaneous readout of charges and corresponding charge storage units VG2 and VG4 are provided for one pixel 20. However, this embodiment is not limited to the structure of pixel 20 shown in Fig. 2, and for example, one pixel may be provided with four read gates and four corresponding charge storage units.

[0024] More specifically, in the pixel 20 shown in FIG. 2 , when the readout gate TG1 is applied to a high level, the charge generated by the photodiode PD (photoelectric conversion) is read out to the charge storage unit VG2 and added to the charge stored in the charge storage unit VG2. On the other hand, when the readout gate TG2 is applied to a high level, the charge is read out to the charge storage unit VG4 and added to the charge stored in the charge storage unit VG4. Although not shown in FIG. 2 , the charge storage units VG1 to VG5 are also arranged continuously in the vertical direction of the pixel 20. Depending on the voltage applied to the charge storage units VG1 to VG5, the charge stored in the charge storage unit can be shifted and transferred to other charge storage units located above and below. Thus, for example, the charge stored in VG2 and VG4 is temporarily transferred downward and evacuated, while the charge evacuated upward is simultaneously pulled back to the positions of VG2 and VG4. This operation allows one pixel to hold four signals.

[0025] Next, the operation of the distance measuring and imaging device 10 according to this embodiment configured as above will be described.

[0026] 3 is a timing chart illustrating the basic function of calculating distance using the TOF method in the distance measuring image pickup device 10 according to the embodiment. Note that, in order to explain only the basic function of distance calculation, the illustration of the "additional exposure period" for detecting outside the distance measurement range is omitted.

[0027] This figure shows the timings of "irradiated light" (FIG. 3(a)), "reflected light" (FIG. 3(b)), "exposure pulse (phase A0)" (FIG. 3(c)), "exposure pulse (phase A1)" (FIG. 3(d)), "exposure pulse (phase A2)" (FIG. 3(e)), and "exposure pulse (phase A3)" (FIG. 3(f)) as an example of driving by the drive circuit 14 in one cycle. Note that dots are added to the timing points where reflected light is received. The irradiated light, reflected light, and four exposure pulses all have a pulse width of time T. The four exposure pulses are an example of "plurality of consecutive exposure periods" corresponding to the distance measurement range.

[0028] In the pulse TOF method employed by the distance measuring imaging device 10, the drive circuit 14 performs one set of exposure drive per period (TOF cycle). In the example shown in the figure, reflected light is acquired across the exposure pulses of phases A0 and A1, and the signal amounts are A0 > A2 and A1 > A3. Here, A0, A1, A2, and A3 refer to the signal amounts (or signals) acquired by the exposure pulses of phases A0, A1, A2, and A3, respectively. Note that the acquired signal amount includes not only the component due to the irradiated light (i.e., reflected light) but also external light components (i.e., background light) due to sunlight, indoor lighting, etc., so the signal processing unit 15 calculates A0 - A2 and A1 - A3 to exclude the external light component (external light subtraction).

[0029] Since the pulse width of the reflected light and the four exposure pulses is T, the delay time Δt from the exposure pulse of phase A0 to the timing of receiving the reflected light can be calculated as Δt = T × (A1 - A3) / (A0 - A2 + A1 - A3), and the distance D between the subject and the distance measuring image pickup device 10 (hereinafter also referred to as the "distance value") can be calculated as D = (T_A0 + Δt) / 2 × c (where T_A0 is the delay time from the irradiated light to the exposure pulse of phase A0, and c is the speed of light). Note that the denominator (A0 - A2 + A1 - A3) in the equation for the delay time Δt corresponds to the net amount of charge generated by the reflected light and is also referred to as the "IR amount." Furthermore, the coefficient by which the time T is multiplied in the calculation of the delay time Δt (in this example, (A1 - A3) / (A0 - A2 + A1 - A3)) is a predetermined formula used to determine whether the subject is outside the distance measurement range and is referred to as the "delay rate."

[0030] 3, if the reflected light is detected across phases A1 and A2, the signal amounts are A2 > A0 and A1 > A3, and the delay time Δt in the reception timing of the reflected light relative to the exposure pulse of phase A1 is calculated as Δt = T × (A2 - A0) / (A2 - A0 + A1 - A3). Therefore, the distance D between the subject and the distance measuring image pickup device 10 is calculated as D = (T_A1 + Δt) / 2 × c (where T_A1 is the delay time from the irradiated light to the exposure pulse of phase A1, and c is the speed of light). In this example, the delay rate is (A2 - A0) / (A2 - A0 + A1 - A3).

[0031] Similarly, when reflected light is detected across phases A2 and A3, the signal amounts are A2 > A0 and A3 > A1, and the delay time Δt in the reception timing of the reflected light relative to the exposure pulse of phase A2 is calculated as Δt = T × (A3 - A1) / (A2 - A0 + A3 - A1). Therefore, the distance D between the subject and the distance measuring image pickup device 10 is calculated as D = (T_A2 + Δt) / 2 × c (where T_A2 is the delay time from the irradiated light to the exposure pulse of phase A2, and c is the speed of light). In this example, the delay rate is (A3 - A1) / (A2 - A0 + A3 - A1).

[0032] When detecting outside the distance measurement range using only multiple consecutive exposure periods without including such additional exposure periods, if reflected light is not detected at the timing of any exposure pulse, or if it is detected only at the timing of the earliest exposure pulse (phase A0), or if it is detected only at the timing of the latest exposure pulse (phase A3), it will be determined to be "outside the distance measurement range."

[0033] The amount of signal (IR amount) generated by reflection from the subject attenuates inversely proportional to the square of the distance to the subject. Pixels that do not meet a predetermined IR amount may be determined by the signal processing unit 15 to be "outside the distance measurement range" because an accurate distance cannot be calculated. The IR level (i.e., threshold) at which an accurate distance cannot be calculated is called the coring level.

[0034] 4 is a timing chart illustrating a problem solved by the distance measuring imaging device 10 according to the embodiment. Here, the timing chart is shown in which examples of timings of reflected light that should be determined to be “outside the distance measuring range” (four reflected lights (i) to (iv)) are added to the timing chart shown in FIG. 3 .

[0035] As described above, in pulse TOF, if reflected light returns at a timing that is different from the timing of a group of consecutive exposure pulses, this can be detected and recognized as "out of the distance measurement range." For example, in the timing example of reflected light (ii) in Figure 4, the signal amount A0 is sufficiently larger than all of the signal amounts A1, A2, and A3, and in the timing example of reflected light (iii), the signal amount A3 is sufficiently larger than all of the signal amounts A0, A1, and A2, so it can be determined to be "out of the distance measurement range." Furthermore, in the timing examples of reflected light (i) and (iv), the reflected light does not overlap in time with any of the exposure pulses (phases A0 to A3), so the signal amounts A0 to A3 are small, usually below the coring level, and it can be determined to be out of range.

[0036] However, signals generated by the parasitic sensitivity of the pixel 20 may cause erroneous determination. Parasitic sensitivity refers to signals that are generated even when the pixel shutter (i.e., the readout gates TG1 / TG2 in FIG. 2) is closed, and is mainly generated when light directly enters the charge storage sections VG1 to VG5 of the pixel 20. There are mainly two types of signals generated by parasitic sensitivity: those generated by external light and those generated by irradiated light (in other words, reflected light).

[0037] Parasitic sensitivity has a large effect in the following situations: 1) when the subject is exposed to strong sunlight, 2) when the subject acts like a retroreflector and the irradiated light is returned with high intensity without being scattered, and 3) when the subject is very close to the distance measuring imaging device and the irradiated light is returned without being attenuated.

[0038] When the reflected light is at the timing of reflected light (i) or (iv), the signal amount obtained in the "several consecutive exposure periods" (i.e., the exposure pulses of phases A0 to A3) used to calculate the distance to the subject is small, and the influence of the charge generated by parasitic sensitivity becomes relatively large. In this case, the random fluctuations in the charge generated by parasitic sensitivity and dark current (i.e., shot noise) rather than the signal amount become large, and even if the signals are subtracted, the residual error may exceed the coring level. For such pixels, a random and incorrect distance value is calculated, but because it exceeds the coring level, conventional methods cannot determine that the pixel is out of range.

[0039] Therefore, in the distance measuring imaging device 10 according to the present embodiment, in order to suppress the influence of parasitic sensitivity and accurately detect out-of-range conditions, the drive circuit 14 exposes the imaging unit 13 to multiple consecutive exposure periods (i.e., exposure pulses of phases A0 to A3) as well as additional exposure periods that are earlier than the multiple consecutive exposure periods and / or later than the multiple consecutive exposure periods. The signal generated in the additional exposure period is added to or subtracted from the signal generated in one of the multiple consecutive exposure periods. This prevents random fluctuations (i.e., shot noise) from causing random, incorrect distance values ​​to be calculated. Below, examples of "additional exposure periods" and examples of criteria for determining "out-of-range conditions" are described for each of the various exposure drive methods.

[0040] First, exposure driving method A according to a conventional technique, which is one of various exposure driving methods, will be described. Fig. 5 is a timing chart illustrating exposure driving method A according to the conventional technique. Fig. 5(a) shows the timing relationship between light emission and each exposure pulse, and Fig. 5(b) shows the timing of light emission and each exposure pulse in one frame. Note that, for convenience of explanation, this figure only explains exposure driving method A, and explains exposure driving method A according to the conventional technique without adding the characteristic "additional exposure period" of the distance measuring imaging device 10 according to the embodiment.

[0041] In order to implement this exposure driving method A, a pixel type is used in which one pixel has two gates (two readout gates TG1 and TG2) and four charge storage sections are provided in one pixel, as shown in Figure 2.

[0042] In exposure driving method A, first, the exposure packets from two readout gates are set to the timing of phase A0 and phase A1 in the figure, and light emission exposure driving is performed to acquire reflected light (Period 1). The acquired signals A0 and A1 are saved in separate charge storage units. Note that phases A0 and A1 in Period 1 are examples of multiple consecutive exposure periods corresponding to the distance measurement range, and are respectively examples of a first exposure period for receiving reflected light and a second exposure period that follows the first exposure period.

[0043] Next, exposure packets for phases A2 and A3 are set, and background light is acquired by performing only exposure driving without emitting light (Period 2). The acquired signals A2 and A3 are saved in separate charge accumulation units. Note that phases A2 and A3 in Period 2 are examples of consecutive exposure periods corresponding to the ranging range, and are respectively examples of a third exposure period for receiving background light corresponding to the first exposure period and a fourth exposure period for receiving background light corresponding to the second exposure period, which follows the third exposure period.

[0044] This process of acquiring reflected light (Period 1) and background light (Period 2) is repeated, for example, 10,000 times in one frame, and signals A0 to A3 are accumulated individually. The four resulting signals A0 to A3 are read out from the pixels and A / D converted, completing one frame (i.e., completing the main light emission exposure drive including the exposure period corresponding to the distance measurement range).

[0045] Although the exposure pulses of phases A0 and A1 are shown as being continuous in time, it is sufficient that the distance corresponding to this time range can be measured continuously, and there may be a slight overlap or gap between the exposure pulses.

[0046] 5, if the reflected light is delayed further than the timing of the final phase A1, both the signals of phases A0 and A1 will have extremely small values. As a result, the signal due to the parasitic sensitivity will become dominant, and if it exceeds the coring level, an indefinite distance value will be calculated.

[0047] For example, when the maximum value (saturation amount) of the signal is set to 4095 LSB (value after A / D conversion), a parasitic sensitivity due to reflected light outside the distance measurement range occurs in signals A0 and A1 by 10 LSB each (at this time, signals A2 and A3 are not irradiated and are therefore zero), and the distance value is calculated as if the light was reflected from a position where signal amount A0 = signal amount A1.

[0048] Depending on the scene to be measured, the closest distance within the measurement range may be set to a relatively long distance (for example, 1 m or more). In this case, if the subject is located closer than 1 m, the irradiated light may return earlier than the timing of phase A0. In this case, both the phase A0 and phase A1 signals may have extremely small values, and an indefinite distance value may be calculated due to charges generated by parasitic sensitivity.

[0049] Fig. 6A is a timing chart showing an example of operation (i.e., an example of operation of Example A-1) of the distance measuring imaging device 10 according to an embodiment that solves the problem with exposure driving method A according to the conventional technology shown in Fig. 5. (a) of Fig. 6A shows the timing relationship between light emission and each exposure pulse, and (b) of Fig. 6A shows the timing of light emission and each exposure pulse in one frame.

[0050] In this embodiment, after the light emission exposure driving of phases A0 and A1 (Period 1) is completed, the drive circuit 14 sets the exposure packet for the final phase A1 signal to an even later timing and executes additional light emission exposure driving (Period 5). Note that phase A1 in Period 5 is an example of a sixth exposure period for receiving reflected light at a later timing than the second exposure period (phase A1 in Period 1) and adding it to the signal generated in the second exposure period.

[0051] In (a) of Figure 6A, the phase "A1" in Period 1 and the phase "A1" in Period 5 are expressed as the same phase, which means that signals expressed as the same phase in multiple places are added together in pixel 20. The same applies to other figures.

[0052] As a result, the reflected light acquired during the sixth exposure period outside the ranging range is added as signal A1 in pixel 20, and signal processing unit 15 uses the added signal A1 to calculate distance D using a calculation formula including the above-mentioned delay time Δt. Here, since signal A1 obtained at the latest timing is sufficiently larger than the other signals, for example, when the above-mentioned delay rate is close to 1 (for example, greater than 0.99), signal processing unit 15 can determine that the subject is located at the farthest end and invalidate distance D calculated using the above-mentioned calculation formula (for example, replace it with a value indicating a distance farther than the ranging range).

[0053] In the presence of external light such as sunlight, adding an exposure to only signal A1 for the sixth exposure period results in a large amount of external light being accumulated only in signal A1, preventing the above-described external light subtraction (A1-A3) from being performed correctly. Therefore, when adding an exposure to signal A1, the drive circuit 14 adds the same number of exposures with the same exposure width to signal A3 (Period 6). In this case, no light is emitted because only external light is exposed. Phase A3 in Period 6 is an example of an eighth exposure period for receiving background light corresponding to the sixth exposure period (phase A1 in Period 5) and subtracting it from the signal generated in the sixth exposure period.

[0054] Furthermore, since the amount of accumulated external light increases as the number of additional exposures increases, it is desirable to set the number of additional exposures (Periods 5 and 6) to be less than the main light emission exposure drive (i.e., the light emission exposure drive used for distance calculation), and more preferably, to 1 / 10 or less of the main light emission exposure drive.

[0055] Furthermore, to prevent an indefinite distance value from being calculated when reflected light returns at a timing earlier than the timing of phase A0, after the light emission exposure driving of phases A0 and A1 is completed, the drive circuit 14 sets the exposure packet for the first phase A0 signal to an even earlier timing and executes light emission exposure driving (Period 3). Note that phase A0 in Period 3 is an example of a fifth exposure period for receiving reflected light at a timing earlier than the first exposure period (phase A0 in Period 1) and adding it to the signal generated in the first exposure period.

[0056] This allows the reflected light acquired during the fifth exposure period outside the ranging range to be added in pixel 20 as signal A0, and therefore signal processing unit 15 determines that the subject is outside the ranging range on the close-up side when, for example, the above-mentioned delay rate is close to zero (for example, smaller than 0.01), since signal A0, which is the earliest timing, is sufficiently larger than the other signals, and can invalidate the distance D calculated by the above-mentioned formula (for example, replace it with a value indicating a distance closer than the ranging range).

[0057] As in the case where the subject is determined to be located at the farthest end, in this example, in order to correctly subtract external light, when additional exposure is performed on signal A0, the drive circuit 14 adds the same number of exposures to signal A2 (Period 4). In this case, no light is emitted because only external light is exposed. Phase A2 in Period 4 is an example of a seventh exposure period for receiving background light corresponding to the fifth exposure period (phase A0 in Period 3) and subtracting it from the signal generated in the fifth exposure period.

[0058] Both Periods 3 and 4 and Periods 5 and 6 may be implemented, or only one of them may be implemented.

[0059] However, in this exposure driving method A, when the additional exposure of phase A1 is performed, the resolution of the lens (for example, defocus) may cause some of the light from the additional exposure to leak into the pixels around the subject, causing the coring level to be exceeded at the contour of the subject that should normally be determined as background, resulting in an inconsistent distance output. In this case, the following example may be used.

[0060] Figure 6B is a timing chart showing another example of operation (i.e., an example of operation of Example A-2) of the distance measuring imaging device 10 according to an embodiment that solves the problems with the exposure driving method A according to the conventional technology shown in Figure 5.

[0061] In this embodiment, after the light emission exposure drive (Period 1) for phases A0 and A1 is completed, the drive circuit 14 executes additional light emission exposure drive (Period 4) by setting the exposure timing for the phase A2 signal that acquires the amount of external light at phase A0 to a timing that is even later than phase A1. Phase A2 in Period 4 is an example of a fifth exposure period for receiving reflected light at a timing later than phase A1 in the second exposure period (Period 1) and adding it to the signal generated in the third exposure period (Phase A2 in Period 2).

[0062] Before the additional exposure (only Periods 1 and 2), if reflected light returned from outside the distance measurement range on the far side, the signal A0 after subtracting external light (i.e., A0-A2) generated an output due to parasitic sensitivity and could not be distinguished from reflected light from a subject within the range, but with the additional exposure, the signal A2 exceeds the parasitic sensitivity of the signal A0, and A0-A2 becomes negative. Because A0-A2 will never be negative if it is within the distance measurement range, the signal processing unit 15 can determine whether it is outside the distance measurement range by determining whether A0-A2 is negative.

[0063] As in Example A-1, if exposure is added only to signal A2 in the presence of external light such as sunlight, a large amount of external light accumulates only in signal A2, preventing the above-described external light subtraction (A0-A2) from being performed correctly. Therefore, when exposure is added to signal A2, the drive circuit 14 adds the same number of exposures with the same exposure width to signal A0 (Period 3). In this case, no light is emitted because only external light is exposed. Phase A0 in Period 3 is an example of a seventh exposure period for receiving background light corresponding to the fifth exposure period (phase A2 in Period 4) and subtracting it from the signal generated in the fifth exposure period.

[0064] Furthermore, since the amount of accumulated external light increases as the number of additional exposures increases, it is desirable to set the number of additional exposures (Periods 3 and 4) to be less than that of the main light emission exposure drive, and more preferably to set it to 1 / 10 or less of the main light emission exposure drive.

[0065] The additional exposure may be phase A3 (Period 6), which is paired with phase A0 (the first exposure), instead of or in addition to phase A2, which is paired with phase A1 (the final exposure). Phase A3 in Period 6 is an example of a sixth exposure period in which reflected light is received later than phase A1 in Period 1 and added to the signal generated in the fourth exposure period (phase A3 in Period 2). In this case, the same number of exposures with the same exposure width are added to signal A1 (Period 5), as described above, to ensure proper ambient light subtraction. Phase A1 in Period 5 is an example of an eighth exposure period in which background light corresponding to the sixth exposure period (phase A3 in Period 6) is received and subtracted from the signal generated in the sixth exposure period.

[0066] Both Periods 3 and 4 and Periods 5 and 6 may be implemented, or only one of them may be implemented.

[0067] In Example A-1, additional exposures are performed at phases A0 and A1, so if there is lens defocus or the like, additional indefinite signals are generated in the background around the subject, which can cause the coring level to be exceeded. However, in Example A-2, the indefinite signals generated by the additional exposures at phases A2 and A3 act to reduce signals A0 and A1 when background light is subtracted, so the signal levels of surrounding pixels do not exceed the coring level, and an erroneous distance value is not calculated, which can occur in Example A-1.

[0068] Next, exposure driving method B according to a conventional technique, which is one of various exposure driving methods, will be described. Fig. 7 is a timing chart illustrating exposure driving method B according to the conventional technique. Fig. 7(a) shows the timing relationship between light emission and each exposure pulse, and Fig. 7(b) shows the timing of light emission and each exposure pulse in one frame. Note that, for convenience of explanation, this figure only explains exposure driving method B, and explains exposure driving method B according to the conventional technique, which does not include the characteristic "additional exposure period" of the distance measuring imaging device 10 according to the embodiment.

[0069] In order to implement this exposure driving method B, a pixel type is used in which one pixel has two gates (two readout gates TG1 and TG2) and four charge storage sections are provided in one pixel, as shown in Figure 2.

[0070] In exposure driving method B, first, the exposure packets from the two readout gates are set to the timing of phase A0 and phase A1 in the figure, and light emission exposure driving is performed to acquire reflected light (Period 1). The acquired signals A0 and A1 are saved in separate charge storage units. Note that phases A0 and A1 in Period 1 are examples of a first exposure period for receiving the first reflected light, and a second exposure period that follows the first exposure period, respectively.

[0071] Next, exposure packets for phases A2 and A3 are set, light emission exposure drive is performed, and reflected light is acquired (Period 2). The acquired signals A2 and A3 are saved in separate charge accumulation units. Note that phases A2 and A3 in Period 2 are examples of a third exposure period that follows the second exposure period (phase A1 in Period 1) for receiving second reflected light, and a fourth exposure period that follows the third exposure period, respectively.

[0072] Furthermore, phases A0 and A1 in Period 1 and phases A2 and A3 in Period 2 are examples of a plurality of consecutive exposure periods corresponding to the distance measurement range.

[0073] This process of acquiring reflected light (Period 1) and background light (Period 2) is repeated, for example, 10,000 times in one frame, and signals A0 to A3 are accumulated individually. The four resulting signals A0 to A3 are read out from the pixels and A / D converted, completing one frame (i.e., completing the main light emission exposure drive including the exposure period corresponding to the distance measurement range).

[0074] In addition, in exposure driving method B, distance measurement is performed using four exposure packets, so the distance measurement range is wider than in exposure driving method A, but power consumption is higher because twice as much irradiation light is required.

[0075] Furthermore, although the exposure pulses of phases A0 to A3 are shown as being continuous in time, it is sufficient that the distance corresponding to this time range can be measured continuously, and there may be some overlap or gap between the exposure pulses.

[0076] 7, if the reflected light is delayed further than the timing of the final phase A3, all of the signals for phases A0 to A3 become extremely low, and as a result, signals due to dark output and parasitic sensitivity become dominant, and if they exceed the coring level, an indefinite distance value will be calculated.

[0077] Depending on the scene to be measured, the closest distance within the measurement range may be set to a relatively long distance (e.g., 1 m or more). In this case, if the subject is located closer than 1 m, the irradiated light will return earlier than the timing of phase A0. Because the signal amount is inversely proportional to the square of the distance, light reflected from outside the close-distance measurement range returns to the pixel while maintaining very high intensity. As a result, the signal due to parasitic sensitivity at phases A0 to A3 also becomes very large. Due to external light subtraction (i.e., A0 - A2, A1 - A3), the signal becomes zero on average, but when viewed pixel by pixel, the shot noise alone may exceed the coring level, resulting in an indeterminate distance value being calculated for that pixel.

[0078] Fig. 8 is a timing chart showing an example of operation (i.e., an example of operation of Example B-1) of the distance measuring imaging device 10 according to an embodiment that solves the problem with exposure driving method B according to the conventional technology shown in Fig. 7. Fig. 8(a) shows the timing relationship between light emission and each exposure pulse, and Fig. 8(b) shows the timing of light emission and each exposure pulse in one frame.

[0079] In this embodiment, after the light emission exposure driving of phases A0 to A3 (Periods 1 and 2) is completed, the exposure packet for the final phase A3 signal is set to an even later timing and light emission exposure driving is executed (Period 4). Note that phase A3 in Period 4 is an example of a sixth exposure period for receiving the second reflected light at a later timing than the fourth exposure period (phase A3 in Period 2) and adding it to the signal generated in the fourth exposure period.

[0080] As a result, the reflected light acquired during the sixth exposure period outside the ranging range is added as signal A3, and signal A3 of the last phase A3 becomes sufficiently larger than the other signals. Therefore, for example, when the delay rate described above is close to 1 (for example, greater than 0.99), the signal processing unit 15 determines that the subject is located at the farthest end, and can invalidate the distance D calculated by the above-mentioned formula (for example, replace it with a value indicating a distance farther than the ranging range).

[0081] In the presence of external light such as sunlight, adding exposure only to signal A3 results in excessive accumulation of external light only in signal A3, preventing proper external light subtraction (A3-A1) as described above. Therefore, when adding exposure in phase A3, the drive circuit 14 adds the same number of exposures with the same exposure width to signal A1 (Period 3). In this case, no light is emitted because only external light is exposed. Phase A1 in Period 3 is an example of an eighth exposure period for receiving background light corresponding to the sixth exposure period (phase A3 in Period 4) and subtracting it from the signal generated in the sixth exposure period.

[0082] Furthermore, since the amount of accumulated external light increases as the number of additional exposures increases, it is desirable to set the number of additional exposures (Periods 5 and 6) to be less than that of the main light emission exposure drive, and more preferably to set it to 1 / 10 or less of the main light emission exposure drive.

[0083] Furthermore, to prevent an indefinite distance value from being calculated when reflected light returns earlier than the timing of phase A0, after completing the light emission exposure drive of phases A0 to A3 (Periods 1 and 2), the drive circuit 14 sets the exposure packet for the first phase A0 signal to an even earlier timing and executes light emission exposure drive (Period 5). Note that phase A0 in Period 5 is an example of a fifth exposure period for receiving the first reflected light earlier than the first exposure period (phase A0 in Period 1) and adding it to the signal generated in the first exposure period.

[0084] At this time, in order to correctly subtract external light as described above, when the drive circuit 14 performs an additional exposure at phase A0, it also performs an additional exposure at phase A2 the same number of times (Period 6) and exposes to external light during the added phase A2. Phase A2 in Period 6 is an example of a seventh exposure period for receiving background light corresponding to the fifth exposure period (phase A0 in Period 5) and subtracting it from the signal generated during the fifth exposure period.

[0085] As a result, reflected light outside the short-distance measurement range can be acquired by signal A0, and therefore signal A0 with the earliest timing becomes sufficiently larger than the other signals. Therefore, for example, when the delay rate described above is close to zero (for example, smaller than 0.01), signal processing unit 15 can determine that the subject is outside the short-distance measurement range and invalidate the distance D calculated by the above-mentioned formula (for example, replace it with a value indicating a distance closer than the measurement range). Note that, because the light reflected outside the short-distance measurement range is very strong, signal A0 can reach saturation even after only a few exposures, making it possible to determine that an object is present outside the short-distance measurement range.

[0086] Next, exposure driving method C according to a conventional technique, which is one of various exposure driving methods, will be described. Fig. 9 is a timing chart illustrating exposure driving method C according to the conventional technique. Fig. 9(a) shows the timing relationship between light emission and each exposure pulse, and Fig. 9(b) shows the timing of light emission and each exposure pulse in one frame. Note that, for convenience of explanation, this figure only explains exposure driving method C, and explains exposure driving method C according to the conventional technique, which does not include the characteristic "additional exposure period" of the distance measuring imaging device 10 according to the embodiment.

[0087] In order to implement this exposure driving method C, a pixel type is used that differs from the pixel shown in FIG. 2 in that one pixel has four gates (four readout gates) and one pixel has four charge storage sections.

[0088] In exposure driving method C, the exposure packets generated by the four readout gates are set to the timing of phases A0 to A3 in the figure to perform light emission exposure driving, thereby acquiring reflected light (Period 1). Note that phases A0 to A3 in Period 1 are an example of a plurality of consecutive exposure periods corresponding to the ranging range, and are respectively an example of a first exposure period for receiving reflected light and background light, a second exposure period that follows the first exposure period, a third exposure period that follows the second exposure period, and a fourth exposure period that follows the third exposure period. The four resulting signals A0 to A3 are read out from the pixels and A / D converted, thereby completing one frame (i.e., completing the main light emission exposure driving including the exposure period corresponding to the ranging range).

[0089] Although the exposure pulses of phases A0 to A3 are shown as being continuous in time, it is sufficient that the distance corresponding to this time range can be measured continuously, and there may be some overlap or gap between the exposure pulses.

[0090] 9, if the reflected light is delayed further than the timing of the final phase A3, all signals will be extremely low. In this case, signals due to dark output and parasitic sensitivity will become dominant, and if they exceed the coring level, an indefinite distance value will be calculated.

[0091] Depending on the scene to be measured, the closest distance within the measurement range may be set to a relatively long distance (e.g., 1 m or more). In this case, if the subject is located closer than 1 m, the irradiated light will return earlier than the timing of phase A0. Because the signal amount is inversely proportional to the square of the distance, light reflected from outside the close-distance measurement range returns to the pixel while maintaining very high intensity. As a result, the signal due to the parasitic sensitivity of phases A0 to A3 also becomes very large. Due to external light subtraction (i.e., A0 - A2, A1 - A3), the signal becomes zero on average, but when viewed pixel by pixel, the shot noise alone may exceed the coring level, resulting in an indeterminate distance value being calculated for that pixel.

[0092] Fig. 10 is a timing chart showing an example of operation (i.e., an example of operation of Example C-1) of the distance measuring imaging device 10 according to an embodiment that solves the problem with exposure driving method C according to the conventional technology shown in Fig. 9. (a) of Fig. 10 shows the timing relationship between light emission and each exposure pulse, and (b) of Fig. 10 shows the timing of light emission and each exposure pulse in one frame.

[0093] In this embodiment, after the light emission exposure driving of phases A0 to A3 (Period 1) is completed, the exposure packet for the final phase A3 signal is set to an even later timing and an additional light emission exposure driving is performed, and in addition, the exposure packet for the first phase A0 signal is set to an earlier timing and an additional light emission exposure driving is performed (Period 2). Note that phases A0 and A3 in Period 2 are an example of a fifth exposure period for receiving reflected light at an earlier timing than the first exposure period (phase A0 in Period 1) and adding it to the signal generated in the first exposure period, and a sixth exposure period for receiving reflected light at a later timing than the fourth exposure period (phase A3 in Period 1) and adding it to the signal generated in the fourth exposure period.

[0094] As a result, if the reflected light returns at a time outside the distance measurement range on the short distance side, a signal can be acquired at phase A0, and if the reflected light returns at a time outside the distance measurement range on the long distance side, a signal can be acquired at phase A3.Therefore, an indefinite distance value is not calculated, and signal A0 or signal A3 becomes sufficiently larger than other signals.Therefore, for example, if the above-mentioned delay rate is close to zero (for example, smaller than 0.01) or close to 1 (for example, larger than 0.99), the signal processing unit 15 can determine that the subject is at the closest or farthest end, and can invalidate the distance D calculated by the above-mentioned formula (for example, replace it with a value indicating a distance closer or farther than the distance measurement range).

[0095] However, in the presence of external light such as sunlight, adding exposures only to phases A0 and A3 results in excessive accumulation of external light only in phases A0 and A3, preventing the aforementioned external light subtraction (A0-A2, A3-A1) from being performed correctly. Therefore, when exposures to phases A0 and A3 are added, the same number of exposures with the same exposure width are added to signals A2 and A1 (Period 3). In this case, no light is emitted because only external light is exposed. Phases A2 and A1 in Period 3 are examples of a seventh exposure period for receiving background light corresponding to the fifth exposure period (phase A0 in Period 2) and subtracting it from the signal generated in the fifth exposure period, and an eighth exposure period for receiving background light corresponding to the sixth exposure period (phase A3 in Period 2) and subtracting it from the signal generated in the sixth exposure period.

[0096] Furthermore, since the amount of accumulated external light increases as the number of additional exposures increases, it is desirable to make the number of additional exposures (Periods 2 and 3) less than that of the main light emission exposure drive, and more preferably, 1 / 10 or less of that of the main light emission exposure drive.

[0097] 11 is a flowchart showing an example of operation (i.e., a driving method) of the distance measuring image capturing device 10 according to the embodiment. This flowchart shows the operation procedure of the distance measuring image capturing device 10 common to the above-mentioned Examples A-1 and A-2, Example B-1, and Example C-1, and shows the procedure of processing performed on each pixel in one frame unit.

[0098] First, the drive circuit 14 performs main light emission exposure drive including an exposure period corresponding to the distance measurement range (i.e., light emission exposure drive including a plurality of consecutive exposure periods) (S10). In the example A-1 shown in FIG. 6A, this corresponds to the processing of Period1 and Period2.

[0099] Next, the drive circuit 14 performs light emission exposure drive including an additional exposure period for detecting outside the distance measurement range (S11). In other words, the drive circuit 14 performs light emission exposure drive including an additional exposure period that is earlier or later than the multiple consecutive exposure periods. In the example A-1 shown in FIG. 6A, this corresponds to the processing of Periods 3 to 6.

[0100] Here, the signal obtained in the additional exposure period is added to the signal obtained in the exposure period of the same phase in the pixel 20 (S12).

[0101] When the light emission exposure drive is completed, the signal processing unit 15 reads out the signals obtained from the pixels in each exposure period, and calculates the distance D for each pixel using the read out signals according to the above-mentioned calculation formula (S13).

[0102] At this time, the signal processing unit 15 determines for each pixel whether the delay rate included in the calculation formula for calculating the distance D is close to zero (e.g., <0.01) or close to 1 (e.g., >0.99) (S14). If the determination is affirmative (Yes in S14), the signal processing unit 15 determines that the distance to the subject is outside the ranging range (the subject is at the nearest or farthest end), and invalidates the distance D obtained in step S13 (S15). For example, the signal processing unit 15 replaces the distance D obtained in step S13 with a value indicating a distance closer or farther than the ranging range. Note that if the determination is negative in step S14 (No in S14), the signal processing unit 15 determines that the distance D obtained in step S13 is correct distance information and leaves it as is.

[0103] In Example A-2, if the signal amount after subtracting external light becomes negative, rather than the delay rate, it is determined that the distance is outside the distance measurement range, and the distance D is invalidated.

[0104] Thus, according to the distance measuring imaging device 10 of this embodiment, in addition to the main light emission exposure drive (i.e., light emission exposure drive including multiple consecutive exposure periods) including an exposure period corresponding to the distance measuring range, an additional light emission exposure drive is performed to detect outside the distance measuring range, so that the signal obtained in the additional exposure period is added to the signal obtained in the exposure period of the same phase in the pixel 20, thereby avoiding the calculation of an indefinite distance value due to parasitic sensitivity and more reliably suppressing the generation of a distance image including false distance information than conventional methods.

[0105] 12 is a diagram illustrating the results of an experiment on the operation of the distance measuring image pickup device 10 according to the embodiment. As measurement conditions, exposure driving method A was adopted, the distance measurement range was 0.2-1.2 m, and the farthest end of the distance measurement range was trimmed to 4000 LSB.

[0106] 12A is a diagram showing a measurement environment in which two objects (a first object 30 and a second object 31) are viewed from the distance measuring image capturing device 10. The first object 30 is a reflector with a reflectivity of 90% and is placed at a position 0.54 m in front of the distance measuring image capturing device 10. The second object 31 is a retroreflector placed at a position 3.1 m in front of the distance measuring image capturing device 10.

[0107] Figure 12(b) shows an example of a distance image obtained by imaging using exposure drive method A (i.e., exposure drive without an additional exposure period) according to the conventional technique shown in Figure 5 in the measurement environment shown in Figure 12(a). In Figure 12(b), the darker the shaded areas (i.e., areas with higher dot density), the closer the distance value. White indicates a distance of 4000 LSB, which corresponds to the farthest end of the distance measurement range of 0.2-1.2 m or a position farther than the farthest end.

[0108] 12(b), the first subject 30 displays normal distance information 30a, but the second subject 31, which is located outside the distance measurement range, displays false distance information (specifically, 2000 LSB) 31a instead of white (i.e., 4000 LSB). This is because the second subject 31 has extremely high reflectance, and false distance information is generated due to the parasitic sensitivity of the pixels.

[0109] Fig. 12(c) is a diagram showing an example of a distance image obtained by imaging using exposure driving method A (i.e., the light emission exposure driving in Example A-2) including the additional exposure period shown in Fig. 6B in the measurement environment shown in Fig. 12(a). In Fig. 12(c), the false distance information 31a that appeared in the distance image shown in Fig. 12(b) has disappeared. This is because the distance measuring image capturing device 10 in Example A-2 determined that the second subject 31 was located outside the distance measurement range and replaced the false distance information 31a of the second subject 31 with the distance at the farthest end of the distance measurement range (i.e., 4000 LSB).

[0110] As described above, the distance measuring image capturing device 10 according to the embodiment is a device for generating a distance image using a TOF method, and includes a light source unit 11 that emits illumination light toward the subject 5, an image capturing unit 13 having a plurality of pixels that receive reflected light from the subject 5 and background light, a drive circuit 14 that drives the light source unit 11 to emit illumination light and drives the image capturing unit 13 to expose for each of a plurality of successive exposure periods, thereby generating a signal indicating the amount of charge generated by exposure for each of the plurality of exposure periods, and a drive circuit 15 that drives the light source unit 11 to emit illumination light and drives the image capturing unit 13 to expose for each of a plurality of successive exposure periods, thereby generating a signal indicating the amount of charge generated by exposure for each of the plurality of pixels. and a signal processing unit 15 that calculates and outputs the distance from the distance measuring image pickup device 10 to the subject 5 from the signals generated in each of the multiple exposure periods, wherein the multiple consecutive exposure periods correspond to the distance measurement range of the distance measuring image pickup device 10, and the drive circuit 14 exposes the image pickup unit 13 in additional exposure periods in addition to the multiple consecutive exposure periods, at least one of which is earlier than the multiple consecutive exposure periods and which is later than the multiple consecutive exposure periods, and the signal generated in the additional exposure period is added to or subtracted from the signal generated in any of the multiple consecutive exposure periods.

[0111] As a result, in addition to the light emission exposure drive including a plurality of consecutive exposure periods corresponding to the ranging range, an additional light emission exposure drive is performed to detect outside the ranging range, and the signal obtained in the additional exposure period is added to or subtracted from the signal obtained in the corresponding exposure period of the plurality of consecutive exposure periods. Therefore, by determining whether or not the ranging range is exceeded using the signal after addition or subtraction, it is possible to avoid the calculation of an indefinite distance value due to parasitic sensitivity as in the conventional method, and more reliably than in the conventional method, the generation of a distance image containing false distance information is suppressed.

[0112] At this time, the signal processing unit 15 determines that the distance is outside the ranging range when, for example, a value expressed by a predetermined formula (e.g., a delay rate) using signals generated in a plurality of consecutive exposure periods after addition or subtraction of signals generated in the additional exposure periods exceeds a predetermined range (e.g., a range of 0.01 to 0.99). This makes it possible to reliably determine that the distance is outside the ranging range using numerical values ​​and threshold values.

[0113] Each of the plurality of pixels has a charge storage section for storing charge generated in at least one of the plurality of consecutive exposure periods, and the addition of signals generated in the additional exposure periods is performed in the charge storage section under the control of the drive circuit 14. As a result, the addition of signals is performed in the circuit in the pixel without using a special adder, and the value after addition can be obtained by reading it out only once.

[0114] Furthermore, subtraction of signals generated in the additional exposure period is performed by the signal processing unit 15, which reads out signals generated in the multiple pixels from the multiple pixels. Thus, subtraction of signals generated in the additional exposure period can be performed by calculation in the signal processing unit 15.

[0115] Furthermore, the number of exposures in the additional exposure period is smaller than the number of exposures in the consecutive exposure periods, which can prevent the amount of accumulated external light from increasing due to exposure in the additional exposure period.

[0116] In Example A-1, the consecutive exposure periods include a first exposure period A0 for receiving reflected light, a second exposure period A1 that follows the first exposure period A0, a third exposure period A2 for receiving background light corresponding to the first exposure period A0, and a fourth exposure period A3 that follows the third exposure period A2 and receives background light corresponding to the second exposure period A1, and the additional exposure periods include at least one of a fifth exposure period A0 for receiving reflected light at an earlier timing than the first exposure period A0 and adding it to a signal generated in the first exposure period A0, and a sixth exposure period A1 for receiving reflected light at a later timing than the second exposure period A1 and adding it to a signal generated in the second exposure period A1. As a result, under exposure driving method A, which is one of various exposure driving methods, addition or subtraction of signals generated in the additional exposure periods is performed, and cases outside the distance measurement range can be reliably detected. In addition, the additional exposure period may further include at least one of a seventh exposure period A2 for receiving background light corresponding to the fifth exposure period A0 and subtracting it from the signal generated in the fifth exposure period A0, and an eighth exposure period A3 for receiving background light corresponding to the sixth exposure period A1 and subtracting it from the signal generated in the sixth exposure period A1.

[0117] In addition, in Example A-2, the consecutive exposure periods include a first exposure period A0 for receiving reflected light, a second exposure period A1 that follows the first exposure period A0, a third exposure period A2 for receiving background light corresponding to the first exposure period A0, and a fourth exposure period A3 that follows the third exposure period A2 and receives background light corresponding to the second exposure period A1, and the additional exposure periods include at least one of a fifth exposure period A2 that receives reflected light at a timing earlier than the first exposure period A0 or later than the second exposure period A1 and adds it to the signal generated in the third exposure period A2, and a sixth exposure period A3 that receives reflected light at a timing earlier than the first exposure period A0 or later than the second exposure period A1 and adds it to the signal generated in the fourth exposure period A3. As a result, in this embodiment A-2, the indefinite signal generated by the additional exposure acts to reduce the subtraction result during background light subtraction, so the signal levels of the surrounding pixels do not exceed the coring level, and the calculation of an erroneous distance value that may occur in embodiment A-1 is suppressed. Note that the additional exposure period may further include at least one of a seventh exposure period A0 for receiving background light corresponding to the fifth exposure period A2 and subtracting it from the signal generated in the fifth exposure period A2, and an eighth exposure period A1 for receiving background light corresponding to the sixth exposure period A3 and subtracting it from the signal generated in the sixth exposure period A3.

[0118] In Example B-1, the consecutive exposure periods include a first exposure period A0 for receiving the first reflected light, a second exposure period A1 that follows the first exposure period A0, a third exposure period A2 that follows the second exposure period A1 for receiving the second reflected light, and a fourth exposure period A3 that follows the third exposure period A2, and the additional exposure periods include at least one of a fifth exposure period A0 for receiving the first reflected light at an earlier timing than the first exposure period A0 and adding it to a signal generated in the first exposure period A0, and a sixth exposure period A3 for receiving the second reflected light at a later timing than the fourth exposure period A3 and adding it to a signal generated in the fourth exposure period A3. As a result, addition or subtraction of signals generated in the additional exposure periods is performed under exposure driving method B, which is one of various exposure driving methods, and cases outside the distance measurement range can be reliably detected. In addition, the additional exposure period may further include at least one of a seventh exposure period A2 for receiving background light corresponding to the fifth exposure period A0 and subtracting it from the signal generated in the fifth exposure period A0, and an eighth exposure period A1 for receiving background light corresponding to the sixth exposure period A3 and subtracting it from the signal generated in the sixth exposure period A3.

[0119] In Example C-1, the consecutive exposure periods include a first exposure period A0 for receiving reflected light and background light, a second exposure period A1 that follows the first exposure period A0, a third exposure period A2 that follows the second exposure period A1, and a fourth exposure period A3 that follows the third exposure period A2, and the additional exposure periods include at least one of a fifth exposure period A0 for receiving reflected light at an earlier timing than the first exposure period A0 and adding it to a signal generated in the first exposure period A0, and a sixth exposure period A3 for receiving reflected light at a later timing than the fourth exposure period A3 and adding it to a signal generated in the fourth exposure period A3. As a result, under exposure driving method C, which is one of various exposure driving methods, addition or subtraction of signals generated in the additional exposure periods is performed, and cases outside the distance measurement range can be reliably detected. In addition, the additional exposure period may further include at least one of a seventh exposure period A2 for receiving background light corresponding to the fifth exposure period A0 and subtracting it from the signal generated in the fifth exposure period A0, and an eighth exposure period A1 for receiving background light corresponding to the sixth exposure period A3 and subtracting it from the signal generated in the sixth exposure period A3.

[0120] Furthermore, the driving method of the distance measuring imaging device 10 according to this embodiment is a driving method of the distance measuring imaging device 10 that generates a distance image using a TOF method, and the distance measuring imaging device 10 includes a light source unit 11 that emits irradiation light toward a subject 5, an imaging unit 13 having a plurality of pixels that receives reflected light from the subject 5 and background light, a driving circuit 14 that drives the light source unit 11 to emit the irradiation light and drives the imaging unit 13 to expose for each of a plurality of consecutive exposure periods, thereby generating, for each of the plurality of pixels, a signal that indicates the amount of charge generated by exposure for each of the plurality of exposure periods, and and a signal processing unit 15 that calculates and outputs the distance from the distance measuring imaging device 10 to the subject 5 from the signals generated in each of the multiple exposure periods, wherein the multiple consecutive exposure periods correspond to the distance measurement range of the distance measuring imaging device 10, and the driving method of the distance measuring imaging device 10 includes a step in which the driving circuit 14 exposes the imaging unit 13 in an additional exposure period that is earlier than the multiple consecutive exposure periods and / or later than the multiple consecutive exposure periods, in addition to the multiple consecutive exposure periods, and the signal generated in the additional exposure period is added to or subtracted from the signal generated in any of the multiple consecutive exposure periods.

[0121] As a result, in addition to the light emission exposure drive including a plurality of consecutive exposure periods corresponding to the ranging range, an additional light emission exposure drive is performed to detect outside the ranging range, and the signal obtained in the additional exposure period is added to or subtracted from the signal obtained in the corresponding exposure period of the plurality of consecutive exposure periods. Therefore, by determining whether or not the ranging range is exceeded using the signal after addition or subtraction, it is possible to avoid the calculation of an indefinite distance value due to parasitic sensitivity as in the conventional method, and more reliably than in the conventional method, the generation of a distance image containing false distance information is suppressed.

[0122] While the distance measuring image capturing device and the method for driving the distance measuring image capturing device according to the present disclosure have been described above based on the embodiments and examples, the present disclosure is not limited to these embodiments and examples. As long as they do not deviate from the gist of the present disclosure, various modifications that a person skilled in the art may make to the present embodiments and examples, and other forms constructed by combining some of the components in the embodiments and examples, are also included within the scope of the present disclosure.

[0123] For example, all of the functions of Examples A-1, A-2, B-1, and C-1 in the above embodiments may be implemented in a distance measuring imaging device, and an example selected from the four examples may be executed depending on the pre-settings or the measurement environment, etc.

[0124] The present disclosure can be used as a distance measuring imaging device that generates distance images using a TOF method, and in particular as a distance measuring imaging device that can more reliably suppress the generation of distance images containing false distance information than conventional devices.

[0125] 5 Object 10 Distance measuring imaging device 11 Light source section 12 Lens 13 Imaging section 14 Drive circuit 15 Signal processing section 20 Pixel 30 First object 31 Second object 30a Distance information of first object 31a False distance information of second object PD Photodiode TG1, TG2 Read gates ODG1, ODG2 Discharge gates OD1, OD2 Overflow drains VG1 to VG5 Charge storage section

Claims

1. A distance measuring imaging device that generates a distance image using a TOF (Time of Flight) method, comprising: a light source unit that emits illumination light toward an object; an imaging unit having a plurality of pixels that receive reflected light from the object and background light; a drive circuit that drives the light source unit to emit illumination light and drives the imaging unit to expose for each of a plurality of consecutive exposure periods, thereby generating, for each of the plurality of pixels, a signal indicating the amount of charge generated by exposure for each of the plurality of exposure periods; and a signal processing unit that calculates and outputs, for each of the plurality of pixels, the distance from the distance measuring imaging device from the signal generated for each of the plurality of exposure periods, wherein the plurality of consecutive exposure periods correspond to the distance measurement range of the distance measuring imaging device; and the drive circuit exposes the imaging unit for at least one additional exposure period that is earlier than the plurality of consecutive exposure periods and later than the plurality of consecutive exposure periods in addition to the plurality of consecutive exposure periods, A signal generated in the additional exposure period is added to or subtracted from a signal generated in any one of the plurality of consecutive exposure periods.

2. The distance measuring imaging device according to claim 1, wherein the signal processing unit determines that the distance is outside the distance measurement range if a value expressed by a predetermined formula using the signal after the addition or subtraction is outside a predetermined range.

3. A distance measuring imaging device according to claim 1 or 2, wherein each of the plurality of pixels has a charge accumulation section for accumulating charge generated in at least one of the plurality of consecutive exposure periods, and the addition is performed in the charge accumulation section under the control of the drive circuit.

4. The distance measuring imaging device according to claim 1 or 2, wherein the subtraction is performed by the signal processing unit that reads out signals generated in the plurality of pixels from the plurality of pixels.

5. A distance measuring imaging device according to any one of claims 1 to 4, wherein the number of exposures in the additional exposure period is less than the number of exposures in the plurality of consecutive exposure periods.

6. A distance measuring imaging device according to any one of claims 1 to 5, wherein the consecutive exposure periods include: a first exposure period A0 for receiving reflected light; a second exposure period A1 that follows consecutively from the first exposure period A0; a third exposure period A2 for receiving background light corresponding to the first exposure period A0; and a fourth exposure period A3 that follows consecutively from the third exposure period A2 and receives background light corresponding to the second exposure period A1; and the additional exposure periods include at least one of: a fifth exposure period A0 for receiving reflected light at an earlier timing than the first exposure period A0 and adding it to the signal generated in the first exposure period A0; and a sixth exposure period A1 for receiving reflected light at a later timing than the second exposure period A1 and adding it to the signal generated in the second exposure period A1.

7. A distance measuring imaging device according to any one of claims 1 to 5, wherein the consecutive exposure periods include: a first exposure period A0 for receiving reflected light; a second exposure period A1 that follows consecutively from the first exposure period A0; a third exposure period A2 for receiving background light corresponding to the first exposure period A0; and a fourth exposure period A3 that follows consecutively from the third exposure period A2 and receives background light corresponding to the second exposure period A1; and the additional exposure periods include at least one of: a fifth exposure period A2 that receives reflected light at a timing earlier than the first exposure period A0 or later than the second exposure period A1 and adds it to a signal generated in the third exposure period A2; and a sixth exposure period A3 that receives reflected light at a timing earlier than the first exposure period A0 or later than the second exposure period A1 and adds it to a signal generated in the fourth exposure period A3.

8. A distance measuring imaging device according to any one of claims 1 to 5, wherein the consecutive exposure periods include a first exposure period A0 for receiving first reflected light, a second exposure period A1 that follows consecutively from the first exposure period A0, a third exposure period A2 that follows consecutively from the second exposure period A1 for receiving second reflected light, and a fourth exposure period A3 that follows consecutively from the third exposure period A2, and the additional exposure periods include at least one of a fifth exposure period A0 for receiving the first reflected light at an earlier timing than the first exposure period A0 and adding it to the signal generated in the first exposure period A0, and a sixth exposure period A3 for receiving the second reflected light at a later timing than the fourth exposure period A3 and adding it to the signal generated in the fourth exposure period A3.

9. A distance measuring imaging device according to any one of claims 1 to 5, wherein the consecutive exposure periods include a first exposure period A0 for receiving reflected light and background light, a second exposure period A1 that follows consecutively from the first exposure period A0, a third exposure period A2 that follows consecutively from the second exposure period A1, and a fourth exposure period A3 that follows consecutively from the third exposure period A2, and the additional exposure periods include at least one of a fifth exposure period A0 for receiving reflected light at an earlier timing than the first exposure period A0 and adding it to the signal generated in the first exposure period A0, and a sixth exposure period A3 for receiving reflected light at a later timing than the fourth exposure period A3 and adding it to the signal generated in the fourth exposure period A3.

10. A method for driving a distance measuring imaging device that generates a distance image using a TOF (Time of Flight) system, wherein the distance measuring imaging device comprises: a light source unit that emits illumination light toward an object; an imaging unit having a plurality of pixels that receive reflected light from the object and background light; a drive circuit that drives the light source unit to emit illumination light and drives the imaging unit to expose in each of a plurality of consecutive exposure periods, thereby generating, for each of the plurality of pixels, a signal indicating the amount of charge generated by exposure in each of the plurality of exposure periods; and a signal processing unit that calculates and outputs, for each of the plurality of pixels, a distance from the distance measuring imaging device to the object from the signal generated in each of the plurality of exposure periods, wherein the plurality of consecutive exposure periods correspond to a distance measurement range of the distance measuring imaging device, and the method for driving the distance measuring imaging device comprises: A driving method for a distance measuring imaging device, comprising a step in which the driving circuit exposes the imaging unit to an additional exposure period in addition to the plurality of consecutive exposure periods, at least one of a timing earlier than the plurality of consecutive exposure periods and a timing later than the plurality of consecutive exposure periods, and a signal generated in the additional exposure period is added to or subtracted from a signal generated in any of the plurality of consecutive exposure periods.

Images