Range image capturing device and method for capturing a range image

Abstract

To improve the total performance of a distance image pickup device.SOLUTION: A distance image pickup device 1 comprises: a light source 2; a photodiode 11 which generates an electric charge corresponding to the received light; and a peripheral circuit 4 which controls the operation of a pixel circuit part 10. The peripheral circuit 4 generates a first transfer control pulse SG1 and a second transfer control pulse SG2 on the basis of a magnitude relation between a first electric charge amount Q1 accumulated in a first floating diffusion part FD1 and a second electric charge amount Q2 accumulated in a second floating diffusion part FD2.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a distance image capturing device and a method for capturing a distance image. [Background technology]

[0002] The distance image capture device outputs a distance image. The distance image has distance information to the capture object. The distance image capture device emits a measurement light to the capture object. The measurement light is reflected by the capture object and enters the distance image capture device again. By using the timing of emission of the measurement light and the timing of incidence of the reflected light from the capture object, the time required for the measurement light to reach the capture object from the distance image capture device can be determined. This time is also called the light flight time. By using the light flight time and the speed of light, the distance from the distance image capture device to the capture object can be determined. Such a distance measurement method is called a time-of-flight measurement method. And a distance image measurement device based on such a distance measurement method is also called a TOF sensor. Patent documents 1 to 5 disclose techniques related to TOF sensors.

[0003] TOF sensors include so-called direct type TOF sensors and indirect type TOF sensors. Direct type TOF sensors directly measure the time of flight of light. For example, Patent Documents 6 and 7 disclose technologies related to direct type TOF sensors. Indirect type TOF sensors distribute electric charges generated by reflected light to multiple regions over time. Then, the ratio of electric charges accumulated in each region is used to obtain the time of flight of light. For example, Patent Documents 8 to 10 disclose technologies related to indirect type TOF sensors. Indirect type TOF sensors have several advantages over direct type TOF sensors, such as a smaller pixel size, a simpler circuit, and high distance resolution. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2019-082331 A [Patent Document 2] JP 2016-090436 A [Patent Document 3] Re-tabled publication No. 2016 / 075885 [Patent Document 4] JP 2008-076390 A [Patent Document 5] Japanese Patent Application Publication No. 06-066940 [Patent Document 6] Japanese Patent Application Publication No. 07-191144 [Patent Document 7] Japanese Patent Application Publication No. 08-313631 [Patent Document 8] International Publication No. 2019 / 078366 [Patent Document 9] International Publication No. 2019 / 031510 [Patent Document 10] International Publication No. 2016 / 133053 Summary of the Invention [Problem to be solved by the invention]

[0005] Characteristics that indicate the performance of an indirect TOF sensor include distance resolution and linearity between distance and output signal. In this field, there is a demand for an improvement in overall performance including these indicators. For example, in order to improve the distance resolution, the irradiation time of the measurement light should be shortened. In other words, a short pulse of measurement light should be irradiated. On the other hand, using a shorter pulse is effective for improving the distance resolution, but the nonlinearity of the pixel's source follower amplifier, distortion of the optical pulse waveform, the finite optical carrier response time in the photodiode, distortion of the transfer control pulse for demodulation, and skew of the transfer control pulse are major issues that must be resolved. As a result, linearity may be impaired. In other words, when trying to improve one characteristic, another characteristic may be degraded. Therefore, the inventors have investigated a technology that can improve the overall performance of the indirect TOF sensor.

[0006] The present invention provides a range image capturing device and a range image capturing method that enable overall performance to be improved. [Means for solving the problem]

[0007] A distance image pickup device according to one embodiment of the present invention includes a light source that generates pulsed light, a pixel circuit array including a plurality of pixel circuits that generate charges corresponding to the received light, and a peripheral circuit that provides the pixel circuit array with a transfer control pulse that controls the operation of the pixel circuit array. The pixel circuit includes a photoelectric conversion region that converts light into charges, a first charge readout region that accumulates the charges transferred from the photoelectric conversion region as first charges, a first control electrode that receives a first transfer control pulse that controls the transfer of charges from the photoelectric conversion region to the first charge readout region, a second charge readout region that accumulates the charges transferred from the photoelectric conversion region as second charges, and a second control electrode that receives a second transfer control pulse that controls the transfer of charges from the photoelectric conversion region to the second charge readout region. The peripheral circuit generates a first transfer control pulse and a second transfer control pulse based on the magnitude relationship between the amount of the first charge accumulated in the first charge readout region and the amount of the second charge accumulated in the second charge readout region.

[0008] This device controls the timing of the transfer control pulse based on a calculated value obtained from the difference in the amount of charge related to the time of flight of light. As a result, it is possible to reduce the bias in the amount of charge for each charge readout region, which is expected to improve overall performance.

[0009] The peripheral circuit may have a first calculation unit that outputs a calculation value relating to the difference between a first voltage based on the first charge and a second voltage based on the second charge, and a second calculation unit that uses the calculation value to determine a delay time for determining the timing of outputting the first transfer control pulse and the second transfer control pulse. The peripheral circuit may repeat an accumulation operation for accumulating the first charge and the second charge, an operation of the first calculation unit, and an operation of the second calculation unit. This configuration also makes it possible to reduce the bias in the amount of charge for each charge readout region. As a result, an improvement in overall performance can be expected.

[0010] When the calculated value in the n-th operation of the second calculation unit is the first information, the second calculation unit may determine the n-th delay time used for the n-th accumulation operation to be further delayed by a unit delay time as the n+1-th delay time used for the n+1-th accumulation operation. This operation can suitably suppress bias in the amount of charge.

[0011] The second calculation unit may have a counter that adds 1 to a control variable when the calculation value is the first information in the n-th operation of the second calculation unit, and a delay time determination unit that determines a value obtained by multiplying the control variable by a unit delay time as the n+1-th delay time to be used for the n+1-th accumulation operation. With this configuration, bias in the amount of charge can be suitably suppressed.

[0012] The first calculation unit may output the difference between the first voltage and the second voltage as a differential voltage, and then calculate the integral of the m-th differential voltage by adding the m-th differential voltage to the integral of the first to (m-1)th differential voltages. The second calculation unit may determine the n+1 delay time used for the n+1 accumulation operation as the first delay time when the m-th integral is the second information, and may determine the m+1 delay time as the second delay time when the m-th integral is the first information. This operation makes it possible to obtain a time of flight of light in which an error caused by a unit delay time is suppressed.

[0013] The first calculation unit may have an integrator that calculates the m-th integral value by adding the m-th differential voltage to the integral value of the differential voltages from the 1st to the (m-1)th times. The second calculation unit may have a delay time determination unit that determines the first delay time as the (m+1)th delay time when the m-th integral value is the second information, and determines the second delay time as the (m+1)th delay time when the m-th integral value is the first information. With this configuration, it is possible to obtain a time of flight of light in which an error caused by a unit delay time is suppressed.

[0014] The distance image capturing device may further include an A / D converter that converts the calculated value output by the first calculation unit into a digital signal and outputs the digital signal to the second calculation unit, and that cooperates with the first calculation unit and the second calculation unit to form a delta-sigma modulator, and a decimation filter that receives the digital signal output by the A / D converter and performs low-pass filtering and downsampling on the digital signal each time the accumulation operation, the operation of the first calculation unit, the operation of the A / D converter, and the operation of the second calculation unit are repeated.

[0015] The decimation filter has an oversampling ratio of 2 for low-pass filtering. B―1 Greater than 2 B B may be a smaller integer. B may be the minimum number of bits capable of expressing the oversampling ratio.

[0016] The peripheral circuit may perform a first measurement in which a process including: an accumulation operation of accumulating the first charge and the second charge; and an operation by the second calculation unit of determining, when a calculated value in the n-th operation of the second calculation unit is the first information, the n-th delay time used for the n-th accumulation operation, further delayed by a unit delay time, as the n+1-th delay time to be used for the n+1-th accumulation operation; and a second measurement in which a process including: an accumulation operation; and an operation by the first calculation unit of outputting a difference between the first voltage and the second voltage as a differential voltage, and then calculating a result of adding the m-th differential voltage to an integral value of the differential voltages from the 1st to the (m-1)th times as an integral value of the m-th differential voltage, and an operation by the second calculation unit of determining the first delay time as the (m+1)th delay time when the m-th integral value is the second information, and determining the second delay time as the (m+1)th delay time when the m-th integral value is the first information. According to this operation loop, the approximate time of flight of light can be measured in the first measurement, and the result can be used to measure the precise time of flight of light in the second measurement.

[0017] The first calculation unit may include an integrator that obtains a result of adding the m-th differential voltage to an integral value of the differential voltages from the 1st to the (m-1)th times. The second calculation unit may include a counter that performs an operation of adding 1 to a control variable when the calculation value is the first information in the n-th operation of the second calculation unit, and a delay time determination unit that selectively performs an operation of determining a value obtained by multiplying the control variable and the unit delay time as the (n+1)th delay time to be used for the (n+1)th accumulation operation, or an operation of determining the first delay time as the (m+1)th delay time when the m-th integral value is the second information and determining the second delay time as the (m+1)th delay time when the m-th integral value is the first information. With this configuration of the loop, the approximate time of flight of light can be measured in the first measurement, and the result can be used to measure the precise time of flight of light in the second measurement.

[0018] The first operational unit may include a differential amplifier that outputs a difference between the first voltage and the second voltage as a differential voltage. The differential amplifier may include an operational amplifier having a first input terminal, a second input terminal, a third input terminal, a first output terminal, and a second output terminal, a feedback capacitor that receives a first charge from the first charge readout region and is connected to the first input terminal and the first output terminal, a feedback capacitor that receives a second charge from the second charge readout region and is connected to the second input terminal and the second output terminal, and a common mode feedback control unit that is connected to the third input terminal, the first output terminal, and the second output terminal and provides a signal to the third input terminal so that a difference between an output value from the first output terminal and an output value from the second output terminal is equal to a common voltage.

[0019] A method for capturing a distance image using a distance image capturing device that is another form of the present invention includes a charge accumulation step of accumulating a first charge in a first charge readout region based on a first transfer control pulse applied to a first control electrode, and accumulating a second charge in a second charge readout region based on a second transfer control pulse applied to a second control electrode, and a pulse generation step of generating a first transfer control pulse and a second transfer control pulse based on the magnitude relationship between the amount of the first charge and the amount of the second charge.

[0020] This method controls the timing of the transfer control pulse based on the calculated value of the charge amount difference related to the light flight time, which makes it possible to reduce the bias of the charge amount for each charge readout region, thereby improving the overall performance.

[0021] The pulse generating step may include a calculation step of outputting a calculation value relating to a difference between a first voltage based on the first charge and a second voltage based on the second charge, and a delay time determination step of determining a delay time for determining a timing for outputting the first transfer control pulse and the second transfer control pulse using the calculation value. The charge accumulation step and the pulse generating step may be repeated. According to this method, the bias of the charge amount can be suitably suppressed.

[0022] The delay time determination step may include a step of determining, when the calculated value in the n-th calculation step is the first information, the n-th delay time used for the n-th charge accumulation step, which is further delayed by a unit delay time, as the n+1-th delay time used for the n+1-th charge accumulation step. According to this method, bias in the amount of charge can be suitably suppressed.

[0023] The delay time determination step may include a step of outputting the difference between the first voltage and the second voltage as a differential voltage, and then determining the result of adding the m-th differential voltage to the integral value of the differential voltage from the 1st to the (m-1)th times as the integral value of the m-th differential voltage as the integral value of the m-th differential voltage, and a step of determining the (m+1)th delay time used for the (m+1)th charge accumulation step as the first delay time when the m-th integral value is the second information, and determining the (m+1)th delay time as the second delay time when the m-th integral value is the first information. According to this method, it is possible to obtain a time of flight of light in which an error caused by a unit delay time is suppressed.

[0024] The delay time determination step may include a step of determining, when the calculated value in the n-th calculation step is the first information, the n-th delay time used for the n-th charge accumulation step, which is further delayed by a unit delay time, as the n+1-th delay time used for the n+1-th charge accumulation step, a step of outputting the difference between the first voltage and the second voltage as a differential voltage, and then calculating the result of adding the m-th differential voltage to the integral value of the differential voltage from the 1st to the (m-1)th times as the integral value of the m-th differential voltage, as the integral value of the m-th differential voltage, and a step of determining, when the m-th integral value is the second information, the m+1-th delay time used for the m+1-th charge accumulation step as the first delay time, and determining, when the m-th integral value is the first information, the m+1-th delay time as the second delay time. According to the loop of this method, the approximate time of flight of light can be measured, and the result can be used to measure the precise time of flight of light. Effect of the Invention

[0025] The distance image capturing device and the distance image capturing method of the present invention can improve overall performance. [Brief description of the drawings]

[0026] [Figure 1] FIG. 1 is a diagram showing the configuration of a range image capturing device. [Diagram 2] FIG. 2 is a flow diagram showing the main steps of a method for capturing a range image. [Diagram 3] FIG. 3 is a diagram showing the principle of the distance image pickup device. [Figure 4] 4(a) and 4(b) are diagrams illustrating problems in measuring the time of flight of light. [Diagram 5] FIG. 5 is a diagram for explaining the C measurement. [Figure 6] 6(a) and 6(b) are diagrams conceptually showing the change in unit delay time. [Figure 7] FIG. 7 is a diagram showing the transition of the number of exposure operations and the integral value of the charge difference. [Figure 8]FIG. 8 is an equivalent block diagram of the configuration for performing the C measurement. [Figure 9] Fig. 9(a) is a diagram conceptually explaining the operation of C measurement, and Fig. 9(b) is a diagram conceptually showing quantization error. [Figure 10] FIG. 10 is an equivalent block diagram of a configuration for performing F measurements. [Figure 11] FIG. 11 is a diagram showing the shape of the quantization error due to the noise transfer function. [Figure 12] FIG. 12 is a timing chart showing both the C measurement and the F measurement. [Figure 13] FIG. 13 is a diagram showing a configuration of a pixel circuit unit. [Figure 14] FIG. 14 is a diagram showing a circuit configuration of a pre-amplifier. [Figure 15] FIG. 15 is a timing chart showing the operations of the pixel circuit unit and the demodulator. [Figure 16] FIG. 16 is a diagram showing a circuit configuration of the analog processing unit. [Figure 17] FIG. 17 is a diagram showing the circuit configuration of a DT converter and a transfer control pulse generator. [Figure 18] FIG. 18 is a diagram showing a specific example of a distance image pickup device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0027] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same elements are given the same reference numerals, and duplicated explanations will be omitted.

[0028] The distance image pickup device 1 shown in FIG. 1 obtains a distance image. The distance image is an image including distance information from the distance image pickup device 1 to the object 100. The distance image pickup device 1 emits irradiated light L1 (measurement light) toward the object 100. The irradiated light L1 is reflected on the surface of the object 100. The reflected light L2 is incident on the distance image pickup device 1. The distance image pickup device 1 obtains the time difference between the timing of emitting the irradiated light L1 and the timing of receiving the reflected light L2. This time difference is also called the light flight time. The light flight time is the time required for the irradiated light L1 to reach the object 100 and then to reach the distance image pickup device 1 from the object 100. In other words, the light flight time is determined according to the distance from the distance image pickup device 1 to the object 100. Therefore, by obtaining the light flight time, the distance from the distance image pickup device 1 to the object 100 can be obtained.

[0029] First, the principle of distance measurement will be explained. FIG. 3 is a diagram showing the principle of the distance image pickup device 1. The distance image pickup device 1 indirectly measures the time of flight of light. The distance image pickup device 1 has multiple regions for accumulating electric charges. The distance image pickup device 1 distributes electric charges generated in the photoelectric conversion region by the incidence of light to each of the accumulation regions. This distribution is performed by a gate that controls the transfer of electric charges from the photoelectric conversion region to the accumulation region. In the first period, electric charges are transferred to the first accumulation region. In the first period, electric charges are not transferred to the second accumulation region. Then, in the second period following the first period, the transfer of electric charges to the first accumulation region is stopped and electric charges are transferred to the second accumulation region. Time T between the first period and the second period W By using the ratio of the first charge amount Q1 in the first accumulation region to the second charge amount Q2 in the second accumulation region, the light flight time can be obtained.

[0030] Here, it is assumed that the time relationship between the timing of emitting the irradiation light L1 and the charge transfer period to the first accumulation region does not change. Similarly, it is assumed that the time relationship does not change for the charge transfer period to the second accumulation region. Under this assumption, the distance image pickup device 1 is operated. As shown in FIG. 4(a), when the distance to the object 100 is short, the time from emitting the irradiation light L1 to the incidence of the reflected light L2 is short. Therefore, the first charge amount Q1 in the first accumulation region operated by the first transfer control pulse SG1 is large. On the other hand, the second charge amount Q2 in the second accumulation region operated by the second transfer control pulse SG2 is small. As shown in FIG. 4(b), when the distance to the object 100 is long, the time from emitting the irradiation light L1 to the incidence of the reflected light L2 is long. Therefore, the first charge amount Q1 is small. On the other hand, the second charge amount Q2 is large. That is, as the distance to the target object 100 increases, the first charge amount Q1 decreases and the second charge amount Q2 increases.

[0031] When the accumulation operation of accumulating charges in the first accumulation region and the second accumulation region is performed, charges not caused by the reflected light L2 are also accumulated. In other words, the charges accumulated in the accumulation region include charges caused by the reflected light L2 and charges not caused by the reflected light L2. Such charges not caused by the reflected light L2 become noise components. Noise components may occur not only when accumulating charges but also when reading out charges.

[0032] Then, when the amount of charge caused by the reflected light L2 is small, the proportion of the amount of charge not caused by the reflected light L2 becomes relatively large. As a result, the performance of the distance image pickup device 1 is affected, such as a decrease in the distance resolution. This problem becomes greater as the upper or lower limit of the measurement range of the distance image pickup device 1 is approached. In other words, if the timing of transfer to the first accumulation region and the second accumulation region is always constant, as was adopted in the conventional distance image pickup device, the amount of charge accumulated in one accumulation region becomes very small depending on the distance to the target object 100. As a result, the distance resolution is decreased.

[0033] In order to solve the above problem, the inventors have intensively studied and have come up with the idea that the problem can be solved by changing the temporal relationship between the timing of emitting the irradiation light L1 and the period during which the charges are transferred to the first accumulation region. In other words, the timing of opening the first transfer control gate G1 and the second transfer control gate G2 is variable, not fixed, relative to the timing of emitting the irradiation light L1. Furthermore, the inventors have come up with the idea of ​​controlling the timing of opening the first transfer control gate G1 and the second transfer control gate G2 according to the magnitude relationship between the first charge amount Q1 transferred by the first transfer control gate G1 and the second charge amount Q2 transferred by the second transfer control gate G2.

[0034] Control according to the magnitude relationship of the charge amount makes it possible to reduce the bias between the first charge amount Q1 and the second charge amount Q2, that is, to operate to reduce the difference in the charge amount. As a result, the influence of noise can be suppressed. Furthermore, control according to the magnitude relationship of the charge amount also makes it possible to apply so-called delta-sigma modulation technology. The delta-sigma modulation technology can reduce quantization error. In other words, by eliminating the bias of the accumulated charge and reducing the quantization error by introducing the delta-sigma modulation technology, it is possible to improve the distance resolution over the entire measurement range.

[0035] Furthermore, in control using the magnitude relationship of the charge amounts, a feedback loop is formed that controls the timing of charge transfer depending on the magnitude relationship of the charge amounts. First, the difference in the charge amounts stored in the storage region is converted into a difference in analog voltage. Next, the difference in analog voltage is converted into a binary digital value. Next, the timing of charge transfer is determined using the digital value. When the pixel is operated based on the determined timing, charge is again accumulated in the storage region. A technical feature of this feedback loop is that the timing of charge transfer is determined using a digital value. In other words, the digital value is converted into time. The inventors named this technology "time domain feedback control."

[0036] The control for transferring charges according to the magnitude relationship of the charge amounts can take two forms.

[0037] In the first embodiment, the control goal is to reduce the difference in the charge amount. According to the first embodiment, it is possible to obtain an approximate light flight time including a known deviation. The inventors named the measurement for obtaining an approximate light flight time according to the first embodiment "Coarse measurement". In the following description, it is also simply referred to as "C measurement" (first measurement).

[0038] Fig. 5 is a diagram for explaining the C measurement. Fig. 5 shows an example in which the feedback operation is performed four times. The time difference between the emission period of the irradiated light L1 and the incidence period of the reflected light L2 corresponds to the distance to the object 100 that generated the reflected light L2.

[0039] In the first operation (n=1), the end of the first transfer period is made to coincide with the midpoint of the emission period of the irradiated light L1. The end of the first transfer period coincides with the start of the second transfer period. Therefore, the start of the second transfer period also coincides with the midpoint of the emission period. The reflected light L2 is received during such first and second transfer periods. In the following description, the operation of receiving the reflected light L2 and generating charges is also referred to as an "exposure operation." In the first exposure operation, the incidence period of the reflected light L2 does not overlap with the first transfer period, but overlaps with a part of the second transfer period. As a result of the first exposure operation, the first charge amount Q1 of the first accumulation region is smaller than the second charge amount Q2 of the second accumulation region. When evaluated based on the charge amount difference (= second charge amount Q2-first charge amount Q1), it can be said that the sign is plus (positive).

[0040] Based on this result, the first and second transfer periods in the second (n=2) exposure operation are set. When the charge amount difference is positive, the first and second transfer periods are delayed by a unit delay time Δt D The second exposure operation is delayed by a unit delay time Δt from the first exposure operation. DThe first and second transfer periods are set with a delay of 100 μs. In the second exposure operation, the incidence period of the reflected light L2 does not overlap with the first transfer period, but overlaps with the entire second transfer period. As a result, the sign of the difference in the amount of charge is still positive.

[0041] In the third exposure operation (n=3), there is an additional unit delay time Δt D In other words, the first transfer period and the second transfer period are delayed by 2Δt from the first exposure operation. D In the third exposure operation, the incidence period of reflected light L2 overlaps with part of the first transfer period and also with part of the second transfer period. The overlapping period of the second transfer period is longer than the overlapping period of the first transfer period. Therefore, the sign of the difference in the amount of charge is still positive.

[0042] In the fourth (n=4) exposure operation, there is an additional unit delay time Δt D In other words, the first and second transfer periods are delayed by 3Δt from the first exposure operation. D In the fourth exposure operation, the incidence period of reflected light L2 overlaps with part of the first transfer period and also with part of the second transfer period. The overlapping period of the first transfer period is longer than the overlapping period of the second transfer period. Therefore, the sign of the difference in the amount of charge is negative.

[0043] Here, when the first charge amount Q1 accumulated in the first transfer period is equal to the second charge amount Q2 accumulated in the second transfer period, it can be said that the end of the first transfer period coincides with the center of the incidence period of the reflected light L2. If the irradiation timing (start, center, or end) of the irradiated light L1 is known and the timings of the first and second transfer periods at which the difference in charge amount becomes zero are known, the incidence timing of the reflected light L2 relative to the irradiated light L1 can be known. In other words, the light flight time can be known.

[0044] In the first to fourth operations described above, the unit delay time Δt DThe first transfer period and the second transfer period were shifted for each transfer period. In this way, the difference in the amount of electric charge may become zero, but it may also not become zero. However, it can be seen that there is a timing when the difference in the amount of electric charge becomes zero between the timing when the sign of the difference in the amount of electric charge is positive and the timing when the sign of the difference in the amount of electric charge is negative.

[0045] Therefore, by searching for the delay time at which the sign of the difference is switched, we can obtain an approximate time of flight of light. D This means that the measurement may include an error equivalent to . The negative feedback used in the C measurement converts a digital value based on the charge amount difference into time. The converted time corresponds to the delay time of the transfer control pulse for the first transfer period and the second transfer period. In other words, the control that reduces the charge amount difference can be said to be negative feedback control that reduces the difference between the delay time of the transfer control pulse and the light flight time. This type of control makes it possible to maintain the operating point of the analog readout circuit constant. As a result, the linearity can be effectively improved.

[0046] The result of the C measurement includes the delay time and the difference in the amount of charge obtained by the operation during the delay time. The result of the C measurement is used in the processing of the F measurement described next.

[0047] A second aspect of the control for transferring charges according to the magnitude relationship of the charge amounts will be described. In the second aspect, the difference in the charge amounts is integrated each time the exposure operation is repeated. In this aspect, the delay time to be applied to the first transfer period and the second transfer period is changed for each exposure operation. The change in the delay time to be applied is determined by the result of integrating the difference in the charge amounts. The result of the integration is converted into a binary digital value of 0 or 1. By repeating the exposure operation multiple times, information expressed by 0 or 1 is obtained. The number of times 0 appears and the number of times 1 appears relate to the true light flight time. Therefore, by using the number of times 0 appears and the number of times 1 appears, a more precise light flight time can be obtained from the approximate light flight time obtained by the C measurement. The inventors named the measurement for obtaining a precise light flight time by the second aspect "Fine measurement". In the following description, it is also simply referred to as "F measurement" (second measurement).

[0048] The basic operation of F measurement will be described. As mentioned above, in F measurement, the delay time given to the first transfer period and the second transfer period is changed for each exposure operation. Figure 6 shows the concept of changing the delay time. In F measurement, the first delay time (2Δt D ) and the second delay time (3Δt D ) and the first delay time (2Δt D The second delay time (3Δt D ) is a case where the charge difference becomes negative as a result of the exposure operation (see FIG. 6(b)). The delay time to be used depends on the unit delay time Δt D It is determined based on the integral value of the charge and the multiple of 2Δt in Figure 5. D from 3Δt D When the charge difference is changed from positive to negative, the first delay time is 2Δt D , the second delay time is 3Δt D However, this is merely an example. The first delay time and the second delay time may be combined in such a way that the difference in the amount of charge is positive and negative.

[0049] FIG. 7 shows the transition of the integral value of the charge difference versus the number of exposure operations. FIG. 7 also shows the relationship between the sign of the integral value and the delay time corresponding to that sign. The 0th time in the F measurement corresponds to the operation before the final time in the C measurement. In the C measurement described above, the result of the exposure operation before the final time (the third time) was a differential charge amount ΔQ indicating the difference in the charge amount. The sign of the differential charge amount ΔQ was positive. When the sign of the differential charge amount ΔQ is positive, the second delay time (3Δt D ) is used. As a result of the first exposure operation in F measurement, a differential charge amount ΔQ (with a negative sign) is obtained. This differential charge amount ΔQ is added to the zeroth integral value. As a result, a new integral value is obtained. The sign of the new integral value is still positive. Therefore, in the second exposure operation, the second delay time (3Δt D ) is used. In this way, the accumulation of charges by exposure operations, obtaining the difference in the amount of charges, adding up the obtained differences, and selecting a delay time according to the sign of the result of the addition are repeated in sequence. In the example of FIG. 7, the sign of the integral value is also positive as a result of the third exposure operation. On the other hand, the absolute value of the integral value gradually becomes smaller.

[0050] As a result of the fourth exposure operation, the sign of the integral value changes from positive to negative. When the sign of the integral value is negative, the first delay time (2Δt D ) is used. As a result of the sixth exposure operation, the charge difference has a positive sign (see Figure 6(a)). When the sixth charge difference is added to the integral value of the fifth exposure, the integral value becomes positive again. Figure 7 shows the exposure operations up to the tenth exposure.

[0051] In the integral operation of F measurement, the value added to the integral value increases in proportion to the magnitude of the error between the delay time given to the first transfer period and the second transfer period and the desired light flight time. Therefore, the frequency of occurrence of delay times with small errors increases. On the other hand, the frequency of occurrence of delay times with large errors decreases. In other words, the light flight time can be obtained by weighting and averaging the first delay time and the second delay time by their frequency of occurrence.

[0052] Next, in the F measurement, a precise light flight time is calculated. For example, as shown in FIG. 7, the first delay time (2Δt D ) appears twice. The second delay time (3Δt D ) appears eight times. As a result, the light flight time is given by equation (1).

number

[0053] The above-mentioned operation of F measurement that applies time domain feedback control, which is a negative feedback technique in the time domain, can be said to be a so-called first-order delta-sigma modulation (DSM) operation. Delta-sigma modulation has self-sufficient low distortion and low noise characteristics using oversampling signal processing. As a result, the range image pickup device 1 can achieve high linearity and high range resolution.

[0054] To summarize the discussion so far, the distance image pickup device 1 employs a technical concept called time domain feedback control. Then, as a first mode of time domain feedback control, a C measurement is performed to obtain an approximate light flight time. Then, as a second mode of time domain feedback control, an F measurement is performed to obtain a precise light flight time. With this type of operation, the distance image pickup device 1 can achieve high linearity and high distance resolution, and can improve overall performance.

[0055] Next, we will explain in detail the short-pulse-based indirect TOF image sensor using time-domain feedback control.

[0056] FIG. 1 is a functional block diagram of a distance image pickup device 1. The distance image pickup device 1 includes a light source 2 and a distance image sensor 3. The light source 2 emits irradiation light L1. The irradiation light L1 is reflected by an object 100. The reflected irradiation light L1 is converted into a voltage signal by a photodiode 11 and a demodulator 12. The distance image sensor 3 includes a photodiode 11, a demodulator 12, an analog processing unit 20 (first calculation unit), an A / D converter 30, a digital processing unit 40 (second calculation unit), and a DT converter 50 (second calculation unit, delay time determination unit). These elements form a time domain negative feedback loop. The distance image sensor 3 obtains the time of flight of light. By obtaining the time of flight of light, the distance to the object 100 can be obtained.

[0057] The distance image sensor 3 performs the above-mentioned C measurement and F measurement. These operations can improve the performance of the distance image sensor 3. More specifically, the distance image sensor 3 has high linearity indicating the relationship between the light flight time and distance. In addition, the distance image sensor 3 has high resolution of the light flight time. In other words, the distance image sensor 3 has high resolution of distance. Furthermore, the distance image sensor 3 can maintain high linearity and high resolution over a wide measurement range.

[0058] The range image sensor 3 switches between a time domain negative feedback loop for C measurement and a time domain negative feedback loop for F measurement. This operation switching is performed by selecting two types of operations (buffer operation, integration operation) realized by the analog processing unit 20 and selecting two types of operations (counting operation, addition operation) realized by the digital processing unit 40. In the C measurement, the operation as the buffer 21 realized by the analog processing unit 20 is selected, and the operation as the counter 41 realized by the digital processing unit 40 is selected. In the F measurement, the operation as the integrator 22 realized by the analog processing unit 20 is selected, and the operation as the adder 42 realized by the digital processing unit 40 is selected. When the F measurement is performed, the range image sensor 3 can be regarded as a first-order delta-sigma modulator.

[0059] The light source 2 generates irradiation light L1 to be emitted to the target 100 in order to perform distance measurement by the optical time of flight (TOF: TIME OF FLIGHT) method. The light source 2 has a semiconductor light-emitting element and a drive circuit. The semiconductor light-emitting element generates light in a wavelength region such as the near-infrared region or the visible light region. A light-emitting diode or a laser diode can be used as the semiconductor light-emitting element.

[0060] The range image sensor 3 receives reflected light L2 from the object 100. The range image sensor 3 then outputs a range image using the received reflected light L2. The range image sensor 3 has a photodiode 11, a demodulator 12, an analog processing unit 20, an A / D converter 30, a digital processing unit 40, and a DT converter 50. The range image sensor 3 also has a decimation filter 60.

[0061] The photodiode 11 of the pixel circuit unit 10 receives the reflected light L2. The distance image capturing device 1 has a plurality of pixel circuit units 10 arranged two-dimensionally. The plurality of pixel circuit units 10 arranged two-dimensionally constitute a pixel circuit unit array 10S. The photodiode 11 generates an electric charge according to the reflected light L2. The pixel circuit unit 10 transfers the electric charge generated in the photodiode 11 to a demodulator 12. The output value of the pixel circuit unit 10 is based on a first electric charge amount Q1, a second electric charge amount Q2, and a differential electric charge amount ΔQ. The physical configuration of the pixel circuit unit 10 will be described later.

[0062] The demodulator 12 controls the transfer of charges from the photodiode 11. The demodulator 12 outputs the difference in the amount of charges received from the photodiode 11 as a voltage difference. The output value of the demodulator 12 is the first pre-stage voltage signal V P1 , the second pre-stage voltage signal V P2 and the differential front-stage voltage signal ΔV P The physical configuration of the demodulator 12 will be described later.

[0063] The analog processing unit 20 , the A / D converter 30 , the digital processing unit 40 and the DT converter 50 constitute a peripheral circuit 4 .

[0064] The analog processing unit 20 processes the voltage difference. The processes performed by the analog processing unit 20 include a process of integer-multiplying the voltage difference and a process of integrating the voltage difference. The analog processing unit 20 includes a buffer 21 that performs a process of integer-multiplying the voltage difference. Furthermore, the analog processing unit 20 includes an integrator 22 that performs a process of integrating the voltage difference. The output value of the analog processing unit 20 is a first subsequent-stage voltage signal V O1 (first voltage), second post-stage voltage signal V O2 (Second voltage) and differential post-stage voltage signal ΔV O (Differential voltage) The specific circuit configuration of the analog processing unit 20 will be described later.

[0065] The A / D converter 30 quantizes the voltage output by the analog processing unit 20. Specifically, the A / D converter 30 converts the voltage output by the analog processing unit 20 into a binary digital value of 0 or 1. The output value of the A / D converter 30 is a digital signal D FB Includes.

[0066] The digital processing unit 40 converts the digital signal D output by the A / D converter 30 into FB The digital processing unit 40 processes the input digital signal D FB The process of counting 0s or 1s in the input digital signal D FB The digital processing unit 40 includes a counter 41 that performs a process of counting 0 or 1. The output terminal of the counter 41 is connected to the DT converter 50 and the adder 42. The digital processing unit 40 includes an adder 42 that performs an adding process. The A / D converter 30 and the counter 41 are connected to the input terminal of the adder 42. That is, the adder 42 can receive the output value of the A / D converter 30 and the output value of the counter 41. The output terminal of the adder 42 is connected to the DT converter 50. The output value of the digital processing unit 40 is a digital signal D M Includes.

[0067] In the F measurement, the adder 42 and the DT converter 50 determine the delay time for the next accumulation operation according to the sign of the integration result. The sign of the integration result is the digital signal D M The adder 42 and the DT converter 50 set a delay time of 3Δt when the sign of the integration result is zero or plus (first information). D Determine 3Δt D is the delay time when the charge difference becomes negative. The adder 42 and the DT converter 50 set the delay time to 2Δt when the sign of the integration result is negative (second information). D Determine 2Δt D is the delay time when the charge difference becomes positive.

[0068] The decimation filter 60 is connected to the input terminal of the adder 42 of the digital processing unit 40. The decimation filter 60, together with the adder 42, converts the digital signal D FB The decimation filter 60 receives the digital signal D FB The decimation filter 60 performs digital filtering on the input signal. The digital filtering includes low-pass filtering and downsampling. For example, the decimation filter 60 performs low-pass filtering with an oversampling ratio of 2 B―1 Greater than 2 B B may be a smaller integer. B may be the minimum number of bits capable of expressing the oversampling ratio.

[0069] According to the functional block diagram of FIG. 1 and the flow diagram of FIG. 2, the method for capturing a distance image using the distance image capturing device 1 includes charge accumulation steps S10C, S10F and pulse generation steps S20C, S20F. In the method for capturing a distance image, the charge accumulation step S10C and the pulse generation step S20C are repeatedly executed. The charge accumulation step S10C and the pulse generation step S20C constitute a C measurement. Thereafter, in the method for capturing a distance image, the charge accumulation step S10F and the pulse generation step S20F are repeatedly executed. The charge accumulation step S10F and the pulse generation step S20F constitute an F measurement.

[0070] The charge accumulation step S10C is performed by the pixel circuit section 10. As the charge accumulation step S10C, the pixel circuit section 10 accumulates a first charge in the first floating diffusion section FD1 based on a first transfer control pulse SG1 provided to the first transfer control gate G1. Furthermore, as the charge accumulation step S10C, the pixel circuit section 10 accumulates a second charge in the second floating diffusion section FD2 based on a second transfer control pulse SG2 provided to the second transfer control gate G2. The pulse generation step S20C is performed by the peripheral circuit 4. As the pulse generation step S20C, the peripheral circuit 4 generates a first transfer control pulse SG1 and a second transfer control pulse SG2 based on the magnitude relationship between the first charge amount Q1 and the second charge amount Q2.

[0071] The pulse generating step S20C includes a calculation step S21C, a judgment step S30, a delay time determining step S22C, and a pulse output step S24C. The calculation step S21C is performed by the demodulator 12 and the analog processing unit 20. The A / D converter 30 may be included as an element that performs the calculation step S21C. The calculation step S21C outputs a calculation value related to the difference between a first voltage based on the first charge and a second voltage based on the second charge. The calculation value may be the differential voltage itself, or may be a digital value obtained by binarizing the differential voltage.

[0072] After the calculation step S21C, it is determined whether or not the TOF has been obtained (step S30). If it is determined that the TOF has been obtained, the process proceeds to a loop of F measurement. If it is not determined that the TOF has been obtained, the process proceeds to a delay time determination step S22C.

[0073] The delay time determination step S22C is executed by the digital processing unit 40. The delay time determination step S22C uses a calculated value to determine a delay time for determining the timing of outputting the first transfer control pulse SG1 and the second transfer control pulse SG2. The delay time determination step S22C for C measurement is executed by the counter 41 of the digital processing unit 40. The counter 41 can realize an operation in which, when the calculated value in the nth calculation step S21C is the first information (judging step S30), the nth delay time used for the nth charge accumulation step S10C is further delayed by a unit delay time to be determined as the n+1th delay time to be used for the n+1th charge accumulation step.

[0074] The pulse output step S23C is executed by the DT converter 50. The pulse output step S23C outputs a first transfer control pulse SG1 and a second transfer control pulse SG2 based on the determined delay time.

[0075] The steps for F measurement include a charge accumulation step S10F and a pulse generation step S20F. The pulse generation step S20F includes a calculation step S21F, a delay time determination step S22F, and a pulse output step S23F. The calculation step S21F is performed by the integrator 22 of the analog processing unit 20. The integrator 22 can realize an operation of outputting the difference between the first voltage and the second voltage as a differential voltage, and then adding the m-th differential voltage to the integral value of the 1st to (m-1)th differential voltages to obtain the integral value of the m-th differential voltage. The delay time determination step S22F is performed by the adder 42 of the digital processing unit 40. According to the adder 42, when the mth integral value is the second information, the m+1th delay time used for the m+1th charge accumulation process is determined as the first delay time, and when the mth integral value is the first information, the m+1th delay time is determined as the second delay time.

[0076] As described above, in the time domain feedback control, C measurement and F measurement can be performed. That is, the distance image pickup device 1 shown in FIG. 1 can realize a configuration for performing C measurement and a configuration for performing F measurement. The configuration for performing C measurement and the configuration for performing F measurement can be switched between each other. That is, an approximate light flight time is obtained by the configuration for performing C measurement. After the approximate light flight time is obtained, the configuration for performing C measurement is switched to the configuration for performing F measurement. Then, a more precise light flight time is obtained by the F measurement. The switching between the configuration for performing C measurement and the configuration for performing F measurement can be realized by selecting the buffer 21 or the integrator 22 in the analog processing unit 20 and the counter 41 or the adder 42 in the digital processing unit 40.

[0077] When performing the C measurement, the analog processing unit 20 functions as a buffer 21. Furthermore, the digital processing unit 40 functions as a counter 41. Figure 8 is an equivalent block diagram of a configuration for performing the C measurement.

[0078] In the configuration for performing the C measurement, the range image sensor 3 has a summing point F8a, a first transfer element F8b, a second transfer element F8c, a third transfer element F8d, a fourth transfer element F8e, and a fifth transfer element F8f. The summing point F8a and the first transfer element F8b correspond to the demodulator 12. The second transfer element F8c corresponds to the buffer 21. The third transfer element F8d corresponds to the A / D converter 30. The fourth transfer element F8e corresponds to the counter 41. The fifth transfer element F8f corresponds to the DT converter 50.

[0079] The measurement of C will be described with reference to the equivalent block diagram of Fig. 8. In the equivalent block diagram, the output value of the second transfer element F8c (buffer 21), which is the rear voltage signal V O (Calculated value) is shown by equation (2).

number

[0080] The third transfer element F8d (A / D converter 30) receives the downstream voltage signal V O The quantization process follows equation (3). The third transfer element F8d (A / D converter 30) converts the result of the quantization process into a digital signal D FB Generate.

number

[0081] The fourth transfer element F8e (counter 41) receives the digital signal D FB The fourth transfer element F8e receives the digital signal D FB When the internal variable (control variable) is 1, the counter 41 performs a count-up operation of adding 1 to the internal variable (control variable) for the (n-1)th time. In other words, the counter 41 counts up the digital signal D FB For example, when the charge difference value in the C measurement is the first information (0 or positive), the A / D converter 30 outputs a digital value of "1". On the other hand, when the charge difference value in the C measurement is the second information (negative), the A / D converter 30 outputs a digital value of "0". The counter 41 may perform a counting operation based on the digital value. The digital signal D output by the counter 41 after performing the operation n times is M (n) is shown in equation (4).

number

[0082] The fifth transfer element F8f (DT converter 50) receives the digital signal D M The fifth transfer element F8f (DT converter 50) receives the delay time signal T M That is, the delay time signal T M is the unit delay time Δt D and digital signal D M It is obtained by multiplying with

number

[0083] FIG. 9(a) conceptually explains the operation of the C measurement. FB and delay time signal T M First, in the first feedback operation, the delay time signal T M is set to zero. As a result of the n-th feedback operation, the digital signal D FB When is 1, in the n+1th feedback operation, the delay time signal T M has a unit delay time Δt D Thus, in the repeated feedback operation, the digital signal D FB As long as is 1, the delay time signal T M is the unit delay time Δt D In other words, the delay time approaches the optical time of flight (TOF).

[0084] As a result of multiple feedback operations, the delay time signal T M becomes greater than the optical time of flight (TOF). At this time, the A / D converter 30 outputs a digital signal D FB = 0 is output. Digital signal D FB When is zero, the delay time signal T M Unit delay time Δt DIn other words, the addition of the delay time signal T M The delay time signal T M The increase in n C If we assume that the feedback operation occurs during the th feedback operation, the output value of counter 41, which is the digital signal D M is shown by equation (6).

number

[0085] n C is the delay time signal T M The maximum value of the final step (n C The delay time signal T M (n C ) is shown by equation (7).

number

[0086] In the C measurement, the DT converter 50 performs an incremental operation. Therefore, the optical time of flight (TOF) and the delay time signal T M (n C ) is the error. As shown in Figure 9(b), the error is -Δt D / 2 and +Δt D The measurement range of the time of flight (TOF) of light (T M,Max ) is shown by equation (8).

number

[0087] The final output result of the C measurement is the output Y coarse Output Y coarse is the final code stored in counter 41. The final code is given by equation (9).

number

[0088] In addition, the time required for one feedback operation is defined as time (T CM0 Then, the time required to perform the C measurement is T CM is shown by equation (10).

number

[0089] When performing F measurement, the analog processing unit 20 functions as an integrator 22. Furthermore, the digital processing unit 40 functions as an adder 42. Figure 10 is an equivalent block diagram of the configuration for performing F measurement. Figure 10 is an equivalent block diagram of F measurement employing first order delta-sigma modulation (DSM) with time domain negative feedback.

[0090] The range image sensor 3 has a first summing point F10a, a second summing point F10b, a sixth transfer element F10c, a seventh transfer element F10d, a third summing point F10e, an eighth transfer element F10f, a ninth transfer element F10g, and a tenth transfer element F10h. The sixth transfer element F10c and the second summing point F10b correspond to the demodulator 12. The seventh transfer element F10d corresponds to the integrator 22. The summing point F8a corresponds to the A / D converter 30. The eighth transfer element F10f corresponds to a differential circuit arranged in a feedback system of the delta-sigma modulator F10. The ninth transfer element F10g and the tenth transfer element F10h correspond to the DT converter 50.

[0091] The measurement of F will be described with reference to the equivalent block diagram of Fig. 10. From the equivalent block diagram shown in Fig. 10, the output value D FB is shown by equation (11).

number

number

[0092] By substituting equation (12) into equation (11), the overall system response in the z-domain to the delta-sigma modulation portion of FIG. 10 is given by equation (13).

number

number

[0093] From equation (14), the transfer function of the signal and the quantization noise is obtained. The transfer function of the signal, i.e., the digital signal D FB and the differential optical time of flight ΔTOF are expressed by the unit delay time Δt D The noise transfer function (NTF), i.e., the reciprocal of the digital signal D FB and the quantization noise E q The relationship is 1-z -1 It is.

[0094] To estimate the quantization noise of the delta-sigma converter, equation (15) is used and the square of the noise transfer function is calculated in the frequency domain to obtain equation (16).

number

number

[0095] FIG. 11 shows the shape of the power spectral density of the quantization error due to the noise transfer function given by equation (16). This shape of the quantization error is expressed as a function of the cutoff frequency f LPF This effectively reduces the effective number of bits (ENOB) in the F measurement by the ratio of the square root of the residual noise power (hatched area) to the total noise power. SO / 2f LPF The oversampling ratio N given by F When using, the area ratio of the residual noise power to the total noise power is approximately 1 / N F 3 In this case, if an ideal low-pass filter is used, the increase in the effective number of bits, ΔENOB i is given by equation (17).

number

[0096] This technology uses first-order delta-sigma conversion, F For ENOB = 4 we obtain ΔENOB = 3. However, this is the case for an ideal low-pass filter with a very steep cutoff. In a practical implementation, a very simple counter that counts the number of 1s in the bitstream is used as the low-pass filter. The counter does not have a very steep cutoff. Its frequency response resembles a moving average filter, and the residual quantization noise power after the low-pass filter is 1 / N of the total quantization noise power. F 2 It becomes.

[0097] In this case, the increase in the effective number of bits ΔENOB1 is given by equation (18).

number

[0098] N F = 64 corresponds exactly to the TD conversion in 6-bit F measurement. This may deteriorate the depth resolution at the connection boundary of the conversion code of C measurement and F measurement. This is because a discontinuous step appears at the boundary of the maximum ΔTOF in the linearity characteristic. At this discontinuity, a time variable error occurs. At this singular point, the distance resolution (depth noise) deteriorates. This is because N F This can be avoided by setting N to a value greater than 64, the power of 2. For example, F This can be avoided by setting =66.

[0099] Fig. 12 is a timing chart showing the C measurement and the F measurement together. That is, Fig. 12 shows conceptual driving waveforms of a technique for measuring the time of flight (TOF) of light by the C measurement and the F measurement. The range image pickup device 1 first performs the C measurement. In the C measurement, the DT converter 50 of the range image pickup device 1 calculates Δt D ·n C The distance image pickup device 1 then performs F measurement. In the F measurement, the distance image pickup device 1 operates as a delta-sigma modulator. As a result, the DT converter 50 of the distance image pickup device 1 outputs a signal that is a bit stream waveform. The bit stream waveform is expressed as Δt D· n C and Δt D ·(n C The output value of the delta-sigma modulator F10 is output to a low-pass filter. Then, as a result of downsampling of the data, the output value of the F measurement is Y fineFIG. 12 shows the filter output when a digital counter (first-order integrator) is used in the decimation filter.

[0100] The output value of the F measurement is Y fine is also shown by equation (19). Y fine takes on a value between 0 and 1.

number

[0101] According to the technique of measuring the time of flight of light, which includes the C measurement and the F measurement, an output value Y indicating the time of flight of light is obtained. TOF is shown by equation (20) or equation (21).

number

number

[0102] N C and N F , 2M C and 2M F Then, the light flight time measurement technique, including C measurement and F measurement, can calculate the light flight time (TOF) as (m C +m F ) bit digital code.

[0103] The specific circuit configuration of the distance image pickup device 1 will be described below. Fig. 13 and Fig. 14 show the circuit configuration of the pixel circuit section 10, pixel 121, and pre-amplifier 122. The pixel circuit section 10 has a photodiode 11 (photoelectric conversion section) and a demodulator 12. The photodiode 11 receives reflected light L2 from the object 100. The pixel 121 outputs a first charge amount Q1 and a second charge amount Q2 to the pre-amplifier 122.

[0104] The photodiode 11 outputs the generated charge to the demodulator 12. The photodiode 11 is connected to a reference potential Vref.

[0105] The demodulator 12 includes a first floating diffusion region FD1 (first charge read region), a first transfer control gate G1 (first control electrode), a second floating diffusion region FD2 (second charge read region), a second transfer control gate G2 (second control electrode), and a voltage source V D and the third transfer control gate G D The demodulator 12 receives the charge from the photodiode 11. The demodulator 12 outputs a first pre-stage voltage signal V P1 and the second pre-stage voltage signal V P2 Output.

[0106] The output of the photodiode 11 is input to a first transfer control gate G1, a second transfer control gate G2, and a third transfer control gate G D and are connected to the first and second transfer control gates G1 and G2. The first and second transfer control gates G1 and G2 are lateral electric field controlled charge modulator gates. By employing the lateral electric field controlled charge modulator gates, it is possible to extremely quickly switch between permission and prohibition of the transfer of the first charge amount Q1 from the photodiode 11 to the first floating diffusion FD1. For example, the switching time is on the order of sub-nanoseconds. As a result, it is possible to improve the resolution of distance measurement.

[0107] The first transfer control gate G1 is connected to the first floating diffusion FD1. Therefore, the first transfer control gate G1 is disposed between the photodiode 11 and the first floating diffusion FD1. The first transfer control gate G1 permits the transfer of charges from the photodiode 11 to the first floating diffusion FD1. The first transfer control gate G1 also prohibits the transfer of charges from the photodiode 11 to the first floating diffusion FD1. The permission and prohibition of the transfer are switched according to the voltage of a control signal given to the first transfer control gate G1. When the voltage of the control signal given to the first transfer control gate G1 is so-called "HIGH", the first transfer control gate G1 permits the transfer of charges from the photodiode 11 to the first floating diffusion FD1. On the other hand, when the voltage of the control signal given to the first transfer control gate G1 is so-called "LOW", the first transfer control gate G1 prohibits the transfer of charges from the photodiode 11 to the first floating diffusion FD1. The first floating diffusion region FD1 is connected to a first input terminal of the pre-stage amplifier 122. The first floating diffusion region FD1 accumulates the charge transferred from the photodiode 11 as a first charge.

[0108] The connection configuration of the second transfer control gate G2 and the second floating diffusion portion FD2 is similar to that of the first transfer control gate G1 and the first floating diffusion portion FD1, and therefore a detailed description thereof will be omitted.

[0109] Third transfer control gate G D is the gate of a MOS transistor. The drain of this MOS transistor is connected to a voltage source V D Therefore, the third transfer control gate G D is the photodiode 11 and the voltage source V D The third transfer control gate G D When a control signal corresponding to HIGH is given, the third transfer control gate G D When a control signal corresponding to LOW is given, it inhibits the transfer of charge from the photodiode 11 to the drain.

[0110] The preamplifier 122 is a fully differential charge-sensitive amplifier (CSA). The preamplifier 122 receives a first charge amount Q1 and a second charge amount Q2 from the pixel 121. The preamplifier 122 outputs a first preamplifier voltage signal V P1 and the second pre-stage voltage signal V P2 The pre-amplifier 122 includes an operational amplifier 122S and a feedback capacitor C S1 , C S2 and reset switch P R11 , P R12 The pre-amplifier 122 further includes a first common-mode feedback controller 122A.

[0111] The operational amplifier 122S has a first input terminal, a second input terminal, and a third input terminal, and further has a first output terminal, and a second output terminal.

[0112] The first input terminal is connected to the first floating diffusion region FD1. The first input terminal receives a first charge amount Q1 accumulated in the first floating diffusion region FD1. The first output terminal is connected to the input terminal of the analog processing unit 20. The first output terminal outputs a first pre-stage voltage signal V P1 The first output terminal is connected to the first common mode feedback controller 122A. The first output terminal outputs the first pre-stage voltage signal V P1 Output.

[0113] The connection configuration of the second input terminal and the second output terminal is similar to the connection configuration of the first input terminal and the first output terminal, and therefore a detailed description thereof will be omitted.

[0114] The third input is connected to the first common mode feedback controller 122A. The third input receives a control signal from the first common mode feedback controller 122A. The first common mode feedback controller 122A controls the average level of the fully differential output to be equal to the common voltage V COM The operational amplifier 122S is controlled so that the input voltage Vout becomes equal to

[0115] Feedback capacitor CS1 is connected between the first input terminal and the first output terminal. R11 is connected between the first input terminal and the first output terminal. R11 is the feedback capacitor C S1 is electrically connected in parallel to

[0116] The first pre-stage voltage signal V of the pre-stage amplifier 122 P1 is expressed by the formula (22). Also, the second pre-stage voltage signal V P2 is shown by equation (23).

number

number

[0117] Furthermore, the first charge Q1 is expressed by equation (24), and the second charge Q2 is expressed by equation (25).

number

number

[0118] By using a fully differential amplifier with common-mode feedback, the differential front-stage voltage signal ΔV P According to the equation (26), the differential pre-stage voltage signal ΔV P is the background light component (Q B ) is cancelled. In addition, the differential pre-stage voltage signal ΔV P is the signal charge Q S1 and the signal charge Q S2 is proportional to the difference between

number

[0119] Charge-to-voltage conversion gain G C is shown by equation (27).

number

[0120] Note that the number of photons contained in one short pulse of light is small, so the signal is amplified by periodically receiving many cycles of reflected light.

[0121] 15 is a timing chart showing the operation of the pixel 10 circuit section and the demodulator 12. In FIG. P The relative delay time signal t D The relative delay signal T D is 0 and the signal charge amount Q S1 and the signal charge Q S2 If they are equal, the differential front-stage voltage signal ΔV P is 0. The relative delay time signal t D If is greater than 0, then (t D >0), differential front-stage voltage signal ΔV P The absolute value of the relative delay time signal T D If it is smaller than (t D <0), differential front-stage voltage signal ΔV P The absolute value of increases gradually. In this case, the sign is negative. Therefore, it is possible to measure the relative optical pulse delay by the optical flight time, and a demodulator 12 including a charge-sensitive preamplifier 122 can be used to measure the optical pulse delay.

[0122] 16 shows the circuit configuration of the analog processing unit 20. The analog processing unit 20 receives a first pre-stage voltage signal V P1 and the second pre-stage voltage signal V P2 The analog processing unit 20 receives the first post-stage voltage signal V O1 and the second rear voltage signal V O2The analog processing unit 20 includes a pre-processing section 23 and a post-amplifying section 24.

[0123] The pre-processing unit 23 is connected to the input capacitor C I1 , C I2 and input switch P S1 , P S2 and drain switch P R1D1 , P R1D2 and an input capacitor C I1 , C I2 is connected to the output terminal of the preamplifier 122. I1 , C I2 is the input switch P S1 , P S2 and drain switch P R1D1 , P R1D2 The input switch P S1 , P S2 is the input capacitor C I1 , C I2 and the input terminal of the operational amplifier 20S of the rear amplifier 24. R1D1 , P R1D2 is the input capacitor C I1 , C I2 and the reference potential V ref is connected to

[0124] The rear amplifier 24 includes an operational amplifier 20S and a feedback capacitor C F1 and the feedback capacitor C F2 and reset switch P R21 , P R22 and a second common mode feedback controller 20A (common mode feedback control section). The circuit configuration of the post-amplifier 24 is the same as the circuit configuration of the pre-amplifier 122. Therefore, detailed description of the connections of the elements constituting the post-amplifier 24 will be omitted.

[0125] As described above, the analog processing unit 20 functions as a buffer 21 during C measurement. Also, the analog processing unit 20 functions as an integrator 22 during F measurement. The analog processing unit 20 is connected to the input switch PS1 , P S2 , drain switch P R1D1 , P R1D2 , Reset switch P R21 , P R22 Each operation is realized by controlling the above.

[0126] The operation of the analog processing unit 20 to function as the buffer 21 will be described. According to this operation, the analog processing unit 20 can be regarded as a fixed gain amplifier. When functioning as the buffer 21, the analog processing unit 20 can take two operation modes.

[0127] In the first operating mode, the drain switch P R1D1 , P R1D2 As a result, the input capacitor C I1 , C I2 The differential front-stage voltage signal ΔV P During this time, the reset switch P R21 , P R22 As a result, the feedback capacitor C F1 and feedback capacitor C F2 will be reset.

[0128] In the second operating mode, the drain switch P R1D1 , P R1D2 In addition, input switch P S1 , P S2 As a result, the input capacitor C I1 , C I2 The charge sampled at is the capacitance ratio (C I / C F ) so that the input signal is amplified by the feedback capacitor C F1 , C F2 Also, the reset switch P R21 , P R22 In addition, input switch P S1 , P S2 As a result, the input capacitor C I1 , C I2 The charge sampled at is the capacitance ratio (C I / C F ) so that the input signal is amplified by the feedback capacitor C F1 , C F2 will be forwarded to.

[0129] Gain A of buffer 21 I is expressed by the formula (28). The gain of the buffer 21 is the gain of the differential output with respect to the differential input.

number

[0130] The operation of the analog processing unit 20 will now be described so as to function as an integrator 22. According to this operation, the analog processing unit 20 can be considered as a fully differential switched-capacitor integrator. In this operating mode, the reset switch P R21 , P R22 is the feedback capacitor C F1 and feedback capacitor C F2 It is turned ON to reset the charge.

[0131] Input capacitor C I1 , C I2 and drain switch P R1D1 , P R1D2 Input signal sampling by and input switch P S1 , P S2 The feedback capacitor C F1 and feedback capacitor C F2The charge transfer to is repeated so as to perform switched capacitor integration. That is, each time the loop is repeated, the integrator 22 adds the m-th differential front-stage voltage signal ΔV P Add N F The final output after the switched capacitor integration iterations, i.e., the differential post-stage voltage signal ΔV O (N F ) (calculated value) is given by equation (29).

number

[0132] 17 shows the circuit configuration of the DT converter 50 and the transfer control pulse generator 70. The DT converter 50 receives the digital signal D M The DT converter 50 receives the signal T S The DT converter 50 receives the signal T S The operation starts as a trigger.

[0133] The DT converter 50 includes a first delay line 51a, a second delay line 51b, a third delay line 51c, a first digital comparator 52a, a second digital comparator 52b, and a third digital comparator 52c. The first delay line 51a, the second delay line 51b, and the third delay line 51c are digitally controllable delay circuits.

[0134] The first delay line 51a is connected to an optical trigger controller 84a (see FIG. 18). The first delay line 51a receives a signal T SThe first delay line 51a is connected to an input terminal of the second delay line 51b. The second delay line 51b is connected to an input terminal of the third delay line 51c. In other words, the first delay line 51a, the second delay line 51b, and the third delay line 51c are electrically connected in series. Each of the first delay line 51a, the second delay line 51b, and the third delay line 51c has a 5-bit counter.

[0135] The first delay line 51a outputs a first binary counting code to the first digital comparator 52a. The first digital comparator 52a outputs a first binary counting code and a digital signal D M The first digital comparator 52a outputs the result of the comparison as a signal DCO0 to the transfer control pulse generator 70. This signal DCO0 is for the first transfer control pulse SG1 provided to the first transfer control gate G1.

[0136] The connection configuration and operation of the second delay line 51b and the second digital comparator 52b are similar to those of the first delay line 51a and the first digital comparator 52a, and therefore a detailed description thereof will be omitted.

[0137] The connection configuration and operation of the third delay line 51c and the third digital comparator 52c are similar to those of the first delay line 51a and the first digital comparator 52a, and therefore a detailed description thereof will be omitted.

[0138] The transfer control pulse generator 70 receives the signals DCO0, DCO1, and DCO2 from the DT converter 50. The transfer control pulse generator 70 outputs a first transfer control pulse SG1, a second transfer control pulse SG2, and a third transfer control pulse SG D The transfer control pulse generator 70 has a first logic element 71a, a second logic element 71b, a third logic element 71c, a first element 72a, a second element 72b, and a third element 72c.

[0139] The first logic element 71a is an AND circuit. The first digital comparator 52a is connected to a non-inverting input terminal of the first logic element 71a. The second digital comparator 52b is connected to an inverting input terminal of the first logic element 71a. The output terminal of the first logic element 71a is connected to the first element 72a. The first element 72a outputs a first transfer control pulse SG1 to the demodulator 12.

[0140] The second logic element 71b is also an AND circuit. The second digital comparator 52b is connected to a non-inverting input terminal of the second logic element 71b. The third digital comparator 52c is connected to an inverting input terminal of the second logic element 71b. The output terminal of the second logic element 71b is connected to the second element 72b. The second element 72b outputs a second transfer control pulse SG2 to the demodulator 12.

[0141] The third logic element 71c is an OR circuit. The third digital comparator 52c is connected to a non-inverting input terminal of the third logic element 71c. The first digital comparator 52a is connected to an inverting input terminal of the third logic element 71c. The output terminal of the third logic element 71c is connected to the third element 72c. The third element 72c outputs a third transfer control pulse SG D Output.

[0142] 18, the distance image pickup device 1 has a light source 2, a chip component 83, and an FPGA 84. The chip component 83 and the FPGA 84 are arranged on a printed circuit board 81. A comparator 85 is also arranged on the printed circuit board 81. The printed circuit board 81 has a JTAG interface that outputs a signal output by the chip component 83 to the outside. The printed circuit board 81 outputs a signal output by the chip component 83 to the computer 200 via the JTAG interface.

[0143] The chip component 83 includes a digital delay line 51, a digital comparator 52, a transfer control pulse generator 70, a photodiode 11, a demodulator 12, a preamplifier 122, an analog processing unit 20, and a multiplexer 83a.

[0144] The chip component 83 receives various control signals from the FPGA 84. The chip component 83 receives the reflected light L2 from the target 100. The analog processing unit 20 of the chip component 83 is connected to the comparator 85 via a multiplexed channel. The chip component 83 outputs a first post-stage voltage signal V O1 and the second rear voltage signal V O2 Output.

[0145] The comparator 85 outputs the first rear stage voltage signal V O1 and the second rear voltage signal V O2 The comparator 85 outputs the bitstream signal to the FPGA 84.

[0146] The FPGA 84 includes an optical trigger controller 84a, a digital processing unit 40, and a readout controller 84b. The FPGA 84 receives a digital signal D, which is a bit stream signal, from a comparator 85. FB The FPGA 84 outputs a signal to the chip component 83. Specifically, the optical trigger controller 84a outputs a signal TL to the digital delay controller 82. The optical trigger controller 84a outputs a signal TS to the digital delay line 51 of the chip component 83. The digital processing unit 40 has a counter 41, an adder 42, and a decimation filter 60. The digital processing unit 40 outputs a digital signal D M The readout controller 84b outputs a clock signal PC[n] to the preamplifier 122, the analog processing unit 20 and the multiplexer 83a.

[0147] Below, the problems with the conventional distance image measurement device will be described in detail, followed by a description of the operation and effect of the distance image pickup device 1 of the embodiment.

[0148] There are two types of distance image measuring devices: direct type and indirect type. Direct type devices directly measure the time from emitting irradiated light to receiving reflected light. On the other hand, indirect type devices distribute the electric charge generated by reflected light to two or more areas (taps) over time. This distribution is also called demodulation. The light flight time is obtained based on the ratio of the amount of electric charge distributed to each area.

[0149] The distance image capture device 1 is required to have high accuracy, precision, linearity, and resistance to ambient light. Usually, a direct type device employs a SPAD (SINGLE PHOTON AVALANCHE DIODE) as an element for measuring an optical signal. A direct type device using a SPAD and full digital processing is suitable for measuring the time of flight of light with high accuracy. In the direct type device, the time of flight of light is digitized by a time-to-digital (TD) converter. A direct type device equipped with a TD converter has high resolution and linearity. However, the circuit scale becomes large, and the resolution of the TD converter limits the distance resolution. Therefore, it is difficult for the direct type device to measure short distances. And when an extremely high resolution and resistance to ambient light are required for the direct type device, the hardware configuration of the direct type device becomes complicated.

[0150] On the other hand, indirect devices employ pixels with two or more charge storage regions. Such pixels are also called lock-in pixels. The indirect device synchronizes (locks in) the photocharge signal with a light emission trigger pulse signal and demodulates it. The indirect device then converts the demodulated photocharge signal into a digital signal by an A / D converter. The indirect device then obtains the ratio of the charge amount in the first charge storage region to the charge amount in the second charge storage region. The indirect device then calculates the distance based on the ratio of the charge amounts.

[0151] The resolution and linearity of the indirect type device depend on the system response of the photocharge demodulator and the A / D converter. The resolution and linearity of the indirect type device are lower than those of the direct type device. However, the indirect type device has the advantages of smaller pixel size, less complicated circuitry, and relatively reliable distance resolution, especially for distance measurements of several meters.

[0152] Indirect devices are further classified into two types according to the waveform of the modulated light. The first type is continuous wave (CW). The second type is short pulse wave. Indirect devices using short pulse waves concentrate the power of the irradiated light into a short pulse wave. As a result, the charge drain function of the pixel reduces the influence of ambient light. Therefore, indirect devices using short pulse waves have high resistance to ambient light.

[0153] However, the indirect type device using a short pulse wave has a problem of nonlinearity of the pixel source follower amplifier and a problem of distortion of the optical pulse waveform due to the analog domain processing in the measurement of the optical flight time. Furthermore, the indirect type device using a short pulse wave also has a problem of the finite photocarrier response time inside the photodiode and a problem of distortion of the transfer control pulse for demodulation. In other words, the indirect type device using a short pulse wave has a problem of analog imperfection.

[0154] For example, when considering the problem of photon shot noise limiting the range resolution, the full well capacity of the pixel limits the range resolution in an indirect device, and the full well capacity of the pixel limits the depth noise in an indirect device.

[0155] In indirect devices using short pulse waves, the use of extremely short pulse light is effective in improving distance resolution. For example, a sensor using optical pulses of less than 100 picoseconds has been reported. The report states that this sensor has a distance resolution of sub-100 μm. However, problems such as nonlinearity caused by short pulse light and skew of transfer control pulses remain issues. Indirect devices using short pulse waves require complex offline processing to correct the nonlinearity. Indirect devices using short pulse waves also require an on-chip skew correction circuit.

[0156] In the indirect type device, the transfer function of the demodulator and the A / D converter limits the resolution and linearity of the system response. In reality, the system response of the indirect type device has an offset and a nonlinear coefficient. The offset can be easily corrected. However, it is not easy to remove the nonlinear coefficient from the system response. In order to correct the nonlinear coefficient, a large-scale processing circuit is required. In addition, in order to correct the nonlinear coefficient, complex signal processing is required.

[0157] Therefore, according to the distance image measurement device 1 of the embodiment, the calculated value obtained from the difference in the amount of charge related to the time of flight of light is converted into a digital signal based on a predetermined condition. The digital signal is then used to control the timing of the transfer control pulse. As a result, it is possible to reduce the bias in the amount of charge for each charge readout region, which is expected to improve overall performance.

[0158] In other words, the distance image pickup device 1 of this embodiment performs measurement when the charge amounts in both charge storage sections are large, so that the effects of noise accompanying charge storage and readout can be minimized. Furthermore, in fine measurement, the application of delta-sigma modulation technology causes quantization error to be small in the low frequency range and large in the high frequency range, so that the quantization error can be effectively reduced by providing a low-pass filter in the subsequent stage.

[0159] The distance imaging device has been described in detail above based on the embodiment. However, the distance imaging device is not limited to the above embodiment. The distance imaging device can be modified in various ways without departing from the gist of the device.

[0160] The above-mentioned distance image pickup device operates while switching between C measurement and F measurement. For example, the distance image pickup device may perform either C measurement or F measurement. That is, the distance image pickup device may perform only C measurement to obtain approximate distance information. In this case, the distance image pickup device may omit the circuit configuration for F measurement. Furthermore, the distance image pickup device that performs only C measurement may perform an operation using two different unit delay times. That is, the distance image pickup device performs an operation based on a first unit delay time and an operation based on a second unit delay time that is shorter than the first delay time. According to such an operation, the operation based on the second unit delay time can also be regarded as a pseudo F measurement.

[0161] Furthermore, the distance image pickup device may only perform F measurement. In this case, the distance image pickup device may omit the circuit configuration for C measurement. [Explanation of symbols]

[0162] 1...distance image capturing device, 2...light source, 4...peripheral circuit, 10...pixel circuit unit, 10S...pixel circuit unit array, 21...buffer, 22...integrator, 30...A / D converter, 41...counter, 42...adder, 60...decimation filter, F10...delta-sigma modulator, FD1...first floating diffusion unit (first charge readout region), FD2...second floating diffusion unit (second charge readout region), G1...first transfer control gate (first control electrode), G2...second transfer control gate (second control electrode), SG1...first transfer control pulse, SG2...second transfer control pulse.

Claims

1. A light source that generates pulsed light; a pixel circuit array including a plurality of pixel circuits that generate charges corresponding to received light; A peripheral circuit that provides the pixel circuit array with a transfer control pulse for controlling an operation of the pixel circuit array, The pixel circuit unit includes: A photoelectric conversion region that converts light into the electric charge; a first charge readout region that accumulates the charges transferred from the photoelectric conversion region as first charges; a first control electrode that receives a first transfer control pulse that controls the transfer of the charges from the photoelectric conversion region to the first charge readout region; a second charge readout region that accumulates the charges transferred from the photoelectric conversion region as second charges; a second control electrode that receives a second transfer control pulse that controls the transfer of the charges from the photoelectric conversion region to the second charge readout region; The peripheral circuitry uses at least one of the first measurement and the second measurement to obtain a time of flight of light for obtaining a distance to an object, and obtains the distance to the object using the time of flight of light; The first measurement comprises: generating the first transfer control pulse and the second transfer control pulse based on a difference between an amount of the first charge stored in the first charge readout region and an amount of the second charge stored in the second charge readout region; searching for a delay time at which a sign of a difference between the amount of the first charge and the amount of the second charge obtained as a result of the first transfer control pulse and the second transfer control pulse is switched, thereby obtaining a time difference from a timing at which the light source emits the pulsed light to a timing at which the pixel circuit unit array receives reflected light caused by the pulsed light, and obtaining the light flight time using the time difference; The second measurement is repeating the generation of the first transfer control pulse and the second transfer control pulse by selecting a first delay time or a second delay time as a delay time to be applied to the first transfer control pulse and the second transfer control pulse based on an integral value of a difference between an amount of the first charge stored in the first charge readout region and an amount of the second charge stored in the second charge readout region; and obtaining the time of flight of light by taking a weighted average of the first delay time and the second delay time using the frequency of occurrence of the first delay time and the second delay time selected in the repetition.

2. The peripheral circuit includes: a first calculation unit that outputs a calculation value relating to a difference between a first voltage based on the first charge and a second voltage based on the second charge; a second calculation unit that uses the calculation value to determine a delay time for determining a timing for outputting the first transfer control pulse and the second transfer control pulse; The range imaging device according to claim 1 , wherein the peripheral circuit repeats a storage operation of storing the first charge and the second charge, an operation of the first calculation unit, and an operation of the second calculation unit.

3. The distance imaging device of claim 2, wherein, in an nth (n is an integer greater than or equal to 1) operation of the second calculation unit, when the calculation value is the first information, the second calculation unit determines the nth delay time used for the nth accumulation operation to be further delayed by a unit delay time as the n+1th delay time to be used for the n+1th accumulation operation.

4. The second calculation unit is a counter that performs an operation of adding 1 to a control variable when the calculated value is the first information in an n-th operation of the second calculation unit; The distance imaging device according to claim 2 or 3, further comprising a delay time determination unit that determines a value obtained by multiplying the control variable by a unit delay time as the (n+1)th delay time to be used for the (n+1)th accumulation operation.

5. the first calculation unit outputs a difference between the first voltage and the second voltage as a differential voltage, and then calculates an integral value of the m-th differential voltage by adding an integral value of the m-th differential voltage to an integral value of the differential voltage from a first time to an m-1-th time (m is an integer of 2 or more); 3. The distance imaging device of claim 2, wherein the second calculation unit determines the m+1th delay time used for the m+1th accumulation operation as the first delay time when the mth integration value is the second information, and determines the m+1th delay time as the second delay time when the mth integration value is the first information.

6. the first calculation unit has an integrator that calculates an m-th integral value by adding an m-th differential voltage to an integral value of differential voltages from a first time to an m-1-th time, The distance imaging device according to claim 2 or 5, wherein the second calculation unit has a delay time determination unit that determines a first delay time as the (m+1)th delay time when the mth integral value is the second information, and that determines a second delay time as the (m+1)th delay time when the mth integral value is the first information.

7. an A / D converter that converts the calculation value output by the first calculation unit into a digital signal, outputs the digital signal to the second calculation unit, and configures a delta-sigma modulator in cooperation with the first calculation unit and the second calculation unit; The distance imaging device of claim 5 or 6, further comprising a decimation filter that receives the digital signal output by the A / D converter and performs low-pass filtering and downsampling on the digital signal each time the accumulation operation, the operation of the first calculation unit, the operation of the A / D converter, and the operation of the second calculation unit are repeated.

8. the decimation filter has an oversampling ratio of the low-pass filter processing that is an integer greater than 2B−1 and less than 2B; 8. The range imaging device according to claim 7, wherein B is a minimum number of bits capable of expressing the oversampling ratio.

9. The peripheral circuit includes: repeating a process including: a storage operation for storing the first charge and the second charge when it is the first measurement; and an operation by the second calculation unit, when the calculation value is the first information in an n-th operation of the second calculation unit, of determining, as the n+1-th delay time to be used for the n+1-th storage operation, the n-th delay time used for the n-th storage operation, which is further delayed by a unit delay time; 3. The distance imaging device according to claim 2, wherein, in the second measurement, a process including the accumulation operation, an operation in which the first calculation unit outputs a difference between the first voltage and the second voltage as a differential voltage, and then calculates an integral value of the m-th differential voltage by adding an integral value of the differential voltage from the first time to the (m-1)th time to the m-th time differential voltage as an integral value of the m-th time differential voltage, and an operation in which the second calculation unit determines a first delay time as the delay time for the (m+1)th time when the m-th integral value is the second information, and determines a second delay time as the delay time for the (m+1)th time when the m-th integral value is the first information.

10. the first calculation unit includes an integrator that obtains a result of adding an m-th differential voltage to an integral value of a first to (m−1)-th differential voltage; The second calculation unit is a counter that performs an operation of adding 1 to a control variable when the calculated value is the first information in an n-th operation of the second calculation unit; and a delay time determination unit that selectively performs an operation of determining a value obtained by multiplying the control variable by a unit delay time as the n+1th delay time to be used for the n+1th accumulation operation, or an operation of determining a first delay time as the m+1th delay time when the mth integration value is second information, and determining a second delay time as the m+1th delay time when the mth integration value is first information.

11. the first calculation unit includes a differential amplifier that outputs a difference between the first voltage and the second voltage as a differential voltage; The differential amplifier comprises: an operational amplifier having a first input, a second input, a third input, a first output, and a second output; a feedback capacitor that receives the first charge from the first charge readout region and is connected to the first input terminal and the first output terminal; a feedback capacitor that receives the second charge from the second charge readout region and is connected to the second input terminal and the second output terminal; A range imaging device as described in any one of claims 4, 6 and 10, further comprising: a common mode feedback control unit connected to the third input terminal, the first output terminal and the second output terminal, and providing a signal to the third input terminal so that a difference between an output value from the first output terminal and an output value from the second output terminal is equal to a common voltage.

12. A method for capturing a distance image using a distance image capturing device, comprising: The distance image capturing device includes: a photoelectric conversion region that converts light into an electric charge; a first charge readout region that accumulates the charges transferred from the photoelectric conversion region as first charges; a first control electrode that receives a first transfer control pulse that controls the transfer of the charges from the photoelectric conversion region to the first charge readout region; a second charge readout region that accumulates the charges transferred from the photoelectric conversion region as second charges; a second control electrode that receives a second transfer control pulse that controls the transfer of the charges from the photoelectric conversion region to the second charge readout region; The method for capturing a distance image includes: a charge accumulation step of accumulating the first charge in the first charge readout region based on a first transfer control pulse applied to the first control electrode, and accumulating the second charge in the second charge readout region based on a second transfer control pulse applied to the second control electrode; a pulse generating step of generating the first transfer control pulse and the second transfer control pulse based on a magnitude relationship between the amount of the first charge and the amount of the second charge; acquiring a time of flight of light for acquiring a distance to an object, and acquiring the distance to the object using the time of flight of light; In the pulse generating step, a first operation of determining a delay time for determining a timing for outputting the first transfer control pulse and the second transfer control pulse based on a difference between an amount of the first charge and an amount of the second charge; a second operation of repeatedly selecting a first delay time or a second delay time as the delay time to be applied to the first transfer control pulse and the second transfer control pulse based on an integral value of a difference between the amount of the first charge and the amount of the second charge; In the step of acquiring a distance to the object, During the first operation, a delay time at which a sign of a difference between the amount of the first charge and the amount of the second charge is switched is searched for to obtain a time difference from a timing at which a pulsed light is emitted toward an object to a timing at which reflected light caused by the pulsed light is received, and the time difference is used to obtain the light flight time; A method for capturing a distance image, during the second operation, obtaining the time of flight of light by weighting the first delay time and the second delay time using the frequency of occurrence of the first delay time and the second delay time selected in the repetition of the second operation.

13. The pulse generating step includes: a calculation step of outputting a calculated value relating to a difference between a first voltage based on the first charge and a second voltage based on the second charge; a delay time determination step of determining a delay time for determining a timing to output the first transfer control pulse and the second transfer control pulse using the calculated value; The method of claim 12 , further comprising repeating the charge accumulation and pulse generation steps.

14. The method for capturing a distance image according to claim 13, wherein the delay time determination process includes a process of determining, when the calculated value in the nth calculation process is first information, the nth delay time used for the nth charge accumulation process, further delayed by a unit delay time, as the n+1th delay time to be used for the n+1th charge accumulation process.

15. The delay time determining step includes: a step of outputting a difference between the first voltage and the second voltage as a differential voltage, and then calculating an integral value of the m-th differential voltage by adding an integral value of the m-th differential voltage to an integral value of the 1st to (m-1)th differential voltages; The method for capturing a distance image according to claim 13, further comprising a step of determining the delay time for the (m+1)th integration step as the first delay time when the mth integration value is the second information, and determining the delay time for the (m+1)th integration step as the second delay time when the mth integration value is the first information.

16. The delay time determining step includes: a step of determining, when the calculated value is the first information in the n-th calculation step, the n-th delay time used for the n-th charge accumulation step, further delayed by a unit delay time, as the n+1-th delay time to be used for the n+1-th charge accumulation step; a step of outputting a difference between the first voltage and the second voltage as a differential voltage, and then calculating an integral value of the m-th differential voltage by adding an integral value of the m-th differential voltage to an integral value of the 1st to (m-1)th differential voltages; The method for capturing a distance image according to claim 13, further comprising a step of determining the delay time for the (m+1)th integration step as the first delay time when the mth integration value is the second information, and determining the delay time for the (m+1)th integration step as the second delay time when the mth integration value is the first information.

Images