Distance image capturing device and distance image capturing method

Abstract

This distance image capturing device comprises: a light source unit that emits radiation light into a measurement space, which is a space being measured; a distance image sensor that receives, as incident light, light including that reflected from an object in the space being measured, accumulates charges generated by the incident light in each pixel, and generates a distance image formed from the charge amount of the charges accumulated in each pixel; and a distance image processing unit that stores a signal value based on non-controlled charges not subject to a control for storing charges in the distance image sensor, from the charge amount in the distance image, and corrects and acquires the distance to the object in the space by using the stored signal value.

Description

Technical Field

[0001] The present invention relates to a distance image capturing device and a distance image capturing method. This application claims priority to PCT / JP2020 / 20221 filed on May 22, 2020, and incorporates its content by reference.

Background Art

[0002] Conventionally, a time-of-flight (TOF) distance image sensor that measures the distance between a measuring device and an object based on the flight time of light in space (measurement space) by utilizing the fact that the speed of light is known has been realized. In a TOF distance image sensor, an object to be measured is irradiated with a light pulse (e.g., near-infrared light, etc.), and the distance between the measuring device and the object is measured based on the difference between the time when the light pulse is irradiated and the time when the light pulse (reflected light) reflected by the object in the measurement space returns, that is, based on the flight time of light between the measuring device and the object (see, for example, Patent Document 1).

[0003] In a TOF distance image sensor, a photoelectric conversion element converts the amount of incident light into electric charge, accumulates the converted electric charge in a charge accumulation unit, and an analog-to-digital converter converts an analog voltage corresponding to the amount of charge accumulated into a digital value. Further, a TOF distance image sensor obtains the distance between the measuring device and the object based on the analog voltage corresponding to the amount of charge and the information on the flight time of light between the measuring device and the object included in the digital value.

[0004] At this time, background light in the environment of the measurement space is included when the reflected light used for distance measurement is incident. In order to accurately obtain the distance, it is necessary to remove the background light from the incident light and obtain only the information of the reflected light. Therefore, for the purpose of removing (canceling) the influence of background light in the environment of the measurement space in distance measurement, a period during which the irradiation light is not irradiated is always set, and the amount of received light of only the background light is accumulated. When calculating the distance to the object, the charge due to the incident light input during the period of receiving the reflected light from the object (including the information on the distance to the object) is subtracted from the charge due to the background light accumulated during the period when the irradiation light is not irradiated, and the charge of only the reflected light is obtained.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] In the case of a TOF sensor, Patent Document 1 performs multiple irradiations of the irradiation light, accumulates the charges generated by the light received each time the irradiation is performed, and uses the analog voltage corresponding to the accumulated charges for distance measurement. At this time, each time the irradiation light is irradiated, the reflected light reflected from the object is incident, and the distance is calculated from the ratio of each of the charges Q2 and Q3 generated by the incident light. At this time, the light received by the TOF sensor includes background light in the environment in addition to the reflected light reflected from the object by the irradiation light.

[0007] As a result, each of the charges Q2 and Q3 generated by the incident light includes, in addition to the charge generated by the reflected light, the charge Q1 generated by the background light. The charge Q1 due to the background light causes a decrease in accuracy when calculating the distance from the TOF sensor to the object. Therefore, before irradiating the irradiation light, only the background light is made incident on the TOF sensor to obtain the charge Q1, and the distance L is calculated by the following formula (1).

[0008] L = ((Q3 - Q1) / (Q2 + Q3 - 2Q1)) × (cTw) / 2 …(1) In equation (1), c is the speed of light and Tw is the pulse width of the irradiated light. By using equation (1), the distance L from the TOF sensor to the object can be calculated. Here, (cTw) / 2 indicates the maximum distance (maximum measurement distance) that can be measured by the irradiation of the optical pulse PO. However, as shown in FIG. 11, in a distance image capturing device using a TOF sensor, a lens 31 is provided to collect light in the pixel region of a distance image sensor 32 (sensor chip) on which pixel elements are formed. In FIG. 11(A), the lens 31 forms an image of incident light with a predetermined viewing angle α for each pixel in the pixel region of the distance image sensor 32. Therefore, for each pixel of the distance image sensor 32, as shown in FIG. 11(B), incident light with various incident angles from the entire lens surface such as a spherical lens surface like the lens 31 is imaged on each pixel in the distance image sensor 32.

[0009] Here, in the distance image sensor 32, the incident angle θ2 of the incident light with respect to the pixel of the imaging point 502 at the end corresponding to the viewing angle α2 is larger than the incident angle θ1 of the incident light with respect to the pixel of the imaging point 501 corresponding to the viewing angle α1 on the axis f of the lens 31. Corresponding to the change in the incident angle θ of the incident light for each pixel, the charge Q1 generated by the background light is different. Also, the charge accumulated for each pixel includes different non-control charges QB1 respectively. Further, the charges Q2 and Q3 accumulated for each pixel also include different non-control charges QB2 and QB3 respectively. The non-control charges described above refer to charges other than the control charges that are collected by a photoelectric conversion element (a photodiode PD described later) and accumulated in a charge accumulation section FD by a control that turns on / off a read gate transistor G to distribute the charges.

[0010] FIG. 12 is a diagram for explaining the generation of uncontrolled charges QB1, QB2, and QB3 included in each of charges Q1, Q2, and Q3 depending on the incident angle of incident light with respect to a pixel. FIG. 12(A) shows a plan view configuration of a pixel in an example of a TOF sensor. The charge generated in a photodiode PD, which is a photoelectric conversion element of the pixel, and distributed to a charge accumulation section FD1 (corresponding to a charge accumulation section CS1 described later) becomes charge Q1. More precisely, the charge generated by photoelectric conversion in a silicon substrate on which a pixel including the photodiode PD is formed is collected by the photodiode PD, and by turning on a read gate transistor G1, the charge distributed to the charge accumulation section FD1 becomes charge Q1. Similarly, the charge collected by the photodiode PD and distributed to a charge accumulation section FD2 (corresponding to a charge accumulation section CS2 described later) becomes charge Q2, and the charge distributed to a charge accumulation section FD3 (corresponding to a charge accumulation section CS2 described later) becomes charge Q3. By turning on the read gate transistor G1, the charge collected by the photodiode PD is distributed to the charge accumulation section FD1. By turning on the read gate transistor G2, the charge collected by the photodiode PD is distributed to the charge accumulation section FD2. Also, by turning on the read gate transistor G3, the charge collected by the photodiode PD is distributed to the charge accumulation section FD3.

[0011] FIG. 12(B) shows a cross-sectional configuration along line segment A-A' in FIG. 12(A). It shows a case where the incident light with respect to the surface of the sensor chip of the distance image sensor 32 is not 0°, but the incident angle is -20° with respect to a plane parallel to line segment A-A' and perpendicular to the sensor chip plane. Here, in each of the charge storage parts FD1, FD2, and FD3, there are control charges QA1, QA2, and QA3, which are charges distributed through readout gate transistors G1, G2, and G3 respectively, of the charges collected by the photodiode PD, and non-control charges QB1, QB2, and QB3, which are charges flowing in without passing through the readout gate transistors G1, G2, and G3. Here, each of the non-control charges QB1, QB2, and QB3 indicates the charges flowing into the charge storage parts FD1, FD2, and FD3 without readout control being performed in the pixel circuit. These non-control charges QB1, QB2, and QB3 are not the same for background light, and are different in each of the charge storage parts FD1, FD2, and FD3 depending on the incident angle of the incident light.

[0012] In the case of FIG. 12(B), since more incident light enters the charge storage part FD2 side compared to the charge storage part FD3 side, the non-control charge QB2 flowing into the charge storage part FD2 is larger than the non-control charge QB3 flowing into the charge storage part FD3.

[0013] FIG. 13 is a diagram showing that the charges stored in the charge storage part change according to the incident angle of the incident light on the sensor chip. FIG. 13(A) shows the concept of an experiment in which, with respect to the radiation (irradiation) direction of the collimated light, the angle of the surface on which the pixels of the distance image sensor 32 are formed is tilted, and the change in the incident angle of the incident light on the pixels of the distance image sensor 32 by the lens 31 is pseudo-simplyfied. FIG. 13(B) shows the incident angle on the horizontal axis and the digital values (LSB) corresponding to the amounts of charge Q1 (solid line), Q2 (dashed-dotted line), and Q3 (two-dot chain line) in each of the charge storage parts FD1, FD2, and FD3 on the vertical axis. The incident light amount decreases by the reduction of the projected area depending on the incident angle of the incident light. Specifically, if the incident angle is θ, it changes by cos(ine) θ times. Therefore, ideally, the amount of charge collected by the photodiode PD also changes by cos(ine) θ times if the incident angle is θ. Here, the control charges QA1, QA2, and QA3 collected by the photodiode PD and distributed to the charge storage parts FD1, FD2, and FD3 by the respective read gate transistors G1, G2, and G3 are the same for each. Therefore, for example, when the incident angle θ is negative, the charge Q2 becomes a larger charge amount than the other charges Q1 and Q3 because the non-control charges QB1, QB2, and QB3 are different depending on the incident angle of the incident light.

[0014] FIG. 14 is a conceptual diagram showing the relationship of the non-control charges QB1, QB2, and QB3 in each of the charges Q1, Q2, and Q3. FIG. 14(A) shows the case where each of the non-control charges QB1, QB2, and QB3 is the same, and shows each of the charges Q1, Q2, and Q3 at a small incident angle (less than 20° described later) such as an incident angle of 0°. In the case of FIG. 14(A), by removing the charge Q1 from each of the charges Q2 and Q3 as shown in Equation (1), all the charges generated by the background light can be removed, and the distance can be calculated with high accuracy only from the charges generated by the reflected light. On the other hand, FIG. 14(B) shows the case where each of the non-control charges QB1, QB2, and QB3 is not the same, and shows each of the charges Q1, Q2, and Q3 at a large incident angle such as an incident angle of 20°. In the case of FIG. 14(B), even if the charge Q1 is subtracted from each of the charges Q2 and Q3 as shown in Equation (1), the amount of charge generated by the background light in each of the charges Q2 and Q3 is different from that of the charge Q1, and a difference due to the difference in each of the non-control charges QB1, QB2, and QB3 remains in the subtraction result, so that the charges generated by the background light cannot be completely removed, and the distance cannot be calculated with high accuracy.

[0015] Conventionally, the amounts of non-controlled charges QB1, QB2, and QB3 flowing from the photodiode PD into the charge storage units FD1, FD2, and FD3 respectively are generated by reflected light, collected by the photodiode PD, and by turning on the readout gate transistors G1, G2, and G3, the amounts of charges distributed to the charge storage units FD1, FD2, and FD3 respectively are in a ratio that can be ignored (the ratio within the error range), and the accuracy of the distance calculated from the above formula (1) is also of such a magnitude that it is included within the error range. However, in order to increase the resolution of the distance image or reduce the chip size of the distance image sensor 32, it is necessary to reduce the area for forming pixels. Therefore, the amount of charge generated by the reflected light decreases corresponding to the area ratio, and the difference in the amounts of non-controlled charges QB1, QB2, and QB3 becomes a ratio that cannot be ignored compared to the amount of charge generated by the reflected light, and the accuracy of the distance calculated by the formula (1) decreases according to the reduction of the pixel area.

[0016] In view of the above problems, even when reducing the pixel area in a distance image sensor, without being affected by each non-controlled charge whose charge amount varies depending on the incident angle of the incident light included in each of the charges accumulated in the charge storage unit, an object and itself To provide a distance image capturing apparatus and a distance image capturing method for obtaining the distance with the same accuracy as when the pixels are not reduced.

Means for Solving the Problems

[0017] The distance image capturing apparatus of the present invention includes a light source unit that irradiates irradiation light to a measurement space that is a space of a measurement target, receives light including reflected light from an object in the measurement space as incident light, accumulates charges generated by the incident light for each pixel, and generates a distance image composed of the amounts of charges accumulated for each pixel. A distance image sensor, and a distance image processing unit that stores a signal value based on a non-controlled charge included in the amount of charge without being affected by the control for accumulating the charge in the distance image sensor from the amount of charge in the distance image, and corrects the distance to the object in the space using the stored signal value.

[0018] The distance image capturing device of the present invention includes a photoelectric conversion element that collects charges generated in response to the incident light by the distance image sensor, and a charge accumulation unit that accumulates the charges in a frame period, and includes a pixel circuit for each pixel that controls the accumulation of the charges in the charge accumulation unit. The distance image processing unit subtracts an adjustment voltage corresponding to the charge amount of the non-control charge, which is the charge flowing into the charge accumulation unit regardless of the control by the pixel circuit, from the input voltage corresponding to the charge amount accumulated in the charge accumulation unit, and measures the distance between the distance image sensor and the measurement target.

[0019] The distance image capturing device of the present invention includes at least one first charge accumulation unit that accumulates background light charges generated by receiving the background light in the space, and two or more second charge accumulation units that accumulate reflected light charges generated by receiving the reflected light from the object of the irradiation light.

[0020] The distance image capturing device of the present invention stores, in advance, charges controlled by the pixel circuit in each of the first charge accumulation unit and the second charge accumulation unit measured in a state where the irradiation light is not irradiated under predetermined ambient light, and stores each of the reference background light voltages obtained from each of the first charge accumulation unit and the second charge accumulation unit, and each of the reference reference voltages obtained from each of the first charge accumulation unit and the second charge accumulation unit without accumulating the charges controlled by the pixel circuit in each of the first charge accumulation unit and the second charge accumulation unit, and includes a storage unit.

[0021] The distance image capturing device of the present invention irradiates the measurement space with irradiation light, receives light including reflected light from an object in the measurement space as incident light, accumulates the charge generated by the incident light in each of the first charge accumulation unit and the second charge accumulation unit, and the distance image processing unit divides the background light voltage generated by the charge accumulated in the first charge accumulation unit by the reference background light voltage measured in advance and stored in the storage unit to obtain an adjustment ratio, and multiplies each of the reference reference voltages by the adjustment ratio to calculate the adjustment voltage for each of the input voltages.

[0022] The distance image capturing device of the present invention includes each of a first frame period and a second frame period in the frame period. In the first frame period, the distance image sensor irradiates the measurement space with irradiation light, receives light including reflected light from an object in the measurement space as incident light, accumulates the charge controlled by the pixel circuit in each of the first charge accumulation unit and the second charge accumulation unit, and the distance image processing unit acquires the input voltage generated by the accumulated charge. In the second frame period, the distance image sensor irradiates the measurement space with irradiation light, receives light including reflected light from an object in the measurement space as incident light, does not accumulate the charge controlled by the pixel circuit in each of the first charge accumulation unit and the second charge accumulation unit, and the distance image processing unit acquires the adjustment voltage corresponding to the non-controlled charge in each of the first charge accumulation unit and the second charge accumulation unit.

[0023] The distance image capturing device of the present invention includes a storage unit that stores, respectively, the reference reflected light voltages obtained from each of the first charge accumulation unit and the second charge accumulation unit, in which the pixel circuit has accumulated the charge controlled thereby, in a state where the irradiation light is irradiated in a shielded environment, and the reference reference voltages obtained from each of the first charge accumulation unit and the second charge accumulation unit, without the pixel circuit accumulating the charge controlled thereby.

[0024] In the distance image capturing device of the present invention, a distance image processing unit divides the result of adding each of the input voltages generated by the charges accumulated in each of the first charge accumulation unit and the second charge accumulation unit by the result of adding each of the reference reflected light voltages to obtain an adjustment ratio, and multiplies the reference reference voltage by the adjustment ratio to calculate each of the adjustment voltages for each of the input voltages.

[0025] The distance image capturing device of the present invention includes the frame period including each of a first frame period and a second frame period. In the first frame period, the distance image sensor irradiates the measurement space with irradiation light in a light-shielded environment, receives light including reflected light from an object in the measurement space as incident light, accumulates the charges controlled by the pixel circuit in each of the first charge accumulation unit and the second charge accumulation unit, and the distance image processing unit acquires the input voltage generated by the charges accumulated in each of the first charge accumulation unit and the second charge accumulation unit. In the second frame period, the distance image sensor irradiates the measurement space with irradiation light in a light-shielded environment, receives light including reflected light from an object in the measurement space as incident light, does not accumulate the charges controlled by the pixel circuit in each of the first charge accumulation unit and the second charge accumulation unit, and the distance image processing unit acquires an adjustment voltage corresponding to the non-controlled charge by charge in each of the first charge accumulation unit and the second charge accumulation unit.

[0026] The distance image capturing device of the present invention stores, in a storage unit, each of: the charge accumulated in each of the first charge storage unit and the second charge storage unit under a state where the irradiation light is not irradiated under predetermined ambient light, which is controlled by the pixel circuit; the reference background photovoltage acquired from each of the first charge storage unit and the second charge storage unit; the first reference reference voltage acquired as a reference reference voltage from each of the first charge storage unit and the second charge storage unit without accumulating the charge controlled by the pixel circuit in each of the first charge storage unit and the second charge storage unit; the charge accumulated in each of the first charge storage unit and the second charge storage unit under a state where the irradiation light is irradiated under a shielded environment, which is measured in advance and is controlled by the pixel circuit; the reference reflected photovoltage acquired from each of the first charge storage unit and the second charge storage unit; and the second reference reference voltage acquired as a reference reference voltage from each of the first charge storage unit and the second charge storage unit without accumulating the charge controlled by the pixel circuit in each of the first charge storage unit and the second charge storage unit.

[0027] In the distance image capturing device of the present invention, the distance image sensor irradiates irradiation light onto the measurement space, receives light including reflected light from an object in the measurement space as incident light, accumulates charges generated by the incident light for each pixel, the distance image processing unit divides the input voltage generated by the accumulated charges by the reference background photovoltage measured in advance and stored in the storage unit to obtain a first adjustment ratio as an adjustment ratio, multiplies each of the first reference reference voltages by the first adjustment ratio to calculate a first adjustment voltage for each of the input voltages, divides the result of adding each of the input voltages by the addition result of adding each of the reference reflected photovoltages to obtain a second adjustment ratio as an adjustment ratio, multiplies each of the second reference reference voltages by the second adjustment ratio to calculate each of the second adjustment voltages for each of the input voltages, and adds the first adjustment voltage and the second adjustment voltage to calculate the adjustment voltage.

[0028] The distance image capturing device of the present invention accumulates the charges controlled by the pixel circuit in each of N (N is an integer of 3 or more) charge accumulation units that have been measured in advance in a state where the irradiation light is not irradiated under predetermined ambient light. It acquires each of the reference background photovoltages obtained from each of the charge accumulation units, the first reference reference voltage obtained as a reference reference voltage from each of the charge accumulation units without accumulating the charges controlled by the pixel circuit in each of the charge accumulation units, and the irradiation light is irradiated in a shielded environment. The charges controlled by the pixel circuit are accumulated in each of the N charge accumulation units that have been measured in advance, the reference reflected photovoltages obtained from each of the charge accumulation units, and the charges controlled by the pixel circuit are not accumulated in each of the charge accumulation units. Each of the second reference reference voltages obtained as reference reference voltages from each of the charge accumulation units is acquired, and the control charges generated by the background light and collected by the photoelectric conversion element and distributed to and accumulated in each of the N charge accumulation units are obtained from each of the reference background photovoltages and the first reference reference voltage. A reference background light control voltage corresponding to the amount of charge, a reference background light non-control voltage corresponding to the amount of non-control charge that has flowed in without being distributed to each of the N charge accumulation units generated by the background light, and the control charges generated by the reflected light and collected by the photoelectric conversion element and distributed to and accumulated in each of the N charge accumulation units. A storage unit that stores each of the reference reflected light control voltages and the reference reflected light non-control voltages corresponding to the amount of non-control charge that has flowed into and accumulated in each of the N charge accumulation units without distribution generated by the reflected light is provided.

[0029] In the state where the distance image capturing device of the present invention captures the distance image, a first ratio is calculated and obtained as the ratio between the amount of background light in the pre-measured environment obtained from a part or all of the reference background photovoltage, the first reference reference voltage, the reference background light control voltage, and the reference background light non-control voltage stored in the storage unit, and the amount of background light in the imaging state. A second ratio is calculated and obtained as the ratio between the amount of reflected light in the pre-measured environment obtained from a part or all of the reference reflected photovoltage, the second reference reference voltage, the reference reflected light control voltage, and the reference reflected light non-control voltage stored in the storage unit, and the amount of reflected light in the imaging state. A simultaneous equation obtained by using each of the first ratio and the second ratio is solved, and a reflected light control voltage corresponding to the amount of charge of the control charge generated by the reflected light in the imaging state and accumulated in the charge accumulation unit is calculated, and a distance calculation unit for obtaining the distance to the object is provided.

[0030] The distance image capturing device of the present invention performs control for distributing the irradiation light to each of the N charge accumulation units in a pre-measured state in a shielded environment, and the reference reflected photovoltage corresponding to the amount of charge including the control charge accumulated by the distribution and the non-control charge that flows in and accumulates regardless of the distribution, and the second reference reference voltage obtained as the reference reference voltage corresponding to the amount of charge of the non-control charge that flows in and accumulates without performing control for accumulating in each of the charge accumulation units are each obtained, and the reference reflected photovoltage and the second reference reference voltage are each obtained from the reference reflected photovoltage and the second reference reference voltage, and the reference reflected light control voltage corresponding to the amount of charge of the control charge generated by the reflected light and collected by the photoelectric conversion element and distributed and accumulated in each of the N charge accumulation units, and the reference reflected light non-control voltage corresponding to the amount of charge of the non-control charge generated by the reflected light and flowing into and accumulating in each of the N charge accumulation units without performing distribution are each stored in a storage unit.

[0031] In the acquisition state of acquiring the distance, the frame period of the distance image capturing device of the present invention includes each of a first frame period and a second frame period. In one of the two frame periods of the first frame period and the second frame period, the distance image sensor acquires a first reference reference voltage as a reference reference voltage corresponding to each of the amounts of non-controlled charges accumulated in each of the N charge accumulation units without irradiating the measurement space with the irradiation light. In the other of the two frame periods, the distance image sensor irradiates the measurement space with the irradiation light and performs control to distribute charges to each of the N charge accumulation units, and acquires a voltage for obtaining a distance corresponding to the amount of charge including the controlled charge accumulated by the distribution and the non-controlled charge that has flowed in regardless of the distribution from each of the charge accumulation units. The ratio of the amount of reflected light in the pre-measured environment obtained from a part or all of the reference reflected light voltage, the second reference reference voltage, the first reference reference voltage, and the voltage for obtaining the distance stored in the storage unit to the amount of reflected light in the acquisition state is calculated and obtained, and a simultaneous equation is solved to calculate a reflected light control voltage corresponding to the amount of charge of the controlled charge, and a distance calculation unit for obtaining the distance to the object is provided.

[0032] When the distance image capturing device of the present invention acquires the distance in a light-shielded environment or a dark environment where ambient light can be ignored, the frame period includes each of a first frame period and a second frame period. In one of the two frame periods of the first frame period and the second frame period, the distance image sensor irradiates the irradiation light, receives the incident light from the measurement space, and accumulates non-controlled charges that flow into each of the N charge accumulation units in a state where the charges controlled by the pixel circuit are not accumulated. A second reference reference voltage is acquired from each of the charge accumulation units, and in the other of the two frame periods, the distance image sensor irradiates the irradiation light, receives the incident light from the measurement space, and distributes and accumulates charges from the photoelectric conversion element to each of the charge accumulation units under the control of the pixel circuit. A reference reflected light voltage that is also a voltage for acquiring a distance corresponding to the charge amount including each of the non-controlled charges and the controlled charges accumulated under the control of the pixel circuit is acquired. A simultaneous equation using the second reference reference voltage and the reference reflected light voltage is solved, and only the reflected light control voltage corresponding to the charge amount of the controlled charges is calculated, and a distance calculation unit for obtaining the distance to the object is provided.

[0033] The distance image capturing device of the present invention further includes a lens that receives the incident light from the space, and the distance image sensor receives the incident light through the lens.

[0034] The distance image capturing device of the present invention further includes a lens that receives the incident light from the space. Since the incident light is incident on each of the pixels through the lens, corresponding to the characteristics of the lens, the pixels within a predetermined difference range of the adjustment ratio are divided into groups, and the median value of the reference reference voltage in the group is used as the reference reference voltage for all the pixels in the group.

[0035] Since the incident light of the distance image capturing device of the present invention is incident on each of the pixels through the lens, a plurality of characteristics of the lens are stored in a storage unit corresponding to the characteristics of the lens.

[0036] In the distance image capturing device of the present invention, since the incident light enters each of the pixels via the lens, a adjustment function that outputs the adjustment ratio corresponding to each position of the pixel corresponding to the characteristics of the lens is stored in the storage unit.

[0037] The distance image capturing method of the present invention includes: a distance image sensor irradiating irradiation light from a light source unit onto a measurement space that is a space to be measured, receiving light including reflected light from an object in the measurement space as incident light, accumulating charges generated by the incident light for each pixel, and generating a distance image having the amount of charge accumulated for each pixel; and a distance image processing process of obtaining the distance to the object in the space based on a corrected charge amount obtained by removing non-controlled charges included in the amount of charge that are not affected by the control of accumulating the charges in the distance image sensor from the amount of charge in the distance image.

Effect of the Invention

[0038] The present invention can provide a distance image capturing device and a distance image capturing method that can obtain the distance between an object and itself with the same accuracy as when the area of the pixels in the distance image sensor is not reduced, without being affected by non-controlled charges each having a different charge amount depending on the incident angle of the incident light included in each of the charges accumulated in the charge accumulation unit even when the area of the pixels in the distance image sensor is reduced.

Brief Description of the Drawings

[0039]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Embodiments for Carrying Out the Invention

[0040] <First Embodiment> Hereinafter, a first embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a schematic configuration of a distance image capturing device according to the first embodiment of the present invention. Note that FIG. 1 also shows a subject S whose distance is measured in the distance image capturing device 1. The distance image capturing device 1 having the configuration shown in FIG. 1 includes a light source unit 2, a light receiving unit 3, and a distance image processing unit 4.

[0041] The light source unit 2 irradiates a space (measurement space P) where a subject S, which is an object to be measured for distance in the distance image capturing device 1, exists with intermittent light pulses PO at a predetermined period in accordance with control from the distance image processing unit 4. The light source unit 2 is, for example, a surface emitting semiconductor laser module such as a vertical cavity surface emitting laser (VCSEL).

[0042] The light source device 21 is a light source that emits laser light in the near-infrared wavelength band (for example, a wavelength band with a wavelength of 850 nm to 940 nm) that becomes the light pulse PO for irradiating the subject S. The light source device 21 is, for example, a semiconductor laser light-emitting element. The light source device 21 emits a light pulse as pulsed laser light in response to the control from the timing control unit 41.

[0043] The diffusion plate 22 is an optical component that diffuses the laser light in the near-infrared wavelength band emitted by the light source device 21 to a predetermined cross-sectional area size for irradiating the measurement space P where the subject S is located. The pulsed laser light diffused by the diffusion plate 22 is emitted from the light source unit 2 as the light pulse PO and irradiates the subject S in the measurement space P.

[0044] The light receiving unit 3 receives the reflected light RL of the light pulse PO reflected by the subject S, which is the object to be measured in the distance image capturing device 1, and outputs a pixel signal corresponding to the received reflected light RL.

[0045] The lens 31 is an optical lens that guides the incident reflected light RL to the distance image sensor 32. The lens 31 emits the incident reflected light RL toward the distance image sensor 32 side and causes it to be received (incident) by the pixels provided in the light receiving region of the distance image sensor 32.

[0046] The distance image sensor 32 is an imaging element used in the distance image capturing device 1. The distance image sensor 32 includes a plurality of pixels in a two-dimensional light receiving region, and in each pixel, there is provided a photoelectric conversion element, a plurality of charge storage units corresponding to this one photoelectric conversion element, and a component for distributing charges to each charge storage unit, which is an imaging element having a distribution configuration. The distance image sensor 32 distributes the charges generated by the photoelectric conversion elements constituting the pixels to the respective charge storage units in response to the control from the timing control unit 41, and outputs a pixel signal corresponding to the amount of charge distributed to each charge storage unit.

[0047] Note that in the distance image sensor 32, a plurality of pixels are arranged in a two-dimensional grid (matrix) form, and each pixel outputs a pixel signal for one frame corresponding thereto.

[0048] The distance image processing unit 4 is a control unit that controls the entire distance image capturing device 1, and is also an arithmetic unit that calculates the distance to the subject S measured in the distance image capturing device 1. This distance image processing unit 4 includes a timing control unit 41, a distance calculation unit 42, and a correction parameter storage unit 43.

[0049] The timing control unit 41 controls the timing at which the light source unit 2 irradiates the subject S with the light pulse PO, the timing at which the distance image sensor 32 provided in the light receiving unit 3 receives and accumulates the reflected light RL, and the like.

[0050] The distance calculation unit 42 outputs distance information obtained by calculating the distance between the distance image capturing device 1 and the subject S based on the pixel signal output from the distance image sensor 32. Further, when calculating the distance information between the distance image capturing device 1 and the subject S, the distance calculation unit 42 uses, as an adjustment voltage, a voltage component corresponding to the non-control charge included in the input voltage obtained from the charge amount in the pixel signal and used for the calculation of the distance, subtracts and removes it from the input voltage, obtains a corrected input voltage, and performs the calculation of the distance according to the formula (1) (described in detail later).

[0051] The correction parameter storage unit 43 stores correction parameters for generating an adjustment voltage to be subtracted from the input voltage when the distance calculation unit 42 removes the voltage corresponding to the non-control charge from the input voltage (described in detail later).

[0052] With such a configuration, in the distance image capturing device 1, the light receiving unit 3 receives the reflected light RL of the light pulse PO in the near-infrared wavelength band irradiated by the light source unit 2 to the subject S, and the distance image processing unit 4 calculates and outputs the distance information measuring the distance to the subject S by using the input voltage from which the voltage component corresponding to the non-control charge corresponding to the incident light is removed.

[0053] In FIG. 1, a distance image capturing device 1 having a configuration including a distance image processing unit 4 inside is shown. However, the distance image processing unit 4 may be a component provided outside the distance image capturing device 1.

[0054] Next, the configuration of the distance image sensor 32 used as an imaging element in the distance image capturing device 1 will be described. FIG. 2 is a block diagram showing a schematic configuration of the imaging element (distance image sensor 32) used in the distance image capturing device 1 according to the first embodiment of the present invention. In FIG. 2, the distance image sensor 32 includes a light receiving pixel unit 320 in which a plurality of pixels 321 are arranged, a control circuit 322, a vertical scanning circuit 323, a horizontal scanning circuit 324, a pixel signal processing circuit 325, and a pixel driving circuit 326. Note that the distance image sensor 32 shown in FIG. 2 shows an example of a light receiving pixel unit 320 in which a plurality of pixels 321 are arranged in a two-dimensional lattice of 8 rows and 8 columns.

[0055] The control circuit 322 controls components provided in the distance image sensor 32, such as the vertical scanning circuit 323, the horizontal scanning circuit 324, the pixel signal processing circuit 325, and the pixel driving circuit 326. The control circuit 322 controls the operation of the components provided in the distance image sensor 32, for example, in response to control from the distance image processing unit 4 (more specifically, the timing control unit 41) provided in the distance image capturing device 1. Note that the control of the components provided in the distance image sensor 32 by the control circuit 322 may be a configuration in which the distance image processing unit 4 (more specifically, the timing control unit 41) directly performs the control. In this case, the distance image sensor 32 may be configured not to include the control circuit 322.

[0056] The pixel drive circuit 326 distributes and accumulates the charges generated by the photoelectric conversion elements (also referred to as the photoelectric conversion element PD or the photodiode PD described later) provided in the pixels 321 arranged in a grid pattern to a plurality of charge accumulation parts (charge accumulation parts CS1, CS2, CS3 described later) provided in the pixel 321, and outputs an accumulation drive signal (accumulation drive signals TX1, TX2, TX3 described later), a reset signal (reset signals RST1, RST2, RST3 described later), and a reset drive signal (reset drive signal RSTD described later) in units of columns of the pixels 321 arranged in a grid pattern within the light-receiving pixel unit 320. The vertical scanning circuit 323 is a drive circuit that controls each of the pixels 321 arranged within the light-receiving pixel unit 320 in accordance with the control from the control circuit 322, and causes each of the pixels 321 to output (read out) a voltage signal (hereinafter referred to as a "voltage signal") corresponding to the amount of charge obtained by photoelectrically converting the incident light to the corresponding vertical signal line 327. The vertical scanning circuit 323 outputs control signals (selection drive signals SEL1, SEL2, SEL3 described later) for driving (controlling) and reading out the pixels 321 in units of rows of the pixels 321 arranged in a grid pattern within the light-receiving pixel unit 320. As a result, voltage signals corresponding to the amounts of charge distributed to the respective charge accumulation parts (charge accumulation parts CS1, CS2, CS3 described later) in the pixel 321 are read out to each of the corresponding vertical signal lines 327 for each column of the light-receiving pixel unit 320 and output to the pixel signal processing circuit 325.

[0057] In the light-receiving pixel section 320, the pixel 321 receives the reflected light RL reflected by the subject S from the light pulse PO irradiated by the light source section 2 onto the subject S, and generates charges corresponding to the amount of the received reflected light RL (light reception amount). In each pixel 321, the pixel drive circuit 326 outputs an accumulation drive signal to distribute and accumulate charges corresponding to the amount of the received reflected light RL (light reception amount) to any one of a plurality of provided charge accumulation sections. Then, in the pixel 321, the vertical scanning circuit 323 outputs a selection drive signal as a read drive signal, and outputs a voltage signal having a magnitude corresponding to the amount of charge of the charges distributed and accumulated in each charge accumulation section to the corresponding vertical signal line 327. Note that a detailed description of the configuration and drive (control) method of the pixel 321 will be described later.

[0058] The pixel signal processing circuit 325 is a signal processing circuit that performs predetermined signal processing on the voltage signals output to the corresponding vertical signal lines 327 from the pixels 321 in each column according to the control from the vertical scanning circuit 323. Examples of the predetermined signal processing include noise suppression processing for suppressing noise included in the voltage signal by, for example, correlated double sampling (CDS).

[0059] Note that the pixel signal processing circuit 325 may be a pixel signal processing circuit group including a plurality of pixel signal processing circuits corresponding to each column of the light-receiving pixel section 320. In this case, the pixel signal processing circuit 325 outputs the voltage signal after performing predetermined signal processing to the AD conversion circuit provided inside according to the control from the control circuit 322, and the AD conversion circuit outputs the AD-converted digital value to the horizontal signal line 329 for each row of the light-receiving pixel section 320 according to the control of the horizontal scanning circuit 324.

[0060] The vertical scanning circuit 323 sequentially outputs read drive signals for outputting voltage signals corresponding to the pixels 321 in each column to the pixel signal processing circuit 325. In response to the control from the control circuit 322, the horizontal scanning circuit 324 sequentially outputs (reads out) the digital values obtained by AD-converting the voltage signals after signal processing to the horizontal signal line 329. As a result, the voltage signals for one frame after the signal processing output by the pixel signal processing circuit 325 are sequentially output to the outside of the distance image sensor 32 as pixel signals for one frame via the horizontal signal line 329. At this time, the distance image sensor 32 outputs the voltage signal after signal processing as a pixel signal to the outside of the distance image sensor 32 from an output circuit (not shown) such as an output amplifier, for example.

[0061] In the following description, it is assumed that the pixel signal processing circuit 325 provided in the distance image sensor 32 performs noise suppression processing on the voltage signal output from the pixel 321, and then performs A / D conversion processing in the AD conversion circuit and outputs it, that is, the voltage signal converted into a digital value is output from the horizontal signal line 329.

[0062] Next, the configuration of the pixel 321 disposed in the light-receiving pixel portion 320 provided in the distance image sensor 32 will be described. FIG. 3 is a circuit diagram showing an example of the configuration of the pixel 321 disposed in the light-receiving pixel portion 320 of the imaging device (distance image sensor 32) used in the distance image imaging apparatus 1 according to the embodiment of the present invention. FIG. 3 shows an example of the configuration of one pixel 321 among a plurality of pixels 321 disposed in the light-receiving pixel portion 320. The pixel 321 is an example of a configuration including three pixel signal readout portions.

[0063] Pixel 321 includes one photoelectric conversion element PD, a drain gate transistor GD, and three pixel signal readout units RU that output voltage signals from corresponding output terminals O. Each of the pixel signal readout units RU includes a readout gate transistor G, a floating diffusion FD, a charge storage capacitor C, a reset gate transistor RT, a source follower gate transistor SF, and a selection gate transistor SL. In each pixel signal readout unit RU, a charge storage section CS is configured by the floating diffusion FD and the charge storage capacitor C. The drain gate transistor GD, the readout gate transistor G, the reset gate transistor RT, the source follower gate transistor SF, and the selection gate transistor SL are N-channel MOS transistors.

[0064] In FIG. 3, each pixel signal readout unit RU is distinguished by assigning a number "1", "2", or "3" after the symbol "RU" of the three pixel signal readout units RU. Similarly, each component provided in the three pixel signal readout units RU is also represented by indicating the number representing each pixel signal readout unit RU after the symbol, so as to distinguish and represent the pixel signal readout unit RU to which each component corresponds. In the pixel 321 shown in FIG. 3, the pixel signal readout unit RU1 that outputs a voltage signal from the output terminal O1 includes a gate transistor G1, a floating diffusion FD1, a charge storage capacitor C1, a reset gate transistor RT1, a source follower gate transistor SF1, and a selection gate transistor SL1. In the pixel signal readout unit RU1, a charge storage section CS1 is configured by the floating diffusion FD1 and the charge storage capacitor C1. The pixel signal readout unit RU2 and the pixel signal readout unit RU3 have the same configuration.

[0065] The photoelectric conversion element PD is an embedded photodiode that photoelectrically converts incident light to generate charges and accumulates the generated charges. Note that in the present invention, the structure of the photoelectric conversion element PD provided in the pixel 321 is not particularly defined. For this reason, the photoelectric conversion element PD may be, for example, a PN photodiode having a structure in which a P-type semiconductor and an N-type semiconductor are joined, or a PIN photodiode having a structure in which an I-type semiconductor is sandwiched between a P-type semiconductor and an N-type semiconductor. Further, the photoelectric conversion element provided in the pixel 321 is not limited to a photodiode, and may be, for example, a photoelectric conversion element of a photogate method.

[0066] The drain gate transistor GD is a transistor for discarding the charges generated and accumulated by the photoelectric conversion element PD and not transferred to the respective pixel signal readout units RU in response to the drive signal input from the vertical scanning circuit 323. That is, the drain gate transistor GD is a transistor that resets the charges generated by the photoelectric conversion element PD and not used for measuring the distance to the subject S.

[0067] The read gate transistor G is a transistor for transferring the charges generated and accumulated by the photoelectric conversion element PD to the corresponding charge storage unit CS in response to the drive signal input from the vertical scanning circuit 323. The charges transferred by the read gate transistor G are held (accumulated) in the corresponding charge storage unit CS. Here, in the pixel signal readout unit RU1, the read gate transistor G1 has its source connected to the second terminal of the photoelectric conversion element PD, its gate connected to the signal line LTX1 that propagates the storage drive signal TX1, and its drain connected to the floating diffusion FD1 and the first terminal of the charge storage capacitor C1. Similarly, in the pixel signal readout unit RU2, the read gate transistor G2 has its source connected to the second terminal of the photoelectric conversion element PD, its gate connected to the signal line LTX2 that propagates the storage drive signal TX2, and its drain connected to the floating diffusion FD2 and the first terminal of the charge storage capacitor C2. Similarly, in the pixel signal readout section RU3, for the readout gate transistor G3, the source is connected to the second terminal of the photoelectric conversion element PD, the gate is connected to the signal line LTX3 that propagates the accumulation drive signal TX3, and the drain is connected to the floating diffusion FD3 and the first terminal of the charge storage capacitor C3. Each of the above-described accumulation drive signals TX1, TX2, and TX3 is supplied from the pixel drive circuit 326 via the signal lines LTX1, LTX2, and LTX3, respectively.

[0068] The charge storage capacitor C is a capacitor that holds (accumulates) the charge transferred by the corresponding readout gate transistor G.

[0069] The reset gate transistor RT is a transistor for discharging the charge held in the corresponding charge storage section CS in response to the drive signal input from the vertical scanning circuit 323. That is, the reset gate transistor RT is a transistor that resets the charge held in the corresponding charge storage section CS.

[0070] The source follower gate transistor SF is a transistor that amplifies the voltage signal corresponding to the amount of charge accumulated in the charge storage section CS connected to the gate terminal and outputs it to the corresponding selection gate transistor SL.

[0071] The selection gate transistor SL is a transistor that outputs the voltage signal amplified by the corresponding source follower gate transistor SF from the corresponding output terminal O in response to the drive signal input from the vertical scanning circuit 323.

[0072] With the above-described configuration, in the pixel 321, the charge generated by photoelectrically converting the incident light by the photoelectric conversion element PD is distributed to each of the three charge storage sections CS, and each voltage signal corresponding to the amount of the distributed charge is output to the pixel signal processing circuit 325.

[0073] The configuration of the pixels arranged in the distance image sensor 32 is not limited to the configuration including three pixel signal readout units RU as shown in FIG. 3. Any pixel configuration may be used as long as it includes one photoelectric conversion element PD and a plurality of pixel signal readout units RU for distributing the charges generated and accumulated by the photoelectric conversion element PD. That is, the number of pixel signal readout units RU (charge storage unit CS) provided in the pixels arranged in the distance image sensor 32 may be two, or may be four or more.

[0074] In addition, in the pixel 321 having the configuration shown in FIG. 3, an example of configuring the charge storage unit CS by a floating diffusion FD and a charge storage capacitor C is shown. However, the charge storage unit CS only needs to be configured by at least the floating diffusion FD. That is, the pixel 321 may have a configuration without each charge storage capacitor C. In this configuration, there is an effect of increasing the charge detection sensitivity (charge-voltage conversion gain CG). However, considering widening the dynamic range in distance measurement in the distance image capturing apparatus 1, a configuration capable of holding (accumulating) more charges is more advantageous. For this reason, in the pixel 321, the charge storage capacitor C is provided in the pixel signal readout unit RU, and the charge storage unit CS is configured by the floating diffusion FD and the charge storage capacitor C, so that more charges can be held (accumulated) than in the case where the charge storage unit CS is configured only by the floating diffusion FD.

[0075] In addition, in the pixel 321 having the configuration shown in FIG. 3, an example of a configuration including a drain gate transistor GD is shown. However, when it is not necessary to discard the charges accumulated (remaining) in the photoelectric conversion element PD, the pixels arranged in the distance image sensor 32 may have a configuration without the drain gate transistor GD.

[0076] Next, the driving (control) method (timing) of the pixel 321 in the distance image capturing device 1 will be described. FIG. 4 is a timing chart showing the timing for driving the pixel 321 disposed in the light receiving pixel portion 320 of the imaging element (distance image sensor 32) used in the distance image capturing device 1 according to the embodiment of the present invention. FIG. 4 shows the timing of the driving signal of the pixel 321 when causing the distance image sensor 32 to output pixel signals for one frame, together with the timing of the light pulse PO irradiated from the light source unit 2 to the subject S.

[0077] First, the driving (control) of pixel 321 during the charge accumulation period will be described, in which the charge generated and accumulated by the photoelectric conversion element PD according to the amount of received light (received light amount) is distributed to each pixel signal readout unit RU. During the charge accumulation period, the light pulse PO is irradiated onto the subject S by the light source unit 2. Then, by driving pixel 321 in synchronization with the timing of irradiating the light pulse PO, the charge corresponding to the received background light and reflected light RL is distributed to each charge accumulation unit CS. The pixel driving circuit 326 distributes and accumulates the charge to each charge accumulation unit CS provided in all the pixels 321 by so-called global shutter driving that drives all the pixels 321 arranged in the light receiving pixel unit 320 simultaneously. Note that the time when the light source device 21 emits pulsed laser light, that is, the pulse width Tw of the light pulse PO, is a very short time determined in advance, such as 10 nS. The reason is that in the distance measurement by the pulse modulation method, the maximum distance that can be measured (hereinafter referred to as the "maximum measurement distance") is determined by the pulse width Tw of the light pulse PO. When the pulse width Tw of the above-described light pulse PO is 10 nS, the maximum measurement distance is 1.5 m. Also, simply widening the pulse width Tw of the light pulse PO, that is, increasing the emission time of the laser light in the light source device 21, allows the photoelectric conversion element PD to receive more reflected light RL, but the resolution of the distance to the subject S to be measured decreases. On the other hand, when the pulse width Tw of the light pulse PO is short, the amount of charge generated by the photoelectric conversion element PD by photoelectric conversion also decreases. For this reason, in the distance image capturing device 1, the irradiation of the light pulse PO and the distribution of the charge are performed a plurality of times so that a sufficient amount of charge is accumulated in each charge accumulation unit CS during the charge accumulation period. Here, a configuration will be described in which each of the vertical scanning circuit 323 and the pixel driving circuit 326 drives (controls) the pixel 321. In the following description, the control circuit 322 outputs clock signals CK1, CK2, CK3, and CKRSTD that generate the storage drive signals TX1, TX2, TX3, and the reset drive signal RSTD to the pixel driving circuit 326, respectively. Further, the control circuit 322 outputs clock signals that generate the selection drive signals SEL1, SEL2, SEL3, and the reset signals RST1, RST2, RST3 to the vertical scanning circuit 323, respectively.

[0078] The timing chart shown in FIG. 4 shows the driving timing of the pixel 321 when the light pulse PO is irradiated and the charge is distributed to all the pixels 321 a plurality of times during the charge storage period. Note that the light pulse PO during the charge storage period in the timing chart shown in FIG. 4 is described as being irradiated when the light pulse PO is at the "H (High)" level (the light source device 21 emits laser light) and the irradiation of the light pulse PO is stopped (the light source device 21 is turned off) when it is at the "L (Low)" level. Further, the timing chart shown in FIG. 4 is described as starting from a state where all the pixels 321 are reset, that is, no charge is stored in the photoelectric conversion element PD and the charge storage section CS.

[0079] The timing chart shown in FIG. 4 shows the driving timing of the pixel 321 when the light pulse PO is irradiated and the charge is distributed to all the pixels 321 a plurality of times during the charge storage period. As the signal levels shown in FIG. 4, the higher voltage value of the binary voltage pulse is the "H" level, and the lower voltage value is the "L" level. In the following description, times tA1 to tA5 are storage cycles for distributing charge, and a plurality of storage cycles are repeated during the charge storage period. Further, for example, the time widths between times tA1, tA2, tA3, and tA4, that is, the pulse widths of the light pulse PO, the storage drive signals TX1, TX2, and TX3 are the same Tw.

[0080] During the charge accumulation period, first, the pixel driving circuit 326 transfers, via the readout gate transistor G1, the charge generated by the photoelectric conversion of the photoelectric conversion element PD in response to the background light before the irradiation of the optical pulse PO, which is the same time as the pulse width Tw during which the light source unit 2 irradiates the optical pulse PO, from the time tA1 that is the same time before, and accumulates the charge in the charge accumulation unit CS1.

[0081] Thereafter, the pixel driving circuit 326 transfers, via the readout gate transistor G2, the charge generated in response to the light currently being photoelectrically converted by the photoelectric conversion element PD from the time tA2 that is the same time as the timing at which the light source unit 2 irradiates the optical pulse PO, and accumulates the charge in the charge accumulation unit CS2. Here, the charge accumulated in the charge accumulation unit CS2 is the charge corresponding to the reflected light RL reflected by the subject S within the pulse width Tw during which the optical pulse PO is irradiated. This charge includes, in addition to the charge corresponding to the background light, the charge corresponding to the reflected light RL that has entered with a small delay time proportional to the distance (absolute distance) to the subject S. More specifically, for example, when the subject S is present at a close position, since the irradiated optical pulse PO is reflected by the subject S in a short time and returns as the reflected light RL, the charge accumulation unit CS2 contains more charge corresponding to the reflected light RL reflected by the subject S present at the close position.

[0082] Thereafter, at the same time tA3 as the timing when the light source unit 2 stops irradiating the optical pulse PO, the pixel drive circuit 326 transfers and accumulates the charge generated by the photoelectric conversion element PD in response to the light currently being photoelectrically converted, via the readout gate transistor G3, to the charge storage unit CS3. Here, the charge accumulated in the charge storage unit CS3 is the charge corresponding to the reflected light RL reflected by the subject S outside the time of the pulse width Tw during which the optical pulse PO is irradiated. This charge includes, in addition to the charge corresponding to the background light, the charge corresponding to the reflected light RL that has entered with a large delay time proportional to the distance (absolute distance) to the subject S. More specifically, for example, when the subject S is present at a distant position, since the irradiated optical pulse PO takes a longer time to be reflected by the subject S and return as the reflected light RL, the charge storage unit CS3 contains more charge corresponding to the reflected light RL reflected by the subject S present at the distant position.

[0083] Thereafter, at the time tA4 after the same time as the pulse width Tw during which the light source unit 2 irradiates the optical pulse PO has elapsed, the pixel drive circuit 326 causes the charge generated by the photoelectric conversion element PD in response to the light currently being photoelectrically converted, that is, the charge not used for measuring the distance to the subject S, to be discarded via the drain gate transistor GD. In other words, the photoelectric conversion element PD is reset.

[0084] Thereafter, at the previous time tA5, which is the same time as the pulse width Tw during which the light source unit 2 irradiates the optical pulse PO next, the pixel drive circuit 326 releases the reset of the photoelectric conversion element PD. Then, in the same timing as from the time tA1, the pixel drive circuit 326 transfers and accumulates the charge generated by the photoelectric conversion element PD when it is next photoelectrically converted, that is, the charge corresponding to the background light before the next optical pulse PO is irradiated, via the readout gate transistor G1, to the charge storage unit CS1.

[0085] Thereafter, the pixel driving circuit 326 repeats the driving of the pixels 321 (hereinafter referred to as "charge distribution driving") similar to that from time tA1 to time tA5. As a result, during the charge accumulation period, the charge amount corresponding to the number of times the charge distribution driving is repeated is accumulated and held in each charge accumulation unit CS provided in all the pixels 321. Note that the maximum number of times the charge distribution driving is repeated during the charge accumulation period is determined by the period in which the distance image sensor 32 outputs (acquires) the pixel signals for one frame. More specifically, it is the number of times obtained by dividing the time obtained by subtracting the pixel signal readout period from the time in which the distance image sensor 32 acquires the pixel signals for one frame by the time in which the light source device 21 emits pulsed laser light, that is, the pulse period time To of the optical pulse PO. Note that in the distance image sensor 32, the larger the number of times of the charge distribution driving, the larger the charge amount accumulated (integrated) in each charge accumulation unit CS, and the higher the sensitivity. Thereby, in the distance image sensor 32, the resolution of the distance to the subject S to be measured can be improved.

[0086] Subsequently, after the charge accumulation period ends, the driving (control) of the pixels 321 in the pixel signal readout period in which voltage signals corresponding to the charge amounts distributed to the respective charge accumulation units CS provided in the respective pixel signal readout units RU are sequentially output for each row of the pixels 321 arranged in the light receiving pixel unit 320 will be described. In the pixel signal readout period, by so-called rolling driving that drives the pixels 321 arranged in the light receiving pixel unit 320 row by row, voltage signals corresponding to the charge amounts accumulated (integrated) and held in the charge accumulation units CS provided in the pixels 321 arranged in the corresponding rows are output to the pixel signal processing circuit 325 row by row.

[0087] As described above, in the distance image sensor 32, for the voltage signals output by the respective pixels 321, the pixel signal processing circuit 325 performs predetermined signal processing such as noise suppression processing and A / D conversion processing. Here, the correlated double sampling (CDS) processing performed by the pixel signal processing circuit 325 as noise suppression processing is a process of taking the difference between a voltage signal (hereinafter referred to as "distance pixel voltage signal PS") corresponding to the amount of charge accumulated (integrated) and held in the charge accumulation unit CS and a voltage signal (hereinafter referred to as "reset voltage signal PR") corresponding to the amount of charge in a state where the charge accumulation unit CS is reset (reset state). Therefore, in the pixel signal readout period, the respective voltage signals of the distance pixel voltage signal PS and the reset voltage signal PR corresponding to the respective charge accumulation units CS provided in the respective pixels 321 are output to the pixel signal processing circuit 325 in row order.

[0088] In the pixel signal readout period of the timing chart shown in FIG. 4, when a plurality of pixels 321 are arranged in the horizontal direction (row direction) of the light receiving pixel unit 320 by y rows (y is an integer of 1 or more) and in the vertical direction (column direction) by x columns (x is an integer of 1 or more), the respective voltage signals of the distance pixel voltage signal PS(i) and the reset voltage signal PR(i) are output from each of the pixels 321(i) arranged in the i-th row (1 ≤ i ≤ y) of the light receiving pixel unit 320. The driving timing of the pixel 321 in this case is shown. In the timing chart shown in FIG. 4, the respective voltage signals are output in the order of the charge accumulation units CS1(i), CS2(i), and CS3(i) provided in each pixel 321(i).

[0089] In the pixel signal readout period, first, in the period from time tR1 to time tR2, the vertical scanning circuit 323 causes the distance pixel voltage signal PS1(i) to be output from the output terminal O1(i) to the pixel signal processing circuit 325 via the vertical signal line. As a result, the pixel signal processing circuit 325 temporarily holds the distance pixel voltage signal PS1(i) output from the pixel signal readout unit RU1(i) via the vertical signal line.

[0090] Thereafter, during the period from time tR3 to time tR4, the vertical scanning circuit 323 causes the reset voltage signal PR1(i) to be output from the output terminal O1(i) to the pixel signal processing circuit 325 via the vertical signal line. As a result, the pixel signal processing circuit 325 takes the difference between the distance pixel voltage signal PS1(i) that it has once held and the reset voltage signal PR1(i) output from the pixel signal reading unit RU1(i) via the vertical signal line, that is, suppresses the noise included in the voltage signal corresponding to the amount of charge accumulated (integrated) and held in the charge accumulation unit CS1(i).

[0091] Thereafter, during the period from time tR4 to time tR7, the vertical scanning circuit 323 causes the distance pixel voltage signal PS2(i) and the reset voltage signal PR2(i) to be output from the output terminal O2(i) to the pixel signal processing circuit 325 via the vertical signal line, in the same manner as during the period from time tR1 to time tR4. Further, during the period from time tR7 to time tR10, the vertical scanning circuit 323 also causes the distance pixel voltage signal PS3(i) and the reset voltage signal PR3(i) to be output from the output terminal O3(i) to the pixel signal processing circuit 325 via the vertical signal line, in the same manner as during the period from time tR1 to time tR4.

[0092] Thereafter, the vertical scanning circuit 323 sequentially performs the driving of the pixels 321 (hereinafter referred to as "pixel signal reading driving") similar to that from time tR1 to time tR10 on each of the pixels 321 arranged in the other rows of the light-receiving pixel unit 320 (for example, each of the pixels 321 arranged in the (i + 1)-th row), and sequentially outputs the respective voltage signals from all the pixels 321 arranged in the light-receiving pixel unit 320.

[0093] By such a driving (control) method (timing), the pixel driving circuit 326 distributes the charges generated and accumulated by the photoelectric conversion element PD in each of the pixels 321 arranged in the light-receiving pixel unit 320 to the respective pixel signal reading units RU a plurality of times. Also, the vertical scanning circuit 323 sequentially outputs, via the vertical signal line, voltage signals corresponding to the amounts of charge accumulated (integrated) in the charge accumulation units CS provided in the pixel signal reading units RU to the pixel signal processing circuit 325.

[0094] Note that the pixel signal processing circuit 325 performs A / D conversion processing on each of the voltage signals with noise suppressed. Then, the horizontal scanning circuit 324 sequentially outputs the voltage signals (digitized voltage signals) of each row after the pixel signal processing circuit 325 performs A / D conversion processing, via the horizontal signal lines in the order of the columns of the light-receiving pixel unit 320. As a result, the distance image sensor 32 outputs the pixel signals (pixel signals VQ1, VQ2, and VQ3 corresponding to the amounts of charge Q1, Q2, and Q3 of the charge storage units CS1, CS2, and CS3, respectively) of all the pixels 321 for one frame to the outside. As a result, in the distance image capturing device 1, the pixel signals (pixel signals VQ1, VQ2, and VQ3) for one frame of the captured image are output to the distance calculation unit 42 in the so-called raster order.

[0095] Note that, as can be seen from the driving (control) timing of the pixel 321 shown in FIG. 4, each of the pixel signals for one frame includes three voltage signals corresponding to each of the three pixel signal readout units RU (charge storage unit CS) provided in the corresponding pixel 321. The distance calculation unit 42 calculates the distance to the subject S for each pixel signal, that is, for each pixel 321, based on the pixel signals for one frame output from the distance image sensor 32.

[0096] Here, a method for calculating the distance between the distance image capturing device 1 and the subject S in the distance calculation unit 42 will be described. Here, the amount of charge corresponding to the background light before the light pulse PO allocated to the charge storage unit CS1 of the pixel signal readout unit RU1 is irradiated is defined as charge Q1. Also, the amount of charge corresponding to the reflected light RL incident with a small delay time with respect to the background light allocated to the charge storage unit CS2 of the pixel signal readout unit RU2 is defined as charge Q2. Further, the amount of charge corresponding to the reflected light RL incident with a large delay time with respect to the background light allocated to the charge storage unit CS3 of the pixel signal readout unit RU3 is defined as charge Q3. The distance calculation unit 42 obtains the distance L to the subject S for each pixel 321 by the formula (1) already described.

[0097] As described above, the distance image capturing device 1 obtains the distance L between itself and the subject S for each pixel 321 arranged in the light-receiving pixel portion 320 of the distance image sensor 32.

[0098] Note that, as described above, the configuration of the pixels arranged in a grid pattern in the distance image sensor 32 is not limited to the configuration including the three pixel signal readout units RU1, RU2, and RU3 as shown in FIG. 3, and any pixel 321 having a configuration including one photoelectric conversion element PD and two or more pixel signal readout units RU for distributing the charges generated and accumulated by the photoelectric conversion element PD may be used. In this case, even in a distance image sensor in which pixels having different configurations with different numbers of pixel signal readout units RU are arranged, the driving (control) method (timing) of the pixels can be easily realized by considering it in the same way as the driving (control) method (timing) of the pixels 321 in the distance image capturing device 1 shown in FIG. 4. More specifically, by repeating the charge distribution driving for the pixels in a cycle maintaining a phase relationship such that the phases of the driving signals input to the readout gate transistors G and drain gate transistors GD provided in each pixel signal readout unit RU do not overlap with each other, similar to the distance image sensor 32, charges corresponding to the corresponding light can be accumulated (integrated) in the charge accumulation portions CS provided in each pixel signal readout unit RU. Then, by sequentially outputting respective voltage signals from all the pixels by pixel signal readout driving, similar to the distance image sensor 32, pixel signals for one frame can be output to the outside of the distance image sensor. Thereby, the distance calculation unit 42 can similarly obtain the distance L between the distance image capturing device 1 and the subject S for each pixel signal (for each pixel).

[0099] In this embodiment, in order to remove the voltage component of the non-controlled charge, as correction parameters in advance, the voltage corresponding to the charge amount of the non-controlled charge is used as a reference voltage by the following process, and is obtained in each of the charge storage units CS1, CS2, and CS3, and written and stored in the correction parameter storage unit 43. Similarly, the voltage corresponding to the charge amount generated by the background light is used as a reference background light voltage, and is obtained in each of the charge storage units CS1, CS2, and CS3, and written and stored in the correction parameter storage unit 43.

[0100] In this embodiment, the data of the reference voltage corresponding to the non-controlled charge generated by the background light and the reference background light voltage corresponding to the background light charge generated by the background light are obtained in advance, for example, at the time of shipment or startup of the distance image capturing device 1, that is, before capturing a distance image, and written and stored in the correction parameter storage unit 43. FIG. 5 is a timing chart for explaining the process of obtaining each of the reference background light voltage and the reference voltage in the first embodiment of the present invention. For example, when the distance image capturing device 1 is activated, the distance calculation unit 42 causes the timing control unit 41 to perform timing output processing in each of the reference background light voltage acquisition frame and the reference voltage acquisition frame.

[0101] FIG. 5(A) shows the process of obtaining, in the reference background light voltage acquisition frame, the reference background light voltage VA1 corresponding to the charge amount generated by the background light accumulated in the charge storage unit CS1, the reference background light voltage VA2 corresponding to the charge amount generated by the background light accumulated in the charge storage unit CS2, and the reference background light voltage VA3 corresponding to the charge amount generated by the background light accumulated in the charge storage unit CS3. In FIG. 5(A), the distance image sensor 32 supplies each of the accumulation drive signals TX1, TX2, and TX3 to the readout gate transistors G1, G2, and G3 with pulse widths Tw1, Tw2, and Tw3, respectively, by the same processing as the operation explanation of the frame period in FIG. 4. As a result, the charges collected by the photodiode PD due to incident light are distributed to the charge accumulation units CS1, CS2, and CS3 to accumulate background light charges. At this time, unlike the processing in FIG. 4, the timing control unit 41 controls the light source device 21 not to emit the optical pulse PO. Therefore, the incident light input to the photodiode PD of each pixel 321 of the distance image sensor 32, more precisely, the incident light input to the silicon region of the pixel including the photodiode PD, is only the background light in the imaging environment.

[0102] As a result, during the accumulation period in the reference background light voltage acquisition frame, charges generated by being incident at different angles for each of the pixels 321 are accumulated in each of the charge accumulation units CS1, CS2, and CS3 in each pixel circuit. Then, in the same manner as the description of FIG. 4, a readout process is performed, and from the pixel signal processing circuit 325, a reference background light voltage VA1 corresponding to the amount of charge of the background light charge (i.e., the reference background light charge) due only to the background light accumulated in the charge accumulation unit CS1, a reference background light voltage VA2 corresponding to the amount of charge of the background light charge accumulated in the charge accumulation unit CS2, and a reference background light voltage VA3 corresponding to the amount of charge of the background light charge accumulated in the charge accumulation unit CS3 are output as pixel signals. The distance calculation unit 42 writes each of the acquired reference background light voltages VA1, VA2, and VA3 into the correction parameter storage unit 43 together with identification information for identifying each pixel 321 in the light receiving pixel unit 320 and stores them. Alternatively, the distance calculation unit 42 may be configured to write each of the reference background light voltages VA1, VA2, and VA3 acquired in order in the same order as the order in which the pixel signals are read out from the pixel signal processing circuit 325 into the correction parameter storage unit 43 and store them. Here, in the same pixel 321, among the charges generated by photoelectric conversion by the incident light incident on the silicon region of the pixel including the photodiode PD, the charges collected by the photodiode PD and distributed and accumulated in the charge accumulation units CS1, CS2, and CS3 respectively under the control of the respective read gate transistors G1, G2, and G3 are the same. The reason why each of the reference background light voltages VA1, VA2, and VA3 is different is that the non-control charges flowing into and accumulated in the charge accumulation units CS1, CS2, and CS3 are different.

[0103] FIG. 5(B) shows a process of acquiring, in the reference voltage acquisition frame, a reference reference voltage VB1 corresponding to the amount of charge of the non-control charge that is generated by the background light and flows into and accumulates in the charge accumulation unit CS1 without passing through the read gate transistor G1, a reference reference voltage VB2 corresponding to the amount of charge of the non-control charge that flows into and accumulates in the charge accumulation unit CS2 without passing through the read gate transistor G2, and a reference reference voltage VB3 corresponding to the amount of charge of the non-control charge that flows into and accumulates in the charge accumulation unit CS3 without passing through the read gate transistor G3. In FIG. 5(B), the distance image sensor 32 accumulates and reads charges through the same process as the operation explanation of the frame period in FIG. 4. On the other hand, the timing control unit 41 controls the light source device 21 not to emit the optical pulse PO, as in the case of FIG. 5(A). Different from the case of FIG. 5(A), the timing control unit 41 controls the control circuit 322 not to output each of the accumulation drive signals TX1, TX2, and TX3, and does not distribute charges (i.e., control charges to be accumulated in the charge accumulation units) to the charge accumulation units CS1, CS2, and CS3 respectively. Also, the timing control unit 41 controls the control circuit 322 to output a reset drive signal RSTD that turns on the drain gate transistor GD at the timing when charge distribution is performed to each of the charge accumulation units CS1, CS2, and CS3, and discards the charges corresponding to the control charges that are collected by the photodiode PD and distributed under the control of the respective read gate transistors G1, G2, and G3 and accumulated in the respective charge accumulation units. However, the period during which the read gate transistors G1, G2, and G3 perform distribution is an extremely short time of about 30 ns, and since the charges collected by the photodiode PD are small, in most cases, since the photodiode PD can hold them, the drain gate transistor GD may be in an off state during the period when each read gate transistor G performs distribution, as in FIG. 5(A). In that case, when the drain gate transistor GD is turned on immediately after the distribution period, the charges held by the photodiode PD are discarded to the drain.

[0104] For this reason, each of the read gate transistors G1, G2, and G3 remains off, and the control charges due to the background light are not accumulated in the charge accumulation units CS1, CS2, and CS3 respectively. That is, since the control charges due to the background light are not propagated through each of the read gate transistors G1, G2, and G3, only the non-control charges QB1, QB2, and QB3 that flow in without passing through each of the read gate transistors G1, G2, and G3 are accumulated in each of the charge accumulation units CS1, CS2, and CS3. Then, also in FIG. 5(B), similar to FIG. 5(A), a readout process is performed. As pixel signals from the pixel signal processing circuit 325, a reference reference voltage VB1 corresponding to the charge amount of the non-controlled charge QB1 accumulated in the charge accumulation unit CS1, a reference reference voltage VB2 corresponding to the charge amount of the non-controlled charge QB2 accumulated in the charge accumulation unit CS2, and a reference reference voltage VB3 corresponding to the charge amount of the non-controlled charge accumulated in the charge accumulation unit CS3 are output as pixel signals. The distance calculation unit 42 writes and stores each of the acquired reference reference voltages VB1, VB2, and VB3 in the correction parameter storage unit 43 together with identification information for identifying each of the pixels 321 in the light receiving pixel unit 320. Alternatively, the distance calculation unit 42 may be configured to write and store each of the sequentially acquired reference reference voltages VB1, VB2, and VB3 in the correction parameter storage unit 43 in the same order as the order in which the pixel signals are read from the pixel signal processing circuit 325.

[0105] Then, in the frame period for measuring the distance between the distance image sensor 32 and the subject S, as already described, the pixel signal processing circuit 325 outputs, as pixel signals (digital values), the pixel signals VQ1, VQ2, and VQ3 corresponding to the charge accumulation units CS1, CS2, and CS3, respectively, for each pixel 321 corresponding to one frame to the distance calculation unit 42. At this time, the distance calculation unit 42 corresponds to each of the pixel signals VQ1, VQ2, and VQ3 for each pixel 321 supplied from the pixel signal processing circuit 325, and sequentially reads out the reference background light voltages VA1, VA2, and VA3 corresponding to this pixel 321 and the reference reference voltages VB1, VB2, and VB3 from the correction parameter storage unit 43. Then, the distance calculation unit 42 divides the pixel signal VQ1 by the reference background light voltage VA1 (VQ1 / VA1) to calculate the adjustment ratio β. This adjustment ratio β is a value indicating the intensity ratio of the background light at the time of obtaining the correction parameter and at the time of calculating the distance. Also for VQ2’ / VA2 and VQ3’ / VA3, it becomes the same value in the linearity region (non-saturation region) of the signal intensity. Here, the corrected pixel signal VQ2’ is a voltage that does not include the component of the charge corresponding to the reflected light RL among the charge amounts corresponding to the background light distributed to the charge accumulation section CS2 and the reflected light RL that has entered with a small delay time. Similarly, the corrected pixel signal VQ3’ is a voltage that does not include the component of the charge corresponding to the reflected light RL among the charge amounts corresponding to the background light distributed to the charge accumulation section CS3 and the reflected light RL that has entered with a large delay time.

[0106] The distance calculation unit 42 calculates the adjustment voltages VP1, VP2, and VP3 corresponding to the pixel signals VQ1, VQ2, and VQ3 respectively, using the calculated adjustment ratio β and each of the reference voltages VB1, VB2, and VB3 according to the following equations (2), (3), and (4). Here, each of the adjustment voltages VP1, VP2, and VP3 is the voltage generated by the accumulation of non-controlled charge in the charge accumulation sections CS1, CS2, and CS3 respectively during distance measurement. VP1 = β × VB1 = (VQ1 / VA1) × VB1 …(2) VP2 = β × VB2 = (VQ1 / VA1) × VB2 …(3) VP3 = β × VB3 = (VQ1 / VA1) × VB3 …(4) According to the above equations (2), (3), and (4), it is possible to obtain each of the adjustment voltages VP1, VP2, and VP3 corresponding to the adjustment ratio β, that is, based on each of the reference voltages VB1, VB2, and VB3 corresponding to the intensity of the background light at the time of calculating the distance L. Then, the distance calculation unit 42 uses each of the obtained adjustment voltages VP1, VP2, and VP3 to calculate the corrected pixel signals VQ1’, VQ2’, and VQ3 by removing the voltage components generated by the accumulation of non-controlled charge in the charge accumulation section CS during distance measurement from each of the pixel signals VQ1, VQ2, and VQ3 according to the following equations (5), (6), and (7). VQ1’ = VQ1 - VP1 …(5) VQ2’ = VQ2 - VP2 …(6) VQ3’ = VQ3 - VP3 …(7)

[0107] The distance calculation unit 42 calculates the distance L between the distance image sensor 32 and the subject S by the following equation (8) corresponding to equation (1) already described, using each of the obtained corrected pixel signals VQ1’, VQ2’, and VQ3’. L = [(VQ3’ - VQ1’) / (VQ2’ + VQ3’ - 2VQ1’)] × Dm …(8) In the above equation (8), Dm is (c / 2)Tw. As described above, the distance image capturing device 1 obtains the distance L between the distance image sensor 32 and the subject S for each pixel 321 disposed in the light receiving pixel unit 320 of the distance image sensor 32. According to the present embodiment, by using the corrected pixel signals VQ1’, VQ2’, and VQ3 obtained by correcting each of the pixel signals VQ1, VQ2, and VQ3 with the adjustment voltages VP1, VP2, and VP3 respectively, the equation (1) for obtaining the distance conventionally used can be utilized as it is.

[0108] FIG. 6 is a timing chart for explaining the period during which non-controlled charges are accumulated. FIG. 6 is the same timing chart as shown in FIG. 4, and is shown over two frame periods. For each frame period, as described in FIG. 4, there is a charge accumulation period (Integuration) in which a light pulse PO is irradiated to accumulate charges in the charge accumulation section CS, and a pixel signal readout period (Read) in which charges are read out from the charge accumulation section CS. Here, the controlled charges collected by the photodiode PD and distributed under the control of the respective read gate transistors G1, G2, and G3 are accumulated in each of the charge accumulation sections CS1, CS2, and CS3 only during the charge accumulation period (Integuration). After the charges are read out from the charge accumulation section CS, the charge accumulation section CS is once discarded and reset by the reset gate transistor RT, but the charges are not discarded until the next charge readout time. That is, since the charge discard is performed only once per frame (Integuration + Read), the non-controlled charges are accumulated in the charge accumulation section CS for approximately one frame period. That is, the period during which the non-controlled charges are accumulated in the charge accumulation section CS is longer than the period during which the controlled charges are accumulated in the charge accumulation section CS. Therefore, in the prior art, when the area of the pixel 321 is reduced, the ratio of the amount of non-controlled charges accumulated in the charge accumulation section CS without passing through the read gate transistor G to the amount of controlled charges collected by the photodiode PD and distributed to the charge accumulation section CS through the read gate transistor G increases, and the accuracy of the required distance L decreases. Furthermore, as the area of the pixel 321 is reduced, the accumulation ratio of the non-controlled charges increases step by step, so the accuracy of the required distance L gradually decreases.

[0109] However, according to the present embodiment, even if the ratio of the amount of non-controlled charge accumulated in the charge accumulation unit CS without passing through the read gate transistor G to the amount of controlled charge passing through the read gate transistor G increases, or even if the area of the pixel 321 is reduced, it is possible to eliminate the influence of each non-controlled charge included in each of the charges accumulated in the charge accumulation unit CS, whose charge amount varies depending on the incident angle of the incident light, and the distance L between the subject S and itself can be obtained with the same or higher accuracy. Further, according to the present embodiment, after the lens 31 is attached and the distance image capturing device 1 is assembled, the reference background photovoltages VA1, VA2, and VA3 and the reference reference voltages VB1, VB2, and VB3 can be measured without using special calibration equipment, and the pixel signals VQ1, VQ2, and VQ3 can be easily corrected in accordance with the individual characteristics of the attached lens 31 and the differences in the relative positions between the attached lens 31 and the distance image sensor 32.

[0110] Also, in the present embodiment, when the lens 31 has a plurality of different characteristics such as having a plurality of F values and changing the incident angle of the incident light to the distance image sensor 32, for each characteristic, the reference background photovoltages VA1, VA2, and VA3 and the reference reference voltages VB1, VB2, and VB3 are obtained and written and stored in the correction parameter storage unit 43 in advance corresponding to each of the characteristics, and the pixel signals VQ1, VQ2, and VQ3 corresponding to the characteristics may be corrected. In this case, the distance calculation unit 42 reads correction parameters such as the reference background photovoltages VA1, VA2, and VA3 and the reference reference voltages VB1, VB2, and VB3 corresponding to the characteristics of the lens used during the charge accumulation period in the frame period from the correction parameter storage unit 43, and corrects each of the pixel signals VQ1, VQ2, and VQ3 with the read correction parameters.

[0111] In the present embodiment, the reference background photovoltages VA1, VA2, and VA3 corresponding to each of all the pixels 321 in the light-receiving pixel unit 320 and the reference reference voltages VB1, VB2, and VB3 are stored in the correction parameter storage unit 43. However, since incident light is incident on each of the pixels 321 through the lens 31 provided in the front part of the light-receiving pixel unit 320 with respect to the subject S, each of the pixels 321 is divided into groups so that the adjustment ratio β falls within a predetermined difference range corresponding to the characteristics of the lens 31. The median values of the reference background photovoltages VA1, VA2, and VA3 and the reference reference voltages VB1, VB2, and VB3 in the pixels 321 within each group are used as the reference background photovoltages VA1, VA2, and VA3 and the reference reference voltages VB1, VB2, and VB3 for all the pixels 321 within the group, and the storage capacity of the correction parameter storage unit 43 may be reduced.

[0112] FIG. 7 is a diagram showing the tendency of the change in the incident angle of incident light in the characteristics of the lens 31 incident on each pixel. As shown in FIG. 7, since the incident angle of the incident light gradually changes continuously in a concentric circle shape (concentric circle E) centered on the area centroid CO of the chip of the distance image sensor 32, it is possible to adopt a configuration in which each group of the predetermined pixels 321 having the same incident angle is grouped as described above. As another configuration, correction parameter values of the reference background photovoltages VA1, VA2, and VA3 and the reference reference voltages VB1, VB2, and VB3 in the pixels 321 at a position of a predetermined radius of the concentric circle E centered on the area centroid CO may be stored in the correction parameter storage unit 43. In this case, a complement function for complementing the correction parameter values of the pixels 321 in the region sandwiched by the concentric circle E with the correction parameter values of the concentric circle E at the predetermined radius is written and stored in the correction parameter storage unit 43 in advance for each region. Then, the distance calculation unit 42 reads out the correction parameter values and the complement function in the concentric circle E as necessary, and generates the correction parameter values of the pixels 321 in the region sandwiched by the concentric circle E by the complement function using the correction parameter values of the pixels 321 corresponding to the concentric circle E sandwiching the region. In this embodiment, the characteristics of the lens 31 were described by way of an example in which the incident angle of the incident light gradually changes continuously in a concentric circle shape (concentric circle E). However, depending on the characteristics of the lens to be used, it is not necessarily required to change in a concentric circle shape, and it may be changed in correspondence with the characteristics of the incident angle of the incident light that the lens has. Also, in this embodiment, an example in which the circuit in the distance image sensor 32 is formed by an n-channel type transistor was shown. However, by changing the polarity of the semiconductor, it may be formed by a p-channel type transistor.

[0113] <Second Embodiment> The distance image capturing device according to the second embodiment has the same configuration as the first embodiment shown in FIG. 1. However, while the pixel 321 in the first embodiment has three pixel signal readout units RU1, RU2, and RU3, in this embodiment, it has four pixel signal readout units RU1, RU2, RU3, and RU4 (not shown). Each of the pixel signal readout units RU1, RU2, RU3, and RU4 includes charge storage units CS1, CS2, CS3, CS4 (not shown), respectively. In the case of the configuration of this embodiment, the charge storage unit CS1, similar to the first embodiment, accumulates the background light charges collected by the photodiode PD by the incident light due to only the background light at the distribution time Tw1 (the same time width as the pulse width Tw, and the same applies to Tw2, Tw3, and Tw4 shown hereinafter). Each of the distribution times Tw1, Tw2, Tw3, and Tw4 (not shown) is the pulse width of the storage drive signal TX1 applied to the readout gate transistor G1, the storage drive signal TX2 applied to the readout gate transistor G2, the storage drive signal TX3 applied to the readout gate transistor G3, and the storage drive signal TX4 (not shown) applied to the readout gate transistor G4 (not shown). Each of Tw1, Tw2, Tw3, and Tw4 is the same as the pulse width Tw of the optical pulse PO. In the charge storage section CS1, similar to the timing chart of FIG. 4, the background light charges collected by the photodiode PD are distributed at the distribution time Tw1 before the light pulse PO is irradiated. In each of the charge storage sections CS2, CS3, and CS4, similar to the timing chart of FIG. 4, the background light charges collected by the photodiode PD and the charges corresponding to the reflected light RL are distributed at the distribution times Tw2, Tw3, and Tw4, respectively, when the light pulse PO is irradiated, at the next distribution time Tw3, and at the further next distribution time Tw4.

[0114] In the present embodiment, similar to the first embodiment, each data of the reference reference voltages VB1, VB2, VB3, and VB4 corresponding to the non-controlled charges generated by the background light and each data of the reference background light voltages VA1, VA2, VA3, and VA4 corresponding to the background light charges generated by the background light are acquired in advance, for example, at the time of shipment or startup of the distance image capturing device 1, that is, before capturing a distance image, and written and stored in the correction parameter storage unit 43. FIG. 8 is a timing chart for explaining the process of acquiring each of the reference background light voltage and the reference reference voltage in the second embodiment of the present invention. For example, when the distance image capturing device 1 is activated, the distance calculation unit 42 causes the timing control unit 41 to perform the process of acquiring the reference background light voltage in the reference background light voltage acquisition frame and the process of acquiring the reference reference voltage in the reference reference voltage acquisition frame. FIG. 8(A) shows the process of acquiring the reference background light voltage VA4 accumulated in the charge storage section CS4 together with each of the reference background light voltages VA1, VA2, and VA3, similar to FIG. 5(A), in the reference background light voltage acquisition frame. In FIG. 8(A), the distance image sensor 32 accumulates charges in the charge storage section CS and reads out charges from the charge storage section CS. When accumulating these charges, the timing control unit 41 controls the light source device 21 not to emit the light pulse PO. Therefore, the incident light input to each pixel 321 of the distance image sensor 32 is only the background light in the imaging environment.

[0115] As a result, during the charge accumulation period in the reference background photovoltage acquisition frame, only the incident light of the background light with the incident angle changed to different angles by the lens 31 is incident on each of the charge accumulation parts CS1, CS2, CS3, and CS4 in each pixel 321. Then, in the same way as the description of Fig. 5(A), a readout process is performed, and from the pixel signal processing circuit 325, as pixel signals, a reference background photovoltage VA1 corresponding to the amount of charge of the background photocurrent (i.e., the reference background photocurrent) due to only the background light accumulated in the charge accumulation part CS1, a reference background photovoltage VA2 corresponding to the amount of charge of the background photocurrent accumulated in the charge accumulation part CS2, a reference background photovoltage VA3 corresponding to the amount of charge of the background photocurrent accumulated in the charge accumulation part CS3, and a reference background photovoltage VA4 corresponding to the amount of charge of the background photocurrent accumulated in the charge accumulation part CS4 are output as pixel signals. The distance calculation unit 42 writes each of the acquired reference background photovoltages VA1, VA2, VA3, and VA4 together with the identification information for identifying each pixel 321 in the light receiving pixel part 320 into the correction parameter storage unit 43 for storage. Alternatively, the distance calculation unit 42 may be configured to write each of the sequentially acquired reference background photovoltages VA1, VA2, VA3, and VA4 into the correction parameter storage unit 43 for storage in the same order as the order in which the pixel signals are read out from the pixel signal processing circuit 325.

[0116] Fig. 8(B) shows a process of acquiring, in the reference reference voltage acquisition frame, each of the reference reference voltages VB1, VB2, VB3 and a reference reference voltage VB4 corresponding to the amount of charge of the non-controlled charge flowing into and accumulated in the charge accumulation part CS4 along with the background light, in the same way as Fig. 5(B). In Fig. 8(B), the distance image sensor 32 performs charge accumulation and readout by the same process as the operation description of the frame period in Fig. 4. On the other hand, the timing control unit 41 controls the light source device 21 not to emit the optical pulse PO, and controls the control circuit 322 not to output each of the accumulation drive signals TX1, TX2, TX3, and TX4, and does not perform charge distribution to each of the charge accumulation parts CS1, CS2, CS3, and CS4. Further, the timing control unit 41 controls the control circuit 322 to output a reset drive signal RSTD that turns on the drain-gate transistor GD at a timing when charge distribution is performed to each of the charge storage units CS1, CS2, CS3, and CS4, thereby discharging the charges. However, since the period during which the charge distribution is performed by each of the read gate transistors G1, G2, G3, and G4 is an extremely short time of about 40 ns, the charges collected by the photodiode PD are few, and in most cases, since they can be held by the photodiode PD, it is also possible to adopt a configuration in which the drain-gate transistor GD is turned off during the period of performing the distribution as shown in FIG. 8(A). In that case, the charges held by the photodiode PD when the drain-gate transistor GD is turned on immediately after the distribution period are discharged to the drain.

[0117] As a result, each of the read gate transistors G1, G2, G3, and G4 remains off, and the charges due to the background light are not propagated to each of the charge storage units CS1, CS2, CS3, and CS4 through each of the read gate transistors G1, G2, G3, and G4. For this reason, only the non-control charges QB1, QB2, QB3, and QB4 that flow into each of the charge storage units CS1, CS2, CS3, and CS4 without passing through each of the read gate transistors G1, G2, G3, and G4 are accumulated. Then, a read process is performed in the same manner as in FIG. 8(A), and as a pixel signal, a reference voltage VB4 corresponding to the amount of the non-control charges accumulated in the charge storage unit CS4 is output from the pixel signal processing circuit 325 as a pixel signal together with the reference voltages VB1, VB2, and VB3. The distance calculation unit 42 writes and stores each of the acquired reference voltages VB1, VB2, VB3, and VB4 in the correction parameter storage unit 43 together with the identification information for identifying each of the pixels 321 in the light receiving pixel unit 320. Alternatively, the distance calculation unit 42 may be configured to write and store each of the sequentially acquired reference voltages VB1, VB2, VB3, and VB4 in the correction parameter storage unit 43 in the same order as the order in which the pixel signals are read from the pixel signal processing circuit 325.

[0118] Then, in the frame period for measuring the distance between the distance image sensor 32 and the subject S, as already described, the pixel signal processing circuit 325 outputs, as pixel signals (digital), the pixel signals VQ1, VQ2, VQ3, and VQ4 corresponding to the charge storage units CS1, CS2, CS3, and CS4, respectively, for each pixel 321 corresponding to one frame, to the distance calculation unit 42. At this time, the distance calculation unit 42 sequentially reads out the reference background photovoltages VA1, VA2, VA3, and VA4 corresponding to the read pixel 321 and the reference reference voltages VB1, VB2, VB3, and VB4 from the correction parameter storage unit 43. The distance calculation unit 42 divides the pixel signal VQ1 by the reference background photovoltage VA1 (VQ1 / VA1) to calculate the adjustment ratio β. This adjustment ratio β is a value indicating the intensity ratio of the background light at the time of obtaining the correction parameter and the time of calculating the distance, and VQ2' / VA2, VQ3' / VA3, and VQ4' / VA4 also have the same value in the linearity region (non-saturation region) of the signal intensity. Here, the corrected pixel signal VQ2' is a voltage that does not include the component of the charge corresponding to the reflected light RL among the charge amounts corresponding to the background light and the reflected light RL distributed to the charge storage unit CS2. Similarly, the corrected pixel signal VQ3' is a voltage that does not include the component of the charge corresponding to the reflected light RL among the charge amounts corresponding to the background light and the reflected light RL distributed to the charge storage unit CS3, and the corrected pixel signal VQ4' is a voltage that does not include the component of the charge corresponding to the reflected light RL among the charge amounts corresponding to the background light and the reflected light RL distributed to the charge storage unit CS4.

[0119] The distance calculation unit 42 uses the adjustment ratio β and the reference reference voltages VB1, VB2, VB3, and VB4 to calculate the adjustment voltages VP1, VP2, VP3, and VP4 generated at the time of distance measurement corresponding to the pixel signals VQ1, VQ2, VQ3, and VQ4, respectively, according to the following equations (9), (10), (11), and (12), which are caused by the accumulation of non-controlled charges in the charge storage unit CS. VP1 = β × VB1 = (VQ1 / VA1) × VB1 …(9) VP2 = β × VB2 = (VQ1 / VA1) × VB2 …(10) VP3 = β × VB3 = (VQ1 / VA1) × VB3 …(11) VP4 = β × VB4 = (VQ1 / VA1) × VB4 …(12) Then, the distance calculation unit 42 uses each of the adjustment voltages VP1, VP2, VP3, and VP4 to calculate the corrected pixel signals VQ1', VQ2', VQ3', and VQ4 of the pixel signals VQ1, VQ2, VQ3, and VQ4 according to the following equations (13), (14), (15), and (16). VQ1' = VQ1 - VP1 …(13) VQ2' = VQ2 - VP2 …(14) VQ3' = VQ3 - VP3 …(15) VQ4' = VQ4 - VP4 …(16)

[0120] As described above, even when the pixel signal reading units are four, namely RU1, RU2, RU3, and RU4, the corrected pixel signals VQ1', VQ2', VQ3', and VQ4 can be calculated in the same manner as when the pixel signal reading units are three, namely RU1, RU2, and RU3. That is, when there are three or more pixel signal reading units, the voltage values of the pixel signals VQ1, VQ2, VQ3, and VQ4 can be accurately corrected by the above-described correction method.

[0121] <Third Embodiment> In each of the first embodiment and the second embodiment, the reference background photovoltage and the reference reference voltage were written and stored in the correction parameter storage unit 43 in advance before measuring the distance between the subject S and the distance image sensor 32. However, in the third embodiment, the correction parameters are acquired for each distance measurement process. This embodiment has the same configuration as the distance image capturing apparatus 1 shown in FIG. 1. Hereinafter, the operations different from those of the first embodiment and the second embodiment will be described. In this embodiment, two frames, namely the first frame and the second frame, are used to acquire one distance image. For this reason, in this embodiment, the distance measurement is performed 30 times per second, as compared with 60 times per second in the first embodiment and the second embodiment.

[0122] FIG. 9 is a timing chart for explaining the process of acquiring each of the reference background photovoltage and the reference reference voltage in the third embodiment of the present invention. FIG. 9(A) shows the timing chart of the first frame, and FIG. 9(B) shows the timing chart of the second frame. In FIG. 9(A), in the first frame, similar to the process described in FIG. 4, the light pulse PO is irradiated, and the reflected light RL (delay time Td from the irradiation timing of the light pulse PO) from the subject S is received to capture a distance image, and each of the pixel signals VQ1, VQ2, and VQ3 is acquired. Then, the distance calculation unit 42 writes each of the acquired pixel signals VQ1, VQ2, and VQ3 into the correction parameter storage unit 43 and stores them. The pixel signal VQ1 is a voltage corresponding to the amount of charge accumulated in the charge accumulation unit CS1 after being collected by the photodiode PD by the incident light of only the background light. The pixel signal VQ2 is a voltage corresponding to the amount of charge accumulated in the charge accumulation unit CS2 after being collected by the photodiode PD by the incident light including a part of the background light and the reflected light RL. The pixel signal VQ3 is a voltage corresponding to the amount of charge accumulated in the charge accumulation unit CS3 after being collected by the photodiode PD by the incident light including a part of the background light and the reflected light RL. Each of the pixel signals VQ1, VQ2, and VQ3 includes the reference reference voltages VB1, VB2, and VB3, which are voltage components of the non-controlled charge corresponding to the amount of charge of the incident light.

[0123] FIG. 9(B) emits the optical pulse PO in the second frame in the same manner as the first frame in FIG. 9(A), and receives the reflected light RL (delay time Td from the emission timing of the optical pulse PO) from the subject S. Similar to the processing in FIG. 5(B), the distance calculation unit 42 obtains the reference voltages VB1, VB2, and VB3 for each of the charge storage units CS1, CS2, and CS3. Here, in the timing chart of FIG. 9(B), at the timing when the charge is distributed to each of the charge storage units CS1, CS2, and CS3, the reset drive signal RSTD that turns on the drain gate transistor GD is output to discard the charge. Here too, since the period for distribution by each of the read gate transistors G1, G2, and G3 is an extremely short time of about 30 ns, the charge collected by the photodiode PD is small and can be held by the photodiode PD in most cases. Similar to FIG. 9(A), the drain gate transistor GD may be in the off state during the distribution period. In the case of this configuration, when the drain gate transistor GD immediately after the distribution period is turned on, the charge held by the photodiode PD is discarded to the drain. Then, the distance calculation unit 42 reads each of the pixel signals VQ1, VQ2, and VQ3 from the correction parameter storage unit 43. The distance calculation unit 42 sets each of the reference voltages VB1, VB2, and VB3 as the adjustment voltages VP1, VP2, and VP3 of the pixel signals VQ1, VQ2, and VQ3 respectively, and calculates the corrected pixel signals VQ1’, VQ2’, and VQ3’ according to the following equations (17), (18), and (19). VQ1’ = VQ1 - VP1 …(17) VQ2’ = VQ2 - VP2 …(18) VQ3’ = VQ3 - VP3 …(19) Then, the distance calculation unit 42 calculates the distance L between the subject S and the distance image sensor 32 in the distance image capturing device 1 using the obtained corrected pixel signals VQ1’, VQ2’, and VQ3’ according to equation (8).

[0124] According to this embodiment, for each process of calculating the distance, since the reference standard voltages VB1, VB2, and VB3 are acquired as correction parameters, it is not necessary to write and store the reference background light voltages VA1, VA2, VA3 corresponding to all the pixels 321 and the reference standard voltages VB1, VB2, VB3 in the correction parameter storage unit 43 in advance as in the first and second embodiments. Therefore, the capacity of the correction parameter storage unit 43 can be reduced. Further, according to this embodiment, in the first frame, the pixel signal for measurement is acquired, and in the next second frame, in order to acquire the pixel signal of the reference standard voltage for correcting the pixel signal for measurement, not only the method of measuring the delay of the single light pulse shown in the present invention, but also, for example, it can be applied to a TOF sensor using CW (continuous wave) modulation in which continuous modulated light is irradiated and the phase shift amount between the irradiated light and the reflected light is obtained and used for distance calculation, and it can be applied to various other TOF sensors. For example, in a TOF sensor using CW (continuous wave) modulation, distance measurement is usually performed using two frames. However, in order to perform the same correction as in this embodiment, two additional frames for correction may be added and four frames may be used to acquire a single distance image.

[0125] <Fourth Embodiment> In the first to third embodiments, the voltage components corresponding to the non-controlled charges generated by the background light and flowing into the charge storage units CS1, CS2, and CS3 are removed from the pixel signals VQ1, VQ2, and VQ3 respectively. On the other hand, in the fourth embodiment, the voltage components corresponding to the non-controlled charges generated by the reflected light RL reflected by the subject S of the light pulse PO (pulse width Tw) and flowing into each of the charge storage units CS1, CS2, and CS3 are removed from the pixel signals VQ1, VQ2, and VQ3 respectively. As described above, if any of the first to third embodiments is implemented, non-controlled charges due to background light can be corrected. In the present embodiment, since it is already known that correction is possible for the fourth embodiment, the description will be made using a state where there is no background light, that is, a state where the background light is blocked from the ambient light in a dark room or the like and does not enter the distance image sensor 32. In the present embodiment, a reference reflected light voltage VC1 corresponding to the charge amount of the reflected light charge C1 generated by the reflected light accumulated in the charge accumulation unit CS1, a reference reflected light voltage VC2 corresponding to the charge amount of the reflected light charge C2 generated by the reflected light accumulated in the charge accumulation unit CS2, and a reference reflected light voltage VC3 corresponding to the charge amount of the reflected light charge C3 generated by the reflected light accumulated in the charge accumulation unit CS3, and a reference reference voltage VD1 corresponding to the charge amount of the non-controlled charge D1 generated by the reflected light accumulated in the charge accumulation unit CS1, a reference reference voltage VD2 corresponding to the charge amount of the non-controlled charge D2 generated by the reflected light accumulated in the charge accumulation unit CS2, and a reference reference voltage VD3 corresponding to the charge amount of the non-controlled charge D3 generated by the reflected light accumulated in the charge accumulation unit CS3 are obtained in advance, for example, at the time of shipment or startup of the distance image capturing device 1, that is, before capturing a distance image, and written and stored in the correction parameter storage unit 43.

[0126] FIG. 10 is a timing chart for explaining the process of acquiring each of the reference reflected light voltage and the reference reference voltage in the fourth embodiment of the present invention. FIG. 10(A) shows the process of acquiring a reference reflected light voltage VC1 corresponding to the charge amount generated by the incident light of only the reflected light accumulated in the charge accumulation unit CS1, a reference reflected light voltage VC2 corresponding to the charge amount generated by the incident light of only the reflected light accumulated in the charge accumulation unit CS2, and a reference reflected light voltage VC3 corresponding to the charge amount generated by the incident light of only the reflected light accumulated in the charge accumulation unit CS3 in the reference reflected light voltage acquisition frame. In FIG. 10(A), the distance image sensor 32 performs the same processes as the distance measurement for emitting the optical pulse PO, i.e., the accumulation of charges in each of the charge accumulation units CS1, CS2, and CS3, and the readout from each of the charge accumulation units CS1, CS2, and CS3, in the same manner as the operation explanation of the frame period in FIG. 4.

[0127] As a result, during the accumulation period in the reference reflected light voltage acquisition frame, only the reflected light that is incident on each of the charge accumulation units CS1, CS2, and CS3 in each pixel 321 at different incident angles due to the lens 31 accumulates the charges generated as the incident light. Then, the readout process is performed in the same manner as the explanation in FIG. 4, and from the pixel signal processing circuit 325 as pixel signals, the reference reflected light voltage VC1 corresponding to the charge amount of the reflected light charge (i.e., the reference reflected light charge) only by the reflected light accumulated in the charge accumulation unit CS1, the reference reflected light voltage VC2 corresponding to the charge amount of only the reflected light accumulated in the charge accumulation unit CS2, and the reference reflected light voltage VC3 corresponding to the charge amount of only the reflected light accumulated in the charge accumulation unit CS3 are output as pixel signals. The distance calculation unit 42 writes and stores each of the acquired reference reflected light voltages VC1, VC2, and VC3 in the correction parameter storage unit 43 together with the identification information for identifying each pixel 321 in the light receiving pixel unit 320. Alternatively, the distance calculation unit 42 may be configured to write and store each of the sequentially acquired reference reflected light voltages VC1, VC2, and VC3 in the correction parameter storage unit 43 in the same order as the order in which the pixel signals are read out from the pixel signal processing circuit 325.

[0128] FIG. 10(B) shows the process of acquiring the reference reference voltage VD1 corresponding to the charge amount of the non-controlled charge that flows into and accumulates in the charge accumulation unit CS1 along with the reflected light, the reference reference voltage VD2 corresponding to the charge amount of the non-controlled charge that flows into and accumulates in the charge accumulation unit CS2 along with the reflected light, and the reference reference voltage VD3 corresponding to the charge amount of the non-controlled charge that flows into and accumulates in the charge accumulation unit CS3 along with the reflected light in the reference reference voltage acquisition frame. In FIG. 10(B), the distance image sensor 32 performs charge accumulation and reading through the same process as the operation explanation of the reference voltage acquisition frame in FIG. 5(B). That is, the timing control unit 41 controls the light source device 21 to emit the optical pulse PO. On the other hand, the control circuit 322 is controlled not to output the accumulation drive signals TX1, TX2, and TX3, and the charge distribution to the charge accumulation units CS1, CS2, and CS3 is not performed. Also, the timing control unit 41 controls the control circuit 322 to output a reset drive signal RSTD that turns on the drain gate transistor GD at the timing of charge distribution to each of the charge accumulation units CS1, CS2, and CS3, and discards the charge generated by the reflected light. However, since the period of distribution by each of the read gate transistors G1, G2, and G3 is an extremely short time of about 30 ns, the charge collected by the photodiode PD is small and can be held by the photodiode PD in most cases. Therefore, the drain gate transistor GD may be in an off state during the period of charge distribution as in FIG. 10(A). In that case, the charge held by the photodiode PD is discarded to the drain when GD immediately after the distribution period is turned on.

[0129] As a result, each of the read gate transistors G1, G2, and G3 remains off, and the charge due to the reflected light is not propagated to each of the charge accumulation units CS1, CS2, and CS3 through each of the read gate transistors G1, G2, and G3. For this reason, only the non-controlled charges QD1, QD2, and QD3 that flow in without passing through each of the read gate transistors G1, G2, and G3 are accumulated in each of the charge accumulation units CS1, CS2, and CS3. Then, readout processing is performed in the same manner as in FIG. 10(A), and from the pixel signal processing circuit 325 as pixel signals, a reference reference voltage VD1 corresponding to the amount of non-controlled charge QD1 stored in the charge storage section CS1, a reference reference voltage VD2 corresponding to the amount of non-controlled charge QD2 stored in the charge storage section CS2, and a reference reference voltage VD3 corresponding to the amount of non-controlled charge QD3 stored in the charge storage section CS3 are output as pixel signals. The distance calculation section 42 writes and stores each of the acquired reference reference voltages VD1, VD2, and VD3 in the correction parameter storage section 43 together with identification information for identifying each of the pixels 321 in the light receiving pixel section 320. Alternatively, the distance calculation section 42 may be configured to write and store each of the sequentially acquired reference reference voltages VD1, VD2, and VD3 in the correction parameter storage section 43 in the same order as the order in which pixel signals are read out from the pixel signal processing circuit 325.

[0130] Then, in the frame period for measuring the distance between the distance image sensor 32 and the subject S, as already described, the pixel signal processing circuit 325 outputs, as pixel signals (digital values), each of the pixel signals VQ1, VQ2, and VQ3 corresponding to each of the charge storage sections CS1, CS2, and CS3 for each pixel 321 corresponding to one frame to the distance calculation section 42. At this time, the distance calculation section 42 sequentially reads out the reference reflected light voltages VC1, VC2, and VC3 corresponding to the read pixel 321 and the reference reference voltages VD1, VD2, and VD3 from the correction parameter storage section 43.

[0131] Then, the distance calculation section 42 calculates a total reflected light charge voltage VCALL corresponding to the amount of charge of the charge generated by the reflected light in the reference reflected light voltage acquisition frame by the following equation (20). VCALL=(VC2-VC1)+(VC3-VC1) …(20) The distance calculation section 42 extracts, as the total reflected light charge voltage VCALL, the components of the reflected light charges generated by the reflected light included in each of the reference reflected light voltages VC1, VC2, and VC3 by the above equation (20). Then, the distance calculation unit 42 calculates the total reflected light charge voltage VQALL corresponding to the amount of charge of the charge generated by the reflected light in the frame for measuring the distance between the distance image sensor 32 and the subject S by the following equation (21). VQALL=(VQ2 - VQ1)+(VQ3 - VQ1) …(21) The distance calculation unit 42 extracts, as the total reflected light charge voltage VQALL, the components of the reflected light charges generated by the reflected light included in each of the pixel signals VQ1, VQ2, and VQ3 according to the above equation (21).

[0132] The distance calculation unit 42 divides the total reflected light charge voltage VQALL by the total reflected light charge voltage VCALL (VQALL / VCALL) to calculate the adjustment ratio β. This adjustment ratio β indicates the intensity ratio of the reflected light RL in the reference reflected light voltage acquisition frame and the frame for measuring the distance between the distance image sensor 32 and the subject S. Therefore, the distance calculation unit 42 calculates adjustment voltages VR1, VR2, and VR3 for correcting each of the pixel signals VQ1, VQ2, and VQ3 according to the following equations (22), (23), and (24) respectively. VR1 = β×VD1=(VQALL / VCALL)×VD1 …(22) VR2 = β×VD2=(VQALL / VCALL)×VD2 …(23) VR3 = β×VD3=(VQALL / VCALL)×VD3 …(24) Then, the distance calculation unit 42 calculates corrected pixel signals VQ1’, VQ2’, and VQ3’ respectively according to the following equations (25), (26), and (27). VQ1’ = VQ1 - VR1 …(25) VQ2’ = VQ2 - VR2 …(26) VQ3’ = VQ3 - VR3 …(27) The distance calculation unit 42 uses the obtained corrected pixel signals VQ1’, VQ2’, and VQ3’ to calculate the distance L between the distance image sensor 32 and the subject S according to equation (8) in the first embodiment.

[0133] <Fifth Embodiment> In the fourth embodiment described above, before measuring the distance between the subject S and the distance image sensor 32, the reference reflected light voltage and the reference reference voltage were written and stored in the correction parameter storage unit 43 in advance. However, in the fifth embodiment, a configuration is adopted in which correction parameters are acquired for each distance measurement process. This embodiment has the same configuration as the distance image capturing device 1 shown in the fourth embodiment already described. Hereinafter, operations different from those of the fourth embodiment will be described. In this embodiment, two frames, i.e., the first frame and the second frame, are used to acquire a single distance image. For this reason, in this embodiment, 30 distance measurements are performed per second, as compared with 60 distance measurements per second in the fourth embodiment, for example. Also in this embodiment, similar to the fourth embodiment, the description will be made using a state in which there is no background light, that is, a state in which the background light is blocked from the ambient light in a dark room or the like and is not incident on the distance image sensor 32.

[0134] In this embodiment, it is assumed that FIG. 10(A) in the timing chart of FIG. 10 shows the timing chart of the first frame, and FIG. 10(B) shows the timing chart of the second frame. In FIG. 10(A), in the first frame, similar to the process in the fourth embodiment, a light pulse PO is irradiated, and the reflected light RL (delay time Td from the irradiation timing of the light pulse PO) from the subject S is received to capture a distance image, and each of the pixel signals VQ1, VQ2, and VQ3 is acquired. Then, the distance calculation unit 42 writes and stores each of the acquired pixel signals VQ1, VQ2, and VQ3 in the correction parameter storage unit 43. The pixel signal VQ1 is a voltage corresponding to the amount of charge stored in the charge storage unit CS1. Further, the pixel signal VQ2 is a voltage corresponding to the amount of charge collected by the photodiode PD by incident light including a part of the reflected light RL and stored in the charge storage unit CS2, and the pixel signal VQ3 is a voltage corresponding to the amount of charge collected by the photodiode PD by incident light including a part of the reflected light RL and stored in the charge storage unit CS3. Each of the pixel signals VQ1, VQ2, and VQ3 includes reference reference voltages VD1, VD2, and VD3, which are voltage components of non-controlled charges corresponding to the amount of charge corresponding to the incident light.

[0135] In the second frame shown in FIG. 10(B), similar to the first frame shown in FIG. 10(A), in order to emit the optical pulse PO, the reflected light RL (delay time Td from the emission timing of the optical pulse PO) from the subject S is received. However, by the same processing as in FIG. 5(B), the distance calculation unit 42 acquires each of the reference reference voltages VD1, VD2, and VD3 for each of the charge storage units CS1, CS2, and CS3. Then, the distance calculation unit 42 reads each of the pixel signals VQ1, VQ2, and VQ3 from the correction parameter storage unit 43. The distance calculation unit 42 sets each of the reference reference voltages VD1, VD2, and VD3 as adjustment voltages VO1, VO2, and VO3 of the pixel signals VQ1, VQ2, and VQ3 respectively, and calculates corrected pixel signals VQ1’, VQ2’, and VQ3’ according to the following equations (28), (29), and (30). VQ1’ = VQ1 - VO1 …(28) VQ2’ = VQ2 - VO2 …(29) VQ3’ = VQ3 - VO3 …(30) Then, the distance calculation unit 42 calculates the distance L between the subject S and the distance image sensor 32 in the distance image capturing device 1 using the obtained corrected pixel signals VQ1’, VQ2’, and VQ3’ according to equation (8).

[0136] According to the present embodiment, since the reference reference voltages VD1, VD2, and VD3 are acquired as correction parameters for each process of calculating the distance, it is not necessary to write and store the reference reflected light voltages VC1, VC2, VC3 corresponding to all the pixels 321 and the reference reference voltages VD1, VD2, VD3 in the correction parameter storage unit 43 in advance as in the fourth embodiment. Therefore, the capacity of the correction parameter storage unit 43 can be reduced. Further, according to the present embodiment, in the first frame, the pixel signal for measurement is acquired, and in the next second frame, the pixel signal of the reference reference voltage for correcting the pixel signal for measurement is acquired. Therefore, not only the method of measuring the delay of the single light pulse shown in the present invention, but also the CW modulation using the continuous modulated light as in the fourth embodiment, irradiating the light, obtaining the amount of phase shift between the irradiated light and the reflected light, and using it for distance calculation can be applied to the TOF sensor, and it can be applied to various other TOF sensors.

[0137] <Sixth Embodiment> The distance image capturing device according to the sixth embodiment has the same configuration as the first embodiment shown in FIG. 1. As shown in FIG. 3, the pixel 321 includes three pixel signal reading units RU1, RU2, and RU3, and each of the pixel signal reading units RU1, RU2, and RU3 includes a charge storage unit CS1, CS2, and CS3, respectively. In the case of the configuration of the present embodiment, the charge storage unit CS1, similar to the first embodiment, accumulates the background light charges collected by the photodiode PD by the incident light only by the background light at the distribution time Tw1 (the same time width as the pulse width Tw, and the same applies to Tw2 and Tw3 shown hereinafter). Each of the distribution times Tw1, Tw2, and Tw3 is the pulse width of the accumulation drive signal TX1 applied to the read gate transistor G1, the accumulation drive signal TX2 applied to the read gate transistor G2, and the accumulation drive signal TX3 applied to the read gate transistor G3, and each of Tw1, Tw2, and Tw3 is the same as the pulse width Tw of the light pulse PO. In the charge storage unit CS1, similar to the timing chart of FIG. 4, the background photocurrent collected by the photodiode PD is distributed at the distribution time Tw1 before the light pulse PO is irradiated. In each of the charge storage units CS2 and CS3, similar to the timing chart of FIG. 4, the background photocurrent collected by the photodiode PD and the charge corresponding to the reflected light RL are distributed at the distribution time Tw2 when the light pulse PO is irradiated and at the next distribution time Tw3, respectively.

[0138] Further, in the sixth embodiment, the operation of measuring the distance is different from that of the first embodiment. The voltage component corresponding to the non-controlled charge generated by the reflected light RL (pulse width Tw) of the light pulse PO reflected by the subject S and flowing into each of the charge storage units CS1, CS2, and CS3, the voltage component corresponding to the controlled charge generated by the background light RL and distributed to each of the charge storage units CS1, CS2, and CS3, and the voltage component corresponding to the non-controlled charge generated by the background light and flowing into each of the charge storage units CS1, CS2, and CS3 are each subtracted from the pixel signals VQ1, VQ2, and VQ3 for correction. That is, a voltage (reflected light control voltages VCL2 and VCL3 described later) corresponding only to the amount of charge (charge amounts QCL2 and QCL3 described later) generated by the reflected light RL and collected by the photoelectric conversion element PD is calculated, and the distance between the distance image pickup device 1 and the subject S is obtained.

[0139] FIG. 15 is a conceptual diagram showing the relationships among the charge amounts QCB1, QCB2, QCB3, the charge amounts QFB1, QFB2, QFB3, the charge amounts QCL1 (not shown, described later), QCL2, QCL3, and the charge amounts QFL1, QFL2, QFL3 in the respective charge amounts Q1, Q2, Q3 stored in the charge storage units CS1, CS2, and CS3. Here, each of the charge amounts QCB1, QCB2, and QCB3 is the charge amount of the control charge generated by the background light, collected by the photoelectric conversion element PD, and distributed and stored in the charge storage units CS1, CS2, and CS3 respectively by the readout gate transistors G1, G2, and G3. Also, each of the charge amounts QFB1, QFB2, and QFB3 is the charge amount of the non-control charge generated by the background light and flowing into and stored in each of the charge storage units CS1, CS2, and CS3. Each of the charge amounts QCL1 (not shown, described later), QCL2, and QCL3 is the charge amount of the control charge generated by the reflected light RL, collected by the photoelectric conversion element PD, and distributed and stored in the charge storage units CS1, CS2, and CS3 respectively by the readout gate transistors G1, G2, and G3. The charge amounts QFL1, QFL2, and QFL3 are the charge amounts of the non-control charge generated by the reflected light RL and flowing into and stored in each of the charge storage units CS1, CS2, and CS3. Here, among the electrons (charges) generated by the incident light, the electrons (charges) collected by the photoelectric conversion element PD and distributed to the charge storage unit CS via the readout gate transistor G are control charges, while the electrons (charges) flowing into the charge storage unit CS without passing through the readout gate transistor G are non-control charges.

[0140] That is, the control charge due to the reflected light RL is the charge generated by the reflected light RL, collected by the photoelectric conversion element PD, and distributed to each of the charge storage units CS1, CS2, and CS3 via the readout gate transistors G1, G2, and G3 respectively (see FIGS. 3 and 12). Also, the non-control charge due to the reflected light RL is the charge generated by the reflected light RL and flowing into each of the charge storage units CS1, CS2, and CS3 without passing through the readout gate transistors G1, G2, and G3. Similarly, the control charge due to the background light is the charge generated by the background light and collected by the photoelectric conversion element PD, and is the charge distributed to each of the charge storage units CS1, CS2, and CS3 via the readout gate transistors G1, G2, and G3, respectively. Also, the non-control charge due to the background light is the charge generated by the background light and is the charge flowing into each of the charge storage units CS1, CS2, and CS3 without passing through the readout gate transistors G1, G2, and G3.

[0141] In the present embodiment, before starting the frame for measuring the distance, as parameters necessary for calculating the reflected light control voltages VCL2 and VCL3 corresponding only to the charge amounts QCL2 and QCL3 of the control charge generated by the reflected light RL, collected by the photoelectric conversion element PD, and distributed by the readout gate transistors G1, G2, and G3, each of the following parameters is acquired in advance and written and stored in the correction parameter storage unit 43 by subtracting the charge amounts QFB1, QFB2, and QFB3 and the non-control voltages corresponding to the charge amounts QFL1, QFL2, and QFL3 due to the non-control charge from the signal voltages corresponding to the charge amounts accumulated in each of the charge storage units CS1, CS2, and CS3 for correction. That is, the reference background light control voltage VPCB1 corresponding to the charge amount (control charge amount) QPCB1 of the control charge generated by the background light, which is distributed from the photoelectric conversion element PD via the read gate transistor G1 and accumulated in the charge accumulation unit CS1 by the reference background light acquisition frame (the frame for turning on and off the read gate transistor G in a state where the optical pulse PO shown in FIG. 5(A) is not irradiated) and the reference reference voltage acquisition frame (the frame for not driving the read gate transistor G in a state where the optical pulse PO shown in FIG. 5(B) is not irradiated) in the first embodiment already described, the reference background light control voltage VPCB2 corresponding to the control charge amount QPCB2 generated by the background light, which is distributed from the photoelectric conversion element PD via the read gate transistor G2 and accumulated in the charge accumulation unit CS2, and the reference background light control voltage VPCB3 corresponding to the control charge amount QPCB3 generated by the background light, which is distributed from the photoelectric conversion element PD via the read gate transistor G3 and accumulated in the charge accumulation unit CS3, and the reference background light non-control voltage VPFB1 corresponding to the charge amount (non-control charge amount) QPFB1 of the non-control charge generated by the background light and flowing into and accumulating in the charge accumulation unit CS1 in an uncontrolled state, the reference background light non-control voltage VPFB2 corresponding to the non-control charge amount QPFB2 generated by the background light and flowing into and accumulating in the charge accumulation unit CS2 in an uncontrolled state, and the reference background light non-control voltage VPFB3 corresponding to the non-control charge amount QPFB3 generated by the background light and flowing into and accumulating in the charge accumulation unit CS3 in an uncontrolled state are acquired in advance, for example, at the time of shipment or startup of the distance image capturing device 1, that is, before capturing a distance image, and written and stored in the correction parameter storage unit 43. Alternatively, the reference background light voltage VA acquired by the reference background light voltage acquisition frame and the reference reference voltage VB acquired by the reference reference voltage acquisition frame are acquired in advance before capturing a distance image, and written and stored in the correction parameter storage unit 43. Of course, both the reference background light control voltage VPCB and the reference background light non-control voltage VPFB, and the reference background light voltage VA and the reference reference voltage VB may be acquired in advance before capturing a distance image, and written and stored in the correction parameter storage unit 43. The acquisition processes of the above-described reference background light control voltages VPCB1, VPCB2, and VPCB3 and the reference background light non-control voltages VPFB1, VPFB2, and VPFB3 will be described later.

[0142] Also, the reference reflected light voltage acquisition frame in the fourth embodiment (the frame for turning on and off the read gate transistor G in the state where the light pulse PO is irradiated in the shielded environment shown in FIG. 10(A)) and the reference reference voltage acquisition frame (the frame for not driving the read gate transistor G in the state where the light pulse PO is irradiated in the shielded environment shown in FIG. 10(B)) are used to distribute the control charge amount QPCL1 generated by the reflected light, which corresponds to the reference reflected light control voltage VPCL1, from the photoelectric conversion element PD through the read gate transistor G1 and accumulate it in the charge accumulation unit CS1, the control charge amount QPCL2 generated by the reflected light, which corresponds to the reference reflected light control voltage VPCL2, from the photoelectric conversion element PD through the read gate transistor G2 and accumulate it in the charge accumulation unit CS2, and the control charge amount QPCL3 generated by the reflected light, which corresponds to the reference reflected light control voltage VPCL3, from the photoelectric conversion element PD through the read gate transistor G3 and accumulate it in the charge accumulation unit CS3, and the reference reflected light non-control voltage VPFL1 corresponding to the non-control charge amount QPFL1 generated by the reflected light and flowing into and accumulating in the charge accumulation unit CS1 in a non-controlled state, the reference reflected light non-control voltage VPFL2 corresponding to the non-control charge amount QPFL2 generated by the reflected light and flowing into and accumulating in the charge accumulation unit CS2 in a non-controlled state, and the reference reflected light non-control voltage VPFL3 corresponding to the non-control charge amount QPFL3 generated by the reflected light and flowing into and accumulating in the charge accumulation unit CS3 in a non-controlled state are acquired in advance, for example, at the time of shipment or startup of the distance image capturing device 1, that is, before capturing a distance image, and written and stored in the correction parameter storage unit 43. Alternatively, the reference reflected light voltage VC acquired in the reference reflected light voltage acquisition frame and the reference reference voltage VD acquired in the reference reference voltage acquisition frame are acquired in advance before capturing a distance image, and written and stored in the correction parameter storage unit 43. Of course, both the reference reflected light control voltage VPCL and the reference reflected light non-control voltage VPFL, and the reference reflected light voltage VC and the reference reference voltage VD may be acquired in advance before imaging the distance image, and written and stored in the correction parameter storage unit 43. The acquisition processes of the above-described reference reflected light control voltages VPCL1, VPCL2, and VPCL3 and the reference reflected light non-control voltages VPFL1, VPFL2, and VPFL3 will be described later.

[0143] Also, as described with reference to FIG. 6 in the first embodiment, the non-control charge due to the background light is generated by irradiating the light pulse PO and accumulating charge in the charge accumulation unit CS during the charge accumulation period (Integuration) and the pixel signal readout period (Read) for reading the charge from the charge accumulation unit CS. During this period, the background light irradiates the distance image sensor 32 to generate electrons, which are accumulated in the charge accumulation unit CS. That is, the non-control charge due to the background light continuously flows into the charge accumulation unit CS and is accumulated as it is during a period of approximately one frame (Integuration + Read). On the other hand, the non-control charge due to the reflected light RL is accumulated in the charge accumulation unit CS only during the charge accumulation period (Integuration). That is, the period during which the non-control charge due to the background light is accumulated is longer than the period during which the non-control charge due to the reflected light RL is accumulated. Therefore, the amount of non-control charge flowing into and accumulated in the charge accumulation unit CS is mainly due to the background light. However, as the intensity of the background light decreases, the influence of the amount of non-control charge due to the reflected light RL increases. Therefore, in this embodiment, the non-control voltage corresponding to the amount of non-control charge due to both the background light and the reflected light RL is corrected to further improve the accuracy of distance measurement.

[0144] Next, in the present embodiment, a non-controlled voltage due to non-controlled charges flowing into the charge storage unit CS without passing through the read gate transistor G is corrected, and only the reflected light control voltages (reflected light control voltages VCL2 and VCL3) due to the amount of control charges generated by the reflected light RL, collected by the photoelectric conversion element PD, and distributed to the charge storage unit CS through the read gate transistor G are extracted, and the process of calculating the distance between the subject S and the distance imaging device 1 based on the reflected light control voltage will be described. Corresponding to the amounts of charges Q1, Q2, and Q3 stored in the charge storage units CS1, CS2, and CS3 in FIG. 15, respectively, the pixel signals VQ1, VQ2, and VQ3 supplied from the pixel circuit 321 of the light receiving unit 3 are represented by the following equations (31) to (33). VQ1 = VCB1 + VCL1 + VFB1 + VFL1 …(31) VQ2 = VCB2 + VCL2 + VFB2 + VFL2 …(32) VQ3 = VCB3 + VCL3 + VFB3 + VFL3 …(33) In the above equations (31) to (33), VCB is a voltage corresponding to the amount of charge generated by background light, distributed from the photoelectric conversion element PD by the read gate transistor G, and stored in the charge storage unit CS, that is, the background light control voltage corresponding to the amount of control charge QCB due to background light. VCL is a voltage corresponding to the amount of charge generated by reflected light, distributed from the photoelectric conversion element PD by the read gate transistor G, and stored in the charge storage unit CS, that is, the reflected light control voltage corresponding to the amount of control charge QCL due to the reflected light RL. VFB is a voltage corresponding to the amount of charge generated by background light and flowing into and stored in the charge storage unit CS in an uncontrolled state, that is, the background light uncontrolled voltage corresponding to the amount of non-controlled charge QFB due to background light. VFL is a voltage corresponding to the amount of charge generated by reflected light and flowing into and stored in the charge storage unit CS in an uncontrolled state, that is, the reflected light uncontrolled voltage corresponding to the amount of non-controlled charge QFL due to the reflected light RL. In the above formula (31), since the optical pulse PO is emitted after the charge is distributed from the photoelectric conversion element PD to the charge storage unit CS1, the reflected light RL does not enter the distance image sensor 32 when the charge is distributed to the charge storage unit CS1. Therefore, the control charge amount QCB1 and the non-control charge amount QFB1 due to the background light, and the non-control charge amount QFL1 due to the reflected light RL are accumulated as the charge amount Q1 in the charge storage unit CS1, and the control charge amount QCL1 due to the reflected light RL is not accumulated. For this reason, the reflected light control voltage VCL1 becomes 0 as shown by the following formula (34). VCL1 = 0 …(34)

[0145] Also, each of the background light control voltages VCB1, VCB2, and VCB3 is generated by the background light and the charge collected by the photoelectric conversion element PD is distributed to the charge storage units CS1, CS2, and CS3 through the readout gate transistors G1, G2, and G3 by the accumulation drive signals TX1, TX2, and TX3 having the same pulse width. Therefore, they have the same voltage value as shown by the following formula (35). VCB1 = VCB2 = VCB3 …(35) Accordingly, each of formulas (31), (32), and (33) becomes the following formulas (36), (37), and (38) respectively based on formulas (34) and (35). VQ1 = VCB1 + VFB1 + VFL1 …(36) VQ2 = VCB1 + VCL2 + VFB2 + VFL2 …(37) VQ3 = VCB1 + VCL3 + VFB3 + VFL3 …(38)

[0146] The above-described pixel signals VQ1, VQ2, and VQ3 are the actually measured values actually measured in the frame for measuring the distance and are known values. However, the background light control voltage VCB1, the background light non-control voltages VFB1, VFB2, VFB3, the reflected light control voltages VCL2, VCL3, and the reflected light non-control voltages VFL1, VFL2, VFL3, which are the parameters constituting each of the pixel signals VQ1, VQ2, and VQ3, are unknown because they are mixed as charge amounts as shown in FIG. 15. Among the above-mentioned unknowns, if the background light control voltage VCB1, reflected light control voltages VCL2 and VCL3 can be obtained respectively, the distance between the subject S and the imaging device 1 can be calculated.

[0147] As already described with reference to FIG. 5 in the first embodiment, in the reference background light voltage acquisition frame (FIG. 5(A)) in a state where the light pulse PO is not irradiated, the accumulation drive signals TX1, TX2, and TX3 are driven to obtain reference background light voltages VA1, VA2, and VA3 corresponding to the charges accumulated in the charge accumulation units CS1, CS2, and CS3 respectively. Also, in the reference reference voltage acquisition frame (FIG. 5(B)), the accumulation drive signals TX1, TX2, and TX3 are not driven, and reference reference voltages VB1, VB2, and VB3 corresponding to the charges accumulated in the charge accumulation units CS1, CS2, and CS3 respectively are obtained. Here, the reference background light voltage VA obtained in the reference background light control voltage acquisition frame is the sum of a reference background light control voltage VPCB corresponding to the amount of charge generated by the background light, distributed by the readout gate transistor G from the photoelectric conversion element PD, and accumulated in the charge accumulation unit CS, and a reference background light non-control voltage VPFB corresponding to the amount of charge flowing into and accumulated in the charge accumulation unit CS in an uncontrolled state. Also, the reference reference voltage VB obtained in the reference reference voltage acquisition frame is a reference background light non-control voltage VPFB corresponding to the amount of charge generated by the background light and flowing into and accumulated in the charge accumulation unit CS in an uncontrolled state. Therefore, each of the reference background light control voltages VPCB1, VPCB2, and VPCB3 is represented by the following formula (39) using the reference background light voltage VA1 and the reference reference voltage VB1. VPCB1 = VPCB2 = VPCB3 = VA1 - VB1 …(39) Also, each of the reference background light non-control voltages VPFB1, VPFB2, and VPFB3 is represented by the following formulas (40) to (42) using the reference reference voltages VB1, VB2, and VB3. VPFB1 = VB1 …(40) VPFB2 = VB2 …(41) VPFB3 = VB3 …(42) From the above equations (39) to (42), the reference background light control voltage VPCB corresponding to the control charge amount due to the background light and the reference background light non-control voltage VPFB corresponding to the non-control charge amount due to the background light can be represented by the reference background light voltages VA1, VA2, and VA3 and the reference reference voltages VB1, VB2, and VB3 obtained in the reference background light voltage acquisition frame and the reference reference voltage acquisition frame.

[0148] The difference between the reference background light control voltage VPCB and the reference background light non-control voltage VPFB obtained from the reference background light voltage VA obtained in the reference background light voltage acquisition frame (Fig. 5(A)) and the reference reference voltage VB obtained in the reference reference voltage acquisition frame (Fig. 5(B)), and the background light control voltage VCB and the background light non-control voltage VFB obtained in the frame for measuring the distance, is the difference in the intensity of the incident background light due to different environments. Therefore, the ratio CP1 of the reference background light control voltage VPCB obtained using the reference background light voltage acquisition frame and the reference reference voltage acquisition frame to the background light control voltage VCB obtained in the frame for measuring the distance, and the ratio CP2 of the reference background light non-control voltage VPFB to the background light non-control voltage VFB obtained in the frame for measuring the distance are the same if each voltage is at a signal level with linearity. As described above, since the ratios CP1 and CP2 are the same, the background light correction coefficient R_B between the reference background light non-control voltage VPFB, the background light non-control voltage VFB, the reference background light control voltage VPCB, and the background light control voltage VCB is represented by the following equation (43). R_B = (background light non-control voltage of the distance measurement frame) / (reference background light non-control voltage) = VFB / VPFB = (background light control voltage of the distance measurement frame) / (reference background light control voltage) = VCB / VPCB …(43) Also, the above equation (43) can also be represented as the following equation (44) using the relationships of equations (35) and (39). R_B = VCB1 / VPCB1 …(44)

[0149] By using the above-described equations (39), (43), and (44), the background light correction coefficient \(R_B\) is represented by the following equation (45). \(R_B = V_{CB1} / V_{PCB1}\) \(= V_{CB1} / (V_{A1}-V_{B1})\) \(= V_{FB1} / V_{PFB1}\) \(= V_{FB2} / V_{PFB2}\) \(= V_{FB3} / V_{PFB3}\ \cdots(45)\) From the above-described equation (45), each of the background light non-control voltages \(V_{FB1}\), \(V_{FB2}\), and \(V_{FB3}\) corresponding to the background light is represented by each of the following equations (46), (47), and (48). \(V_{FB1}=R_B\times V_{PFB1}=R_B\times V_{B1}\ \cdots(46)\) \(V_{FB2}=R_B\times V_{PFB2}=R_B\times V_{B2}\ \cdots(47)\) \(V_{FB3}=R_B\times V_{PFB3}=R_B\times V_{B3}\ \cdots(48)\) According to each of the above-described equations (46), (47), and (48), each of the background light non-control voltages \(V_{FB1}\), \(V_{FB2}\), and \(V_{FB3}\) at the time of the frame for measuring the distance can be obtained by the respective reference background light non-control voltages \(V_{PFB1}\), \(V_{PFB2}\), \(V_{PFB3}\) and the background light correction coefficient \(R_B\). Also, regarding the background light correction coefficient \(R_B\), when paying attention to the part of " \(R_B = V_{CB1} / (V_{A1}-V_{B1})\)" in the above equation (45), it can be obtained by referring to the desired background light control voltage \(V_{CB1}\), the known reference background light voltage \(V_{A1}\) and the reference reference voltage \(V_{B1}\) stored in the correction parameter storage unit 43.

[0150] Next, a process of obtaining the reflected light non-control voltage \(V_{FL}\) in the frame for measuring the distance from the reference reflected light control voltage \(V_{PCL}\) and the reference reflected light non-control voltage \(V_{PFL}\) obtained by the reference reflected light voltage \(V_C\) obtained in the reference reflected light voltage acquisition frame (FIG. 10(A)) and the reference reference voltage \(V_D\) obtained in the reference reference voltage acquisition frame (FIG. 10(B)) will be described. First, create an environment where the background light does not enter due to light shielding, and in the reference reflected light voltage acquisition frame (Figure 10(A)) in the state of irradiating the optical pulse PO, drive the accumulation drive signals TX1, TX2, and TX3 to obtain the reference reflected light voltages VC1, VC2, and VC3 corresponding to the amounts of charge accumulated in the charge accumulation units CS1, CS2, and CS3, respectively. Also, in the reference reference voltage acquisition frame (Figure 10(B)) in the state of irradiating the optical pulse PO, without driving the accumulation drive signals TX1, TX2, and TX3, obtain the reference reference voltages VD1, VD2, and VD3 corresponding to the charges generated by the reflected light RL flowing into the charge accumulation unit CS (obtained in the same manner as in the first embodiment). Here, the reference reflected light voltage VC obtained in the reference reflected light control voltage acquisition frame is the sum of the reference reflected light control voltage VPCL corresponding to the amount of charge generated by the reflected light and distributed by the readout gate transistor G from the photoelectric conversion element PD and accumulated in the charge accumulation unit CS, and the reference background light non-control voltage VPFL corresponding to the amount of charge flowing into and accumulated in the charge accumulation unit CS in an uncontrolled state. Also, the reference reference voltage VD obtained in the reference reference voltage acquisition frame is the reference reflected light non-control voltage VPFL corresponding to the amount of charge generated by the reflected light and flowing into and accumulated in the charge accumulation unit CS in an uncontrolled state. Therefore, each of the reference reflected light control voltages VPCL1, VPCL2, and VPCL3 is represented by the following equations (49), (50), and (51) using the reference reflected light voltages VC1, VC2, and VC3 and the reference reference voltages VD1, VD2, and VD3, respectively. VPCL1 = VC1 - VD1 …(49) VPCL2 = VC2 - VD2 …(50) VPCL3 = VC3 - VD3 …(51)

[0151] Also, each of the reference reflected light non-control voltages VPFL1, VPFL2, and VPFL3 corresponding to the reflected light is represented by the following equations (52), (53), and (54) using the reference reference voltages VD1, VD2, and VD3, respectively. VPFL1 = VD1 …(52) VPFL2 = VD2 …(53) VPFL3 = VD3 …(54) From the above equations (49) to (54), the reference reflected light control voltage VPCL corresponding to the control charge by the reflected light and the reference reflected light non-control voltage VPFL corresponding to the non-control charge by the reflected light can be represented by the reference reflected light voltages VC1, VC2, and VC3 obtained in the reference reflected light voltage acquisition frame (Fig. 10(A)) and the reference reference voltages VD1, VD2, and VD3 obtained in the reference reference voltage acquisition frame (Fig. 10(B)).

[0152] Then, in the frame for measuring the distance, when the charge storage unit CS1 is a charge storage unit that stores only the background light, since the control charge generated by the reflected light is stored in the charge storage units CS2 and CS3, the total control charge voltage VCALL corresponding to the total charge amount of the control charge generated by the reflected light RL is expressed by the following equation (55). VCALL = VCL2 + VCL3 …(55) Similarly, the reference reflected light total control charge voltage VPCALL corresponding to the total charge amount of the control charge generated by the reflected light RL, which is obtained in the reference reflected light voltage acquisition frame (Fig. 10(A)) and the reference reference voltage acquisition frame (Fig. 10(B)), is expressed by the following equation (56). VPCALL = VPCL2 + VPCL3 =(VC2 - VD2)+(VC3 - VD3) …(56)

[0153] Also, the differences between the reference reflected light control voltage VPCL, the reference reflected light non-control voltage VPFL obtained from the reference reflected light voltage VC obtained in the reference reflected light voltage acquisition frame (Fig. 10(A)) and the reference reference voltage VD obtained in the reference reference voltage acquisition frame (Fig. 10(B)), and the reflected light control voltage VCL and the reflected light non-control voltage VFL obtained in the frame for measuring the distance are the differences in the intensity of the incident reflected light RL. Therefore, the ratio CP3 between the reference reflected light control voltage VPCL obtained using the reference reflected light voltage acquisition frame and the reference reference voltage acquisition frame, and the reflected light control voltage VCL acquired in the frame for measuring the distance, and the ratio CP4 between the reference reflected light non-control voltage VPFL and the reflected light non-control voltage VFL acquired in the frame for measuring the distance are the same if each voltage is at a signal level with linearity. As described above, since the ratios CP3 and CP4 are the same, the reflected light correction coefficient RL between the reference reflected light non-control voltage VPFL, the reflected light non-control voltage VFL, the reference reflected light control voltage VPCL, and the reflected light control voltage VCL is expressed by the following equation (57). RL = (reflected light non-control voltage in the distance measurement frame) / (reference reflected light non-control voltage) = VFL / VPFL = (reflected light control voltage in the distance measurement frame) / (reference reflected light control voltage) = VCL / VPCL …(57) Also, the above equation (57) can also be expressed as the following equation (58) using the relationships of equations (55) and (56) indicating the total charge amount of the control charge generated by the reflected light RL. RL = (total control charge voltage of the reflected light in the distance measurement frame) / (total control charge voltage of the reference reflected light) = VCALL / VPCALL …(58)

[0154] By using the above equations (57) and (58), the reflected light correction coefficient RL is expressed by the following equation (59). RL = (VCL2 + VCL3) / ((VC2 - VD2) + (VC3 - VD3)) = VFL1 / VPFL1 = VFL2 / VPFL2 = VFL3 / VPFL3 …(59)

[0155] From the above equation (59), each of the reflected light non-control voltages VFL1, VFL2, and VFL3 is expressed by the following equations (60), (61), and (62) respectively using the reflected light correction coefficient RL. VFL1 = R_L × VPFL1 = R_L × VD1 …(60) VFL2 = R_L × VPFL2 = R_L × VD2 …(61) VFL3 = R_L × VPFL3 = R_L × VD3 …(62) From each of the above equations (60), (61), and (62), each of the reflected light non-control voltages VFL1, VFL2, and VFL3 at the frame time of measuring the distance can be obtained by the reference reflected light non-control voltage and the reflected light correction coefficient R_L. Also, regarding the reflected light correction coefficient R_L, when paying attention to the part of equation (59) "R_L = (VCL2 + VCL3) / ((VC2 - VD2)+(VC3 - VD3))", it can be obtained by referring to the desired reflected light control voltages VCL2 and VCL3, and the known reference reflected light voltages VC2 and VC3 and reference reference voltages VD2 and VD3 stored in the correction parameter storage unit 43. From the above, each of the background light non-control voltages VFB1, VFB2, and VFB3 and the reflected light non-control voltages VFL1, VFL2, and VFL3, which were the unknowns in equations (36), (37), and (38), becomes a system of three linear equations with three unknowns consisting of the reference background light voltage VA1, the reference reference voltages VB1, VB2, and VB3, the reference reflected light voltages VC2 and VC3, the reference reference voltages VB1, VB2, and VB3 stored in the correction parameter storage unit 43, the desired background light control voltage VCB1, and the reflected light control voltages VCL2 and VCL3. Since the unknowns in this system of three linear equations are only the three desired background light control voltage VCB1 and the reflected light control voltages VCL2 and VCL3, the solution can be obtained precisely. That is, the distance calculation unit 42 obtains the background light control voltage VCB1 and the reflected light control voltages VCL2 and VCL3 by solving the above system of three linear equations, and calculates the distance L between the distance image capturing device 1 and the subject S according to equation (63) described later. Alternatively, the solution obtained by solving the above system of three linear equations is stored in the storage unit or described in the program to obtain the background light control voltage VCB1, the reflected light control voltages VCL2, VCL3, and calculate the distance L between the distance image capturing device 1 and the subject S.

[0156] As described above, in the reference background light voltage acquisition frame (Fig. 5(A)), the reference reference voltage acquisition frame (Fig. 5(B)), the reference reflected light voltage acquisition frame (Fig. 10(A)), and the reference reference voltage acquisition frame (Fig. 10(B)), the reference background light voltages VA1, VA2, and VA3, the reference reflected light voltages VC1, VC2, and VC3, and the reference reference voltages VB1, VB2, VB3, VD1, VD2, and VD3 are acquired as measured values, and are written and stored in the correction parameter storage unit 43 together with the identification information for identifying each of the pixels 321 in the light receiving pixel unit 320. Alternatively, using the equations (39) to (42) and (49) to (54), from each of the reference background light voltages VA1, VA2, and VA3, the reference reflected light voltages VC1, VC2, and VC3, and the reference reference voltages VB1, VB2, VB3, VD1, VD2, and VD3, the reference background light control voltages VPCB1, VPCB2, and VPCB3, the reference background light non-control voltages VPFB1, VPFB2, and VPFB3, the reference reflected light control voltages VPCL1, VPCL2, and VPCL3, and the reference reflected light non-control voltages VPFL1, VPFL2, and VPFL3 are respectively obtained, and are written and stored in the correction parameter storage unit 43 together with the identification information for identifying each of the pixels 321 in the light receiving pixel unit 320. Further, each of the reference background light voltages VA1, VA2, and VA3, the reference reflected light voltages VC1, VC2, and VC3, and the reference reference voltages VB1, VB2, VB3, VD1, VD2, and VD3 is written and stored in the correction parameter storage unit 43 together with the identification information for identifying each of the pixels 321 in the light receiving pixel unit 320. Furthermore, the reference background light control voltages VPCB1, VPCB2, and VPCB3, the reference background light non-control voltages VPFB1, VPFB2, and VPFB3, the reference reflected light control voltages VPCL1, VPCL2, and VPCL3, and the reference reflected light non-control voltages VPFL1, VPFL2, and VPFL3, which are obtained with reference to these, may also be written and stored in the correction parameter storage unit 43.

[0157] Then, in the frame for measuring the distance, every time the distance calculation unit 42 acquires each of the pixel signals VQ1, VQ2, and VQ3, it refers to the reference background photovoltages VA1, VA2, and VA3, the reference reflected photovoltages VC1, VC2, and VC3, the reference reference voltages VB1, VB2, VB3, VD1, VD2, and VD3, or the reference background light control voltages VPCB1, VPCB2, and VPCB3, the reference background light non-control voltages VPFB1, VPFB2, and VPFB3, the reference reflected light control voltages VPCL1, VPCL2, and VPCL3, and the reference reflected light non-control voltages VPFL1, VPFL2, and VPFL3 stored in the correction parameter storage unit 43, and solves the above simultaneous ternary linear equations to calculate the background light control voltage VCB1 and the reflected light control voltages VCL2 and VCL3. Alternatively, the solutions obtained by solving the above simultaneous ternary linear equations are stored in the storage unit or described in the program to calculate the background light control voltage VCB1, the reflected light control voltages VCL2, and VCL3. Further, the distance calculation unit 42 uses each of the background light control voltage VCB1 and the reflected light control voltages VCL2 and VCL3 to calculate the distance L between the distance image sensor 32 and the subject S according to the following equation (63) corresponding to equation (1) described above. L = ((V3 - V1) / (V2 + V3 - 2V1)) × Dm = (((VCB1 + VCL3) - VCB1) / ((VCB1 + VCL2) + (VCB1 + VCL3) - 2 × VCB1)) × Dm = (VCL3 / (VCL2 + VCL3)) × Dm …(63) In the above equation (63), Dm is (c / 2)Tw. Tw is the pulse width of the optical pulse PO.

[0158] According to the present embodiment, even if the ratio of the amount of non-control charge accumulated without passing through the read gate transistor G to the amount of control charge accumulated in the charge storage unit CS through the read gate transistor G increases, or even if the area of the pixel 321 is reduced, even if the non-control charges included in each of the charges accumulated in the charge storage unit CS vary greatly in amount depending on the incident angle of the incident light, each of the effects can be eliminated, and the distance L between the distance image capturing device 1 and the subject S can be obtained with the same accuracy as when the area of the pixel is not reduced, or with higher accuracy. Further, according to the present embodiment, after the lens 31 is attached and the distance image capturing device 1 is assembled, the reference background light voltages VA1, VA2, and VA3, the reference reflected light voltages VC1, VC2, and VC3, and the reference reference voltages VB1, VB2, VB3, VD1, VD2, and VD3 can be measured without using special calibration equipment, and the individual characteristics of the attached lens 31 and the difference in the relative position between the attached lens 31 and the distance image sensor 32, etc. can be accommodated, and the reflected light control voltages VCL2 and VCL3 corresponding to the amount of charge generated by the reflected light RL, collected by the photoelectric conversion element PD, and distributed to the charge storage units CS2 and CS3 by the read gate transistors G2 and G3, which do not include the background light non-control voltage VFB and the reflected light non-control voltage VFL, can be calculated easily and accurately.

[0159] <Seventh Embodiment> The distance image capturing device according to the seventh embodiment has the same configuration as the sixth embodiment described above, but the pixel 321 of the sixth embodiment has three pixel signal reading units RU1, RU2, and RU3 as shown in FIG. 3, whereas in the present embodiment, the pixel signal reading unit has four, namely RU1, RU2, RU3, and RU4 (not shown). Each of the pixel signal reading units RU1, RU2, RU3, and RU4 includes charge storage units CS1, CS2, CS3, and CS4 (not shown), respectively. In the case of the configuration of this embodiment, the charge storage unit CS1, similar to the sixth embodiment, distributes and accumulates the background light charges collected by the photodiode PD by the incident light due to only the background light at the distribution time Tw1 (the same time width as the pulse width Tw, and the same applies to Tw2, Tw3, and Tw4 shown later). Each of the distribution times Tw1, Tw2, Tw3, and Tw4 (not shown) is the pulse width of the storage drive signal TX1 applied to the read gate transistor G1, the storage drive signal TX2 applied to the read gate transistor G2, the storage drive signal TX3 applied to the read gate transistor G3, and the storage drive signal TX4 (not shown) applied to the read gate transistor G4 (not shown). Each of Tw1, Tw2, Tw3, and Tw4 is the same as the pulse width Tw of the optical pulse PO. In the charge storage unit CS1, similar to the timing chart of FIG. 4, the background light charges collected by the photodiode PD are distributed at the distribution time Tw1 before the optical pulse PO is irradiated. In each of the charge storage units CS2, CS3, and CS4, similar to the timing chart of FIG. 4, the background light charges collected by the photodiode PD and the charges corresponding to the reflected light RL are distributed at the distribution times Tw2 when the optical pulse PO is irradiated, the next distribution time Tw3, and the next distribution time Tw4, respectively.

[0160] FIG. 16 is a conceptual diagram showing the relationships between the charge amounts QCB1, QCB2, QCB3, QCB4, the charge amounts QFB1, QFB2, QFB3, QFB4, the charge amounts QCL1 (not shown, described later), QCL2, QCL3, QCL4 (not shown, described later), and the charge amounts QFL1, QFL2, QFL3, QFL4 in the respective charge amounts Q1, Q2, Q3, Q4 accumulated in the charge storage units CS1, CS2, CS3, and CS4. That is, each of the charge amounts QCB1, QCB2, QCB3, and QCB4 is the charge amount of control charges generated by background light, collected by the photoelectric conversion element PD, and distributed and stored in the charge storage units CS1, CS2, CS3, and CS4 respectively by the readout gate transistors G1, G2, G3, and G4. Further, each of the charge amounts QFB1, QFB2, QFB3, and QFB4 is the charge amount of non-control charges generated by background light and flowing into and stored in the charge storage units CS1, CS2, CS3, and CS4 respectively. Each of the charge amounts QCL1 (not shown, described later), QCL2, QCL3, and QCL4 (not shown, described later) is the charge amount of control charges generated by reflected light, collected by the photoelectric conversion element PD, and distributed and stored in the charge storage units CS1, CS2, CS3, and CS4 respectively by the readout gate transistors G1, G2, G3, and G4. Each of the charge amounts QFL1, QFL2, QFL3, and QFL4 is the charge amount of non-control charges generated by reflected light and flowing into and stored in the charge storage units CS1, CS2, CS3, and CS4 respectively.

[0161] Here, the control charges due to the reflected light RL are the charges that the charges collected by the photodiode PD are distributed to each of the charge storage units CS1, CS2, CS3, and CS4 through the readout gate transistors G1, G2, G3, and G4 respectively (see FIGS. 3 and 12). Further, the non-control charges due to the reflected light RL are the charges that flow in without passing through the readout gate transistors G1, G2, G3, and G4. Similarly, the control charges due to the background light are the charges that the charges collected by the photodiode PD are distributed to each of the charge storage units CS1, CS2, CS3, and CS4 through the readout gate transistors G1, G2, G3, and G4 respectively. Further, the non-control charges due to the background light are the charges that flow in without passing through the readout gate transistors G1, G2, G3, and G4.

[0162] In this embodiment, in addition to the reference background light control voltages VPCB1, VPCB2, and VPCB3 and the reference background light non-control voltages VPFB1, VPFB2, and VPFB3 in the sixth embodiment, a reference background light control voltage VPCB4 corresponding to the background light charge amount QPCB4 generated by background light and distributed from the photoelectric conversion element PD via the read gate transistor G4 and accumulated in the charge accumulation unit CS4, and a reference background light non-control voltage VPFB4 corresponding to the non-control charge amount QPFB4 generated by background light and flowing into and accumulated in the charge accumulation unit CS4 in an uncontrolled state are acquired in advance, for example, at the time of shipment or startup of the distance image capturing device 1, that is, before capturing a distance image, and written and stored in the correction parameter storage unit 43. Alternatively, a reference background light voltage VA acquired in a reference background light voltage acquisition frame and a reference reference voltage VB acquired in a reference reference voltage acquisition frame are acquired in advance before capturing a distance image, and written and stored in the correction parameter storage unit 43. Of course, both the reference background light control voltage VPCB and the reference background light non-control voltage VPFB, and the reference background light voltage VA and the reference reference voltage VB may be acquired in advance before capturing a distance image, and written and stored in the correction parameter storage unit 43. Similarly, in addition to the reference reflected light control voltages VPCL1, VPCL2, and VPCL3 and the reference reflected light non-control voltages VPFL1, VPFL2, and VPFL3 in the sixth embodiment, a reference reflected light control voltage VPCL4 corresponding to the reflected light charge amount QPCL4 generated by the reflected light RL, collected by the photoelectric conversion element PD, and distributed via the read gate transistor G4, and a reference reflected light non-control voltage VPFL4 corresponding to the non-control charge amount QPFL4 generated by the reflected light and flowing into and accumulated in the charge accumulation unit CS4 in an uncontrolled state are acquired in advance, for example, at the time of shipment or startup of the distance image capturing device 1, that is, before capturing a distance image, and written and stored in the correction parameter storage unit 43. Alternatively, a reference reflected light voltage VC acquired in a reference reflected light voltage acquisition frame and a reference reference voltage VD acquired in a reference reference voltage acquisition frame are acquired in advance before capturing a distance image, and written and stored in the correction parameter storage unit 43. Of course, both the reference reflected light control voltage VPCL and the reference reflected light non-control voltage VPFL, and the reference reflected light voltage VC and the reference reference voltage VD may be acquired in advance before imaging the distance image, and written and stored in the correction parameter storage unit 43.

[0163] Next, the non-control voltage due to the non-control charge flowing in without passing through the read gate transistor in the present embodiment is corrected, and only the control voltage due to the charge amount of the control charge generated by the reflected light RL and collected by the photoelectric conversion element PD and distributed to the charge storage unit CS through the read gate transistor G is extracted, and the process of calculating the distance between the subject S and the distance imaging device based on the charge amount will be described. The pixel signals VQ1, VQ2, VQ3, and VQ4 corresponding to the charge amounts Q1, Q2, Q3, and Q4 respectively accumulated in the charge storage units CS1, CS2, CS3, and CS4 in FIG. 16 are represented by the following equations (64) to (67). VQ1 = VCB1 + VCL1 + VFB1 + VFL1 …(64) VQ2 = VCB2 + VCL2 + VFB2 + VFL2 …(65) VQ3 = VCB3 + VCL3 + VFB3 + VFL3 …(66) VQ4 = VCB4 + VCL4 + VFB4 + VFL4 …(67) In the above equations (64) to (67), VCB is the background light control voltage corresponding to the control charge amount QCB generated by the background light, distributed from the photoelectric conversion element PD by the read gate transistor G, and accumulated in the charge storage unit CS. VCL is the reflected light control voltage corresponding to the control charge amount QCL generated by the reflected light RL, distributed from the photoelectric conversion element PD by the read gate transistor G, and accumulated in the charge storage unit CS. VFB is the background light non-control voltage corresponding to the non-control charge amount QFB generated by the background light and flowing into and accumulating in the charge storage unit CS in a non-controlled state. VFL is the reflected light non-control voltage corresponding to the non-control charge amount QFL generated by the reflected light RL and flowing into and accumulating in the charge storage unit CS in a non-controlled state. In the above-described equation (64), since the optical pulse PO is emitted after the charge is distributed from the photoelectric conversion element PD to the charge storage unit CS1, the reflected light RL does not enter the distance image sensor 32 when the charge is distributed to the charge storage unit CS1. Therefore, the control charge amount QCB1 and the non-control charge amount QFB1 due to the background light and the non-control charge amount QFL1 due to the reflected light RL are accumulated as the charge amount Q1 in the charge storage unit CS1, and the control charge amount QCL1 due to the reflected light RL is not accumulated. For this reason, the reflected light control voltage VCL1 becomes 0 as shown by the following equation (68). Also, in the present embodiment, since the state in which the reflected light RL enters the distance image sensor 32 is taken as an example at the timing when the charge is distributed from the photoelectric conversion element PD to the charge storage units CS2 and CS3, the control charge amount QCB4 and the non-control charge amount QFB4 due to the background light and the non-control charge amount QFL4 due to the reflected light RL are also accumulated in the charge storage unit CS4, and the control charge amount QCL4 due to the reflected light RL is not accumulated. For this reason, in the case of this example, the reflected light control voltage VCL4 becomes 0 as shown by the following equation (68), similarly to the reflected light control voltage VCL1. VCL1 = VCL4 = 0 …(68)

[0164] Also, each of the background light control voltages VCB1, VCB2, VCB3, and VCB4 is the same as shown in the following equation (69) because the charge generated by the background light and collected by the photoelectric conversion element PD is distributed to the charge storage units CS1, CS2, CS3, and CS4 via the readout gate transistors G1, G2, G3, and G4 by the accumulation drive signals TX1, TX2, TX3, and TX4 having the same pulse width. VCB1 = VCB2 = VCB3 = VCB4 …(69) As a result, each of equations (64) to (67) becomes the following equations (70), (71), (72), and (73), respectively. VQ1 = VCB1 + VFB1 + VFL1 …(70) VQ2 = VCB1 + VCL2 + VFB2 + VFL2 …(71) VQ3 = VCB1 + VCL3 + VFB3 + VFL3 …(72) VQ4 = VCB1 + VFB4 + VFL4 …(73)

[0165] The pixel signals VQ1, VQ2, VQ3, and VQ4 described above are the actually measured values measured in the frame for distance measurement and are known values. However, the background light control voltage VCB1, the background light non-control voltages VFB1, VFB2, VFB3, and VFB4, the reflected light control voltages VCL2 and VCL3, and the reflected light non-control voltages VFL1, VFL2, VFL3, and VFL4, which are the parameters constituting each of the pixel signals VQ1, VQ2, VQ3, and VQ4, are unknown because they are mixed as electric charge amounts as shown in FIG. 16. If each of the background light control voltage VCB1, the reflected light control voltages VCL2 and VCL3 among the above unknowns can be obtained, the distance between the subject S and the distance imaging device 1 can be calculated.

[0166] Then, similar to the sixth embodiment, the background light correction coefficient R_B between the reference background light non-control voltage VPFB and the background light non-control voltage VFB is represented by the following equation (74) according to the relationship that the ratio of the background light intensities of the reference background light voltage acquisition frame (FIG. 5(A)), the reference reference voltage acquisition frame (FIG. 5(B)), and the frame for measuring the distance is the same. R_B = VCB1 / (VA1 - VB1) …(74) Accordingly, each of the background light non-control voltages VFB1, VFB2, VFB3, and VFB4 corresponding to the background light is represented by the following equations (75), (76), (77), and (78), respectively. VFB1 = R_B × VPFB1 = R_B × VB1 …(75) VFB2 = R_B × VPFB2 = R_B × VB2 …(76) VFB3 = R_B × VPFB3 = R_B × VB3 …(77) VFB4 = R_B × VPFB4 = R_B × VB4 …(78) According to each of the above equations (75), (76), (77), and (78), each of the background light non-control voltages VFB1, VFB2, VFB3, and VFB4 at the time of measuring the distance in the frame can be obtained by the reference background light non-control voltages VPFB1, VPFB2, VPFB3, VPFB4 and the background light correction coefficient R_B. Also, the background light correction coefficient R_B of equation (74) can also be obtained by referring to the desired background light control voltage VCB1, the known reference background light voltage VA1 and the reference reference voltage VB1 stored in the correction parameter storage unit 43.

[0167] Also, similar to the sixth embodiment, the reflection light correction coefficient R_L between the reference reflection light non-control voltage VPFL and the reflection light non-control voltage VFL is represented by the following equation (79) according to the relationship that the ratio of the intensities of the reflection lights in the reference reflection light voltage acquisition frame (FIG. 10(A)) and the reference reference voltage acquisition frame (FIG. 10(B)) is the same as that in the frame for measuring the distance. R_L=(VCL2+VCL3) / ((VC2-VD2)+(VC3-VD3)) …(79) Thereby, each of the reflection light non-control voltages VFL1, VFL2, VFL3, and VFL4 corresponding to the reflection light is represented by each of the following equations (80), (81), (82), and (83). VFL1=R_L×VPFL1=R_L×VD1 …(80) VFL2=R_L×VPFL2=R_L×VD2 …(81) VFL3=R_L×VPFL3=R_L×VD3 …(82) VFL4=R_L×VPFL4=R_L×VD4 …(83)

[0168] Similarly, according to equations (80) to (83), each of the reflection light non-control voltages VFL1, VFL2, VFL3, and VFL4 is represented by the reflection light correction count ratio R_L and each of the reference reflection light non-control voltages VPFL1, VPFL2, VPFL3, and VPFL4. Also, the background light correction coefficient \(R_L\) in the formula (79) can be obtained by referring to the desired reflected light control voltages \(V_{CL2}\) and \(V_{CL3}\), and the known reference reflected light voltages \(V_C2\) and \(V_C3\), and reference reference voltages \(V_D2\) and \(V_D3\) stored in the correction parameter storage unit 43. From the above, each of the unknown background light non-control voltages \(V_{FB1}\), \(V_{FB2}\), \(V_{FB3}\), and \(V_{FB4}\), and reflected light non-control voltages \(V_{FL1}\), \(V_{FL2}\), \(V_{FL3}\), and \(V_{FL4}\) forms a system of linear equations consisting of the reference background light voltage \(V_A1\), reference reference voltages \(V_B1\), \(V_B2\), \(V_B3\), \(V_B4\), reference reflected light voltages \(V_C2\) and \(V_C3\), and reference reference voltages \(V_D1\), \(V_D2\), \(V_D3\), and \(V_D4\) stored in the correction parameter storage unit 43, the desired background light control voltage \(V_{CB1}\), and the reflected light control voltages \(V_{CL2}\) and \(V_{CL3}\). That is, the distance calculation unit 42 solves the above-described system of linear equations to obtain the background light control voltage \(V_{CB1}\), and the reflected light control voltages \(V_{CL2}\) and \(V_{CL3}\), and calculates the distance \(L\) between the distance image capturing device 1 and the subject S. Alternatively, the solutions obtained by solving the above system of linear equations are stored in the storage unit or described in the program to obtain the background light control voltage \(V_{CB1}\), the reflected light control voltages \(V_{CL2}\) and \(V_{CL3}\), and calculate the distance \(L\) between the distance image capturing device 1 and the subject S.

[0169] As described above, in the reference background light voltage acquisition frame (Fig. 5(A)), reference reference voltage acquisition frame (Fig. 5(B)), reference reflected light voltage acquisition frame (Fig. 10(A)), and reference reference voltage acquisition frame (Fig. 10(B)), the reference background light voltages \(V_A1\), \(V_A2\), \(V_A3\), and \(V_A4\), the reference reflected light voltages \(V_C1\), \(V_C2\), \(V_C3\), and \(V_C4\), and the reference reference voltages \(V_B1\), \(V_B2\), \(V_B3\), \(V_B4\), \(V_D1\), \(V_D2\), \(V_D3\), and \(V_D4\) are acquired as measured values, and written and stored in the correction parameter storage unit 43 together with the identification information for identifying each pixel 321 in the light receiving pixel unit 320. Alternatively, the distance calculation unit 42 calculates, from each of the reference background light voltages VA1, VA2, VA3, and VA4, the reference reflected light voltages VC1, VC2, VC3, and VC4, and the reference reference voltages VB1, VB2, VB3, VB4, VD1, VD2, VD3, and VD4, the reference background light control voltages VPCB1, VPCB2, VPCB3, and VPCB4, the reference reflected light control voltages VPCL1, VPCL2, VPCL3, and VPCL4, the reference background light non-control voltages VPFB1, VPFB2, VPFB3, and VPFB4, and the reference reflected light non-control voltages VPFL1, VPFL2, VPFL3, and VPFL4, respectively, and writes and stores them in the correction parameter storage unit 43 together with the identification information for identifying each of the pixels 321 in the light receiving pixel unit 320.

[0170] Then, in the frame for measuring the distance, each time the distance calculation unit 42 acquires each of the pixel signals VQ1, VQ2, VQ3, and VQ4, it refers to the reference background light voltages VA1, VA2, VA3, and VA4, the reference reflected light voltages VC1, VC2, VC3, and VC4, the reference reference voltages VB1, VB2, VB3, VB4, VD1, VD2, VD3, and VD4, or the reference background light control voltages VPCB1, VPCB2, VPCB3, and VPCB4, the reference background light non-control voltages VPFB1, VPFB2, VPFB3, and VPFB4, the reference reflected light control voltages VPCL1, VPCL2, VPCL3, and VPCL4, and the reference reflected light non-control voltages VPFL1, VPFL2, VPFL3, and VPFL4 stored in the correction parameter storage unit 43, and solves the above simultaneous linear equations to calculate the background light control voltage VCB1 and the reflected light control voltages VCL2 and VCL3. Alternatively, the solutions obtained by solving the above simultaneous three-variable linear equations are stored in the storage unit or described in the program, and the background light control voltage VCB1, the reflected light control voltages VCL2, and VCL3 are calculated. According to the present embodiment, even if the ratio of the amount of non-control charge accumulated without passing through the read gate transistor G to the amount of control charge accumulated in the charge storage unit CS through the read gate transistor G increases, or even if the area of the pixel 321 is reduced, non-control charges included in each of the charges accumulated in the charge storage unit CS and having different charge amounts depending on the incident angle of the incident light can be eliminated even if they vary greatly, and the distance L between the subject S and the distance image capturing device 1 can be obtained with the same or higher accuracy as in the case where the area of the pixel is not reduced. According to the sixth embodiment, even if there is a large variation in non-control charge when the pixel signal readout unit RU has three elements, the distance L between the subject S and the distance image capturing device 1 can be obtained with high distance accuracy. Also, according to the seventh embodiment, even when the pixel signal readout unit RU has four elements and there is a large variation in non-control charge, the distance L between the subject S and the distance image capturing device 1 can be obtained with high distance accuracy. As is clear from the description of the seventh embodiment, similar to the case where the pixel signal readout unit RU has four elements, even when the pixel signal readout unit RU has five, six, or more than a plurality of elements and there is a large variation in non-control charge, the distance L between the subject S and the distance image capturing device 1 can be obtained with high distance accuracy. That is, according to the present embodiment, when the pixel signal readout unit RU has N (N is an integer of 3 or more) elements, even if there is a large variation in non-control charge, high distance accuracy can be achieved in distance measurement.

[0171] <Eighth Embodiment> The eighth embodiment, similar to the sixth embodiment, subtracts and corrects from the pixel signals VQ1, VQ2, and VQ3 each of a voltage component corresponding to non-controlled charges generated by the reflected light RL obtained by reflecting the optical pulse PO (pulse width Tw) by the subject S and flowing into each of the charge accumulation units CS1, CS2, and CS3, a voltage component corresponding to controlled charges generated by the background light and distributed to each of the charge accumulation units CS1, CS2, and CS3, and a voltage component corresponding to non-controlled charges generated by the background light and flowing into each of the charge accumulation units CS1, CS2, and CS3, that is, calculates a voltage (reflected light control voltages VCL2 and VCL3 described later) corresponding only to the amount of charge generated by the reflected light. Also, in this embodiment, similar to the third embodiment, two frames of each of the first frame and the second frame are used to acquire a single distance image.

[0172] And in the present embodiment, as described in the sixth embodiment, in a light-shielded environment, with the light pulse PO irradiated, the reference reflected light voltage acquisition frame (FIG. 10(A)) and the reference reference voltage acquisition frame (FIG. 10(B)) are used to read from the photoelectric conversion element PD via the read gate transistor G1 and distribute the light to the charge storage unit CS1. The reference reflected light voltage VPCL1 corresponding to the reflected light charge amount QPCL1 generated by the reflected light RL, the reference reflected light voltage VPCL2 corresponding to the reflected light charge amount QPCL2 generated by the reflected light RL, which is distributed from the photoelectric conversion element PD via the read gate transistor G2 and stored in the charge storage unit CS2, and the reference reflected light voltage VPCL3 corresponding to the reflected light charge amount QPCL3 generated by the reflected light RL, which is distributed from the photoelectric conversion element PD via the read gate transistor G3 and stored in the charge storage unit CS3, and the reference reflected light non-control voltage VPFL1 corresponding to the non-control charge amount QPFL1 generated by the reflected light RL and flowing into and stored in the charge storage unit CS1 in an uncontrolled state, the reference reflected light non-control voltage VPFL2 corresponding to the non-control charge amount QPFL2 generated by the reflected light RL and flowing into and stored in the charge storage unit CS2 in an uncontrolled state, and the reference reflected light non-control voltage VPFL3 corresponding to the non-control charge amount QPFL3 generated by the reflected light RL and flowing into and stored in the charge storage unit CS3 in an uncontrolled state are, for example, acquired in advance at the time of shipment or startup of the distance image capturing device 1, that is, before capturing a distance image, and written and stored in the correction parameter storage unit 43.

[0173] In this embodiment, two frames for measuring distance are used, namely the first frame and the second frame. In the first frame, the light source device 21 is not controlled to irradiate the optical pulse PO, and the control circuit 322 is not controlled to output the accumulation drive signals TX1, TX2, and TX3. Instead, a reference voltage acquisition frame (corresponding to FIG. 5(B)) is used to acquire each of the reference voltages VB1, VB2, and VB3 corresponding to the non-controlled charges generated by the background light. Then, in the second frame, the light source device 21 is controlled to irradiate the optical pulse PO, and the control circuit 322 is controlled to output the accumulation drive signals TX1, TX2, and TX3, and each of the pixel signals VQ1, VQ2, and VQ3 is acquired to measure the distance. At this time, the order of executing the first frame and the second frame may be configured to be controlled as an operation with completely reversed order.

[0174] Each of the pixel signals VQ1, VQ2, and VQ3 can be expressed by equations (36), (37), and (38) as in the sixth embodiment. In this embodiment, since the intensity of the background light in the reference voltage acquisition frame for acquiring each of the reference voltages VB1, VB2, and VB3 corresponding to the non-controlled charges generated by the background light, that is, the first frame, and the background light in the second frame for measuring the distance is the same, equations (40), (41), and (42) are VB1 = VPFB1 = VFB1 …(84) VB2 = VPFB2 = VFB2 …(85) VB3 = VPFB3 = VFB3 …(86) can be expressed as. Thus, equation (43) becomes R_B = VFB / VPFB = 1. That is, the background light correction coefficient R_B becomes 1 (R_B = 1). By subtracting the reference voltage VB1 acquired in the first frame from the pixel signal VQ1, which is the voltage corresponding to the amount of charge accumulated in the charge accumulation unit CS1 in the second frame, the sum of the background light control voltage VCB1 and the reflected light non-control voltage VFL1 is obtained as the subtraction result. Here, the reflected light correction coefficient R_L is R_L = (reference voltage corresponding to the distance measurement frame (first frame)) / (reference voltage) = VB / VD = VB / VPFL …(87) It can also be expressed as If the reflected light correction coefficient RL is obtained, from "VFL1 = RL × VD1" in Equation (60), the non-controlled reflected light voltage VFL1 can be obtained, and only the desired background light control voltage VCB1 can be extracted from the sum of the background light control voltage VCB1 and the non-controlled reflected light voltage VFL1. Thus, similar to the sixth embodiment, Equations (36), (37), and (38) become a system of three linear equations with one unknown consisting of the desired background light control voltage VCB1, and the reflected light control voltages VCL2 and VCL3, and the background light control voltage VCB1, and the reflected light control voltages VCL2 and VCL3 can be accurately obtained as the solutions.

[0175] Then, the distance calculation unit 42 solves a system of linear equations based on the reference reflected light control voltages VPCL1, VPCL2, and VPCL3, the reference reference voltages VPFL1, VPFL2, and VPFL3, the calculated reference background light control voltages VPCB1, VPCB2, and VPCB3, the reference reference voltages VPFB1, VPFB2, and VPFB3, and the actually measured pixel signals VQ1, VQ2, and VQ3 in the correction parameter storage unit 43 to obtain the background light control voltage VCB1, and the reflected light control voltages VCL2 and VCL3. Alternatively, the solutions obtained by solving the above system of linear equations are stored in the storage unit or described in the program to obtain the background light control voltage VCB1, and the reflected light control voltages VCL2 and VCL3. Then, the distance calculation unit 42 uses Equation (63) to obtain the distance from the distance image capturing device 1 (distance image sensor 32) to the subject S based on the reflected light control voltages VCL2 and VCL3. As described above, according to the present embodiment, without executing the reference background light voltage acquisition frame (FIG. 5(A)) and the reference reference voltage acquisition frame (FIG. 5(B)) shown in the sixth embodiment before capturing the distance image, two frames are used when capturing the distance image. Thereby, when calculating the distance, the process of obtaining the background light correction coefficient RB (since RB = 1) can be omitted, and the load in the distance calculation can be reduced. Note that in this embodiment, the case where there are three pixel signal readout units RU has been described. However, as in the seventh embodiment, even when there are four pixel signal readout units RU, the solution can be obtained in the same manner. As described in the seventh embodiment, even when there are five, six, or a plurality of pixel signal readout units RU, the solution can be obtained in the same manner, and the distance L between the subject S and the distance image capturing device 1 can be obtained with high distance accuracy. That is, according to this embodiment, when there are N (N is an integer of 3 or more) pixel signal readout units RU, even if the variation in non-controlled charges is large, high distance accuracy can be realized in distance measurement.

[0176] <Ninth Embodiment> The ninth embodiment is premised on capturing a distance image in a shielded state (i.e., a state without background light) or a very dark state (i.e., a state where the intensity of background light is negligibly small). The voltage components corresponding to the non-controlled charges generated by the reflected light RL when the optical pulse PO (pulse width Tw) is reflected by the subject S and flows into each of the charge accumulation units CS1, CS2, and CS3 are subtracted and corrected from the pixel signals VQ1, VQ2, and VQ3 respectively. That is, it is a configuration for calculating the voltage (reflected light control voltages VCL2 and VCL3 described later) corresponding only to the amount of charge generated by the reflected light RL. Also, in this embodiment, as in the eighth embodiment, in order to acquire a single distance image, two frames, namely the first frame and the second frame, are used.

[0177] Then, in this embodiment, in the first frame of the two frames for measuring the distance, the light source device 21 is controlled to irradiate the optical pulse PO, and the control circuit 322 is controlled not to output the accumulation drive signals TX1, TX2, and TX3, and a reference voltage acquisition frame (corresponding to FIG. 10(B)) for acquiring each of the reference voltages VD1, VD2, and VD3 corresponding to the non-controlled charges generated by the reflected term RL is set. Then, in the second frame, the light source device 21 is controlled to irradiate the optical pulse PO, and the control circuit 322 is controlled to output the accumulation drive signals TX1, TX2, and TX3, and the pixel signals VQ1, VQ2, and VQ3 are acquired to measure the distance. At this time, the order of executing the first frame and the second frame may be configured to be controlled as an operation in which the order is completely reversed.

[0178] Each of the pixel signals VQ1, VQ2, and VQ3 can be expressed by equations (36), (37), and (38) as in the sixth embodiment. In this embodiment, since the second frame for measuring the distance also serves as the reference reflected light voltage acquisition frame, the intensity of the reflected light RL in the frame for measuring the distance and the reference reflected light voltage acquisition frame is the same, and the reflected light correction coefficient RL becomes 1 (RL = 1). That is, since the first frame is the reference reference voltage acquisition frame for obtaining the reference voltage VD of the reflected light, and the second frame for measuring the distance is used as the reference reflected light voltage acquisition frame, the total control charge voltage VCALL of the reflected light in the distance measurement frame and the total control charge voltage VPCALL of the reference reflected light are the same. Therefore, the numerator and denominator in equation (57) are the same. In this embodiment, the background light correction coefficient RB can be expressed by equation (45) as in the sixth embodiment. Since it is in a shielded state (that is, a state without background light) or a very dark state (that is, a state where the intensity of the background light is negligibly small), the background light correction coefficient RB becomes 0 (RB = 0). If the background light correction coefficient RB and the reflected light correction coefficient RL are determined, equations (36), (37), and (38) become a system of three linear equations with three unknowns consisting of the desired background light control voltage VCB1 and the reflected light control voltages VCL2 and VCL3 as in the sixth embodiment. Since the background light control voltage VCB1 and the reflected light control voltages VCL2 and VCL3 can be accurately obtained, the distance from the distance image capturing device 1 (distance image sensor 32) to the subject S can be obtained with high accuracy. Note that, in this embodiment, the case where there are three pixel signal readout units RU has been described. However, as in the seventh embodiment, even when there are four pixel signal readout units RU, solutions can be obtained in the same way. As described in the seventh embodiment, even when there are five, six, or a plurality of pixel signal readout units RU, solutions can be obtained in the same way, and the distance L between the subject S and the distance image capturing device 1 can be obtained with high distance accuracy. That is, according to this embodiment, when there are N (N is an integer of 3 or more) pixel signal readout units RU, even if the variation of the non-controlled charge is large, high distance accuracy can be realized in distance measurement.

[0179] As described above, the embodiments of the present invention have been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and designs and the like within the scope not departing from the gist of the present invention are also included.

Explanation of Reference Numerals

[0180] 1... Distance image capturing device 2... Light source unit 3... Light receiving unit 4... Distance image processing unit 21... Light source device 22... Diffusion plate 31... Lens 32... Distance image sensor 41... Timing control unit 42... Distance calculation unit 43... Correction parameter storage unit 320... Light receiving pixel unit 321... Pixel 322... Control circuit 323... Vertical scanning circuit 324... Horizontal scanning circuit 325... Pixel signal processing circuit 326... Pixel driving circuit C1, C2, C3... Charge storage capacitance CS1, CS2, CS3 Charge storage unit FD1, FD2, FD3... Floating diffusion G1, G2, G3... Readout gate transistor GD... Drain gate transistor O1, O2, O3... Output terminal P... Measurement space PD... Photoelectric conversion element PO... Optical pulse (irradiation light) RL... Reflected light RT1, RT2, RT3... Reset gate transistor RU1, RU2, RU3... Pixel signal readout unit S... Subject (object) SF1, SF2, SF3... Source follower gate transistor SL1, SL2, SL3... Selection gate transistor

Claims

1. A light source unit that irradiates irradiation light onto a measurement space that is a space to be measured; A distance image sensor that receives light including reflected light from an object in the measurement space as incident light, accumulates charges generated by the incident light for each pixel, and generates a distance image composed of the amounts of charges accumulated for each pixel; A distance image processing unit that stores a signal value based on non-controlled charges included in the amount of charge, which is not based on the control of accumulating the charge in the distance image sensor, from the amount of charge in the distance image, and corrects the distance to the object in the space using the stored signal value A distance image imaging device comprising the above.

2. The distance image sensor includes A photoelectric conversion element that collects charges generated in response to the incident light, and a charge accumulation unit that accumulates the charges in a frame period, and includes a pixel circuit that controls accumulation of the charges in the charge accumulation unit for each pixel, The distance image processing unit Subtracts an adjustment voltage corresponding to the amount of the non-controlled charge, which is the charge flowing into the charge accumulation unit without being controlled by the control by the pixel circuit, from an input voltage corresponding to the amount of charge accumulated in the charge accumulation unit, and measures the distance between the distance image sensor and the measurement object The distance image imaging device according to claim 1.

3. The charge accumulation unit includes At least one first charge accumulation unit that accumulates background light charges generated by receiving background light in the space, and two or more second charge accumulation units that accumulate reflected light charges generated by receiving the reflected light from the object of the irradiation light The distance image imaging device according to claim 2.

4. In a state where the irradiation light is not irradiated under predetermined ambient light, the charges controlled by the pixel circuit are accumulated in each of the first charge accumulation unit and the second charge accumulation unit in advance, and the reference background photovoltages obtained from each of the first charge accumulation unit and the second charge accumulation unit, and the reference reference voltages obtained from each of the first charge accumulation unit and the second charge accumulation unit without accumulating the charges controlled by the pixel circuit in each of the first charge accumulation unit and the second charge accumulation unit are stored in a storage unit The distance image capturing device according to claim 3.

5. The distance image sensor irradiates the measurement space with irradiation light, receives light including reflected light from an object in the measurement space as incident light, and accumulates the charges generated by the incident light in each of the first charge accumulation unit and the second charge accumulation unit. The distance image processing unit divides the background photovoltage generated by the charges accumulated in the first charge accumulation unit by the reference background photovoltage measured in advance and stored in the storage unit to obtain an adjustment ratio, and multiplies each of the reference reference voltages by the adjustment ratio to calculate the adjustment voltage for each of the input voltages. The distance image capturing device according to claim 4.

6. The frame period includes each of a first frame period and a second frame period. In the first frame period, The distance image sensor irradiates the measurement space with irradiation light, receives light including reflected light from an object in the measurement space as incident light, and accumulates the charges controlled by the pixel circuit in each of the first charge accumulation unit and the second charge accumulation unit. The distance image processing unit acquires the input voltage generated by the accumulated charges. In the second frame period, The distance image sensor irradiates the measurement space with irradiation light, receives light including reflected light from an object in the measurement space as incident light, and does not accumulate the charges controlled by the pixel circuit in each of the first charge accumulation unit and the second charge accumulation unit. The distance image processing unit acquires adjustment voltages corresponding to the non-controlled charges in each of the first charge storage unit and the second charge storage unit. The distance image capturing device according to claim 3.

7. In a state where the irradiation light is irradiated in a shielded environment, the charges controlled by the pixel circuit are accumulated in each of the first charge storage unit and the second charge storage unit, which are measured in advance. A storage unit is provided for storing each of the reference reflected light voltages obtained from each of the first charge storage unit and the second charge storage unit, and the reference reference voltages obtained from each of the first charge storage unit and the second charge storage unit without accumulating the charges controlled by the pixel circuit in each of the first charge storage unit and the second charge storage unit. The distance image capturing device according to claim 3.

8. The distance image processing unit divides the result of adding the input voltages respectively generated by the charges accumulated in each of the first charge storage unit and the second charge storage unit by the result of adding the reference reflected light voltages respectively to obtain an adjustment ratio, and multiplies the reference reference voltage by the adjustment ratio to calculate the adjustment voltages for the respective input voltages. The distance image capturing device according to claim 7.

9. The frame period includes each of a first frame period and a second frame period. In the first frame period, The distance image sensor irradiates the measurement space with irradiation light in a shielded environment, receives light including reflected light from an object in the measurement space as incident light, and accumulates the charges controlled by the pixel circuit in each of the first charge storage unit and the second charge storage unit. The distance image processing unit acquires the input voltages respectively generated by the charges accumulated in each of the first charge storage unit and the second charge storage unit. In the second frame period, The distance image sensor irradiates the measurement space with irradiation light in a light-shielded environment, receives light including reflected light from an object in the measurement space as incident light, and does not accumulate the charge controlled by the pixel circuit in each of the first charge storage unit and the second charge storage unit. The distance image processing unit acquires an adjustment voltage corresponding to the non-controlled charge by charge for each of the first charge storage unit and the second charge storage unit. The distance image capturing device according to claim 3.

10. In a state where the irradiation light is not irradiated under predetermined ambient light, the charge controlled by the pixel circuit is accumulated in each of the first charge storage unit and the second charge storage unit in advance, and the reference background photovoltage obtained from each of the first charge storage unit and the second charge storage unit, and the charge controlled by the pixel circuit is not accumulated in each of the first charge storage unit and the second charge storage unit, and the first reference reference voltage obtained as a reference reference voltage from each of the first charge storage unit and the second charge storage unit, Also, in a state where the irradiation light is irradiated in a light-shielded environment, the charge controlled by the pixel circuit is accumulated in each of the first charge storage unit and the second charge storage unit in advance, and the reference reflected photovoltage obtained from each of the first charge storage unit and the second charge storage unit, and the charge controlled by the pixel circuit is not accumulated in each of the first charge storage unit and the second charge storage unit, and the second reference reference voltage obtained as a reference reference voltage from each of the first charge storage unit and the second charge storage unit, and includes a storage unit that stores each of them. The distance image capturing device according to claim 3.

11. The distance image sensor irradiates the measurement space with irradiation light, receives light including reflected light from an object in the measurement space as incident light, and accumulates the charge generated by the incident light for each pixel. The distance image processing unit divides the input voltage generated by the accumulated charge by the reference background photovoltage measured in advance and stored in the storage unit to obtain a first adjustment ratio as an adjustment ratio, and multiplies each of the first reference reference voltages by the first adjustment ratio to calculate a first adjustment voltage for each of the input voltages. Further, the result of adding each of the input voltages is divided by the addition result of adding each of the reference reflected photovoltages to obtain a second adjustment ratio as an adjustment ratio, and each of the input voltages is calculated by multiplying the second reference reference voltage by the second adjustment ratio. A second adjustment voltage is calculated, and the first adjustment voltage and the second adjustment voltage are added to calculate the adjustment voltage. The distance image capturing device according to claim 10.

12. The pixel circuit controls each of the N (N is an integer of 3 or more) charge storage units to accumulate the charge measured in advance in a state where the irradiation light is not irradiated under predetermined ambient light, and the reference background photovoltage obtained from each of the charge storage units, and the pixel circuit does not control each of the charge storage units to accumulate the charge, and the first reference reference voltage obtained as the reference reference voltage from each of the charge storage units, The pixel circuit controls each of the N charge storage units to accumulate the charge measured in advance in a state where the irradiation light is irradiated in a light-shielded environment, and the reference reflected photovoltage obtained from each of the charge storage units, and the pixel circuit does not control each of the charge storage units to accumulate the charge, and the second reference reference voltage obtained as the reference reference voltage from each of the charge storage units each of which is obtained, The reference background light control voltage corresponding to the amount of control charge generated by the background light, collected by the photoelectric conversion element, and distributed and stored in each of the N charge storage units, obtained from each of the reference background light voltage and the first reference voltage; the reference background light non-control voltage corresponding to the amount of non-control charge that flows in without being distributed to each of the N charge storage units generated by the background light; the reference reflected light control voltage corresponding to the amount of control charge generated by the reflected light, collected by the photoelectric conversion element, and distributed and stored in each of the N charge storage units; and the reference reflected light non-control voltage corresponding to the amount of non-control charge that flows into and is stored in each of the N charge storage units without distribution, generated by the reflected light. Comprising a storage unit for storing each of them The distance image capturing device according to claim 3.

13. In a state of capturing the distance image, a first ratio is calculated and obtained as a ratio between the amount of background light in the pre-measured environment obtained from some or all of the reference background light voltage, the first reference voltage, the reference background light control voltage, and the reference background light non-control voltage stored in the storage unit, and the amount of background light in the state of capturing the image; a second ratio is calculated and obtained as a ratio between the amount of reflected light in the pre-measured environment obtained from some or all of the reference reflected light voltage, the second reference voltage, the reference reflected light control voltage, and the reference reflected light non-control voltage stored in the storage unit, and the amount of reflected light in the state of capturing the image; a system of simultaneous equations obtained using each of the first ratio and the second ratio is solved; and a distance calculation unit is provided that calculates a reflected light control voltage corresponding to the amount of control charge generated by the reflected light in the state of capturing the image and stored in the charge storage unit, and obtains the distance to the object. The distance image capturing device according to claim 12.

14. In a state where the irradiation light is irradiated in a shielded environment, perform control of the distribution to be stored in each of the N charge storage units measured in advance, and a reference reflected photovoltage corresponding to the charge amount including the control charge stored by the distribution and the non-control charge that flows in and is stored regardless of the distribution, and a second reference reference voltage obtained as a reference reference voltage corresponding to the charge amount of the non-control charge that flows in and is stored without performing control of storage in each of the charge storage units Obtain each of them, A reference reflected light control voltage corresponding to the charge amount of the control charge generated by the reflected light, collected by the photoelectric conversion element, and distributed and stored in each of the N charge storage units, obtained from each of the reference reflected photovoltage and the second reference reference voltage, and a reference reflected light non-control voltage corresponding to the charge amount of the non-control charge that is generated by the reflected light, flows into each of the N charge storage units without distribution, and is stored A storage unit that stores each of them The distance image capturing device according to claim 3.

15. In the acquisition state of acquiring the distance, the frame period includes each of a first frame period and a second frame period, In one of the two frame periods of the first frame period and the second frame period, The distance image sensor acquires a first reference reference voltage as a reference reference voltage corresponding to each of the charge amounts of the non-control charges stored in each of the N charge storage units without irradiating the irradiation light on the measurement space, In the other of the two frame periods, The distance image sensor irradiates the measurement space with the irradiation light, performs control to distribute charges to each of the N charge storage units, and from each of the charge storage units, acquires a voltage for obtaining a distance corresponding to the amount of charge including the control charge accumulated by the distribution and the non-control charge that has flowed in regardless of the distribution. The distance calculation unit calculates the ratio of the amount of reflected light in the pre-measured environment obtained from a part or all of the reference reflected light voltage, the second reference voltage, the first reference voltage, and the voltage for obtaining the distance stored in the storage unit to the amount of reflected light in the acquisition state, solves a system of simultaneous equations, calculates a reflected light control voltage corresponding to the amount of charge of the control charge, and obtains the distance to the object. The distance image capturing device according to claim 14.

16. When acquiring the distance in a shielded environment or a dark environment where ambient light can be ignored, the frame period includes each of a first frame period and a second frame period. In one of the two frame periods of the first frame period and the second frame period. The distance image sensor irradiates the irradiation light, receives incident light from the measurement space, accumulates non-control charges that flow into each of the N charge storage units without accumulating the charges controlled by the pixel circuit, and acquires a second reference voltage from each of the charge storage units. In the other of the two frame periods. The distance image sensor irradiates the irradiation light, receives incident light from the measurement space, distributes and accumulates charges from the photoelectric conversion element to each of the charge storage units under the control of the pixel circuit, and acquires a reference reflected light voltage that is also a voltage for obtaining a distance corresponding to the amount of charge including each of the non-control charge and the control charge accumulated under the control of the pixel circuit. Solve a system of simultaneous equations using the second reference voltage and the reference reflected light voltage, calculate only the reflected light control voltage corresponding to the amount of charge of the control charge, and obtain the distance to the object. Comprising a distance calculation unit. The distance image capturing device according to claim 3.

17. Further comprising a lens for incident light from the space to be incident, The distance image sensor receives the incident light through the lens The distance image capturing device according to any one of claims 1 to 16.

18. Further comprising a lens for incident light from the space to be incident, Since the incident light is incident on each of the pixels through the lens, corresponding to the characteristics of the lens, the pixels within a predetermined difference range of the adjustment ratio are grouped, and the median value of the reference reference voltages in the group is used as the reference reference voltage for all the pixels in the group The distance image capturing device according to any one of claims 5, 8 or 11.

19. Since the incident light is incident on each of the pixels through the lens, corresponding to the characteristics of the lens, a plurality of characteristics of the lens are stored in the storage unit The distance image capturing device according to any one of claims 5, 8, 11 or 18.

20. Since the incident light is incident on each of the pixels through the lens, corresponding to the characteristics of the lens, an adjustment function for outputting the adjustment ratio corresponding to each position of the pixel is stored in the storage unit The distance image capturing device according to any one of claims 5, 8, 11, 18 or 19.

21. The distance image sensor irradiates irradiation light from the light source unit onto a measurement space that is a space to be measured, receives light including reflected light from an object in the measurement space as incident light, accumulates charges generated by the incident light for each pixel, and generates a distance image having the amount of charge of the charges accumulated for each pixel; a distance image generation process, A distance image processing process for obtaining the distance to the object in the space based on the corrected charge amount obtained by removing the non-controlled charge included in the charge amount from the charge amount in the distance image without depending on the control for accumulating the charge in the distance image sensor A distance image capturing method including the above

Images