Range image capturing device and range image capturing method

Abstract

To provide a distance image capture device that captures a correct distance image without changing an optical structure in the device and without interrupting the size reduction, the height reduction, the higher definition, and the improvement of the quantum efficiency of the device.SOLUTION: A distance image capture device includes: a light source unit that irradiates a measurement space with optical pulses; a light-reception unit including a photoelectric conversion element that generates charges according to light entering from the measurement space, a plurality of pixel circuits including a plurality of charge accumulation units for accumulating charges in a frame period, and a pixel driving circuit that allocates and accumulates charges in the charge accumulation unit by performing on / off processing of a transfer transistor at a predetermined accumulation timing synchronizing with the emission of the optical pulses; and a distance calculation unit that calculates a distance between a subject and the light-reception unit in the measurement space in accordance with the amount of charges determined based on a first charge amount of the charges accumulated in the charge accumulation unit. The distance calculation unit calculates a distance by subtracting a second charge amount by noise charges being the accumulated charges other than the charges allocated and accumulated in the on / off processing of the transfer transistor from the first charge amount.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a distance image capturing device and a distance image capturing method. [Background technology]

[0002] Conventionally, there have been time-of-flight (ToF) distance imaging devices that utilize the known speed of light to measure the distance to a subject based on the flight time of light (see, for example, Patent Document 1). A ToF distance imaging device includes a light source unit that emits light and an imaging unit that includes a pixel array in which a plurality of pixel circuits that detect light for measuring distance are arranged in a two-dimensional matrix (array). Each of the pixel circuits has a photoelectric conversion element (e.g., a photodiode) that generates a charge corresponding to the intensity of light. With this configuration, the ToF range image pickup device can acquire (capture) information about the distance between itself and a subject, and an image of the subject, in a measurement space (three-dimensional space).

[0003] Furthermore, for use in mobile terminals such as smartphones and tablet terminals, there is a demand for the housing (imaging device) of the range image capturing device to be smaller and thinner (thinner). On the other hand, in order to obtain higher definition images from distance imaging devices, there is a demand for an increase in the number of pixels in the imaging section. Therefore, the angle of incidence of light incident on pixel circuits arranged at the ends of the pixel array is larger than the angle of incidence of light incident on pixel circuits arranged near the center.

[0004] As the angle of incidence of the incident light on the pixel circuit increases, the light is irradiated not only onto the photoelectric conversion element in the pixel circuit but also onto the circuitry surrounding the photoelectric conversion element, such as a charge accumulation portion that accumulates charge. Then, not only in the photoelectric conversion element but also in the peripheral circuit (including the charge storage section), charges corresponding to the irradiated light (hereinafter referred to as noise charges) are generated due to the photoelectric effect, and these charges flow into the charge storage section with a high potential.

[0005] This causes noise charges from the peripheral circuits to be added to the charges transferred from the photoelectric conversion element to the charge accumulation unit and used for distance measurement, reducing the accuracy of the distance calculated based on the charges accumulated in the charge accumulation unit. Therefore, as a measure to suppress the generation of the above-mentioned noise charge, the pixel circuit is configured so that incident light is not incident on peripheral circuits other than the photoelectric conversion element.

[0006] For example, as a measure to suppress the incidence of light on peripheral circuits other than the photoelectric conversion element in the pixel circuit, a configuration is used in which a microlens is formed above each pixel circuit, the microlens collects light, and the collected light is made incident on the photoelectric conversion element. In this way, the light incident on the pixel circuit is collected by the microlens and irradiated only on the photoelectric conversion element, suppressing the light from being irradiated on other peripheral circuits.

[0007] In addition, as a measure to suppress the incidence of light on peripheral circuits other than the photoelectric conversion elements in other pixel circuits, a configuration is adopted in FSI (Front Side Illuminated) CMOS image sensors in which a light-shielding layer made of a metal pattern is provided directly above the pixel circuit to shield circuit parts other than the photoelectric conversion elements from light. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-29054 Summary of the Invention [Problem to be solved by the invention]

[0009] However, in light collection by a microlens, due to the demand for a low profile of the range imaging device, as the range imaging device is made thinner, the thickness of the microlens must also be made correspondingly thinner. For this reason, the microlens does not have a sufficient curvature to focus the incident light onto the photoelectric conversion element, making it difficult to focus the light so that it is irradiated only onto the photoelectric conversion element.

[0010] Furthermore, in the configuration in which the above-mentioned light-shielding layer is provided, since in the FSI method a wiring layer for the wiring in the pixel circuit is arranged between the photoelectric conversion element and the microlens, the metal pattern of this wiring layer can be used to form a metal pattern for light-shielding. On the other hand, in the BSI (Back Side Illuminated) method, light is incident from the back surface of a semiconductor substrate on which pixel circuits are formed, rather than from the front surface of the semiconductor substrate.

[0011] For this reason, since no circuit is configured, a wiring layer does not exist on the back surface of the semiconductor substrate, and in the BSI configuration, it is not possible to form a metal pattern for shading using a metal pattern in the wiring layer, as is the case with the FSI method. Furthermore, in the BSI type configuration, as in Patent Document 1, in order to improve sensitivity to incident light and increase quantum efficiency, a configuration may be adopted in which a scattering / reflecting structure is provided to lengthen the optical path length of light within the semiconductor substrate. In this configuration, the light collected by the microlens is scattered within the semiconductor substrate, causing more light to be incident on the peripheral circuitry, which causes more noise charges to be generated in the peripheral circuitry other than the photoelectric conversion element.

[0012] The present invention has been made in consideration of the above circumstances, and aims to provide a distance image capturing device and a distance image capturing method that capture more accurate distance images without changing the optical configuration within the device and without hindering the miniaturization, low profile, high definition, and improved quantum efficiency of the device. [Means for solving the problem]

[0013] In order to solve the above-mentioned problems, the distance image pickup device of the present invention includes a light source unit that irradiates a measurement space with a light pulse, a photoelectric conversion element that generates an electric charge according to the light incident from the measurement space, Indicates the frame repeat period a light receiving unit having a plurality of pixel circuits each including a plurality of charge accumulation units that accumulate the charge in a frame period; a pixel drive circuit that performs on / off processing on each transfer transistor in each of the charge accumulation units to allocate and accumulate the charge at a predetermined accumulation timing synchronized with the irradiation of the light pulse; and a distance calculation unit that calculates a distance between the subject and the light receiving unit in the measurement space based on a first charge amount of the charge accumulated in each of the charge accumulation units, wherein the distance calculation unit calculates the distance by subtracting a second charge amount of noise charge, which is charge generated other than at the photoelectric conversion element, from each of the first charge amounts. a frame in which the amount of charge accumulated in the charge accumulation unit is acquired as the third amount of charge in a state in which the pixel driving circuit turns off the transfer transistor and does not accumulate charge from the photoelectric conversion element in the charge accumulation unit when the light pulse is not irradiated from the light source unit, and a frame in which the amount of charge accumulated in the charge accumulation unit is acquired as the second amount of charge in a state in which the pixel driving circuit turns off the transfer transistor and does not accumulate charge from the photoelectric conversion element in the charge accumulation unit after the light pulse is irradiated from the light source unit, and a frame in which the amount of charge accumulated in the charge accumulation unit is acquired as the second amount of charge in a state in which the pixel driving circuit turns off the transfer transistor and does not accumulate charge from the photoelectric conversion element in the charge accumulation unit when the light pulse is not irradiated from the light source unit, and a frame in which the amount of charge accumulated in the charge accumulation unit is acquired as the second amount of charge, and the distance calculation unit subtracts the third amount of charge from the second amount of charge to obtain a fourth amount of charge which is an amount of charge generated by the light pulse other than the photoelectric conversion element, and corrects the second amount of charge based on the fourth amount of charge to use in calculating the distance. It is characterized by: a light receiving unit having a light source unit that irradiates a measurement space with a light pulse; a photoelectric conversion element that generates a charge according to the light incident from the measurement space; a plurality of pixel circuits each having a plurality of charge accumulation units that accumulate the charge in a frame period that indicates a period in which frames are repeated; a pixel drive circuit that performs on / off processing on each transfer transistor in each of the charge accumulation units to allocate and accumulate the charge at a predetermined accumulation timing synchronized with the irradiation of the light pulse; and a distance calculation unit that calculates the distance between a subject in the measurement space and the light receiving unit using a charge amount determined by a first charge amount of the charge accumulated in each of the charge accumulation units, wherein the distance calculation unit calculates the distance by subtracting a second charge amount due to noise charge that is accumulated charge other than the charge allocated and accumulated by the on / off processing of the transfer transistor from each of the first charge amounts, and A frame in which the charge amount is acquired as the third charge amount is defined as a first noise charge amount acquisition frame, a frame in which the charge amount accumulated in the charge accumulation unit is acquired as the second charge amount in a state in which the pixel drive circuit turns off the transfer transistor after the light pulse is irradiated from the light source unit and charge is not accumulated in the charge accumulation unit from the photoelectric conversion element, is defined as a second noise charge amount acquisition frame, and the distance calculation unit subtracts the third charge amount from the second charge amount to acquire a fourth charge amount which is the charge amount generated by the light pulse at a location other than the photoelectric conversion element. and when the second number of allocations for each of the frames after the second noise charge amount acquisition frame changes with respect to the first number of allocations for each of the first noise charge amount acquisition frame and the second noise charge amount acquisition frame, the distance calculation unit divides the second number of allocations by the first number of allocations to obtain an adjustment coefficient, multiplies the fourth charge amount by the adjustment coefficient to calculate a fifth charge amount, adds the fifth charge amount to the third charge amount, and corrects the second charge amount for use in calculating the distance.

[0014] The distance image capturing device of the present invention is characterized in that one of the frames is a noise charge amount acquisition frame in which the second charge amount is acquired, and in the noise charge amount acquisition frame, after the light pulse is irradiated from the light source unit, the pixel drive circuit turns off the transfer transistor so that the charge accumulated in the charge accumulation unit is not distributed from the photoelectric conversion element to the charge accumulation unit, and in frames after the noise charge amount acquisition frame, the pixel drive circuit turns on and off the transfer transistor so that the charge is distributed to and accumulated in each of the charge accumulation units, resulting in the first charge amount.

[0015] The distance image capturing device of the present invention is characterized in that it divides each frame to generate a first sub-frame and a second sub-frame, and in the first sub-frame, after the light pulse is irradiated from the light source unit, the pixel driving circuit turns off the transfer transistor so that no charge is accumulated from the photoelectric conversion element in the charge accumulation unit, and the amount of charge accumulated in the charge accumulation unit is the second charge amount, and in the second sub-frame, the pixel driving circuit turns on and off the transfer transistor so that the charge is distributed and accumulated in each of the charge accumulation units, and is the first charge amount.

[0016] The distance image capturing device of the present invention is characterized in that the frame is composed of an accumulation period in which the charge is allocated and accumulated in each of the charge accumulation sections, and a readout period in which the amount of charge accumulated from each of the charge accumulation sections is read out, and the first sub-frame and the second sub-frame are each formed by shortening the accumulation period.

[0017] The distance image capturing device of the present invention comprises a light source unit that irradiates a measurement space with a light pulse, a photoelectric conversion element that generates a charge in response to the light incident from the measurement space, a plurality of pixel circuits each having a plurality of charge accumulation units that accumulate the charge in a frame period, and a pixel drive circuit that turns on and off each of the transfer transistors in each of the charge accumulation units at a predetermined accumulation timing synchronized with the irradiation of the light pulse to allocate and accumulate the charge, and a distance calculation unit that calculates the distance between a subject in the measurement space and the light receiving unit using a charge amount determined by a first charge amount of the charge accumulated in each of the charge accumulation units, and is characterized in that the distance calculation unit calculates the distance by subtracting a second charge amount due to noise charge, which is accumulated charge other than the charge allocated and accumulated by the on / off processing of the transfer transistors, from each of the first charge amounts.

[0018] The distance image capturing device of the present invention is characterized in that, when the second number of allocations in each of the frames after the second noise charge amount acquisition frame changes compared to the first number of allocations in each of the first noise charge amount acquisition frame and the second noise charge amount acquisition frame, the distance calculation unit divides the second number of allocations by the first number of allocations to obtain an adjustment coefficient, multiplies the fourth amount of charge by the adjustment coefficient to calculate a fifth amount of charge, adds the fifth amount of charge to the third amount of charge, and corrects the second amount of charge for use in calculating the distance.

[0019] The distance image pickup device of the present invention is characterized in that the light pulse irradiated from the light source section is a pulse of light in the near-infrared wavelength band with a predetermined width.

[0020] The distance image pickup device of the present invention is characterized in that the pixel circuit has a BSI (Back Side Illumination) structure.

[0021] The distance image pickup device of the present invention is characterized in that the pixel circuit has three or more charge storage sections. The distance image capturing device of the present invention is characterized in that the pixel circuit is provided with one or more charge drain transistors that drain charge from the photoelectric conversion element except for the period during which the charge is distributed and stored in each charge storage section.

[0022] The distance image capturing method of the present invention includes: picture The element drive circuit and a light receiving unit having , a light source unit; a distance calculation unit; and a distance image capturing method for controlling the distance image capturing device, the distance image capturing method including: a step of the light source unit irradiating a measurement space with a light pulse; and a step of the pixel circuit Indicates the frame repeat periodThe method includes a step of accumulating, in a frame period, charges generated by the photoelectric conversion elements in response to light incident from the measurement space in the charge accumulation units; a pixel driving step in which the pixel driving circuit performs on / off processing on each transfer transistor in each of the charge accumulation units at a predetermined accumulation timing synchronized with the irradiation of the light pulse to allocate and accumulate the charges; and a distance calculation step in which the distance calculation unit calculates a distance between the object in the measurement space and the light receiving unit from an amount of charge determined by a first amount of charge of the charges accumulated in each of the charge accumulation units, wherein the distance calculation unit calculates the distance by subtracting, from each of the first amounts of charge, a second amount of charge due to noise charge, which is accumulated charge other than the charges allocated and accumulated by the on / off processing of the transfer transistors. a frame in which the amount of charge accumulated in the charge accumulation unit is acquired as the third amount of charge in a state in which the pixel drive circuit turns off the transfer transistor and does not accumulate charge from the photoelectric conversion element in the charge accumulation unit when the light pulse is not irradiated from the light source unit, and a frame in which the amount of charge accumulated in the charge accumulation unit is acquired as the second amount of charge in a state in which the pixel drive circuit turns off the transfer transistor and does not accumulate charge from the photoelectric conversion element in the charge accumulation unit after the light pulse is irradiated from the light source unit, and a frame in which the amount of charge accumulated in the charge accumulation unit is acquired as the second amount of charge in a state in which the pixel drive circuit turns off the transfer transistor and does not accumulate charge from the photoelectric conversion element in the charge accumulation unit when the light pulse is not irradiated from the light source unit, and a frame in which the amount of charge accumulated in the charge accumulation unit is acquired as the second amount of charge, and in the distance calculation step, the distance calculation unit subtracts the third amount of charge from the second amount of charge to obtain a fourth amount of charge which is an amount of charge generated by the light pulse other than the photoelectric conversion element, and corrects the second amount of charge based on the fourth amount of charge to use in calculating the distance. It is characterized by: The present invention also provides a distance image capturing method for controlling a distance image capturing device including a light receiving unit having a plurality of pixel circuits each including a photoelectric conversion element and a plurality of charge accumulation units, and a pixel drive circuit, a light source unit, and a distance calculation unit, the method comprising the steps of: a step in which the light source unit irradiates a measurement space with a light pulse; a step in which the pixel circuit accumulates in the charge accumulation unit, in a frame period indicating a period in which frames are repeated, charges generated by the photoelectric conversion element in response to light incident from the measurement space; and a step in which the pixel drive circuit accumulates, at a predetermined accumulation timing synchronized with the irradiation of the light pulse. a pixel driving step of performing on / off processing of each transfer transistor in each of the charge accumulation units to allocate and accumulate the charges; and a distance calculation step of the distance calculation unit calculating a distance between the object and the light receiving unit in the measurement space from an amount of charge determined by a first amount of charge of the charges accumulated in each of the charge accumulation units, wherein the distance calculation unit calculates the distance by subtracting, from each of the first amounts of charge, a second amount of charge due to noise charge, which is accumulated charge other than the charges allocated and accumulated by on / off processing of the transfer transistors, and A frame in which the pixel driving circuit turns off the transfer transistor and does not store charge from the photoelectric conversion element in the charge storage unit while no light pulse is irradiated, and the amount of charge stored in the charge storage unit is acquired as a third amount of charge, is defined as a first noise charge amount acquisition frame. A frame in which the pixel driving circuit turns off the transfer transistor and does not store charge from the photoelectric conversion element in the charge storage unit after the light pulse is irradiated from the light source unit, and the amount of charge stored in the charge storage unit is acquired as the second amount of charge, is defined as a third noise charge amount acquisition frame. and in the distance calculation step, the distance calculation unit subtracts the third charge amount from the second charge amount to obtain a fourth charge amount, which is the charge amount of the charge generated by the light pulse at a location other than the photoelectric conversion element, and when a second number of allocations of the allocation in each of the frames after the second noise charge amount acquisition frame has changed with respect to a first number of allocations of the allocation in each of the first noise charge amount acquisition frame and the second noise charge amount acquisition frame, the distance calculation unit divides the second number of allocations by the first number of allocations and uses the result as an adjustment coefficient,The fourth charge amount is multiplied by the adjustment coefficient to calculate a fifth charge amount, and the fifth charge amount is added to the third charge amount to correct the second charge amount and use it in calculating the distance. [Effects of the Invention]

[0023] As described above, according to the present invention, it is possible to provide a distance image capturing device and a distance image capturing method that capture more accurate distance images without changing the optical configuration within the device and without hindering the miniaturization, low profile, high definition, and improved quantum efficiency of the device. [Brief explanation of the drawings]

[0024] [Figure 1] FIG. 1 is a block diagram showing a schematic configuration of a range image pickup device according to a first embodiment of the present invention. [Figure 2] 1 is a circuit diagram showing an example of the configuration of a pixel circuit 321 arranged in a range image sensor 32 in the range image pickup device according to the first embodiment of the present invention. FIG. [Figure 3] 10 is a timing chart showing the transfer of charges generated by the photoelectric conversion element PD to each of the charge accumulation sections CS. FIG. [Figure 4] 3 is a diagram showing an example of the arrangement (layout pattern) of each transistor of a pixel circuit 321 in this embodiment. FIG. [Figure 5]10A and 10B are diagrams illustrating a process of acquiring the amount of noise charge accumulated in a charge accumulation unit CS in the second embodiment. [Figure 6] 10A and 10B are conceptual diagrams illustrating the process of accumulating charges in a charge accumulation unit CS in a noise charge amount acquisition mode. [Figure 7] 10A and 10B are conceptual diagrams illustrating the process of accumulating charges in a charge accumulation unit CS in a distance measurement charge amount acquisition mode. [Figure 8] 10A and 10B are diagrams illustrating a process of acquiring the amount of noise charge accumulated in a charge accumulation unit CS in the second embodiment. [Figure 9] 10A and 10B are diagrams illustrating a process of acquiring the amount of noise charge accumulated in a charge accumulation unit CS in the third embodiment. [Figure 10] 13 is a conceptual diagram illustrating noise charges corresponding to reflected light RL in a charge accumulation unit CS in a distance measurement charge amount acquisition mode in the third embodiment. FIG. [Figure 11] 10 is a flowchart showing an example of the operation of a process for calculating the distance between the distance image sensor 32 and the subject S in the distance image pickup device 1 according to the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0025] First Embodiment A first embodiment of the present invention will be described below with reference to the drawings. Figure 1 is a block diagram showing the schematic configuration of a distance image pickup device according to a first embodiment of the present invention. The distance image pickup device 1 shown in Figure 1 comprises a light source unit 2, a light receiving unit 3, and a distance image processing unit 4. Figure 1 also shows a subject S, which is an object whose distance is to be measured by the distance image pickup device 1. The distance image pickup element is, for example, a distance image sensor 32 (described below) in the light receiving unit 3.

[0026] The light source section 2 irradiates a light pulse PO into a space to be photographed, in which a subject S, the distance of which is to be measured by the distance image pickup device 1, is present, under the control of the distance image processor 4. The light source section 2 is, for example, a surface-emitting semiconductor laser module such as a vertical cavity surface-emitting laser (VCSEL). The light source section 2 includes a light source device 21 and a diffuser plate 22.

[0027] The light source device 21 is a light source that emits laser light in a near-infrared wavelength band (for example, a wavelength band of 850 nm to 940 nm) that becomes the light pulse PO to be irradiated onto the subject S. The light source device 21 is, for example, a semiconductor laser light-emitting element. The light source device 21 emits pulsed laser light in response to control from the timing control unit 41. The diffusion plate 22 is an optical component that diffuses the laser light in the near-infrared wavelength band emitted by the light source device 21 to the extent of the surface to be irradiated onto the subject S. The pulsed laser light diffused by the diffusion plate 22 is emitted as a light pulse PO and is irradiated onto the subject S.

[0028] The light receiving unit 3 receives reflected light RL of the light pulse PO reflected by a subject S, the distance of which is to be measured in the range image pickup device 1, and outputs a pixel signal corresponding to the received reflected light RL. The light receiving unit 3 includes a lens 31 and a range image sensor 32. The lens 31 is an optical lens that guides the incident reflected light RL to the range image sensor 32. The lens 31 outputs the incident reflected light RL to the range image sensor 32 side, and causes the light to be received (incident) by pixel circuits provided in the light receiving region of the range image sensor 32.

[0029] The range image sensor 32 is an imaging element used in the range image capturing device 1. The range image sensor 32 includes a plurality of pixel circuits 321 in a two-dimensional light receiving area, and a pixel drive circuit 322 that controls each of the pixel circuits 321. The pixel circuit 321 includes one photoelectric conversion element (for example, a photoelectric conversion element PD described later), a plurality of charge accumulation units (for example, charge accumulation units CS1 to CS4 described later) corresponding to the one photoelectric conversion element, and components that distribute charge to each charge accumulation unit.

[0030] The range image sensor 32 distributes the charges generated by the photoelectric conversion elements to the respective charge accumulation sections under the control of the timing control section 41. The range image sensor 32 also outputs pixel signals according to the amount of charge distributed to the charge accumulation sections. The range image sensor 32 has a plurality of pixel circuits arranged in a two-dimensional matrix, and outputs pixel signals for one frame corresponding to each pixel circuit.

[0031] The distance image processing unit 4 controls the distance image pickup device 1 and calculates the distance to the subject S. Distance image processing unit 4 includes timing control unit 41, distance calculation unit 42, and measurement control unit 43. The timing control unit 41 controls the timing of outputting various control signals required for distance measurement in accordance with the control of the measurement control unit 43. The various control signals here include, for example, a signal for controlling the irradiation of the light pulse PO, a signal for distributing the reflected light RL to multiple charge accumulation units, a signal for controlling the number of distributions per frame, etc. The number of distributions is the number of times the process of distributing electric charges to the charge accumulation units CS (see FIG. 3) is repeated.

[0032] The distance calculation unit 42 outputs distance information calculated based on the pixel signals output from the range image sensor 32 under the control of the measurement control unit 43. The distance calculation unit 42 calculates the delay time Td from when the light pulse PO is emitted until when the reflected light RL is received, based on the amount of charge accumulated in the multiple charge accumulation units CS. The distance calculation unit 42 calculates the distance from the range image pickup device 1 to the range image pickup unit S in accordance with the calculated delay time Td.

[0033] The measurement control unit 43 selects the mode of each frame repeated in a frame cycle between a noise charge amount acquisition mode, which is a frame for acquiring charge noise, and a distance measurement charge amount acquisition mode, which is a normal frame for performing distance measurement. The measurement control unit 43 controls the timing in the timing control unit 41 and the calculation in the distance calculation unit 42 in accordance with the noise charge amount acquisition mode and the distance measurement charge amount acquisition mode (described in detail later). Here, the noise charge refers to the charge generated in an area other than the photoelectric conversion element by incident light (background light and reflected light) from the measurement space and accumulated in the charge accumulation unit CS. In other words, the noise charge refers to the accumulated charge other than the charge distributed and accumulated by the on / off processing of the transfer transistor G.

[0034] That is, the distance image pickup device according to this embodiment calculates the distance between the subject and the distance image sensor 32 based on the charges accumulated in the charge accumulation unit CS. Therefore, in the calculation of calculated distance measurement, noise charges due to background light and the like are accumulated in the charge accumulation unit CS until the time of readout in each frame, regardless of the charges read out from the photoelectric conversion elements at predetermined timings, which reduces the accuracy of the calculated distance.

[0035] With this configuration, in the distance image capturing device 1, the light source unit 2 irradiates a light pulse PO in the near-infrared wavelength band onto the subject S, and the light receiving unit 3 receives the reflected light RL reflected by the subject S, and the distance image processing unit 4 outputs distance information measuring the distance between the subject S and the distance image capturing device 1. Although FIG. 1 shows the distance image pickup device 1 having the distance image processing unit 4 built therein, the distance image processing unit 4 may be an element provided outside the distance image pickup device 1.

[0036] Here, we will explain the configuration of the pixel circuit 321 in the range image sensor 32. Figure 2 is a circuit diagram showing an example of the configuration of the pixel circuit 321 arranged in the range image sensor 32 in the range image pickup device according to the first embodiment of the present invention. The pixel circuit 321 in Figure 2 is an example configuration including four pixel signal readout units RU1 to RU4.

[0037] The pixel circuit 321 includes one photoelectric conversion element PD, a charge discharging transistor GD (GD1 and GD2 described below), and four pixel signal readout units RU (RU1 to RU4) that output voltage signals from corresponding output terminals O. Each pixel signal readout unit RU includes a transfer transistor G, a floating diffusion FD, a charge storage capacitance C, a reset transistor RT, a source follower transistor SF, and a selection transistor SL. The floating diffusion FD and the charge storage capacitance C form a charge storage unit CS.

[0038] 2, the pixel signal readout unit RU1, which outputs a voltage signal from the output terminal O1, includes a transfer transistor G1 (transfer MOS transistor), a floating diffusion FD1, a charge storage capacitor C1, a reset transistor RT1, a source follower transistor SF1, and a selection transistor SL1. In the pixel signal readout unit RU1, the floating diffusion FD1 and the charge storage capacitor C1 form a charge storage unit CS1. The pixel signal readout units RU2, RU3, and RU4 have a similar configuration.

[0039] The photoelectric conversion element PD is a buried photodiode that photoelectrically converts incident light, generates charges corresponding to the incident light, and accumulates the generated charges. In this embodiment, the incident light is incident from the space to be measured. In the pixel circuit 321, the photoelectric conversion element PD photoelectrically converts incident light to generate electric charges, which are then distributed to each of the four charge accumulation units CS (CS1 to CS4), and voltage signals corresponding to the amount of the distributed electric charges are output to the pixel signal processing circuit 325. Furthermore, the configuration of the pixel circuit arranged in the distance image sensor 32 is not limited to the configuration having four pixel signal readout units RU (RU1 to RU4) as shown in Figure 2, but may be a pixel circuit having a configuration having one or more pixel signal readout units RU.

[0040] In driving the pixel circuit 321 of the range image pickup device 1, a light pulse PO is emitted for an irradiation time To, and reflected light RL is received by the range image sensor 32 after a delay time Td. The pixel drive circuit 322 synchronizes with the irradiation of the light pulse PO under the control of the timing control unit 41, and supplies accumulation drive signals TX1 to TX4 to transfer transistors G1, G2, G3, and G4 at their respective timings to redirect the charges generated in the photoelectric conversion element PD, causing them to be accumulated in the charge accumulation units CS1, CS2, CS3, and CS4 in that order.

[0041] The pixel driving circuit 322 controls the reset transistor RT and the selection transistor SL by driving signals RST and SEL, respectively, converts the charges accumulated in the charge storage unit CS into an electrical signal by the source follower transistor SF, and outputs the generated electrical signal to the distance calculation unit 42 via terminal O. Furthermore, under the control of the timing control section 41, the pixel driving circuit 322 causes the charge generated in the photoelectric conversion element PD to flow to the power supply VDD in response to the driving signal RSTD, thereby discharging the charge (erasing the charge).

[0042] FIG. 3 is a timing chart showing the transfer of the charges generated by the photoelectric conversion elements PD to the charge accumulation sections CS. 3, the vertical axis represents the pulse level and the horizontal axis represents time. The timing chart shows the relative relationship on the time axis between the optical pulse PO and the reflected light RL, the timing of each of the accumulation drive signals TX1 to TX4 supplied to the transfer transistors G1 to G4, and the timing of the drive signal RSTD supplied to the charge discharge transistor GD.

[0043] The timing control unit 41 causes the light source unit 2 to irradiate the measurement space with a light pulse PO. As a result, the light pulse PO is reflected by the subject and received by the light receiving unit 3 as reflected light RL. The photoelectric conversion element PD then generates charges corresponding to the background light and the reflected light RL. The pixel drive circuit 322 controls the on / off of each of the transfer transistors G1 to G4 to transfer the charges generated by the photoelectric conversion element PD to each of the charge accumulation units CS1 to CS4. That is, the pixel drive circuit 322 supplies each of the accumulation drive signals TX1 to TX4 as an "H" level signal for a predetermined time duration (the same duration as the irradiation time To) to the transfer transistors G1 to G4, respectively.

[0044] The pixel driving circuit 322, for example, turns on a transfer transistor G1 provided on a transfer path that transfers charges from the photoelectric conversion element PD to the charge accumulation unit CS1. As a result, charges photoelectrically converted by the photoelectric conversion element PD are accumulated in the charge accumulation unit CS1 via the transfer transistor G1. Thereafter, the pixel driving circuit 322 turns off the transfer transistor G1. This stops the transfer of charges to the charge accumulation unit CS1. In this way, the vertical scanning circuit 322 accumulates charges in the charge accumulation unit CS1. The same applies to the other charge accumulation units CS2, CS3, and CS4.

[0045] At this time, during the charge accumulation period in which charge is distributed to the charge accumulation sections CS (the period in which charge is accumulated in each of the charge accumulation sections CS in a frame), an accumulation cycle in which each of the accumulation drive signals TX1, TX2, TX3, and TX4 is supplied to the transfer transistors G1, G2, G3, and G4, respectively, is repeated. Then, charges corresponding to incident light are transferred from the photoelectric conversion element PD to the charge accumulation units CS1, CS2, CS3, and CS4, respectively, via the transfer transistors G1, G2, G3, and G4. A plurality of accumulation cycles are repeated during the charge accumulation period. As a result, charges are accumulated in the charge accumulation units CS1, CS2, CS3, and CS4, respectively, for each accumulation cycle of the charge accumulation units CS1, CS2, CS3, and CS4 during the charge accumulation period.

[0046] Furthermore, when repeating the accumulation cycles of the charge accumulation units CS1, CS2, CS3, and CS4, after the transfer (relocation) of charges to the charge accumulation unit CS4 is completed, the pixel driving circuit 322 supplies an "H" level driving signal RSTD to the charge discharging transistor GD provided on the discharge path that discharges charges from the photoelectric conversion element PD, to turn it on. As a result, before the accumulation cycle for the charge accumulation unit CS1 starts, the charge drain transistor GD discards the charge generated in the photoelectric conversion element PD after the accumulation cycle for the immediately preceding charge accumulation unit CS4 (i.e., resets the photoelectric conversion element PD). That is, one or more charge drain transistors GD are provided, and drain charge from the photoelectric conversion element PD except for the period during which charge generated from the photoelectric conversion element PD by incident light is distributed and accumulated in each of the charge accumulation units CS1, CS2, CS3, and CS4.

[0047] The pixel driving circuit 322 then sequentially performs signal processing such as A / D conversion on the voltage signals from all of the pixel circuits 321 arranged in the light receiving section 3 in units of rows (horizontal arrangement) of the pixel circuits 321. Thereafter, the pixel driving circuit 322 outputs the processed voltage signals to the distance calculation unit 42 in the order of the columns arranged in the light receiving unit 3.

[0048] As described above, the pixel driving circuit 322 accumulates charges in the charge accumulation units CS and discards the charges photoelectrically converted by the photoelectric conversion elements PD over one frame. As a result, charges corresponding to the amount of light received by the distance image pickup device 1 over a predetermined time period are accumulated in each charge accumulation unit CS. The pixel driving circuit 322 outputs an electrical signal corresponding to the amount of charge accumulated in each charge accumulation unit CS for one frame to the distance calculation unit 42.

[0049] Due to the relationship between the timing of irradiating the light pulse PO and the timing of accumulating charges in each of the charge accumulation units CS (CS1 to CS4), the charge accumulation unit CS1 holds an amount of charge corresponding to external light components such as background light before irradiating the light pulse PO. Furthermore, the charge accumulation units CS2, CS3, and CS4 hold charges corresponding to the reflected light RL and external light components, respectively. The distribution (allocation ratio) of the amount of charge allocated to the charge accumulation units CS2 and CS3, or the charge accumulation units CS3 and CS4, is a ratio that corresponds to the delay time Td between when the light pulse PO is reflected by the subject S and when it enters the range image pickup device 1.

[0050] Returning to FIG. 1, the distance calculation unit 42 uses this principle to calculate the delay time Td using the following equation (1) or (2). Td=To×(Q3-Q1) / (Q2+Q3-2×Q1) …(1) Td=To+To×(Q4-Q1) / (Q3+Q4-2×Q1) …(2) Here, To is the period during which the light pulse PO is irradiated, Q1 is the amount of charge accumulated in the charge accumulation unit CS1, Q2 is the amount of charge accumulated in the charge accumulation unit CS2, Q3 is the amount of charge accumulated in the charge accumulation unit CS3, and Q4 is the amount of charge accumulated in the charge accumulation unit CS4. For example, when Q4=Q1, the distance calculation unit 42 calculates the delay time Td using equation (1), and when Q2=Q1, the distance calculation unit 42 calculates the delay time Td using equation (2).

[0051] In equation (1), charges generated by reflected light are accumulated in the charge accumulation units CS2 and CS3, but not in the charge accumulation unit CS4. On the other hand, in equation (2), charges generated by reflected light are accumulated in the charge accumulation units CS3 and CS4, but not in the charge accumulation unit CS2. In addition, equation (1) or (2) is based on the premise that the amount of charge stored in the charge storage units CS2, CS3, and CS4 that corresponds to the external light component is the same as the amount of charge stored in the charge storage unit CS1.

[0052] The distance calculation unit 42 calculates the round trip distance to the subject S by multiplying the delay time obtained by equation (1) or (2) by the speed of light (velocity). Then, the distance calculation unit 42 calculates the distance from the distance image sensor 32 (i.e., the distance image pickup device 1) to the subject S by dividing the calculated round trip distance by 2 (delay time Td×c (speed of light) / 2).

[0053] FIG. 4 is a diagram showing an example of the arrangement (layout pattern) of each transistor of the pixel circuit 321 in this embodiment. The layout pattern in FIG. 4 shows the layout pattern of the pixel circuit 321 in FIG. 3 (that is, the pixel circuit 321 in FIG. 2). 4 also shows the pattern layout of transfer transistors G1, G2, G3, and G4, source follower transistors SF1, SF2, SF3, and SF4, select transistors SL1, SL2, SL3, and SL4, reset transistors RT1, RT2, RT3, and RT4, charge ejection transistors GD1 and GD2, and photoelectric conversion element PD. Each of the above-mentioned transistors is an n-channel MOS transistor formed on a p-type semiconductor substrate.

[0054] For example, the reset transistor RT1 is composed of a drain RT1_D (n diffusion layer (n-type impurity diffusion layer)), a source RT1_S (n diffusion layer), and a gate RT1_G on a p-type semiconductor substrate. The contact RT1_C is a pattern indicating a contact that is provided in each diffusion layer of the drain RT1_D (n diffusion layer) and the source RT1_S (n diffusion layer) of the reset transistor RT1 and that connects to wiring (not shown). The other transfer transistors G1 to G4, source follower transistors SF1 to SF4, select transistors SL1 to SL4, reset transistors RT2 to RT4, and charge discharge transistors GD1 and GD2 have the same configuration. The photoelectric conversion element PD is formed in a rectangular shape and is made up of a long side PDL1, a PDL2 parallel to and facing the long side PDL1, a short side PDS1, and a PDS2 parallel to and facing the short side PDS1.

[0055] The transfer transistor G1 is formed of a floating diffusion FD1 as a drain, a gate G1_G, and a source (n diffusion layer of the photoelectric conversion element PD). Here, the floating diffusion FD1 is a diffusion layer (n diffusion layer) as a drain of the transfer transistor G1, and accumulates charges from the photoelectric conversion element PD. The drain G1_D is connected to the gate SF1_G of the source follower transistor SF1 and the source RT1_S of the reset transistor RT1 via a contact G1_C and a wiring (not shown). Each of the other transfer transistors G2, G3, and G4 has the same configuration as the transfer transistor G1.

[0056] 4 shows the arrangement of each transistor on the semiconductor substrate of pixel circuit 321, omitting the wiring patterns and each charge storage capacitor (C1 to C4). Therefore, each of charge storage units CS1, CS2, CS3, and CS4 is arranged at the position of each of floating diffusions FD1, FD2, FD3, and FD4.

[0057] In FIG. 4, for example, the floating diffusion FD1 in the charge storage section CS, and the transfer transistor G1, the selection transistor SL1, the source follower transistor SF1, and the reset transistor RT1 located near the charge storage section CS1 each have a pn junction, and when irradiated with incident light, generate charges that become noise charges. Then, the charge storage unit CS1 is reset to the voltage of the power supply VRD by the reset transistor and is in a high potential state, so that noise charges are injected into the charge storage unit CS1 due to an electric field caused by the difference in potential.

[0058] As a result, noise charges are accumulated in the charge accumulation unit CS1. Furthermore, since incident light is continuously irradiated onto all of the charge accumulation units CS1 to CS4 during the period when charges are accumulated in the charge accumulation unit CS and until the charges are read out from the charge accumulation unit CS, the ratio of noise charges to the amount of charges accumulated in the charge accumulation unit CS increases. Similarly to the charge storage unit CS1 described above, each of the charge storage units CS2 to CS3 also receives and stores noise charges generated by itself and other transistors in the vicinity of itself.

[0059] Therefore, the accuracy of the distance calculated by the above formulas (1) and (2) is reduced due to the inclusion of noise charges in the charge amounts Q1, Q2, Q3, and Q4 used in the calculation. In this embodiment, as shown in Fig. 5, a process for detecting the amount of noise charge accumulated in the charge accumulation unit CS is performed. Fig. 5 is a diagram illustrating a process for acquiring the amount of noise charge accumulated in the charge accumulation unit CS in this embodiment. In Fig. 5, the integration time is a period during which charge is accumulated in the charge accumulation unit CS in a frame, and the process of allocating charge from the photoelectric conversion element PD to the charge accumulation unit CS, as already explained, is repeated every predetermined number of accumulation periods. The read time is a period during which the amount of charge accumulated in the charge accumulation unit CS in a frame is read out, and the amount of charge accumulated in each charge accumulation unit CS of the pixel circuits 321 in the distance image sensor 32 is sequentially read out and output to the distance calculation unit 42. The measurement control unit 43 controls the timing control unit 41 to change the output timing of each of the accumulation drive signals TX1, TX2, TX3, and TX4 from the pixel drive circuit 322 and the drive signal RSTD between the noise charge amount acquisition mode and the ranging charge amount acquisition mode.

[0060] Here, the noise charge amount acquisition mode indicates the processing mode performed in the first frame when distance measurement is started, and is a mode for obtaining the noise charge amounts QNS1 to QNS4 accumulated in each of the charge storage units CS1 to CS4. On the other hand, the distance measurement charge amount acquisition mode is a mode for acquiring the amount of charge accumulated in each of the charge accumulation units CS1 to CS4, the charge amount being generated by the photoelectric conversion element PD due to incident light including reflected light RL. Then, when distance measurement is started in the distance image capturing device 1, the measurement control unit 43 controls the timing control unit 41 so that the operation of the pixel driving circuit 322 in the first frame is set to noise charge amount acquisition mode, and from the second frame onwards, the operation of the pixel driving circuit 322 is set to distance measurement charge amount acquisition mode.

[0061] FIG. 6 is a conceptual diagram illustrating the process of accumulating charges in the charge accumulation units CS (CS1, CS2, CS3, and CS4) in the noise charge amount acquisition mode. 6(a) is a timing chart showing the timing of the emission of the light pulse PO, the accumulation drive signals TX1, TX2, TX3, and TX4, and the drive signal RSTD in the noise charge amount acquisition mode. In FIG. 6(a), the horizontal axis represents time, and the vertical axis represents the signal level (H level (ON) or L level (OFF)). FIG. 6(b) is a conceptual diagram that simplifies FIG. 4 and shows the concept of the configuration of the photoelectric conversion element PD, transfer transistors G1, GS2, GS3, and GS4, and charge accumulation units CS1, CS2, CS3, and CS4.

[0062] As shown in FIG. 6(a), in the noise charge amount acquisition mode, during the integration time, i.e., during all accumulation cycles in the charge accumulation period, all of the accumulation drive signals TX1, TX2, TX3, and TX4 are set to the L level, so that the transfer transistors G1, G2, G3, and G4 are maintained in the off state. At this time, the light pulse PO is irradiated at a predetermined time in each accumulation period, the drive signal RSTD is set to H level, and the charge discharging transistors GD1 and GD2 are maintained in the ON state.

[0063] As a result, all the charges generated by the photoelectric conversion element PD are discharged to the power supply VDD. Since the transfer transistors G1, G2, G3, and G4 are each in an off state, no charges from the photoelectric conversion element PD are accumulated in the charge accumulation units CS1, CS2, CS3, and CS4. Therefore, the charges accumulated in the charge accumulation unit CS are noise charges including charges generated in the charge accumulation unit CS by incident light (background light and reflected light RL) and charges generated in the circuit near the charge accumulation unit CS. Then, noise charges of respective amounts QNS1, QNS2, QNS3, and QNS4 are accumulated in the charge accumulation units CS1, CS2, CS3, and CS4 during the integration time.

[0064] Furthermore, during the read time, the pixel drive circuit 322 reads out the noise charge amounts QNS1, QNS2, QNS3, and QNS4 of the charge storage units CS1, CS2, CS3, and CS4 from each of the pixel circuits 321, and outputs them as predetermined electrical signals to the distance calculation unit 42. For simplicity of explanation, the following explanation will also refer to these as noise charge amounts QNS1, QNS2, QNS3, and QNS4. Distance calculation unit 42 writes and stores noise charge amounts QNS1, QNS2, QNS3, and QNS4 for each pixel circuit 321 in a storage unit (not shown) (for example, provided in distance image processing unit 4). The noise charge amount acquisition mode is completed by the above-described processing.

[0065] FIG. 7 is a conceptual diagram illustrating the process of accumulating charges in the charge accumulation units CS (CS1, CS2, CS3, and CS4) in the distance measurement charge amount acquisition mode. 7(a) is a timing chart showing the timing of the emission of the light pulse PO, the accumulation drive signals TX1, TX2, TX3, and TX4, and the drive signal RSTD in the distance measurement charge amount acquisition mode. In FIG. 7(a), the horizontal axis represents time, and the vertical axis represents the signal level (H level (ON) or L level (OFF)). FIG. 7(b) is a conceptual diagram that simplifies FIG. 4 and shows the concept of the configuration of the photoelectric conversion element PD, transfer transistors G1, G2, G3, and G4, and charge accumulation units CS1, CS2, CS3, and CS4.

[0066] As shown in Figure 7(a), in the distance measurement charge amount acquisition mode, during the integration time, i.e., during all accumulation cycles in the charge accumulation period, each of the accumulation drive signals TX1 to TX4 is controlled to be at H level / L level at a predetermined timing, and each of the transfer transistors G1, G2, G3, and G4 is controlled to be on or off, and charge is distributed from the photoelectric conversion element PD to each of the charge accumulation units CS1, CS2, CS3, and CS4. At this time, the light pulse PO is irradiated at a predetermined time for each accumulation cycle, and during the period in which the charge is distributed to the charge accumulation sections CS1, CS2, CS3, and CS4, the drive signal RSTD is set to L level, and the charge discharging transistors GD1 and GD2 are maintained in the off state.

[0067] As a result, the charges generated by the photoelectric conversion element PD are transferred to the charge storage units CS1, CS2, CS3, and CS4 by the transfer transistors G1, G2, G3, and G4, respectively, and the charges are stored in the charge storage units CS1, CS2, CS3, and CS4. Therefore, the charge amount Q of the charge accumulated in the charge accumulation unit CS is a mixture of the charge distributed from the photoelectric conversion element PD and noise charges, which are charges generated in the charge accumulation unit CS by the incident light (background light and reflected light RL) and charges generated in the circuit near the charge accumulation unit CS. That is, noise charges QNS1, QNS2, QNS3, and QNS4 are mixed into the charges Q1, Q2, Q3, and Q4 stored in the charge storage units CS1, CS2, CS3, and CS4, respectively.

[0068] Furthermore, during the read time, the pixel driving circuit 322 reads out the amounts of charges Q1, Q2, Q3, and Q4 stored in the charge storage units CS1, CS2, CS3, and CS4 from each of the pixel circuits 321, and outputs them as predetermined electrical signals to the distance calculation unit 42. For the sake of simplicity, the following description will also refer to the amounts of charges as Q1, Q2, Q3, and Q4. The distance calculation unit 42 calculates the delay time Td corresponding to the distance between the distance image sensor 32 (i.e., each pixel circuit 321) and the subject S using equation (1) or (2) based on each of the charge amounts Q1, Q2, Q3, and Q4 supplied from the distance image sensor 32, as already explained.

[0069] At this time, the distance calculation unit 42 reads out the noise charge amounts QNS1, QNS2, QNS3, and QNS4 stored in the storage unit for each pixel circuit 321. Then, the distance calculation unit 42 subtracts the noise charge amounts QNS1, QNS2, QNS3, and QNS4 from the charge amounts Q1, Q2, Q3, and Q4, respectively, to calculate corrected charge amounts QC1, QC2, QC3, and QC4. That is, the correction charge QC1 (= Q1 - QNS1) is the charge amount obtained by subtracting the noise charge QNS1 from the charge Q1, supplied from the photoelectric conversion element PS by the allocation process, and accumulated in the charge storage unit CS1. Similarly, the correction charge QC2 (= Q2 - QNS2) is the charge amount obtained by subtracting the noise charge QNS2 from the charge Q2, supplied from the photoelectric conversion element PS by the allocation process, and accumulated in the charge storage unit CS2. The correction charge QC3 (= Q3 - QNS3) is the charge amount obtained by subtracting the noise charge QNS3 from the charge Q3, supplied from the photoelectric conversion element PS by the allocation process, and accumulated in the charge storage unit CS3. The correction charge QC4 (= Q4 - QNS4) is the charge amount obtained by subtracting the noise charge QNS4 from the charge Q4, supplied from the photoelectric conversion element PS by the allocation process, and accumulated in the charge storage unit CS4.

[0070] Then, the distance calculation unit 42 calculates the delay time Td by substituting each of the corrected charge amounts QC1, QC2, QC3, and QC4 into equation (1) or (2) as shown below. Td=To×(QC3-QC1) / (QC2+QC3-2×QC1) …(1)' Td=To+To×(QC4-QC1) / (QC3+QC4-2×QC1)…(2)' Then, in calculating the distance for each frame from the second frame onwards, the noise charge amounts QNS1, QNS2, QNS3, and QNS4 stored in the memory unit are read out and subtracted from the charge amounts Q1, Q2, Q3, and Q4, respectively, to sequentially obtain corrected charge amounts QC1, QC2, QC3, and QC4, and the distance is calculated using equation (1)' or equation (2)'.

[0071] According to this embodiment, with the above-described configuration, in a frame of the noise charge amount acquisition mode, the noise charge amount QNS of the noise charge generated by incident light and accumulated in the charge accumulation unit CS is acquired in advance in each of the charge accumulation units CS (CS1, CS2, CS3, and CS4) and in the circuitry near the charge accumulation unit CS, and in a subsequent frame of the distance measurement charge amount acquisition mode, the noise charge amount QNS is subtracted from each of the charge amounts Q allocated and accumulated in each of the charge accumulation units CS by the allocation process from the photoelectric conversion element PS, the noise charge mixed in with the charge amount Q accumulated in the charge accumulation unit CS is removed, and the corrected charge amount QC generated by the incident light in the photoelectric conversion element PD is extracted, thereby preventing a decrease in accuracy due to the influence of the noise charge and enabling the distance between the subject S and the distance image sensor 32 to be calculated with high accuracy.

[0072] In addition, in this embodiment, when the distance image capturing device 1 starts distance measurement, the first frame is set to the noise charge amount acquisition mode frame. However, the first frame in a group of a predetermined number of consecutive frames may be set to the noise charge amount acquisition mode frame. For example, if a group consists of 100 consecutive frames, the first frame of the group is set to the noise charge amount acquisition mode frame, and the second to 99th frames are set to the distance measurement charge amount acquisition mode frame. The noise charge amount information in the memory unit is overwritten with the newly acquired noise charges QNS1, QNS2, QNS3, and QNS4, respectively. Alternatively, a frame in the noise charge amount acquisition mode may be inserted at regular intervals (for example, every 5 seconds, every minute, etc.) to perform distance measurement processing.

[0073] Furthermore, in this embodiment, the noise charge amounts QNS1, QNS2, QNS3, and QNS4 in all pixel circuits 321 in the distance image sensor 32 are stored in a memory unit, but in order to reduce the capacity of the memory unit, an average value of the noise charge amounts QNS1, QNS2, QNS3, and QNS4 in each charge storage unit CS1, CS2, CS4, and CS4 in the pixel circuit 321 may be calculated, and the average value may be used as the noise charge of all charge storage units CS1, CS2, CS4, and CS4. Alternatively, each of the pixel circuits 321 in the distance image sensor 32 may be divided into pixel circuit groups consisting of a predetermined number of pixel circuits 321, and the pixel circuits 321 in the distance image sensor 32 may be divided, and the noise charge amounts QNS1, QNS2, QNS3, and QNS4 of any of the pixel circuits 321 in each group may be used as a representative value, or an average value of each may be calculated and used by the group.

[0074] <Second embodiment> A second embodiment of the present invention will be described below with reference to the drawings. The second embodiment has the same configuration as the first embodiment shown in Figures 1 to 4. The following describes the operations of the second embodiment that differ from those of the first embodiment. FIG. 8 is a diagram illustrating a process for acquiring the amount of noise charge accumulated in the charge accumulation unit CS in this embodiment. 8, the integration time is the period during which charge is accumulated in the charge accumulation unit CS in a frame, and the process of allocating charge from the photoelectric conversion element PD to the charge accumulation unit CS, as already explained, is repeated. The read time is the period during which the amount of charge accumulated in the charge accumulation unit CS in a frame is read out, and the amount of charge accumulated in each charge accumulation unit CS of the pixel circuits 321 in the range image sensor 32 is read out sequentially. In this embodiment, the frame has a frame width of 33.3 msec (milliseconds), and the frame cycle is 30 frames per second (30 fps (frames per second)).

[0075] In the second embodiment, each frame is divided into a first subframe and a second subframe as shown in Fig. 8, and processing in the noise charge amount acquisition mode is performed in the first subframe, and processing in the distance measurement charge amount acquisition mode is performed in the second subframe. The processing in the noise charge amount acquisition mode and the distance measurement charge amount acquisition mode is the same as in the first embodiment. The first subframe and the second subframe each have an integration time and a lead time.

[0076] During the integration time of the first subframe, the noise charge amounts QNS1, QNS2, QNS3, and QNS4 are acquired, read out during the read time, and output to the distance image processing unit 4. This causes the distance calculation unit 42 to write and store the input noise charge amounts QNS1, QNS2, QNS3, and QNS4 in the memory unit. Then, during the integration time of the second sub-frame, the charge amounts Q1, Q2, Q3, and Q4 are acquired, read out during the read time, and output to the distance image processing unit 4.

[0077] As a result, the distance calculation unit 42 subtracts the noise charge amounts QNS1, QNS2, QNS3, and QNS4 from the input charge amounts Q1, Q2, Q3, and Q4, respectively, to obtain corrected charge amounts QC1, QC2, QC3, and QC4. Further, the distance calculation unit 42 calculates the delay time Td for calculating the distance from the distance image sensor 32 to the subject S using the calculated corrected charge amounts QC1, QC2, QC3, and QC4, according to equation (1)' or equation (2)'.

[0078] Here, the read time cannot be shortened because the noise charge amounts QNS1, QNS2, QNS3, and QNS4 or the charge amounts Q1, Q2, Q3, and Q4 are read from each of the pixel circuits 321 in the range image sensor 32. Therefore, in order to maintain the lead time for reading out charges from the charge storage unit CS, the integration time for storing charges in the charge storage unit CS is shortened (the number of storage cycles is reduced).

[0079] With the above-described configuration, in this embodiment, the noise charge amount is acquired and the distance is measured in the same frame. Therefore, when the ambient light changes in a short period of time (for example, in seconds), the noise charge amount QNS generated by the incident light can be removed from the charge amount Q in real time. This makes it possible to suppress the influence of the noise charge in response to changes in the ambient light, thereby improving the accuracy of the measured distance.

[0080] <Third embodiment> A third embodiment of the present invention will now be described with reference to the drawings. The third embodiment has the same configuration as the first embodiment shown in Figures 1 to 4. The following describes the operations of the third embodiment that differ from those of the first embodiment. 9 is a diagram illustrating the process of acquiring the amount of noise charge accumulated in the charge accumulation unit CS in this embodiment. In FIG. 9, the integration time is the period during which charge is accumulated in the charge accumulation unit CS in a frame, and the process of allocating charge from the photoelectric conversion element PD to the charge accumulation unit CS, as already explained, is repeated. The lead time is the period during which the amount of charge accumulated in the charge accumulation unit CS in a frame is read out, and the amount of charge accumulated in each charge accumulation unit CS of the pixel circuits 321 in the range image sensor 32 is read out sequentially.

[0081] 9, in this embodiment, under the control of the measurement control unit 43, two types of frames for the noise charge amount acquisition mode are used: a frame for the first noise charge amount acquisition mode (first frame) and a frame for the second noise charge amount acquisition mode (second frame). The frames for the distance measurement charge amount acquisition mode are the third and subsequent frames. In this embodiment, as in the first embodiment, when distance image capture is started, the first one and two frames may be set to the noise charge amount acquisition mode, or the first noise charge amount acquisition mode and the second noise charge amount acquisition mode may be executed periodically or temporally between blocks of frames in the time series of the distance measurement charge amount acquisition mode.

[0082] In this embodiment, the first noise charge amount acquisition mode is a mode in which the first noise charge amount that is generated by only background light as incident light and accumulated in the charge accumulation unit CS is acquired. The second noise charge amount acquisition mode is a mode for acquiring the second noise charge amount (corresponding to the noise charge amount in the first and second embodiments) that is generated by background light and reflected light RL as incident light and accumulated in the charge accumulation unit CS. The third frame and onwards are frames in the distance measurement charge amount acquisition mode for acquiring a distance image.

[0083] FIG. 10 is a conceptual diagram illustrating noise charges corresponding to reflected light RL in the charge accumulation unit CS in the distance measurement charge amount acquisition mode. 10 is a timing chart showing the timing of the emission of the light pulse PO, the reflected light RL, each of the accumulation drive signals TX1, TX2, TX3, and TX4, the drive signal RSTD, and the generation of the noise charge amounts QNSP (QNSP1, QNSP2, QNSP3, and QNSP4) in the distance measurement charge amount acquisition mode. The noise charge amount QNSP is the amount of noise charge generated by the reflected light RL and accumulated in the charge accumulation unit CS. That is, the noise charge amount QNSP1 is the amount of noise charge due to the reflected light RL and accumulated in the charge accumulation unit CS1. The noise charge amount QNSP2 is the amount of noise charge due to the reflected light RL and accumulated in the charge accumulation unit CS2. The noise charge amount QNSP3 is the amount of noise charge due to the reflected light RL and accumulated in the charge accumulation unit CS3. The noise charge amount QNSP4 is the amount of noise charge due to the reflected light RL and accumulated in the charge accumulation unit CS4.

[0084] As shown in Figure 10, in the distance measurement charge amount acquisition mode, during the integration time, i.e., during all accumulation cycles in the charge accumulation period, each of the accumulation drive signals TX1, TX2, TX3, and TX4 is controlled to be at H level / L level at a predetermined timing, each of the transfer transistors G1, G2, G3, and G4 is controlled on and off, and charge is distributed from the photoelectric conversion element PD to each of the charge accumulation units CS1, CS2, CS3, and CS4. At this time, the light pulse PO is irradiated at a predetermined time for each accumulation cycle, and during the period in which the charge is distributed to the charge accumulation sections CS1, CS2, CS3, and CS4, the drive signal RSTD is set to L level, and the charge discharging transistors GD1 and GD2 are maintained in the off state. At this time, each of the noise charge amounts QNSP1, QNSP2, QNSP3, and QNSP4 changes depending on the number of allocations made in each frame in the distance measurement charge amount acquisition mode.

[0085] As a result, the charges generated by the photoelectric conversion element PD are transferred to the charge storage units CS1, CS2, CS3, and CS4 by the transfer transistors G1, G2, G3, and G4, respectively, and the charges stored in the charge storage units CS1, CS2, CS3, and CS4 are mixed with noise charges generated by the reflected light RL and noise charges generated by the background light. When a subject with significantly different reflectances is imaged, each of the noise charge amounts QNSP1, QNSP2, QNSP3, and QNSP4 generated by this reflected light RL must be acquired each time the number of allocations changes due to the auto exposure function. For this reason, it is not possible to predict when the number of allocations will change due to auto exposure, and therefore it is necessary to frequently execute frames in the noise charge amount acquisition mode. However, frequent processing of frames in noise charge amount acquisition mode reduces the resolution of the distance image on the time axis, so in this embodiment, the method described below is used to deal with changes in the number of allocations made by auto exposure.

[0086] In the first frame of the first noise charge amount acquisition mode in FIG. 9, the process of the noise charge amount acquisition mode is performed during the integration time, similarly to the first embodiment. However, in the first noise charge amount acquisition mode, unlike the noise charge amount acquisition mode of the first embodiment, the measurement control unit 43 does not cause the light source unit 2 to emit a light pulse PO, but instead uses only background light as incident light, and acquires the noise charge amount QNSN of the noise charge generated by the background light and accumulated in the charge accumulation unit CS. That is, the distance calculation unit 42 acquires, from the range image sensor 32, the noise charge amounts QNSN1, QNSN2, QNSN3, and QNSN4 of the charge accumulation units CS1, CS2, CS3, and CS4, respectively. Then, the distance calculation unit 42 writes and stores each of the acquired noise charge amounts QNSN1, QNSN2, QNSN3, and QNSN4 in the storage unit as the noise charge amount (first noise charge amount) acquired in the first noise charge amount acquisition mode.

[0087] Furthermore, in the second noise charge amount acquisition mode, similarly to the noise charge amount acquisition mode of the first embodiment, the measurement control unit 43 causes the light source unit 2 to emit a light pulse PO, sets incident light as background light and reflected light RL, and acquires noise charge amounts QNS of noise charges generated by the background light and reflected light RL and accumulated in the charge accumulation unit CS. That is, the distance calculation unit 42 acquires, from the distance image sensor 32, the noise charge amounts QNS1, QNS2, QNS3, and QNS4 in the charge accumulation units CS1, CS2, CS3, and CS4, respectively. Then, the distance calculation unit 42 writes and stores each of the acquired noise charge amounts QNS1, QNS2, QNS3, and QNS4 in the storage unit as the noise charge amounts (second noise charge amounts) acquired in the second noise charge amount acquisition mode.

[0088] In addition, the distance calculation unit 42 subtracts the first noise charge amounts QNSN1, QNSN2, QNSN3, and QNSN4 from the second noise charge amounts QNS1, QNS2, QNS3, and QNS4, respectively, to obtain third noise charge amounts QNSP1 (= QNS1 - QNSN1), QNSP2 (= QNS2 - QNSN2), QNSP3 (= QNS3 - QNSN4), and QNSP4 (= QNS4 - QNSN4). Each of the third noise charge amounts QNSP1, QNSP2, QNSP3, and QNSP4 is the amount of noise charge generated by only the light pulse PO in each of the charge accumulation units CS1, CS2, CS3, and CS4. Here, each of the third noise charge amounts QNSP1, QNSP2, QNSP3, and QNSP4 is the amount of noise charge generated by the reflected light RL corresponding to the number of allocations in the second noise charge amount acquisition mode, that is, the number of times the light source unit 2 emits the light pulse PO. Then, the distance calculation unit 42 writes and stores each of the calculated third noise charge amounts QNSP1, QNSP2, QNSP3, and QNSP4 in the storage unit.

[0089] In the frames of the distance measurement charge amount acquisition mode from the third frame onwards, similar to the frames of the distance measurement charge amount acquisition mode in the first embodiment, the measurement control unit 43 causes the light source unit 2 to emit a light pulse PO, sets the incident light as background light and reflected light RL, allocates the charges generated in the photoelectric conversion element PD by the background light and reflected light RL, and acquires the charge amount Q of the charges (including noise charges) accumulated in the charge accumulation unit CS. That is, the distance calculation unit 42 acquires from the distance image sensor 32 the charge amounts Q1, Q2, Q3, and Q4 into which the charges generated by the photoelectric conversion element PD are allocated in the charge accumulation units CS1, CS2, CS3, and CS4, respectively. Then, the distance calculation unit 42 subtracts the noise charge amount from each of the acquired charge amounts Q1, Q2, Q3, and Q4 in the same manner as in the first embodiment, to calculate corrected charge amounts QC1, QC2, QC3, and QC4, respectively.

[0090] In this embodiment, when calculating each of the correction charge amounts QC1, QC2, QC3, and QC4, the third noise charge amounts QNSP1, QNSP2, QNSP3, and QNSP4 are adjusted based on the number of allocations during the integration time in the frame of the distance measurement charge amount acquisition mode. That is, if the distance measuring imaging device has an auto-exposure function, the measurement control unit 43 changes the number of distributions during the integration time depending on the intensity of the reflected light RL so that any of the charge storage units CS does not become saturated. Therefore, each of the third noise charge amounts QNSP1, QNSP2, QNSP3, and QNSP4 changes successively depending on the number of times of allocation (number of times of emitting light pulses).

[0091] When calculating the distance in a frame in the distance measurement charge amount acquisition mode, the distance calculation unit 42 reads out the third noise charge amounts QNSP1, QNSP2, QNSP3, and QNSP4 that have been calculated in advance and stored in the storage unit. Then, the distance calculation unit 42 divides the number of allocations in the frame of the distance measurement charge amount acquisition mode for which distance is to be calculated by the number of allocations in the second frame of the second noise charge amount acquisition mode to calculate the adjustment coefficient k. The distance calculation unit 42 multiplies each of the third noise charge amounts QNSP1, QNSP2, QNSP3, and QNSP4 by the calculated adjustment coefficient k to calculate adjusted third noise charge amounts kQNSP1, kQNSP2, kQNSP3, and kQNSP4.

[0092] Then, the distance calculation unit 42 adds the adjusted third noise charge amounts kQNSP1, kQNSP2, kQNSP3, and kQNSP4 to each of the first noise charge amounts QNSN1, QNSN2, QNSN3, and QNSN4, respectively, to calculate fourth noise charge amounts QN1 (= QNSN1 + kQNSP1), QN2 (= QNSN2 + kQNSP2), QN3 (= QNSN3 + kQNSP3), and QN4 (= QNSN4 + kQNSP4). Furthermore, the distance calculation unit 42 subtracts the fourth noise charge amounts QN1, QN2, QN3, and QN4 from the charge amounts Q1, Q2, Q3, and Q4, respectively, to calculate corrected charge amounts QC1, QC2, QC3, and QC4. Then, as in the first embodiment, the distance calculation unit 42 calculates the delay time Td used to determine the distance between the subject S and the distance image sensor 32 using equation (1)' or (2)', using each of the above-mentioned correction charge amounts QC1, QC2, QC3, and QC4.

[0093] According to the present embodiment, with the above-described configuration, in a frame of the first and second noise charge amount acquisition mode, the first noise charge amount QNSN and the third noise charge amount QNSP of the noise charge generated by incident light and accumulated in the charge accumulation unit CS are calculated in advance for each of the charge accumulation units CS (CS1, CS2, CS3, and CS4) and in a circuit near the charge accumulation unit CS, and an adjusted third noise charge amount QNSP of each of the frames is calculated using an adjustment coefficient k corresponding to the number of allocations in each frame of the distance measurement charge amount acquisition mode. kQNSP is calculated, the first noise charge QNSN and the adjusted third noise charge kQNSP are added together, and the fourth noise charge QN obtained as a result of the addition is subtracted from the charge Q to obtain each of the correction charges QC1, QC2, QC3, and QC4, and the adjustment time is calculated.Since the delay time Td is obtained from the correction charges QC1, QC2, QC3, and QC4 based on the fourth noise charge QN, which corresponds to the number of times of allocation by the auto exposure function, the distance between the subject S and the range image sensor 32 can be calculated with high accuracy.

[0094] Furthermore, according to this embodiment, in the first and second noise charge amount acquisition modes, the first noise charge amount QNSN and the third noise charge amount QNSP are obtained in the first and second frames, respectively, and therefore it is possible to obtain the noise charge amount corresponding to the number of times of allocation for each frame. This eliminates the need to perform processing to obtain the noise charge amount for each frame as in the second embodiment, and makes it possible to obtain the noise charge amount corresponding to the number of times of allocation for each frame, thereby easily improving the accuracy of the distance to be measured.

[0095] 11 is a flowchart showing an example of the operation of the process of calculating the distance between the distance image sensor 32 and the subject S in the distance image pickup device 1 according to the third embodiment. In the following explanation, in each of the pixel circuits 321 in the distance image sensor 32, the distance from the pixel circuit 321 (i.e., the distance image sensor 32) to the subject S is calculated.

[0096] Step S101: When the distance image pickup device 1 starts distance measurement, the measurement control unit 43 outputs an instruction to the timing control unit 41 to perform processing in the first noise charge amount acquisition mode. The timing control unit 41 outputs accumulation drive signals TX1, TX2, TX3, and TX4 and a drive signal RSTD to the range image sensor 32 in order to acquire the noise charge amount QNSN of the noise charge generated only by background light in the first frame. As a result, the distance image sensor 32 outputs the first noise charge amounts QNSN (QNSN1, QNSN2, QNSN3, QNSN4) acquired in the first frame to the distance calculation unit .

[0097] Step S102: After the first frame ends, the measurement control unit 43 outputs an instruction to the timing control unit 41 to perform processing in the second noise charge amount acquisition mode. In the second frame, the timing control unit 41 causes the light source unit 2 to emit a light pulse PO, and outputs accumulation drive signals TX1, TX2, TX3, and TX4 and a drive signal RSTD to the distance image sensor 32 to acquire the noise charge amount QNS (QNS1, QNS2, QNS3, QNS4) of the noise charge generated by the background light and reflected light RL. As a result, the distance image sensor 32 outputs the second noise charge amount QNS acquired in the second frame to the distance calculation unit .

[0098] Step S103: Then, the distance calculation section 42 subtracts each of the first noise charge amounts QNSN from each of the second noise charge amounts QNS to calculate third noise charge amounts QNSP (QNSP1, QNSP2, QNSP3, QNSP4). The distance calculation unit 42 writes and stores the third noise charge amount QNSP together with the first noise charge amount QNSN in the storage unit for each charge accumulation unit CS (CS1, CS2, CS3, CS4) in pixel circuit 321 units.

[0099] Step S104: After the second frame ends, the measurement control unit 43 outputs an instruction to the timing control unit 41 to perform processing in the distance measurement charge amount acquisition mode. In the third frame (i.e., a frame after the third frame), the timing control unit 41 causes the light source unit 2 to emit a light pulse PO and outputs accumulation drive signals TX1, TX2, TX3, and TX4 and a drive signal RSTD to the distance image sensor 32 as control for distributing charge from the photoelectric conversion element PD to each charge accumulation unit CS in order to measure the distance between the subject S and the distance image sensor 32. As a result, the distance image sensor 32 outputs to the distance calculation unit 42 each of the amounts of charge Q (Q1, Q2, Q3, Q4) acquired in the third frame.

[0100] Step S105: The distance calculation unit 42 divides the number of allocations in the third frame, which is the target of distance measurement, by the number of allocations in the second frame to calculate an adjustment coefficient k.

[0101] Step S106: The distance calculation unit 42 reads out the third noise charge amounts QNSP from the storage unit, and multiplies each of the third noise charge amounts QNSP by an adjustment coefficient k to calculate adjusted third noise charge amounts kQNSP (kQNSP1, kQNSP2, kQNSP3, kQNSP4).

[0102] Step S107: The distance calculation unit 42 reads out each of the first noise charge amounts QNSN from the storage unit, and adds each of the read out first noise charge amounts QNSN to the adjusted third noise charge amount kQNSP to calculate the fourth noise charge amount QN (QN1, QN2, QN3, QN4).

[0103] Step S108: The distance calculation unit 42 subtracts each of the calculated fourth noise charge amounts QN from each of the charge amounts Q to calculate corrected charge amounts QC (QC1, QC2, QC3, QC4).

[0104] Step S109: The distance calculation unit 42 uses the calculated corrected charge amount QC to calculate the delay time Td according to equation (1)' or (2)'.

[0105] Step S110: The distance calculation unit 42 calculates the distance from the distance image sensor 32 to the subject S by multiplying the obtained delay time Td by the high speed and dividing the result by "2".

[0106] Step S111: The measurement control unit 43 determines whether a control signal to end distance measurement has been input. At this time, the measurement control unit 43 terminates the processing if a control signal to terminate the distance measurement processing is input, and proceeds to step S104 if a control signal to terminate the distance measurement processing is not input.

[0107] Although the configurations of the first, second, and third embodiments described above are distance imaging devices using TOF technology, the application of the present invention is not limited to this, and it can also be applied to sensors having a structure in which a photodiode supplies one charge storage section, such as an RGB-IR (Red Green Blue-Infrared) sensor. Furthermore, if the charge generated by the photodiode due to incident light is stored in the charge storage section, the present invention can also be applied to a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor. [Explanation of symbols]

[0108] 1...Distance image capturing device 2...Light source section 3...Light receiving section 31...Lens 32...Distance image sensor (distance image sensor) 321...Pixel circuit 322...Pixel driving circuit 4...Distance image processing section 41...Timing control section 42...Distance calculation section 43...Measurement control section CS1, CS2, CS3, CS4...Charge storage section FD1, FD2, FD3, FD4...Floating diffusion G1, G2, G3, G4...Transfer transistors GD: Charge drain transistor ML...micro lens PD...photoelectric conversion element PO...light pulse RT1, RT2, RT3, RT4...Reset transistors S…Subject SF1, SF2, SF3, SF4...Source follower transistors SL1, SL2, SL3, SL4...Selection transistors

Claims

1. a light source unit that irradiates a measurement space with a light pulse; a light receiving unit including a photoelectric conversion element that generates charges according to light incident from the measurement space, a plurality of pixel circuits each including a plurality of charge accumulation units that accumulate the charges in a frame period that indicates a period in which frames are repeated, and a pixel drive circuit that performs on / off processing of each transfer transistor in each of the charge accumulation units at a predetermined accumulation timing synchronized with the irradiation of the light pulse, thereby distributing and accumulating the charges; a distance calculation unit that calculates a distance between the object in the measurement space and the light receiving unit based on an amount of charge determined by a first amount of charge of the charges accumulated in each of the charge accumulation units; Equipped with the distance calculation unit calculates the distance by subtracting, from each of the first charge amounts, a second charge amount due to noise charge, which is accumulated charge other than the charge distributed and accumulated by the on / off processing of the transfer transistor; a frame in which the amount of charge accumulated in the charge accumulation unit is acquired as a third amount of charge in a state in which the light pulse is not irradiated from the light source unit and the pixel drive circuit turns off the transfer transistor so that charge is not accumulated in the charge accumulation unit from the photoelectric conversion element, is defined as a first noise charge amount acquisition frame; a frame in which the amount of charge accumulated in the charge accumulation unit is acquired as the second amount of charge in a state in which the pixel drive circuit turns off the transfer transistor after the light pulse is irradiated from the light source unit and charge is not accumulated in the charge accumulation unit from the photoelectric conversion element, is defined as a second noise charge amount acquisition frame; The distance calculation unit subtracts the third charge amount from the second charge amount to obtain a fourth charge amount, which is a charge amount generated by the light pulse at a location other than the photoelectric conversion element, and corrects the second charge amount based on the fourth charge amount for use in calculating the distance. A distance image capturing device characterized by:

2. A light source unit that irradiates a measurement space with a light pulse; a light receiving unit including a photoelectric conversion element that generates charges according to light incident from the measurement space, a plurality of pixel circuits each including a plurality of charge accumulation units that accumulate the charges in a frame period that indicates a period in which frames are repeated, and a pixel drive circuit that performs on / off processing of each transfer transistor in each of the charge accumulation units at a predetermined accumulation timing synchronized with the irradiation of the light pulse, thereby distributing and accumulating the charges; a distance calculation unit that calculates a distance between the object in the measurement space and the light receiving unit based on an amount of charge determined by a first amount of charge of the charges accumulated in each of the charge accumulation units; Equipped with the distance calculation unit calculates the distance by subtracting, from each of the first charge amounts, a second charge amount due to noise charge, which is accumulated charge other than the charge distributed and accumulated by the on / off processing of the transfer transistor; a frame in which the amount of charge accumulated in the charge accumulation unit is acquired as a third amount of charge in a state in which the light pulse is not irradiated from the light source unit and the pixel drive circuit turns off the transfer transistor so that charge is not accumulated in the charge accumulation unit from the photoelectric conversion element, is defined as a first noise charge amount acquisition frame; a frame in which the amount of charge accumulated in the charge accumulation unit is acquired as the second amount of charge in a state in which the pixel drive circuit turns off the transfer transistor after the light pulse is irradiated from the light source unit and charge is not accumulated in the charge accumulation unit from the photoelectric conversion element, is defined as a second noise charge amount acquisition frame; the distance calculation unit subtracts the third charge amount from the second charge amount to obtain a fourth charge amount, which is a charge amount of charges generated by the light pulse at a location other than the photoelectric conversion element; The distance calculation unit when the number of second allocations in each of the frames after the second noise charge amount acquisition frame has changed with respect to the number of first allocations in each of the first noise charge amount acquisition frame and the second noise charge amount acquisition frame, the second allocation number is divided by the first allocation number to obtain an adjustment coefficient, and the result is multiplied by the fourth charge amount to calculate a fifth charge amount; The fifth charge amount is added to the third charge amount to correct the second charge amount and used to calculate the distance. A distance image capturing device characterized by:

3. one of the frames is designated as a noise charge amount acquisition frame for acquiring the second charge amount, In the noise charge amount acquisition frame, after the light pulse is irradiated from the light source unit, the pixel drive circuit turns off the transfer transistor and does not distribute charge from the photoelectric conversion element to the charge storage unit, the amount of charge stored in the charge storage unit being the second amount of charge; In the frames after the noise charge amount acquisition frame, The pixel driving circuit turns on and off the transfer transistors to distribute and store the charges in the charge storage units, thereby achieving the first charge amount.

3. The distance imaging device according to claim 1 or 2.

4. The light pulse irradiated from the light source unit is a pulse of light in the near-infrared wavelength band with a predetermined width.

4. The distance imaging device according to claim 1, wherein the distance imaging device is a distance sensor.

5. The pixel circuit has a BSI (Back Side Illumination) structure.

5. The distance imaging device according to claim 1, wherein the distance imaging device is a distance sensor.

6. The pixel circuit has three or more charge storage units.

6. The distance imaging device according to claim 1, wherein the distance imaging device is a distance measuring device.

7. The pixel circuit is provided with one or more charge drain transistors that drain charges from the photoelectric conversion elements during periods other than the period in which the charges are distributed and stored in the charge storage sections.

7. The distance imaging device according to claim 1, wherein the distance imaging device is a distance sensor.

8. A distance image capturing method for controlling a distance image capturing device including a light receiving unit having a plurality of pixel circuits, each of which is composed of a photoelectric conversion element and a plurality of charge accumulation units, and a pixel drive circuit, a light source unit, and a distance calculation unit, a step in which the light source unit irradiates a measurement space with a light pulse; a step in which the pixel circuit accumulates, in the charge accumulation unit, charges generated by the photoelectric conversion element in response to light incident from the measurement space during a frame period indicating a period in which frames are repeated; a pixel driving process in which the pixel driving circuit performs on / off processing of each transfer transistor in each of the charge accumulation units at a predetermined accumulation timing synchronized with the irradiation of the light pulse, thereby distributing and accumulating the charges; a distance calculation step in which the distance calculation unit calculates a distance between the object in the measurement space and the light receiving unit based on an amount of charge determined by a first amount of charge of the charges accumulated in each of the charge accumulation units; Including, the distance calculation unit calculates the distance by subtracting, from each of the first charge amounts, a second charge amount due to noise charge, which is accumulated charge other than the charge distributed and accumulated by the on / off processing of the transfer transistor; a frame in which the amount of charge accumulated in the charge accumulation unit is acquired as a third amount of charge in a state in which the light pulse is not irradiated from the light source unit and the pixel drive circuit turns off the transfer transistor so that charge is not accumulated in the charge accumulation unit from the photoelectric conversion element, is defined as a first noise charge amount acquisition frame; a frame in which the amount of charge accumulated in the charge accumulation unit is acquired as the second amount of charge in a state in which the pixel drive circuit turns off the transfer transistor after the light pulse is irradiated from the light source unit and charge is not accumulated in the charge accumulation unit from the photoelectric conversion element, is defined as a second noise charge amount acquisition frame; In the distance calculation step, the distance calculation unit subtracts the third charge amount from the second charge amount to obtain a fourth charge amount, which is the charge amount of the charge generated by the light pulse at a location other than the photoelectric conversion element, and corrects the second charge amount based on the fourth charge amount for use in calculating the distance. A distance image capturing method comprising:

9. A distance image capturing method for controlling a distance image capturing device including a light receiving unit having a plurality of pixel circuits each including a photoelectric conversion element and a plurality of charge accumulation units, a pixel drive circuit, a light source unit, and a distance calculation unit, a step in which the light source unit irradiates a measurement space with a light pulse; a step in which the pixel circuit accumulates, in the charge accumulation unit, charges generated by the photoelectric conversion element in response to light incident from the measurement space during a frame period indicating a period in which frames are repeated; a pixel driving process in which the pixel driving circuit performs on / off processing of each transfer transistor in each of the charge accumulation units at a predetermined accumulation timing synchronized with the irradiation of the light pulse, thereby distributing and accumulating the charges; a distance calculation step in which the distance calculation unit calculates a distance between the object in the measurement space and the light receiving unit based on an amount of charge determined by a first amount of charge of the charges accumulated in each of the charge accumulation units; Including, the distance calculation unit calculates the distance by subtracting, from each of the first charge amounts, a second charge amount due to noise charge, which is accumulated charge other than the charge distributed and accumulated by the on / off processing of the transfer transistor; a frame in which the amount of charge accumulated in the charge accumulation unit is acquired as a third amount of charge in a state in which the light pulse is not irradiated from the light source unit and the pixel drive circuit turns off the transfer transistor so that charge is not accumulated in the charge accumulation unit from the photoelectric conversion element, is defined as a first noise charge amount acquisition frame; a frame in which the amount of charge accumulated in the charge accumulation unit is acquired as the second amount of charge in a state in which the pixel drive circuit turns off the transfer transistor after the light pulse is irradiated from the light source unit and charge is not accumulated in the charge accumulation unit from the photoelectric conversion element, is defined as a second noise charge amount acquisition frame; In the distance calculation step, the distance calculation unit subtracts the third charge amount from the second charge amount to obtain a fourth charge amount, which is a charge amount of a charge generated by the light pulse other than the photoelectric conversion element, when the number of second allocations in each of the frames after the second noise charge amount acquisition frame has changed with respect to the number of first allocations in each of the first noise charge amount acquisition frame and the second noise charge amount acquisition frame, the second allocation number is divided by the first allocation number to obtain an adjustment coefficient, and the result is multiplied by the fourth charge amount to calculate a fifth charge amount; The fifth charge amount is added to the third charge amount to correct the second charge amount and used to calculate the distance. A distance image capturing method comprising:

Images