Distance image capturing device and distance image capturing method

Abstract

To drive pixels so that a drive condition for a first charge storage unit in which charge is stored first and a drive condition for other charge storage units in which charge is stored next and onwards among a plurality of charge storage units are brought closer to each other.SOLUTION: The present invention: executes a drive for a unit store period as many times as needed for storage in a frame cycle; executes a store phase in which charges are sequentially stored in a charge storage unit at store timing in the unit store period and a discharge phase in which charges are discharged via a charge discharge unit; executes an adjustment phase synchronously with switching from the unit store period where the discharge phase was executed previously to the unit store period to be executed this time; and executes a first drive for causing the charge discharge unit not to discharge charges and the charge storage unit not to store charges and a second drive for discharging charges via the charge discharge unit in the adjustment phase, in the order stated. The time at which the second drive is executed is the same as a store drive time at which a store drive for causing one charge storage unit to store charges in the store phase is executed.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a distance image capturing device and a distance image capturing method.

Background Art

[0002] A time-of-flight (ToF) type distance image capturing device that measures the distance between a measuring device and an object based on the flight time of light in space (measurement space) by utilizing the fact that the speed of light is known has been realized (see, for example, Patent Document 1).

[0003] In the image sensor (pixels) of such a distance image capturing device, a photoelectric conversion unit, a plurality of charge storage units, a charge discharge unit, etc. are provided. Driving of a unit accumulation period in which pulsed light is irradiated onto a subject and the reflected light reflected by the measurement object (subject) is made incident on the pixel is repeatedly executed for the number of accumulations, thereby driving the pixel in one frame period. In the driving of the unit accumulation period, an accumulation phase for accumulating charges and a discharge phase for discharging charges are executed.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, even when the same amount of light is incident, a difference may occur in the amount of charge accumulated between the first charge storage unit that accumulates charges first in the accumulation phase and other charge storage units that accumulate charges after the second time, which has been a factor in deteriorating the accuracy of the distance. One of the factors that causes a difference in the amount of charge accumulated between the first charge accumulation unit and other charge accumulation units is the difference in driving conditions. In the driving during the unit accumulation period, the time for executing the accumulation phase is relatively short, and the time for executing the discharge phase is often set relatively long. For this reason, the first charge accumulation unit starts accumulating charge at the timing when the state in which charge is not discharged after a long period in which charge is discharged through the charge discharge unit in the discharge phase of the previous unit accumulation period is switched to the state in which charge is not discharged. On the other hand, the other charge accumulation units start accumulating charge at the timing when the short-time charge accumulation in the previous charge accumulation unit is completed. Such a difference in driving conditions is considered to be one of the factors that deteriorate the distance accuracy.

[0006] The present invention has been made based on the above problems, and an object thereof is to provide a distance image capturing device and a distance image capturing method capable of driving pixels so that the driving conditions of a first charge accumulation unit that first accumulates charge among a plurality of charge accumulation units and other charge accumulation units that accumulate charge after the second approach each other.

Means for Solving the Problems

[0007] The distance image capturing device of the present invention includes a light source unit that irradiates a subject with light pulses, a photoelectric conversion element that generates charges in response to the incident light, a charge discharge unit that discharges the charges generated by the photoelectric conversion element, and a distance image sensor in which a plurality of pixels each including a plurality of charge storage units that store the charges generated by the photoelectric conversion element are arranged in a two-dimensional matrix; a pixel drive circuit that distributes and stores charges in each of the charge storage units at a storage timing synchronized with the irradiation timing of the light pulses according to a frame period; a light receiving unit having the pixel drive circuit; and a distance image processing unit that calculates the distance to the subject based on the amount of charge stored in each of the charge storage units. In the frame period, the drive for a unit storage period is executed a plurality of times, and in the unit storage period, a storage phase in which charges are sequentially stored in the charge storage units at the storage timing and a discharge phase in which charges are discharged through the charge discharge unit are executed. An adjustment phase is executed at a timing when switching from the discharge phase of the previous unit storage period to the unit storage period to be executed this time. In the adjustment phase, a first drive in which charges are not discharged from the charge discharge unit and charges are not stored in the charge storage unit and a second drive in which charges are discharged through the charge discharge unit are sequentially executed. The time for executing the second drive is the same as the storage drive time for executing a storage drive in which charges are stored in one of the charge storage units in the storage phase.

[0008] The distance image capturing method of the present invention includes a light source unit that irradiates a subject with light pulses, a photoelectric conversion element that generates charges in response to the incident light, a charge discharge unit that discharges the charges generated by the photoelectric conversion element, and a plurality of charge accumulation units that accumulate the charges generated by the photoelectric conversion element. A distance image sensor in which a plurality of pixels are arranged in a two-dimensional matrix, a pixel driving circuit that distributes and accumulates charges in each of the charge accumulation units at an accumulation timing synchronized with the irradiation timing of the light pulses according to a frame period, a light receiving unit having the pixel driving circuit, and a distance image processing unit that calculates the distance to the subject based on the amount of charge accumulated in each of the charge accumulation units. The distance image capturing method performed by the distance image capturing apparatus includes executing driving for a unit accumulation period a plurality of times in the frame period, and performing an accumulation phase in which charges are sequentially accumulated in the charge accumulation units at the accumulation timing in the unit accumulation period and a discharge phase in which charges are discharged through the charge discharge unit. An adjustment phase is executed at a timing when switching from the discharge phase of the previous unit accumulation period to the current unit accumulation period. In the adjustment phase, a first drive in which charges are not discharged from the charge discharge unit and charges are not accumulated in the charge accumulation unit, and a second drive in which charges are discharged through the charge discharge unit are sequentially executed. The time for executing the second drive is the same as the accumulation drive time for executing an accumulation drive in which charges are accumulated in one of the charge accumulation units in the accumulation phase.

Advantages of the Invention

[0009] According to the present invention, it is possible to drive the pixels so that the driving conditions of the first charge accumulation unit that first accumulates charges and the other charge accumulation units that accumulate charges after the second one approach each other.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4A

Figure 4B

Figure 4C

Figure 5

Figure 6

Figure 7

Figure 8A

Figure 8B

Figure 8C

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Mode for Carrying Out the Invention

[0011] Hereinafter, the distance image capturing device according to the embodiment will be described with reference to the drawings.

[0012] FIG. 1 is a block diagram showing a schematic configuration of the distance image capturing device according to the embodiment. The distance image capturing device 1 includes, for example, a light source unit 2, a light receiving unit 3, and a distance image processing unit 4. FIG. 1 also shows a subject OB which is an object to be measured for distance in the distance image capturing device 1.

[0013] The light source unit 2 irradiates the subject OB with an optical pulse PO in accordance with the control from the distance image processing unit 4. The light source unit 2 is, for example, a surface-emitting semiconductor laser module such as a vertical cavity surface emitting laser (VCSEL). The light source unit 2 includes a light source device 21 and a diffuser plate 22.

[0014] The light source device 21 is a light source that emits laser light in a near-infrared wavelength band (for example, a wavelength band with a wavelength of 850 nm to 940 nm) that becomes the optical pulse PO for irradiating the subject OB. The light source device 21 is, for example, a semiconductor laser emitting element. The light source device 21 emits pulsed laser light in accordance with the control from the timing control unit 41.

[0015] The diffuser plate 22 is an optical component that diffuses the laser light in the near-infrared wavelength band emitted by the light source device 21 to the area of the surface for irradiating the subject OB. The pulsed laser light diffused by the diffuser plate 22 is emitted as the optical pulse PO and irradiates the subject OB.

[0016] The light receiving unit 3 receives the reflected light RL of the optical pulse PO reflected by the subject OB and outputs a pixel signal corresponding to the received reflected light RL. The light receiving unit 3 includes a lens 31 and a distance image sensor 32.

[0017] The lens 31 is an optical lens that guides the incident reflected light RL to the distance image sensor 32. The lens 31 emits the incident reflected light RL toward the distance image sensor 32 side and causes it to be received (incident) by the pixels provided in the light receiving area of the distance image sensor 32.

[0018] The distance image sensor 32 is an imaging device. The distance image sensor 32 includes a plurality of pixels arranged in a two-dimensional matrix. In each pixel of the distance image sensor 32, one photoelectric conversion element, a plurality of charge storage parts corresponding to this one photoelectric conversion element, and a component for distributing charges to each charge storage part are provided. That is, the pixel is an imaging device with a distribution configuration that distributes and accumulates charges in a plurality of charge storage parts.

[0019] The distance image sensor 32 distributes the charges generated by the photoelectric conversion element to each charge storage part according to the control from the timing control unit 41. Further, the distance image sensor 32 outputs a pixel signal corresponding to the amount of charge distributed to the charge storage part. In the distance image sensor 32, a plurality of pixels are arranged in a two-dimensional matrix, and the pixel signals for one frame corresponding to each pixel are output.

[0020] Here, the configuration of the distance image sensor 32 will be described with reference to FIG. 2. FIG. 2 is a block diagram showing the schematic configuration of the imaging device (distance image sensor 32) used in the distance image capturing device 1 of the embodiment.

[0021] As shown in FIG. 2, the distance image sensor 32 includes, for example, a light receiving region 320 in which a plurality of pixels 321 are arranged in a two-dimensional matrix, and a pixel driving circuit 322. The pixel driving circuit 322 includes, for example, a vertical scanning circuit 323 having a distribution operation, a horizontal scanning circuit 324, a pixel signal processing circuit 325, and a control circuit 326.

[0022] The light receiving region 320 is a region in which a plurality of pixels 321 are arranged in a two-dimensional matrix. In FIG. 2, an example of being arranged in an 8-row and 8-column two-dimensional matrix is shown. The pixel 321 accumulates charges corresponding to the amount of received light and outputs an accumulation signal corresponding to the accumulated amount of charges.

[0023] The control circuit 326 comprehensively controls the distance image sensor 32. The control circuit 326 controls the operations of the components of the distance image sensor 32, for example, in response to an instruction from the timing control unit 41 of the distance image processing unit 4. Note that the control of the components provided in the distance image sensor 32 may be configured such that the timing control unit 41 directly performs the control. In this case, it is also possible to omit the control circuit 326.

[0024] The vertical scanning circuit 323 controls the pixels 321 arranged in the light receiving region 320 row by row in response to the control from the control circuit 326. The vertical scanning circuit 323 causes the pixel signal processing circuit 325 to output a voltage signal corresponding to the amount of charge accumulated in each charge accumulation unit CS of the pixel 321. For example, the vertical scanning circuit 323 distributes and accumulates the charges converted by the photoelectric conversion element at the accumulation timing synchronized with the irradiation of the optical pulse PO in each charge accumulation unit of the pixel 321. Further, the vertical scanning circuit 323 discharges the charges converted by the photoelectric conversion element from a charge discharge unit (a charge discharge transistor GD described later) during a period different from the accumulation period for accumulating charges in the charge accumulation unit CS (for example, a read period).

[0025] The pixel signal processing circuit 325 performs predetermined signal processing (for example, noise suppression processing, A / D conversion processing, etc.) on the voltage signals output from the pixels 321 of each column to the corresponding vertical signal lines in response to the control from the control circuit 326.

[0026] The horizontal scanning circuit 324 sequentially outputs the signals output from the pixel signal processing circuit 325 in time series in response to the control from the control circuit 326. Thereby, the accumulation signals for one frame are sequentially output to the distance image processing unit 4. Hereinafter, it will be described on the assumption that the pixel signal processing circuit 325 performs A / D conversion processing and the accumulation signals are digital signals.

[0027] Here, the configuration of pixel 321 will be described with reference to FIG. 3. FIG. 3 is a circuit diagram showing an example of pixel 321. FIG. 3 shows an example of the configuration of one pixel 321 among a plurality of pixels 321 arranged in the light-receiving region 320. In this figure, an example in which pixel 321 includes four signal readout units RU (signal readout units RU1 to RU4) is shown.

[0028] Pixel 321 includes one photoelectric conversion element PD, a charge discharge transistor GD, and four signal readout units RU that output voltage signals from corresponding output terminals O. Each of the signal readout units RU includes a transfer transistor G, a floating diffusion FD, a charge storage capacitor C, a reset transistor RT, a source follower transistor SF, and a selection transistor SL. The charge storage unit CS is composed of the floating diffusion FD and the charge storage capacitor C.

[0029] In FIG. 3, each signal readout unit RU is distinguished by assigning one of the numbers from "1" to "4" after the symbol "RU" of the four signal readout units RU. Similarly, each component provided in the four signal readout units RU is also represented by indicating the number representing each signal readout unit RU after the symbol, so that each component is distinguished and represented by the corresponding signal readout unit RU.

[0030] In FIG. 3, each signal readout unit RU is distinguished by assigning one of the numbers from "1" to "4" after the symbol "RU" of the four signal readout units RU. Similarly, each component provided in the four signal readout units RU is also represented by indicating the number representing each signal readout unit RU after the symbol, so that each component is distinguished and represented by the corresponding signal readout unit RU.

[0031] In pixel 321, signal readout unit RU1 outputs a voltage signal from output terminal O1. Signal readout unit RU1 includes transfer transistor G1, floating diffusion FD1, charge storage capacitor C1, reset transistor RT1, source follower transistor SF1, and selection transistor SL1. Charge storage unit CS1 is constituted by floating diffusion FD1 and charge storage capacitor C1. Signal readout units RU2 to RU4 have the same configuration.

[0032] The photoelectric conversion element PD is an embedded photodiode that photoelectrically converts incident light to generate charges corresponding to the intensity of the incident light and accumulates the generated charges. The structure of the photoelectric conversion element PD may be arbitrary. The photoelectric conversion element PD may be, for example, a PN photodiode having a structure in which a P-type semiconductor and an N-type semiconductor are joined, or a PIN photodiode having a structure in which an I-type semiconductor is sandwiched between a P-type semiconductor and an N-type semiconductor. Further, the photoelectric conversion element PD is not limited to a photodiode, and may be, for example, a photoelectric conversion element of a photogate method.

[0033] Charge discharge transistor GD is a transistor for discarding the charges generated in photoelectric conversion element PD. When charge discharge transistor GD is controlled to be in an on state by pixel drive circuit 322, the charges generated in photoelectric conversion element PD are discarded (that is, photoelectric conversion element PD is reset).

[0034] Pixel drive circuit 322 drives pixel 321, distributes the charges generated by photoelectrically converting the light incident on photoelectric conversion element PD to each of the four charge storage units CS, and outputs respective voltage signals corresponding to the amounts of the distributed charges to pixel signal processing circuit 325.

[0035] For example, in driving the pixel 321, the pixel driving circuit 322 controls the storage driving signals TX1 to TX4 corresponding to the respective charge storage units CS1 to CS4 to be turned on in sequence in synchronization with the irradiation timing of the optical pulse PO. As a result, the transfer transistors G1 to G4 corresponding to the respective charge storage units CS are turned on in sequence, and charges are distributed and stored in the corresponding charge storage units CS. Thereby, charges are stored in the charge storage units CS1, CS2, CS3, and CS4 in this order.

[0036] Note that the pixel 321 is not limited to a configuration including four signal readout units RU as shown in FIG. 3, and any pixel having a configuration including a plurality of signal readout units RU may be used. That is, the number of signal readout units RU (charge storage units CS) provided in the pixels arranged in the distance image sensor 32 may be two, three, or five or more.

[0037] Also, in FIG. 3, an example in which the charge storage unit CS is composed of a floating diffusion FD and a charge storage capacitor C is shown. However, the charge storage unit CS only needs to be composed of at least the floating diffusion FD, and the pixel 321 may have a configuration without the charge storage capacitor C.

[0038] Returning to the description of FIG. 1, the distance image processing unit 4 controls the distance image capturing device 1 and calculates the distance to the subject OB. The distance image processing unit 4 includes a timing control unit 41, a distance calculation unit 42, and a measurement control unit 43.

[0039] The timing control unit 41 controls the timing for outputting various control signals required for measurement in accordance with the control of the measurement control unit 43. The various control signals here are, for example, a signal for controlling whether or not to irradiate the optical pulse PO, a signal for controlling whether or not to accumulate charges in the charge accumulation unit, a signal for setting the number of accumulation times per frame, and the like. The number of accumulation times is the number of times the process of distributing and accumulating charges in the charge accumulation unit CS is repeated, and corresponds to the preset distribution number in the frame period. The product of this number of accumulation times and the time (accumulation time) for accumulating charges in each charge accumulation unit per one process of distributing and accumulating charges is the exposure time.

[0040] The distance calculation unit 42 outputs distance information obtained by calculating the distance to the subject OB based on the pixel signal output from the distance image sensor 32. The distance calculation unit 42 calculates the delay time from when the optical pulse PO is irradiated until the reflected light RL is received based on the amount of charges accumulated in the plurality of charge accumulation units CS. The distance calculation unit 42 calculates the distance to the subject OB according to the calculated delay time. The measurement control unit 43 controls the timing control unit 41. For example, the measurement control unit 43 sets the number of accumulation times and the accumulation time width per frame, and controls the timing control unit 41 so that imaging is performed with the set content. That is, the measurement control unit 43 sets the frame period, and controls the timing control unit 41 so that imaging is performed with the set content.

[0041] Here, the problems of the present embodiment will be described with reference to FIG. 4 (FIGS. 4A to 4C).

[0042] FIG. 4A schematically shows the layout pattern of the pixel 321. As shown in FIG. 4A, the pixel 321 is an integrated circuit in which a transistor, transfer transistors G (transfer transistors G1 to G4), and charge discharge transistors GD (charge discharge transistors GD1 and GD2) are mounted on the photoelectric conversion element PD. The transistor is, for example, an n-channel type MOS transistor formed on a p-type semiconductor substrate, and is composed of each of a drain D (n-diffusion layer (diffusion layer of n-type impurities)), a source (n-diffusion layer), and a gate G. In the example of FIG. 4A, descriptions of transistors other than the transfer transistors G and the charge discharge transistors GD, specifically, transistors such as the charge discharge transistors GD, the source follower transistors SF1 to SF4, the selection transistors SL1 to SL4, and the reset transistors RT1 to RT4 are omitted. The photoelectric conversion element PD is formed in a shape of a long regular hexagon in which two opposite sides of a regular hexagon are longer than the other four sides. The transfer transistors G1 and G3 are arranged symmetrically with respect to an axis passing through the center of the long regular hexagon and perpendicular to the two long sides of the long regular hexagon on one of the two long sides of the long regular hexagon of the photoelectric conversion element PD. The transfer transistors G2 and G4 are arranged symmetrically with respect to an axis passing through the center of the long regular hexagon and perpendicular to the two long sides of the long regular hexagon on a side different from the side on which the transfer transistors G1 and G3 are arranged among the two long sides of the long regular hexagon of the photoelectric conversion element PD. The transfer transistors G1 and G2 are arranged symmetrically with respect to an axis passing through the center of the long regular hexagon and parallel to the two long sides of the long regular hexagon. The transfer transistors G3 and G4 are arranged symmetrically with respect to an axis passing through the center of the long regular hexagon and parallel to the two long sides of the long regular hexagon. The charge discharge transistors GD1 and GD2 are arranged symmetrically with respect to an axis passing through the center of the long regular hexagon and perpendicular to the two long sides of the long regular hexagon at the positions of the vertices where two adjacent sides of the four short sides of the long regular hexagon of the photoelectric conversion element PD are connected. In the example of this figure, the gate G of the charge discharge transistor GD1 is connected to a control signal GD_CL that controls the discharge of charges. When the control signal GD_CL becomes High (1), charges are discharged, and when it becomes Low (0 (zero)), the charges are not discharged. Also, the gate G of the charge discharge transistor GD2 is fixed to a fixed value (0 (zero)). Therefore, charges are not discharged at all times in the charge discharge transistor GD2. Also, each gate G of transfer transistors G1 to G4 is connected to control signals G1_CL to G4_CL that control the charge accumulation in charge accumulation units CS1 to CS4 as corresponding drains D. When the control signal Gk_CL goes High (1), charge is accumulated, and when it goes Low (0 (zero)), charge is not accumulated. Here, k is any one of 1 to 4.

[0043] Figure 4B is a timing chart showing a conventional driving example of pixel 321. The driving of the pixel is performed according to the frame period, and as shown in Figure 4B, an accumulation period and a readout period are provided in one frame. The accumulation period is a period for accumulating charge in the charge accumulation unit CS, and is a period for repeating the driving of pixel 321 shown in the unit accumulation period a predetermined number of times. The readout period is a period for reading out pixel signals Q corresponding to the amount of charge accumulated in each of the charge accumulation units CS.

[0044] In Figure 4B, the timing charts of elements corresponding to each item of "GD", "0 fixed", "G1" to "G4", and "LIGHT" are shown. "GD" indicates the operation timing of the control signal GD_CL that controls the charge discharge transistor GD1. "0 fixed" indicates that the charge discharge transistor GD2 is fixed to Low (0 (zero)). "G1" to "G4" indicate the operation timing of the control signals G1_CL to G4_CL that control the transfer transistors G1 to G4. "LIGHT" indicates the irradiation timing of the optical pulse PO. Specifically, it shows that in the on state (the state where the timing signal is set to High (1)), the irradiation state is achieved, and in the off state (the state where the timing signal is set to Low (0 (zero))), the light is turned off.

[0045] As shown in Figure 4B, the unit accumulation period includes an accumulation phase A and a discharge phase B. In the accumulation phase A, first, the charge discharge transistor GD is controlled to be in the off state, and then the transfer transistor G1 is controlled to be in the on state. When the accumulation time (for example, the accumulation time To set corresponding to the irradiation time of the optical pulse PO) elapses after the charge discharge transistor GD is controlled to be in the off state, the transfer transistor G1 is controlled to be in the off state. Here, the period during which the transfer transistor G1 is controlled to be in the on state is the accumulation drive time Tc. The accumulation drive time Tc is set to be smaller than the accumulation time To. During the non-accumulation drive period (=To - Tc) in which the transfer transistor G1 is controlled to be in the off state within the accumulation time To, the charges converted by the photoelectric conversion element PD are accumulated in the photoelectric conversion element PD (not in the charge storage unit CS1). By controlling the transfer transistor G1 to be in the on state, the charges accumulated in the photoelectric conversion element PD during the non-accumulation drive period move from the photoelectric conversion element PD to the floating diffusion FD1 and are accumulated in the charge storage unit CS1. Also, the charges converted by the photoelectric conversion element PD during the accumulation drive time Tc when the transfer transistor G1 is controlled to be in the on state are accumulated in the charge storage unit CS1. That is, the transfer transistor G1 is controlled to be in the on state during the accumulation drive time Tc, and the charges converted by the photoelectric conversion element PD during the accumulation time To are accumulated in the charge storage unit CS1. At the timing when the transfer transistor G1 is controlled to be in the off state, the optical pulse PO is irradiated for the irradiation time To. Also, the transfer transistor G1 is controlled to be in the off state, and then the transfer transistor G2 is controlled to be in the on state. When the accumulation time To elapses after the transfer transistor G1 is controlled to be in the off state, the transfer transistor G2 is controlled to be in the off state. The period during which the transfer transistor G2 is controlled to be in the on state is the accumulation drive time Tc. The transfer transistor G2 is controlled to be in the off state, and then the transfer transistor G3 is controlled to be in the on state. When the accumulation time To elapses after the transfer transistor G2 is controlled to be in the off state, the transfer transistor G3 is controlled to be in the off state. The period during which the transfer transistor G3 is controlled to be in the on state is the accumulation drive time Tc. The transfer transistor G3 is controlled to be in the off state, and then the transfer transistor G4 is controlled to be in the on state. When the accumulation time To has elapsed since the transfer transistor G3 was controlled to be in the off state, the transfer transistor G4 is controlled to be in the off state. The period during which the transfer transistor G4 is controlled to be in the on state is the accumulation drive time Tc. Thereafter, the charge discharge transistor GD is controlled to be in the on state. In the discharge phase B, until the start of the next unit accumulation period, the charge discharge transistor GD is maintained in the on state, and the transfer transistors G1 to G4 are maintained in the off state.

[0046] FIG. 4C is a schematic diagram showing the magnitudes of the signal values of the pixel signals Q1 to Q4 corresponding to the amounts of charge accumulated in the charge accumulation units CS1 to CS4 when the driving shown in FIG. 4B is performed. Examples of patterns P1 and P2 are shown in FIG. 4C. In pattern P1, the signal value of the pixel signal Q1 shows a smaller value than the other pixel signals Q2 to Q4 because GD (charge discharge) is excessive compared to the other pixel signals. In pattern P2, the signal value of the pixel signal Q1 shows a larger value than the other pixel signals Q2 to Q4 because GD (charge discharge) is insufficient compared to the other pixel signals. It is assumed that the amount of light incident on the pixel 321 does not change (is constant) at the timing of accumulating charge in each of the charge accumulation units CS1 to CS4. In this way, there is a tendency for a difference to occur in the amount of charge accumulated between the pixel signal Q1 and the other pixel signals Q2 to Q4. One of the factors for this is the difference in driving conditions. As shown in FIG. 4B, in the driving during the unit accumulation period, the pixel 321 is often driven such that the accumulation phase A is implemented for a relatively short time and the discharge phase B is implemented for a relatively long time. For this reason, the charge accumulation unit CS1 starts accumulating charge at the timing when the state where charge is discharged in the discharge phase B of the previous unit accumulation period continues for a long time and then switches to a state where charge is not discharged. On the other hand, the other charge accumulation units start accumulating charge at the timing when the short-time charge accumulation to the charge accumulation unit CS in the previous stage is completed. Specifically, the charge accumulation unit CS2 starts accumulating charge at the timing when the short-time charge accumulation to the charge accumulation unit CS1 in the previous stage is completed. The charge accumulation unit CS3 starts accumulating charge at the timing when the short-time charge accumulation to the charge accumulation unit CS2 in the previous stage is completed. The charge accumulation unit CS4 starts accumulating charge at the timing when the short-time charge accumulation to the charge accumulation unit CS3 in the previous stage is completed. Such a difference in driving conditions has been one of the factors for the deterioration of the distance accuracy. As a countermeasure, in this embodiment, among the plurality of charge accumulation units CS1 to CS4, the driving conditions of the charge accumulation unit CS1 that first accumulates charge and the other charge accumulation units CS2 to CS4 that accumulate charge after the second are made to approach each other so as to drive the pixel. Hereinafter, a method of driving the pixel 321 of this embodiment will be described with reference to FIGS. 5 to 13. FIGS. 5 to 7 are timing charts showing driving examples of the pixel 321 of the embodiment. FIG. 8 (FIGS. 8A to 8C) is a diagram showing a layout example of the pixel 321 of the embodiment. FIGS. 9 to 13 are timing charts showing driving examples of the pixel 321 of the embodiment.

[0047] The items of "GD", "0 fixed", "G1" to "G4", and "LIGHT" in FIGS. 5 to 7 are the same as those in FIG. 4B, and thus the description thereof is omitted.

[0048] FIG. 5 shows a driving example (first example) of the pixel 321 of the present embodiment. As shown in FIG. 5, in the present embodiment, the distance image capturing device 1 executes the adjustment phase X at the timing TG1 in FIG. 4B, that is, the timing at which the unit accumulation period to be executed this time is switched from the discharge phase B of the previously executed unit accumulation period. In the adjustment phase X, the distance image capturing device 1 sequentially executes the first drive KN1 and the second drive KN2. In the first drive KN1, no charge is discharged to the charge discharge unit (charge discharge transistor GD), and no charge is accumulated in the charge accumulation unit CS. In the second drive KN2, charge is discharged through the charge discharge unit (charge discharge transistor GD). Here, the execution time Tg for executing the second drive KN2 is the same as the accumulation drive time Tc for executing the accumulation drive for accumulating charge in one charge accumulation unit CS in the accumulation phase, that is, Tg = Tc. In this way, in the present embodiment, the adjustment phase X is provided in the driving of the unit accumulation period of the pixel 321. Thereby, immediately before the charge is accumulated in the charge accumulation unit CS1, the charge discharge unit (charge discharge transistor GD) is pulse-driven so that the charge accumulation in the charge accumulation unit CS1 is started at the timing as if the short-time charge accumulation in the previous charge accumulation unit has been completed. Therefore, the driving conditions of the charge accumulation unit CS1 and the other charge accumulation units CS2 to CS4 for accumulating charge after the second one can be made closer, and it is possible to suppress the deterioration of the distance accuracy.

[0049] FIG. 6 shows a driving example (second example) of the pixel 321 of the present embodiment. As shown in FIG. 6, the distance image capturing device 1 may alternately execute the first drive KN1 and the second drive KN2 a plurality of times in the adjustment phase X.

[0050] FIG. 7 shows a driving example (third example) of the pixel 321 of the present embodiment. As shown in FIG. 7, the distance image capturing device 1 may execute the driving of the adjustment phase X instead of maintaining the charge discharge transistor GD in the on state continuously in the discharge phase B, that is, alternately execute the first drive KN1 and the second drive KN2.

[0051] FIG. 8A schematically shows the layout of pixel 321 in which charge discharge transistor GD2 of FIG. 4A functions as a charge discharge section. In the example of this figure, gate G of charge discharge transistor GD1 is connected to control signal GD1_CL that controls charge discharge. When control signal GD1_CL goes High (1), charge is discharged, and when it goes Low (0 (zero)), charge is not discharged. Also, gate G of charge discharge transistor GD2 is connected to control signal GD2_CL that controls charge discharge. When control signal GD2_CL goes High (1), charge is discharged, and when it goes Low (0 (zero)), charge is not discharged. By providing two charge discharge sections (charge discharge transistors GD1 and GD2) in pixel 321, different applications can be used appropriately. That is, it can be used separately for the charge discharge section (charge discharge transistor GD1) used for applications that discharge charge for a long time, and the charge discharge section (charge discharge transistor GD2) used for applications that align the driving conditions of charge storage section CS1 and other charge storage sections CS2 to CS4. This makes it easy to align the driving conditions of charge storage section CS while maintaining the function of discharging charge.

[0052] FIG. 8B schematically shows the layout in pixel 321 of FIG. 8A with the buffer configurations of control signals G1_CL to G4_CL and GD2_CL indicated by reference sign GP aligned. By aligning the buffer configurations, for example, by using the same power supply for the buffer power supply, the waveforms (rise waveforms and fall waveforms) of the control signals of charge discharge transistor GD2 can be made equivalent to the waveforms (rise waveforms and fall waveforms) of the control signals of transfer transistors G1 to G4. Thereby, the electrical characteristics of the control waveform of the charge discharge section (charge discharge transistor GD2) used for the purpose of aligning the driving conditions of charge storage section CS can be made closer to the electrical characteristics of the control waveform of charge storage section CS.

[0053] FIG. 8C schematically shows a layout in which the photoelectric conversion element PD is formed in a regular hexagon shape. In this way, the buffer configurations of the control signals G1_CL to G4_CL and GD2_CL indicated by the symbol GP are made uniform, and the six gates G (the gates G of the two charge discharge transistors GD and the four transfer transistors G1 to G4) included in the pixel 321 are symmetrically arranged so as to be point-symmetrical with respect to the center of the regular hexagon. Thereby, the load applied to each gate can be made equal, and the electrical characteristics of each control waveform can be made closer to each other.

[0054] FIGS. 9 to 10 show drive examples using two charge discharge units (charge discharge transistors GD1, GD2) as shown in FIG. 8.

[0055] FIG. 9 shows a drive example (fourth example) of the pixel 321 of the present embodiment. In the distance image capturing device 1, in the adjustment phase X, the charge discharge unit (charge discharge transistor GD1) used for charge discharge discharges charges, and the charge discharge unit (charge discharge transistor GD2) used for aligning drive conditions executes the first drive KN1 and the second drive KN2 in order.

[0056] FIG. 10 shows a drive example (fifth example) of the pixel 321 of the present embodiment. The distance image capturing device 1 executes the adjustment phase X at the timing of switching from the discharge phase B of the previously executed unit accumulation period to the unit accumulation period to be executed this time, and also executes the adjustment phase X at the timing when the accumulation phase A ends.

[0057] FIGS. 11 to 13 show drive examples in which one frame is composed of a plurality of sub-frames. Here, a case where two sub-frames, a first sub-frame and a second sub-frame, are provided in one frame will be exemplified and described. FIGS. 11 to 13 also show drive examples using one charge discharge unit in the same manner as the drive shown in FIG. 4A.

[0058] In FIGS. 11 to 13, timing charts of elements corresponding to respective items of "GD.1", "G1.1" to "G4.1", "GD.2", "G1.2" to "G4.2", and "LIGHT" are shown. "GD.1" shows the operation timing of the control signal GD_CL that controls the charge discharge transistor GD1 of the first sub-frame. "G1.1" to "G4.1" show the operation timing of the control signals G1_CL to G4_CL that control the transfer transistors G1 to G4 of the first sub-frame. "GD.2" shows the operation timing of the control signal GD_CL that controls the charge discharge transistor GD1 of the second sub-frame. "G1.2" to "G4.2" show the operation timing of the control signals G1_CL to G4_CL that control the transfer transistors G1 to G4 of the second sub-frame. "LIGHT" shows the irradiation timing of the optical pulse PO. The irradiation timing of the optical pulse PO is common to the first sub-frame and the second sub-frame.

[0059] In FIGS. 11 to 13, the distance image capturing device 1 first repeatedly executes the driving of the first unit accumulation period as the driving of the first sub-frame a predetermined number of times (the first accumulation number of times), and acquires pixel signals Q1 to Q4 as the driving result of the first sub-frame and stores them in the memory. Next, the driving of the second unit accumulation period is repeatedly executed as the driving of the second sub-frame a predetermined number of times (the second accumulation number of times), and pixel signals Q1 to Q4 are acquired as the driving result of the second sub-frame and stored in the memory. Then, the distance image capturing device 1 calculates the distance using the pixel signals Q1 to Q4 that are the driving results of the first sub-frame stored in the memory and the pixel signals Q1 to Q4 as the driving result of the second sub-frame.

[0060] FIG. 11 shows a driving example (sixth example) of the pixel 321 of the present embodiment. In FIG. 11, in the first unit accumulation period, after executing the adjustment phase X, the distance image capturing device 1 sequentially executes a first accumulation phase A1, a first discharge phase B1, a second accumulation phase A2, and a second discharge phase B2. Further, in the second unit accumulation period, the distance image capturing device 1 does not provide a charge accumulation unit dedicated to external light, and executes the accumulation phase A at the same timing as the timing of executing the first discharge phase B1 in the first unit accumulation period, and then executes the discharge phase B.

[0061] In the first unit accumulation period, the charge accumulation unit CS1 is used as a charge accumulation unit dedicated to external light that accumulates only the external light component. The reason for providing the first discharge phase B1 in the first unit accumulation period is to delay the accumulation timing of charging each of the charge accumulation units CS2 to CS4 with respect to the irradiation timing of the light pulse PO compared to the driving shown in FIG. 5. By delaying the accumulation timing, a component of the reflected light RL that arrives after being reflected by the subject OB at a relatively long distance is accumulated in any of the charge accumulation units CS2 to CS4. In the second unit accumulation period, the charge accumulation unit CS1 is used as a charge accumulation unit that accumulates a component of the reflected light RL that arrives after being reflected by the subject OB at a short distance. For each of the charge accumulation units CS2 to CS4, a component of the reflected light RL that arrives after being reflected by the subject OB at a relatively long distance is accumulated in the same manner as in the first unit accumulation period. Generally, the reflected light RL that arrives after being reflected by the subject OB at a short distance has a large amount of light, while the reflected light RL that arrives after being reflected by the subject OB at a long distance has a small amount of light. By performing the driving shown in FIG. 11, the number of times of accumulating the component of the reflected light RL that arrives after being reflected by the subject OB at a short distance can be reduced, and the number of times of accumulating the component of the reflected light RL that arrives after being reflected by the subject OB at a long distance can be increased. By reducing the number of times of accumulating the component of the reflected light RL that arrives after being reflected by the subject OB at a short distance, it is possible to suppress a situation where the amount of charge accumulated in the charge accumulation unit CS saturates and the distance cannot be calculated accurately. By increasing the number of times of accumulating the component of the reflected light RL that arrives after being reflected by the subject OB at a long distance, it is possible to accumulate an amount of charge in the charge accumulation unit CS that enables accurate distance calculation.

[0062] Specifically, in the adjustment phase X of the first unit accumulation period, the distance image capturing device 1 sequentially executes the first drive KN1 and the second drive KN2. In the first accumulation phase A1, during the accumulation drive time Tc of the charge accumulation unit CS1, an accumulation drive for accumulating charge is executed to accumulate the charge of the external light component in the charge accumulation unit CS. The distance image capturing device 1 discharges the charge through the charge accumulation unit (charge discharge transistor GD) in the first discharge phase B1. In the second accumulation phase A2, the distance image capturing device 1 sequentially executes an accumulation drive for accumulating charge in each of the charge accumulation units CS2 to CS4 for the accumulation drive time Tc. The distance image capturing device 1 discharges the charge through the charge accumulation unit (charge discharge transistor GD) in the second discharge phase B2. Also, in the adjustment phase X of the second unit accumulation period, the distance image capturing device 1 sequentially executes the first drive KN1 and the second drive KN2. In the accumulation phase A, the distance image capturing device 1 sequentially executes an accumulation drive for accumulating charge in each of the charge accumulation units CS1 to CS4 for the accumulation drive time Tc. The distance image capturing device 1 discharges the charge through the charge accumulation unit (charge discharge transistor GD) in the discharge phase B.

[0063] In this way, in each of the plurality of sub-frames provided in one frame, the distance image capturing device 1 executes the adjustment phase X at the timing of switching from the discharge phase B of the previously executed unit accumulation period to the unit accumulation period to be executed this time. Thereby, before accumulating charge in the charge accumulation unit CS1 in the first accumulation phase A1 of the first unit accumulation period and before accumulating charge in the charge accumulation unit CS1 in the accumulation phase A of the second unit accumulation period, by providing the adjustment phase X respectively, the driving and driving conditions of the other charge accumulation units CS2 to CS4 can be made closer.

[0064] FIG. 12 shows a driving example (seventh example) of the pixel 321 of the present embodiment. The first accumulation phase A1, the first discharge phase B1, the second accumulation phase A2, the second discharge phase B2 in the first unit accumulation period shown in FIG. 12, and the accumulation phase A and the discharge phase B in the second unit accumulation period are the same as those in FIG. 11. As shown in FIG. 12, the distance image capturing device 1 may start executing the adjustment phase X of the second unit accumulation period at the same timing as the adjustment phase X of the first unit accumulation period, and execute the adjustment phase X of the second unit accumulation period for a longer time than the adjustment phase X of the first unit accumulation period. In this case, the distance image capturing device 1 alternately executes the first drive KN1 and the second drive KN2 a plurality of times in the adjustment phase X of the second unit accumulation period.

[0065] Specifically, in the example of FIG. 12, the adjustment phase X of the first unit accumulation period starts at timing TG1. Also, the first accumulation phase A1 in the first unit accumulation period starts at timing TG2. The accumulation phase A in the second unit accumulation period starts at timing TG3. Timing TG2 arrives earlier than timing TG3. If the elapsed time from timing TG1 to timing TG3 (i.e., the time for executing adjustment phase X in the second unit accumulation period) is greater than the time for executing the first drive KN1 and the second drive KN2 (once each), the distance image capturing device 1 may alternately execute the first drive KN1 and the second drive KN2 a plurality of times in adjustment phase X of the second unit accumulation period.

[0066] FIG. 13 shows a driving example (eighth example) of pixel 321 of the present embodiment. In the driving example of FIG. 13, components of the reflected light RL arriving after being reflected by a subject OB that is even further away than the driving shown in FIGS. 11 and 12 are accumulated in pixel 321. Specifically, in the first unit accumulation period, a range shift period sft is provided between the irradiation timing and the accumulation timing for accumulating charges in charge accumulation unit CS2. In the second unit accumulation period, in conjunction with the driving of the first unit accumulation period, an interlock period tm is provided from the irradiation timing to the accumulation timing for accumulating charges in charge accumulation unit CS1. When such a range shift period sft is provided, the distance image capturing device 1 may alternately execute the first drive KN1 and the second drive KN2 a plurality of times during the range shift period sft.

[0067] Specifically, in the example of FIG. 13, the charge accumulation in charge accumulation unit CS1 in the first unit accumulation period ends at timing TG4, and the charge accumulation in the next charge accumulation unit CS2 starts from timing TG5. If the elapsed time from timing TG4 to timing TG5 (i.e., the time for executing adjustment phase X2 in the first unit accumulation period) is greater than the time for executing the first drive KN1 and the second drive KN2 (once each), the distance image capturing device 1 may alternately execute the first drive KN1 and the second drive KN2 a plurality of times in adjustment phase X of the second unit accumulation period. Incidentally, when the range shift period sft is provided during the driving with a frame period without providing a sub-frame, the distance image capturing device 1 may execute the adjustment phase X during the range shift period. For example, assume that the distance image capturing device 1 provides the range shift period sft between the timings TG4 and TG5 during the accumulation phase of the unit accumulation period in the frame period. In this case, the distance image capturing device 1 ends the charge accumulation in the charge accumulation unit CS1 at the timing TG4 and starts the charge accumulation in the next charge accumulation unit CS2 at the timing TG5 during the accumulation phase of the unit accumulation period. Since the elapsed time from the timing TG4 to the timing TG5 is longer than the time for executing the first drive KN1 and the second drive KN2 once each, the distance image capturing device 1 alternately executes the first drive KN1 and the second drive KN2 a plurality of times during the elapsed time.

[0068] As described above, the distance image capturing apparatus 1 of the embodiment includes a light source unit 2, a light receiving unit 3, and a distance image processing unit 4. The light source unit 2 irradiates a subject OB with an optical pulse PO. The light receiving unit 3 includes a distance image sensor 32 (pixel circuit) in which a plurality of pixels 321 each including a photoelectric conversion element PD and three or more charge storage units CS are arranged in a two-dimensional matrix, and a pixel driving circuit 322. The photoelectric conversion element PD generates charges corresponding to the incident light. The charge storage unit CS stores charges. The pixel driving circuit 322 distributes and stores charges in each of the charge storage units CS at a storage timing synchronized with the irradiation timing of irradiating the optical pulse PO in a frame period. The distance image processing unit 4 calculates the distance to the subject OB based on the amount of charges stored in each of the charge storage units CS. The distance image capturing apparatus 1 executes driving in a unit storage period a storage number of times in a frame period. The distance image capturing apparatus 1 executes a storage phase A and a discharge phase B in a unit storage period, and executes an adjustment phase X at a timing when switching from the discharge phase B of the previously executed unit storage period to the unit storage period to be executed this time. In the storage phase A, charges are sequentially stored in the charge storage unit CS at a storage timing synchronized with the irradiation timing. In the discharge phase B, the charges are discharged via a charge discharge unit. The distance image capturing apparatus 1 executes a first drive KN1 and a second drive KN2 in this order in the adjustment phase X. In the first drive KN1, the charge discharge unit is not caused to discharge charges and the charge storage unit is not caused to store charges. In the second drive KN2, the charges are discharged via the charge discharge unit. The time Tg for executing the second drive KN2 is the same as the storage drive time Tc for executing a storage drive for storing charges in one charge storage unit CS in the storage phase A. Thereby, in the distance image capturing apparatus 1 of the embodiment, the pixels can be driven so as to mitigate the difference in driving conditions between a first charge storage unit that stores charges first among the plurality of charge storage units and other charge storage units that store charges second and later.

[0069] Also, in the distance image capturing device 1 of the embodiment, as shown in FIG. 6, in the adjustment phase X, the first drive KN1 and the second drive KN2 may be executed a plurality of times. Thereby, an effect similar to the above-described effect can be obtained.

[0070] Also, in the distance image capturing device 1 of the embodiment, as shown in FIG. 7, in the discharge phase B, the first drive KN1 and the second drive KN2 in the adjustment phase X may be alternately executed. Thereby, in the distance image capturing device 1 of the embodiment, an effect similar to the above-described effect can be obtained.

[0071] Also, in the distance image capturing device 1 of the embodiment, the charge discharging unit includes two parts: a first charge discharging unit (charge discharging transistor GD1) and a second charge discharging unit (charge discharging transistor GD2). As shown in FIGS. 9 to 10, in the adjustment phase X of the distance image capturing device 1, the first charge discharging unit (charge discharging transistor GD1) discharges the charge, and the second charge discharging unit (charge discharging transistor GD2) may execute the first drive KN1 and the second drive KN2 in order. Thereby, it becomes easy to make the driving conditions of the charge storage unit CS uniform while maintaining the function of discharging the charge.

[0072] Also, in the distance image capturing device 1 of the embodiment, as shown in FIG. 10, after executing the adjustment phase X at the timing when the accumulation phase A ends, the discharge phase B may be executed. Thereby, the driving conditions of all of the charge storage units CS1 to CS4 can be made closer.

[0073] Further, in the distance image capturing device 1 according to the embodiment, as shown in FIGS. 11 to 13, a plurality of sub-frames may be provided in a frame period, and the pixel 321 may be driven such that the accumulation timing with respect to the irradiation timing is different for each of the plurality of sub-frames. The distance image capturing device 1 executes the adjustment phase X at the timing when switching from the discharge phase B of the previously executed unit accumulation period to the unit accumulation period to be executed this time in each of the plurality of sub-frames. Thereby, in the distance image capturing device 1 according to the embodiment, in driving the sub-frames, the driving conditions of the charge storage unit CS1 can be made closer to the driving conditions of the other charge storage units CS2 to CS4.

[0074] Further, in the distance image capturing device 1 according to the embodiment, as shown in FIGS. 11 to 13, when two sub-frames are provided in a frame, the adjustment phase X may be executed at the same timing TG1 (first timing) with respect to the irradiation timing of the optical pulse PO in the first sub-frame and the second sub-frame. The distance image capturing device 1 executes the adjustment phase X at the timing TG1 in the first sub-frame, executes the first accumulation phase A1 at the timing TG2 (second timing), and executes the accumulation phase at the timing TG3 (third timing) in the second sub-frame. The timing TG2 arrives earlier than the timing TG3. The distance image capturing device 1 starts executing the adjustment phase X at the timing TG1 in the second unit accumulation period of the second sub-frame, and when the elapsed time from the timing TG2 to the timing TG3 is longer than the time for executing each of the first drive KN1 and the second drive TN2 (once each), the first drive KN1 and the second drive KN2 are alternately executed during the elapsed time. Thereby, in the distance image capturing device 1 according to the embodiment, in driving the frame period including the sub-frames, the driving conditions of the charge storage unit CS1 can be made closer to the driving conditions of the other charge storage units CS2 to CS4.

[0075] Also, in the distance image capturing device 1 of the embodiment, as shown in FIG. 13, when providing a range shift period sft, in that range shift period sft, instead of the discharge phase B, an adjustment phase X may be executed. The distance image capturing device 1 ends the charge accumulation in the charge accumulation unit CS1 at timing TG4 (the fourth timing) in the accumulation phase A, and starts the charge accumulation in the charge accumulation unit CS2 that accumulates charge next to the charge accumulation unit CS1 at timing TG5 (the fifth timing). The elapsed time from timing TG4 to timing TG5 is greater than the time for executing each of the first drive KN1 and the second drive KN2. The distance image capturing device 1 alternately executes the first drive KN1 and the second drive KN2 during the elapsed time. Thereby, in the distance image capturing device 1 of the embodiment, in the drive including the range shift period, the drive conditions of each of the charge accumulation units CS1 to CS4 can be made closer to each other.

[0076] All or part of the distance image capturing device 1 and the distance image processing unit 4 in the above-described embodiment may be realized by a computer. In that case, a program for realizing this function may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be read into a computer system and executed to realize it. Here, the "computer system" shall include hardware such as an OS and peripheral devices. Further, the "computer-readable recording medium" refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, etc., and a storage device such as a hard disk built in a computer system. Furthermore, the "computer-readable recording medium" refers to a medium that dynamically holds a program for a short time, such as a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line, and also includes a volatile memory inside a computer system that becomes a server or a client in that case and holds a program for a certain period of time. Also, the above program may be for realizing a part of the aforementioned functions, and may further be realizable in combination with a program already recorded in a computer system for realizing the aforementioned functions, and may also be realized using a programmable logic device such as an FPGA.

[0077] As described above, the embodiments of the present invention have been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and designs and the like within the scope not departing from the gist of the present invention are also included.

Explanation of Reference Numerals

[0078] 1... Distance image capturing device 2... Light source unit 3... Light receiving unit 32... Distance image sensor 321... Pixel 322... Pixel drive circuit 4... Distance image processing unit 41... Timing control unit 42... Distance calculation unit CS... Charge storage unit PO… Optical pulse RL… Reflected light

Claims

1. A light source unit that irradiates a subject with light pulses, a photoelectric conversion element that generates charges in response to the incident light, a charge discharge unit that discharges the charges generated by the photoelectric conversion element, and a plurality of charge storage units that store the charges generated by the photoelectric conversion element, and a distance image sensor in which a plurality of pixels are arranged in a two-dimensional matrix, and a pixel drive circuit that distributes and stores charges in each of the charge storage units at a storage timing synchronized with the irradiation timing of the light pulses according to a frame period, and a light receiving unit having the same; a distance image processing unit that calculates the distance to the subject based on the amount of charge stored in each of the charge storage units; comprising performing driving of a unit storage period a storage number of times in the frame period, executing a storage phase in which charges are sequentially stored in the charge storage unit at the storage timing in the unit storage period and a discharge phase in which charges are discharged through the charge discharge unit, and executing an adjustment phase at a timing when switching from the discharge phase of the previous unit storage period to the unit storage period to be executed this time, executing in order a first drive in which charges are not discharged to the charge discharge unit and charges are not stored in the charge storage unit and a second drive in which charges are discharged through the charge discharge unit in the adjustment phase, the time for executing the second drive is the same as the storage drive time for executing a storage drive for storing charges in one of the charge storage units in the storage phase, a distance image imaging device.

2. executing the first drive and the second drive a plurality of times in the adjustment phase, The distance image imaging device according to claim 1.

3. in the discharge phase, alternately executing the first drive and the second drive in the adjustment phase, The distance image imaging device according to claim 1.

4. the charge discharge unit consists of two, a first charge discharge unit and a second charge discharge unit, in the adjustment phase, the first charge discharge unit discharges charges, and the second charge discharge unit executes the first drive and the second drive in order, The distance image imaging device according to claim 1.

5. executing the discharge phase after executing the adjustment phase at the timing when the storage phase ends, The distance image imaging device according to claim 4.

6. a plurality of sub-frames are provided in the frame period, In each of the plurality of sub-frames, the accumulation timing with respect to the irradiation timing is different from each other, In each of the plurality of sub-frames, the adjustment phase is executed at a timing when switching from the discharge phase of the previously executed unit accumulation period to the unit accumulation period to be executed this time. The distance image capturing device according to claim 1.

7. In the frame period, two sub-frames, a first sub-frame and a second sub-frame, are provided, In the first sub-frame, the adjustment phase is executed at a first timing when switching from the discharge phase of the previously executed unit accumulation period to the unit accumulation period to be executed this time, and the accumulation phase is executed at a second timing based on the irradiation timing. In the second sub-frame, the accumulation phase is executed at a third timing based on the irradiation timing. The second timing arrives earlier than the third timing. In the second sub-frame, the execution of the adjustment phase is started at the first timing. The elapsed time from the second timing to the third timing is greater than the time for executing each of the first drive and the second drive. The first drive and the second drive are alternately executed during the elapsed time. The distance image capturing device according to claim 1.

8. In the accumulation phase, the charge accumulation in the first charge accumulation part among the charge accumulation parts is ended at a fourth timing, and the charge accumulation in the second charge accumulation part for accumulating charge next to the first charge accumulation part is started at a fifth timing. The elapsed time from the fourth timing to the fifth timing is greater than the time for executing each of the first drive and the second drive. The first drive and the second drive are alternately executed during the elapsed time. The distance image capturing device according to claim 1.

9. A distance image capturing apparatus includes a light source unit that irradiates a subject with light pulses, a photoelectric conversion element that generates charges in response to the incident light, a charge discharging unit that discharges the charges generated by the photoelectric conversion element, and a plurality of charge storage units that store the charges generated by the photoelectric conversion element. A plurality of pixels are arranged in a two-dimensional matrix. The distance image capturing apparatus further includes a pixel driving circuit that distributes and stores charges in each of the charge storage units at a storage timing synchronized with the irradiation timing of the light pulses according to a frame period, and a distance image processing unit that calculates the distance to the subject based on the amount of charge stored in each of the charge storage units. A distance image capturing method performed by the distance image capturing apparatus includes: performing driving for a unit storage period a storage number of times within the frame period; executing a storage phase in which charges are sequentially stored in the charge storage units at the storage timing within the unit storage period and a discharge phase in which charges are discharged through the charge discharging unit, and executing an adjustment phase at a timing when switching from the discharge phase of the previous unit storage period to the current unit storage period; executing, in the adjustment phase, a first drive in which charges are not discharged from the charge discharging unit and charges are not stored in the charge storage unit, and a second drive in which charges are discharged through the charge discharging unit, in this order; wherein the time for executing the second drive is the same as the storage drive time for executing a storage drive in which charges are stored in one of the charge storage units in the storage phase; A distance image capturing method.

Images