Range image capturing device and range image capturing method

Abstract

The invention relates to a range image capturing apparatus and a range image capturing method. In a frame period, the distance image capturing apparatus performs driving for a unit accumulation period in which an accumulation phase in which a charge accumulation unit accumulates charges in order at an accumulation timing and a discharge phase in which the charges are discharged via a charge discharge unit are performed at an accumulation number of times. A timing adjustment stage is switched from the discharge stage of the previous unit accumulation period to the timing adjustment stage of the unit accumulation period executed this time. In the adjustment stage, a first drive that does not cause the charge discharge unit to discharge charges and does not cause the charge accumulation unit to accumulate charges, and a second drive that discharges charges via the charge discharge unit are sequentially performed. The time during which the second drive is performed is the same as the accumulation drive time during which accumulation drive for causing one of the charge accumulation units to accumulate charges is performed in the accumulation phase.

Description

Technical Field

[0001] The present invention relates to a distance image capturing device and a distance image capturing method.

[0002] This application claims the priority of Japanese Patent Application No. 2023-216662 filed in Japan on December 22, 2023, and incorporates its content herein. Background Art

[0003] A distance image capturing device using a Time of Flight (hereinafter referred to as "ToF") method that measures the distance between a detector and an object based on the known speed of light and the flight time of light in space (measurement space) has been realized (for example, refer to Japanese Patent No. 4235729).

[0004] In the imaging element (pixel) of such a distance image capturing device, a photoelectric conversion section, a plurality of charge storage sections, a charge discharge section, etc. are provided. By repeatedly performing the driving of the unit storage period in which pulsed light is irradiated onto the subject and the reflected light reflected by the object to be measured (subject) is incident on the pixel for the storage times, the driving of the pixel for one frame period is thus executed. In the driving of the unit storage period, a storage stage for storing charges and a discharge stage for discharging charges are executed.

[0005] However, even when light of the same light amount is incident, a difference in the amount of stored charges sometimes occurs between the first charge storage section that first stores charges in the storage stage and other charge storage sections that store charges after the second one, which becomes an important factor in the deterioration of distance accuracy.

[0006] As one of the important factors for the difference in the amount of stored charges between the first charge storage section and other charge storage sections, there is a difference in driving conditions. In the driving of the unit storage period, the time for executing the storage stage is set relatively short, and the time for executing the discharge stage is set relatively long in many cases. Therefore, the timing at which the first charge storage section switches from the state of discharging charges via the charge discharge section during the discharge stage of the previous unit storage period to the state of not discharging charges is the timing to start storing charges. On the other hand, other charge storage sections start storing charges at the timing when the short-time charge storage in the previous charge storage section is completed. It is considered that this difference in driving conditions is one of the important factors in the deterioration of distance accuracy. Summary of the Invention

[0007] The present invention has been completed based on the above problems, and an object thereof is to provide a distance image capturing device and a distance image capturing method that can drive a pixel in such a way that the driving conditions of the first charge storage section that first stores charges and other charge storage sections that store charges after the second one among a plurality of charge storage sections are close to each other.

[0008] The distance image capturing device of the present invention includes: a light source unit that irradiates a light pulse onto a subject; a light receiving unit that includes a distance image sensor and a pixel driving circuit. In the distance image sensor, a plurality of pixels are arranged in a two-dimensional matrix. Each pixel includes a photoelectric conversion element that generates charge according to incident light, a charge discharging unit that discharges the charge generated by the photoelectric conversion element, and a plurality of charge storage units that store the charge generated by the photoelectric conversion element. The pixel driving circuit distributes and stores charge in the charge storage units at a storage timing synchronized with an irradiation timing at which the light pulse is irradiated in a frame period; and a distance image processing unit that calculates the distance to the subject based on the amounts of charge stored in the charge storage units respectively. In the frame period, the number of driving execution storage times in a unit storage period is set. In the unit storage period, a storage stage in which charge is sequentially stored in the charge storage units at the storage timing and a discharging stage in which the charge is discharged via the charge discharging unit are executed. An adjustment stage is executed at a timing when switching from the discharging stage of the previous executed unit storage period to the current executed unit storage period. In the adjustment stage, a first drive in which the charge discharging unit does not discharge charge and the charge storage unit does not store charge is sequentially executed, and a second drive in which the charge is discharged via the charge discharging unit is executed. The time for executing the second drive is the same as the storage drive time for executing storage drive for storing charge in one of the charge storage units in the storage stage.

[0009] The distance image capturing method of the present invention is performed by a distance image capturing device, which includes: a light source unit that irradiates a light pulse onto a subject; a light receiving unit having a distance image sensor and a pixel driving circuit. In the distance image sensor, a plurality of pixels are arranged in a two-dimensional matrix. Each pixel includes a photoelectric conversion element that generates charges according to incident light, a charge discharging unit that discharges the charges generated by the photoelectric conversion element, and a plurality of charge storage units that store the charges generated by the photoelectric conversion element. The pixel driving circuit distributes and stores charges in the charge storage units respectively at a storage timing synchronized with an irradiation timing of irradiating the light pulse according to a frame period. And a distance image processing unit that calculates the distance to the subject based on the amounts of charges stored in the charge storage units respectively. In the distance image capturing method, during the frame period, the number of driving execution storage times in a unit storage period is set. In the unit storage period, a storage stage of sequentially storing charges in the charge storage units at the storage timing and a discharging stage of discharging charges via the charge discharging unit are executed. An adjustment stage is executed at a timing of switching from the discharging stage of the previous executed unit storage period to the current executed unit storage period. In the adjustment stage, a first drive of not discharging charges by the charge discharging unit and not storing charges in the charge storage unit and a second drive of discharging charges via the charge discharging unit are sequentially executed. The time for executing the second drive is the same as the storage drive time for executing a storage drive of storing charges in one of the charge storage units in the storage stage.

[0010] According to the present invention, it is possible to drive pixels in such a manner that the driving conditions of a first charge storage unit that stores charges first among a plurality of charge storage units are close to those of other charge storage units that store charges after the second one. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 It is a block diagram showing the configuration of the distance image capturing device 1 of the embodiment.

[0012] Figure 2 It is a block diagram showing the configuration of the distance image sensor 32 of the embodiment.

[0013] Figure 3 It is a circuit diagram showing an example of the configuration of a pixel 321 of the embodiment.

[0014] Figure 4A It is a diagram schematically showing an example of the layout pattern of the pixel 321 of the embodiment.

[0015] Figure 4B It is a timing chart showing the timing of driving the pixel 321 in the prior art.

[0016] Figure 4C It is shown thatFigure 4B Diagram of the states of the pixel signals Q1 to Q4 driven.

[0017] Figure 5 It is a timing chart showing a first example of the timing for driving the pixel 321 of the embodiment.

[0018] Figure 6 It is a timing chart showing a second example of the timing for driving the pixel 321 of the embodiment.

[0019] Figure 7 It is a timing chart showing a third example of the timing for driving the pixel 321 of the embodiment.

[0020] Figure 8A It is a diagram schematically showing an example of the layout pattern of the pixel 321 of the embodiment.

[0021] Figure 8B It is a diagram schematically showing an example of the layout pattern of the pixel 321 of the embodiment.

[0022] Figure 8C It is a diagram schematically showing an example of the layout pattern of the pixel 321 of the embodiment.

[0023] Figure 9 It is a timing chart showing a fourth example of the timing for driving the pixel 321 of the embodiment.

[0024] Figure 10 It is a timing chart showing a fifth example of the timing for driving the pixel 321 of the embodiment.

[0025] Figure 11 It is a timing chart showing a sixth example of the timing for driving the pixel 321 of the embodiment.

[0026] Figure 12 It is a timing chart showing a seventh example of the timing for driving the pixel 321 of the embodiment.

[0027] Figure 13 It is a timing chart showing an eighth example of the timing for driving the pixel 321 of the embodiment. Detailed Embodiment

[0028] Hereinafter, while referring to the attached Figure 1 the distance image capturing device of the embodiment will be described.

[0029] Figure 1 It is a block diagram showing a schematic configuration of the distance image capturing device of the embodiment. The distance image capturing device 1 includes, for example, a light source unit 2, a light receiving unit 3, and a distance image processing unit 4. Also shown in Figure 1 is the object OB to be photographed, which is the object for measuring the distance in the distance image capturing device 1.

[0030] The light source unit 2 irradiates the subject OB with a light pulse PO according to the control from the distance image processing unit 4. The light source unit 2 is, for example, a surface-emitting semiconductor laser module such as a vertical cavity surface emitting laser (VCSEL). The light source unit 2 includes a light source device 21 and a diffuser plate 22.

[0031] The light source device 21 is a light source that emits laser light in the near-infrared band (for example, a band with a wavelength of 850 nm to 940 nm) that becomes the light pulse PO irradiated to the subject OB. The light source device 21 is, for example, a semiconductor laser light-emitting element. The light source device 21 emits pulsed laser light according to the control from the timing control unit 41.

[0032] The diffuser plate 22 is an optical component that diffuses the near-infrared band laser light emitted by the light source device 21 to the area of the surface irradiated to the subject OB. The pulsed laser light diffused by the diffuser plate 22 is emitted as the light pulse PO and irradiated to the subject OB.

[0033] The light receiving unit 3 receives the reflected light RL of the light pulse PO reflected by the subject OB and outputs a pixel signal corresponding to the received reflected light RL. The light receiving unit 3 includes a lens 31 and a distance image sensor 32.

[0034] The lens 31 is an optical lens that guides the incident reflected light RL to the distance image sensor 32. The lens 31 emits the incident reflected light RL toward the distance image sensor 32 side, causing the pixels included in the light receiving area of the distance image sensor 32 to receive light (incidence).

[0035] The distance image sensor 32 is an imaging element. The distance image sensor 32 has a plurality of pixels arranged in a two-dimensional matrix. In each pixel of the distance image sensor 32, one photoelectric conversion element, a plurality of charge storage units corresponding to the one photoelectric conversion element, and a component that distributes charges to each charge storage unit are provided. That is, the pixel is an imaging element configured to distribute and store charges in a plurality of charge storage units.

[0036] The distance image sensor 32 distributes the charges generated by the photoelectric conversion element to each charge storage unit according to the control from the timing control unit 41. In addition, the distance image sensor 32 outputs a pixel signal corresponding to the amount of charge distributed to the charge storage unit. In the distance image sensor 32, a plurality of pixels are arranged in a two-dimensional matrix, and a pixel signal of one frame amount corresponding to each pixel is output.

[0037] Here, Figure 2 The configuration of the distance image sensor 32 will be described.Figure 2 This is a block diagram showing a schematic configuration of an imaging element (distance image sensor 32) used in the distance image capturing device 1 according to an embodiment.

[0038] As Figure 2 shown, the distance image sensor 32 includes, for example, a light receiving area 320 in which a plurality of pixels 321 are arranged in a two-dimensional matrix, and a pixel driving circuit 322. The pixel driving circuit 322 includes, for example, a vertical scanning circuit 323 having an assignment operation, a horizontal scanning circuit 324, a pixel signal processing circuit 325, and a control circuit 326.

[0039] The light receiving area 320 is an area where a plurality of pixels 321 are arranged in a two-dimensional matrix, and an example of a two-dimensional matrix arranged in 8 rows and 8 columns is shown in Figure 2 . Each pixel 321 accumulates charge corresponding to the amount of received light and outputs an accumulation signal corresponding to the accumulated charge amount.

[0040] The control circuit 326 comprehensively controls the distance image sensor 32. The control circuit 326 controls the operations of the components of the distance image sensor 32, for example, according to an instruction from the timing control unit 41 of the distance image processing unit 4. In addition, the control of the components included in the distance image sensor 32 may be configured such that it is directly performed by the timing control unit 41. In this case, the control circuit 326 can be omitted.

[0041] The vertical scanning circuit 323 controls the pixels 321 arranged in the light receiving area 320 row by row according to the control from the control circuit 326. The vertical scanning circuit 323 outputs a voltage signal corresponding to the amount of charge accumulated in the charge accumulation unit CS of each pixel 321 to the pixel signal processing circuit 325. For example, at an accumulation timing synchronized with the irradiation of the light pulse PO, the vertical scanning circuit 323 distributes and accumulates the charge converted by the photoelectric conversion element to the charge accumulation parts of the pixels 321. In addition, during a period different from the accumulation period in which the charge accumulation unit CS accumulates charge (for example, a readout period), the vertical scanning circuit 323 discharges the charge converted by the photoelectric conversion element from a charge discharge unit (a charge discharge transistor GD described later).

[0042] The pixel signal processing circuit 325 performs predetermined signal processing (such as noise suppression processing, A / D conversion processing, etc.) on the voltage signals output from the pixels 321 of each column to the corresponding vertical signal lines according to the control from the control circuit 326.

[0043] The horizontal scanning circuit 324, under the control from the control circuit 326, sequentially outputs the signals output from the pixel signal processing circuit 325 in time series. Thereby, the accumulation signals of one frame amount are sequentially output to the distance image processing unit 4. Hereinafter, it is assumed that the pixel signal processing circuit 325 performs A / D conversion processing and the accumulation signals are digital signals for explanation.

[0044] Here, Figure 3 the configuration of the pixel 321 will be described. Figure 3 FIG. is a circuit diagram showing an example of the pixel 321. Figure 3 FIG. shows an example of the configuration of one pixel 321 among the plurality of pixels 321 arranged in the light receiving region 320. In this figure, an example in which the pixel 321 includes four signal readout units RU (signal readout units RU1 to RU4) is shown.

[0045] The pixel 321 includes one photoelectric conversion element PD, a charge discharge transistor GD, and four signal readout units RU that output voltage signals from the corresponding output terminals O. Each signal readout unit RU includes a transfer transistor G, a floating diffusion region FD, a charge storage capacitor C, a reset transistor RT, a source follower transistor SF, and a selection transistor SL. The charge storage unit CS is composed of the floating diffusion region FD and the charge storage capacitor C.

[0046] In addition, in Figure 3 , each signal readout unit RU is distinguished by adding any one of the numbers "1" to "4" after the symbol "RU" of the four signal readout units RU. Similarly, each component included in the four signal readout units RU is also distinguished by showing the number indicating each signal readout unit RU after the symbol to indicate the signal readout unit RU corresponding to each component.

[0047] In addition, in Figure 3 , each signal readout unit RU is distinguished by adding any one of the numbers "1" to "4" after the symbol "RU" of the four signal readout units RU. Similarly, each component included in the four signal readout units RU is also distinguished by showing the number indicating each signal readout unit RU after the symbol to indicate the signal readout unit RU corresponding to each component.

[0048] In the pixel 321, the signal readout unit RU1 outputs a voltage signal from the output terminal O1. The signal readout unit RU1 includes a transfer transistor G1, a floating diffusion region FD1, a charge storage capacitor C1, a reset transistor RT1, a source follower transistor SF1, and a selection transistor SL1. The charge storage unit CS1 is composed of the floating diffusion region FD1 and the charge storage capacitor C1. The signal readout units RU2 to RU4 have the same configuration.

[0049] The photoelectric conversion element PD is a buried-type photodiode that performs photoelectric conversion on the incident light to generate charges corresponding to the intensity of the incident light and accumulates the generated charges. The structure of the photoelectric conversion element PD can be arbitrary. For example, the photoelectric conversion element PD can be a PN photodiode with a structure in which a P-type semiconductor is joined to an N-type semiconductor, or a PIN photodiode with a structure in which an I-type semiconductor is sandwiched between a P-type semiconductor and an N-type semiconductor. In addition, the photoelectric conversion element PD is not limited to a photodiode, and can be, for example, a photoelectric conversion element of a grating type.

[0050] The charge discharge transistor GD is a transistor used to discard the charges generated in the photoelectric conversion element PD. When the charge discharge transistor GD is controlled to be in an on state by the pixel drive circuit 322, the charges generated in the photoelectric conversion element PD are discarded (that is, the photoelectric conversion element PD is reset).

[0051] The pixel drive circuit 322 drives the pixel 321, distributes the charges generated by the photoelectric conversion element PD performing photoelectric conversion on the incident light to the four charge accumulation parts CS respectively, and outputs each voltage signal corresponding to the amount of charge of the distributed charges to the pixel signal processing circuit 325.

[0052] For example, in driving the pixel 321, the pixel drive circuit 322 synchronizes with the irradiation timing of the light pulse PO and sequentially controls the accumulation drive signals TX1 to TX4 corresponding to the charge accumulation parts CS1 to CS4 to be in an on state. Thereby, the transfer transistors G1 to G4 corresponding to the charge accumulation parts CS are sequentially turned on, and charges are distributed to the corresponding charge accumulation parts CS and accumulated therein. Thereby, charges are accumulated in the order of the charge accumulation parts CS1, CS2, CS3, CS4.

[0053] In addition, the pixel 321 is not limited to Figure 3 the configuration having four signal readout parts RU as shown. As long as it is a pixel having a configuration with a plurality of signal readout parts RU. That is, the number of signal readout parts RU (charge accumulation parts CS) provided in the pixels arranged in the image sensor 32 can be two, can be three, or can be five or more.

[0054] In addition, in Figure 3 an example in which the charge accumulation part CS is composed of a floating diffusion region FD and a charge accumulation capacitor C is shown. However, as long as the charge accumulation part CS is at least composed of the floating diffusion region FD, the pixel 321 can also be a configuration that does not have the charge accumulation capacitor C.

[0055] Return Figure 1As described above, the distance image processing unit 4 controls the distance image capturing device 1 to calculate the distance to the object OB. The distance image processing unit 4 includes a timing control unit 41, a distance calculation unit 42, and a measurement control unit 43.

[0056] The timing control unit 41 controls the timing of outputting various control signals required for measurement according to the control of the measurement control unit 43. The various control signals here are, for example, a signal for controlling whether to irradiate the light pulse PO, a signal for controlling whether to accumulate charges in the charge accumulation unit, a signal for setting the number of accumulation times per frame, etc. The number of accumulation times is the number of times the process of distributing charges to the charge accumulation unit CS and accumulating them is repeated, and corresponds to the number of distribution times preset in the frame cycle. The product of the number of accumulation times and the time (accumulation time) for accumulating charges in each charge accumulation unit for each process of distributing and accumulating charges becomes the exposure time.

[0057] The distance calculation unit 42 outputs distance information of the distance to the object OB calculated based on the pixel signal output from the distance image sensor 32. The distance calculation unit 42 calculates the delay time from the irradiation light pulse PO to the reception of the reflected light RL based on the amount of charge accumulated in the plurality of charge accumulation units CS. The distance calculation unit 42 calculates the distance to the object OB based on the calculated delay time.

[0058] The measurement control unit 43 controls the timing control unit 41. For example, the measurement control unit 431 sets the number of frame accumulations and the accumulation time width, and controls the timing control unit 41 to perform imaging according to the set contents. That is, the measurement control unit 43 sets the frame cycle, and controls the timing control unit 41 to perform imaging according to the set contents.

[0059] Here, using Figure 4 ( Figures 4A to 4C ) describes the subject matter of this embodiment.

[0060] Figure 4A Schematically shows the layout mode of the pixel 321. Figure 4A As shown, the pixel 321 is an integrated circuit having a transistor, a transfer transistor G (transfer transistors G1 to G4), and a charge discharge transistor GD (charge discharge transistors GD1 and GD2) mounted on a photoelectric conversion element PD. The transistor is, for example, an n-channel MOS transistor formed on a p-type semiconductor substrate, and is composed of a drain D (n diffusion layer (diffusion layer of n-type impurities)), a source (n diffusion layer), and a gate G, respectively.

[0061] In addition, Figure 4AIn the example, transistors other than the transfer transistors G and the charge discharge transistors GD, specifically, transistors such as the charge discharge transistors GD, source follower transistors SF1 to SF4, selection transistors SL1 to SL4, and reset transistors RT1 to RT4 are omitted from description.

[0062] The photoelectric conversion element PD is formed in a long hexagonal shape in which two opposite sides of a regular hexagon are longer than the other four sides.

[0063] The transfer transistors G1 and G3 are symmetrically arranged with respect to an axis passing through the center of the long hexagon of the photoelectric conversion element PD and orthogonal to the two longer sides of the long hexagon on one of the two longer sides of the long hexagon of the photoelectric conversion element PD. The transfer transistors G2 and G4 are symmetrically arranged with respect to an axis passing through the center of the long hexagon of the photoelectric conversion element PD and orthogonal to the two longer sides of the long hexagon on a side different from the side on which the transfer transistors G1 and G3 are arranged among the two longer sides of the long hexagon of the photoelectric conversion element PD. The transfer transistors G1 and G2 are symmetrically arranged with respect to an axis passing through the center of the long hexagon and parallel to the two longer sides of the long hexagon. The transfer transistors G3 and G4 are symmetrically arranged with respect to an axis passing through the center of the long hexagon and parallel to the two longer sides of the long hexagon.

[0064] The charge discharge transistors GD1 and GD2 are symmetrically arranged at the vertex positions where two adjacent sides among the four shorter sides of the long hexagon of the photoelectric conversion element PD are connected, with respect to an axis passing through the center of the long hexagon and orthogonal to the two longer sides of the long hexagon.

[0065] In the example of this figure, the gate G of the charge discharge transistor GD1 is connected to a control signal GD_CL that controls the discharge of charges. When the control signal GD_CL becomes High (1), charges are discharged, and when it becomes Low (0 (zero)), it becomes a state where charges are not discharged.

[0066] In addition, the gate G of the charge discharge transistor GD2 is fixed to a fixed value (0 (zero)). Therefore, in the charge discharge transistor GD2, charges are never discharged.

[0067] In addition, the gates G of the transfer transistors G1 to G4 are each connected to control signals G1_CL to G4_CL that control the charge storage in the charge storage units CS1 to CS4 that are the respective corresponding drains D. When the control signal Gk_CL becomes High (1), charges are stored, and when it becomes Low (0 (zero)), it becomes a state where charges are not stored. Here, k is one of 1 to 4.

[0068] Figure 4B It is a timing chart showing a conventional driving example of the pixel 321. The driving of the pixel is performed in a frame period, asFigure 4B As shown, an accumulation period and a readout period are set in one frame. The accumulation period is a period during which the charge storage section CS stores charges, and is a period during which the driving of the pixel 321 shown in the unit accumulation period is repeated a predetermined number of accumulation times.

[0069] The readout period is a period during which the pixel signal Q corresponding to the amount of charge stored in each of the charge storage sections CS is read out.

[0070] In Figure 4B shows a timing chart of elements corresponding to each item of "GD", "0 fixed", "G1" to "G4", and "LIGHT". "GD" represents the operation timing of the control signal GD_CL for controlling the charge discharge transistor GD1. "0 fixed" means that the charge discharge transistor GD2 is fixed to Low (fixed to 0 (zero)). "G1" to "G4" represent the operation timings of the control signals G1_CL to G4_CL for controlling the transfer transistors G1 to G4. "LIGHT" represents the irradiation timing of the light pulse PO. Specifically, it means that it becomes the irradiation state in the conduction state (the state where the timing signal is set to High (1)), and the light becomes the extinguished state in the cutoff state (the state where the timing signal is set to Low (0 (zero))).

[0071] As Figure 4B shown, the unit accumulation period includes an accumulation stage A and a discharge stage B.

[0072] In the accumulation stage A, first, the charge discharge transistor GD is controlled to be in the cutoff state, and then the transfer transistor G1 is controlled to be in the conduction state. When the accumulation time (for example, the accumulation time To set corresponding to the irradiation time of the light pulse PO) has elapsed since the charge discharge transistor GD was controlled to be in the cutoff state, the transfer transistor G1 is controlled to be in the cutoff state.

[0073] Here, the period during which the transfer transistor G1 is controlled to be in the on state is the storage drive time Tc. The storage drive time Tc is set to be less than the storage time To. During the non-storage drive period (= To - Tc) in which the transfer transistor G1 is controlled to be in the off state during the storage time To, the charge converted by the photoelectric conversion element PD is stored in the photoelectric conversion element PD (not the charge storage section CS1). By controlling the transfer transistor G1 to be in the on state, the charge stored in the photoelectric conversion element PD during its non-storage drive period moves from the photoelectric conversion element PD to the floating diffusion region FD1 and is stored in the charge storage section CS1. In addition, the charge converted by the photoelectric conversion element PD during the storage drive time Tc in which the transfer transistor G1 is controlled to be in the on state is stored in the charge storage section CS1. That is, the transfer transistor G1 is controlled to be in the on state during the storage drive time Tc, and the charge converted by the photoelectric conversion element PD is stored in the charge storage section CS1 during the storage time To.

[0074] At the timing when the transfer transistor G1 is controlled to be in the off state, a light pulse PO is irradiated during the irradiation time To. In addition, the transfer transistor G1 is controlled to be in the off state, and then the transfer transistor G2 is controlled to be in the on state. When the storage time To has elapsed since the transfer transistor G1 was controlled to be in the off state, the transfer transistor G2 is controlled to be in the off state. The period during which the transfer transistor G2 is controlled to be in the on state is the storage drive time Tc.

[0075] The transfer transistor G2 is controlled to be in the off state, and then the transfer transistor G3 is controlled to be in the on state. When the storage time To has elapsed since the transfer transistor G2 was controlled to be in the off state, the transfer transistor G3 is controlled to be in the off state. The period during which the transfer transistor G3 is controlled to be in the on state is the storage drive time Tc.

[0076] The transfer transistor G3 is controlled to be in the off state, and then the transfer transistor G4 is controlled to be in the on state. When the storage time To has elapsed since the transfer transistor G3 was controlled to be in the off state, the transfer transistor G4 is controlled to be in the off state. The period during which the transfer transistor G4 is controlled to be in the on state is the storage drive time Tc. After that, the charge discharge transistor GD is controlled to be in the on state.

[0077] In the discharge stage B, during the period before the start of the next unit storage period, the charge discharge transistor GD is maintained in the on state, and the transfer transistors G1 to G4 are maintained in the off state.

[0078] Figure 4C It is a schematic diagram showing the magnitudes of the pixel signals Q1 to Q4 corresponding to the amounts of charge stored in the charge storage sections CS1 to CS4 respectively in the case of performing the Figure 4B shown driving.Figure 4C Examples of patterns P1 and P2 are shown. In pattern P1, the signal value of pixel signal Q1 has an excess of GD (charge discharge) compared to other pixel signals Q2 to Q4, and pixel signal Q1 shows a value smaller than other pixel signals. In pattern P2, the signal value of pixel signal Q1 has a shortage of GD (charge discharge) compared to other pixel signals Q2 to Q4, and pixel signal Q1 shows a value larger than other pixel signals. Additionally, it is premised that the amount of light incident on pixel 321 does not change (is constant) at the timing when charge storage units CS1 to CS4 store charge respectively.

[0079] In this way, there is a tendency for a difference to occur in the amount of charge stored in pixel signal Q1 and other pixel signals Q2 to Q4. One of the important factors is the difference in driving conditions. As Figure 4B shown, during the driving in the unit storage period, in many cases, pixel 321 is driven such that storage phase A is implemented for a relatively short time and discharge phase B is implemented for a relatively long time. Therefore, for charge storage unit CS1, charge storage starts at the timing when the state in which charge was discharged during discharge phase B of the previous unit storage period has continued for a long time and then switches to a state where charge is not discharged. On the other hand, for other charge storage units, charge storage starts at the timing when short-time charge storage in the previous charge storage unit CS is completed. Specifically, for charge storage unit CS2, charge storage starts at the timing when short-time charge storage in the previous charge storage unit CS1 is completed. For charge storage unit CS3, charge storage starts at the timing when short-time charge storage in the previous charge storage unit CS2 is completed. For charge storage unit CS4, charge storage starts at the timing when short-time charge storage in the previous charge storage unit CS3 is completed. This difference in driving conditions becomes one of the important factors for the deterioration of distance accuracy.

[0080] As a countermeasure, in the present embodiment, pixel 321 is driven such that the driving conditions of charge storage unit CS1, which stores charge first among the multiple charge storage units CS1 to CS4, are close to those of other charge storage units CS2 to CS4 that store charge after the second one.

[0081] Hereinafter, Figures 5 to 13 a method for driving pixel 321 of the present embodiment will be described. Figures 5 to 7 is a timing chart showing a driving example of pixel 321 of the embodiment.

[0082] FIG. 8( Figures 8A to 8C ) is a diagram showing a layout example of pixel 321 of the embodiment. Figures 9 to 13 is a timing chart showing a driving example of pixel 321 of the embodiment.

[0083] Figures 5 to 7Each item of "GD", "0 fixed", "G1" to "G4", and "LIGHT" is the same as Figure 4B and thus the description thereof is omitted.

[0084] Figure 5 Shows a driving example (first example) of the pixel 321 of the present embodiment. As Figure 5 shown, in the present embodiment, at the timing TG1 of the distance image capturing device 1, that is, the timing switched from the discharge phase B of the previous unit accumulation period to the current unit accumulation period, the adjustment phase X is executed. Figure 4B

[0085] The distance image capturing device 1 sequentially executes the first drive KN1 and the second drive KN2 in the adjustment phase X. In the first drive KN1, the charge discharge unit (charge discharge transistor GD) does not discharge charge and the charge storage unit CS does not store charge. In the second drive KN2, charge is discharged via the charge discharge unit (charge discharge transistor GD).

[0086] Here, the execution time Tg of the second drive KN2 is the same as the accumulation drive time Tc of the accumulation drive for storing charge in one charge storage unit CS in the accumulation phase, that is, Tg = Tc.

[0087] Thus, in the present embodiment, the adjustment phase X is provided in the driving of the unit accumulation period of the pixel 321. Thereby, immediately before storing charge in the charge storage unit CS1, the charge discharge unit (charge discharge transistor GD) is pulse-driven, as if the storage of charge in the charge storage unit CS1 starts at the timing when the short-time charge storage in the previous charge storage unit is completed. Therefore, the driving conditions of the charge storage unit CS1 can be made close to those of the other charge storage units CS2 to CS4 that store charge after the second one, and the deterioration of the distance accuracy can be suppressed.

[0088] Figure 6 Shows a driving example (second example) of the pixel 321 of the present embodiment. As Figure 6 shown, the distance image capturing device 1 may also execute the first drive KN1 and the second drive KN2 alternately multiple times in the adjustment phase X.

[0089] Figure 7 Shows a driving example (third example) of the pixel 321 of the present embodiment. As Figure 7 shown, the distance image capturing device 1 may be such that, instead of continuously maintaining the charge discharge transistor GD in the on state in the discharge phase B, the adjustment phase X is driven, that is, the first drive KN1 and the second drive KN2 are executed alternately.

[0090] Figure 8A Schematically represents makingFigure 4A Layout of pixel 321 in which charge discharge transistor GD2 functions as a charge discharge unit.

[0091] In the example of this figure, the gate G of charge discharge transistor GD1 is connected to control signal GD1_CL that controls charge discharge. When control signal GD1_CL becomes High (1), charge is discharged, and when it becomes Low (0 (zero)), it is in a state where charge is not discharged.

[0092] In addition, the gate G of charge discharge transistor GD2 is connected to control signal GD2_CL that controls charge discharge. When control signal GD2_CL becomes High (1), charge is discharged, and when it becomes Low (0 (zero)), it is in a state where charge is not discharged.

[0093] By providing two charge discharge units (charge discharge transistors GD1 and GD2) in pixel 321, they can be used according to different purposes. That is, they are used as a charge discharge unit (charge discharge transistor GD1) for discharging charge over a long period and a charge discharge unit (charge discharge transistor GD2) for making the driving conditions of charge storage unit CS1 consistent with those of other charge storage units CS2 - CS4. Thus, it is easy to make the driving conditions of charge storage unit CS consistent while maintaining the charge discharge function.

[0094] Figure 8B Schematically shows the layout in pixel 321 where buffer configurations of control signals G1_CL - G4_CL and GD2_CL indicated by symbol GP are made consistent. By making the buffer configurations consistent, for example, using the same power supply for the buffer power supply, the waveforms (rise waveform and fall waveform) of the control signal of charge discharge transistor GD2 can be made the same as those of the control signals of transfer transistors G1 - G4. Thus, the electrical characteristics of the control waveform of the charge discharge unit (charge discharge transistor GD2) used for making the driving conditions of charge storage unit CS consistent can be made close to those of the control waveform of charge storage unit CS. Figure 8A

[0095] Figure 8CSchematically shows a layout in which the photoelectric conversion element PD is formed in a regular hexagon shape. In this way, the buffers of the control signals G1_CL to G4_CL and GD2_CL shown by the symbol GP are made consistent, and the six gates G (the gates G of the two charge discharge transistors GD and the four transfer transistors G1 to G4) included in the pixel 321 are symmetrically arranged in a point-symmetrical manner with respect to the center of the regular hexagon. Thereby, the load applied to each gate can be made the same, and the electrical characteristics of each control waveform can be made close to each other.

[0096] Figures 9 to 10 Shows a driving example using two charge discharge units (charge discharge transistors GD1, GD2) as shown in FIG. 8.

[0097] Figure 9 Shows a driving example (fourth example) of the pixel 321 of the present embodiment. At a distance from the image pickup device 1, in the adjustment stage X, the charge discharge unit (charge discharge transistor GD1) used for charge discharge discharges the charge, and the charge discharge unit (charge discharge transistor GD2) used for making the driving conditions consistent sequentially executes the first drive KN1 and the second drive KN2.

[0098] Figure 10 Shows a driving example (fifth example) of the pixel 321 of the present embodiment. The distance image pickup device 1 switches to the timing of the current unit accumulation period from the discharge stage B of the previous unit accumulation period and executes the adjustment stage X, and also executes the adjustment stage X at the timing when the accumulation stage A ends.

[0099] Figures 11 to 13 Shows a driving example in which one frame is composed of a plurality of sub-frames. Here, a case where two sub-frames, that is, a first sub-frame and a second sub-frame are provided in one frame is illustrated for explanation. In addition, Figures 11 to 13 Shows the same driving as Figure 4A using one charge discharge unit.

[0100] In Figures 11 to 13A timing chart showing elements corresponding to each item of "GD.1", "G1.1" to "G4.1", "GD.2", "G1.2" to "G4.2", and "LIGHT" is shown. "GD.1" represents the operation timing of the control signal GD_CL for controlling the charge discharge transistor GD1 of the first sub-frame. "G1.1" to "G4.1" represent the operation timing of the control signals G1_CL to G4_CL for controlling the transfer transistors G1 to G4 of the first sub-frame. "GD.2" represents the operation timing of the control signal GD_CL for controlling the charge discharge transistor GD1 of the second sub-frame. "G1.2" to "G4.2" represent the operation timing of the control signals G1_CL to G4_CL for controlling the transfer transistors G1 to G4 of the second sub-frame. "LIGHT" represents the irradiation timing of the optical pulse PO. The irradiation timing of the optical pulse PO is common to the first sub-frame and the second sub-frame.

[0101] In Figures 11 to 13 the distance image pickup device 1 first repeats the driving of the first unit accumulation period a specified number of times (the first accumulation number) as the driving of the first sub-frame, obtains the pixel signals Q1 to Q4 as the driving result of the first sub-frame, and stores them in the memory. Next, the driving of the second unit accumulation period is repeated a specified number of times (the second accumulation number) as the driving of the second sub-frame, obtains the pixel signals Q1 to Q4 as the driving result of the second sub-frame, and stores them in the memory. Then, the distance image pickup device 1 calculates the distance using the pixel signals Q1 to Q4, which are the driving results of the first sub-frame stored in the memory, and the pixel signals Q1 to Q4, which are the driving results of the second sub-frame.

[0102] Figure 11 An example of driving the pixel 321 of the present embodiment (the sixth example) is shown. In Figure 11 the distance image pickup device 1 sequentially executes the first accumulation stage A1, the first discharge stage B1, the second accumulation stage A2, and the second discharge stage B2 after executing the adjustment stage X during the first unit accumulation period. In addition, the distance image pickup device 1 does not provide a charge accumulation section dedicated to external light during the second unit accumulation period, but executes the accumulation stage A at the same timing as the timing of the first discharge stage B1 during the first unit accumulation period, and then executes the discharge stage B.

[0103] During the first unit accumulation period, the charge accumulation section CS1 is used as a charge accumulation section dedicated to external light that accumulates only the external light component. The first discharge stage B1 is provided during the first unit accumulation period so that the charge accumulation timings of the charge accumulation sections CS2 to CS4 relative to the irradiation timing of the optical pulse PO respectively accumulate charges Figure 5The driving delay shown. By delaying the accumulation timing, a component of the reflected light RL that arrives after being reflected from a subject OB at a relatively long distance is accumulated in one of the charge accumulation units CS2 to CS4.

[0104] During the second unit accumulation period, the charge accumulation unit CS1 is used as a charge accumulation unit for accumulating a component of the reflected light RL that arrives after being reflected from a subject OB at a short distance. For each of the charge accumulation units CS2 to CS4, the component of the reflected light RL that arrives after being reflected from a subject OB at a relatively long distance is accumulated, the same as in the first unit accumulation period.

[0105] Generally, the amount of the reflected light RL that arrives after being reflected from a subject OB at a short distance is large, and the amount of the reflected light RL that arrives after being reflected from a subject OB at a long distance is small. By performing Figure 11 the driving shown, the component of the reflected light RL that arrives after being reflected from a subject OB at a short distance can be accumulated fewer times, and the component of the reflected light RL that arrives after being reflected from a subject OB at a long distance can be accumulated more times. By reducing the number of times of accumulating the component of the reflected light RL that arrives after being reflected from a subject OB at a short distance, it is possible to suppress the saturation of the amount of charge accumulated in the charge accumulation unit CS and prevent the situation where the distance cannot be calculated with high precision. By increasing the number of times of accumulating the component of the reflected light RL that arrives after being reflected from a subject OB at a long distance, the charge accumulation unit CS can accumulate an amount of charge that allows for high-precision distance calculation.

[0106] Specifically, the distance image capturing device 1 sequentially executes the first drive KN1 and the second drive KN2 in the adjustment stage X during the first unit accumulation period. In the first accumulation stage A1, by executing an accumulation drive that causes the charge accumulation unit CS1 to accumulate charge during the accumulation drive time Tc, the charge accumulation unit CS accumulates charge of the external light component. The distance image capturing device 1 discharges the charge via the charge accumulation unit (charge discharge transistor GD) in the first discharge stage B1. The distance image capturing device 1 executes an accumulation drive that causes the charge accumulation units CS2 to CS4 to sequentially accumulate charge during the accumulation drive time Tc, respectively, in the second accumulation stage A2. The distance image capturing device 1 discharges the charge via the charge accumulation unit (charge discharge transistor GD) in the second discharge stage B2.

[0107] In addition, the distance image capturing device 1 sequentially executes the first drive KN1 and the second drive KN2 in the adjustment stage X during the second unit accumulation period. The distance image capturing device 1 executes an accumulation drive that causes the charge accumulation units CS1 to CS4 to sequentially accumulate charge during the accumulation drive time Tc, respectively, in the accumulation stage A. The distance image capturing device 1 discharges the charge via the charge accumulation unit (charge discharge transistor GD) in the discharge stage B.

[0108] Thus, in each of the plurality of sub-frames of one frame for the distance image capturing device 1, at the timing of switching from the discharge phase B of the previous unit accumulation period to the current unit accumulation period, the adjustment phase X is executed. Accordingly, by providing the adjustment phase X before the charge storage unit CS1 stores charge in the first charge storage phase A1 of the first unit accumulation period and before the charge storage unit CS1 stores charge in the charge storage phase A of the second unit accumulation period, respectively, the driving conditions can be made closer to those of the other charge storage units CS2 to CS4.

[0109] Figure 12 Shows a driving example (seventh example) of the pixel 321 of the present embodiment. Figure 12 The first charge storage phase A1, the first discharge phase B1, the second charge storage phase A2, the second discharge phase B2 in the shown first unit accumulation period, the charge storage phase A and the discharge phase B in the second unit accumulation period are respectively the same as Figure 11 the same.

[0110] As Figure 12 shown, the distance image capturing device 1 may also start executing the adjustment phase X of the second unit accumulation period at the same timing as the adjustment phase X of the first unit accumulation period, and execute the adjustment phase X of the second unit accumulation period for a longer time than the adjustment phase X of the first unit accumulation period. In this case, the distance image capturing device 1 alternately executes the first drive KN1 and the second drive KN2 a plurality of times in the adjustment phase X of the second unit accumulation period.

[0111] Specifically, in the example of Figure 12 , the adjustment phase X of the first unit accumulation period starts at the timing TG1. In addition, the first charge storage phase A1 in the first unit accumulation period starts at the timing TG2. The charge storage phase A in the second unit accumulation period starts at the timing TG3. The timing TG2 arrives earlier than the timing TG3.

[0112] When the elapsed time from the timing TG1 to the timing TG3 (i.e., the time for executing the adjustment phase X in the second unit accumulation period) is longer than the time for executing (once each) the first drive KN1 and the second drive KN2, the distance image capturing device 1 may also alternately execute the first drive KN1 and the second drive KN2 a plurality of times in the adjustment phase X of the second unit accumulation period.

[0113] Figure 13 Shows a driving example (eighth example) of the pixel 321 of the present embodiment. In the driving example of Figure 13 , from being in a state where it is higher than Figure 11 and Figure 12The component of the reflected light RL that is reflected by the subject OB driven over a longer distance is accumulated in the pixel 321. Specifically, during the first unit accumulation period, a range drift period sft is set during the period from the irradiation timing to the accumulation timing at which the charge accumulation unit CS2 accumulates charge. During the second unit accumulation period, a linked period tm is set from the irradiation timing to the accumulation timing at which the charge accumulation unit CS1 accumulates charge, in linkage with the drive in the first unit accumulation period.

[0114] In the case where such a range drift period sft is set, the distance image pickup device 1 can also alternately execute the first drive KN1 and the second drive KN2 multiple times during the range drift period sft.

[0115] Specifically, in Figure 13 the example of, the charge accumulation in the charge accumulation unit CS1 during the first unit accumulation period ends at the timing TG4, and the charge accumulation in the next charge accumulation unit CS2 starts from the timing TG5.

[0116] In the case where the elapsed time from the timing TG4 to the timing TG5 (i.e., the time for executing the adjustment stage X2 during the first unit accumulation period) is longer than the time for executing (once each) the first drive KN1 and the second drive KN2, the distance image pickup device 1 can also alternately execute the first drive KN1 and the second drive KN2 multiple times in the adjustment stage X during the second unit accumulation period.

[0117] In addition, the distance image pickup device 1 can also be such that, in the drive of the frame period without setting a sub-frame, in the case where a range drift period sft is set, the adjustment stage X is executed during the range drift period. For example, the distance image pickup device 1 sets a range drift period sft during the period from the timing TG4 to TG5 in the accumulation stage during the unit accumulation period in the frame period. In this case, in the accumulation stage during the unit accumulation period of the distance image pickup device 1, the charge accumulation in the charge accumulation unit CS1 ends at the timing TG4, and the charge accumulation in the next charge accumulation unit CS2 starts at the timing TG5. The elapsed time from the timing TG4 to the timing TG5 is longer than the time for executing the first drive KN1 and the second drive KN2 once each, so the distance image pickup device 1 alternately executes the first drive KN1 and the second drive KN2 multiple times during the elapsed time.

[0118] As described above, the distance image capturing device 1 of the embodiment includes a light source unit 2, a light receiving unit 3, and a distance image processing unit 4. The light source unit 2 irradiates a light pulse PO to the subject OB. The light receiving unit 3 includes a distance image sensor 32 (pixel circuit) in which a plurality of pixels 321 are arranged in a two-dimensional matrix and a pixel driving circuit 322. Each pixel 321 includes a photoelectric conversion element PD and three or more charge storage units CS. The photoelectric conversion element PD generates charges corresponding to the incident light. The charge storage unit CS stores charges. The pixel driving circuit 322 distributes and stores charges in the charge storage units CS at a storage timing synchronized with the irradiation timing of the light pulse PO according to the frame period. The distance image processing unit 4 calculates the distance to the subject OB based on the amounts of charges stored in the charge storage units CS respectively. The distance image capturing device 1 executes the storage execution times per unit storage period within the frame period. The distance image capturing device 1 executes a storage stage A and a discharge stage B within the unit storage period, and executes an adjustment stage X at the timing of switching from the discharge stage B of the previous executed unit storage period to the current executed unit storage period. In the storage stage A, charges are sequentially stored in the charge storage units CS at a storage timing synchronized with the irradiation timing. In the discharge stage B, charges are discharged via a charge discharge unit. The distance image capturing device 1 sequentially executes a first drive KN1 and a second drive KN2 in the adjustment stage X. In the first drive KN1, the charge discharge unit does not discharge charges and the charge storage units do not store charges. In the second drive KN2, charges are discharged via the charge discharge unit. The time Tg for executing the second drive KN2 is the same as the storage drive time Tc for executing the storage drive for storing charges in one charge storage unit CS in the storage stage A.

[0119] Accordingly, in the distance image capturing device 1 of the embodiment, the pixels can be driven in such a manner that the difference in driving conditions between the first charge storage unit that stores charges first among the plurality of charge storage units and the other charge storage units that store charges after the second one is alleviated.

[0120] In addition, in the distance image capturing device 1 of the embodiment, it may also be that, as Figure 6 shown, the first drive KN1 and the second drive KN2 are executed multiple times in the adjustment stage X. Accordingly, an effect similar to the above-described effect can be achieved.

[0121] In addition, in the distance image capturing device 1 of the embodiment, it may also be that, as Figure 7 shown, in the discharge stage B, the first drive KN1 and the second drive KN2 in the adjustment stage X are alternately executed. Accordingly, in the distance image capturing device 1 of the embodiment, an effect similar to the above-described effect can be achieved.

[0122] In addition, in the distance image capturing device 1 of the embodiment, the charge discharging unit includes two components: a first charge discharging unit (charge discharging transistor GD1) and a second charge discharging unit (charge discharging transistor GD2). The distance image capturing device 1 may also be, as Figures 9 to 10 shown, in the adjustment stage X, the first charge discharging unit (charge discharging transistor GD1) discharges charges, and the second charge discharging unit (charge discharging transistor GD2) sequentially executes the first drive KN1 and the second drive KN2. Thus, it is easy to make the driving conditions of the charge storage unit CS consistent while maintaining the function of discharging charges.

[0123] In addition, in the distance image capturing device 1 of the embodiment, it may also be, as Figure 10 shown, after the adjustment stage X is executed at the timing when the accumulation stage A ends, the discharge stage B is executed. Thus, it is possible to make all the driving conditions of the charge storage units CS1 to CS4 close to each other.

[0124] In addition, in the distance image capturing device 1 of the embodiment, it may also be, as Figures 11 to 13 shown, a plurality of sub-frames are set in the frame period, and in each of the plurality of sub-frames, the pixels 321 are driven in such a way that the accumulation timing is different from the irradiation timing with respect to each other. In each of the plurality of sub-frames of the distance image capturing device 1, the adjustment stage X is executed at the timing when switching from the discharge stage B of the previous execution unit accumulation period to the current execution unit accumulation period. Thus, in the distance image capturing device 1 of the embodiment, in the driving of the sub-frames, it is possible to make the driving conditions of the charge storage unit CS1 close to the driving conditions of the other charge storage units CS2 to CS4.

[0125] In addition, in the distance image capturing device 1 of the embodiment, it may also be, as Figures 11 to 13 shown, when two sub-frames are set in the frame, in the first sub-frame and the second sub-frame, the adjustment stage X is executed at the same timing TG1 (first timing) with respect to the irradiation timing of the optical pulse PO. The distance image capturing device 1 executes the adjustment stage X at the timing TG1 in the first sub-frame, executes the first accumulation stage A1 at the timing TG2 (second timing), and executes the accumulation stage at the timing TG3 (third timing) in the second sub-frame. The timing TG2 arrives earlier than the timing TG3.

[0126] In the second unit accumulation period of the second sub-frame of the distance image capturing device 1, when the adjustment stage X starts to be executed at the timing TG1 and the elapsed time from the timing TG2 to the timing TG3 is greater than the time for executing (once each) the first drive KN1 and the second drive TN2, the first drive KN1 and the second drive KN2 are alternately executed during the elapsed time.

[0127] Accordingly, in the distance image capturing device 1 of the embodiment, in the driving of the frame period including sub-frames, the driving conditions of the charge storage unit CS1 can be made close to those of the other charge storage units CS2 to CS4.

[0128] In addition, in the distance image capturing device 1 of the embodiment, it may also be the case that, as Figure 13 shown, when the range drift period sft is set, the adjustment period X is executed instead of the discharge period B during the range drift period sft. The distance image capturing device 1 ends the charge storage in the charge storage unit CS1 at the timing TG4 (fourth timing) during the storage period A, and starts the charge storage in the charge storage unit CS2 that subsequently stores charge after the charge storage unit CS1 at the timing TG5 (fifth timing). The elapsed time from the timing TG4 to the timing TG5 is longer than the respective times for executing the first drive KN1 and the second drive KN2. The distance image capturing device 1 alternately executes the first drive KN1 and the second drive KN2 during the elapsed time. Accordingly, in the distance image capturing device 1 of the embodiment, in the driving including the range drift period, the driving conditions of each of the charge storage units CS1 to CS4 can be made close.

[0129] In the distance image capturing device 1 in the above embodiment, all or part of the distance image processing unit 4 may be implemented by a computer. In this case, a program for implementing this function can be recorded on a computer-readable recording medium, and is implemented by reading the program recorded on the recording medium into a computer system and executing it. In addition, the "computer system" mentioned here includes hardware such as an OS and peripheral devices. In addition, the "computer-readable recording medium" refers to a removable medium such as a floppy disk, an optical disk, a ROM, a CD-ROM, or a storage device such as a hard disk built into the computer system. Also, the "computer-readable recording medium" may include a medium that dynamically holds a program for a short period of time, such as a communication line when a program is transmitted via a network such as the Internet or a communication line such as a telephone line, and a medium that holds a program for a certain time, such as a volatile memory inside a computer system that is a server or a client in this case. Furthermore, the above program can be used to implement a part of the above function, can be further combined with a program already recorded in the computer system to implement the above function, or can be implemented using a programmable logic device such as an FPGA.

[0130] As described above, the preferred embodiments of the present invention have been described and explained, but these are examples of the present invention, and it should be understood that they should not be considered in a limiting manner. Additions, omissions, substitutions, and other changes can be made without departing from the scope of the present invention. Therefore, the present invention should not be considered limited by the above description, but by the claims.

Claims

1. A distance image camera device, comprising: A light source unit irradiates light pulses to the subject; a light receiving unit having a distance image sensor and a pixel driving circuit, wherein the distance image sensor has a plurality of pixels arranged in a two-dimensional matrix, the pixels having a photoelectric conversion element that generates charges according to incident light, a charge discharge unit that discharges the charges generated by the photoelectric conversion element, and a plurality of charge accumulation units that accumulate the charges generated by the photoelectric conversion element, the pixel driving circuit distributing the charges to the charge accumulation units at accumulation timings synchronized with irradiation timings of irradiating the light pulses according to a frame period, and causing the charges to be accumulated; and a distance image processing unit that calculates the distance to the subject based on the amount of charge accumulated by each of the charge accumulation units, The driving execution accumulation number of the unit accumulation period is performed in the frame cycle, An accumulation phase of sequentially accumulating charges in the charge accumulation section at the accumulation timing and a discharge phase of discharging charges via the charge discharge section are performed during the unit accumulation period, and an adjustment phase is performed at a timing switching from the discharge phase of the unit accumulation period executed last time to the unit accumulation period executed this time. In the adjustment phase, a first drive in which the charge discharge unit is not caused to discharge charge and the charge accumulation unit is not caused to accumulate charge, and a second drive in which the charge discharge unit discharges charge, are sequentially performed. The time for executing the second drive is the same as the accumulation drive time for executing the accumulation drive for causing one of the charge accumulation units to accumulate charge in the accumulation phase.

2. The distance image camera device according to claim 1, wherein: The first driving and the second driving are performed multiple times in the adjustment phase.

3. The distance image camera device according to claim 1, wherein: The first driving and the second driving in the adjustment phase are performed alternately in the discharge phase.

4. The distance image camera device according to claim 1, wherein: The charge discharge section is composed of two parts: a first charge discharge section and a second charge discharge section. In the adjustment phase, the first charge discharge section discharges charges, and the second charge discharge section performs the first driving and the second driving in sequence.

5. The distance image camera device according to claim 4, wherein: The discharge phase is executed at the timing when the accumulation phase ends and after the adjustment phase is executed.

6. The distance image camera device according to claim 1, wherein: A plurality of subframes are arranged in the frame period. In each of the plurality of subframes, the accumulation timing relative to the irradiation timing is different from each other, In each of the plurality of subframes, the adjustment phase is executed at a timing of switching from the discharge phase of the unit accumulation period executed last time to the unit accumulation period executed this time.

7. The distance image camera device according to claim 1, wherein: Two subframes, a first subframe and a second subframe, are provided in the frame period. In the first subframe, the adjustment phase is performed at a first timing when the discharge phase of the unit accumulation period performed last time is switched to the unit accumulation period performed this time, and the accumulation phase is performed at a second timing based on the irradiation timing. In the second subframe, the accumulation phase is performed at a third timing based on the irradiation timing, The second timing arrives earlier than the third timing, In the second subframe, the adjustment phase is performed starting at the first timing, The elapsed time from the second timing to the third timing is longer than the time for executing the first drive and the second drive, respectively. The first drive and the second drive are performed alternately during the elapsed time.

8. The distance image camera device according to claim 1, wherein: In the accumulation phase, charge accumulation in a first charge accumulation unit among the charge accumulation units ends at a fourth timing, and charge accumulation in a second charge accumulation unit subsequent to the first charge accumulation unit that accumulates charge starts at a fifth timing. The elapsed time from the fourth timing to the fifth timing is longer than the time for executing the first drive and the second drive, respectively. The first drive and the second drive are performed alternately during the elapsed time.

9. A distance image capturing method, performed by a distance image capturing device, The distance image camera device comprises: A light source unit irradiates light pulses to the subject; a light receiving unit having a distance image sensor and a pixel driving circuit, wherein the distance image sensor has a plurality of pixels arranged in a two-dimensional matrix, the pixels having a photoelectric conversion element that generates charges according to incident light, a charge discharge unit that discharges the charges generated by the photoelectric conversion element, and a plurality of charge accumulation units that accumulate the charges generated by the photoelectric conversion element, the pixel driving circuit distributing the charges to the charge accumulation units at accumulation timings synchronized with irradiation timings of irradiating the light pulses according to a frame period, and causing the charges to be accumulated; and a distance image processing unit that calculates the distance to the subject based on the amount of charge accumulated by each of the charge accumulation units, In the range image capturing method, The driving execution accumulation number of the unit accumulation period is performed in the frame cycle, An accumulation phase of sequentially accumulating charges in the charge accumulation section at the accumulation timing and a discharge phase of discharging charges via the charge discharge section are performed during the unit accumulation period, and an adjustment phase is performed at a timing switching from the discharge phase of the unit accumulation period executed last time to the unit accumulation period executed this time. In the adjustment phase, a first drive in which the charge discharge unit is not caused to discharge charge and the charge accumulation unit is not caused to accumulate charge, and a second drive in which the charge discharge unit discharges charge, are sequentially performed. The time for executing the second drive is the same as the accumulation drive time for executing the accumulation drive for causing one of the charge accumulation units to accumulate charge in the accumulation phase.

Images