Range image capturing device and range image capturing method

Abstract

To provide a distance image pickup device which maintains the sensitivity to the resolution of a photographed image and incident light without increasing the capacity of a charge storage part and suppresses saturation due to increase in the intensity of the incident light.SOLUTION: A distance image pickup device comprises: a light reception unit that includes a plurality of pixel circuits having a photoelectric conversion element which generates a charge according to incident light that is incident from a measurement space of a measurement object, N (N≥3) charge storage units which store the charges in a frame period and a transfer transistor which transfers the charges from the photoelectric conversion element to the charge storage units, and a pixel drive circuit which performs on / off processing of the transfer transistor to the charge storage units in a storage period synchronized with the irradiation of a light pulse and distributes the charges for storage; a measurement control unit which divides the charges generated by the photoelectric conversion element into each of the combinations of the charge storage units in any transfer period of charge storage cycles and distributes the charges for storage; and a distance calculation unit which obtains a distance to a subject in the measurement space as a measurement distance on the basis of the charge amount stored in the charge storage units.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a distance image capturing device and a distance image capturing method. [Background technology]

[0002] Conventionally, there have been time-of-flight (ToF) distance imaging devices that utilize the known speed of light to measure the distance to a subject based on the flight time of light (see, for example, Patent Document 1). A ToF distance imaging device includes a light source unit that emits light and an imaging unit that includes a pixel array in which a plurality of pixel circuits that detect light for measuring distance are arranged in a two-dimensional matrix (array). Each of the pixel circuits has a photoelectric conversion element (e.g., a photodiode) that generates a charge corresponding to the intensity of light. With this configuration, the ToF range image pickup device can acquire (capture) information about the distance between itself and a subject, and an image of the subject, in a measurement space (three-dimensional space).

[0003] A ToF range imaging device measures distance based on the delay time between when radiated light is emitted and when the light reflected by the subject is received. However, since the amount of charge generated by the photosensor changes depending on the intensity of the incident light, the intensity of the reflected light increases as the distance to the subject decreases (light intensity is inversely proportional to the square of the distance).

[0004] As described above, as the intensity of reflected light and the intensity of background light in the environment increase, the amount of charge generated by the photoelectric conversion element increases, and depending on the intensity of the reflected light and background light, the charge storage section that stores this amount of charge may become saturated. On the other hand, since a ToF range imaging device calculates the delay time based on the amount of charge accumulated in the charge accumulation unit, the measurement accuracy improves as the signal-to-noise ratio increases. Therefore, in order to prevent the charge storage section from being saturated, it is conceivable to increase the capacitance of the charge storage section. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2004-294420 Summary of the Invention [Problem to be solved by the invention]

[0006] However, when increasing the capacity of the charge storage unit, if the area of ​​the pixel circuit is increased and the capacity of the charge storage unit is increased for the same chip size, the number of pixels must be reduced, which reduces the resolution of the distance image capturing device. On the other hand, if the chip size is increased in proportion to the increase in the area of ​​the pixel circuit, the resolution of the range image pickup device can be maintained, but the cost per chip increases. Furthermore, if the capacity of the charge storage section is increased without changing the area of ​​the pixel circuit, the area of ​​the photoelectric conversion section must be reduced by the amount of the increase in the area of ​​the charge storage section, which results in a decrease in sensitivity to incident light and a decrease in the accuracy of the desired distance.

[0007] The present invention has been made in consideration of the above circumstances, and aims to provide a distance image capturing device and a distance image capturing method that can suppress saturation caused by an increase in the intensity of incident light while maintaining the resolution of the captured image and the sensitivity to incident light without increasing the capacity of the charge storage section. [Means for solving the problem]

[0008] In order to solve the above-mentioned problems, the distance image pickup device of the present invention includes a light receiving unit having a photoelectric conversion element that generates a charge corresponding to incident light that is light that is incident from a measurement space that is a space to be measured, N (N≧3) charge accumulation units that accumulate the charge in a frame period, a plurality of pixel circuits each including a transfer transistor that transfers the charge from the photoelectric conversion element to each of the charge accumulation units, and a pixel drive circuit that performs on / off processing on each of the transfer transistors in each of the charge accumulation units to allocate and accumulate the charge in a predetermined accumulation period synchronized with the irradiation of a light pulse, and Match a distance calculation unit that calculates the distance to a subject present in the measurement space as a measurement distance based on the amount of charge accumulated in each of the charge accumulation units. In the capacity increase mode, the measurement control unit sets, for each of the plurality of charge storage units of the combination, a divided accumulation count obtained by multiplying the accumulation count in the frame period by a predetermined ratio, turns on the transfer transistors in the different accumulation periods in a transfer order for each of the charge storage units of the combination, and transfers the charges generated by the photoelectric conversion elements by the divided accumulation count to each of the charge storage units of the combination, thereby dividing and storing the charges generated by the photoelectric conversion elements in each of the charge storage units of the combination. It is characterized by:

[0009] The distance image capturing device of the present invention comprises: The aforementioned The measurement control unit The aforementioned In the capacity increase mode, the transfer transistors corresponding to each of the plurality of charge storage units of the combination are turned on simultaneously in the transfer order for each storage period, and the charge generated by the photoelectric conversion element is transferred to each of the charge storage units of the combination, and the charge generated by the photoelectric conversion element is divided and stored in each of the charge storage units of the combination.

[0010] The distance image capturing device of the present invention is characterized in that, in a capacity increase mode, the measurement control unit sets, for each of the plurality of charge storage units of the combination, a split accumulation count that is multiplied by a predetermined ratio with respect to the accumulation count in the frame period, turns on the transfer transistor in the transfer order for each of the charge storage units of the combination at different accumulation periods, and transfers the charge generated by the photoelectric conversion element for the split accumulation count to each of the charge storage units of the combination, thereby dividing and storing the charge generated by the photoelectric conversion element in each of the charge storage units of the combination.

[0011] The distance image capturing device of the present invention comprises: The measurement control unit In the capacity increase mode, different The aforementioned The transfer transistor is repeatedly turned on for each accumulation period, and the charge generated by the photoelectric conversion element is alternately allocated and accumulated in each of the plurality of charge accumulation sections of the combination, and the charge generated by the photoelectric conversion element is divided and accumulated.

[0012] The distance image capturing device of the present invention comprises: a light receiving unit having a photoelectric conversion element that generates a charge corresponding to incident light that is light that is incident from a measurement space that is a space to be measured; N (N≧3) charge accumulation units that accumulate the charge in a frame period; a plurality of pixel circuits each including a transfer transistor that transfers the charge from the photoelectric conversion element to each of the charge accumulation units; a pixel drive circuit that performs on / off processing on each of the transfer transistors in each of the charge accumulation units to allocate and accumulate the charge in a predetermined accumulation period synchronized with the irradiation of a light pulse; a measurement control unit that divides, allocates, and accumulates the charge generated by the photoelectric conversion element to each combination of the charge accumulation units in any transfer period of the charge accumulation cycle; and a distance calculation unit that calculates the distance to a subject present in the measurement space as a measured distance based on the amount of charge accumulated in each of the charge accumulation units; The measurement control unit In the capacitance increasing mode, the charge storage portions are provided in a plurality of combinations of two or more.

[0013] In the distance image pickup device of the present invention, the measurement control unit The aforementioned The charges generated by the photoelectric conversion elements are transferred to the charge storage units at each accumulation period. The aforementioned a normal mode in which the transfer transistors are turned on in the transfer order to accumulate and transfer the charges generated by the photoelectric conversion elements, and a combination of the charge accumulation units in any transfer period of the charge accumulation cycle. Match Divide, allocate and accumulate The aforementioned The capacity increase mode is switched according to a predetermined condition.

[0014] The distance image capturing device of the present invention is characterized in that the specified condition is when the amount of charge stored in the charge storage unit exceeds a predetermined ratio to the maximum capacity of the charge storage unit, and the combination includes a charge storage unit that exceeds the ratio.

[0015] The distance image capturing method of the present invention is a distance image capturing method for controlling a distance image capturing device including a plurality of pixel circuits each including a photoelectric conversion element, a plurality of charge accumulation units, and a transfer transistor, a pixel drive circuit, a distance calculation unit, and a measurement control unit, and includes a step in which the pixel drive circuit allocates and accumulates, in a frame period, charges generated by the photoelectric conversion elements in response to incident light from a measurement space, in each of N (N≧3) charge accumulation units by turning on and off the transfer transistors that transfer the charges from the photoelectric conversion elements to the charge accumulation units, and a step in which the measurement control unit divides, allocates, and accumulates the charges generated by the photoelectric conversion elements to each combination of the charge accumulation units in any transfer period of a charge accumulation cycle. and a process of calculating the distance to the subject present in the measurement space as a measurement distance based on the amount of charge accumulated in each of the charge accumulation units, wherein in the process of dividing, allocating, and accumulating the charge among each combination of the charge accumulation units, the measurement control unit, in a capacity increase mode, sets a divided accumulation count for each of the plurality of charge accumulation units of the combination, which is multiplied by a predetermined ratio with respect to the accumulation count in the frame period, turns on the transfer transistor in a transfer order for each of the charge accumulation units of the combination at different accumulation periods, and transfers the charge generated by the photoelectric conversion element by the divided accumulation count to each of the charge accumulation units of the combination, thereby dividing and accumulating the charge generated by the photoelectric conversion element in each of the charge accumulation units of the combination. Further, a distance image capturing method of the present invention is a distance image capturing method for controlling a distance image capturing device including a plurality of pixel circuits each including a photoelectric conversion element, a plurality of charge accumulation units, and a transfer transistor, a pixel drive circuit, a distance calculation unit, and a measurement control unit, and includes the steps of: the pixel drive circuit dividing and accumulating charges generated by the photoelectric conversion elements in response to incident light from a measurement space into each of N (N≧3) charge accumulation units by turning on and off each of the transfer transistors that transfer the charges from the photoelectric conversion elements to the charge accumulation units at a predetermined accumulation period synchronized with the irradiation of a light pulse; the measurement control unit dividing and accumulating the charges generated by the photoelectric conversion elements into each combination of the charge accumulation units at any transfer period of the charge accumulation cycle; and the distance calculation unit calculating the distance to a subject present in the measurement space as a measured distance based on the amount of charge accumulated in each of the charge accumulation units. In the process of dividing and allocating the charge to each combination of the charge storage units and storing the charge, the measurement control unit In the capacitance increasing mode, the charge storage portions are provided in a plurality of combinations of two or more. [Effects of the Invention]

[0016] As described above, according to the present invention, it is possible to provide a distance image capturing device and a distance image capturing method that can suppress saturation due to an increase in the intensity of incident light while maintaining the resolution of the captured image and the sensitivity to incident light without increasing the capacity of the charge storage section. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a block diagram showing a schematic configuration of a distance imaging device according to a first embodiment of the present invention. [Figure 2] 1 is a circuit diagram showing an example of the configuration of a pixel circuit 321 arranged in a range image sensor 32 in the range image pickup device according to the first embodiment of the present invention. FIG. [Figure 3] 10 is a timing chart showing the transfer of charges generated in the photoelectric conversion element PD to each charge accumulation unit CS in the normal mode. FIG. [Figure 4]10 is a timing chart showing the transfer of charges generated in the photoelectric conversion element PD to each charge accumulation unit CS in the capacitance increase mode when the charge accumulation amount Q2 of the charge accumulation unit CS2 exceeds the capacitance threshold. FIG. [Figure 5] FIG. 10 is a timing chart showing the transfer of charges generated in the photoelectric conversion element PD to each of the charge accumulation units CS in the capacitance increase mode when the charge accumulation amount Q2 of the charge accumulation unit CS2 exceeds the capacitance threshold in the first embodiment. [Figure 6] 10 is a flowchart showing an example of the operation of the distance image pickup device 1 of the first embodiment in the process of calculating the distance between the distance image sensor 32 and the subject S. [Figure 7] FIG. 10 is a timing chart showing the transfer of charges generated in the photoelectric conversion element PD to each of the charge accumulation units CS in the capacitance increase mode when the charge accumulation amount Q2 of the charge accumulation unit CS2 exceeds the capacitance threshold in the second embodiment. [Figure 8] FIG. 11 is a timing chart showing the transfer of charges generated in the photoelectric conversion element PD to each charge accumulation unit CS in the capacitance increase mode in an environment where the influence of background light is negligible in the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0018] First Embodiment A first embodiment of the present invention will be described below with reference to the drawings. Fig. 1 is a block diagram showing the schematic configuration of a distance image pickup device according to a first embodiment of the present invention. The distance image pickup device 1 shown in Fig. 1 is a ToF distance image pickup device, and includes a light source unit 2, a light receiving unit 3, and a distance image processing unit 4. Fig. 1 also shows a subject S, which is an object whose distance is to be measured by the distance image pickup device 1. The distance image pickup element is, for example, a distance image sensor 32 (described below) in the light receiving unit 3.

[0019] The light source section 2 irradiates a light pulse PO into a space to be photographed, in which a subject S, the distance of which is to be measured by the distance image pickup device 1, is present, under the control of the distance image processor 4. The light source section 2 is, for example, a surface-emitting semiconductor laser module such as a vertical cavity surface-emitting laser (VCSEL). The light source section 2 includes a light source device 21 and a diffuser plate 22.

[0020] The light source device 21 is a light source that emits laser light in a near-infrared wavelength band (for example, a wavelength band of 850 nm to 940 nm) that becomes the light pulse PO to be irradiated onto the subject S. The light source device 21 is, for example, a semiconductor laser light-emitting element. The light source device 21 emits pulsed laser light in response to control from the timing control unit 41. The diffusion plate 22 is an optical component that diffuses the laser light in the near-infrared wavelength band emitted by the light source device 21 to the extent of the surface to be irradiated onto the subject S. The pulsed laser light diffused by the diffusion plate 22 is emitted as a light pulse PO and is irradiated onto the subject S.

[0021] The light receiving unit 3 receives reflected light RL of the light pulse PO reflected by a subject S, the distance of which is to be measured in the range image pickup device 1, and outputs a pixel signal corresponding to the received reflected light RL. The light receiving unit 3 includes a lens 31 and a range image sensor 32. The lens 31 is an optical lens that guides the incident reflected light RL to the range image sensor 32. The lens 31 outputs the incident reflected light RL to the range image sensor 32 side, and causes the light to be received (incident) by pixel circuits provided in the light receiving region of the range image sensor 32.

[0022] The range image sensor 32 is an imaging element used in the range image capturing device 1. The range image sensor 32 includes a plurality of pixel circuits 321 in a two-dimensional light receiving area, and a pixel drive circuit 322 that controls each of the pixel circuits 321. The pixel circuit 321 includes one photoelectric conversion element (for example, a photoelectric conversion element PD described later), a plurality of charge accumulation units (for example, charge accumulation units CS1 to CS4 described later) corresponding to the one photoelectric conversion element, and components that distribute charge to each charge accumulation unit.

[0023] The range image sensor 32 distributes the charges generated by the photoelectric conversion elements to the respective charge accumulation sections under the control of the timing control section 41. The range image sensor 32 also outputs pixel signals according to the amount of charge distributed to the charge accumulation sections. The range image sensor 32 has a plurality of pixel circuits arranged in a two-dimensional matrix, and outputs pixel signals for one frame corresponding to each pixel circuit.

[0024] The distance image processing unit 4 controls the distance image pickup device 1 and calculates the distance to the subject S. Distance image processing unit 4 includes timing control unit 41, distance calculation unit 42, and measurement control unit 43. The timing control unit 41 controls the timing of outputting various control signals required for distance measurement in accordance with the control of the measurement control unit 43. The various control signals here include, for example, a signal that controls the irradiation of the light pulse PO, a signal that distributes the reflected light RL to multiple charge accumulation units, a signal that controls the number of distributions per frame, etc. The number of distributions is the number of times that the process of distributing electric charges to the charge accumulation units CS (see FIG. 2) is repeated.

[0025] The distance calculation unit 42 outputs distance information calculated based on the pixel signals output from the range image sensor 32 under the control of the measurement control unit 43. The distance calculation unit 42 calculates the delay time Td from when the light pulse PO is emitted until when the reflected light RL is received, based on the amount of charge accumulated in the multiple charge accumulation units CS. The distance calculation unit 42 calculates the distance (measured distance) from the range image pickup device 1 to the range image pickup unit S according to the calculated delay time Td.

[0026] In a frame repeated at a frame period, when distributing the charge generated by the photoelectric conversion element PD due to incident light to each charge accumulation unit CS according to the amount of accumulated charge accumulated in the charge accumulation unit CS, the measurement control unit 43 selects the distribution algorithm from either the normal mode (a mode used when the charge accumulation unit is not saturated) or the capacity increase mode (a mode used when the charge accumulation unit is saturated) and controls the operation of each of the timing control unit 41, the distance calculation unit 42 and the pixel driving circuit 322. That is, the measurement control unit 43 controls the timing in the timing control unit 41, controls the calculation in the distance calculation unit 42, and distributes the charge from the photoelectric conversion element PD to each of the charge accumulation units CS by the pixel driving circuit 322, corresponding to each of the normal mode and the capacity increase mode (described in detail later).

[0027] With this configuration, in the distance image capturing device 1, the light source unit 2 irradiates a light pulse PO in the near-infrared wavelength band onto the subject S, and the light receiving unit 3 receives the reflected light RL reflected by the subject S, and the distance image processing unit 4 outputs distance information measuring the measured distance between the subject S and the distance image capturing device 1. Although FIG. 1 shows the distance image pickup device 1 having the distance image processing unit 4 built therein, the distance image processing unit 4 may be an element provided outside the distance image pickup device 1.

[0028] That is, the range image pickup device according to this embodiment calculates the distance between the subject and the range image sensor 32 based on the charges accumulated in the charge accumulation unit CS. Therefore, when calculating the measured distance, if the intensity of reflected light or background light is high, the intensity of the incident light will increase and the charge generated in the photoelectric conversion element will exceed the maximum storage capacity of the charge storage unit CS, which is the capacity at which the charge can be stored.In this case, the amount of charge generated by the photoelectric conversion element cannot be accurately obtained, and the measured distance between the distance image capturing device 1 and the subject S cannot be accurately determined by calculating the distance using the amount of charge in the charge storage unit.

[0029] Furthermore, as the distance from the range image capturing device 1 to the subject S becomes shorter and the reflectance of the subject S becomes higher, the intensity of the reflected light generated by the reflection of the light pulse PO on the subject S increases. Therefore, the electric charge generated by the reflected light in the photoelectric conversion element increases. Therefore, the maximum storage capacity of the charge storage unit is exceeded, and the measured distance between the range image pickup device 1 and the subject S cannot be accurately determined by distance calculation using the amount of charge in the charge storage unit.

[0030] The above-described process is performed as an auto-exposure process, which controls the number of times of integration or the number of times of emission of the light pulse PO depending on the amount of charge accumulated in the charge accumulation section. Here, when the intensity of the reflected light increases, the number of times that the charge generated in the photoelectric conversion element by the reflected light is accumulated is reduced, and the amount of charge required to calculate the measured distance is accumulated in the charge accumulation element, thereby improving the accuracy of the measured distance obtained.

[0031] However, it is generally rare for the reflectance of each subject to be the same; the reflectance of each subject is different from each other, and the intensity of light is inversely proportional to the distance to the subject. Therefore, if the number of accumulations is set in advance to match an object at a close distance, the charge generated by reflected light from a subject with low reflectance that is at a distance from the distance image capture device will be reduced, lowering the S / N ratio and reducing the accuracy of the measured distance.

[0032] On the other hand, when the number of accumulations is set to match a distant subject in order to improve the accuracy of the measurement distance, if the distance from each of the subjects other than the distant subject to the distance image capturing device is unknown and a subject is present at a close distance to the distance image capturing device 1, the intensity of the reflected light from the close subject will be greater than that of a subject at a long distance, and the charge storage section will become saturated with the charge generated by the reflected light from the close subject. For this reason, in this embodiment, as will be described later, the measurement control unit 43 controls the timing control unit 41 to select either the normal mode or the capacity increase mode as the distribution algorithm when distributing the charge generated by the photoelectric conversion element PD due to incident light to each of the charge accumulation units so that the charge accumulation units do not become saturated.

[0033] Next, the configuration of the pixel circuit 321 in the range image sensor 32 will be described. Figure 2 is a circuit diagram showing an example of the configuration of a pixel circuit 321 arranged in the range image sensor 32 in the range image pickup device according to the first embodiment of the present invention. The pixel circuit 321 in Figure 2 is an example configuration including, for example, four pixel signal readout units RU1 to RU4. The configuration of the pixel circuit 321 in this embodiment is just one example, and the pixel circuit 321 has a configuration including three or more pixel signal readout units, i.e., n units (n≧3).

[0034] The pixel circuit 321 includes one photoelectric conversion element PD, a charge discharging transistor GD, and four pixel signal readout units RU (RU1 to RU4) that output voltage signals from corresponding output terminals O. Each pixel signal readout unit RU includes a transfer transistor G, a floating diffusion FD, a charge storage capacitance C, a reset transistor RT, a source follower transistor SF, and a selection transistor SL. The floating diffusions FD (FD1, FD2, FD3, FD4) and the charge storage capacitances C (C1, C2, C3, C4) form charge storage units CS (CS1, CS2, CS3, CS4).

[0035] 2, the pixel signal readout unit RU1, which outputs a voltage signal from the output terminal O1, includes a transfer transistor G1 (transfer MOS transistor), a floating diffusion FD1, a charge storage capacitor C1, a reset transistor RT1, a source follower transistor SF1, and a selection transistor SL1. In the pixel signal readout unit RU1, the floating diffusion FD1 and the charge storage capacitor C1 form a charge storage unit CS1. The pixel signal readout units RU2, RU3, and RU4 have a similar configuration.

[0036] The photoelectric conversion element PD is a buried photodiode that photoelectrically converts incident light, generates charges corresponding to the incident light, and accumulates the generated charges. In this embodiment, the incident light is incident from the space to be measured. In pixel circuit 321, the photoelectric conversion element PD photoelectrically converts incident light to generate electric charges, which are then distributed to each of four charge accumulation units CS (CS1 to CS4), and voltage signals corresponding to the amount of distributed electric charges are output to distance image processing unit 4. Furthermore, the configuration of the pixel circuit arranged in the distance image sensor 32 is not limited to the configuration having four pixel signal readout units RU (RU1 to RU4) as shown in Figure 2, but may be a pixel circuit having a configuration having one or more pixel signal readout units RU.

[0037] In driving the pixel circuit 321 of the range image pickup device 1, a light pulse PO is emitted for an irradiation time To, and reflected light RL is received by the range image sensor 32 after a delay time Td. The pixel drive circuit 322 synchronizes with the irradiation of the light pulse PO under the control of the timing control unit 41, and supplies accumulation drive signals TX1 to TX4 to transfer transistors G1, G2, G3, and G4 at their respective timings to redirect the charges generated in the photoelectric conversion element PD, causing them to be accumulated in the charge accumulation units CS1, CS2, CS3, and CS4 in that order.

[0038] The pixel driving circuit 322 controls each of the reset transistor RT and the selection transistor SL by driving signals RST and SEL, respectively, converts the charges accumulated in the charge storage unit CS into an electrical signal by the source follower transistor SF, and outputs the generated electrical signal to the distance calculation unit 42 via the output terminal O. Furthermore, under the control of the timing control section 41, the pixel driving circuit 322 causes the charge generated in the photoelectric conversion element PD to flow to the power supply VDD in response to the driving signal RSTD, thereby discharging the charge (erasing the charge).

[0039] FIG. 3 is a timing chart showing the transfer of charges generated by the photoelectric conversion elements PD to the charge accumulation units CS in the normal mode. In the timing chart of Fig. 3, the vertical axis represents the pulse level, and the horizontal axis represents time. Fig. 3 also shows the accumulation cycle repeated during the charge accumulation period of a frame. The diagram shows the relative relationship on the time axis between the light pulse PO and the reflected light RL in normal mode, the timing of each of the accumulation drive signals TX1 to TX4 supplied to the transfer transistors G1 to G4, and the timing of the drive signal RSTD supplied to the charge discharge transistor GD.

[0040] The timing control unit 41 causes the light source unit 2 to irradiate the measurement space with a light pulse PO. As a result, the light pulse PO is reflected by the subject and received by the light receiving unit 3 as reflected light RL. The photoelectric conversion element PD then generates charges corresponding to the background light and the reflected light RL. The pixel drive circuit 322 controls the on / off of each of the transfer transistors G1 to G4 to transfer the charges generated by the photoelectric conversion element PD to each of the charge accumulation units CS1 to CS4. That is, the pixel drive circuit 322 supplies each of the accumulation drive signals TX1 to TX4 as an "H" level signal with a predetermined time width (the same width as the irradiation time To, ie, the pulse width) to the transfer transistors G1 to G4, respectively.

[0041] The pixel driving circuit 322, for example, turns on a transfer transistor G1 provided on a transfer path that transfers charges from the photoelectric conversion element PD to the charge accumulation unit CS1. As a result, charges photoelectrically converted by the photoelectric conversion element PD are accumulated in the charge accumulation unit CS1 via the transfer transistor G1. Thereafter, the pixel driving circuit 322 turns off the transfer transistor G1. This stops the transfer of charges to the charge accumulation unit CS1. In this way, the pixel driving circuit 322 accumulates charges in the charge accumulation unit CS1. The same applies to the other charge accumulation units CS2, CS3, and CS4.

[0042] At this time, during the charge accumulation period in which charge is distributed to the charge accumulation sections CS (the period in which charge is accumulated in each of the charge accumulation sections CS in a frame), transfer periods T1, T2, T3 and T4 (periods in which charge is accumulated and integrated, transfer order) in which each of the accumulation drive signals TX1, TX2, TX3 and TX4 is supplied to the transfer transistors G1, G2, G3 and G4, respectively, are repeated in the accumulation period. Here, for example, in transfer period T1, charges generated by incident light (background light only) are accumulated in charge accumulation unit CS1, in transfer period T2, charges generated by incident light (background light + reflected light) are accumulated in charge accumulation unit CS2, in transfer period T3, charges generated by incident light (background light + reflected light) are accumulated in charge accumulation unit CS3, and in transfer period T4, charges generated by incident light (background light + reflected light) are accumulated in charge accumulation unit CS4.

[0043] Then, charges corresponding to incident light are transferred from the photoelectric conversion element PD to the charge accumulation units CS1, CS2, CS3, and CS4, respectively, via the transfer transistors G1, G2, G3, and G4. That is, during the charge accumulation period of each frame period, a plurality of accumulation periods are repeated in which charge is transferred in the order of transfer to each of the charge accumulation units CS1, CS2, CS3, and CS4. As a result, charges are accumulated in the charge accumulation units CS1, CS2, CS3, and CS4 in the transfer periods T1, T2, T3, and T4, respectively, for each accumulation period of the charge accumulation units CS1, CS2, CS3, and CS4 in the charge accumulation period.

[0044] Furthermore, when repeating the accumulation cycles of the charge accumulation units CS1, CS2, CS3, and CS4, after the transfer (distribution) of charges to the charge accumulation unit CS4 is completed, the pixel driving circuit 322 supplies an "H" level driving signal RSTD to the charge discharging transistor GD provided on the discharge path that discharges charges from the photoelectric conversion element PD, to turn it on. As a result, before the transfer period T1 for the charge storage unit CS1 starts, the charge discharging transistor GD discards the charge generated in the photoelectric conversion element PD after the transfer period T4 for the previous charge storage unit CS4 (i.e., resets the photoelectric conversion element PD).

[0045] The pixel driving circuit 322 then sequentially performs signal processing such as A / D conversion on the voltage signals from all of the pixel circuits 321 arranged in the light receiving section 3 in units of rows (horizontal arrangement) of the pixel circuits 321. Thereafter, the pixel driving circuit 322 outputs the processed voltage signals to the distance calculation unit 42 in the order of the columns arranged in the light receiving unit 3.

[0046] As described above, the pixel driving circuit 322 accumulates charges in the charge accumulation units CS and discards the charges photoelectrically converted by the photoelectric conversion elements PD over one frame. As a result, charges corresponding to the amount of light received by the distance image pickup device 1 over a predetermined time period are accumulated in each charge accumulation unit CS. The pixel driving circuit 322 outputs an electrical signal corresponding to the amount of charge accumulated in each charge accumulation unit CS for one frame to the distance calculation unit 42.

[0047] Due to the relationship between the timing of irradiating the light pulse PO and the timing of accumulating charges in each of the charge accumulation units CS (CS1 to CS4), the charge accumulation unit CS1 holds an amount of charge corresponding to external light components such as background light before irradiating the light pulse PO. Furthermore, the charge accumulation units CS2, CS3, and CS4 hold charges corresponding to the reflected light RL and external light components, respectively. The distribution (allocation ratio) of the amount of charge allocated to the charge accumulation units CS2 and CS3, or the charge accumulation units CS3 and CS4, is a ratio that corresponds to the delay time Td between when the light pulse PO is reflected by the subject S and when it enters the range image pickup device 1.

[0048] Returning to FIG. 1, the distance calculation unit 42 uses this principle to calculate the delay time Td using the following equation (1) or (2). Td=To×(Q3-Q1) / (Q2+Q3-2×Q1) …(1) Td=To+To×(Q4-Q1) / (Q3+Q4-2×Q1) …(2) Here, To is the period during which the light pulse PO is irradiated, Q1 is the amount of charge accumulated in the charge accumulation unit CS1, Q2 is the amount of charge accumulated in the charge accumulation unit CS2, Q3 is the amount of charge accumulated in the charge accumulation unit CS3, and Q4 is the amount of charge accumulated in the charge accumulation unit CS4. For example, when Q4=Q1, the distance calculation unit 42 calculates the delay time Td using equation (1), and when Q2=Q1, the distance calculation unit 42 calculates the delay time Td using equation (2).

[0049] In equation (1), charges generated by reflected light are accumulated in the charge accumulation units CS2 and CS3, but not in the charge accumulation unit CS4. On the other hand, in equation (2), charges generated by reflected light are accumulated in the charge accumulation units CS3 and CS4, but not in the charge accumulation unit CS2. In addition, equation (1) or (2) is based on the premise that the amount of charge stored in the charge storage units CS2, CS3, and CS4 that corresponds to the external light component is the same as the amount of charge stored in the charge storage unit CS1.

[0050] The distance calculation unit 42 calculates the measured distance to the subject S and back by multiplying the delay time obtained by equation (1) or (2) by the speed of light (velocity). Then, the distance calculation unit 42 calculates the distance from the distance image sensor 32 (i.e., the distance image pickup device 1) to the subject S by dividing the calculated round trip distance by 2 (delay time Td×c (speed of light) / 2).

[0051] Furthermore, time Trs indicates the period during which the drive signal RSTD supplied to the charge discharging transistor GD is set to the “H” level after the distribution of charge from the photoelectric conversion element PD to the charge storage section CS4 in one cycle of the accumulation period in FIG. 3 is completed so that the charge generated by the input light does not remain (accumulate) in the photoelectric conversion element PD.

[0052] 4 is a block diagram showing an example of the configuration of the measurement control unit 43 in the distance image pickup device of Embodiment 1. In Fig. 4, the measurement control unit 43 includes a normal mode operating condition setting unit 431, a mode determination unit 432, a capacity increase mode operating condition setting unit 433, an operation control unit 434, a mode determination condition storage unit 435, and an operating condition combination storage unit 436.

[0053] The normal mode operating condition setting unit 431 reads out the control settings of the timing control unit 41 in the normal mode of the timing chart shown in FIG. Then, the normal mode operating condition setting unit 431 supplies the read normal mode settings to the operation control unit 434. By setting the normal mode, the operation control unit 434 controls the operations of the timing control unit 41, the distance calculation unit 42, and the pixel drive circuit 322, and causes them to operate in the normal mode shown in FIG.

[0054] The mode determination unit 432 determines whether to operate the timing control unit 41 in the normal mode or the capacity increase mode. Here, when the charge accumulation unit CS1 accumulates charges generated by background light, the mode determination unit 432 extracts the number of pixel circuits 321 in which the charge accumulation amount Q2 of the charge accumulation unit CS2, the charge accumulation amount Q3 of the charge accumulation unit CS3, and the charge accumulation amount Q4 of the charge accumulation unit CS4 each exceed a preset capacitance threshold.

[0055] The mode determination unit 432 reads out the capacitance threshold value from the mode determination condition storage unit 435 and compares it with each of the accumulated charge amounts Q2, Q3, and Q4. This first threshold value is set, for example, as either 80% of the maximum storage capacity of the charge storage unit CS or an amount of charge equal to or greater than 80%. In actual operation, the capacitance threshold is set to an empirical range of, for example, about 90% to 95% of the maximum storage capacity, taking into consideration fluctuations in the charge stored in the charge storage section CS due to reflected light.

[0056] However, when the capacitance threshold is set to 90% to 95% of the maximum storage capacity, the photoelectric conversion element PD generates charges due to incident light that includes not only reflected light but also background light. Therefore, taking into account the influence of background light, it is expected that the charge storage unit CS will become saturated depending on the change in the intensity of the background light. Therefore, when a margin is provided for fluctuations in the amount of charge generated by the photoelectric conversion element PD due to incident light caused by changes in background light, experimental results showed that a margin of 60% or more is desirable.

[0057] The mode determination unit 432 determines, for each pixel circuit 321, whether or not the charge accumulation amount Q2, the charge accumulation amount Q3, and the charge accumulation amount Q4, which are the amounts of charge accumulated in the charge accumulation unit CS2, the charge accumulation unit CS3, and the charge accumulation unit CS4, respectively, exceed a preset capacitance threshold. Then, the mode determination unit 432 counts the number of charge accumulation units CS2, CS3, and CS4 in all pixel circuits 321 whose accumulation capacitance exceeds the capacitance threshold. Here, the mode determination unit 432 extracts the charge storage unit CS having the largest number of times the storage capacitance exceeds the capacitance threshold from among the charge storage units CS2, CS3, and CS4. The method for extracting the charge accumulation portion is not limited to the above-described method. For example, the ratio of saturated pixels to the total number of pixels (for example, 20% of the total number of pixels) may be set in advance as an extraction threshold, the saturated pixels of each charge accumulation portion may be counted, and the charge accumulation portion may be extracted when the saturated pixels exceed the extraction threshold.

[0058] At this time, the mode determination unit 432 distributes the charges generated by the photoelectric conversion element PD in the transfer period T2 (transfer order) of the accumulation period (charge accumulation cycle) in the normal mode only to the charge accumulation unit CS2. However, for example, when the charge storage amount Q2 of the charge storage unit CS2 exceeds the capacity threshold, the mode determination unit 432 stores the charge generated by the photoelectric conversion element PD in the transfer period T2 in the charge storage units CS2 and CS3 as a capacity increase mode. Furthermore, the mode determination unit 432 causes the charge storage unit CS4 to store the charge generated by the photoelectric conversion element PD in the transfer period T3. The charge storage unit combined with the charge storage unit CS2 is not limited to the charge storage unit CS3. For example, the same operation as described above can be achieved by combining the charge storage unit CS4 with the charge storage unit CS2 at the timing of the transfer period T2, and allocating the charge storage unit CS3 to the transfer period T3 and the charge storage unit CS1 to the transfer period T1. That is, the mode determination unit 432 divides and allocates and accumulates the charges generated by the photoelectric conversion element PD in response to incident light for each combination of the charge accumulation units CS in any transfer period of the charge accumulation cycle.

[0059] Figure 5 is a timing chart showing the transfer of charges generated in the photoelectric conversion element PD to each of the charge storage units CS in the capacitance increase mode when the charge storage amount Q2 of the charge storage unit CS2 in the first embodiment exceeds the capacitance threshold. In the timing chart of Fig. 5, the vertical axis indicates the pulse level (H level / L level), and the horizontal axis indicates time. Fig. 5 also shows the transfer order in the accumulation cycle that is repeated during the charge accumulation period of a frame. It shows the timing of each of the accumulation drive signals TX1 to TX4 supplied to the transfer transistors G1 to G4 in the capacitance increase mode, and the timing of the drive signal RSTD supplied to the charge discharging transistor GD.

[0060] The timing control unit 41 causes the light source unit 2 to irradiate the measurement space with a light pulse PO. As a result, the light pulse PO is reflected by the subject and received by the light receiving unit 3 as reflected light RL. The photoelectric conversion element PD then generates charges corresponding to the background light and the reflected light RL. The pixel drive circuit 322 controls the on / off of each of the transfer transistors G1 to G4 to transfer the charges generated by the photoelectric conversion element PD to each of the charge accumulation units CS1 to CS4.

[0061] That is, the pixel drive circuit 322 supplies each of the accumulation drive signals TX1 to TX4 as an "H" level signal with a predetermined time width (the same width as the irradiation time To, ie, the pulse width) to the transfer transistors G1 to G4, respectively. The pixel driving circuit 322, for example, turns on a transfer transistor G1 provided on a transfer path that transfers charges from the photoelectric conversion element PD to the charge accumulation unit CS1. As a result, the charges photoelectrically converted by the photoelectric conversion element PD are accumulated in the charge accumulation unit CS1 via the transfer transistor G1. Thereafter, the pixel driving circuit 322 turns off the transfer transistor G1. As a result, the transfer of charges to the charge accumulation unit CS1 is stopped.

[0062] In addition, in the capacity increase mode, the pixel driving circuit 322, under the timing control of the timing control unit 41, sets the accumulation driving signals TX2 and TX3 to the “H” level at the same timing (simultaneously in the same transfer order) during the transfer period T2 in which charge is accumulated in the charge accumulation unit CS2. Each of the transfer transistors G2 and G3 divides and distributes the charge generated by the photoelectric conversion element PD to the charge accumulation unit CS2 and the charge accumulation unit CS3, respectively. As a result, the amount of charge that would have been saturated in only the charge storage unit CS2 is divided and stored in each of the charge storage units CS2 and CS3, making it possible to prevent the amount of stored charge from exceeding the maximum storage capacity and becoming saturated.

[0063] In addition, in the normal mode, the storage drive signal TX4 is set to the "H" level in the transfer period T4. However, in the capacitance increase mode, the charge storage unit CS4 has the function of the charge storage unit CS3 in the normal mode, so the pixel drive circuit 322 sets the storage drive signal TX4 to the "H" level in the transfer period T3. As a result, the charge generated by the photoelectric conversion element PD in the transfer period T3 is distributed and stored in the charge storage unit CS4 via the transfer transistor G4.

[0064] Then, charges corresponding to incident light are transferred from the photoelectric conversion element PD to the charge accumulation units CS1, CS2, CS3, and CS4 via the transfer transistors G1, G2, G3, and G4, respectively. During the charge accumulation period, multiple accumulation periods each consisting of transfer periods T1 to T3 and Trs are repeated. As a result, charges are accumulated in the charge accumulation units CS1, CS2 and CS3, and CS4 for each transfer period of the charge accumulation period, i.e., the charge accumulation unit CS1 (transfer period T1), CS2 and CS3 (transfer period T2), and CS4 (transfer period T3).

[0065] Furthermore, when the pixel driving circuit 322 repeats each transfer cycle in which charge is transferred from the photoelectric conversion element PD to each of the charge accumulation units CS1, CS2, CS3, and CS4, after the transfer (distribution) of charge to the charge accumulation unit CS4 is completed, it supplies an "H" level drive signal RSTD to the charge discharging transistor GD provided on the discharge path that discharges charge from the photoelectric conversion element PD to turn it on.

[0066] As a result, before the transfer period T1 for the charge storage unit CS1 begins, the charge discharging transistor GD discards the charge generated in the photoelectric conversion element PD after the transfer period T3 for the previous charge storage unit CS4 (i.e., resets the photoelectric conversion element PD). In the capacity increase mode, the transfer periods T2 and T3 are essentially integrated and the transfer period T4 is eliminated. Therefore, as shown in FIG. 5, at the time Trs after the transfer period T3, the charge discharging transistor GD is turned on by supplying an "H" level drive signal RSTD.

[0067] After the accumulation period in the frame cycle ends, the pixel drive circuit 322 performs signal processing such as A / D conversion on the voltage signals from all of the pixel circuits 321 arranged in the light receiving section 3, sequentially in units of rows (horizontal arrangement) of the pixel circuits 321. The process of reading out the charge accumulation amounts Q1, Q2, Q3, and Q4 from the charge accumulation sections CS1, CS2, CS3, and CS4 of the pixel circuits 321, respectively, is the same as in the normal mode. Thereafter, the pixel driving circuit 322 outputs the processed voltage signals to the distance calculation unit 42 in the order of the columns arranged in the light receiving unit 3.

[0068] Here, the distance calculation unit 42 adds the digital values ​​of the charge accumulation amounts Q2 and Q3 supplied by the pixel driving circuit 322 (treating them as the charge amounts accumulated in one charge accumulation unit CS2), and uses this addition result Q2+Q3 and each of the charge accumulation amounts Q1 and Q4 to calculate the delay time Td using equation (4) described below, thereby obtaining the distance between the subject S and the distance image pickup device 1. Furthermore, an adder may be provided before the A / D conversion process to add the analog values ​​of the charge accumulation amounts Q2 and Q3, the addition result Q2+Q3 by the adder is A / D converted, and the resulting digital value of the addition result Q2+Q3 may be output to the distance calculation unit 42.

[0069] As described above, the pixel driving circuit 322 accumulates charges in the charge accumulation units CS and discards the charges photoelectrically converted by the photoelectric conversion elements PD over one frame. As a result, charges corresponding to the amount of light received by the distance image pickup device 1 over a predetermined time period are accumulated in each charge accumulation unit CS. The pixel driving circuit 322 outputs an electrical signal corresponding to the amount of charge accumulated in each charge accumulation unit CS for one frame to the distance calculation unit 42.

[0070] Due to the relationship between the timing of irradiating the light pulse PO and the timing of accumulating charges in each of the charge accumulation units CS (CS1 to CS4), the charge accumulation unit CS1 holds an amount of charge corresponding to external light components such as background light before irradiating the light pulse PO. In addition, the charge accumulation units CS2, CS3, and CS4 hold divided amounts of charge corresponding to the reflected light RL and external light components. Here, in Figure 5, the distribution (distribution ratio) of the amount of charge distributed to the charge accumulation units CS2 and CS3 and the charge accumulation unit CS4 is a ratio that corresponds to the delay time Td from when the light pulse PO is reflected by the subject S until it enters the distance image capturing device 1.

[0071] Furthermore, in the case of the combination of the charge storage units CS shown in FIG. 5 in the capacitance increasing mode, the distance calculation unit 42 calculates the delay time Td by the following equation (4). Td=To×(Q4-Q1) / (Q2+Q3+Q4-3×Q1)…(4) At this time, the distance calculation unit 42 calculates the measured distance to the subject S and back by multiplying the delay time Td calculated by the above equation (4) by the speed of light (velocity). Then, the distance calculation unit 42 calculates the distance from the distance image sensor 32 (i.e., the distance image pickup device 1) to the subject S by dividing the calculated round trip distance by 2 (delay time Td×c (speed of light) / 2).

[0072] Furthermore, in the capacitance increasing mode when the charge storage amount Q3 of the charge storage unit CS3 exceeds the capacitance threshold, the charge storage units CS3 and CS4 are used in combination. That is, for example, in the capacity increase mode in which the charge accumulation amount Q3 of the charge accumulation unit CS3 exceeds the capacity threshold, the pixel driving circuit 322 sets the accumulation driving signal TX1 to the “H” level at the timing of the transfer period T1 in which charge is accumulated in the charge accumulation unit CS1 under the timing control of the timing control unit 41. Under timing control by the timing control section 41, the pixel drive circuit 322 goes to the "H" level at the timing of the accumulation drive signal TX2 in the transfer period T2 during which charges are accumulated in the charge accumulation section CS2.

[0073] Then, under the timing control of the timing control section 41, the pixel driving circuit 322 sets the accumulation driving signals TX3 and TX4 to the "H" level at the same timing in the transfer cycle T3 in which charges are accumulated in the charge accumulation section CS3. Here, each of the transfer transistors G3 and G4 divides and distributes the charge generated by the photoelectric conversion element PD to the charge accumulation unit CS3 and the charge accumulation unit CS4, respectively. As a result, the amount of charge that would have been saturated in only the charge storage unit CS3 is divided and stored in each of the charge storage units CS3 and CS4, making it possible to prevent the amount of stored charge from exceeding the maximum storage capacity and becoming saturated.

[0074] In addition, in the normal mode, the storage drive signal TX4 is set to the "H" level in the transfer period T4. However, in the capacitance increase mode, the charge storage unit CS4 has the function of the charge storage unit CS3 in the normal mode, and therefore the pixel drive circuit 322 sets the storage drive signal TX4 to the “H” level in the transfer period T3 as described above. As a result, in the capacity increase mode, the transfer periods T2 and T3 are essentially integrated and the transfer period T4 is eliminated, so that at the time Trs after the transfer period T3, the charge discharging transistor GD is supplied with an "H" level drive signal RSTD to turn it on.

[0075] Due to the relationship between the timing of irradiating the light pulse PO and the timing of accumulating charges in each of the charge accumulation units CS (CS1 to CS4), the charge accumulation unit CS1 holds an amount of charge corresponding to external light components such as background light before irradiating the light pulse PO. In addition, the charge accumulation units CS2, CS3, and CS4 hold divided amounts of charge corresponding to the reflected light RL and external light components. Here, when the charge accumulation amount Q3 of the charge accumulation unit CS3 exceeds the capacitance threshold, the distribution (allocation ratio) of the charge amount allocated to the charge accumulation unit CS2 and the charge accumulation units CS3 and CS4 becomes a ratio corresponding to the delay time Td from when the light pulse PO is reflected by the subject S to when it is incident on the distance image capturing device 1.

[0076] Therefore, in the case of a combination of charge storage units CS in which the charge storage amount Q3 of the charge storage unit CS3 in the capacitance increasing mode exceeds the capacitance threshold, the distance calculation unit 42 calculates the delay time Td by the following equation (5). Td=To×(Q3+Q4-2×Q1) / (Q2+Q3+Q4-3×Q1) …(5) At this time, the distance calculation unit 42 calculates the measured distance to the subject S and back by multiplying the delay time Td calculated by the above equation (5) by the speed of light (velocity). Then, the distance calculation unit 42 calculates the distance from the distance image sensor 32 (i.e., the distance image pickup device 1) to the subject S by dividing the calculated round trip distance by 2 (delay time Td×c (speed of light) / 2).

[0077] Here, the distance calculation unit 42 adds the digital values ​​of the charge accumulation amounts Q3 and Q4 supplied by the pixel driving circuit 322 (treating them as the charge amounts accumulated in one charge accumulation unit CS3), and calculates the delay time Td using the sum Q3+Q4 and each of the charge accumulation amounts Q1 and Q2 according to the above equation (5).

[0078] Furthermore, an adder may be provided before the A / D conversion process in the pixel driving circuit 322 to add the analog values ​​of the charge accumulation amounts Q3 and Q4, and the result of addition Q3+Q4 by the adder may be A / D converted, and the resulting digital value of addition result Q3+Q4 may be output to the distance calculation unit 42.

[0079] 6 is a flowchart showing an example of the operation of the distance image pickup device 1 of the first embodiment in calculating the distance between the distance image sensor 32 and the subject S. When the distance image pickup device 1 is started, the distance measurement process starts from the following step S1. Step S1: The normal mode operating condition setting unit 431 in the measurement control unit 43 reads out the normal mode operating conditions (control information for the timing control unit 41, distance calculation unit 42, and pixel drive circuit 322 in the normal mode) that are the settings for the normal mode from the operating condition combination storage unit 436. Then, the normal mode operating condition setting unit 431 supplies the read normal mode operating conditions to the operation control unit 434. The operation control unit 434 controls the operations of the timing control unit 41, the distance calculation unit 42, and the pixel drive circuit 322 according to the normal mode operation conditions, and causes the operation in the normal mode shown in FIG.

[0080] Step S2: The pixel drive circuit 322 sequentially supplies the accumulation drive signals TX1, TX2, TX3, and TX4 to the transfer transistors G1, G2, G3, and G4, respectively, and distributes the charges generated by the photoelectric conversion element PD to the charge accumulation units CS1, CS2, CS3, and CS4. Then, the pixel driving circuit 322 outputs the amounts of charge Q1, Q2, Q3, and Q4 accumulated in the charge accumulation units CS1, CS2, CS3, and CS4, respectively, to the distance calculation unit 42 and the measurement control unit 43 by a predetermined number of accumulation cycles during one frame. As a result, the distance calculation unit 42 obtains the distance from the distance image pickup device 1 to the subject S using equation (1).

[0081] Step S3: The mode determination unit 432 reads out from the mode determination condition storage unit 435 a preset capacitance threshold value to be compared with the storage capacitance in each of the charge storage units CS. When the current operation is in normal mode, the mode determination unit 432 determines for each pixel circuit 321 whether the charge accumulation amount Q1, charge accumulation amount Q2, charge accumulation amount Q3, and charge accumulation amount Q4, which are the amounts of charge accumulated in the charge accumulation units CS1, CS2, CS3, and CS4, respectively, exceed a predetermined capacitance threshold.

[0082] Furthermore, the mode determination unit 432 counts the number of charge accumulation units CS whose accumulation capacitance exceeds the capacitance threshold in all pixel circuits 321 for each type of charge accumulation unit CS2, charge accumulation unit CS3, and charge accumulation unit CS4. After this counting, the mode determination unit 432 extracts the charge storage unit CS with the largest number of accumulated capacitances exceeding the capacitance threshold from among the types of the charge storage units CS2, CS3, and CS4.

[0083] On the other hand, when the current operation is in the capacity increase mode, the mode determination unit 432 determines, for each pixel circuit 321, whether the amount of charge accumulated in each of the two (or three or more) charge storage units CS operating in combination and the charge storage unit CS operating independently exceeds a preset capacity threshold.

[0084] For example, when the combination of the charge storage units CS2 and CS3 operates as one charge storage and the charge storage unit CS4 operates independently, the mode determination unit 432 determines whether the sum of the charge storage amounts Q2 and Q3 and the charge storage amount Q4 each exceed a preset capacitance threshold. That is, the mode determination unit 432 compares the charge storage amount Q2+Q3 in the combination of the charge storage units CS2 and CS3 and the charge storage amount Q4 in the charge storage unit CS with the capacitance threshold.

[0085] In addition, the mode determination unit 432 counts the number of charge storage units CS whose storage capacity exceeds the capacity threshold in all pixel circuits 321, by type: two charge storage units CS operating in combination, and charge storage units CS operating independently. After this counting, the mode determination unit 432 extracts the charge storage unit CS whose storage capacity exceeds the capacity threshold the greatest number of times from each type of charge storage unit CS, that is, two charge storage units CS operating in combination and one charge storage unit CS operating independently.

[0086] Here, when the combination of charge storage units CS2 and CS3 operates as one charge storage and the charge storage unit CS4 operates independently, the mode determination unit 432 counts the number of units (i.e., the number of pixels) whose storage capacitance exceeds the capacitance threshold for each type of the combination of charge storage units CS2 and CS3 and the charge storage unit CS4. Then, the mode determination unit 432 sets the number of types with the larger total number as the maximum number, and extracts either the combination of the charge accumulation units CS2 and CS3 or the charge accumulation unit CS4 as the corresponding type.

[0087] Step S4: The mode determination unit 432 reads out from the mode determination condition storage unit 435 a preset number (threshold number) to be compared with the number of charge storage units CS (or a combination of two charge storage units) whose storage capacity exceeds the capacity threshold by the largest number, i.e., the maximum number. Then, the mode determination unit 432 determines whether or not the extracted maximum number exceeds the set number. At this time, if the maximum number exceeds the set number, mode determination unit 432 advances the process to step S5. On the other hand, if the maximum number is equal to or less than the set number, mode determination unit 432 advances the process to step S6.

[0088] Step S5: The mode determination unit 432 outputs to the capacity increase mode operating condition setting unit 433 the type of the maximum number of charge storage units CS, that is, which of the charge storage units CS2, CS3, and CS4 has the maximum number. The capacity increase mode operating condition setting unit 433 reads out the capacity increase mode operating conditions (control information for the timing control unit 41, distance calculation unit 42, and pixel driving circuit 322 in the capacity increase mode) of the capacity increase mode corresponding to the maximum number of charge storage units CS, for example, charge storage unit CS2, from the operating condition combination memory unit 436.

[0089] Here, the capacity increase mode operating condition of the capacity increase mode corresponding to the charge storage unit CS2 is a setting in which the charge storage units CS2 and CS3 are used as one charge storage unit CS, as already explained. Therefore, in the transfer period T2 under the operating conditions of the increased capacitance mode, the pixel drive circuit 22 sets the accumulation drive signals TX2 and TX3 to the "H" level at the same timing, and divides and distributes the signals to the transfer transistors G2 and G3.

[0090] As described above, the mode determination unit 432 reads out the capacity increase mode operating conditions for each of the charge storage units CS2, CS3, and CS4 from the operating condition combination storage unit 436, corresponding to the type of charge storage unit CS extracted as the maximum number. Then, the capacity increase mode operating condition setting unit 433 outputs the read capacity increase mode operating conditions to the operation control unit 434. As a result, the operation control unit 434 controls the operation of the timing control unit 41, the distance calculation unit 42, and the pixel driving circuit 322 according to the capacity increase mode operation conditions corresponding to each of the charge storage units CS, and causes them to operate in the capacity increase mode. It should be noted that the steps are not limited to those described above, and the device may be driven in a capacity increasing mode in advance.

[0091] Step S6: The mode determination unit 432 determines whether the current operation mode is the normal mode, that is, whether it is the normal mode or the capacity increase mode. At this time, if the current operation mode is the normal mode, the mode determination unit 432 advances the process to step S2. On the other hand, if the mode determination unit 432 determines that the current operation mode is not the normal mode (the capacity increase mode), the process proceeds to step S1.

[0092] As described above, according to this embodiment, when the number of pixels at which the charge storage unit CS becomes saturated exceeds a preset number, multiple (for example, two) charge storage units CS are combined to operate as a single charge storage unit. For example, when the number of charge storage units CS2 whose capacity (storage capacity) of accumulated charge exceeds a capacity threshold exceeds a preset number, the charge storage units CS2 and CS3 are combined to divide and distribute the charge generated by the photoelectric conversion element PD, thereby preventing the charge storage unit CS2 from becoming saturated and reducing the accuracy of the distance measurement calculation results. Therefore, it is possible to suppress saturation due to an increase in the intensity of incident light while maintaining the resolution of the captured image and the sensitivity to incident light without increasing the capacity of the charge storage units.

[0093] <Second embodiment> A second embodiment of the present invention will be described below with reference to the drawings. The range imaging device according to the second embodiment of the present invention has the same configuration as the range imaging device 1 according to the first embodiment shown in FIG. 1, which has already been described. In the following, only the operations of the second embodiment that are different from those of the first embodiment will be described.

[0094] As already explained, in the capacity increase mode, the distance image capturing device of the first embodiment turns on the transfer transistors G corresponding to the combined charge storage units CS simultaneously in the same transfer cycle in the same accumulation cycle, and performs a process of dividing and distributing the charge generated in the photoelectric conversion element PD by incident light to each of the combined charge storage units CS. However, in the second embodiment, in the capacity increase mode, in each of the same transfer periods in different accumulation periods (in each of the same transfer orders in different accumulation periods), the transfer transistors G corresponding to the combined charge accumulation units CS are turned on, and the charge generated in the photoelectric conversion element PD by incident light is divided and distributed to each of the combined charge accumulation units CS.

[0095] 7 is a timing chart showing the transfer of charges generated in the photoelectric conversion element PD to each charge accumulation unit CS in the capacitance increase mode when the charge accumulation amount Q2 of the charge accumulation unit CS2 exceeds the capacitance threshold in the second embodiment. In the timing chart of FIG. 7, the vertical axis represents the pulse level ("H" level / "L" level), and the horizontal axis represents time. FIG. 7 also shows an accumulation cycle repeated during the charge accumulation period in a frame cycle. The timings of the accumulation drive signals TX1 to TX4 supplied to the transfer transistors G1 to G4, respectively, and the timing of the drive signal RSTD supplied to the charge discharging transistor GD are shown in the capacitance increase mode.

[0096] 5, the timing control unit 41 causes the light source unit 2 to irradiate the measurement space with a light pulse PO. As a result, the light pulse PO is reflected by the subject and received by the light receiving unit 3 as reflected light RL. The photoelectric conversion element PD then generates charges corresponding to the background light and the reflected light RL. The pixel drive circuit 322 controls the on / off of each of the transfer transistors G1 to G4 to transfer the charges generated by the photoelectric conversion element PD to each of the charge accumulation units CS1 to CS4.

[0097] That is, the pixel drive circuit 322 supplies each of the accumulation drive signals TX1 to TX4 as an "H" level signal with a predetermined time width (the same width as the irradiation time To, ie, the pulse width) to the transfer transistors G1 to G4, respectively. The pixel driving circuit 322, for example, turns on a transfer transistor G1 provided on a transfer path that transfers charges from the photoelectric conversion element PD to the charge accumulation unit CS1. As a result, the charges photoelectrically converted by the photoelectric conversion element PD are accumulated in the charge accumulation unit CS1 via the transfer transistor G1. Thereafter, the pixel driving circuit 322 turns off the transfer transistor G1. As a result, the transfer of charges to the charge accumulation unit CS1 is stopped.

[0098] Furthermore, when measuring the distance to the subject S in the normal mode shown in FIG. 3, if the mode determination unit 432 detects that the number of pixels where the charge accumulation amount Q2 of the charge accumulation unit CS2 exceeds the capacity threshold exceeds a set number, the capacity increase mode operating condition setting unit 433 reads out, from the operating condition combination storage unit 436, a capacity increase mode operating condition that operates the charge accumulation units CS2 and CS3 in combination. Then, the capacity increase mode operating condition setting unit 433 outputs the read capacity increase mode operating conditions to the operation control unit 434. The operation control unit 434 controls the operation of the timing control unit 41, the distance calculation unit 42, and the pixel driving circuit 322 according to the capacity increase mode operation conditions corresponding to each of the charge storage units CS, and causes them to operate in the capacity increase mode.

[0099] As a result, in the case of the capacitance increase mode, under the timing control of the timing control unit 41, during an accumulation period in which charge is accumulated in a frame period, if there are, for example, m accumulation periods, the pixel driving circuit 322 sets the accumulation driving signal TX2 to the “H” level during the transfer period T2, turns on the transfer transistor G2, and accumulates charge in the charge accumulation unit CS2, from the first accumulation period to the m / 2th accumulation period in the accumulation period. Furthermore, the pixel driving circuit 322 sets the accumulation driving signal TX3 to the “H” level in the transfer period T2 from the (m / 2)+1th accumulation period to the mth accumulation period, and turns on the transfer transistor G3 to accumulate charge in the charge accumulation section CS3.

[0100] That is, in the capacity increase mode when the charge storage amount Q2 of the charge storage unit CS2 exceeds the capacity threshold, during the storage period of each frame cycle, in the same transfer cycle in different storage cycles, the charge generated by the photoelectric conversion element PD due to incident light is not distributed to one charge storage unit CS2, but is divided and distributed to multiple units, i.e., two charge storage units CS2 and CS3. As a result, the amount of charge that would have been saturated in only the charge storage unit CS2 is divided and stored in each of the charge storage units CS2 and CS3, making it possible to prevent the amount of stored charge from exceeding the maximum storage capacity and becoming saturated.

[0101] In addition, in the normal mode, the storage drive signal TX4 is set to the "H" level in the transfer period T4. However, in the capacitance increase mode, the charge storage unit CS4 has the function of the charge storage unit CS3 in the normal mode, and therefore, as in the first embodiment already described, the pixel drive circuit 322 sets the storage drive signal TX4 to the “H” level in the transfer period T3. As a result, the charge generated by the photoelectric conversion element PD in the transfer period T3 is distributed and stored in the charge storage unit CS4 via the transfer transistor G4.

[0102] Furthermore, under the operating conditions of the above-mentioned capacity increase mode, if there are, for example, m accumulation cycles, charge is accumulated in the charge accumulation unit CS2 from the first accumulation cycle to the m / 2 accumulation cycle in the accumulation period, and charge is accumulated in the charge accumulation unit CS3 from the (m / 2)+1 accumulation cycle to the m accumulation cycle. However, the configuration may also be such that the accumulation drive signal TX2 is set to the "H" level in the transfer period T2 of an odd-numbered accumulation period, and the accumulation drive signal TX3 is set to the "H" level in the transfer period T2 of an even-numbered accumulation period, and the charge generated by the photoelectric conversion unit PD is divided and distributed alternately to each of the charge accumulation units CS2 and CS3. Furthermore, the configuration is not limited to the above, and may be such that, for example, the number of accumulation periods in the accumulation period of a frame period is randomly allocated so that half is divided between the charge accumulation unit CS2 and half is divided between the charge accumulation unit CS3.

[0103] Also, the number of accumulation periods in the accumulation period of a frame period is halved (equally divided) so that the charges generated by the photoelectric conversion unit PD are distributed to each of the charge accumulation units CS2 and CS3. However, a configuration may be adopted in which the number of accumulation cycles in the accumulation period of a frame cycle is set to a predetermined ratio and the charge generated by the photoelectric conversion unit PD is divided and distributed to each of the charge accumulation units CS2 and CS3. In other words, even if the amount of charge divided to each of the charge accumulation units CS2 and CS3 is not equal but is a predetermined ratio, the effect of reducing saturation as in the case of only one charge accumulation unit CS2 can be obtained.

[0104] Then, charges corresponding to incident light are transferred from the photoelectric conversion element PD to the charge accumulation units CS1, CS2, CS3, and CS4 via the transfer transistors G1, G2, G3, and G4, respectively. During the charge accumulation period, a plurality of accumulation periods, each consisting of a transfer period T1 to a transfer period T3 and a time Trs, are repeated. As a result, charges are accumulated in the charge accumulation unit CS1, the charge accumulation units CS2 and CS3, and the charge accumulation unit CS4 for each of the transfer cycle T1 in the charge accumulation period, the transfer cycle T2 having a different accumulation cycle, and the transfer cycle T3.

[0105] Furthermore, when the pixel driving circuit 322 repeats each transfer cycle in which charges are transferred from the conversion element PD to each of the charge accumulation units CS1, CS2, CS3, and CS4 in the above paragraph, after the transfer (distribution) of charges to the charge accumulation unit CS4 is completed, it supplies an "H" level drive signal RSTD to the charge discharging transistor GD provided on the discharge path that discharges charges from the photoelectric conversion element PD to turn it on. As a result, before the transfer period T1 for the charge storage unit CS1 begins, the charge discharging transistor GD discards the charge generated in the photoelectric conversion element PD after the transfer period T3 for the previous charge storage unit CS4 (i.e., resets the photoelectric conversion element PD). In the capacity increase mode, the transfer periods T2 and T3 are essentially integrated and the transfer period T4 is eliminated. Therefore, as shown in FIG. 7, at the time Trs after the transfer period T3, the charge discharging transistor GD is turned on by supplying an "H" level drive signal RSTD.

[0106] Then, as in the first embodiment, after the accumulation period in a frame cycle ends, the pixel drive circuit 322 sequentially performs signal processing such as A / D conversion on the voltage signals from each of all pixel circuits 321 arranged in the light receiving unit 3, in units of rows (horizontal arrangement) of the pixel circuits 321. The process of reading out the charge accumulation amounts Q1, Q2, Q3, and Q4 from the charge accumulation units CS1, CS2, CS3, and CS4 of the pixel circuits 321, respectively, is the same as in the normal mode. Thereafter, the pixel driving circuit 322 outputs the processed voltage signals to the distance calculation unit 42 in the order of the columns arranged in the light receiving unit 3.

[0107] Here, the distance calculation unit 42 adds the digital values ​​of the charge accumulation amount Q2 and the charge accumulation amount Q3 supplied by the pixel driving circuit 322 (treating them as the charge amounts accumulated in one charge accumulation unit CS2), and uses this addition result Q2+Q3 and each of the charge accumulation amount Q1 and the charge accumulation amount Q4 to calculate the delay time Td in the same manner as in the first embodiment, and finds the distance between the subject S and the distance image pickup device 1. Furthermore, an adder may be provided before the A / D conversion process to add the analog values ​​of the charge accumulation amounts Q2 and Q3, the addition result Q2+Q3 by the adder is A / D converted, and the resulting digital value of the addition result Q2+Q3 may be output to the distance calculation unit 42.

[0108] As described above, the pixel driving circuit 322 accumulates charges in the charge accumulation units CS and discards the charges photoelectrically converted by the photoelectric conversion elements PD over one frame. As a result, charges corresponding to the amount of light received by the distance image pickup device 1 over a predetermined time period are accumulated in each charge accumulation unit CS. The pixel driving circuit 322 outputs an electrical signal corresponding to the amount of charge accumulated in each charge accumulation unit CS over one frame (frame period) to the distance calculation unit 42.

[0109] Due to the relationship between the timing of irradiating the light pulse PO and the timing of accumulating charges in each of the charge accumulation units CS (CS1 to CS4), the charge accumulation unit CS1 holds an amount of charge corresponding to external light components such as background light before irradiating the light pulse PO. In addition, the charge accumulation units CS2, CS3, and CS4 hold divided amounts of charge corresponding to the reflected light RL and external light components. Here, in Figure 7, the distribution (distribution ratio) of the amount of charge distributed to the charge accumulation units CS2 and CS3 and the charge accumulation unit CS4 is a ratio that corresponds to the delay time Td from when the light pulse PO is reflected by the subject S until it enters the distance image capturing device 1.

[0110] In addition, in the capacitance increasing mode when the charge accumulation amount Q3 of the charge accumulation unit CS3 exceeds the capacitance threshold, the charge accumulation unit CS2 is used alone (one unit) and the charge accumulation units CS3 and CS4 are used in combination. That is, when measuring the distance to the subject S in the normal mode shown in FIG. 3, if the mode determination unit 432 detects that the number of pixels where the charge accumulation amount Q3 of the charge accumulation unit CS3 exceeds the capacity threshold exceeds a set number, the capacity increase mode operating condition setting unit 433 reads out, from the operating condition combination memory unit 436, the capacity increase mode operating conditions that operate the charge accumulation units CS3 and CS4 in combination.

[0111] Then, the capacity increase mode operating condition setting unit 433 outputs the read capacity increase mode operating conditions to the operation control unit 434. The operation control unit 434 controls the operation of the timing control unit 41, the distance calculation unit 42, and the pixel driving circuit 322 according to the capacity increase mode operation conditions corresponding to each of the charge storage units CS, and causes them to operate in the capacity increase mode.

[0112] As a result, in the case of the capacitance increase mode, under the timing control of the timing control unit 41, during an accumulation period in which charge is accumulated in a frame period, if there are, for example, m accumulation periods, the pixel driving circuit 322 sets the accumulation driving signal TX3 to the “H” level in the transfer period T3, turns on the transfer transistor G3, and accumulates charge in the charge accumulation unit CS3, from the first accumulation period to the m / 2th accumulation period in the accumulation period. Furthermore, from the (m / 2)+1th accumulation period to the mth accumulation period, the pixel driving circuit 322 sets the accumulation driving signal TX4 to the “H” level in the transfer period T3, turns on the transfer transistor G4, and accumulates charge in the charge storage section CS4.

[0113] That is, in the capacity increase mode when the charge storage amount Q3 of the charge storage unit CS3 exceeds the capacity threshold, during the storage period of each frame cycle, in the same transfer cycle in different storage cycles, the charge generated by the photoelectric conversion element PD due to incident light is not distributed to one charge storage unit CS3, but is divided and distributed to multiple units, i.e., two charge storage units CS3 and CS4. As a result, the amount of charge that would have been saturated in only the charge storage unit CS3 is divided and stored in each of the charge storage units CS3 and CS4, making it possible to prevent the amount of stored charge from exceeding the maximum storage capacity and becoming saturated.

[0114] Furthermore, in the normal mode, the accumulation drive signal TX2 is set to the "H" level in the transfer period T2, as in the normal mode. As a result, in the charge storage unit CS2, the charge generated by the photoelectric conversion element PD in the transfer period T2 is distributed and stored via the transfer transistor G2, similar to the normal mode.

[0115] Furthermore, in the driving mode shown in FIG. 7 in the capacitance increasing mode, when the charge storage units CS3 and CS4 are combined in the transfer period T3, the distance calculation unit 42 calculates the delay time Td by the following equation (6). Td=To×(Q3+Q4-2×Q1) / (Q2+Q3+Q4-3×Q1)…(6) At this time, the distance calculation unit 42 calculates the measured distance to the subject S and back by multiplying the delay time Td obtained by the above equation (6) by the speed of light (velocity). Then, the distance calculation unit 42 calculates the distance from the distance image sensor 32 (i.e., the distance image pickup device 1) to the subject S by dividing the calculated round trip distance by 2 (delay time Td×c (speed of light) / 2).

[0116] Furthermore, under the operating conditions of the above-mentioned capacity increase mode, if there are, for example, m accumulation cycles, charge is accumulated in the charge accumulation unit CS3 from the first accumulation cycle to the m / 2 accumulation cycle in the accumulation period, and charge is accumulated in the charge accumulation unit CS4 from the (m / 2)+1 accumulation cycle to the m accumulation cycle. However, for example, the accumulation drive signal TX3 may be set to the "H" level in the transfer period T3 of the odd-numbered accumulation period, and the accumulation drive signal TX4 may be set to the "H" level in the transfer period T3 of the even-numbered accumulation period, and the charge generated by the photoelectric conversion unit PD may be divided and distributed alternately to each of the charge accumulation units CS3 and CS4.

[0117] Also, the number of accumulation periods in the accumulation period of a frame period is halved (equally divided) so that the charges generated by the photoelectric conversion unit PD are distributed to each of the charge accumulation units CS3 and CS4. However, a configuration may be adopted in which the number of accumulation cycles in the accumulation period of a frame cycle is set to a predetermined ratio and the charges generated by the photoelectric conversion unit PD are divided and distributed to the charge accumulation units CS3 and CS4. In other words, even if the amount of charge divided to the charge accumulation units CS3 and CS4 is not equal but is a predetermined ratio, the effect of reducing saturation, as in the case of only one charge accumulation unit CS3, can be obtained.

[0118] Then, charges corresponding to incident light are transferred from the photoelectric conversion element PD to the charge accumulation units CS1, CS2, CS3, and CS4 via the transfer transistors G1, G2, G3, and G4, respectively. A plurality of accumulation periods, each consisting of transfer periods T1 to T4, are repeated during the charge accumulation period. As a result, charges are accumulated in the charge accumulation units CS1, CS2, CS3 and CS4, respectively, for each of the transfer cycles T1, T2 and T3, which has a different accumulation cycle, during the charge accumulation period.

[0119] As described above, according to this embodiment, similarly to the first embodiment, when the number of pixels at which the charge storage unit CS becomes saturated exceeds a preset number, multiple (for example, two) charge storage units CS are combined to operate as a single charge storage unit. For example, when the number of charge storage units CS2 whose capacity (storage capacity) of accumulated charge exceeds a capacity threshold exceeds a preset number, the charge storage units CS2 and CS3 are combined to divide and distribute the charge generated by the photoelectric conversion element PD, thereby preventing the charge storage unit CS2 from becoming saturated and reducing the accuracy of the distance measurement calculation results. Therefore, it is possible to suppress saturation due to an increase in the intensity of incident light while maintaining the resolution of the captured image and the sensitivity to incident light without increasing the capacity of the charge storage units.

[0120] <Third embodiment> A third embodiment of the present invention will be described below with reference to the drawings. The range imaging device according to the second embodiment of the present invention has the same configuration as the range imaging device 1 according to the first embodiment shown in FIG. 1, which has already been described. In the following, only the operations of the second embodiment that are different from those of the first embodiment will be described.

[0121] As already explained, in the capacity increase mode, the distance image capturing device of the first embodiment turns on the transfer transistors G corresponding to the combined charge storage units CS simultaneously in the same transfer cycle in the same accumulation cycle, and performs a process of dividing and distributing the charge generated in the photoelectric conversion element PD by incident light to each of the combined charge storage units CS. However, the third embodiment has a capacity increase mode that is premised on an environment where the influence of background light can be ignored, for example, a state in which there is almost no background light in the dark environment. Note that the driving in the above-described embodiment is for pixels having 4 taps (4 charge storage sections), and for pixels having charge storage sections of 5 taps (5 charge storage sections) or more, a capacitance increase mode is used that takes into account the influence of background light.

[0122] 8 is a timing chart showing the transfer of charges generated in the photoelectric conversion element PD to each charge accumulation unit CS in the capacitance increase mode in an environment where the influence of background light is negligible in the third embodiment. In the timing chart of FIG. 8, the vertical axis indicates the pulse level ("H" level / "L" level), and the horizontal axis indicates time. FIG. 8 also shows an accumulation cycle repeated during the charge accumulation period in a frame cycle. The timings of the accumulation drive signals TX1 to TX4 supplied to the transfer transistors G1 to G4, respectively, and the timing of the drive signal RSTD supplied to the charge discharging transistor GD are shown in the capacitance increase mode.

[0123] In this embodiment, since there is no need to acquire background light, in the capacity increase mode, the light source unit 2 irradiates the light pulse PO at the same timing as the accumulation period T1 according to the capacity increase mode operating conditions in the operating condition combination storage unit 436. Moreover, the mode determination to set the capacity increase mode is performed in the normal mode, as in the first and second embodiments. Then, the capacity increase mode operating condition setting unit 433 outputs the read capacity increase mode operating conditions to the operation control unit 434.

[0124] The operation control unit 434 controls the operation of the timing control unit 41, the distance calculation unit 42, and the pixel driving circuit 322 according to the capacity increase mode operation conditions corresponding to each of the charge storage units CS, and causes them to operate in the capacity increase mode. The above-mentioned capacity increase mode operating conditions do not use the charge storage unit CS1 to store background light, so the charge storage units CS1 and CS2 are combined with each other and with the charge storage units CS3 and CS4, and are operated in correspondence with each of the charge storage units CS1 and CS2 in the normal mode.

[0125] That is, in the capacity increase mode, the pixel driving circuit 322, under the timing control of the timing control unit 41, sets the accumulation driving signals TX1 and TX2 to the “H” level at the same timing during the transfer period T1 in which charge is accumulated in the charge accumulation unit CS1 during normal driving. Each of the transfer transistors G1 and G2 divides and distributes the charge generated by the photoelectric conversion element PD to the charge accumulation unit CS1 and the charge accumulation unit CS2, respectively. As a result, the amount of charge that would have been saturated in only the charge storage unit CS1 is divided and stored in each of the charge storage units CS1 and CS2, making it possible to prevent the amount of stored charge from exceeding the maximum storage capacity and becoming saturated.

[0126] Similarly, in the case of the capacitance increase mode, the pixel drive circuit 322, under the timing control of the timing control unit 41, sets the accumulation drive signals TX3 and TX4 to the “H” level at the same timing during the transfer period T2 in which charge is accumulated in the charge accumulation unit CS2 in normal drive. Each of the transfer transistors G3 and G4 divides and distributes the charge generated by the photoelectric conversion element PD to the charge accumulation unit CS3 and the charge accumulation unit CS4, respectively. As a result, the amount of charge that would have been saturated in only the charge storage unit CS2 is divided and stored in each of the charge storage units CS3 and CS4, making it possible to prevent the amount of stored charge from exceeding the maximum storage capacity and becoming saturated.

[0127] Then, charges corresponding to incident light are transferred from the photoelectric conversion element PD to the charge accumulation units CS1, CS2, CS3, and CS4 via the transfer transistors G1, G2, G3, and G4, respectively. A plurality of accumulation periods, each consisting of transfer periods T1 to T4, are repeated during the charge accumulation period. As a result, charges are accumulated in the charge accumulation units CS1 and CS2 and the charge accumulation units CS3 and CS4 for each transfer period of the charge accumulation units CS1 and CS2 (transfer period T1) and the charge accumulation units CS3 and CS4 (transfer period T2) during the charge accumulation period.

[0128] Furthermore, when the pixel driving circuit 322 repeats each transfer cycle in which charges are transferred from the conversion element PD to each of the charge accumulation units CS1, CS2, CS3, and CS4 in the above paragraph, after the transfer (distribution) of charges to the charge accumulation unit CS4 is completed, it supplies an "H" level drive signal RSTD to the charge discharging transistor GD provided on the discharge path that discharges charges from the photoelectric conversion element PD to turn it on.

[0129] As a result, before the transfer period T1 for the charge storage unit CS1 starts, the charge discharging transistor GD discards the charge generated in the photoelectric conversion element PD after the transfer period T2 for the immediately preceding charge storage unit CS4 (i.e., resets the photoelectric conversion element PD). In the capacity increase mode, the transfer periods T1 and T2 and the transfer periods T3 and T4 are essentially integrated, and the transfer periods T3 and T4 are eliminated. Therefore, as shown in FIG. 8, at the time Trs after the transfer period T2, the charge discharging transistor GD is supplied with an "H" level drive signal RSTD to turn it on.

[0130] Furthermore, similarly to the second embodiment, the capacitance increase mode when the charge accumulation amounts Q2 and Q3 of the charge accumulation units CS2 and CS3 exceed the capacitance threshold value may be configured such that, during the accumulation period of each frame cycle, in the same transfer cycle in different accumulation cycles, the charge generated by the photoelectric conversion element PD in response to incident light is not distributed to one charge accumulation unit CS2 or CS3, but is divided and distributed to two charge accumulation units CS1 and CS2, respectively, or to two charge accumulation units CS3 and CS4, respectively.

[0131] Furthermore, assuming that background light is ignored, in the three charge accumulation units CS1, CS2, and CS3, distance measurement may be performed using a combination of the charge accumulation units CS1 and CS2 and the charge accumulation unit CS3, as in the configuration of the first embodiment, or distance measurement may be performed using a combination of the charge accumulation unit CS1 and the charge accumulation units CS2 and CS3.

[0132] As described above, according to this embodiment, similarly to the first embodiment, when the number of pixels at which the charge storage unit CS becomes saturated exceeds a preset number, multiple (for example, two) charge storage units CS are combined to operate as a single charge storage unit. For example, when the number of charge storage units CS2 and CS3 whose capacity (storage capacity) of stored charge exceeds a capacity threshold exceeds a preset number, the charge storage units CS1 and CS2, or the charge storage units CS3 and CS4, are combined to divide and distribute the charge generated by the photoelectric conversion element PD, thereby preventing the charge storage units CS2 and CS3 from becoming saturated and reducing the accuracy of the distance measurement calculation results. Therefore, it is possible to suppress saturation due to an increase in the intensity of incident light while maintaining the resolution of the captured image and the sensitivity to incident light without increasing the capacity of the charge storage units. [Explanation of symbols]

[0133] 1...Distance image capturing device 2...Light source section 3...Light receiving section 21...Light source device 22...Diffuser 31...Lens 32...Distance image sensor (distance image sensor) 321...Pixel circuit 322...Pixel driving circuit 4...Distance image processing section 41...Timing control section 42...Distance calculation section 43...Measurement control section 431...Normal mode operating condition setting unit 432...Mode determination unit 433...Capacity increase mode operating condition setting unit 434...Operation control unit 435...Mode determination condition storage unit 436...Operating condition combination memory unit CS1, CS2, CS3, CS4...Charge storage section FD1, FD2, FD3, FD4...Floating diffusion G1, G2, G3, G4...Transfer transistors GD: Charge drain transistor PD...photoelectric conversion element PO...light pulse RL…Reflected light RT1, RT2, RT3, RT4...Reset transistors S…Subject SF1, SF2, SF3, SF4...Source follower transistors SL1, SL2, SL3, SL4...Selection transistors

Claims

1. a plurality of pixel circuits each including a photoelectric conversion element that generates an electric charge according to incident light that is light that is incident from a measurement space that is a space to be measured, N (N≧3) charge accumulation units that accumulate the electric charge in a frame period, and a transfer transistor that transfers the electric charge from the photoelectric conversion element to each of the charge accumulation units; a pixel drive circuit that performs on / off processing of each of the transfer transistors in each of the charge accumulation units at a predetermined accumulation period synchronized with irradiation of a light pulse, and distributes and accumulates the charges; a light receiving unit having a measurement control unit that divides and distributes the charges generated by the photoelectric conversion elements to each combination of the charge accumulation units and accumulates the charges in any transfer period of a charge accumulation cycle; a distance calculation unit that calculates a distance to a subject present in the measurement space as a measured distance based on the amount of charge accumulated in each of the charge accumulation units; Equipped with The measurement control unit In the capacity increase mode, for each of the plurality of charge storage units of the combination, a divided accumulation count is set which is obtained by multiplying the accumulation count in the frame period by a predetermined ratio, and for each of the charge storage units of the combination, the transfer transistors are turned on in a transfer order in different accumulation periods, and charges generated by the photoelectric conversion elements for the divided accumulation count are transferred to each of the charge storage units of the combination, thereby dividing and storing the charges generated by the photoelectric conversion elements in each of the charge storage units of the combination. A distance image capturing device characterized by:

2. The measurement control unit In the capacitance increase mode, the transfer transistor is repeatedly turned on for each of the different accumulation periods, and the charges generated by the photoelectric conversion elements are alternately allocated and accumulated in each of the plurality of charge accumulation units of the combination, and the charges generated by the photoelectric conversion elements are divided and accumulated.

2. The distance imaging device according to claim 1,

3. a plurality of pixel circuits each including a photoelectric conversion element that generates an electric charge according to incident light that is light that is incident from a measurement space that is a space to be measured, N (N≧3) charge accumulation units that accumulate the electric charge in a frame period, and a transfer transistor that transfers the electric charge from the photoelectric conversion element to each of the charge accumulation units; a pixel drive circuit that performs on / off processing of each of the transfer transistors in each of the charge accumulation units at a predetermined accumulation period synchronized with irradiation of a light pulse, and distributes and accumulates the charges; a light receiving unit having a measurement control unit that divides and distributes the charges generated by the photoelectric conversion elements to each combination of the charge accumulation units and accumulates the charges in any transfer period of a charge accumulation cycle; a distance calculation unit that calculates a distance to a subject present in the measurement space as a measured distance based on the amount of charge accumulated in each of the charge accumulation units; Equipped with The measurement control unit provides a combination of the charge storage units as a plurality of sets of two or more in a capacitance increasing mode. A distance image capturing device characterized by:

4. The measurement control unit A normal mode is selected for each accumulation cycle in the accumulation period of the frame cycle, in which the charges generated by the photoelectric conversion elements are accumulated and transferred by turning on transfer transistors in the transfer order set for each of the charge accumulation units, and a capacity increase mode is selected for each of the charge accumulation cycles, in which the charges generated by the photoelectric conversion elements are divided, allocated, and accumulated in each of the combinations of the charge accumulation units in any transfer cycle of the charge accumulation cycle, based on predetermined conditions.

3. The distance imaging device according to claim 1 or 2.

5. The predetermined condition is The charge amount stored in the charge storage unit exceeds a preset ratio with respect to the maximum capacity of the charge storage unit, and the combination includes a charge storage unit that exceeds the ratio.

5. The distance imaging device according to claim 4.

6. A distance image capturing method for controlling a distance image capturing device including a plurality of pixel circuits, each of which includes a photoelectric conversion element, a plurality of charge accumulation units, and a transfer transistor, a pixel drive circuit, a distance calculation unit, and a measurement control unit, a step in which the pixel driving circuit allocates and accumulates charges generated by the photoelectric conversion elements in response to incident light from a measurement space in each of N (N≧3) charge accumulation units by performing on / off processing on the transfer transistors that transfer the charges from the photoelectric conversion elements to the charge accumulation units, at a predetermined accumulation period synchronized with irradiation of a light pulse during a frame period; a step in which the measurement control unit divides and allocates the charges generated by the photoelectric conversion elements to each combination of the charge accumulation units and accumulates them in any transfer period of a charge accumulation cycle; a step in which the distance calculation unit calculates a distance to a subject present in the measurement space as a measurement distance based on the amount of charge accumulated in each of the charge accumulation units; Including, In the process of dividing and allocating the charge to each combination of the charge storage units and storing the charge, the measurement control unit In the capacity increase mode, for each of the plurality of charge storage units of the combination, a divided accumulation count is set which is obtained by multiplying the accumulation count in the frame period by a predetermined ratio, and for each of the charge storage units of the combination, the transfer transistors are turned on in a transfer order in different accumulation periods, and charges generated by the photoelectric conversion elements for the divided accumulation count are transferred to each of the charge storage units of the combination, thereby dividing and storing the charges generated by the photoelectric conversion elements in each of the charge storage units of the combination. A distance image capturing method comprising:

7. A distance image capturing method for controlling a distance image capturing device including a plurality of pixel circuits, each of which includes a photoelectric conversion element, a plurality of charge accumulation units, and a transfer transistor, a pixel drive circuit, a distance calculation unit, and a measurement control unit, a step in which the pixel driving circuit allocates and accumulates charges generated by the photoelectric conversion elements in response to incident light from a measurement space into each of N (N≧3) charge accumulation units by performing on / off processing on each of the transfer transistors that transfer the charges from the photoelectric conversion elements to the charge accumulation units at a predetermined accumulation period synchronized with irradiation of a light pulse; a step in which the measurement control unit divides and allocates the charges generated by the photoelectric conversion elements to each combination of the charge accumulation units and accumulates them in any transfer period of a charge accumulation cycle; a step in which the distance calculation unit calculates a distance to a subject present in the measurement space as a measurement distance based on the amount of charge accumulated in each of the charge accumulation units; Including, In the process of dividing and allocating the charge to each of the combinations of the charge storage units and storing the charge, the measurement control unit provides the combinations of the charge storage units as two or more sets in a capacity increase mode. A distance image capturing method comprising:

Images