Range image capturing device and range image capturing method

Abstract

To provide a distance image pickup device which prevents such a state that electric charges are accumulated only in any of two electric charge storage parts even when a waveform of an optical pulse becomes blunt and the pulse width is deformed to be short, and reduces an error between a distance obtained from a ratio of the electric charge amounts of the electric charge storage parts and an actual distance.SOLUTION: A distance image pickup device comprises: a light source unit which emits an optical pulse; a light reception unit which includes a photoelectric conversion element that generates an electric charge according to the incident light, a plurality of pixel circuits that have electric charge storage parts for accumulating the electric charges and a pixel drive circuit that distributes the electric charges generated by the photoelectric conversion element via a transfer transistor to the electric charge storage parts at timing in synchronization with the optical pulse; and a distance calculation unit which calculates a distance between a subject and the light reception unit from an electric charge ratio of the electric charge amounts generated by reflection light, distributed via the transfer transistor from the photoelectric conversion element and accumulated in the electric charge storage parts. The width of the optical pulse is set to be long in comparison to a period of the rise of the specific transfer transistor and the rise of the transfer transistor that rises the next in the transfer transistors.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a distance image capturing device and a distance image capturing method. [Background technology]

[0002] Conventionally, there have been time-of-flight (ToF) distance imaging devices that utilize the known speed of light to measure the distance to a subject based on the flight time of light (see, for example, Patent Document 1). A ToF distance imaging device includes a light source unit that emits light and an imaging unit that includes a pixel array in which a plurality of pixel circuits that detect light for measuring distance are arranged in a two-dimensional matrix (array). Each of the pixel circuits has a photoelectric conversion element (e.g., a photodiode) that generates a charge corresponding to the intensity of light. With this configuration, the ToF range image pickup device can acquire (capture) information about the distance between itself and a subject, and an image of the subject, in a measurement space (three-dimensional space). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-29054 Summary of the Invention [Problem to be solved by the invention]

[0004] In the above-described distance imaging device, the pixels receive the light reflected from the subject when the emitted light pulse is reflected, and the received reflected light is photoelectrically converted by the photoelectric conversion element, and the charge obtained by the photoelectric conversion is distributed to the charge accumulation section at predetermined time intervals. In the distance image pickup device, charges are accumulated in each of the charge accumulation sections, and the distance between the subject and each pixel is calculated based on the ratio (charge ratio) of the amount of charges accumulated in each of the two charge accumulation sections.

[0005] However, the optical pulse does not have an ideal rectangular wave shape, and the rising and falling edges of the optical pulse are blunt. Therefore, when the time required for distributing charge from the photoelectric conversion element to the charge storage unit (the pulse width of the storage drive signal TX, which will be described later) is the same as the width of the light pulse, the waveform of the light pulse becomes dull, making the pulse width of the light pulse narrower (shorter) than the time required for distributing charge, and there is a situation in which charge can only be distributed to one of the two charge storage units.

[0006] In particular, at the timing when the combination of two charge accumulation units to which the reflected light is allocated changes among the plurality of charge accumulation units, the bias in the allocated charges becomes particularly large. If the waveform of the light pulse becomes dull and the pulse width of the light pulse becomes narrower than the charge distribution time, a state in which charge is accumulated in only one of the charge accumulation sections is likely to occur. As a result, at the timing when the combination of charge storage units is switched, the change in charge ratio becomes insensitive to the actual change in distance, and the distance fluctuates in response to deviations in the charge ratio due to noise, etc., resulting in a decrease in the temporal resolution and spatial resolution of the measured distance.

[0007] The present invention has been made in consideration of the above circumstances, and aims to provide a distance image capturing device and a distance image capturing method that prevent a situation in which charge is accumulated in only one of two charge storage sections, and that reduces the error between the distance obtained from the ratio of the charge amounts in the charge storage sections and the actual distance between the pixel and the subject. [Means for solving the problem]

[0008] In order to solve the above-mentioned problems, the distance image pickup device of the present invention comprises: a light source unit that irradiates a measurement space with a light pulse; a photoelectric conversion element that generates an electric charge according to the light incident from the measurement space; a light receiving unit having a plurality of pixel circuits each including a plurality of charge accumulation units that accumulate the electric charge in a frame period; and a pixel drive circuit that allocates and accumulates the electric charge generated in the photoelectric conversion element in each of the charge accumulation units via each transfer transistor at a predetermined accumulation timing synchronized with the irradiation of the light pulse; and a distance calculation unit that calculates the distance between the subject and the light receiving unit from the respective charge ratios of the amounts of electric charge that are generated by reflected light in the measurement space, allocated from the photoelectric conversion element by the transfer transistors, and accumulated in each of the charge accumulation units, and the pulse width of the light pulse is set longer than the period between the rise of a specific transfer transistor and the rise of the next transfer transistor among the transfer transistors. the pulse width is set by the variation in the rate of change of the charge ratio during a delay time from when the light pulse is irradiated until when the reflected light is incident, the delay time corresponding to a measurable distance, It is characterized by the fact that

[0009] The distance image capturing device of the present invention comprises: The distance calculation unit increases the pulse width by adding a unit width time, calculates the variation in the rate of change of the charge ratio corresponding to each of the inverse delay times after the light pulse having the increased pulse width is irradiated, repeatedly increases the pulse width and calculates the variation corresponding to the increased pulse width until the pulse width of the light pulse exceeds a predetermined width value, and sets the pulse width at which the variation is minimum as the pulse width of the light pulse. It is characterized by:

[0010] The distance image capturing device of the present invention is characterized in that while sequentially changing the pulse width, the variation in the rate of change of the charge ratio at each change in the delay time is calculated for each pulse width, the pulse width at which the variation is minimum is extracted, and the pulse width is set as the pulse width of the light pulse.

[0011] The distance image capturing device of the present invention is characterized in that the distance calculation unit calculates the distance based on table information indicating the relationship between a corresponding distance corresponding to the distance between the subject and the pixel and a charge ratio calculated from each of the amounts of charge accumulated in the charge accumulation units.

[0012] The distance image capturing device of the present invention is characterized in that the distance calculation unit pre-calculates a polynomial that approximates the distance deviation from a known distance, which is the distance calculated using the charge ratio, and calculates the distance with the distance deviation corrected using the polynomial.

[0013] A distance image capturing method of the present invention is a distance image capturing method for controlling a distance image capturing device comprising a plurality of pixel circuits each consisting of a light source unit, a photoelectric conversion element, a plurality of charge accumulation units, and transfer transistors, a pixel drive circuit, and a distance calculation unit, the method comprising the steps of: the pixel drive circuit accumulating charges generated by the photoelectric conversion elements in response to incident light from a measurement space in each of the charge accumulation units in a frame period via each of the transfer transistors that transfer the charges from the photoelectric conversion elements to the charge accumulation units, at a predetermined accumulation period synchronized with the irradiation of a light pulse from the light source unit; and the distance calculation unit calculating the distance between a subject and the distance image capturing device from a charge ratio of the amounts of charge generated by reflected light in the measurement space, distributed from the photoelectric conversion elements by the transfer transistors, and accumulated in each of the charge accumulation units, wherein the pulse width of the light pulse is set longer than the period between the rising edge of a specific transfer transistor and the rising edge of the next transfer transistor among the transfer transistors. the pulse width is set by the variation in the rate of change of the charge ratio during a delay time from when the light pulse is irradiated until when the reflected light is incident, the delay time corresponding to a measurable distance, It is characterized by the fact that [Effects of the Invention]

[0014] As described above, the present invention can provide a distance image capturing device and a distance image capturing method that prevent a situation in which the waveform of an optical pulse becomes dull and the pulse width becomes short, causing charge to be accumulated in only one of the two charge accumulation sections, thereby reducing the error between the distance obtained from the ratio of the charge amounts in the charge accumulation sections and the actual distance between the pixel and the subject. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a block diagram showing a schematic configuration of a distance imaging device according to a first embodiment of the present invention. [Figure 2] 10 is a diagram showing an example of the pulse shape of a light pulse PO emitted from a light source device 21. FIG. [Figure 3]1 is a circuit diagram showing an example of the configuration of a pixel circuit 321 arranged in a range image sensor 32 in the range image pickup device according to the first embodiment of the present invention. FIG. [Figure 4] 10 is a timing chart showing the transfer of charges generated by the photoelectric conversion element PD to each of the charge accumulation sections CS. FIG. [Figure 5] 10 is a diagram showing the correspondence relationship between the charge ratio of each charge storage section CS and the distance between the distance image pickup device 1 and the subject S. FIG. [Figure 6] FIG. 10 is a diagram showing the charge ratio slope SL in the case where the pulse width is the same as the width of the accumulation drive signal TX and the pulse width is wider than the width of the accumulation drive signal TX. [Figure 7] 10 is a graph showing the correspondence relationship between the charge ratio R and the delay time TD in this embodiment. [Figure 8] 1 is a diagram showing the resolution of the distance to the subject measured by the distance image pickup device 1. FIG. [Figure 9] 10 is a flowchart showing the operation of a process for extracting the pulse width of a light pulse PO used in distance measurement in a pulse width setting mode according to the present embodiment. [Figure 10] FIG. 10 is a diagram illustrating a correspondence table showing the relationship between the charge ratio R and the estimated distance in the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0016] First Embodiment A first embodiment of the present invention will be described below with reference to the drawings. Figure 1 is a block diagram showing the schematic configuration of a distance image pickup device according to a first embodiment of the present invention. The distance image pickup device 1 shown in Figure 1 comprises a light source unit 2, a light receiving unit 3, and a distance image processing unit 4. Figure 1 also shows a subject S, which is an object whose distance is to be measured by the distance image pickup device 1. The distance image pickup element is, for example, a distance image sensor 32 (described below) in the light receiving unit 3.

[0017] The light source section 2 irradiates a light pulse PO into a space to be photographed, in which a subject S, the distance of which is to be measured by the distance image pickup device 1, is present, under the control of the distance image processor 4. The light source section 2 is, for example, a surface-emitting semiconductor laser module such as a vertical cavity surface-emitting laser (VCSEL). The light source section 2 includes a light source device 21 and a diffuser plate 22.

[0018] The light source device 21 is a light source that emits laser light in a near-infrared wavelength band (for example, a wavelength band of 850 nm to 940 nm) that becomes the light pulse PO to be irradiated onto the subject S. The light source device 21 is, for example, a semiconductor laser light-emitting element. The light source device 21 emits pulsed laser light in response to control from the timing control unit 41. The diffusion plate 22 is an optical component that diffuses the laser light in the near-infrared wavelength band emitted by the light source device 21 to the extent of the surface to be irradiated onto the subject S. The pulsed laser light diffused by the diffusion plate 22 is emitted as a light pulse PO and is irradiated onto the subject S.

[0019] The light receiving unit 3 receives reflected light RL of the light pulse PO reflected by a subject S, the distance of which is to be measured in the range image pickup device 1, and outputs a pixel signal corresponding to the received reflected light RL. The light receiving unit 3 includes a lens 31 and a range image sensor 32. The lens 31 is an optical lens that guides the incident reflected light RL to the range image sensor 32. The lens 31 outputs the incident reflected light RL to the range image sensor 32 side, and causes the light to be received (incident) by pixel circuits provided in the light receiving region of the range image sensor 32.

[0020] The range image sensor 32 is an imaging element used in the range image capturing device 1. The range image sensor 32 includes a plurality of pixel circuits 321 in a two-dimensional light receiving area, and a pixel drive circuit 322 that controls each of the pixel circuits 321. The pixel circuit 321 includes one photoelectric conversion element (for example, a photoelectric conversion element PD described later), a plurality of charge accumulation units (for example, charge accumulation units CS1 to CS4 described later) corresponding to the one photoelectric conversion element, and components that distribute charge to each charge accumulation unit.

[0021] The range image sensor 32 distributes the charges generated by the photoelectric conversion elements to the respective charge accumulation sections under the control of the timing control section 41. The range image sensor 32 also outputs pixel signals according to the amount of charge distributed to the charge accumulation sections. The range image sensor 32 has a plurality of pixel circuits arranged in a two-dimensional matrix, and outputs pixel signals for one frame corresponding to each pixel circuit.

[0022] The distance image processing unit 4 controls the distance image pickup device 1 and calculates the distance to the subject S. Distance image processing unit 4 includes timing control unit 41, pulse width adjustment unit 42, distance calculation unit 43, and measurement control unit 44. The timing control unit 41 controls the timing of outputting various control signals required for distance measurement in accordance with the control of the measurement control unit 44. The various control signals here include, for example, a signal for controlling the irradiation of the light pulse PO, a signal for distributing the reflected light RL to multiple charge accumulation units (an accumulation drive signal TX for operating a transfer transistor G described later), and a signal for controlling the number of distributions per frame. The number of distributions refers to the number of times the process of distributing the charges generated by the incident light from the photoelectric conversion element PD to the charge accumulation units CS (see FIG. 5) via the transfer transistor G is repeated.

[0023] The pulse width adjusting section 42 adjusts the pulse width of the light pulse PO emitted from the light source device 21 of the light source section 2. That is, the light source device 21 causes the light source unit 2 to emit a light pulse PO in synchronization with an accumulation drive signal TX (described later) that drives a transfer transistor G that distributes charges to the charge accumulation unit CS. At this time, the pulse width adjustment unit 42 causes the light source device 21 to irradiate the light pulse PO with a pulse width TPP (>TP1) that is not the same as the pulse width TP1 of the storage drive signal TX (the distribution time during which the transfer transistor G is turned on and charge is distributed to the charge storage unit CS, i.e., the on period during which the transfer transistor G is turned on to distribute the charge), but is adjusted in accordance with the degree of dullness of the light pulse PO (details will be described later). The pulse width of the optical pulse described here is assumed to be a pulse width set value by a register. Alternatively, the control signal may be defined by the rising timing and rising period of the optical pulse, or the period from the rising timing to the falling timing. The pulse width of the optical pulse may also be defined as the period of 10% or more intensity, or the period of 90% or more intensity, with the output peak of the optical pulse being 100%. That is, in this embodiment, in each of the transfer transistors G, the pulse width of the optical pulse PO is set longer than the rising period (period during which the signal is at H level) of a specific transfer transistor G (which rises first to H level) and the rising period of the next transfer transistor G.

[0024] The distance calculation unit 43, under the control of the measurement control unit 44, outputs distance information (quantized as gradation) calculated based on the pixel signal (voltage value corresponding to the accumulated charge amount Q described later) output from the distance image sensor 32, which indicates the distance from the distance image capturing device 1 (each pixel) to the subject S. The distance calculation unit 43 calculates a delay time Td from when the light pulse PO is emitted until when the reflected light RL is received based on the amount of charge accumulated in the plurality of charge accumulation units CS. The distance calculation unit 43 calculates the distance from the range image pickup device 1 to the subject S in accordance with the calculated delay time Td.

[0025] The measurement control unit 44 selects the mode of each frame repeated at a frame period between a pulse width setting mode in which the pulse width TPP of the optical pulse PO is adjusted and determined by arbitrarily changing the pulse width, and a ranging charge amount acquisition mode in which a normal frame is used to perform ranging using an optical pulse PO of the set pulse width. The measurement control unit 44 controls the timing in the timing control unit 41 and the calculation in the distance calculation unit 43 in accordance with the pulse width setting mode and the distance measurement charge amount acquisition mode (described in detail later).

[0026] 2A and 2B are diagrams showing an example of the pulse shape of the light pulse PO emitted from the light source device 21. FIG. 2A shows, for example, the waveform of the light pulse PO having the same pulse width TPP (=TP1) as the accumulation drive signal TX that drives the transfer transistor G. In FIG. 2A, the horizontal axis represents time, and the vertical axis represents intensity (voltage measured by the photodetector). The light pulse PO is ideally emitted with a waveform identical to the rectangular waveform of the accumulation drive signal TX indicated by the dotted line K1. However, in reality, the rising and falling edges of the waveform of the optical pulse PO become dull (waveform dullness occurs due to the time constant of irradiation), and the pulse width TPP of the optical pulse PO actually becomes narrower than the width of the accumulation drive signal TX, which is a set value.

[0027] As a result, as described in the problem section, charges are distributed from the photoelectric conversion element PD to each of the charge storage units CS, but the pulse width TPP of the light pulse PO becomes shorter than the pulse width TP1 of the storage drive signal TX due to the waveform becoming duller. When the distance is calculated based on the ratio of the amount of accumulated charge Q stored in the two charge storage units CS, there may be a period of time during which the charge is allocated to only one of the two charge storage units CS, or the amount of charge allocated to one of the two charge storage units CS may be so small that it may be susceptible to noise. For this reason, the distance calculated from the ratio of the amounts of charge stored in the two charge storage units CS (charge ratio R, which will be described later) may differ from the actual distance.

[0028] Figure 2(b) shows the waveform of an optical pulse PO with a pulse width set wider than the width of the accumulation drive signal TX. In Figure 2(b), the horizontal axis represents time, and the vertical axis represents intensity (for example, the voltage measured from the optical pulse waveform using a photodetector, etc.). This shows that even if the waveform of the optical pulse PO becomes dull, the pulse width TPP of the optical pulse PO can maintain the same pulse width TP1 as the accumulation drive signal TX indicated by the dotted line K1.

[0029] Therefore, when calculating the distance based on the ratio of the amount of accumulated charge Q accumulated in the two charge accumulation units CS, it is possible to prevent a state in which the charge is allocated to only one of the two charge accumulation units CS for a specified time period, or a state in which the charge allocated to one of the two charge accumulation units CS is very small (i.e., the charge generated in the photoelectric conversion element PD by reflected light is accumulated in a highly biased manner in one of the charge accumulation units CS). Therefore, the distance calculated from the ratio of the amounts of charge accumulated in the two charge accumulation units CS is calculated as a value close to the actual distance between the distance image sensor 32 and the subject S.

[0030] With this configuration, in the distance image capturing device 1, the light source unit 2 irradiates a light pulse PO in the near-infrared wavelength band onto the subject S, and the light receiving unit 3 receives the reflected light RL reflected by the subject S (receives the reflected light RL as incident light mixed with background light), and the distance image processing unit 4 outputs distance information measuring the distance between the subject S and the distance image capturing device 1. Although FIG. 1 shows the distance image pickup device 1 having the distance image processing unit 4 built therein, the distance image processing unit 4 may be an element provided outside the distance image pickup device 1.

[0031] Here, the configuration of the pixel circuit 321 in the range image sensor 32 will be described. Fig. 3 is a circuit diagram showing an example of the configuration of the pixel circuit 321 arranged in the range image sensor 32 in the range image pickup device according to the first embodiment of the present invention. The pixel circuit 321 in Fig. 3 is an example configuration including four pixel signal readout units RU1 to RU4.

[0032] The pixel circuit 321 includes one photoelectric conversion element PD, a charge discharging transistor GD (GD1 and GD2 described below), and four pixel signal readout units RU (RU1 to RU4) that output voltage signals from corresponding output terminals O. Each pixel signal readout unit RU includes a transfer transistor G, a floating diffusion FD, a charge storage capacitance C, a reset transistor RT, a source follower transistor SF, and a selection transistor SL. The floating diffusion FD and the charge storage capacitance C form a charge storage unit CS.

[0033] 3, the pixel signal readout unit RU1, which outputs a voltage signal from the output terminal O1, includes a transfer transistor G1 (transfer MOS transistor), a floating diffusion FD1, a charge storage capacitance C1, a reset transistor RT1, a source follower transistor SF1, and a selection transistor SL1. In the pixel signal readout unit RU1, the floating diffusion FD1 and the charge storage capacitance C1 form a charge storage unit CS1. The pixel signal readout units RU2, RU3, and RU4 have a similar configuration.

[0034] The photoelectric conversion element PD is a buried photodiode that photoelectrically converts incident light, generates charges corresponding to the incident light, and accumulates the generated charges. In this embodiment, the incident light is incident from the space to be measured. In pixel circuit 321, the photoelectric conversion element PD photoelectrically converts incident light to generate electric charges, which are then distributed to each of four charge accumulation units CS (CS1 to CS4), and voltage signals corresponding to the amount of distributed electric charges are output to distance image processing unit 4.

[0035] Furthermore, the configuration of the pixel circuit arranged in the distance image sensor 32 is not limited to the configuration having four pixel signal readout units RU (RU1 to RU4) as shown in Figure 3, but may be a pixel circuit having a configuration having one or more pixel signal readout units RU.

[0036] In driving the pixel circuit 321 of the range image pickup device 1, a light pulse PO is emitted for an irradiation time To, and reflected light RL is received by the range image sensor 32 after a delay time Td. The pixel drive circuit 322 synchronizes with the irradiation of the light pulse PO under the control of the timing control unit 41, and supplies accumulation drive signals TX1 to TX4 to transfer transistors G1, G2, G3, and G4 at their respective timings to redirect the charges generated in the photoelectric conversion element PD, causing them to be accumulated in the charge accumulation units CS1, CS2, CS3, and CS4 in that order.

[0037] The pixel driving circuit 322 controls the reset transistor RT and the selection transistor SL by driving signals RST and SEL, respectively, converts the charges accumulated in the charge storage unit CS into an electrical signal by the source follower transistor SF, and outputs the generated electrical signal to the distance calculation unit 43 via terminal O. Furthermore, under the control of the timing control section 41, the pixel driving circuit 322 causes the charge generated in the photoelectric conversion element PD to flow to the power supply VDD in response to the driving signal RSTD, thereby discharging the charge (erasing the charge).

[0038] FIG. 4 is a timing chart showing the transfer of the charges generated by the photoelectric conversion elements PD to the charge accumulation sections CS. 4, the vertical axis represents the pulse level and the horizontal axis represents time. The timing chart shows the relative relationship on the time axis between the optical pulse PO and the reflected light RL, the timing of each of the accumulation drive signals TX1 to TX4 supplied to the transfer transistors G1 to G4, and the timing of the drive signal RSTD supplied to the charge discharge transistor GD.

[0039] The timing control unit 41 causes the light source unit 2 to irradiate the measurement space with a light pulse PO. As a result, the light pulse PO is reflected by the subject and received by the light receiving unit 3 as reflected light RL. The photoelectric conversion element PD then generates charges corresponding to the background light and the reflected light RL. The pixel drive circuit 322 controls the on / off of each of the transfer transistors G1 to G4 to transfer the charges generated by the photoelectric conversion element PD to each of the charge accumulation units CS1 to CS4.

[0040] That is, the pixel drive circuit 322 supplies each of the accumulation drive signals TX1 to TX4 as an "H" level signal for a predetermined time duration (the same duration as the irradiation time To) to the transfer transistors G1 to G4, respectively.

[0041] The pixel driving circuit 322, for example, turns on a transfer transistor G1 provided on a transfer path that transfers charges from the photoelectric conversion element PD to the charge accumulation unit CS1. As a result, charges photoelectrically converted by the photoelectric conversion element PD are accumulated in the charge accumulation unit CS1 via the transfer transistor G1. Thereafter, the pixel driving circuit 322 turns off the transfer transistor G1. This stops the transfer of charges to the charge accumulation unit CS1. In this way, the pixel driving circuit 322 accumulates charges in the charge accumulation unit CS1. The same applies to the other charge accumulation units CS2, CS3, and CS4.

[0042] At this time, during the charge accumulation period in which charge is distributed to the charge accumulation sections CS (the period in which charge is accumulated in each of the charge accumulation sections CS in the frame), an accumulation cycle (corresponding to the pulse width TP1 of the accumulation drive signal TX that drives the transfer transistors G) in which each of the accumulation drive signals TX1, TX2, TX3, and TX4 is supplied to the transfer transistors G1, G2, G3, and G4, respectively, is repeated. Then, charges corresponding to incident light are transferred from the photoelectric conversion element PD to the charge accumulation units CS1, CS2, CS3, and CS4, respectively, via the transfer transistors G1, G2, G3, and G4. A plurality of accumulation cycles are repeated during the charge accumulation period. As a result, charges are accumulated in the charge accumulation units CS1, CS2, CS3, and CS4, respectively, for each accumulation cycle of the charge accumulation units CS1, CS2, CS3, and CS4 during the charge accumulation period.

[0043] Furthermore, when repeating the accumulation cycles of the charge accumulation units CS1, CS2, CS3, and CS4, after the transfer (relocation) of charges to the charge accumulation unit CS4 is completed, the pixel driving circuit 322 supplies an "H" level driving signal RSTD to the charge discharging transistor GD provided on the discharge path that discharges charges from the photoelectric conversion element PD, to turn it on.

[0044] As a result, before the accumulation cycle for the charge accumulation unit CS1 starts, the charge drain transistor GD discards the charge generated in the photoelectric conversion element PD after the accumulation cycle for the immediately preceding charge accumulation unit CS4 (i.e., resets the photoelectric conversion element PD). That is, one or more charge drain transistors GD are provided, and drain charge from the photoelectric conversion element PD except for the period during which the charge generated from the photoelectric conversion element PD by incident light is distributed and accumulated in each of the charge accumulation units CS1, CS2, CS3, and CS4.

[0045] The pixel driving circuit 322 then sequentially performs signal processing such as A / D conversion on the voltage signals from all of the pixel circuits 321 arranged in the light receiving section 3 in units of rows (horizontal arrangement) of the pixel circuits 321. Thereafter, the pixel driving circuit 322 outputs the processed voltage signals to the distance calculation unit 43 in the order of the columns arranged in the light receiving unit 3.

[0046] As described above, the pixel driving circuit 322 accumulates charges in the charge accumulation units CS and discards the charges photoelectrically converted by the photoelectric conversion elements PD over one frame. As a result, charges corresponding to the amount of light received by the distance image pickup device 1 over a predetermined time period are accumulated in each charge accumulation unit CS. The pixel driving circuit 322 outputs an electrical signal corresponding to the amount of charge accumulated in each charge accumulation unit CS for one frame to the distance calculation unit 43.

[0047] Due to the relationship between the timing of irradiating the light pulse PO and the timing of accumulating charges in each of the charge accumulation units CS (CS1 to CS4), the charge accumulation unit CS1 holds an amount of charge corresponding to external light components such as background light before irradiating the light pulse PO. Furthermore, the charge accumulation units CS2, CS3, and CS4 hold charges corresponding to the reflected light RL and external light components, respectively. The distribution (allocation ratio) of the amount of charge allocated to the charge accumulation units CS2 and CS3, or the charge accumulation units CS3 and CS4, is a ratio that corresponds to the delay time Td between when the light pulse PO is reflected by the subject S and when it enters the range image pickup device 1.

[0048] As described above, in this embodiment, the distance between the distance image capturing device 1 and the subject S is determined by the ratio of the amount of charge stored in a combination of two charge storage units CS, for example, the charge storage units CS1 and CS2 and the charge storage units CS2 and CS3, among the charge storage units CS1, CS2, and CS3. The ratio of the amount of charge stored in the combination of the two charge storage units CS changes linearly with the distance (changes at a predetermined rate), allowing the distance to be measured with high accuracy. However, when the waveform of the light pulse PO is dulled and deformed, the relationship between the charge ratio, which is the ratio of the charge amounts, and the distance is no longer linear.

[0049] FIG. 5 is a diagram showing the correspondence relationship between the charge ratio of each charge storage section CS and the distance between the distance image pickup device 1 and the subject S. In FIG. In FIG. 5, the dashed line indicates a linear relationship, and the solid line indicates the relationship when the pulse width TPP of the optical pulse PO is the same as that of the storage drive signal TX. The solid line deviates from the dashed line, which shows a linear relationship between the charge ratio and distance near 2 m and 4 m. The charge ratio R used here is set as follows based on the amounts of accumulated charge Q1, Q2, Q3, and Q4 accumulated in the charge accumulation units CS1, CS2, CS3, and CS4, respectively, and supplied from the distance image sensor 32. The following calculation is performed by the pulse width adjustment unit 42.

[0050] R1=1-Q 1-3 / Q A …(1) R2=2-Q 2-4 / Q A …(2) In the above formulas (1) and (2), Q 1-3 =|Q1-Q3| …(3) Q 2-4 =|Q2-Q4| …(4) Q A =|Q1-Q3|+|Q2-Q4| …(5) is.

[0051] The charge ratio R1 indicates the ratio of charges in a state where the charge of the reflected light RL is accumulated in the combination of the charge accumulation units CS1 and CS2 (time window Tw1 in FIG. 4). Further, the charge ratio R2 indicates the ratio of charges in a state where the charge of the reflected light RL is accumulated in the combination of the charge accumulation units CS2 and CS3 (time window Tw2 in FIG. 4). The area around a distance of 2 m in FIG. 5 corresponds to the range where the time window switches from Tw1 to Tw2. When the time window is switched, the pulse width TPP of the actually irradiated light pulse PO, which has already been described, becomes narrower than the pulse width TP1 of the accumulation drive signal TX, causing an error with respect to the actual distance.

[0052] Therefore, by determining the pulse width TPP of the optical pulse PO, in which the charge ratios R1 and R2 change linearly with the distance to be measured, and using this for measurement, it is possible to improve the measurement accuracy. Here, when the charge ratio R1 exceeds a preset threshold ThW, the charge ratio R is determined by switching to the charge ratio R2. This threshold ThW is set as the upper limit of the charge ratio R1 in the time window Tw1 near the timing when the time window Tw1 switches to the time window Tw2, i.e., the charge ratio R1 corresponding to the distance that can be measured in the time window Tw1. For this reason, in this embodiment, the pulse width TPP (≧TP1) of the light pulse PO at which the charge ratio becomes linear is found by the following process.

[0053] In the pulse width setting mode, the measurement control unit 44 arbitrarily sets the pulse width TPP, increases it by a predetermined unit delay time ΔTD from the timing synchronized with the accumulation drive signal TX1, and causes the light source device 21 to irradiate the light pulse PO. At this time, the subject S is fixed at a predetermined position on the distance image pickup device 1. The distance between the distance image pickup device 1 and the subject S is virtually changed by irradiating the light pulse PO with a delay time TD shifted from the rising timing of the accumulation drive signal TX1.

[0054] Then, the pulse width adjustment unit 42 extracts the acquired charge ratio R for each predetermined delay time TD, and calculates the charge ratio slope SL (=ΔR / ΔTD) from the difference ΔR between the charge ratios R for adjacent delay times TD and the unit delay time ΔTD. Furthermore, after determining the charge ratio slope SL for all delay times TD in the pulse width TPP of the optical pulse PO, the pulse width adjustment unit 42 determines the standard deviation (i.e., the variation in the rate of change) of the charge ratio slope SL in this pulse width TPP. In this embodiment, the standard deviation is used to determine the variation in the rate of change, but any index for evaluating the variation, such as variance, deviation value, or mean square error, can be used as the index for evaluating the charge ratio slope SL in the pulse width TPP.

[0055] The pulse width adjusting unit 42 performs the above-described process of finding the standard deviation of the pulse width TPP of the optical pulse PO for all the set pulse widths TPP of the optical pulse PO, and extracts the minimum standard deviation. As a result, the pulse width adjusting unit 42 obtains the pulse width of the light pulse PO corresponding to the charge ratio slope SL with the smallest standard deviation, and sets it as the pulse width TPP to be used.

[0056] Then, in the ranging charge amount acquisition mode, the distance calculation unit 43 calculates the distance between each pixel circuit 321 and the subject S based on the delay time Td obtained by the following equation (6) or (7) using the ratio R1 or ratio R2 calculated from each of equations (1) to (5) from the accumulated charge amounts Q1, Q2, Q3, and Q4 of the charge accumulation units CS1, CS2, CS3, and CS4 in the pixel circuit 321.

[0057] That is, the distance calculation unit 43 uses the above-mentioned principle to calculate the delay time Td using the following equation (6) or (7). Td = To × R1 …(6) Td = To × R2 …(7) Here, To is the period during which the light pulse PO is irradiated. R1 is the ratio of the amount of accumulated charge accumulated in the charge accumulation unit CS1 to the amount of accumulated charge accumulated in the charge accumulation unit CS2 during the time window Tw1 in FIG. 4, which will be described later. R2 is the ratio of the amount of accumulated charge accumulated in the charge accumulation unit CS2 to the amount of accumulated charge accumulated in the charge accumulation unit CS3 during the time window Tw2.

[0058] In equation (6), charges generated by reflected light are accumulated in the charge accumulation units CS1 and CS2, but not in the charge accumulation units CS3 and CS4. On the other hand, in equation (7), charges generated by reflected light are accumulated in the charge accumulation units CS2 and CS3, but not in the charge accumulation units CS1 and CS4. In addition, equations (6) and (7) are based on the assumption that the amount of charge stored in the charge storage units CS1, CS2, and CS3 that corresponds to the external light component is the same as the amount of charge stored in the charge storage unit CS4.

[0059] The distance calculation unit 43 calculates the round trip distance to the subject S by multiplying the delay time Td obtained by equation (6) or (7) by the speed of light (velocity). Then, the distance calculation unit 43 calculates the distance from the distance image sensor 32 (i.e., the distance image pickup device 1) to the subject S by dividing the calculated round trip distance by 2 (delay time Td×c (speed of light) / 2).

[0060] FIG. 6 is a diagram showing the charge ratio slope SL in the cases of a pulse width TPP (=TP1) that is the same as the pulse width TP1 of the accumulation drive signal TX and a pulse width TPP (>TP1) that is wider than the width of the accumulation drive signal TX. 6, the horizontal axis represents delay time TD, and the vertical axis represents charge ratio slope SL. The dashed line represents the charge ratio slope SL when an optical pulse PO with a pulse width TPP (=TP1) that is the same as the pulse width TP1 of the accumulation drive signal TX is used, and the solid line represents the charge ratio slope SL when an optical pulse PO with a pulse width TPP (>TP1) that is longer than the width of the accumulation drive signal TX is used. 6 shows that the charge ratio slope SL (solid line) of the light pulse PO having a pulse width TPP longer than the width of the accumulation drive signal TX has less variation than the pulse width TPP (dashed line) of the accumulation drive signal TX having the same pulse width TP1.

[0061] 7A and 7B are graphs showing the correspondence relationship between the charge ratio R and the delay time TD in this embodiment. Fig. 7A shows the correspondence relationship between the delay time TD and the charge ratio R when an optical pulse PO having the same pulse width as the pulse width TP1 of the storage drive signal TX is used. In Fig. 7A, the vertical axis represents the charge ratio R, and the horizontal axis represents the delay time TD. The dashed line indicates an ideal linear line segment between the delay time TD and the charge ratio R, and the dots indicate the values ​​of the charge ratio R corresponding to the actually measured delay time TD.

[0062] As can be seen from FIG. 7(a), the measured circle points deviate from the linear relationship (dashed line) between the delay time TD and the charge ratio R near the delay time (near 20 ns) when the time window switches from Tw1 to Tw2. Since the delay time corresponds to the measured distance, it can be seen that if an optical pulse PO with a pulse width TPP identical to the width of the accumulation drive signal TX is used, a value that deviates from the actual distance will be obtained.

[0063] 7(b) shows the correspondence relationship between delay time TD and charge ratio R when an optical pulse PO with a pulse width TPP determined from the standard deviation of the charge ratio slope SL is used, rather than an optical pulse PO with the same pulse width TPP as the pulse width TP1 of the storage drive signal TX. In FIG. 7(b), the vertical axis represents the charge ratio R, and the horizontal axis represents the delay time TD. The dashed line indicates an ideal linear line segment between the delay time TD and the charge ratio R, and the dots indicate the values ​​of the charge ratio R corresponding to the actually measured delay time TD.

[0064] As can be seen from Figure 7(b), the measured circle points deviate from the linear relationship (dashed line) between the delay time TD and the charge ratio R near the delay time (near 20 ns) when time window Tw1 switches to time window Tw2, just like Figure 7(b). However, compared to Figure 7(a) in which an optical pulse PO with a pulse width TPP (=TP1) identical to the pulse width TP1 of the storage drive signal TX is used, when an optical pulse PO with a pulse width TPP determined from the standard deviation of the charge ratio slope SL is used, the distance that deviates from the linear relationship (dashed line) between the delay time TD and the charge ratio R is significantly improved.

[0065] As a result, by using an optical pulse PO with a pulse width TPP determined from the standard deviation of the charge ratio slope SL, a numerical value closer to the actual distance can be obtained compared to when an optical pulse PO with the same pulse width TPP as the pulse width TP1 of the accumulation drive signal TX is used, thereby improving the accuracy of distance measurement control.

[0066] FIG. 8 is a diagram showing the resolution of the distance to the subject measured by the distance image pickup device 1. As shown in FIG. Fig. 8(a) shows the time resolution in measuring the distance to the subject measured by the range image pickup device 1. In Fig. 8(a), the vertical axis represents the time resolution, and the horizontal axis represents the actual distance (actual distance) between the range image pickup device 1 and the subject S. The broken line indicates the relationship between the time resolution and the actual distance when using an optical pulse PO with the same pulse width TPP as the pulse width TP1 of the storage drive signal TX.

[0067] On the other hand, the solid line shows the relationship between the time resolution and the actual distance when an optical pulse PO with a pulse width TPP is used based on the standard deviation of the charge ratio slope SL. As can be seen from FIG. 8(a), according to this embodiment, by measuring the distance using an optical pulse PO with a pulse width TPP determined from the standard deviation of the charge ratio slope SL, the time resolution of the measured distance can be improved compared to when an optical pulse PO with a pulse width TPP identical to the width of the accumulation drive signal TX is used.

[0068] Fig. 8(b) shows the spatial resolution in measuring the distance to the subject measured by the range image pickup device 1. In Fig. 8(b), the vertical axis represents the spatial resolution, and the horizontal axis represents the actual distance (actual distance) between the range image pickup device 1 and the subject S. The broken line indicates the relationship between the distance resolution and the actual distance when using an optical pulse PO with the same pulse width TPP as the pulse width TP1 of the storage drive signal TX.

[0069] On the other hand, the solid line shows the relationship between the distance resolution and the actual distance when using an optical pulse PO with a pulse width TPP used from the standard deviation of the charge ratio slope SL. As can be seen from FIG. 8(b), according to this embodiment, by measuring the distance using an optical pulse PO with a pulse width TPP determined from the standard deviation of the charge ratio slope SL, the spatial resolution of the measured distance can be improved compared to when an optical pulse PO with a pulse width TPP identical to the pulse width TP1 of the accumulation drive signal TX is used.

[0070] FIG. 9 is a flowchart showing the operation of the process of extracting the pulse width TPP of the light pulse PO used in distance measurement in the pulse width setting mode according to this embodiment. Step S101: The measurement control section 44 controls the timing control section 41 to start the execution of the pulse width setting mode.

[0071] Step S102: Then, the pulse width adjusting unit 42 sets the pulse width of the light pulse PO to the same value as the width (pulse width TP1) of the storage drive signal TX, and sets this in the timing control unit 41 (initialization of the pulse width TPP of the light pulse PO).

[0072] Step S103: The pulse width adjusting unit 42 determines whether or not the pulse width TPP of the optical pulse PO exceeds a preset upper limit value (pulse width specified value). At this time, if the pulse width TPP of the optical pulse PO does not exceed (is equal to or less than) the pulse width specified value, which is a preset upper limit value, the pulse width adjusting unit 42 advances the process to step S104. On the other hand, if the pulse width TPP of the optical pulse PO exceeds the predetermined pulse width specified value, the pulse width adjusting unit 42 advances the process to step S112.

[0073] Step S104: The pulse width adjustment unit 42 controls the timing control unit 41 so that the timing of irradiating the light pulse PO is synchronized with the rising edge of the accumulation drive signal TX1 (initializing the delay time TD of the light pulse PO, i.e., setting the delay time TD to "0").

[0074] Step S105: The pulse width adjustment unit 42 determines whether or not the delay time TD from the rising edge of the accumulation drive signal TX1 to the timing of irradiation of the light pulse PO exceeds a preset upper limit value (delay specification value). At this time, if the pulse width of the optical pulse PO does not exceed (is equal to or less than) the predetermined delay value, the process proceeds to step S106. On the other hand, if the pulse width TPP (≧TP1) of the optical pulse PO exceeds the delay specified value, which is a preset upper limit value, the pulse width adjusting unit 42 advances the process to step S109.

[0075] Step S106: During the accumulation period in the frame period, the light source device 21 irradiates the light pulse PO with a delay of the delay time TD from the rising edge of the accumulation drive signal TX1, with a set pulse width TPP, for each accumulation cycle. Then, for each accumulation period, the pixel driving circuit 322 allocates and accumulates the charges generated by the incident light in the photoelectric conversion element PD in the charge accumulation units CS1, CS2, CS3, and CS4, respectively, using the transfer transistors G1, G2, G3, and G4.

[0076] Here, the accumulation drive signals TX1, TX2, TX3, and TX4 for the transfer transistors G1, G2, G3, and G4, respectively, have the same pulse width TP1. The distribution of the electric charges to each of the charge storage sections CS1, CS2, CS3, and CS4 is performed for a preset number of accumulation periods (number of distributions).

[0077] Step S107: Pixel drive circuit 322 supplies accumulated charge amounts Q1, Q2, Q3, and Q4 from charge accumulation units CS1, CS2, CS3, and CS4 in each pixel circuit 321 to distance image processing unit 4, respectively. The pulse width adjustment unit 42 calculates the charge ratio R for each pixel circuit 321 using equations (3) and (4) based on the amounts of accumulated charges Q1, Q2, Q3, and Q4 supplied from each pixel circuit 321.

[0078] Step S108: The pulse width adjusting unit 42 adds the unit delay time ΔTD to the delay time TD, and sets the result of the addition as a new delay time TD (change of the delay time TD). Then, the pulse width adjusting unit 42 controls the timing control unit 41 so that the timing of irradiating the light pulse PO is delayed by the delay time TD with respect to the rising edge of the accumulation drive signal TX1.

[0079] Step S109: The pulse width adjustment unit 42 calculates the difference ΔR between the charge ratio R between the delay time TD to be processed and the immediately preceding delay time TD for all of the measured delay times TD. Furthermore, the pulse width adjustment unit 42 divides the difference ΔR in each delay time TD obtained for each pixel circuit 321 (ie, pixel) by the unit delay time ΔTD to obtain the charge ratio slope SL for each delay time TD.

[0080] Step S110: Then, the pulse width adjusting unit 42 calculates, for each pixel circuit 321, the standard deviation s of the charge ratio slope SL at the pulse width TPP at this point in time from the charge ratio slope SL at each delay time TD. Furthermore, the pulse width adjustment unit 42 calculates the average value of the standard deviations s in all pixel circuits 321, and sets the average value as the standard deviation s_av.

[0081] Step S111: The pulse width adjusting unit 42 adds the unit width time ΔTPP to the pulse width TPP of the optical pulse PO, and sets the result of the addition as a new pulse width TPP (change of the pulse width TPP). Then, the pulse width adjusting section 42 controls the timing control section 41 so that the pulse width TPP of the irradiated light pulse PO is set to a pulse width TPP obtained by adding the newly set unit width time ΔTPP.

[0082] Step S112: The pulse width adjusting unit 42 extracts the pulse width TPP with the minimum standard deviation s_av from among the pulse widths TPP of the optical pulses PO.

[0083] Step S113: The pulse width adjustment unit 42 sets the extracted pulse width TPP in each pixel circuit 321 as the pulse width TPP that has the least variation (fluctuation) in the charge ratio slope SL and that maximizes the accuracy in measuring the distance between the distance image pickup device 1 and the subject S.

[0084] In the processes in steps S109 and S110 of the flowchart described above, the standard deviation s is found for all pixel circuits 321 in the range image sensor 32, and the standard deviation s_av is calculated as the average value of the standard deviations s. However, instead of the average value of the standard deviations s of all pixel circuits 321 in the distance image sensor 32, the average value of the standard deviations s of the pixel circuits 321 in a predetermined area, for example, an area in the central part of the distance image sensor 32, may be used as the standard deviation s_av. In this configuration, the pulse width adjustment unit 42 performs the process of calculating the standard deviation s only for each of the pixel circuits 321 set in the central area of ​​the range image sensor 32.

[0085] Alternatively, pixel circuits 321 may be set in advance at predetermined positions in the distance image sensor 32 that are spaced apart from one another, and the average value of the standard deviations s of the pixel circuits 321 may be used as the standard deviation s_av. In this configuration, the pulse width adjustment unit 42 performs a process of calculating the standard deviation s only for each of the pixel circuits 321 at a predetermined position set in the distance image sensor 32.

[0086] Also, in step S107, the pulse width adjustment unit 42 may be configured to calculate the average value of the charge ratios R obtained from all pixel circuits 321, use this average value as the charge ratio R of the distance image sensor 32, and calculate the difference ΔR between the charge ratios R at each delay time TD. In this configuration, in steps S109 and S110, the pulse width adjuster 42 calculates the standard deviation s of the charge slope SL calculated from each difference ΔR and the unit delay time ΔTD, and uses this as the standard deviation a_av.

[0087] Furthermore, in step S107, the pulse width adjustment unit 42 may be configured to calculate the difference ΔR in the charge ratio R for each delay time TD using the average value of the charge ratios R of the pixel circuits 321 in a predetermined region, for example, a region in the center of the distance image sensor 32, rather than calculating the average value of the charge ratios R calculated from all of the pixel circuits 321. In this configuration, in steps S109 and S110, the pulse width adjustment unit 42 calculates the standard deviation s of the charge non-gradient SL calculated from each of the differences ΔR calculated from the charge ratios R of the pixel circuits 321 in the central region and the unit delay time ΔTD, and uses this as the standard deviation a_av.

[0088] Furthermore, in step S107, the pulse width adjustment unit 42 may be configured to calculate the difference ΔR in the charge ratio R for each delay time TD using the average value of the charge ratios R of pixel circuits 321 at predetermined positions spaced apart from each other in a predetermined area, for example, in the distance image sensor 32, rather than calculating the average value of the charge ratios R calculated from all pixel circuits 321. In this configuration, in steps S109 and S110, the pulse width adjustment unit 42 calculates the standard deviation s of the charge non-gradient SL calculated from each of the differences ΔR calculated from the charge ratios R of the pixel circuits 321 at the above-mentioned predetermined separated positions and the unit delay time ΔTD, and uses this as the standard deviation s_av.

[0089] In addition, in this embodiment, the delay time Td is calculated from the amount of accumulated charge Q accumulated in each charge storage unit CS in the time window Tw1 or Tw2 using equations (1) to (7), and the distance between the distance image capturing device 1 and the subject S is calculated using this delay time Td. However, by utilizing the principle that the distribution (distribution ratio) of the amount of charge distributed to the charge storage units CS2 and CS3, or the charge storage units CS3 and CS4, is a ratio that corresponds to the delay time Td from when the light pulse PO is reflected by the subject S to when it is incident on the distance image pickup device 1, the distance calculation unit 43 may be configured to calculate the delay time Td using the following equation (8) or (9) and determine the distance between the distance image pickup device 1 and the subject S.

[0090] Td=To×(Q2-Q1) / (Q1+Q2-2×Q4) …(8) Td=To+To×(Q3-Q2) / (Q2+Q3-2×Q4) …(9) Here, To is the period during which the light pulse PO is irradiated. Q1 is the amount of charge accumulated in the charge accumulation unit CS1. Q2 is the amount of charge accumulated in the charge accumulation unit CS2. Q3 is the amount of charge accumulated in the charge accumulation unit CS3. Q4 is the amount of charge accumulated in the charge accumulation unit CS4.

[0091] In equation (8), charges generated by reflected light are accumulated in the charge accumulation units CS1 and CS2, but not in the charge accumulation units CS3 and CS4 (time window Tw1).On the other hand, in equation (9), charges generated by reflected light are accumulated in the charge accumulation units CS2 and CS3, but not in the charge accumulation units CS1 and CS4 (time window Tw2). In addition, equations (8) and (9) are based on the premise that the amount of charge stored in the charge storage units CS1, CS2, and CS3 that corresponds to the external light component is the same as the amount of charge stored in the charge storage unit CS4.

[0092] <Second embodiment> A second embodiment of the present invention will be described below with reference to the drawings. The second embodiment has the same configuration as the range image pickup device 1 in the first embodiment shown in FIG. The following describes the operation of the range image pickup device 1 according to the second embodiment, which differs from that of the first embodiment. The process for determining the pulse width TPP of the optical pulse PO used for measurement is the same as in the first embodiment.

[0093] In the first embodiment, the accumulated charge amounts Q1, Q2, Q3, and Q4 are obtained from the charge storage sections CS1, CS2, CS3, and CS4 of the pixel circuit 321, respectively, and the distance between the pixel circuit 321 of the distance image sensor 32 and the subject S is calculated using the delay time Td obtained from equations (1) to (7). In the second embodiment, the distance between the pixel circuit 321 and the subject S is calculated (estimated) using a correspondence table showing the relationship between the charge ratio R (obtained from each of the charge ratios R1 and R2 in the first embodiment and the threshold value ThW) and the estimated distance (corresponding to the distance between the pixel circuit 321 and the subject S).

[0094] In this embodiment, first, the distance from the distance image capturing device 1 to the subject S is changed in increments of a predetermined unit distance using the pulse width TPP of the light pulse PO calculated in the first embodiment, and a distance image is captured to obtain the charge ratio R. That is, a distance measurement experiment is performed, and the charge ratio R at that distance is calculated using a known distance, and the change curve of FIG. 10(a) is obtained. Then, a change curve of the charge ratio R and the distance is obtained, and a linear region in which the charge ratio R changes linearly with respect to the distance is extracted from the change curve.

[0095] FIG. 10 is a diagram illustrating a correspondence table showing the relationship between the charge ratio R and the estimated distance in the second embodiment. FIG. 10(a) is a graph showing the correspondence between the charge ratio R and the estimated distance, with the vertical axis representing the charge ratio R and the horizontal axis representing the distance (estimated distance). For example, the graph in Figure 10(a) shows the correspondence between the charge ratio R and the estimated distance for the time window Tw1. A similar graph is also obtained for the time window Tw2.

[0096] In Figure 10(a), the regions between charge ratio R_1 and charge ratio R_2, between charge ratio R_2 and charge ratio R_3, ..., between charge ratio R_8 and charge ratio R_9, ... are (considered to be) linear regions in which the charge ratio R changes linearly with distance. By using an optical pulse PO having a pulse width TPP (>TP1) longer than the pulse width TP1 of the accumulation drive signal TX, as determined in the first embodiment, fluctuations in the charge ratio slope SL are suppressed compared to when a pulse width TPP (=TP1) identical to the pulse width TP1 of the accumulation drive signal TX is used, as shown in FIG. 6.

[0097] That is, in the change curve of FIG. 10(a), the region (table interval TK) where the charge ratio R changes linearly with respect to the distance becomes longer. Therefore, by using an optical pulse PO with a pulse width TPP (>TP1) longer than the pulse width TP1 of the accumulation drive signal TX, the number of charge ratios R representing the change curve can be reduced compared to when a pulse width TPP (=TP1) identical to the pulse width TP1 of the accumulation drive signal TX is used.

[0098] Fig. 10(b) is a diagram showing an example of the configuration of a correspondence table showing the relationship between the charge ratio R and the estimated distance. This correspondence table is provided for each time window, and in the case of Fig. 10(b), it corresponds to the time window Tw1. The charge ratio R_5 indicates the threshold value at which the distance interval at which the charge ratio slope SL changes becomes shorter. When the charge ratio is less than R_5, the interval at which the charge ratio R changes is the table interval TK1. However, when the charge ratio is R_5 or more (when the charge ratio is near the boundary of the time window), the interval at which the charge ratio R changes becomes the table interval TK2 ( <TK1)となる。

[0099] 10(b), a distance is associated with each charge ratio R. For example, a distance D1 is associated with a charge ratio R_1, a distance D2 is associated with a charge ratio R_2, etc., and the correspondence table of FIG. 10(b) is written in advance and stored in a storage unit (not shown) for each time window. The correspondence table is written and stored in the storage unit for each of all pixel circuits 321 of the range image sensor 32, or for each of multiple regions obtained by dividing the range image sensor 32.

[0100] The distance calculation unit 43 calculates the charge ratio R using the amounts of accumulated charges Q1, Q2, Q3, and Q4 supplied from the charge accumulation units CS1, CS2, CS3, and CS4 of the pixel circuits 321, respectively, according to equations (1) and (2). Then, the distance calculation unit 43 searches the correspondence table of Figure 10(b) for a charge ratio R_n that is identical to the calculated charge ratio R, and if the same charge ratio R_n is found, it reads out the distance Dn that corresponds to this charge ratio R_n and sets it as the distance between the pixel circuit 321 and the subject S.

[0101] Furthermore, if the distance calculation unit 43 does not find a charge ratio R_n that is identical to the calculated charge ratio R, it extracts the charge ratio R_n-1 and charge ratio Rn that include this charge ratio R by referring to the correspondence table in Figure 10(b). Then, the distance calculation unit 43 reads out the distances Dn-1 and Dn corresponding to the charge ratios R_n-1 and Rn, respectively, and performs linear interpolation to correspond the distances Dn_1 and Dn to the charge ratio R, thereby calculating the distance D corresponding to the distance R, and sets this as the distance between the pixel circuit 321 and the subject S.

[0102] <Third embodiment> A third embodiment of the present invention will be described below with reference to the drawings. The third embodiment has the same configuration as the range image pickup device 1 in the first embodiment shown in FIG. The following describes the operation of the range image pickup device 1 according to the third embodiment, which differs from that of the first embodiment. The process for determining the pulse width TPP of the optical pulse PO used for measurement is the same as in the first embodiment.

[0103] In the second embodiment, a correspondence table showing the correspondence relationship between the charge ratio R and the distance D is referenced, and the distance corresponding to the charge ratio R calculated from the accumulated charge amounts Q1, Q2, Q3, and Q4 of the charge accumulation units CS1, CS2, CS3, and CS4 is extracted or obtained by linear interpolation. On the other hand, in the third embodiment, the relationship between the known distance and the measured distance D is approximated in advance by a polynomial, with the distance D being an indeterminate element, in correspondence with the change curve of the charge ratio R and the distance D shown in FIG. 10(a). Then, the distance calculation unit 43 substitutes the measured distance D into the polynomial to correct the positional deviation of the distance D from the actual distance. That is, a polynomial is calculated in advance to approximate the distance D calculated using the charge ratio to a known distance, and the distance calculation unit 43 corrects the distance deviation by substituting the measured distance D into the polynomial to approximate the distance D to the actual distance, thereby calculating a distance D with a reduced distance deviation from the actual distance.

[0104] In the polynomial approximation, the polynomial approximation is performed in accordance with the charge ratio gradient SL in the change curve of FIG. 10(a). As a result, by using an optical pulse PO having a pulse width TPP (>TP1) longer than the pulse width TP1 of the accumulation drive signal TX, as determined in the first embodiment, fluctuations in the charge ratio slope SL are suppressed compared to when the pulse width TPP (=TP1) identical to the pulse width TP1 of the accumulation drive signal TX is used, as shown in FIG. 6.

[0105] According to this embodiment, by using an optical pulse PO with a pulse width TPP (>TP1) longer than the pulse width TP1 of the accumulation drive signal TX, fluctuations in the charge ratio slope SL are suppressed compared to when the same pulse width TPP (=TP1) as the pulse width TP1 of the accumulation drive signal TX is used. As a result, the number of terms and degree of the indeterminate (charge ratio as a variable) in the polynomial that approximates the change curve are reduced, and an approximation equation can be obtained that can obtain the distance D with high approximation accuracy while reducing the calculation load using the charge ratio R.

[0106] Furthermore, in this embodiment, by using an optical pulse PO having a pulse width TPP (>TP1) longer than the optical pulse width TP1, it is possible that the charges generated by the reflected light RL may be accumulated across three charge accumulation units in the charge accumulation units CS1, CS2, CS3, and CS4, for example, across each of the charge accumulation units CS1, CS2, and CS3. In this way, when the reflected light RL is accumulated across three charge accumulation sections, a distance calculation may be performed to determine the distance between the distance image capturing device 1 and the subject S based on the three accumulated charge amounts Q where the reflected light RL overlaps. [Explanation of symbols]

[0107] 1...Distance image capturing device 2...Light source section 3...Light receiving section 31...Lens 32...Distance image sensor (distance image sensor) 321...Pixel circuit 322...Pixel driving circuit 4...Distance image processing section 41...Timing control section 42...Pulse width adjustment section 43...Distance calculation section 44...Measurement control section CS1, CS2, CS3, CS4...Charge storage section FD1, FD2, FD3, FD4...Floating diffusion G1, G2, G3, G4...Transfer transistors GD: Charge drain transistor ML...micro lens PD...photoelectric conversion element PO...light pulse RT1, RT2, RT3, RT4...Reset transistors S…Subject SF1, SF2, SF3, SF4...Source follower transistors SL1, SL2, SL3, SL4...Selection transistors

Claims

1. a light source unit that irradiates a measurement space with a light pulse; a light receiving unit including a photoelectric conversion element that generates charges according to light incident from the measurement space, a plurality of pixel circuits including a plurality of charge accumulation units that accumulate the charges in a frame period, and a pixel drive circuit that distributes and accumulates the charges generated in the photoelectric conversion element in each of the charge accumulation units via transfer transistors at a predetermined accumulation timing synchronized with the irradiation of the light pulse; a distance calculation unit that calculates the distance between the subject and the light receiving unit from a charge ratio of the amounts of electric charges generated by reflected light in the measurement space, distributed from the photoelectric conversion elements by the transfer transistors, and stored in each of the charge storage units; Equipped with the pulse width of the optical pulse is set to be longer than the period between the rising edge of a specific transfer transistor and the rising edge of the next transfer transistor among the transfer transistors; The pulse width is set by the variation in the rate of change of the charge ratio during the delay time from when the light pulse is irradiated until when the reflected light is incident, which corresponds to a measurable distance. A distance image capturing device characterized by:

2. The distance calculation unit Increasing the pulse width by adding a unit width time, calculating variations in the rate of change of the charge ratio corresponding to each of the delay times after irradiation of the light pulse having the increased pulse width; repeatedly increasing the pulse width of the optical pulse and calculating the variation corresponding to the increased pulse width until the pulse width of the optical pulse exceeds a predetermined width value; setting the pulse width at which the variation is minimum as the pulse width of the optical pulse; 2. The distance imaging device according to claim 1,

3. While sequentially changing the pulse width, the variation in the rate of change of the charge ratio in each change in the delay time is obtained for each pulse width, the pulse width at which the variation is minimum is extracted, and the extracted pulse width is set as the pulse width of the optical pulse.

2. The distance imaging device according to claim 1,

4. The distance calculation unit calculates the distance based on table information indicating a relationship between a corresponding distance corresponding to the distance between the object and the pixel and a charge ratio calculated from the amount of charge accumulated in each of the charge accumulation units.

4. The distance imaging device according to claim 1, wherein the distance imaging device is a distance sensor.

5. The distance calculation unit calculates a polynomial in advance that approximates a distance deviation between the distance calculated using the charge ratio and a known distance, and calculates the distance with the distance deviation corrected using the polynomial.

4. The distance imaging device according to claim 1, wherein the distance imaging device is a distance sensor.

6. The distance calculation unit calculates an average value of the variations of each of the pixel circuits corresponding to all of the pixel circuits, and sets the pulse width of the light pulse to the pulse width at which the average value of the variations is minimum.

3. The distance imaging device according to claim 2.

7. The distance calculation unit calculates an average value of variations in each of the pixel circuits corresponding to a part of the pixel circuits, and sets a pulse width at which the average value of the variations is minimum as the pulse width of the light pulse.

3. The distance imaging device according to claim 2.

8. A distance image capturing method for controlling a distance image capturing device including a plurality of pixel circuits, each of which includes a light source unit, a photoelectric conversion element, a plurality of charge accumulation units, and a transfer transistor, a pixel drive circuit, and a distance calculation unit, a step in which the pixel drive circuit accumulates charges generated by the photoelectric conversion elements in response to incident light from a measurement space in each charge accumulation unit in a frame period via the transfer transistors that transfer the charges from the photoelectric conversion elements to the charge accumulation units, at a predetermined accumulation period synchronized with irradiation of a light pulse from the light source unit; a step in which a distance calculation unit calculates the distance between the subject and the distance image pickup device from a charge ratio of the amounts of electric charges generated by reflected light in the measurement space, distributed by the transfer transistors from the photoelectric conversion elements, and stored in each of the charge storage units; Including, a pulse width of the optical pulse is set to be longer than a period between a rising edge of a specific transfer transistor and a rising edge of the next transfer transistor among the transfer transistors; The pulse width is set by the variation in the rate of change of the charge ratio during the delay time from when the light pulse is irradiated until when the reflected light is incident, which corresponds to a measurable distance. A distance image capturing method comprising:

Images