Range image capturing device and range image capturing method

Abstract

To provide a distance image pickup device which maintains the sensitivity to the resolution of a photographed image and incident light without increasing the capacity of a charge storage part and suppresses saturation due to increase in the intensity of the incident light.SOLUTION: A distance image pickup device comprises: a light reception unit that includes a plurality of pixel circuits having a photoelectric conversion element which generates a charge according to incident light that is incident from a measurement space of a measurement object, N (N≥3) charge storage units which store the charges in a frame period and a transfer transistor which transfers the charges from the photoelectric conversion element to the charge storage units, and a pixel drive circuit which performs on / off processing of the transfer transistor to the charge storage units in a storage period synchronized with the irradiation of a light pulse and distributes the charges for storage; a measurement control unit which divides the charges generated by the photoelectric conversion element into each of the combinations of the charge storage units in any transfer period of charge storage cycles and distributes the charges for storage; and a distance calculation unit which obtains a distance to a subject in the measurement space as a measurement distance on the basis of the charge amount stored in the charge storage units.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a distance image capturing device and a distance image capturing method. [Background technology]

[0002] Conventionally, there have been time-of-flight (TOF) type distance imaging devices that utilize the known speed of light to measure the distance to a subject based on the flight time of light (see, for example, Patent Document 1). A TOF distance imaging device includes a light source unit that emits light, and an imaging unit that includes a pixel array in which a plurality of pixel circuits that detect light for measuring distance are arranged in a two-dimensional matrix (array). Each of the pixel circuits has a photoelectric conversion element (e.g., a photodiode) that generates a charge corresponding to the intensity of light. With this configuration, the TOF range image pickup device can acquire (capture) information about the distance between itself and a subject, and an image of the subject, in a measurement space (three-dimensional space).

[0003] A TOF range imaging device measures distance based on the delay time between the time when synchrotron radiation is emitted and the time when the light reflected by the subject is received. Here, in the indirect TOF method using optical pulses, the electric charges generated in the photoelectric conversion element by the light reflected from the subject are distributed to each of the two electric charge accumulation units. The TOF range imaging device then calculates the distance to the subject based on the ratio of the amounts of charge stored in the charge storage sections. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2004-294420 Summary of the Invention [Problem to be solved by the invention]

[0005] However, if the amount of charge stored in one charge storage section is smaller than that in the other charge storage section, the noise in the charge storage section with the smaller amount of stored charge becomes dominant in the error of the calculated distance. That is, noise that occurs randomly when charges are read out from the charge storage section increases in proportion to the square root of the amount of charge that is read out. Therefore, the smaller the amount of charge, the greater the proportion of noise in the voltage signal read out from the charge storage section, reducing the S(signal) / N(noise) ratio of the voltage signal and the accuracy of the distance calculated from this voltage signal.

[0006] In three or more charge storage units, the highest S / N ratio is achieved when the amount of accumulated charge is in a ratio of 1:1 in each of the two charge storage units to which the charge is distributed. When the subject moves farther or closer from the distance where the charge amount ratio is 1:1, the charge is unevenly distributed and accumulated in one of the charge accumulation sections, resulting in a decrease in the S / N ratio of the voltage signal.

[0007] Therefore, when the ratio of the above-mentioned charge amounts is 1:1, the distance is in the center of the range of distances that can be calculated based on the charges accumulated in each charge accumulation section, and the calculated distance has the highest accuracy. On the other hand, when the distance range is near the upper limit or the lower limit, where the ratio of the amount of charge is most biased, the S / N ratio decreases and the accuracy of the measured distance decreases.

[0008] The present invention has been made in consideration of the above circumstances, and aims to provide a distance image capturing device and a distance image capturing method that make the accuracy of the distance calculated near the upper and lower limits of the distance range that can be calculated using the charges accumulated in each of multiple charge accumulation units the same as the accuracy calculated near the center of the distance range, even when the amount of charge accumulated in each of multiple charge accumulation units is small compared to the other charge accumulation units. [Means for solving the problem]

[0009] In order to solve the above-mentioned problems, the distance image pickup device of the present invention includes a light receiving unit having a photoelectric conversion element that generates a charge corresponding to incident light that is light that is incident from a measurement space that is a space to be measured, N (N≧3) charge accumulation units that accumulate the charge in a frame period, a plurality of pixel circuits each including a transfer transistor that transfers the charge from the photoelectric conversion element to each of the charge accumulation units, and a pixel drive circuit that performs on / off processing on each of the transfer transistors in each of the charge accumulation units at a predetermined accumulation period corresponding to irradiation of a light pulse, and allocates and accumulates the charge; and causes the pixel drive circuit to perform on / off processing on each of the transfer transistors at a first accumulation period synchronized with the irradiation of the light pulse in a first frame period in the frame period, and at other frame periods in the frame period, The irradiation time of the light pulse is 1 / 2 or less. a measurement control unit that performs on / off processing of each of the transfer transistors in another accumulation period shifted by a predetermined time to distribute the charges to the charge accumulation units; and a control unit that calculates a distance to an object present in the measurement space as a first measurement distance and another measurement distance in each of the first frame period and the other frame period based on the amount of charge accumulated in each of the charge accumulation units, and determines whether the first measurement distance or the other measurement distance is to be the distance between the distance image pickup device and the object in each of the pixel circuits according to the amount of charge accumulated in the charge accumulation units. and determining whether a ratio of the amount of charge accumulated in each of the first charge accumulation unit and the second charge accumulation unit, which are the charge accumulation units to which the charge due to the reflected light of the optical pulse is sequentially allocated from the photoelectric conversion element in each of the first frame period and the other frame period, is close to 1:1. The present invention is characterized by comprising a distance calculation unit that determines the distance.

[0010] In the distance image pickup device of the present invention, when the measurement control unit delays the second accumulation period, which is the other accumulation period, by a predetermined time with respect to the first accumulation period, the distance calculation unit determines that, in the first frame period, the ratio of the amount of charge accumulated in each of the first charge accumulation unit and the second charge accumulation unit, which are the charge accumulation units to which the charge caused by the reflected light of the light pulse from the subject is sequentially allocated from the photoelectric conversion element by the first accumulation period, is set to a predetermined value. Out of range In this case, the second measured distance, which is the other measured distance, is set as the distance.

[0011] The distance image capturing device of the present invention is characterized in that the measurement control unit causes the pixel driving circuit to allocate more signals from the photoelectric conversion element in the second frame period, which is the other frame period, than in the first frame period.

[0012] In the distance image pickup device of the present invention, when the measurement control unit advances the third accumulation period, which is another accumulation period, by a predetermined time relative to the first accumulation period, the distance calculation unit determines that, in the first frame period, the ratio of the amount of charge accumulated in each of the first charge accumulation unit and the second charge accumulation unit, which are the charge accumulation units to which the charge caused by the reflected light of the light pulse from the subject is sequentially allocated from the photoelectric conversion element during the first accumulation period, is set to a predetermined value. Out of range In this case, the third measured distance, which is the other measured distance, is set as the distance.

[0013] The distance image capturing device of the present invention is characterized in that the measurement control unit reduces the number of allocations from the photoelectric conversion element in the third frame period, which is the other frame period, by the pixel driving circuit compared to the number of allocations in the first frame period.

[0014] The distance image capturing device of the present invention is characterized in that the measurement control unit sets, as the other frame periods, a second frame period of a second accumulation period that is delayed by a predetermined time relative to the first accumulation period of the first frame period, and a third frame period of a third accumulation period that is advanced by a predetermined time relative to the first accumulation period of the first frame period, and in each of the first frame period, the second frame period, and the third frame period, the pixel driving circuit distributes charge to each of the charge accumulation units, and the distance calculation unit calculates the distance to a subject present in the measurement space as a first measurement distance, a second measurement distance, and a third measurement distance in each of the first frame period, the second frame period, and the third frame period, respectively, based on the amount of charge accumulated in each of the charge accumulation units, and determines whether the distance in each pixel circuit is the first measurement distance, the second measurement distance, or the third measurement distance depending on the amount of charge accumulated in the charge accumulation unit.

[0015] The distance image capturing device of the present invention is characterized in that, in the first frame period, the distance calculation unit determines the third measured distance as the distance if the ratio of the amount of charge accumulated in each of the first charge accumulation unit and the second charge accumulation unit, which are the charge accumulation units to which the charge due to the reflected light of the light pulse from the subject is sequentially allocated from the photoelectric conversion element by the first accumulation period, is equal to or greater than a predetermined first threshold, and determines the second measured distance as the distance if the ratio is equal to or less than a predetermined second threshold.

[0016] The distance image capturing device of the present invention is characterized in that the measurement control unit performs on / off processing of each of the transfer transistors in a first accumulation period synchronized with the irradiation of the light pulse in the first frame period, and determines whether to delay or advance the other accumulation period in the other frame period by a predetermined time relative to the first accumulation period depending on the amount of charge accumulated in the charge accumulation unit in the pixel circuit of the judgment area in the first frame period.

[0017] In the distance image pickup device of the present invention, the measurement control unit is configured to measure whether, in each of the two charge accumulation units in the pixel circuit of the determination area, a ratio between an amount of charge accumulated in the first charge accumulation unit in a previous first accumulation period and an amount of charge accumulated in the second charge accumulation unit in a subsequent first accumulation period is equal to or greater than a predetermined first threshold value in each of the successive first accumulation periods in the first frame period. under In this case, the other accumulation period is delayed with respect to the first accumulation period, and the other accumulation period is delayed until the second threshold value or more is reached. above In this case, the other accumulation period is set earlier than the first accumulation period.

[0018] The distance image capturing method of the present invention is a distance image capturing method for controlling a distance image capturing device comprising a plurality of pixel circuits each consisting of a photoelectric conversion element, a plurality of charge accumulation sections, and transfer transistors, a pixel drive circuit, a distance calculation section, and a measurement control section, and includes a step in which the pixel drive circuit allocates and accumulates charges generated by the photoelectric conversion element in response to incident light from a measurement space in each of N (N≧3) charge accumulation sections in a frame period by turning on and off each of the transfer transistors that transfer the charges from the photoelectric conversion element to the charge accumulation sections, at a predetermined accumulation period synchronized with the irradiation of a light pulse; and the measurement control section causes the pixel drive circuit to turn on and off each of the transfer transistors in a first accumulation period synchronized with the irradiation of the light pulse in a first frame period in the frame period, and in other frame periods in the frame period, The irradiation time of the light pulse is 1 / 2 or less. a step of allocating the charges to the charge accumulation units by performing on / off processing of each of the transfer transistors in another accumulation period shifted by a predetermined time; and a step of the distance calculation unit calculating a distance to an object present in the measurement space as a first measurement distance and another measurement distance in each of the first frame period and the other frame period based on the amount of charge accumulated in each of the charge accumulation units, and determining whether the distance in each pixel circuit is the first measurement distance or the other measurement distance according to the amount of charge accumulated in the charge accumulation unit. in accordance with whether a ratio of the amount of charge accumulated in each of the first charge accumulation section and the second charge accumulation section, which are the charge accumulation sections to which the charge due to the reflected light of the light pulse is sequentially allocated from the photoelectric conversion element in each of the first frame period and the other frame period, is close to 1:1. and determining whether the [Effects of the Invention]

[0019] As described above, according to the present invention, even when the amount of charge stored in each of a plurality of charge storage sections is small compared to the other charge storage sections, it is possible to provide a distance imaging device and a distance imaging method that make the accuracy of the distance calculated near the upper and lower limits of the distance range that can be calculated using the charge stored in each of the charge storage sections the same as the accuracy calculated near the center of the distance range. [Brief explanation of the drawings]

[0020] [Figure 1] 1 is a block diagram showing a schematic configuration of a distance imaging device according to a first embodiment of the present invention. [Figure 2] 1 is a circuit diagram showing an example of the configuration of a pixel circuit 321 arranged in a range image sensor 32 in the range image pickup device according to the first embodiment of the present invention. FIG. [Figure 3] FIG. 10 is a timing chart showing the transfer of charges generated by the photoelectric conversion element PD to each charge accumulation section CS in a predetermined frame period (a first frame period described later). [Figure 4] 10 is a diagram showing a timing chart for transferring charges generated in the photoelectric conversion element PD to each charge accumulation unit CS in each of the first and second frame periods in the first embodiment. FIG. [Figure 5] 4A to 4C are diagrams illustrating distance resolution in a first frame period and a second frame period in the first embodiment. [Figure 6] 10 is a flowchart showing an example of the operation of the distance image pickup device 1 of the first embodiment in the process of calculating the distance between the distance image sensor 32 and the subject S. [Figure 7] FIG. 10 is a timing chart showing the transfer of charges generated in the photoelectric conversion element PD to each charge accumulation unit CS in each of the first and second frame periods in the second embodiment. [Figure 8] 10A and 10B are diagrams illustrating distance resolution in a first frame period and a second frame period in the second embodiment. [Figure 9] 10 is a flowchart showing an example of the operation of the distance image pickup device 1 of the second embodiment in the process of calculating the distance between the distance image sensor 32 and the subject S. DETAILED DESCRIPTION OF THE INVENTION

[0021] First Embodiment A first embodiment of the present invention will be described below with reference to the drawings. Fig. 1 is a block diagram showing the schematic configuration of a distance image pickup device according to a first embodiment of the present invention. The distance image pickup device 1 shown in Fig. 1 is a ToF distance image pickup device, and includes a light source unit 2, a light receiving unit 3, and a distance image processing unit 4. Fig. 1 also shows a subject S, which is an object whose distance is to be measured by the distance image pickup device 1. The distance image pickup element is, for example, a distance image sensor 32 (described below) in the light receiving unit 3.

[0022] The light source section 2 irradiates a light pulse PO into a space to be photographed, in which a subject S, the distance of which is to be measured by the distance image pickup device 1, is present, under the control of the distance image processor 4. The light source section 2 is, for example, a surface-emitting semiconductor laser module such as a vertical cavity surface-emitting laser (VCSEL). The light source section 2 includes a light source device 21 and a diffuser plate 22.

[0023] The light source device 21 is a light source that emits laser light in a near-infrared wavelength band (for example, a wavelength band of 850 nm to 940 nm) that becomes the light pulse PO to be irradiated onto the subject S. The light source device 21 is, for example, a semiconductor laser light-emitting element. The light source device 21 emits pulsed laser light in response to control from the timing control unit 41. The diffusion plate 22 is an optical component that diffuses the laser light in the near-infrared wavelength band emitted by the light source device 21 to the extent of the surface to be irradiated onto the subject S. The pulsed laser light diffused by the diffusion plate 22 is emitted as a light pulse PO and is irradiated onto the subject S.

[0024] The light receiving unit 3 receives reflected light RL of the light pulse PO reflected by a subject S, the distance of which is to be measured in the range image pickup device 1, and outputs a pixel signal corresponding to the received reflected light RL. The light receiving unit 3 includes a lens 31 and a range image sensor 32. The lens 31 is an optical lens that guides the incident reflected light RL to the range image sensor 32. The lens 31 outputs the incident reflected light RL to the range image sensor 32 side, and causes the light to be received (incident) by pixel circuits provided in the light receiving region of the range image sensor 32.

[0025] The range image sensor 32 is an imaging element used in the range image capturing device 1. The range image sensor 32 includes a plurality of pixel circuits 321 in a two-dimensional light receiving area, and a pixel drive circuit 322 that controls each of the pixel circuits 321. The pixel circuit 321 includes one photoelectric conversion element (for example, a photoelectric conversion element PD described later), a plurality of charge accumulation units (for example, charge accumulation units CS1 to CS4 described later) corresponding to the one photoelectric conversion element, and components that distribute charge to each charge accumulation unit.

[0026] The range image sensor 32 distributes the charges generated by the photoelectric conversion elements to the respective charge accumulation sections under the control of the timing control section 41 (described below). The range image sensor 32 also outputs pixel signals according to the amount of charge distributed to the charge accumulation sections. The range image sensor 32 has a plurality of pixel circuits arranged in a two-dimensional matrix, and outputs pixel signals for one frame corresponding to each pixel circuit.

[0027] The distance image processing unit 4 controls the distance image pickup device 1, and calculates the distance from the distance image sensor 32 (that is, the distance image pickup device 1) to the subject S based on the pixel signals supplied from the light receiving unit 3. Distance image processing unit 4 includes a timing control unit 41, a distance calculation unit 42, and a measurement control unit 43.

[0028] The timing control unit 41 controls the timing of outputting various control signals required for distance measurement in accordance with the control of the measurement control unit 43. The various control signals here include, for example, a signal that controls the irradiation of the light pulse PO, a signal that distributes the reflected light RL to multiple charge accumulation units, a signal that controls the number of distributions per frame, etc. The number of distributions is the number of times that the process of distributing electric charges to the charge accumulation units CS (see FIG. 2) is repeated.

[0029] The distance calculation unit 42 outputs distance information calculated based on the pixel signals output from the range image sensor 32 under the control of the measurement control unit 43. The distance calculation unit 42 calculates the delay time Td from when the light pulse PO is emitted until when the reflected light RL is received, based on the amount of charge accumulated in the multiple charge accumulation units CS. The distance calculation unit 42 calculates the distance (measured distance) from the range image pickup device 1 to the range image pickup unit S according to the calculated delay time Td.

[0030] When capturing an image in a frame repeated at a frame period, the measurement control unit 43 controls the operation of each of the timing control unit 41, the distance calculation unit 42, and the pixel driving circuit 322 so that one image is captured using two frame periods (each of a first frame period and a second frame period, which will be described later). Here, the measurement control unit 43 performs control to shift each of the accumulation drive signals TX1, TX2, TX3, and TX4 in the second frame period by a predetermined time relative to each of the accumulation drive signals TX1, TX2, TX3, and TX4 in the first frame period.

[0031] Furthermore, the range image pickup device 1 according to this embodiment calculates the distance between the subject and the range image sensor 32 based on the charges accumulated in the charge accumulation section CS. That is, the measurement control unit 43 shifts the timing of distribution from the photoelectric conversion element PD to each of the charge accumulation units CS1, CS2, CS3, and CS4 in the second frame period relative to the first frame period. For example, when calculating the distance based on the amount of charge accumulated in each of the charge accumulation units CS2 and CS3, the distance is calculated using the amount of charge Q accumulated in the charge accumulation units CS2 and CS3 that is closest to being equal in either the first frame period or the second frame period (details will be described later).The same applies when calculating the distance based on the amount of charge accumulated in each of the charge accumulation units CS3 and CS4.

[0032] Here, during two frame periods in which one distance image is captured, even if the charge amount Q is uneven between the charge storage units CS in one frame period, the charge amount Q between the charge storage units CS will be close to equal in the other frame period. This reduces the deterioration of spatial resolution due to a decrease in the S / N ratio caused by a decrease in the charge amount Q, and by calculating the distance for each pixel using the same charge amount in each of the first and second frame periods, it is possible to capture a distance image with high spatial resolution.

[0033] That is, when driving the distance image sensor 32, random noise increases in proportion to the square root of the amount of charge accumulated in the charge accumulation unit CS, and therefore, as the amount of charge decreases, the proportion of noise increases, resulting in a decrease in the S / N ratio. Therefore, as the ratio of the amounts of charge in the two charge accumulation units CS approaches 1:1, the S / N ratio increases, while as the ratio of the amounts of charge deviates from 1:1, the S / N ratio decreases.

[0034] Therefore, if the amount of charge is unevenly distributed to one of the charge storage sections CS, the amount of charge distributed to the other section will decrease, and the distance resolution of the distance to be measured will deteriorate. As already mentioned, in this embodiment, in order to extend the measurement distance, three or more charge storage units CS are used to perform time window (TW) driving and measure the distance between each of the distance image pickup device 1 and the subject S.

[0035] When performing this time window driving, the distance near the joint between adjacent time windows causes a large amount of charge to be distributed to one of the charge storage units CS, resulting in an uneven amount of charge stored in the charge storage unit CS, and the S / N ratio is reduced the most. In this embodiment, in order to prevent the deterioration of distance resolution due to the random noise described above, one distance image is captured in the first frame period and the second frame period, as already described.

[0036] With this configuration, in the distance image capturing device 1, the light source unit 2 irradiates a light pulse PO in the near-infrared wavelength band onto the subject S, and the light receiving unit 3 receives the reflected light RL reflected by the subject S, and the distance image processing unit 4 outputs distance information measuring the measured distance between the subject S and the distance image capturing device 1. Although FIG. 1 shows the distance image pickup device 1 having the distance image processing unit 4 built therein, the distance image processing unit 4 may be an element provided outside the distance image pickup device 1.

[0037] Next, the configuration of the pixel circuit 321 in the range image sensor 32 will be described. Figure 2 is a circuit diagram showing an example of the configuration of a pixel circuit 321 arranged in the range image sensor 32 in the range image pickup device according to the first embodiment of the present invention. The pixel circuit 321 in Figure 2 is an example configuration including, for example, four pixel signal readout units RU1 to RU4. The configuration of the pixel circuit 321 in this embodiment is just one example, and the pixel circuit 321 has a configuration including three or more pixel signal readout units, i.e., n units (n≧3).

[0038] The pixel circuit 321 includes one photoelectric conversion element PD, a charge discharging transistor GD, and four pixel signal readout units RU (RU1 to RU4) that output voltage signals from corresponding output terminals O. Each pixel signal readout unit RU includes a readout gate transistor G, a floating diffusion FD, a charge storage capacitance C, a reset transistor RT, a source follower transistor SF, and a selection transistor SL. The floating diffusions FD (FD1, FD2, FD3, FD4) and the charge storage capacitances C (C1, C2, C3, C4) form charge storage units CS (CS1, CS2, CS3, CS4).

[0039] 2, the pixel signal readout unit RU1, which outputs a voltage signal from the output terminal O1, includes a readout gate transistor G1 (transfer MOS transistor), a floating diffusion FD1, a charge storage capacitance C1, a reset transistor RT1, a source follower transistor SF1, and a selection transistor SL1. In the pixel signal readout unit RU1, the floating diffusion FD1 and the charge storage capacitance C1 form a charge storage unit CS1. The pixel signal readout units RU2, RU3, and RU4 have a similar configuration.

[0040] The photoelectric conversion element PD is a buried photodiode that photoelectrically converts incident light, generates charges corresponding to the incident light, and accumulates the generated charges. In this embodiment, the incident light is incident from the space to be measured. In pixel circuit 321, the photoelectric conversion element PD photoelectrically converts incident light to generate electric charges, which are then distributed to each of four charge accumulation units CS (CS1 to CS4), and voltage signals corresponding to the amount of distributed electric charges are output to distance image processing unit 4. Furthermore, the configuration of the pixel circuit arranged in the distance image sensor 32 is not limited to the configuration having four pixel signal readout units RU (RU1 to RU4) as shown in Figure 2, but may be a pixel circuit having a configuration having one or more pixel signal readout units RU.

[0041] In driving the pixel circuit 321 of the range image pickup device 1, a light pulse PO is emitted for an irradiation time (also indicated as pulse width or time) To, and reflected light RL is received by the range image sensor 32 after a delay time Td. Under the control of the timing control unit 41, the pixel drive circuit 322 synchronizes with the irradiation of the light pulse PO by supplying accumulation drive signals TX1 to TX4 to readout gate transistors G1, G2, G3, and G4 at their respective timings to redirect the charges generated in the photoelectric conversion element PD and accumulate them in the charge accumulation units CS1, CS2, CS3, and CS4 in that order.

[0042] The pixel driving circuit 322 controls each of the reset transistor RT and the selection transistor SL by driving signals RST and SEL, respectively, converts the charges accumulated in the charge storage unit CS into an electrical signal by the source follower transistor SF, and outputs the generated electrical signal to the distance calculation unit 42 via the output terminal O. Furthermore, under the control of the timing control section 41, the pixel driving circuit 322 causes the charge generated in the photoelectric conversion element PD to flow to the power supply VDD in response to the driving signal RSTD, thereby discharging the charge (erasing the charge).

[0043] FIG. 3 is a timing chart showing the transfer of charges generated by the photoelectric conversion elements PD to the charge accumulation sections CS in a predetermined frame period (a first frame period, which will be described later). In the timing chart of Fig. 3, the vertical axis represents the pulse level, and the horizontal axis represents time. Fig. 3 also shows the accumulation cycle repeated during the charge accumulation period of a frame. The diagram shows the relative relationship on the time axis between the light pulse PO and the reflected light RL in normal mode, the timing of each of the accumulation drive signals TX1 to TX4 supplied to the readout gate transistors G1 to G4, and the timing of the drive signal RSTD supplied to the charge discharge transistor GD.

[0044] The timing control unit 41 causes the light source unit 2 to irradiate the measurement space with a light pulse PO. As a result, the light pulse PO is reflected by the subject and received by the light receiving unit 3 as reflected light RL. The photoelectric conversion element PD then generates charges corresponding to the background light and the reflected light RL. The pixel drive circuit 322 controls the on / off of each of the read gate transistors G1 to G4 to transfer the charges generated by the photoelectric conversion element PD to each of the charge accumulation units CS1 to CS4. That is, the pixel drive circuit 322 supplies each of the accumulation drive signals TX1 to TX4 as an "H" level signal with a predetermined time width (the same width as the irradiation time To, ie, the pulse width) to the read gate transistors G1 to G4, respectively.

[0045] The pixel driving circuit 322 turns on, for example, the read gate transistor G1 provided on a transfer path that transfers charge from the photoelectric conversion element PD to the charge accumulation unit CS1. As a result, the charge photoelectrically converted by the photoelectric conversion element PD is accumulated in the charge accumulation unit CS1 via the read gate transistor G1. Thereafter, the pixel driving circuit 322 turns off the read gate transistor G1. This stops the transfer of charge to the charge accumulation unit CS1. In this way, the pixel driving circuit 322 accumulates charge in the charge accumulation unit CS1. The same applies to the other charge accumulation units CS2, CS3, and CS4.

[0046] At this time, during the charge accumulation period in which charge is distributed to the charge accumulation sections CS (the period in which charge is accumulated in each of the charge accumulation sections CS in the frame), transfer periods T1, T2, T3 and T4 (periods in which charge is accumulated and integrated, transfer order) in which each of the accumulation drive signals TX1, TX2, TX3 and TX4 is supplied to the read gate transistors G1, G2, G3 and G4, respectively, are repeated in the accumulation period. Here, for example, in transfer period T1, charges generated by incident light (background light only) are accumulated in charge accumulation unit CS1, in transfer period T2, charges generated by incident light (background light + reflected light) are accumulated in charge accumulation unit CS2, in transfer period T3, charges generated by incident light (background light + reflected light) are accumulated in charge accumulation unit CS3, and in transfer period T4, charges generated by incident light (background light + reflected light) are accumulated in charge accumulation unit CS4.

[0047] Then, charges corresponding to incident light are transferred from the photoelectric conversion element PD to the charge accumulation units CS1, CS2, CS3, and CS4, respectively, via the read gate transistors G1, G2, G3, and G4. That is, during the charge accumulation period of each frame period, a plurality of accumulation periods are repeated in which charges are transferred in the order of transfer to the charge accumulation units CS1, CS2, CS3, and CS4. As a result, charges are accumulated in the charge accumulation units CS1, CS2, CS3, and CS4 in the transfer periods T1, T2, T3, and T4, respectively, for each accumulation period of the charge accumulation units CS1, CS2, CS3, and CS4 in the charge accumulation period.

[0048] Furthermore, when repeating the accumulation cycles of the charge accumulation units CS1, CS2, CS3, and CS4, after the transfer (distribution) of charges to the charge accumulation unit CS4 is completed, the pixel driving circuit 322 supplies an "H" level driving signal RSTD to the charge discharging transistor GD provided on the discharge path that discharges charges from the photoelectric conversion element PD, to turn it on. As a result, before the transfer period T1 for the charge storage unit CS1 starts, the charge discharging transistor GD discards the charge generated in the photoelectric conversion element PD after the transfer period T4 for the previous charge storage unit CS4 (i.e., resets the photoelectric conversion element PD).

[0049] The pixel driving circuit 322 then sequentially performs signal processing such as A / D conversion on the voltage signals from all of the pixel circuits 321 arranged in the light receiving section 3 in units of rows (horizontal arrangement) of the pixel circuits 321. Thereafter, the pixel driving circuit 322 outputs the voltage signals (voltages corresponding to the amounts of charge accumulated in each of the charge accumulation units CS) after signal processing to the distance calculation unit 42 in the order of the columns arranged in the light receiving unit 3.

[0050] As described above, the pixel driving circuit 322 accumulates charges in the charge accumulation units CS and discards the charges photoelectrically converted by the photoelectric conversion elements PD over one frame. As a result, charges corresponding to the amount of light received by the distance image pickup device 1 over a predetermined time period are accumulated in each charge accumulation unit CS. The pixel driving circuit 322 outputs an electrical signal corresponding to the amount of charge accumulated in each charge accumulation unit CS for one frame to the distance calculation unit 42.

[0051] Due to the relationship between the timing of irradiating the light pulse PO and the timing of accumulating charges in each of the charge accumulation units CS (CS1 to CS4), the charge accumulation unit CS1 holds an amount of charge corresponding to external light components such as background light before irradiating the light pulse PO. Furthermore, the charge accumulation units CS2, CS3, and CS4 hold charges corresponding to the reflected light RL and external light components, respectively. The distribution (allocation ratio) of the amount of charge allocated to the charge accumulation units CS2 and CS3, or the charge accumulation units CS3 and CS4, is a ratio that corresponds to the delay time Td between when the light pulse PO is reflected by the subject S and when it enters the range image pickup device 1.

[0052] Returning to FIG. 1, the distance calculation unit 42 uses this principle to calculate the delay time Td using the following equation (1) or (2). Td=To×(Q3-Q1) / (Q2+Q3-2×Q1) …(1) Td=To+To×(Q4-Q1) / (Q3+Q4-2×Q1) …(2) Here, To is the period (pulse width) during which the optical pulse PO is irradiated, Q1 is the amount of charge accumulated in the charge accumulation section CS1, Q2 is the amount of charge accumulated in the charge accumulation section CS2, Q3 is the amount of charge accumulated in the charge accumulation section CS3, and Q4 is the amount of charge accumulated in the charge accumulation section CS4. When Q2 ≥ Q4, for example, the distance calculation unit 42 calculates the delay time Td using the formula (1), while when Q2 < Q4, the distance calculation unit 42 calculates the delay time Td using the formula (2).

[0053] In the formula (1), charges generated by the reflected light are accumulated in the charge accumulation sections CS2 and CS3, but not in the charge accumulation section CS4. On the other hand, in the formula (2), charges generated by the reflected light are accumulated in the charge accumulation sections CS3 and CS4, but not in the charge accumulation section CS2. Note that in the formula (1) or (2), it is assumed that the component corresponding to the external light component among the amounts of charge accumulated in the charge accumulation sections CS2, CS3, and CS4 is the same as the amount of charge accumulated in the charge accumulation section CS1.

[0054] The distance calculation unit 42 calculates the measured round-trip distance to the subject S by multiplying the delay time obtained by the formula (1) or (2) by the speed of light (velocity). Then, the distance calculation unit 42 obtains the distance from the distance image sensor 32 (that is, the distance image capturing device 1) to the subject S by setting the round-trip distance calculated above to 1 / 2 (delay time Td × c (speed of light) / 2).

[0055] Also, the time Trs indicates the period during which the drive signal RSTD supplied to the charge discharge transistor GD is set to the "H" level so that the charge generated by the input light in the photoelectric conversion element PD does not remain (accumulate) after the charge distribution from the photoelectric conversion element PD to the charge accumulation section CS4 in one cycle of the accumulation period in FIG. 3 is completed.

[0056] The measurement control unit 43 generates one (one sheet) distance image indicating the distance to the subject for each pixel using two frame periods, namely the first frame period and the second frame period, as the frame period of FIG. 3 described above. That is, in the first frame, the measurement control unit 43 controls the timing control unit 41 so that the pixel drive circuit 322 sequentially supplies accumulation drive signals TX1, TX2, TX3, and TX4 having the same width as the light pulse PO to the readout gate transistors G1, G2, G3, and G4, respectively, in a first accumulation period that is an accumulation period synchronized with the light pulse PO, as shown in FIG.

[0057] On the other hand, the measurement control unit 43 controls the timing control unit 41 so that, in the second frame, the pixel driving circuit 322 sequentially supplies each of the accumulation drive signals TX1, TX2, TX3, and TX4, which have the same pulse width as the light pulse PO, to each of the readout gate transistors G1, G2, G3, and G4 in a second accumulation period, which is an accumulation period in which the timing at which the light pulse PO is emitted is delayed by half the pulse width (time To) of the light pulse PO shown in FIG.

[0058] In the above description, the measurement control unit 43 controls the timing control unit 41 to cause the pixel driving circuit 322 to delay the output timing of the accumulation driving signal TX in the second frame period (second accumulation period) by a predetermined time relative to the output timing of the accumulation driving signal TX in the first frame period (first accumulation period). However, the measurement control unit 43 may not change the output timing of the accumulation drive signal TX in the first frame period and the second frame period, and may instead control the timing control unit 41 to advance the timing of irradiation of the light pulse PO by half the pulse width (time To) of the light pulse PO in the second frame period with respect to the first frame period. In this case, the accumulation drive signal TX in the second frame period is delayed by a predetermined time relative to the output timing of the accumulation drive signal TX in the first frame period, using the irradiation timing of the light pulse PO as a reference.

[0059] FIG. 4 is a timing chart showing the transfer of charges generated by the photoelectric conversion elements PD to the charge accumulation sections CS in each of the first and second frame periods in this embodiment. Also, in order to compare the timing at which the accumulation drive signals TX1, TX2, TX3, and TX4 are supplied in the first frame period and the second frame period, the waveform of the drive signal RSTD in FIG. 3 is omitted.

[0060] Here, the first frame period in FIG. 4(a) is the same as the timing chart shown in FIG. 3 already explained. The pixel drive circuit 322 outputs each of the accumulation drive signals TX1, TX2, TX3, and TX4 in a first frame period, which is an accumulation period in which the accumulation drive signal TX2 is synchronized with the light pulse PO. The pixel drive circuit 322 also outputs each of the accumulation drive signals TX1, TX2, TX3, and TX4 sequentially for each successive accumulation period.

[0061] In the case of the first frame period of FIG. 4(a), the time during which reflected light RL is incident in the accumulation periods of the accumulation drive signals TX2 and TX3 is longer in the accumulation period of the accumulation drive signal TX3 than in the accumulation drive signal TX2. On the other hand, in the case of the second frame period in FIG. 4(a), the time during which reflected light RL is incident in the accumulation periods of the accumulation drive signals TX2 and TX3 is longer in the accumulation period of the accumulation drive signal TX2 than in the accumulation drive signal TX3.

[0062] Furthermore, in the second frame period, the pixel drive circuit 322 delays each of the accumulation drive signals TX1, TX2, TX3, and TX4 by a predetermined time (half the time of the pulse width To) compared to the first frame. Here, the distance calculation unit 42 uses the charge amounts Q1, Q2, and Q3 accumulated in the charge accumulation units CS1, CS2, and CS3 in the first frame period to obtain the first measured distance as a distance from the delay time Td calculated by (1) above.

[0063] In addition, in the second frame period, the distance calculation unit 42 also uses the charge amounts Q1, Q2, and Q3 accumulated in the charge accumulation units CS1, CS2, and CS3, respectively, to determine the second measurement distance as the distance from the delay time Td calculated by (3) below. Td=To×(Q3-Q1) / (Q2+Q3-2×Q1)+Ra×To …(3) Here, the coefficient Ra is a coefficient obtained by dividing the delayed (shifted) time by the pulse width To. For example, when the delay is half the time of the pulse width To, the coefficient Ra is 1 / 2.

[0064] Then, if the charge amount ratio α obtained by dividing the charge amount Q2 accumulated in the charge accumulation unit CS2 in the first frame period by the charge amount Q3 accumulated in the predetermined charge accumulation unit CS3 is within a predetermined threshold range, for example, greater than 1 / 2 and less than 2 (1 / 2<α<2), the distance calculation unit 42 determines the first measured distance obtained from the delay time Td calculated in the first frame period to be the distance of the corresponding pixel.

[0065] 4(b), pixel drive circuit 322 delays each of accumulation drive signals TX1, TX2, TX3, and TX4 in the second frame period by a predetermined time (half the pulse width To) compared to the first frame. Furthermore, pixel drive circuit 322 sequentially outputs each of accumulation drive signals TX1, TX2, TX3, and TX4 for each successive accumulation period. In the case of the first frame period of FIG. 4(b), the time during which reflected light RL is incident in the accumulation periods of the accumulation drive signals TX2 and TX3 is longer in the accumulation period of the accumulation drive signal TX3 than in the accumulation drive signal TX2. On the other hand, in the case of the second frame period of FIG. 4(b), the time during which reflected light RL is incident in the accumulation periods of the accumulation drive signals TX2 and TX3 is longer than that of the accumulation drive signal TX3.

[0066] Here, the distance calculation unit 42 uses the amounts of charge Q1, Q2, and Q3 accumulated in the charge accumulation units CS1, CS2, and CS3 in the first frame period to obtain the first measured distance as a distance using the above equation (1). Furthermore, in the second frame period, the distance calculation unit 42 also uses the charge amounts Q1, Q2, and Q3 accumulated in the charge accumulation units CS1, CS2, and CS3, respectively, to determine the second measured distance calculated from the delay time Td calculated by the above equation (3) as the distance to the corresponding pixel.

[0067] Then, if the charge amount ratio α obtained by dividing the charge amount Q2 accumulated in the charge accumulation unit CS2 in the first frame period by the charge amount Q3 accumulated in the predetermined charge accumulation unit CS3 is outside the above threshold range, or if 1 / 2<α<2 is not satisfied (α≦1 / 2 or α≧2), the distance calculation unit 42 sets the second measured distance calculated in the second frame period as the distance to the corresponding pixel.

[0068] In Figure 4(c), pixel drive circuit 322 also delays each of accumulation drive signals TX1, TX2, TX3, and TX4 in the second frame period by a predetermined time (half the pulse width To) compared to the first frame. Furthermore, pixel drive circuit 322 sequentially outputs each of accumulation drive signals TX1, TX2, TX3, and TX4 for each successive accumulation period. In the case of Figure 4(c), charges generated in photoelectric conversion element PD by reflected light RL are distributed and stored in charge accumulation units CS3 and CS4.

[0069] In the case of the first frame period of FIG. 4(c), the time during which reflected light RL is incident in the accumulation periods of the accumulation drive signals TX3 and TX4 is longer in the accumulation period of the accumulation drive signal TX3 than in the accumulation drive signal TX4. On the other hand, in the case of the second frame period of FIG. 4(c), the time during which reflected light RL is incident in the accumulation periods of the accumulation drive signals TX2 and TX3 is longer in the accumulation period of the accumulation drive signal TX3 than in the accumulation drive signal TX2.

[0070] Here, the distance calculation unit 42 uses the charge amounts Q1, Q2, and Q3 accumulated in the charge accumulation units CS1, CS2, and CS3 in the first frame period to obtain the first measurement distance as a distance from the delay time calculated by (2) above. In addition, in the second frame period, the distance calculation unit 42 also uses the charge amounts Q1, Q2, and Q3 accumulated in the charge accumulation units CS1, CS2, and CS3, respectively, to determine the second measurement distance as the distance from the delay time calculated by (4) below. Td=To+To×(Q4-Q1) / (Q3+Q4-2×Q1)+Ra×To…(4) Here, the coefficient Ra is a coefficient obtained by dividing the delayed (shifted) time by the pulse width To. For example, when the delay is half the time of the pulse width To, the coefficient Ra is 1 / 2.

[0071] Then, if the charge amount ratio β obtained by dividing the charge amount Q3 accumulated in the charge accumulation unit CS3 in the first frame period by the charge amount Q4 accumulated in the predetermined charge accumulation unit CS4 is within a predetermined threshold range, for example, greater than 1 / 2 and less than 2 (1 / 2<β<2), the distance calculation unit 42 determines the first measured distance obtained from the delay time Td calculated in the first frame period to be the distance of the corresponding pixel. Furthermore, if the charge amount ratio β obtained by dividing the charge amount Q3 accumulated in the charge accumulation unit CS3 in the first frame period by the charge amount Q4 accumulated in the predetermined charge accumulation unit CS4 is outside the above threshold range, or if 1 / 2<β<2 is not satisfied (β≦1 / 2 or β≧2), the distance calculation unit 42 sets the second measured distance calculated in the second frame period as the distance to the corresponding pixel.

[0072] FIG. 5 is a diagram illustrating distance resolution in the first frame period and the second frame period in the first embodiment. FIG. 5(a) shows the correspondence between the measured distance and distance resolution in the first frame period, with the horizontal axis representing the measured distance and the vertical axis representing the distance resolution. In the first frame period, as shown in the first frame of Figure 4(b), as the amount of charge Q2 accumulated in the charge accumulation unit CS2 decreases compared to the amount of charge Q3 accumulated in the charge accumulation unit CS3, the distance resolution decreases as shown in the distance range of 4 m to 5 m on curve 401 shown in Figure 5(a).

[0073] On the other hand, Fig. 5(b) shows the correspondence relationship between distance and distance resolution when each of the accumulation drive signals TX1, TX2, TX3, and TX4 is delayed by a time To / 2 with respect to the first frame period in the second frame period. In Fig. 5(b), the horizontal axis represents the measured distance, and the vertical axis represents the distance resolution. As shown in the second frame period of Figure 5(b), by delaying each of the storage drive signals TX1, TX2, TX3, and TX4, the ratio α of the amount of charge Q2 stored in the charge storage unit CS2 to the amount of charge Q3 stored in the charge storage unit CS3 approaches 1 / 2. As a result, as shown in the second frame of Figure 4(b), the amount of charge Q2 accumulated in the charge storage unit CS2 approaches and becomes equal to the amount of charge Q3 accumulated in the charge storage unit CS3, so that the distance resolution does not decrease, as shown in the distance range of 4 m to 5 m on curve 402 shown in Figure 5(b).

[0074] However, in the second frame period, since each of the accumulation drive signals TX1, TX2, TX3, and TX4 is delayed with respect to the first frame period, the amount of charge Q3 accumulated in the charge accumulation unit CS3 decreases as shown in FIG. 4(c). As shown in FIG. 5(b), the distance resolution at a distance of 6 m to 8 m is greater than that at a distance of 6 m to 8 m in FIG. 5(a), and the function for accuracy of position detection is reduced. In the case of Figure 4(c), as already explained, when the charge amount ratio β is within the above threshold range, 1 / 2<β<2, so the distance calculation unit 42 determines the first measured distance obtained from the delay time Td calculated in the first frame period to be the distance of the corresponding pixel.

[0075] As shown in each of Figures 5(a) and 5(b) above, the distance resolution changes depending on the distance to be measured, and by delaying each of the accumulation drive signals TX1, TX2, TX3, and TX4, the distance resolution increases, and the area where the accuracy of position detection decreases is also delayed. Therefore, by delaying each of the accumulation drive signals TX1, TX2, TX3, and TX4 to correspond to the distance range where the distance resolution increases and the accuracy of position detection decreases, it is possible to adjust the distance range where the distance resolution increases and the accuracy of position detection decreases.

[0076] Then, the distance calculation unit 42 compares each of the charge amount ratio α and the charge amount ratio β with a threshold value to determine whether to use the first measured distance calculated in the first frame period or the second measured distance calculated in the second frame period as the distance between the distance image capturing device 1 and the subject S. 5(c), in this embodiment, the first measurement distance in the first frame period is used for distance ranges 501 and 503, and the second measurement distance in the second frame period is used for distance ranges 502 and 504. This allows the relationship between the decrease in accuracy of distance measurement due to an increase in distance resolution and the increase in the measured distance to be nearly linear (solid curve 403), and the accuracy of the measured distance can be improved compared to the conventional case where only the first frame period is used.

[0077] Furthermore, in two charge storage units CS used to calculate the distance, for example, charge storage units CS2 and CS3, when the amount of charge Q2 stored in one charge storage unit CS2 approaches 0 (minimum value), the amount of charge Q3 stored in the other charge storage unit CS3 approaches the maximum value. Therefore, by setting the delay time for each of the accumulation drive signals TX1, TX2, TX3, and TX4 to To / 2, the charge amounts Q2 and Q3 approach a charge amount ratio of 1:1, and the distance range with the highest distance resolution and lowest measurement accuracy can be changed to the distance range with the lowest distance resolution and highest measurement accuracy. Therefore, the threshold value is most preferably set to 1 / 2, but since the distance resolution can be improved by delaying the threshold value, the threshold value may be set to any arbitrary value.

[0078] Furthermore, the number of times that the charges are distributed from the photoelectric conversion element PD to the charge accumulation section CS during the accumulation period of the second frame period may be increased compared to the first frame period. That is, when the second measured distance in the second frame period is selected, it indicates that the distance to the subject S is long. Therefore, by increasing the amount of charge Q stored in the charge storage section CS, the accuracy of measuring the distance between the range image pickup device 1 and the subject S can be improved.

[0079] 6 is a flowchart showing an example of the operation of the distance image capturing device 1 of the first embodiment for calculating the distance between the distance image sensor 32 and the subject S. The distance image capturing process is performed in two frame cycles, a first frame cycle and a second frame cycle.

[0080] Step S101: In the first frame, under the control of the measurement control unit 43, the pixel driving circuit 322 performs a process of distributing the charge from the photoelectric conversion element PD to each of the charge accumulation units CS1, CS2, CS3, and CS4. At this time, in each pixel circuit 321 (pixel unit), the pixel driving circuit 322 distributes charges in the corresponding accumulation periods of the charge accumulation units CS1, CS2, CS3, and CS4 according to the accumulation driving signals TX1, TX2, TX3, and TX4 of the first frame period shown in FIG. 3 or FIG. 4. As already mentioned, the number of times (the number of accumulation cycles) that charges are accumulated in each of the charge accumulation units CS1, CS2, CS3, and CS4 is multiple in the accumulation period of a frame cycle.

[0081] Step S102: When the accumulation period in the first frame cycle ends, the pixel driving circuit 322 outputs pixel signals, which are voltages corresponding to the amounts of charge Q1, Q2, Q3, and Q4 accumulated in the charge accumulation units CS1, CS2, CS3, and CS4, respectively, to the distance calculation unit 42. The distance calculation unit 42 calculates the delay time Td based on pixel signals (hereinafter, for the sake of explanation, described as charge amounts) supplied from the pixel circuits 321 provided for each pixel of the distance image sensor 32.

[0082] At this time, the distance calculation unit 42 compares the amounts of electric charge Q2 and Q4, and if the amount of electric charge Q2 is greater, calculates the delay time Td using the above formula (1). On the other hand, the distance calculation unit 42 compares the amounts of electric charge Q2 and Q4, and if the amount of electric charge Q4 is greater, calculates the delay time Td using the above formula (2). Then, the distance calculation unit 42 calculates the first measurement distance based on the calculated delay time Td.

[0083] Step S103: In the second frame period, the pixel driving circuit 322, under the control of the measurement control unit 43, performs a process of distributing the charges from the photoelectric conversion element PD to each of the charge accumulation units CS1, CS2, CS3, and CS4. At this time, in each pixel circuit 321 (pixel unit), the pixel driving circuit 322 distributes charges to the charge accumulation units CS1, CS2, CS3, and CS4, respectively, using the accumulation driving signals TX1, TX2, TX3, and TX4 of the second frame period shown in FIG. 3 or FIG. 4.

[0084] That is, the pixel driving circuit 322 distributes charges to the charge accumulation units CS1, CS2, CS3, and CS4 in their corresponding accumulation periods using the accumulation driving signals TX1, TX2, TX3, and TX4, which are delayed by a predetermined time with respect to the first frame period. As already mentioned, the number of times (the number of accumulation cycles) that charges are accumulated in each of the charge accumulation units CS1, CS2, CS3, and CS4 is multiple in the accumulation period of a frame cycle.

[0085] Step S104: When the accumulation period in the second frame cycle ends, the pixel driving circuit 322 outputs pixel signals, which are voltages corresponding to the amounts of charge Q1, Q2, Q3, and Q4 accumulated in the charge accumulation units CS1, CS2, CS3, and CS4, respectively, to the distance calculation unit 42. The distance calculation unit 42 calculates the delay time Td based on the pixel signals (ie, the amount of charge) supplied from each of the pixel circuits 321 provided for each pixel of the distance image sensor 32.

[0086] At this time, the distance calculation unit 42 compares the amounts of electric charge Q2 and Q4, and if the amount of electric charge Q2 is greater, calculates the delay time Td using the above formula (3). On the other hand, the distance calculation unit 42 compares the amounts of electric charge Q2 and Q4, and if the amount of electric charge Q4 is greater, calculates the delay time Td using the above formula (4). Then, the distance calculation unit 42 calculates the second measurement distance based on the calculated delay time Td.

[0087] Step S105: The distance calculation unit 42 selects one of the pixel circuits 321 for which processing has not been completed as the pixel to be processed, compares the charge amount Q2 with the charge amount Q4 in the first frame period in that pixel circuit 321, and determines whether the charge amount Q2 exceeds the charge amount Q4. At this time, if the charge amount Q2 exceeds the charge amount Q4, the distance calculation unit 42 advances the process to step S106. On the other hand, if the amount of electric charge Q2 is equal to or less than the amount of electric charge Q4, the distance calculation unit 42 advances the process to step S107.

[0088] Step S106: The distance calculation unit 42 obtains the charge amount ratio α by dividing the charge amount Q2 by the charge amount Q3. Then, the distance calculation unit 42 compares the charge amount ratio α with a preset threshold range, and determines whether the charge amount ratio α is within the threshold range. At this time, if the charge amount ratio α is within the threshold range, the distance calculation unit 42 advances the process to step S108. On the other hand, if the charge amount ratio α is outside the threshold range, the distance calculation unit 42 advances the process to step S109.

[0089] Step S107: The distance calculation unit 42 obtains the charge amount ratio β by dividing the charge amount Q2 by the charge amount Q3. Then, the distance calculation unit 42 compares the charge amount ratio β with a preset threshold value, and determines whether the charge amount ratio β is equal to or less than the threshold value. At this time, if the charge amount ratio β is within the threshold range, the distance calculation unit 42 advances the process to step S108. On the other hand, if the charge amount ratio β is outside the threshold range, the distance calculation unit 42 advances the process to step S109.

[0090] Step S108: Since the charge amount ratio α or the charge amount ratio β is within the threshold range, the distance calculation unit 42 determines that the resolution of the first measured distance meets the specified accuracy, and sets the first measured distance as the distance between the distance image capturing device 1 and the subject S at the pixel to be processed.

[0091] Step S109: Since the charge amount ratio α or the charge amount ratio β is outside the threshold range, the distance calculation unit 42 determines that the resolution of the first measured distance does not meet the specified accuracy, and sets the second measured distance as the distance between the distance image capturing device 1 and the subject S at the pixel to be processed.

[0092] Step S110: The distance calculation unit 42 determines whether the process of calculating the distance between the distance image pickup device 1 and the subject S has been performed for all pixel circuits 321, that is, whether calculation of the distance for all pixels in the distance image sensor 32 has been completed. At this time, if the distance calculation unit 42 has finished calculating the distances of all pixels in the distance image sensor 32, the process proceeds to step S111. On the other hand, if the distance calculation unit 42 has not yet finished calculating the distances of all pixels in the distance image sensor 32, the process proceeds to step S105.

[0093] Step S111: When the distance calculation unit 42 determines that each of all pixels in the distance image sensor 32 is the first measurement distance or the second measurement distance, it outputs an image having the distance between the distance image capturing device 1 and the subject S for each pixel to an external device (including the process of writing to an external storage device), and proceeds to step S101.

[0094] <Second embodiment> A second embodiment of the present invention will be described below with reference to the drawings. The configuration of the range image capturing device 1 of the second embodiment is the same as that of the first embodiment shown in Fig. 1. In the following, only the operations of the second embodiment that are different from those of the first embodiment will be described. Furthermore, in the second embodiment, two frame periods, a first frame period and a third frame period, are used to capture one captured image, just like in the first embodiment.

[0095] The operation differs from the first embodiment in the timing at which the accumulation drive signals TX1, TX2, TX3, and TX4 are supplied in the third frame period. In the first embodiment, a process is performed to delay the output timing of each of the accumulation drive signals TX1, TX2, TX3, and TX4 in the second frame period (processing to set it as the second accumulation period) relative to the output timing of each of the accumulation drive signals TX1, TX2, TX3, and TX4 in the first frame period (first accumulation period). However, in the second embodiment, processing is performed to advance the output timing for supplying each of the accumulation drive signals TX1, TX2, TX3, and TX4 in the third frame period relative to the first frame period (processing to set the third accumulation period).

[0096] As in the first embodiment, the measurement control unit 43 controls the timing control unit 41 so that, in the first frame, the pixel drive circuit 322 sequentially supplies accumulation drive signals TX1, TX2, TX3, and TX4 having the same width as the light pulse PO to the readout gate transistors G1, G2, G3, and G4, respectively, in a first accumulation period that is an accumulation period synchronized with the light pulse PO, as shown in FIG.

[0097] On the other hand, in the third frame, the measurement control unit 43 controls the timing control unit 41 so that the pixel driving circuit 322 sequentially supplies each of the accumulation drive signals TX1, TX2, TX3, and TX4, which have the same pulse width as the light pulse PO, to each of the readout gate transistors G1, G2, G3, and G4 in a third accumulation period, which is an accumulation period in which the timing at which the light pulse PO is emitted is advanced by half the pulse width (time To) of the light pulse PO shown in FIG.

[0098] In the above description, the measurement control unit 43 controls the timing control unit 41 to cause the pixel driving circuit 322 to advance the output timing of the accumulation driving signal TX in the third frame period by a predetermined time relative to the output timing of the accumulation driving signal TX in the first frame period. However, the measurement control unit 43 may not change the output timing of the accumulation drive signal TX in the first frame period and the third frame period, and may instead control the timing control unit 41 to delay the timing at which the light source device 21 irradiates the light pulse PO in the third frame period by half the pulse width (time To) of the light pulse PO, relative to the first frame period. In this case, this is equivalent to relatively advancing the accumulation drive signal TX in the third frame period by a predetermined time, relative to the output timing of the accumulation drive signal TX in the first frame period, using the irradiation timing of the light pulse PO as a reference.

[0099] FIG. 7 is a timing chart showing the transfer of charges generated by the photoelectric conversion elements PD to the charge accumulation units CS in each of the first and third frame periods in the first embodiment. Also, in order to compare the timing at which the accumulation drive signals TX1, TX2, TX3, and TX4 are supplied in the first and third frame periods, the waveform of the drive signal RSTD in FIG. 3 is omitted.

[0100] Here, the first frame period in FIG. 7(a) is the same as the timing chart shown in FIG. 3 already explained. The pixel drive circuit 322 outputs each of the accumulation drive signals TX1, TX2, TX3, and TX4 in a first frame period (first accumulation period) in which the accumulation drive signal TX2 is synchronized with the light pulse PO. The pixel drive circuit 322 also outputs each of the accumulation drive signals TX1, TX2, TX3, and TX4 sequentially for each successive accumulation period.

[0101] In the case of the first frame period of FIG. 7(a), the time during which reflected light RL is incident in the accumulation periods of the accumulation drive signals TX2 and TX3 is longer in the accumulation period of the accumulation drive signal TX3 than in the accumulation drive signal TX2. On the other hand, in the case of the third frame period in Figure 7(a), the time during which reflected light RL is incident in each accumulation period (third accumulation period) of accumulation drive signals TX2 and TX3 is longer in the accumulation period (third accumulation period) of accumulation drive signal TX2 than in the accumulation drive signal TX3.

[0102] Furthermore, in the third frame period, the pixel drive circuit 322 supplies each of the accumulation drive signals TX1, TX2, TX3, and TX4 a predetermined time earlier than in the first frame (supplying them earlier by half the pulse width To). Here, the distance calculation unit 42 uses the charge amounts Q1, Q2, and Q3 accumulated in the charge accumulation units CS1, CS2, and CS3 in the first frame period to obtain the first measured distance as a distance from the delay time Td calculated by (1) above.

[0103] Furthermore, in the third frame period, the distance calculation unit 42 also uses the charge amounts Q1, Q2, and Q3 accumulated in the charge accumulation units CS1, CS2, and CS3, respectively, to determine the third measured distance as the distance from the delay time Td calculated by (5) below. Td=To×(Q3-Q1) / (Q2+Q3-2×Q1)-Ra×To …(5) Here, the coefficient Ra is a coefficient obtained by dividing the advanced (shifted) time by the pulse width To. For example, if the time is advanced by half the pulse width To, the coefficient Ra is 1 / 2.

[0104] Then, if the charge amount ratio α obtained by dividing the charge amount Q2 accumulated in the charge accumulation unit CS2 in the first frame period by the charge amount Q3 accumulated in the predetermined charge accumulation unit CS3 is within a predetermined threshold range, for example, greater than 1 / 2 and less than 2 (1 / 2<α<2), the distance calculation unit 42 determines the first measured distance obtained from the delay time Td calculated in the first frame period to be the distance of the corresponding pixel.

[0105] 7(b), the pixel driving circuit 322 also supplies each of the accumulation drive signals TX1, TX2, TX3, and TX4 in the third frame period a predetermined time earlier than in the first frame period (supplying them half the time of the pulse width To earlier). The pixel driving circuit 322 also outputs each of the accumulation drive signals TX1, TX2, TX3, and TX4 sequentially for each successive accumulation period (third accumulation period).

[0106] In the case of the first frame period of FIG. 7(b), the time during which reflected light RL is incident in the accumulation periods of the accumulation drive signals TX2 and TX3 is longer in the accumulation period of the accumulation drive signal TX2 than in the accumulation drive signal TX3. On the other hand, in the case of the third frame period of FIG. 7(b), the time during which reflected light RL is incident in the accumulation periods of the accumulation drive signals TX2 and TX3 is longer in the accumulation period of the accumulation drive signal TX3 than in the accumulation drive signal TX2.

[0107] Here, the distance calculation unit 42 uses the amounts of charge Q1, Q2, and Q3 accumulated in the charge accumulation units CS1, CS2, and CS3 in the first frame period to obtain the first measured distance as a distance using the above equation (1). Furthermore, in the third frame period, the distance calculation unit 42 also uses the charge amounts Q1, Q2, and Q3 accumulated in the charge accumulation units CS1, CS2, and CS3, respectively, to determine the third measured distance calculated from the delay time Td calculated by the above equation (5) as the distance to the corresponding pixel.

[0108] Then, if the charge amount ratio α obtained by dividing the charge amount Q2 accumulated in the charge accumulation unit CS2 in the first frame period by the charge amount Q3 accumulated in the predetermined charge accumulation unit CS3 is outside the above threshold range, or if 1 / 2<α<2 is not satisfied (α≦1 / 2 or α≧2), the distance calculation unit 42 sets the third measured distance calculated in the third frame period as the distance to the corresponding pixel.

[0109] 7(c), the pixel drive circuit 322 also supplies each of the accumulation drive signals TX1, TX2, TX3, and TX4 in the third frame period a predetermined time earlier than in the first frame period (supplied half the time of the pulse width To). Furthermore, the pixel drive circuit 322 sequentially outputs each of the accumulation drive signals TX1, TX2, TX3, and TX4 for each successive accumulation period. In the case of FIG. 7(c), the charges generated in the photoelectric conversion element PD by the reflected light RL are distributed and stored in the charge storage units CS3 and CS4.

[0110] In the case of the first frame period of FIG. 7(c), the time during which reflected light RL is incident in the accumulation periods of the accumulation drive signals TX2 and TX3 is longer in the accumulation period of the accumulation drive signal TX2 than in the accumulation drive signal TX3. On the other hand, in the case of the third frame period of Figure 7(c), the time during which reflected light RL is incident in the accumulation periods of the accumulation drive signals TX2 and TX3 is longer in the accumulation period of the accumulation drive signal TX3 than in the accumulation drive signal TX2.

[0111] Here, the distance calculation unit 42 uses the charge amounts Q1, Q2, and Q3 accumulated in the charge accumulation units CS1, CS2, and CS3 in the first frame period to obtain the first measurement distance as a distance from the delay time calculated by the above equation (2). Furthermore, in the third frame period, the distance calculation unit 42 also uses the charge amounts Q1, Q2, and Q3 accumulated in the charge accumulation units CS1, CS2, and CS3, respectively, to determine the third measurement distance as the distance from the delay time calculated by the above equation (5).

[0112] Then, if the charge amount ratio α obtained by dividing the charge amount Q2 accumulated in the charge accumulation unit CS3 in the first frame period by the charge amount Q3 accumulated in the predetermined charge accumulation unit CS3 is outside the above threshold range, and 1 / 2<α<2 is not satisfied (α≦1 / 2 or α≧2), the distance calculation unit 42 determines the first measured distance obtained from the delay time Td calculated in the first frame period to be the distance of the corresponding pixel. Furthermore, if the charge amount ratio α obtained by dividing the charge amount Q2 accumulated in the charge accumulation unit CS2 in the first frame period by the charge amount Q3 accumulated in the charge accumulation unit CS3 is outside the above threshold range, or if 1 / 2<α<2 is not satisfied (α≦1 / 2 or α≧2), the distance calculation unit 42 determines the third measured distance calculated in the third frame period to be the distance to the corresponding pixel.

[0113] FIG. 8 is a diagram illustrating distance resolution in the first frame period and the third frame period in the second embodiment. FIG. 8(a) shows the correspondence between the measured distance and distance resolution in the first frame period, with the horizontal axis representing the measured distance and the vertical axis representing the distance resolution. In the first frame period, as shown in the first frame of Figures 7(a) and 7(b), as the amount of charge Q3 accumulated in the charge accumulation unit CS3 decreases compared to the amount of charge Q2 accumulated in the charge accumulation unit CS2, the distance resolution increases and the accuracy of distance measurement decreases, as shown in the distance range of 4 m to 5 m on curve 411 shown in Figure 8(a).

[0114] On the other hand, Fig. 8(b) shows the relationship between distance and distance resolution when the accumulation drive signals TX1, TX2, TX3, and TX4 are each advanced by time To / 2 relative to the first frame period in the third frame period. In Fig. 8(b), the horizontal axis represents the measured distance, and the vertical axis represents the distance resolution. As shown in the third frame period of Figures 7(a) and 7(b), by accelerating each of the storage drive signals TX1, TX2, TX3, and TX4, the ratio α of the amount of charge Q3 stored in the charge storage unit CS3 to the amount of charge Q2 stored in the charge storage unit CS2 approaches 1 / 2. As a result, as shown in the third frame of Figures 7(a) and 7(b), the amount of charge Q3 accumulated in the charge accumulation unit CS3 approaches and becomes equal to the amount of charge Q2 accumulated in the charge accumulation unit CS2, so that the distance resolution is reduced and the accuracy of distance measurement is not reduced, as shown in the distance range of 4 m to 5 m on curve 412 shown in Figure 8(b).

[0115] However, in the third frame period, the accumulation drive signals TX1, TX2, TX3, and TX4 are each advanced relative to the first frame period, so the amount of charge Q2 accumulated in the charge accumulation section CS2 decreases, as shown in FIG. 7(c). As shown in FIG. 8(b), the distance resolution at a distance of 6 m to 8 m becomes larger than the distance resolution at a distance of 6 m to 8 m in FIG. 8(a), and the accuracy of distance measurement decreases. In the case of Figure 7(c), as already explained, the charge amount ratio α is outside a predetermined threshold range, for example, outside the range of 1 / 2<α<2, so the distance calculation unit 42 determines the first measured distance obtained from the delay time Td calculated in the first frame period as the distance to the corresponding pixel.

[0116] As shown in each of Figures 8(a) and 8(b) above, the distance resolution changes depending on the distance to be measured, and by accelerating each of the accumulation drive signals TX1, TX2, TX3, and TX4, the distance resolution increases, and the area where the accuracy of distance measurement decreases also similarly advances. Therefore, by accelerating each of the accumulation drive signals TX1, TX2, TX3, and TX4 in accordance with the distance range in which the distance resolution decreases, the distance resolution increases and the distance measurement accuracy decreases, making it possible to adjust the distance range.

[0117] Then, the distance calculation unit 42 compares each of the ratios α and β with a threshold value to determine whether to use the first measured distance calculated in the first frame period or the third measured distance calculated in the third frame period as the distance between the distance image capturing device 1 and the subject S. 8(c), distance ranges 511 and 513 use the first measurement distance in the first frame period, and distance range 512 uses the third measurement distance in the third frame period. This allows the relationship between the decrease in accuracy of distance measurement due to an increase in distance resolution and the increase in measured distance to be nearly linear (solid curve 413), and the accuracy of measured distance can be improved compared to the conventional case where only the first frame period is used.

[0118] Furthermore, in two charge storage units CS used to calculate distance, for example, charge storage units CS2 and CS3, when the amount of charge Q2 stored in one charge storage unit CS2 approaches the minimum value due to the distribution of all of the charge generated in the photoelectric conversion element PD by the reflected light RL, the amount of charge Q3 stored in the other charge storage unit CS3 approaches the minimum value. Therefore, by setting the time for advancing each of the accumulation drive signals TX1, TX2, TX3, and TX4 to To / 2, the charge amounts Q2 and Q3 approach a charge amount ratio of 1:1, and the distance range with the lowest distance resolution can be changed to the distance range with the highest distance resolution. Therefore, the threshold value is most preferably set to 1 / 2, but since it is possible to improve distance resolution by setting it earlier, it may be set to any arbitrary value.

[0119] Furthermore, the number of times that the charge is distributed from the photoelectric conversion element PD to the charge accumulation section CS during the accumulation period of the third frame period may be reduced compared to the first frame period. That is, when the second measured distance in the third frame period is selected, it indicates that the distance to the subject S is short. Therefore, by reducing the amount of charge Q accumulated in the charge accumulation unit CS, the amount of accumulated charge can be prevented from exceeding the capacity of the charge accumulation unit CS, thereby improving the accuracy of measuring the distance between the distance image capturing device 1 and the subject S.

[0120] 9 is a flowchart showing an example of the operation of the distance image capturing device 1 of the second embodiment to calculate the distance between the distance image sensor 32 and the subject S. As in the first embodiment, the distance image capturing process is performed in two frame cycles, the first frame cycle and the third frame cycle.

[0121] Step S201: In the first frame, under the control of the measurement control unit 43, the pixel driving circuit 322 performs a process of distributing the charge from the photoelectric conversion element PD to each of the charge accumulation units CS1, CS2, CS3, and CS4. At this time, in each pixel circuit 321 (pixel unit), the pixel driving circuit 322 distributes charges in the corresponding accumulation periods of the charge accumulation units CS1, CS2, CS3, and CS4 according to the accumulation driving signals TX1, TX2, TX3, and TX4 of the first frame period shown in FIG. 3 or FIG. 7. As already mentioned, the number of times (the number of accumulation cycles) that charges are accumulated in each of the charge accumulation units CS1, CS2, CS3, and CS4 is multiple in the accumulation period of a frame cycle.

[0122] Step S202: When the accumulation period in the first frame cycle ends, the pixel driving circuit 322 outputs pixel signals, which are voltages corresponding to the amounts of charge Q1, Q2, Q3, and Q4 accumulated in the charge accumulation units CS1, CS2, CS3, and CS4, respectively, to the distance calculation unit 42. The distance calculation unit 42 calculates the delay time Td based on the pixel signals (corresponding to the amount of charge) supplied from the pixel circuits 321 provided for each pixel of the distance image sensor 32.

[0123] At this time, the distance calculation unit 42 compares the amounts of electric charge Q2 and Q4, and if the amount of electric charge Q2 is greater, calculates the delay time Td using the above formula (1). On the other hand, the distance calculation unit 42 compares the amounts of electric charge Q2 and Q4, and if the amount of electric charge Q4 is greater, calculates the delay time Td using the above formula (2). Then, the distance calculation unit 42 calculates the first measurement distance based on the calculated delay time Td.

[0124] Step S203: In the third frame, under the control of the measurement control unit 43, the pixel driving circuit 322 performs a process of distributing the charge from the photoelectric conversion element PD to each of the charge accumulation units CS1, CS2, CS3, and CS4. At this time, in each pixel circuit 321 (pixel unit), the pixel driving circuit 322 distributes charges to the charge accumulation units CS1, CS2, CS3, and CS4, respectively, using the accumulation driving signals TX1, TX2, TX3, and TX4 of the third frame period shown in FIG.

[0125] That is, the pixel driving circuit 322 distributes charges in the corresponding accumulation periods of the charge accumulation units CS1, CS2, CS3, and CS4 using the accumulation driving signals TX1, TX2, TX3, and TX4 that are advanced (advanced) by a predetermined time relative to the first frame period. Here, similarly to the first embodiment, there are a plurality of allocations (number of accumulation cycles) for accumulating charges in each of the charge accumulation units CS1, CS2, CS3, and CS4 during the accumulation period of a frame cycle.

[0126] Step S204: When the accumulation period in the third frame cycle ends, the pixel driving circuit 322 outputs pixel signals, which are voltages corresponding to the amounts of charge Q1, Q2, Q3, and Q4 accumulated in the charge accumulation units CS1, CS2, CS3, and CS4, respectively, to the distance calculation unit 42. The distance calculation unit 42 calculates the delay time Td based on the pixel signals (ie, the amount of charge) supplied from each of the pixel circuits 321 provided for each pixel of the distance image sensor 32.

[0127] At this time, the distance calculation unit 42 compares the amounts of electric charge Q2 and Q4, and if the amount of electric charge Q2 is greater, calculates the delay time Td using the above formula (5). On the other hand, the distance calculation unit 42 compares the electric charge amounts Q2 and Q4, and if the electric charge amount Q4 is greater, calculates the delay time Td using the following equation (6). Td=To+To×(Q4-Q1) / (Q3+Q4-2×Q1)-Ra×To…(6) Here, the coefficient Ra is a coefficient obtained by dividing the advanced time by the pulse width To. Then, the distance calculation unit 42 calculates the third measurement distance based on the calculated delay time Td.

[0128] Step S205: The distance calculation unit 42 selects one of the pixel circuits 321 for which processing has not been completed as the pixel to be processed, compares the charge amount Q2 with the charge amount Q4 in the first frame period in that pixel circuit 321, and determines whether the charge amount Q2 exceeds the charge amount Q4. At this time, if the amount of electric charge Q2 exceeds the amount of electric charge Q4, the distance calculation unit 42 advances the process to step S206. On the other hand, if the amount of electric charge Q2 is equal to or less than the amount of electric charge Q4, the distance calculation unit 42 advances the process to step S207.

[0129] Step S206: The distance calculation unit 42 obtains the charge amount ratio α by dividing the charge amount Q2 by the charge amount Q3. Then, the distance calculation unit 42 compares the charge amount ratio α with a preset threshold range, and determines whether the charge amount ratio α is within the threshold range. At this time, if the charge amount ratio α is within the threshold range, the distance calculation unit 42 advances the process to step S208. On the other hand, if the charge amount ratio α is outside the threshold range, the distance calculation unit 42 advances the process to step S209.

[0130] Step S207: The distance calculation unit 42 obtains the charge amount ratio β by dividing the charge amount Q2 by the charge amount Q3. Then, the distance calculation unit 42 compares the charge amount ratio β with a preset threshold range, and determines whether the charge amount ratio β is within the threshold range. At this time, if the charge amount ratio β is within the threshold range, the distance calculation unit 42 advances the process to step S208. On the other hand, if the charge amount ratio β is outside the threshold range, the distance calculation unit 42 advances the process to step S209.

[0131] Step S208: Since the charge amount ratio α or the charge amount ratio β is within the threshold range, the distance calculation unit 42 determines that the resolution of the first measured distance meets the specified accuracy, and sets the first measured distance as the distance between the distance image capturing device 1 and the subject S at the pixel to be processed.

[0132] Step S209: Since the charge amount ratio α or the charge amount ratio β is outside the threshold range, the distance calculation unit 42 determines that the resolution of the first measured distance does not meet the specified accuracy, and sets the third measured distance as the distance between the distance image capturing device 1 and the subject S at the pixel to be processed.

[0133] Step S210: The distance calculation unit 42 determines whether the process of calculating the distance between the distance image pickup device 1 and the subject S has been performed for all pixel circuits 321, that is, whether calculation of the distance for all pixels in the distance image sensor 32 has been completed. At this time, if the distance calculation unit 42 has finished calculating the distances of all pixels in the distance image sensor 32, the process proceeds to step S211. On the other hand, if the distance calculation unit 42 has not yet finished calculating the distances of all pixels in the distance image sensor 32, the process proceeds to step S205.

[0134] Step S211: When the distance calculation unit 42 determines that each of all pixels in the distance image sensor 32 is the first measurement distance or the third measurement distance, it outputs an image having the distance between the distance image capturing device 1 and the subject S for each pixel to an external device (including the process of writing to an external storage device), and proceeds to step S201.

[0135] <Third embodiment> A third embodiment of the present invention will now be described with reference to the drawings. The configuration of the range image capturing device 1 of the third embodiment is the same as that of the first embodiment shown in Fig. 1. In the following, only the operations of the third embodiment that are different from those of the first embodiment will be described. Also, unlike the first embodiment, the third embodiment uses three frame periods, namely a first frame period, a second frame period, and a third frame period, to capture one captured image.

[0136] In addition, in this embodiment, one distance image is captured using three frame periods, with the first frame period being the central frame period, the third frame period in the second embodiment being the frame period immediately preceding the first frame period, and the second frame period in the first embodiment being the frame period immediately following the first frame period. For example, when the charge generated by the reflected light RL is distributed from the photoelectric conversion element PD to each of the charge storage unit CS2 and the charge storage unit CS3, the distance calculation unit 42 calculates the charge amount ratio α by dividing the charge amount Q2 accumulated in the charge storage unit CS2 by the charge amount Q3 accumulated in the charge storage unit CS3.

[0137] Then, when the charge amount ratio α obtained by dividing the charge amount Q2 accumulated in the charge accumulation unit CS2 in the first frame period by the charge amount Q3 accumulated in the predetermined charge accumulation unit CS3 exceeds 1 / 2 and is less than 2 (1 / 2<α<2), the distance calculation unit 42 determines the first measured distance calculated from the delay time Td in the first frame as the distance of the corresponding pixel, as in the first and second embodiments. Furthermore, when the charge amount ratio α obtained by dividing the charge amount Q2 accumulated in the charge accumulation unit CS2 in the first frame period by the charge amount Q3 accumulated in the predetermined charge accumulation unit CS3 is 1 / 2 or less (α≦1 / 2), the distance calculation unit 42 determines the second measured distance obtained from the delay time Td calculated in the second frame as the measured distance, as in the first embodiment. Furthermore, when the charge amount ratio α obtained by dividing the charge amount Q2 accumulated in the charge accumulation unit CS2 in the first frame period by the charge amount Q3 accumulated in the predetermined charge accumulation unit CS3 is 2 or greater (α≧2), the distance calculation unit 42 determines the third measured distance calculated from the delay time Td in the third frame as the measured distance, as in the second embodiment.

[0138] Furthermore, for example, when the charge generated by the reflected light RL is distributed from the photoelectric conversion element PD to each of the charge storage unit CS3 and the charge storage unit CS4, the distance calculation unit 42 calculates the charge amount ratio β by dividing the charge amount Q3 accumulated in the charge storage unit CS3 by the charge amount Q4 accumulated in the charge storage unit CS4. Then, if the charge amount ratio β obtained by dividing the charge amount Q3 accumulated in the charge accumulation unit CS3 in the first frame period by the charge amount Q4 accumulated in the predetermined charge accumulation unit CS4 is within a predetermined threshold range, for example, greater than 1 / 2 and less than 2 (1 / 2<β<2), the distance calculation unit 42 determines the first measured distance obtained from the delay time Td calculated in the first frame period to be the distance of the corresponding pixel.

[0139] In addition, if the charge amount ratio β obtained by dividing the charge amount Q3 accumulated in the charge accumulation unit CS3 in the first frame period by the charge amount Q4 accumulated in the predetermined charge accumulation unit CS4 is outside the above threshold range, or is 1 / 2 or less (β≦1 / 2), the distance calculation unit 42 determines the second measured distance calculated in the second frame to be the measured distance, as in the first embodiment. Furthermore, when the charge amount ratio β obtained by dividing the charge amount Q3 accumulated in the charge accumulation unit CS3 in the first frame period by the charge amount Q4 accumulated in the predetermined charge accumulation unit CS4 is 2 or greater (β≧2), the distance calculation unit 42 determines the third measured distance calculated in the third frame as the measured distance, as in the second embodiment.

[0140] Then, the distance calculation unit 42 compares each of the ratios α and β with the threshold range to determine which of the first measured distance calculated in the first frame period, the second measured distance calculated in the second frame period, and the third measured distance calculated in the third frame period to use as the distance between the distance image capturing device 1 and the subject S. 5(c), distance ranges 501 and 503 use the first measurement distance in the first frame period, and distance ranges 502 and 504 use the second measurement distance in the second frame period. This allows the relationship between the decrease in accuracy of distance measurement due to an increase in distance resolution and the increase in the measured distance to be nearly linear (solid curve 403), and the accuracy of the measured distance can be improved compared to the conventional case where only the first frame period is used.

[0141] 8(c), distance ranges 511 and 513 use the first measurement distance in the first frame period, and distance range 502 uses the third measurement distance in the second frame period. This allows the relationship between the decrease in accuracy of distance measurement due to an increase in distance resolution and the increase in the measured distance to be nearly linear (solid curve 413), and the accuracy of the measured distance can be improved compared to the conventional case where only the first frame period is used.

[0142] Furthermore, in two charge storage units CS used to calculate the distance, for example, charge storage units CS2 and CS3, when the amount of charge Q2 stored in one charge storage unit CS2 approaches 0 (minimum value), the amount of charge Q3 stored in the other charge storage unit CS3 approaches the maximum value. When all of the charges generated in the photoelectric conversion element PD by the reflected light RL are distributed to one charge storage section CS2 (when the charge amount Q2 approaches its maximum value), the charge amount Q3 accumulated in the other charge storage section CS3 approaches its minimum value.

[0143] Therefore, by delaying or advancing each of the accumulation drive signals TX1, TX2, TX3, and TX4 by a time To / 2, the charge amounts Q2 and Q3 approach a charge amount ratio of 1:1, and the distance range with the highest distance resolution and lowest measurement accuracy can be changed to the distance range with the lowest distance resolution and highest measurement accuracy. Therefore, the most desirable value for the threshold value is 1 / 2, but since it is possible to improve the distance resolution by delaying or advancing each of the accumulation drive signals TX1, TX2, TX3, and TX4 in the second frame period, the threshold value may be set to any arbitrary value.

[0144] <Fourth embodiment> A fourth embodiment of the present invention will be described below with reference to the drawings. The configuration of the range image capturing device 1 of the fourth embodiment is the same as that of the first embodiment shown in Fig. 1. In the following, only the operations of the fourth embodiment that differ from those of the first embodiment will be described. Also, in the fourth embodiment, two frame periods are used to capture one captured image, similar to the first embodiment. However, in the fourth embodiment, a distance image is captured by selecting, according to predetermined conditions, from two frame periods, the first frame period and the second frame period, two frame periods, the first frame period and the second frame period (first embodiment), and two frame periods, the first frame period and the third frame period (second embodiment).

[0145] In the range image sensor 32, a pixel region for determining whether to select a frame period, that is, a region of the pixel circuits 321 (for example, the central pixel region) is set in advance as a determination region. This determination region is arbitrarily set by the user as a region in the pixel circuit 321 where particularly high-precision measurement is desired. Then, measurement control section 43 controls timing control section 41 so that pixel drive circuit 322 operates in the first frame period of the two frame periods in which a distance image is captured, as the first frame. The pixel driving circuit 322 supplies the accumulation driving signals TX1, TX2, TX3, and TX4 in the first frame period to each of the pixel circuits 321, and distributes the charges generated by the incident light from the photoelectric conversion element PD to the charge accumulation sections CS1, CS2, CS3, and CS4, respectively.

[0146] Hereinafter, a case will be described in which the charges generated by the photoelectric conversion element PD in response to the reflected light RL are distributed to the charge accumulation units CS2 and CS3. The distance calculation unit 42 divides the amount of charge Q2 accumulated in the charge accumulation unit CS2 by the amount of charge Q3 accumulated in the charge accumulation unit CS3 in each of the pixel circuits 321 in the above-mentioned determination area to calculate the ratio α (similar to the first embodiment). Then, the distance calculation unit 42 calculates the average value of the ratios α of the pixel circuits 321 in the determination region, and sets the average value as the average ratio α'.

[0147] Here, when the average ratio α' is greater than 1 / 2 and less than 2 (1 / 2<α<2), the distance calculation unit 42 calculates the first measured distance obtained from the delay time Td in the first frame period, as in the first and second embodiments. Furthermore, distance calculation unit 42 outputs to measurement control unit 43 a control signal indicating that average ratio α′ is 1 / 2<α<2. As a result, measurement control section 43 controls timing control section 41 so that pixel drive circuit 322 operates in the second frame period during the two frames for capturing a distance image.

[0148] The distance calculation unit 42 calculates a first measured distance determined from the delay time Td in the first frame period in the second frame period. If this average ratio α' is 1 / 2<α'<2, the distance calculation unit 42 determines the first measured distance calculated in either the first first frame period or the second first frame period, or the average value of the first measured distances calculated in two first frame periods, as the distance between the distance image capturing device 1 and the subject S.

[0149] On the other hand, when the average ratio α' is 1 / 2 or less (α'≦1 / 2), the distance calculation unit 42 calculates the first measured distance obtained from the delay time Td in the first frame period, as in the first and second embodiments. Further, the distance calculation unit 42 outputs a control signal to the measurement control unit 43 indicating that the average ratio α′ is α′≦½.

[0150] As a result, measurement control section 43 controls timing control section 41 so that pixel drive circuit 322 operates in the second frame period during the two frames for capturing a distance image. The distance calculation unit 42 calculates a second measured distance determined from the delay time Td in the second frame period in the second frame period. If the average ratio α' is α'≦½, the distance calculation unit 42 determines the second measured distance calculated in the second second frame period as the distance between the distance image pickup device 1 and the subject S.

[0151] Furthermore, when the average ratio α' is 2 or more (α≧2), the distance calculation unit 42 calculates the first measured distance determined from the delay time Td in the first frame period, similarly to the first and second embodiments. Furthermore, the distance calculation unit 42 outputs a control signal to the measurement control unit 43 indicating that the average ratio α′ satisfies α′≧2.

[0152] As a result, measurement control section 43 controls timing control section 41 so that pixel drive circuit 322 operates with the second frame period as the third frame period in two frames for capturing a distance image. The distance calculation unit 42 calculates a third measured distance determined from the delay time Td in the third frame period in the second frame period. If the average ratio α' satisfies α'≧2, the distance calculation unit 42 determines the third measured distance calculated in the second third frame period as the distance between the distance image pickup device 1 and the subject S.

[0153] Furthermore, in each of the pixel circuits 321 in the above-mentioned judgment region, when the charge generated in the photoelectric conversion element PD by the reflected light RL is distributed to the charge accumulation units CS3 and CS4, the distance calculation unit 42 divides the amount of charge Q3 accumulated in the charge accumulation unit CS3 by the amount of charge Q4 accumulated in the charge accumulation unit CS4 to calculate the ratio β (similar to the first embodiment). Further, the distance calculation unit 42 calculates the average value of the ratios β of the pixel circuits 321 in the determination region in the first frame period during two frame periods in which the captured image is captured, and sets this as the average ratio β'.

[0154] Then, the distance calculation unit 42 performs a process of comparing the average ratio β′ with a threshold value, similar to the case of the average ratio α′, to calculate the first measured distance, and outputs the comparison result to the measurement control unit 43. As a result, the measurement control unit 43 controls the timing control unit 41 so that in the two frames in which a distance image is captured, the second frame period is set as the second frame period or the third frame period and the pixel driving circuit 322 operates. Then, the distance calculation unit 42 calculates the distance between the distance image pickup device 1 and the subject S in the distance image, in the same way as in the case of the average ratio α'.

[0155] According to the present embodiment, with the above-described configuration, in the first frame period, which is the first of the two frame periods, if operation based on the first frame period is selected as the second frame period in response to the comparison result between the average ratio (α' or β') of the amount of charge Q accumulated in the charge accumulation unit CS and the threshold, the first measurement distance in the first frame period is adopted in distance ranges 501 and 503, while if operation based on the second frame period is selected as the second frame period, the second measurement distance in the second frame period is adopted in distance ranges 502 and 504. This makes it possible to achieve a nearly linear relationship (solid curve 403) between the decrease in accuracy of distance measurement due to an increase in distance resolution and the increase in the measured distance, and thereby improve the accuracy of the measured distance compared to the conventional case where only the first frame period is used.

[0156] Similarly, according to this embodiment, in the first frame period, which is the first of the two frame periods, if operation based on the first frame period is selected as the second frame period in response to the comparison result between the average ratio (α' or β') of the amount of charge Q accumulated in the charge accumulation unit CS and the threshold, the first measurement distance in the first frame period is adopted in distance ranges 511 and 513, as shown in Fig. 8(c), and if operation based on the third frame period is selected as the second frame period, the third measurement distance in the third frame period is adopted in distance range 502. This makes it possible to achieve a nearly linear relationship (solid curve 413) between the decrease in accuracy of distance measurement due to an increase in distance resolution and the increase in the measured distance, and thereby improve the accuracy of the measured distance compared to the conventional case where only the first frame period is used.

[0157] Furthermore, in two charge storage units CS used to calculate the distance, for example, charge storage units CS2 and CS3, when the amount of charge Q2 stored in one charge storage unit CS2 approaches 0 (minimum value), the amount of charge Q3 stored in the other charge storage unit CS3 approaches the maximum value. When all of the charges generated in the photoelectric conversion element PD by the reflected light RL are distributed to one charge storage section CS2 (when the charge amount Q2 approaches its maximum value), the charge amount Q3 accumulated in the other charge storage section CS3 approaches its minimum value.

[0158] Therefore, by delaying or advancing each of the accumulation drive signals TX1, TX2, TX3, and TX4 by To / 2, in each of the pixel circuits 321 included in the determination area requiring higher measurement accuracy, the charge amounts Q2 and Q3 (or charge amounts Q3 and Q4) approach a charge amount ratio of 1:1, and the distance range with the lowest distance resolution can be changed to the distance range with the highest distance resolution. Therefore, the most desirable value for the threshold value is 1 / 2, but the threshold value may be set to any arbitrary value, since it is possible to improve the distance resolution by delaying or advancing each of the accumulation drive signals TX1, TX2, TX3, and TX4 in the second frame period depending on the result of the judgment in the first frame period of the two frame periods in which the captured image is captured. [Explanation of symbols]

[0159] 1...Distance image capturing device 2...Light source section 3...Light receiving section 21...Light source device 22...Diffuser 31...Lens 32...Distance image sensor (distance image sensor) 321...Pixel circuit 322...Pixel driving circuit 4...Distance image processing section 41...Timing control section 42...Distance calculation section 43...Measurement control section CS1, CS2, CS3, CS4...Charge storage section FD1, FD2, FD3, FD4...Floating diffusion G1, G2, G3, G4...Read gate transistors GD: Charge drain transistor PD...photoelectric conversion element PO...light pulse RL…Reflected light RT1, RT2, RT3, RT4...Reset transistors S…Subject SF1, SF2, SF3, SF4...Source follower transistors SL1, SL2, SL3, SL4...Selection transistors

Claims

1. a plurality of pixel circuits each including a photoelectric conversion element that generates an electric charge according to incident light that is light that is incident from a measurement space that is a space to be measured, N (N≧3) charge accumulation units that accumulate the electric charge in a frame period, and a transfer transistor that transfers the electric charge from the photoelectric conversion element to each of the charge accumulation units; a pixel drive circuit that performs on / off processing of each of the transfer transistors in each of the charge accumulation units at a predetermined accumulation period corresponding to irradiation of a light pulse, and distributes and accumulates the charges; a light receiving unit having a measurement control unit that causes the pixel drive circuit to turn on and off each of the transfer transistors in a first accumulation period synchronized with the irradiation of the light pulse in a first frame period in the frame period, and to turn on and off each of the transfer transistors in other accumulation periods shifted by a predetermined time that is equal to or less than half of the irradiation time of the light pulse in other frame periods in the frame period, thereby distributing the charges to the charge accumulation units; a distance calculation unit that calculates a distance to a subject present in the measurement space as a first measurement distance and another measurement distance in each of the first frame period and the other frame period based on the amount of charge accumulated in each of the charge accumulation units, and determines whether the first measurement distance or the other measurement distance is to be the distance between the distance image pickup device in each of the pixel circuits and the subject, depending on the amount of charge accumulated in the charge accumulation units, based on whether a ratio of the amount of charge accumulated in each of the first charge accumulation unit and the second charge accumulation unit, which are the charge accumulation units to which charge caused by reflected light of the light pulse is sequentially allocated from the photoelectric conversion element, is close to 1:1 in each of the first frame period and the other frame period; Equipped with A distance image capturing device characterized by:

2. The measurement control unit When the second accumulation period, which is the other accumulation period, is delayed by a predetermined time with respect to the first accumulation period, The distance calculation unit In the first frame period, when a ratio of the amount of charge accumulated in each of the first charge accumulation section and the second charge accumulation section, which are the charge accumulation sections to which the charge due to the reflected light of the light pulse from the subject is sequentially allocated from the photoelectric conversion element during the first accumulation period, is outside a preset range, the second measurement distance, which is the other measurement distance, is set as the distance.

2. The distance imaging device according to claim 1,

3. The measurement control unit The number of times that the pixel driving circuit distributes signals from the photoelectric conversion elements in a second frame period, which is the other frame period, is made larger than the number of times that the pixel driving circuit distributes signals in the first frame period.

3. The distance imaging device according to claim 2.

4. The measurement control unit When the third accumulation period, which is the other accumulation period, is advanced by a predetermined time relative to the first accumulation period, The distance calculation unit In the first frame period, when a ratio of the amount of charge accumulated in each of the first charge accumulation section and the second charge accumulation section, which are the charge accumulation sections to which the charge due to the reflected light of the light pulse from the subject is sequentially allocated from the photoelectric conversion element during the first accumulation period, is outside a preset range, the third measurement distance, which is the other measurement distance, is set as the distance.

2. The distance imaging device according to claim 1,

5. The measurement control unit The number of times that the pixel driving circuit distributes the signals from the photoelectric conversion elements in a third frame period, which is another frame period, is set to be smaller than the number of times that the pixel driving circuit distributes the signals in the first frame period.

5. The distance imaging device according to claim 4.

6. The measurement control unit a second frame period of a second accumulation period that is delayed by a predetermined time with respect to the first accumulation period of the first frame period, and a third frame period of a third accumulation period that is advanced by a predetermined time with respect to the first accumulation period of the first frame period, and in each of the first frame period, the second frame period, and the third frame period, charges are distributed to each of the charge storage units by the pixel drive circuit; The distance calculation unit calculating a distance to a subject present in the measurement space as a first measurement distance, a second measurement distance, and a third measurement distance in each of the first frame period, the second frame period, and the third frame period based on the amount of charge accumulated in each of the charge accumulation units, and determining whether the distance in each pixel circuit should be the first measurement distance, the second measurement distance, or the third measurement distance according to the amount of charge accumulated in the charge accumulation unit; 2. The distance imaging device according to claim 1,

7. The distance calculation unit In the first frame period, when a ratio of the amount of charge accumulated in each of the first charge accumulation section and the second charge accumulation section, which are the charge accumulation sections to which the charge caused by the reflected light of the light pulse from the subject is sequentially allocated from the photoelectric conversion element during the first accumulation period, is equal to or greater than a predetermined first threshold, the third measured distance is set to the distance, and when the ratio is equal to or less than a predetermined second threshold, the second measured distance is set to the distance.

7. The distance imaging device according to claim 6,

8. The measurement control unit In the first frame period, the transfer transistors are turned on and off in a first accumulation period synchronized with the irradiation of the light pulse, and a determination is made as to whether the other accumulation period in the other frame period should be delayed or advanced by a predetermined time with respect to the first accumulation period, depending on the amount of charge accumulated in the charge accumulation section in the pixel circuit of the determination region in the first frame period.

2. The distance imaging device according to claim 1,

9. The measurement control unit In each of the two charge accumulation units in the pixel circuit of the determination region, in each of consecutive first accumulation periods in the first frame period, if a ratio between an amount of charge accumulated in the first charge accumulation unit in a previous first accumulation period and an amount of charge accumulated in the second charge accumulation unit in a subsequent first accumulation period is equal to or less than a predetermined first threshold, the other accumulation period is delayed with respect to the first accumulation period, and if the ratio is equal to or greater than a predetermined second threshold, the other accumulation period is advanced with respect to the first accumulation period.

9. The distance imaging device according to claim 8.

10. A distance image capturing method for controlling a distance image capturing device including a plurality of pixel circuits, each of which includes a photoelectric conversion element, a plurality of charge accumulation units, and a transfer transistor, a pixel drive circuit, a distance calculation unit, and a measurement control unit, a step in which the pixel driving circuit allocates and accumulates charges generated by the photoelectric conversion elements in response to incident light from a measurement space in each of N (N≧3) charge accumulation units in a frame period by performing on / off processing on the transfer transistors that transfer the charges from the photoelectric conversion elements to the charge accumulation units, at a predetermined accumulation period synchronized with irradiation of a light pulse; a step in which the measurement control unit causes the pixel drive circuit to turn on and off each of the transfer transistors in a first accumulation period synchronized with the irradiation of the light pulse in a first frame period in the frame period, and to turn on and off each of the transfer transistors in other accumulation periods shifted by a predetermined time that is equal to or less than half of the irradiation time of the light pulse in other frame periods in the frame period, thereby distributing the charges to the charge accumulation units; a step in which the distance calculation unit calculates a distance to a subject present in the measurement space as a first measurement distance and another measurement distance in each of the first frame period and the other frame period based on the amount of charge accumulated in each of the charge accumulation units of the distance calculation unit, and determines whether the distance in each pixel circuit is the first measurement distance or the other measurement distance according to the amount of charge accumulated in the charge accumulation units, depending on whether a ratio of the amount of charge accumulated in each of the first charge accumulation unit and the second charge accumulation unit, which are the charge accumulation units to which charge caused by reflected light of the light pulse is sequentially allocated from the photoelectric conversion element in each of the first frame period and the other frame period, is close to 1:1; Contains A distance image capturing method comprising:

Images