Distance image capturing device, and distance image capturing method

Abstract

To provide a distance image photographing device and distance image photographing method capable of measuring a distance to an object at a short distance and determining presence / absence of an object at a long distance.SOLUTION: A distance image photographing device includes a light source, a light reception section, an electric charge discharge section, and a distance image processing section. The distance image processing section executes unit accumulation processing a plurality of times in one frame period, and determines a measurement distance to a subject with the use of electric charge amounts accumulated respectively in three or more electric charge accumulation sections. The unit accumulation processing comprises: a short-distance object measurement process for allowing a part of the electric charge accumulation section among the three or more electric charge accumulation sections to distribute and accumulate electric charges corresponding to reflection light being an optical pulse reflected against the subject; a discharge process for allowing the electric charge discharge sections to discharge the electric charges generated by photoelectric conversion elements; and a long-distance object measurement process for allowing the electric charge accumulation sections including the electric charge accumulation section different from the part of the electric charge accumulation sections to accumulate the electric charges corresponding to the reflection light. In the unit accumulation processing, the processes are sequentially executed.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a distance image capturing device and a distance image capturing method.

Background Art

[0002] Conventionally, as a technique for measuring the distance to an object, there is a technique for measuring the flight time of an optical pulse. Such a technique is called Time of Flight (hereinafter referred to as ToF). In ToF, the fact that the speed of light is known is utilized, and an object is irradiated with an optical pulse in the near-infrared region. Then, the time difference between the time when the optical pulse is irradiated and the time when the reflected light reflected by the object is received is measured. Based on this time difference, the distance to the object is calculated. A distance measuring sensor (ToF sensor) for detecting light for measuring the distance using a photodiode (photoelectric conversion element) has been put into practical use.

[0003] In recent years, a distance measuring sensor that can obtain not only the distance to an object but also the depth information for each pixel in a two-dimensional image including the object, that is, three-dimensional information about the object, has been put into practical use. Such a distance measuring sensor is also called a distance image capturing device. In a distance image capturing device, a plurality of pixels including photodiodes are arranged in a two-dimensional matrix on a silicon substrate, and the reflected light reflected by the object is received on this pixel surface. In a distance image capturing device, by outputting a photoelectric conversion signal based on the amount of light (charge) received by each pixel for one image, a two-dimensional image including the object and distance information for each pixel constituting this image can be obtained. For example, Patent Document 1 discloses a technique in which three charge storage parts are provided in one pixel, and the distance is calculated based on the amount of charge stored in each charge storage part.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In a ToF sensor, when improving the measurement accuracy in measuring the distance to an object at a short distance, it is conceivable to shorten the irradiation time of the optical pulse and the accumulation time for accumulating charges in the charge accumulation unit. However, in this case, the measurable distance range becomes shorter. As a countermeasure, in order to extend the measurable distance range while maintaining the relationship between the irradiation time and the accumulation time for improving the measurement accuracy, it is conceivable to increase the number of charge accumulation units. However, due to the constraint on the area occupied by one pixel, there is a limit to the number of charge accumulation units, and it has been difficult to increase the measurable distance range while increasing the measurement accuracy for an object at a short distance. In particular, when using a distance image capturing device mounted on a moving object with a high moving speed to measure the distance to an object around the moving object, it is necessary to control the moving object, such as not only measuring the distance to an object at a short distance but also sensing the presence or absence of an object at a long distance that relatively approaches due to the movement of the moving object and taking an avoidance action. For this reason, there is a demand for a distance image capturing device that can accurately measure the distance to an object at a short distance and also determine the presence or absence of an object at a long distance.

[0006] The present invention has been made based on the above problems, and an object thereof is to provide a distance image capturing device and a distance image capturing method capable of measuring the distance to an object at a short distance and determining the presence or absence of an object at a long distance.

Means for Solving the Problems

[0007] The distance image capturing device of the present invention includes a light source unit that irradiates a measurement space where a subject exists with light pulses, a photoelectric conversion element that generates charges in response to the incident light, and a pixel including three or more charge storage units that store the charges, a pixel driving circuit that distributes and stores the charges in each of the charge storage units in the pixel at a predetermined timing synchronized with the irradiation of the light pulses, a charge discharging unit that discharges the charges generated by the photoelectric conversion element, and a distance image processing unit that determines a measurement distance to the subject using the amount of charge stored in each of the charge storage units. The distance image processing unit performs a plurality of unit accumulation processes in one frame period, determines the measurement distance to the subject using the amount of charge stored in each of the three or more charge storage units, and in the unit accumulation process, a short-distance object measurement process of distributing and storing charges corresponding to the reflected light, which is the light pulse reflected by the subject, in some of the three or more charge storage units, a discharging process of discharging the charges generated by the photoelectric conversion element by the charge discharging unit, and a long-distance object measurement process of storing the charges corresponding to the reflected light in charge storage units including charge storage units different from the some charge storage units are sequentially performed.

[0008] The distance image capturing method of the present invention includes a light source unit that irradiates a measurement space where a subject exists with light pulses, a photoelectric conversion element that generates charges according to the incident light, and three or more charge storage units that store the charges, a pixel including the above, and a pixel driving circuit that distributes and stores the charges to each of the charge storage units in the pixel at a predetermined timing synchronized with the irradiation of the light pulses. A light receiving unit having the above, a charge discharging unit that discharges the charges generated by the photoelectric conversion element, and a distance image processing unit that determines the measurement distance to the subject using the amount of charges stored in each of the charge storage units. A distance image capturing method by a distance image capturing device including the above, wherein the distance image processing unit performs a plurality of unit accumulation processes in one frame period, and determines the measurement distance to the subject using the amount of charges stored in each of the three or more charge storage units. In the unit accumulation process, a short-distance object measurement process of distributing and storing charges corresponding to the reflected light, which is the light pulse reflected by the subject, to some of the three or more charge storage units, a discharging process of discharging the charges generated by the photoelectric conversion element by the charge discharging unit, and a long-distance object measurement process of storing the charges corresponding to the reflected light in a charge storage unit including a charge storage unit different from the some charge storage units are sequentially performed.

Effects of the Invention

[0009] According to the present invention, it is possible to measure the distance to an object at a short distance and determine the presence or absence of an object at a long distance.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Modes for Carrying Out the Invention

[0011] Hereinafter, the distance image capturing device according to the embodiment will be described with reference to the drawings.

[0012] (Basic Configuration of the Embodiment) First, the basic configuration of the embodiment will be described. FIG. 1 is a block diagram showing the schematic configuration of the distance image capturing device according to the embodiment of the present invention. The distance image capturing device 1 having the configuration shown in FIG. 1 includes a light source unit 2, a light receiving unit 3, and a distance image processing unit 4. FIG. 1 also shows a subject OB which is an object to be measured for distance in the distance image capturing device 1.

[0013] The light source unit 2 irradiates the light pulse PO onto the imaging target space where the subject OB to be measured for distance exists in the distance image capturing device 1 in accordance with the control from the distance image processing unit 4. The light source unit 2 is, for example, a surface emitting semiconductor laser module such as a vertical cavity surface emitting laser (VCSEL). The light source unit 2 includes a light source device 21 and a diffuser plate 22.

[0014] The light source device 21 is a light source that emits laser light in the near-infrared wavelength band (for example, the wavelength band with a wavelength of 850 nm to 940 nm) that becomes the light pulse PO for irradiating the subject OB. The light source device 21 is, for example, a semiconductor laser emitting element. The light source device 21 emits pulsed laser light in response to the control from the timing control unit 41.

[0015] The diffuser plate 22 is an optical component that diffuses the laser light in the near-infrared wavelength band emitted by the light source device 21 to the area of the surface for irradiating the subject OB. The pulsed laser light diffused by the diffuser plate 22 is emitted as the light pulse PO and irradiates the subject OB.

[0016] The light receiving unit 3 receives the reflected light RL of the light pulse PO reflected by the subject OB to be measured for distance in the distance image capturing device 1, and outputs a pixel signal corresponding to the received reflected light RL. The light receiving unit 3 includes a lens 31 and a distance image sensor 32.

[0017] The lens 31 is an optical lens that guides the incident reflected light RL to the distance image sensor 32. The lens 31 emits the incident reflected light RL toward the distance image sensor 32 side and causes it to be received (incident) by the pixels provided in the light receiving area of the distance image sensor 32.

[0018] The distance image sensor 32 is an imaging device used in the distance image capturing device 1. The distance image sensor 32 includes a plurality of pixels in a two-dimensional light-receiving area. In each pixel of the distance image sensor 32, one photoelectric conversion element, a plurality of charge storage parts corresponding to this one photoelectric conversion element, and a component for distributing charges to each charge storage part are provided. That is, the pixel is an imaging device with a distribution structure that distributes and accumulates charges in a plurality of charge storage parts.

[0019] The distance image sensor 32 distributes the charges generated by the photoelectric conversion element to each charge storage part according to the control from the timing control unit 41. Further, the distance image sensor 32 outputs a pixel signal corresponding to the amount of charge distributed to the charge storage part. In the distance image sensor 32, a plurality of pixels are arranged in a two-dimensional matrix, and a pixel signal for one frame corresponding to each pixel is output.

[0020] The distance image processing unit 4 controls the distance image capturing device 1 and calculates the distance to the subject OB. The distance image processing unit 4 includes a timing control unit 41, a distance calculation unit 42, a measurement control unit 43, and a storage unit 44. Note that a part of the functional units (the timing control unit 41, the distance calculation unit 42, the measurement control unit 43, and the storage unit 44) of the distance image processing unit 4 may be incorporated into the distance image sensor 32.

[0021] The timing control unit 41 controls the timing for outputting various control signals required for measurement according to the control of the measurement control unit 43. The various control signals here are, for example, a signal for controlling the irradiation of the optical pulse PO, a signal for distributing and accumulating the reflected light RL to a plurality of charge storage parts, a signal for controlling the number of distribution times (accumulation times) per frame, etc. The number of distribution times is the number of times the process of distributing charges to the charge storage part CS (see FIG. 3) is repeated.

[0022] The distance calculation unit 42 calculates the ToF distance using the pixel signal output from the distance image sensor 32 and the following formula (1). The distance calculation unit 42 corrects the calculated ToF distance using the correction information 440, and sets the corrected ToF distance as the distance to the subject OB (measured distance). The correction information 440 and the method by which the distance calculation unit 42 corrects the ToF distance using the correction information 440 will be described in detail later.

[0023] The measurement control unit 43 controls the timing control unit 41. For example, the measurement control unit 43 sets the number of allocations per frame, the accumulation time Ta, etc., and controls the timing control unit 41 so that imaging is performed with the set content.

[0024] The storage unit 44 is composed of a storage medium, for example, an HDD (Hard Disk Drive), a flash memory, an EEPROM (Electrically Erasable Programmable Read Only Memory), a RAM (Random Access read / write Memory), a ROM (Read Only Memory), or an arbitrary combination of these storage media.

[0025] With such a configuration, in the distance image capturing device 1, the light receiving unit 3 receives the reflected light RL in which the light pulse PO in the near-infrared wavelength band irradiated by the light source unit 2 to the subject OB is reflected by the subject OB, and the distance image processing unit 4 outputs distance information measuring the distance to the subject OB.

[0026] Note that, in FIG. 1, the distance image capturing device 1 having a configuration including the distance image processing unit 4 inside is shown, but the distance image processing unit 4 may be a component provided outside the distance image capturing device 1.

[0027] Next, the configuration of the distance image sensor 32 used as the imaging element in the distance image capturing device 1 will be described. FIG. 2 is a block diagram showing a schematic configuration of the imaging element (distance image sensor 32) used in the distance image capturing device 1 of the embodiment.

[0028] As shown in FIG. 2, the distance image sensor 32 includes, for example, a light receiving area 320 in which a plurality of pixels 321 are arranged, a control circuit 322, a vertical scanning circuit 323 having a distribution operation, a horizontal scanning circuit 324, and a pixel signal processing circuit 325.

[0029] The light receiving area 320 is an area in which a plurality of pixels 321 are arranged. In FIG. 2, an example of a two-dimensional matrix arrangement in 8 rows and 8 columns is shown. The pixel 321 accumulates charges according to the amount of received light. The control circuit 322 comprehensively controls the distance image sensor 32. The control circuit 322 controls the operations of the components of the distance image sensor 32, for example, in response to an instruction from the timing control unit 41 of the distance image processing unit 4. Note that the control of the components provided in the distance image sensor 32 may be directly performed by the timing control unit 41. In this case, the control circuit 322 can be omitted.

[0030] The vertical scanning circuit 323 is a circuit that controls the pixels 321 arranged in the light receiving area 320 row by row in response to control from the control circuit 322. The vertical scanning circuit 323 causes the pixel signal processing circuit 325 to output a voltage signal corresponding to the amount of charge accumulated in each charge storage unit CS of the pixel 321. In this case, the vertical scanning circuit 323 distributes the charges converted by the photoelectric conversion element to each charge storage unit of the pixel 321. That is, the vertical scanning circuit 323 is an example of a "pixel driving circuit".

[0031] The pixel signal processing circuit 325 is a circuit that performs predetermined signal processing (for example, noise suppression processing, A / D conversion processing, etc.) on the voltage signals output from the pixels 321 of each column to the corresponding vertical signal lines in response to control from the control circuit 322.

[0032] The horizontal scanning circuit 324 is a circuit that sequentially outputs the signals output from the pixel signal processing circuit 325 to the horizontal signal lines in response to control from the control circuit 322. As a result, pixel signals corresponding to the amount of charge accumulated for one frame are sequentially output to the distance image processing unit 4 via the horizontal signal lines.

[0033] Hereinafter, it is assumed that the pixel signal processing circuit 325 performs A / D conversion processing and the pixel signal is a digital signal, and the description will be given accordingly.

[0034] Here, the configuration of the pixel 321 disposed in the light receiving region 320 provided in the distance image sensor 32 will be described. FIG. 3 is a circuit diagram showing an example of the configuration of the pixel 321 disposed in the light receiving region 320 of the distance image sensor 32 according to the embodiment. FIG. 3 shows an example of the configuration of one of the plurality of pixels 321 disposed in the light receiving region 320. In the example of this figure, the pixel 321 is an example of a configuration including four pixel signal readout units.

[0035] As shown in FIG. 3, the pixel 321 includes one photoelectric conversion element PD, a drain gate transistor GD, and four pixel signal readout units RU (pixel signal readout units RU1 to RU4). Each pixel signal readout unit RU outputs a voltage signal from the output terminal O.

[0036] In the following description, by attaching the numbers "1", "2", "3", or "4" after the symbols of the four pixel signal readout units RU, the respective pixel signal readout units RU are distinguished. Similarly, each component provided in the four pixel signal readout units RU is also represented by attaching a number after the respective symbol to distinguish each component.

[0037] Each of the pixel signal readout units RU includes a charge distribution gate transistor G, a floating diffusion FD, a charge storage capacitor C, a reset gate transistor RT, a source follower gate transistor SF, and a selection gate transistor SL. In each pixel signal readout unit RU, a charge storage unit CS is constituted by the floating diffusion FD and the charge storage capacitor C. Specifically, the pixel signal readout unit RU1 includes a charge distribution gate transistor G1, a floating diffusion FD1, a charge storage capacitor C1, a reset gate transistor RT1, a source follower gate transistor SF1, and a selection gate transistor SL1. In the pixel signal readout unit RU1, a charge storage unit CS1 is constituted by the floating diffusion FD1 and the charge storage capacitor C1. The pixel signal readout units RU2 to RU4 have the same configuration. Note that the configuration of the charge distribution gate transistor G is not limited to the transfer method, and may be charge distribution by a photogate method.

[0038] The photoelectric conversion element PD is an embedded photodiode that photoelectrically converts incident light to generate charges and accumulates the generated charges. The structure of the photoelectric conversion element PD may be arbitrary. The photoelectric conversion element PD may be, for example, a PN photodiode having a structure in which a P-type semiconductor and an N-type semiconductor are joined, or a PIN photodiode having a structure in which an I-type semiconductor is sandwiched between a P-type semiconductor and an N-type semiconductor.

[0039] In the pixel 321, the charges generated by the photoelectric conversion element PD photoelectrically converting the incident light are distributed to each of the four charge storage units CS, and each voltage signal corresponding to the amount of the distributed charges is output to the pixel signal processing circuit 325.

[0040] The configuration of the pixels arranged in the distance image sensor 32 is not limited to the configuration including four pixel signal readout units RU as shown in FIG. 3, and any pixel with a configuration including a plurality of pixel signal readout units RU may be used. That is, the number of pixel signal readout units RU (charge storage unit CS) provided in the pixels arranged in the distance image sensor 32 may be two, three, or five or more.

[0041] In addition, in the pixel 321 having the configuration shown in FIG. 3, an example in which the charge storage unit CS is configured by a floating diffusion FD and a charge storage capacitor C is shown. However, the charge storage unit CS only needs to be configured by at least the floating diffusion FD, and the pixel 321 may have a configuration without the charge storage capacitor C.

[0042] In addition, in the pixel 321 having the configuration shown in FIG. 3, an example of a configuration including a drain gate transistor GD is shown, but the present invention is not limited thereto. For example, when it is not necessary to discard the charges remaining in the photoelectric conversion element PD without being stored in the charge storage unit CS, a configuration without the drain gate transistor GD may be used.

[0043] (Conventional driving timing) Here, the conventional driving timing will be described with reference to FIG. 4. FIG. 4 is a timing chart showing the conventional driving timing (the timing for driving the pixels).

[0044] In FIG. 4, in one sorting process, the time required to store charges in each of the charge storage units CS is represented as a "unit storage time UT". After the sorting process (unit storage process) performed in the "unit storage time UT" is repeated the number of storage times corresponding to one frame, a process of reading the amount of charge stored during that time is performed. The time when this process of reading the stored charge amount is performed is represented as a "readout period".

[0045] In addition, in Fig. 4, the timing of irradiating the optical pulse PO is denoted as "L", the timing when the reflected light RL is received is denoted as "R", the timing of driving the charge distribution gate transistor G1 is denoted as "G1", the timing of driving the charge distribution gate transistor G2 is denoted as "G2", the timing of driving the charge distribution gate transistor G3 is denoted as "G3", the timing of driving the charge distribution gate transistor G4 is denoted as "G4", and the timing of the drive signal RSTD is denoted as "GD".

[0046] The vertical scanning circuit 323 accumulates charges in the charge storage units CS1 to CS4 at a timing synchronized with the irradiation of the optical pulse PO. In the example of Fig. 4, charges are accumulated in the charge storage unit CS1 at the same timing as the timing of irradiating the optical pulse PO, and after the charges are accumulated in the charge storage unit CS1, charges are sequentially accumulated in the charge storage units CS2 to CS4.

[0047] In the example of Fig. 4, a timing chart is shown for the case where the reflected light RL is received by the distance image sensor 32 with a delay time Td after the optical pulse PO is irradiated. Depending on the delay time Td, the charges corresponding to the reflected light RL are distributed and accumulated in the charge storage units CS1 and CS2. At the timing when the charge storage units CS3 and CS4 accumulate charges, the reflected light RL is not received, and the charges corresponding to external light components such as background light are accumulated in the charge storage units CS3 and CS4.

[0048] Specifically, first, the vertical scanning circuit 323 irradiates the optical pulse PO. The vertical scanning circuit 323 turns off the drain gate transistor GD at the same timing as the irradiation timing, and turns on the charge distribution gate transistor G1 for the accumulation time Ta. After the vertical scanning circuit 323 turns on the charge distribution gate transistor G1 for the accumulation time Ta, it turns off the charge distribution gate transistor G1. As a result, the charges photoelectrically converted by the photoelectric conversion element PD while the charge distribution gate transistor G1 is controlled to be in the on state are accumulated in the charge storage unit CS1 through the charge distribution gate transistor G1.

[0049] Next, the vertical scanning circuit 323 turns on the charge distribution gate transistor G2 for the accumulation time Ta at the timing when the charge distribution gate transistor G2 is turned off. After turning on the charge distribution gate transistor G2 for the accumulation time Ta, the vertical scanning circuit 323 turns off the charge distribution gate transistor G2. As a result, the charges photoelectrically converted by the photoelectric conversion element PD while the charge distribution gate transistor G2 is controlled to be in the on state are accumulated in the charge accumulation unit CS2 via the charge distribution gate transistor G2.

[0050] Next, the vertical scanning circuit 323 turns on the charge distribution gate transistor G3 for the accumulation time Ta at the timing when the charge distribution gate transistor G2 is turned off. After turning on the charge distribution gate transistor G3 for the accumulation time Ta, the vertical scanning circuit 323 turns off the charge distribution gate transistor G3. As a result, the charges photoelectrically converted by the photoelectric conversion element PD while the charge distribution gate transistor G3 is controlled to be in the on state are accumulated in the charge accumulation unit CS3 via the charge distribution gate transistor G3.

[0051] Next, the vertical scanning circuit 323 turns on the charge distribution gate transistor G4 for the accumulation time Ta at the timing when the accumulation of charges in the charge accumulation unit CS3 is completed. After turning on the charge distribution gate transistor G4 for the accumulation time Ta, the vertical scanning circuit 323 turns off the charge distribution gate transistor G4. The vertical scanning circuit 323 turns on the drain gate transistor GD at the timing when the charge distribution gate transistor G4 is turned off. When the drain gate transistor GD is turned on, the charges photoelectrically converted by the photoelectric conversion element PD during this period are discarded via the drain gate transistor GD without being accumulated in the charge accumulation unit CS.

[0052] The vertical scanning circuit 323 repeats the above-described driving for a predetermined number of distribution times over one frame. Then, the vertical scanning circuit 323 outputs a voltage signal corresponding to the amount of charge accumulated in each charge accumulation unit CS. Specifically, the vertical scanning circuit 323 turns on the selection gate transistor SL1 for a predetermined time, and outputs a voltage signal corresponding to the amount of charge accumulated in the charge accumulation unit CS1 from the output terminal O1 through the pixel signal readout unit RU1. Similarly, the vertical scanning circuit 323 sequentially turns on the selection gate transistors SL2 to SL4, and outputs voltage signals corresponding to the amounts of charge accumulated in the charge accumulation units CS2 to CS4 from the output terminals O2 to O4. Thereby, an electric signal corresponding to the amount of charge for one frame accumulated in each of the charge accumulation units CS is output to the distance calculation unit 42.

[0053] In the above description, the case where the charge distribution gate transistor G1 is turned on at the timing when the optical pulse PO is irradiated has been described as an example. However, the present invention is not limited to this. The optical pulse PO may be irradiated at least at the timing when the charge corresponding to the reflected light RL is distributed and accumulated in the charge accumulation units CS1 and CS2, or CS2 or CS3.

[0054] In FIG. 4, from the relationship between the timing of irradiating the optical pulse PO, the timing of receiving the reflected light RL, and the timing of accumulating charges in each of the charge accumulation units CS, the amounts of charge corresponding to the reflected light RL are distributed and accumulated in the charge accumulation units CS1 and CS2. Further, the amounts of charge corresponding to the external light component are accumulated in the charge accumulation units CS1 to CS4.

[0055] The distribution (distribution ratio) of the amounts of charge distributed to the charge accumulation units CS1 and CS2 is a ratio corresponding to the delay time Td from the irradiation time of the optical pulse PO to the reception time of the reflected light RL. The distance calculation unit 42 calculates the delay time Td by the following formula (1) using this principle. R in the formula (1) is a charge ratio indicating the distribution ratio of the reflected light RL. In the formula (1), it is assumed that the amounts of charge corresponding to the external light component accumulated in each of the charge accumulation units CS1 to CS3 are the same.

[0056] Td = To × R …(1) However, R = (Q2 - Q3) / (Q1 + Q2 - 2×Q3) To is the time interval during which the optical pulse PO is irradiated Q1 is the amount of charge stored in the charge storage unit CS1 Q2 is the amount of charge stored in the charge storage unit CS2 Q3 is the amount of charge stored in the charge storage unit CS3

[0057] The distance calculation unit 42 calculates the round - trip distance to the subject OB by multiplying the delay time Td obtained by the formula (1) by the speed of light (velocity) in the case of the short - distance light - receiving pixels. Then, the distance calculation unit 42 obtains the ToF distance by taking half of the round - trip distance calculated above.

[0058] (Drive Timing of the First Embodiment) Here, the drive timing of the first embodiment will be described with reference to FIG. 5. FIG. 5 is a timing chart showing the drive timing of the pixel 321 in the first embodiment.

[0059] FIG. 5 shows a timing chart of the sorting process (unit accumulation process) performed during the unit accumulation time UT of the present embodiment. In FIG. 5, similar to FIG. 4, the timing of driving the charge sorting gate transistor G1 is denoted as "G1", the timing of driving the charge sorting gate transistor G2 is denoted as "G2", the timing of driving the charge sorting gate transistor G3 is denoted as "G3", the timing of driving the charge sorting gate transistor G4 is denoted as "G4", and the timing of the drive signal RSTD is denoted as "GD" in the item names, respectively. Also, here, at the start of the unit accumulation time UT, each of the charge sorting gate transistors G1 to G4 is in the off state, and the drain - gate transistor GD is in the on state.

[0060] In the present embodiment, in the unit accumulation process, the short - distance object measurement process, the discharge process, and the long - distance object measurement process are sequentially performed. Hereinafter, each process will be described.

[0061] First, the short-distance object measurement process will be described. The short-distance object measurement process is a process for measuring the distance to an object at a relatively short distance, and is implemented by driving the charge storage units CS1 to CS3 of the pixel 321 at the conventional driving timing.

[0062] Specifically, the vertical scanning circuit 323 turns off the drain gate transistor GD and sequentially turns on the charge distribution gates G1 to G3 for the accumulation time Ta at the timing corresponding to the irradiation of the optical pulse PO, for example, at the same timing as the irradiation of the optical pulse PO, as shown in FIG. 4.

[0063] Next, the discharge process will be described. The discharge process is a process for discharging the charge generated by the photoelectric conversion element PD, and is implemented by turning on the drain gate transistor GD.

[0064] Specifically, the vertical scanning circuit 323 turns on the drain gate transistor GD for a certain period of time (the time corresponding to the zone Z3 in this figure) at the timing when the charge accumulation in the charge storage unit CS3 is completed.

[0065] Next, the long-distance object measurement process will be described. The long-distance object measurement process is a process for determining the presence or absence of an object existing at a long distance. The long-distance object measurement process is implemented by turning on the charge distribution gate G4 after the discharge process.

[0066] Specifically, the vertical scanning circuit 323 turns off the drain gate transistor GD and turns on the charge distribution gate G4 for a certain period of time (the accumulation time Tb in this figure). Then, the vertical scanning circuit 323 turns on the drain gate transistor GD at the timing when the charge accumulation in the charge storage unit CS4 is completed.

[0067] Also, in this figure, according to the distance to the subject OB, the time intervals during which the reflected light RL arrives at the distance image capturing device 1 are respectively shown in zones Z1 to Z4. Here, according to in which of the zones Z1 to Z4 the reflected light RL is received, the distance to the subject OB is classified into a first distance to a fourth distance.

[0068] The first distance is the distance corresponding to zone Z1. The reflected light RL reflected by the subject OB existing at the first distance is received in zone Z1. As the first distance, for example, a distance of about 0 (zero) [m] to 2.5 [m] is assumed.

[0069] The second distance is the distance corresponding to zone Z2. The second distance is greater than the first distance, and the reflected light RL reflected by the subject OB existing at the second distance is received in zone Z2. As the second distance, for example, a distance of about 2.5 [m] to 5.0 [m] is assumed.

[0070] The third distance is the distance corresponding to zone Z3. The third distance is greater than the second distance, and the reflected light RL reflected by the subject OB existing at the third distance is received in zone Z3. As the third distance, for example, a distance of about 5.0 [m] to 15.0 [m] is assumed.

[0071] The fourth distance is the distance corresponding to zone Z4. The fourth distance is greater than the third distance, and the reflected light RL reflected by the subject OB existing at the fourth distance is received in zone Z4. As the fourth distance, for example, a distance of about 15.0 [m] to 25.0 [m] is assumed.

[0072] When driving pixel 321 as shown in the example of this figure, the charges corresponding to the reflected light received in zone Z1 are distributed and stored in charge storage section CS1 and charge storage section CS2. In this case, the charges stored in charge storage sections CS3 and CS4 correspond to the ambient light component. When using the charge stored in charge storage section CS4 as the ambient light component, the charge storage sections CS1 to CS3 may be set to have different storage times from that of charge storage section CS4. Therefore, it is necessary to derive the ambient light component corresponding to the storage time of charge storage section CS4 by multiplying Ta / Tb. When executing the calculation of multiplying Ta / Tb, there is a possibility of increasing or decreasing (changing) the noise component that has nothing to do with the storage time. For this reason, it is preferable to derive the charge corresponding to the ambient light component based on the charge stored in charge storage section CS3 for which no calculation is required when deriving the ambient light component.

[0073] The charges corresponding to the reflected light received in zone Z2 are distributed and stored in charge storage section CS2 and charge storage section CS3. In this case, the charges stored in charge storage sections CS1 and CS4 correspond to the ambient light component. When using the charge stored in charge storage section CS4 as the ambient light component, the charge storage sections CS1 to CS3 may be set to have different storage times from that of charge storage section CS4. Therefore, it is necessary to derive the ambient light component corresponding to the storage time of charge storage section CS4 by multiplying Ta / Tb. When executing the calculation of multiplying Ta / Tb, there is a possibility of increasing or decreasing (changing) the noise component that has nothing to do with the storage time. For this reason, it is preferable to derive the charge corresponding to the ambient light component based on the charge stored in charge storage section CS1 for which no calculation is required when deriving the ambient light component.

[0074] The charges corresponding to the reflected light received in zone Z3 are discharged without being stored in any of the charge storage sections CS. That is, the distance to the subject OB existing at the third distance corresponding to zone Z3 is not measured. The third distance is an example of the "distance outside the measurement range".

[0075] The charge corresponding to the reflected light received in zone Z4 is accumulated in the charge storage section CS4. In this case, the charges accumulated in the charge storage sections CS1 to CS3 correspond to the ambient light component.

[0076] After repeating the sorting process shown in FIG. 5 for the number of accumulations corresponding to one frame, a process of reading out the amount of charge accumulated during that period is performed.

[0077] (Flow of processing in the first embodiment) Here, the flow of processing in the first embodiment will be described with reference to FIG. 6. FIG. 6 is a flowchart showing the flow of processing performed by the distance image processing unit 4 in the first embodiment.

[0078] First, the distance image processing unit 4 performs charge accumulation for one frame at the drive timing shown in FIG. 5 (step S10).

[0079] Next, the distance image processing unit 4 calculates the amount of charge Qg corresponding to the ambient light component (step S11). For example, the distance image processing unit 4 sets the smallest amount of charge among the amounts of charge accumulated in each of the charge storage sections CS1 to CS3 as the amount of charge Qg corresponding to the ambient light component.

[0080] Next, the distance image processing unit 4 calculates a corrected charge amount Q4# obtained by correcting the charge amount Q4 accumulated in the charge storage section CS4 (step S12). The corrected charge amount Q4# is a value obtained by correcting the charge amount Q4 accumulated in the charge storage section CS4 to the charge amount corresponding to the accumulation time in the other charge storage sections CS (charge storage sections CS1 to CS3). The distance image processing unit 4 calculates the corrected charge amount Q4# according to the following equation (2).

[0081] Q4# = Ta / Tb × Q4 …(2) However, Ta is the time interval during which charge is accumulated in the charge storage section CS1 Tb is the time interval during which charge is accumulated in the charge storage section CS4 Q4 is the amount of charge accumulated in the charge storage section CS4

[0082] Next, the distance image processing unit 4 determines in which charge storage unit CS the charge amount corresponding to the reflected light is stored (step S13). The distance image processing unit 4 compares the charge amount Qg calculated in step S11 with the charge amounts stored in each of the charge storage units CS. In this case, for the charge storage units CS1 to CS3, the distance image processing unit 4 compares the charge amount Qg with the charge amounts Q1 to Q3 stored in CS1 to CS3 respectively. For the charge storage unit CS4, the distance image processing unit 4 compares the charge amount Qg with the corrected charge amount Q4# calculated in step S12. The distance image processing unit 4 determines the charge storage unit CS in which a charge amount larger than the charge amount Qg is stored as the charge storage unit CS in which the charge amount corresponding to the reflected light RL is stored.

[0083] When it is determined that the charge corresponding to the reflected light is stored in the charge storage units CS1 and CS2, the distance image processing unit 4 determines that there is an object at the distance (first distance) corresponding to the zone Z1, and calculates the distance to the object using the formula (1) (step S14).

[0084] The distance image processing unit 4 performs control when there is an object at the distance (first distance) corresponding to the zone Z1 (step S15). The control when there is an object at the distance (first distance) corresponding to the zone Z1 here is arbitrarily set according to the environment in which the distance image capturing device 1 is mounted. For example, in an environment where the distance image capturing device 1 is mounted on a moving body and is moving, if it continues to move in a situation where there is an object at a short distance, the risk of collision or the like increases. Therefore, processing such as urgently stopping the moving body is performed.

[0085] When it is determined that charges corresponding to the reflected light are accumulated in the charge storage units CS2 and CS3, the distance image processing unit 4 determines that there is an object at the distance (second distance) corresponding to the zone Z2, and calculates the distance to the object using equation (1) (step S16). In this case, the charge ratio R in equation (1) is R = (Q3 - Q1) / (Q2 + Q3 - 2×Q1). When obtaining the distance from the charge ratio R, the distance to the object is determined by adding the base distance corresponding to the zone. For example, in the case of zone Z2, the base distance is 2.5 [m]. In the case of zone Z2, the distance obtained by adding the base distance of 2.5 [m] to the distance calculated based on the charge ratio R is the distance to the subject OB existing in zone Z2.

[0086] When there is an object at the distance (second distance) corresponding to the zone Z2, the distance image processing unit 4 performs control (step S17). The control when there is an object at the distance (second distance) corresponding to the zone Z2 here is arbitrarily set according to the environment in which the distance image capturing device 1 is mounted. For example, processing such as avoidance behavior for changing the traveling direction of the moving body is performed.

[0087] When it is determined that charges corresponding to the reflected light are accumulated in the charge storage unit CS4, the distance image processing unit 4 determines that there is an object at the distance (fourth distance) corresponding to the zone Z4 (step S18).

[0088] When there is an object at the distance (fourth distance) corresponding to the zone Z4, the distance image processing unit 4 performs control (step S19). The control when there is an object at the distance (fourth distance) corresponding to the zone Z4 here is arbitrarily set according to the environment in which the distance image capturing device 1 is mounted. For example, processing such as reducing the moving speed of the moving body is performed.

[0089] In the above description, the case where the distance image processing unit 4 performs the processes shown in steps S15, S17, and S19 respectively has been exemplified and described. However, the present invention is not limited to this. The distance image processing unit 4 may execute at least the processes shown in steps S14, S15, or S18. For example, consider a case where an external device for controlling a moving body on which the distance image capturing device 1 is mounted is provided. The external device is communicably connected to the distance image capturing device 1. The distance image capturing device 1 outputs information indicating the distance to an object calculated in step S14 or S15 to the external device. Alternatively, the distance image capturing device 1 outputs information indicating the presence or absence of an object determined in step S18 to the external device. The external device controls the moving body based on the information received from the distance image capturing device 1.

[0090] As described above, the distance image capturing device 1 of the first embodiment includes a light source unit 2, a light receiving unit 3, a drain gate transistor GD, and a distance image processing unit 4. The light source unit 2 irradiates an optical pulse PO to a measurement space where the subject OB exists. The light receiving unit 3 includes a pixel 321 and a vertical scanning circuit 323 (an example of a driving circuit). The vertical scanning circuit 323 distributes and accumulates charges to each of the charge accumulation units CS in the pixel 321 at a predetermined timing synchronized with the irradiation of the optical pulse PO. The drain gate transistor GD discharges the charges generated by the photoelectric conversion element PD. The distance image processing unit 4 determines the measurement distance to the subject OB using the amount of charge accumulated in each of the charge accumulation units CS. The distance image processing unit 4 performs the unit accumulation process a plurality of times during one frame period, and determines the measurement distance to the subject OB using the amount of charge accumulated in each of the charge accumulation units CS. In the unit accumulation process, the short-distance object measurement process, the discharge process, and the long-distance object measurement process are performed in order. The short-distance object measurement process is a process of distributing and accumulating charges corresponding to the reflected light RL to the charge accumulation units CS1 to CS3 (a part of the charge accumulation units CS among the charge accumulation units CS). The discharge process is a process in which the drain gate transistor GD discharges the charges generated by the photoelectric conversion element PD. The long-distance object measurement process is a process of accumulating charges corresponding to the reflected light RL in the charge accumulation unit CS4 (the charge accumulation unit CS including a charge accumulation unit CS different from a part of the charge accumulation units CS).

[0091] Thereby, the distance image capturing device 1 of the first embodiment can measure the distance to an object at a short distance in the short-distance object measurement process. Also, the presence or absence of an object at a long distance can be determined in the long-distance object measurement process. That is, it is possible to measure the distance to an object at a short distance and determine the presence or absence of an object at a long distance. Moreover, in the distance image capturing device 1 according to the first embodiment, the discharge process is performed after the short-distance object measurement process and before the long-distance object measurement process. For this reason, charges are not accumulated in the charge accumulation unit CS during this period, and charges corresponding to the reflected light reflected from an object at a long distance can be accurately accumulated.

[0092] In the first embodiment, during the unit accumulation time UT, the drain gate transistor GD is turned off so that no extra charge is accumulated at timings other than the time when the charge storage unit CS accumulates charge. As a result, it is possible to avoid continuous charge accumulation during the time when the distance to a nearby object is not measured and the presence or absence of a distant object is not determined (zone Z3).

[0093] (Drive Timing of the Second Embodiment) Here, the drive timing of the second embodiment will be described with reference to FIG. 7. FIG. 7 is a timing chart showing the drive timing of the pixel 321 in the second embodiment.

[0094] FIG. 7 shows a timing chart of the sorting process (unit accumulation process) performed during the unit accumulation time UT of this embodiment. In FIG. 7, similar to FIG. 5, the timing for driving the charge sorting gate transistor G1 is denoted as "G1", the timing for driving the charge sorting gate transistor G2 is denoted as "G2", the timing for driving the charge sorting gate transistor G3 is denoted as "G3", the timing for driving the charge sorting gate transistor G4 is denoted as "G4", and the timing of the drive signal RSTD is denoted as "GD". Also, here, it is assumed that at the start of the unit accumulation time UT, each of the charge sorting gate transistors G1 to G4 is in the off state and the drain gate transistor GD is in the on state.

[0095] In this embodiment, in the long-distance object measurement process, there is a difference from the first embodiment in terms of measuring the distance to the object. The differences from the first embodiment will be described below.

[0096] Since the near-distance object measurement process and the discharge process are the same as those in the first embodiment, their descriptions are omitted.

[0097] The long-distance object measurement process of this embodiment will be described. The long-distance object measurement process is a process for determining the distance to an object existing at a long distance. The long-distance object measurement process is performed by sequentially turning on the charge distribution gate transistors G4 and G1 after the discharge process.

[0098] Specifically, the vertical scanning circuit 323 turns off the drain gate transistor GD and turns on the charge distribution gate transistor G4 for the accumulation time Ta. After the vertical scanning circuit 323 turns on the charge distribution gate transistor G4 for the accumulation time Ta, it turns off the charge distribution gate transistor G4. At the timing when the charging of the charge storage unit CS4 is completed, the vertical scanning circuit 323 turns on the charge distribution gate transistor G1 for the accumulation time Ta. After the vertical scanning circuit 323 turns on the charge distribution gate transistor G1 for the accumulation time Ta, it turns off the charge distribution gate transistor G1. At the timing when the charging of the charge storage unit CS1 is completed, the vertical scanning circuit 323 turns on the drain gate transistor GD.

[0099] The fourth distance corresponding to the zone Z4 in this embodiment is the entire range or a partial range of the fourth distance in the first embodiment. The fourth distance in this embodiment may be arbitrarily determined according to the relationship between the accumulation time Tb in the first embodiment and the accumulation time Ta for the charge distribution gate transistors G4 and G1 in the long-distance object measurement process in this embodiment. For example, the fourth distance in this embodiment is assumed to be a distance of about 17.5 [m] to 20.0 [m].

[0100] (Flow of the process of the second embodiment) Here, the flow of the process of the second embodiment will be described with reference to FIG. 8. FIG. 8 is a flowchart showing the flow of the process performed by the distance image processing unit 4 of the second embodiment.

[0101] Step S20 is the same as step S10 in FIG. 5, so the description thereof will be omitted.

[0102] The distance image processing unit 4 calculates the charge amount Qg corresponding to the external light component (step S21). For example, the distance image processing unit 4 sets the smallest charge amount among the charge amounts accumulated in each of the charge storage units CS2 to CS4 as the charge amount Qg corresponding to the external light component.

[0103] Next, the distance image processing unit 4 calculates a corrected charge amount Q1# obtained by correcting the charge amount Q1 accumulated in the charge storage unit CS1 (step S22). The corrected charge amount Q1# is a value obtained by correcting the charge amount Q1 accumulated in the charge storage unit CS1 with the charge amount corresponding to the accumulation time in other charge storage units CS (charge storage units CS2 to CS4). The distance image processing unit 4 calculates the corrected charge amount Q1# according to the following formula (3).

[0104] Q1# = Q1 - Qg...(3) However, Q1 is the charge amount accumulated in the charge storage unit CS1 Qg is the charge amount corresponding to the external light component In addition, in formula (3), it is assumed that the time interval during which charge is accumulated in the charge storage unit CS1 in the short-distance object measurement process and the time interval during which charge is accumulated in the charge storage unit CS1 in the long-distance object measurement process (Ta in FIG. 7) are the same value.

[0105] Next, the distance image processing unit 4 determines in which charge storage unit CS the charge amount corresponding to the reflected light is accumulated (step S23). The distance image processing unit 4 compares the charge amount Qg calculated in step S21 with the charge amounts accumulated in each of the charge storage units. In this case, for the charge storage units CS2 to CS4, the distance image processing unit 4 compares the charge amount Qg with the charge amounts Q2 to Q4 accumulated in CS2 to CS4 respectively. For the charge storage unit CS1, the distance image processing unit 4 compares the charge amount Qg with the corrected charge amount Q1# calculated in step S22. The distance image processing unit 4 determines that the charge storage unit CS in which a charge amount greater than the charge amount Qg is accumulated is the charge storage unit CS in which the charge amount corresponding to the reflected light is accumulated.

[0106] When it is determined that charges corresponding to the reflected light are accumulated in the charge storage units CS1 and CS2, the distance image processing unit 4 determines that there is an object at a distance (first distance) corresponding to the zone Z1, and calculates the distance to the object using Equation (1) (step S24). In this case, the charge ratio R in Equation (1) is R = (Q2 - Q3) / (Q1#+Q2 - 2×Q3).

[0107] Steps S25 to 27 are the same as steps S15 to 17 in FIG. 5, and thus the description thereof is omitted.

[0108] When it is determined that charges corresponding to the reflected light are accumulated in the charge storage units CS4 and CS1, the distance image processing unit 4 determines that there is an object at a distance (fourth distance) corresponding to the zone Z4, and calculates the distance to the object using Equation (1) (step S28). In this case, the charge ratio R in Equation (1) is R = (Q1# - Q2) / (Q4 + Q1# - 2×Q2), or the charge ratio R is R = (Q1# - Q3) / (Q4 + Q1# - 2×Q3). Note that when the distance is obtained from the charge ratio R, the distance to the object is determined by adding the base distance corresponding to the zone.

[0109] Step S29 is the same as step S19 in FIG. 5, and thus the description thereof is omitted.

[0110] As described above, in the distance image capturing device 1 according to the second embodiment, in the long-distance object measurement process, the charges corresponding to the reflected light RL are distributed and accumulated in the charge storage units CS4 and CS1 (charge storage units CS including charge storage units CS different from some of the charge storage units CS).

[0111] Thereby, the distance image capturing device 1 according to the second embodiment can measure the distance to the object in the long-distance object measurement process. For this reason, processing according to the distance to an object at a long distance becomes possible.

[0112] (Drive Timing of Modification Example 1 According to the Second Embodiment) Here, the driving timing of Modification 1 according to the second embodiment will be described with reference to FIG. 9. FIG. 9 is a timing chart showing the driving timing of the pixel 321 in Modification 1 according to the second embodiment.

[0113] FIG. 9 shows a timing chart of the sorting process (unit accumulation process) performed during the unit accumulation time UT of this modification. In FIG. 9, similar to FIG. 5, the timing for driving the charge sorting gate transistor G1 is denoted as "G1", the timing for driving the charge sorting gate transistor G2 is denoted as "G2", the timing for driving the charge sorting gate transistor G3 is denoted as "G3", the timing for driving the charge sorting gate transistor G4 is denoted as "G4", and the timing of the driving signal RSTD is denoted as "GD" in the item names, respectively. Here, it is assumed that at the start of the unit accumulation time UT, each of the charge sorting gate transistors G1 to G4 is in the off state and the drain gate transistor GD is in the on state.

[0114] This modification is different from the second embodiment in that in the long-distance object measurement process, the range in which the distance to the object can be measured is made larger. Hereinafter, the differences from the second embodiment will be described.

[0115] In this modification, as shown in this figure, in addition to zones Z1 to Z4, zone Z5 is shown. The fourth distance corresponding to zone Z4 is, similar to the second embodiment, for example, a distance of about 17.5 [m] to 20.0 [m].

[0116] In this modification, the distance corresponding to zone Z5 is defined as the fifth distance. The reflected light RL reflected by the subject OB existing at the fifth distance is received in zone Z5. As the fifth distance, for example, a distance of about 20.0 [m] to 22.5 [m] is assumed.

[0117] Since the near-distance object measurement process and the discharge process are the same as those in the second embodiment, the description thereof will be omitted.

[0118] The long-distance object measurement process of this modification example will be described. The long-distance object measurement process is performed by sequentially turning on the charge distribution gate transistors G4, G1, and G2 after the discharge process.

[0119] Specifically, the vertical scanning circuit 323 turns off the drain gate transistor GD, and in the same manner as in the second embodiment, turns on the charge distribution gate transistors G4 and G1 for the accumulation time Ta. After the vertical scanning circuit 323 turns on the charge distribution gate transistor G1 for the accumulation time Ta, it turns off the charge distribution gate transistor G1. At the timing when the charging of the charge storage unit CS4 is completed, the vertical scanning circuit 323 turns on the charge distribution gate transistor G2 for the accumulation time Ta. After the vertical scanning circuit 323 turns on the charge distribution gate transistor G2 for the accumulation time Ta, it turns off the charge distribution gate transistor G2. At the timing when the charging of the charge storage unit CS2 is completed, the vertical scanning circuit 323 turns on the drain gate transistor GD.

[0120] (Flow of the process of Modification Example 1 according to the second embodiment) Here, the flow of the process of Modification Example 1 according to the second embodiment will be described with reference to FIG. 10. FIG. 10 is a flowchart showing the flow of the process performed by the distance image processing unit 4 of Modification Example 1 according to the second embodiment.

[0121] Step S30 is the same as step S10 in FIG. 5, so the description thereof is omitted.

[0122] The distance image processing unit 4 calculates the charge amount Qg corresponding to the external light component (step S31). The distance image processing unit 4 sets, for example, the smallest charge amount among the charge amounts accumulated in the charge storage units CS3 and CS4 to the charge amount Qg corresponding to the external light component.

[0123] Next, the distance image processing unit 4 calculates the corrected charge amounts Q1# and Q2# (step S32). The corrected charge amount Q1# is a value obtained by correcting the charge amount Q1 accumulated in the charge accumulation unit CS1 to the charge amount corresponding to the accumulation time in the other charge accumulation units CS (charge accumulation units CS3 to CS4), and is calculated by the following equation (3). The corrected charge amount Q2# is a value obtained by correcting the charge amount Q2 accumulated in the charge accumulation unit CS2 to the charge amount corresponding to the accumulation time in the other charge accumulation units CS (charge accumulation units CS3 to CS4), and is calculated by the following equation (4).

[0124] Q2# = Q2 - Qg…(4) However,[[]] Q2 is the charge amount accumulated in the charge accumulation unit CS2 Qg is the charge amount corresponding to the ambient light component In the equation (4), it is assumed that the time intervals during which charges are accumulated in the charge accumulation unit CS2 in the short-distance object measurement process and the time intervals during which charges are accumulated in the charge accumulation unit CS2 in the long-distance object measurement process (Ta in FIG. 9) are the same value.

[0125] Next, the distance image processing unit 4 determines in which charge accumulation unit CS the charge amount corresponding to the reflected light is accumulated (step S33). The distance image processing unit 4 compares the charge amount Qg calculated in step S31 with the charge amounts accumulated in each of the charge accumulation units. In this case, for the charge accumulation units CS3 and CS4, the distance image processing unit 4 compares the charge amount Qg with the charge amounts Q3 and Q4 accumulated in CS3 and CS4, respectively. For the charge accumulation units CS1 and CS2, the distance image processing unit 4 compares the charge amount Qg with the corrected charge amounts Q1# and Q2# calculated in step S22. The distance image processing unit 4 determines that the charge accumulation unit CS in which a charge amount larger than the charge amount Qg is accumulated is the charge accumulation unit CS in which the charge amount corresponding to the reflected light is accumulated.

[0126] When it is determined that charges corresponding to the reflected light are accumulated in the charge accumulation units CS1 and CS2, the distance image processing unit 4 determines whether the sum (Q1# + Q2#) of the corrected charge amounts Q1# and Q2# is greater than the threshold Th (step S34).

[0127] The threshold value Th is a value set according to the amount of charge corresponding to the boundary between the intensity of the reflected light from an object at a short distance and the intensity of the reflected light from an object at a long distance. That is, when the sum (Q1#+Q2#) is greater than the threshold value Th, it can be determined that the reflected light from an object at a short distance is distributed and accumulated in the charge accumulation units CS1 and CS2. On the other hand, when the sum (Q1#+Q2#) is less than the threshold value Th, it can be determined that the reflected light from an object at a long distance is distributed and accumulated in the charge accumulation units CS1 and CS2.

[0128] When the sum (Q1#+Q2#) is greater than the threshold value Th, the distance image processing unit 4 determines that there is an object at a distance (first distance) corresponding to the zone Z1, and calculates the distance to the object using the formula (1) (step S35). In this case, the charge ratio R in the formula (1) is R=(Q2#-Q3) / (Q1#+Q2#-2×Q3). Since step S36 is the same as step S15 in FIG. 5, the description thereof is omitted.

[0129] On the other hand, when the sum (Q1#+Q2#) is less than the threshold value Th, the distance image processing unit 4 determines that there is an object at a distance (fifth distance) corresponding to the zone Z5, and calculates the distance to the object using the formula (1) (step S37). In this case, the charge ratio R in the formula (1) is R=(Q2#-Q3) / (Q1#+Q2#-2×Q3). Since step S38 is the same as step S19 in FIG. 5, the description thereof is omitted.

[0130] Regarding the processing (step S39) when it is determined in step S33 that charges corresponding to the reflected light are accumulated in the charge accumulation units CS2 and CS3, since it is the same as step S16 in FIG. 5, the description thereof is omitted. Also, regarding the subsequent processing (step S40), since it is the same as step S17 in FIG. 5, the description thereof is omitted.

[0131] In step S33, when it is determined that charges corresponding to the reflected light are accumulated in the charge storage units CS4 and CS1, the process in step S41 is the same as step S28 in FIG. 8, and thus the description thereof is omitted. Further, for the subsequent process (step S42), since it is the same as step S29 in FIG. 8, the description thereof is omitted.

[0132] As described above, in the distance image capturing device 1 according to the first modification of the second embodiment, in the long-distance object measurement process, the charges corresponding to the reflected light RL are distributed and accumulated in the charge storage units CS4, CS1, and CS2 (charge storage units CS including charge storage units CS different from some of the charge storage units CS).

[0133] Thereby, the distance image capturing device 1 according to the first modification of the second embodiment can measure the distance to the object in a three-tap configuration (charge storage units CS4, CS1, and CS2) in the long-distance object measurement process. Therefore, the distance to an object at a long distance can be measured to a farther distance.

[0134] (Drive Timing of the Second Modification of the Second Embodiment) Here, the drive timing of the second modification of the second embodiment will be described with reference to FIG. 11. FIG. 11 is a timing chart showing the drive timing of the pixel 321 in the second modification of the second embodiment.

[0135] FIG. 11 shows a timing chart of the sorting process (unit accumulation process) performed during the unit accumulation time UT of this modified example. In FIG. 11, similar to FIG. 5, the timing for driving the charge sorting gate transistor G1 is indicated by the item name "G1", the timing for driving the charge sorting gate transistor G2 is indicated by the item name "G2", the timing for driving the charge sorting gate transistor G3 is indicated by the item name "G3", the timing for driving the charge sorting gate transistor G4 is indicated by the item name "G4", and the timing of the drive signal RSTD is indicated by the item name "GD". Here, it is assumed that at the start of the unit accumulation time UT, each of the charge sorting gate transistors G1 to G4 is in the off state and the drain gate transistor GD is in the on state.

[0136] In this modified example, in the long-distance object measurement process, it differs from the second embodiment in that the range in which the distance to the object can be measured is made larger. Hereinafter, the differences from the second embodiment will be described.

[0137] In this modified example, as shown in this figure, there is no zone Z2, and zones Z1, Z3 to Z5 are shown. In this modified example, the first distance corresponding to zone Z1 is, for example, a distance of about 0 (zero) [m] to 2.5 [m], similar to the first embodiment. In this modified example, the third distance corresponding to zone Z3 is, for example, a distance of about 2.5 [m] to 15.0 [m]. In this modified example, the fourth distance corresponding to zone Z4 is, for example, a distance of about 15.0 [m] to 17.5 [m]. In this modified example, the fifth distance corresponding to zone Z5 is, for example, a distance of about 17.5 [m] to 20.0 [m].

[0138] The near-distance object measurement process in this modified example will be described. In this modified example, the near-distance object measurement process is performed by driving the charge accumulation units CS1 and CS2 of the pixel 321 at the conventional driving timing.

[0139] Specifically, the vertical scanning circuit 323 turns off the drain gate transistor GD at a timing corresponding to the irradiation of the optical pulse PO, for example, at the same timing as the irradiation of the optical pulse PO, and turns on the charge distribution gate transistors G1 and G2 sequentially over the accumulation time Ta.

[0140] Since the discharge process in this modification is the same as the discharge process of the first embodiment, the description thereof is omitted.

[0141] The long-distance object measurement process in this embodiment will be described. In this modification, after the discharge process, it is implemented by sequentially turning on the charge distribution gate transistors G3, G4, and G1.

[0142] Specifically, the vertical scanning circuit 323 turns off the drain gate transistor GD and turns on the charge distribution gate transistor G3 over the accumulation time Ta. After the vertical scanning circuit 323 turns on the charge distribution gate transistor G3 over the accumulation time Ta, it turns off the charge distribution gate transistor G3. At the timing when the charging of the charge storage unit CS3 is completed, the vertical scanning circuit 323 turns on the charge distribution gate transistor G4 over the accumulation time Ta. After the vertical scanning circuit 323 turns on the charge distribution gate transistor G4 over the accumulation time Ta, it turns off the charge distribution gate transistor G4. At the timing when the charging of the charge storage unit CS4 is completed, the vertical scanning circuit 323 turns on the charge distribution gate transistor G1 over the accumulation time Ta. After the vertical scanning circuit 323 turns on the charge distribution gate transistor G1 over the accumulation time Ta, it turns off the charge distribution gate transistor G1. At the timing when the charging of the charge storage unit CS1 is completed, the vertical scanning circuit 323 turns on the drain gate transistor GD.

[0143] (Flow of the process of Modification 2 according to the second embodiment) Here, the processing flow of Modification 2 according to the second embodiment will be described with reference to FIG. 12. FIG. 12 is a flowchart showing the processing flow performed by the distance image processing unit 4 of Modification 2 according to the second embodiment.

[0144] Regarding step S50, since it is the same as step S10 in FIG. 5, the description thereof will be omitted.

[0145] The distance image processing unit 4 calculates the charge amount Qg corresponding to the external light component (step S51). For example, the distance image processing unit 4 sets the smallest charge amount among the charge amounts accumulated in each of the charge accumulation units CS2 to CS4 as the charge amount Qg corresponding to the external light component.

[0146] Next, the distance image processing unit 4 calculates a corrected charge amount Q1# obtained by correcting the charge amount Q1 accumulated in the charge accumulation unit CS1 (step S52). The corrected charge amount Q1# is a value obtained by correcting the charge amount Q1 accumulated in the charge accumulation unit CS1 to the charge amount corresponding to the accumulation time in the other charge accumulation units CS (charge accumulation units CS2 to CS4). The distance image processing unit 4 calculates the corrected charge amount Q1# according to the following formula (5).

[0147] Q1# = Q1 - Qg...(5) However, Q1 is the charge amount accumulated in the charge accumulation unit CS1 Qg is the charge amount corresponding to the external light component Note that in formula (5), it is assumed that the time interval during which charge is accumulated in the charge accumulation unit CS1 in the near-distance object measurement process and the time interval during which charge is accumulated in the charge accumulation unit CS1 in the far-distance object measurement process (Ta in FIG. 11) are the same value.

[0148] Next, the distance image processing unit 4 determines in which charge storage unit CS the charge amount corresponding to the reflected light is stored (step S53). The distance image processing unit 4 compares the charge amount Qg calculated in step S51 with the charge amounts stored in each of the charge storage units. In this case, for the charge storage units CS2 to CS4, the distance image processing unit 4 compares the charge amount Qg with the charge amounts Q2 to Q4 stored in CS2 to CS4 respectively. For the charge storage unit CS1, the distance image processing unit 4 compares the charge amount Qg with the corrected charge amount Q1# calculated in step S52. The distance image processing unit 4 determines the charge storage unit CS in which a charge amount larger than the charge amount Qg is stored as the charge storage unit CS in which the charge amount corresponding to the reflected light is stored.

[0149] When it is determined that the charge corresponding to the reflected light is stored in the charge storage units CS1 and CS2, it is determined that there is an object at the distance (first distance) corresponding to the zone Z1, and the distance to the object is calculated using equation (1) (step S54). In this case, the charge ratio R in equation (1) is R = (Q2 - Q3) / (Q1# + Q2 - 2×Q3). Since step S55 is the same as step S15 in FIG. 5, its description is omitted.

[0150] When it is determined that the charge corresponding to the reflected light is stored in the charge storage units CS3 and CS4, it is determined that there is an object at the distance (fourth distance) corresponding to the zone Z4, and the distance to the object is calculated using equation (1) (step S56). In this case, the charge ratio R in equation (1) is R = (Q4 - Q2) / (Q3 + Q4 - 2×Q2). Since step S57 is the same as step S19 in FIG. 5, its description is omitted.

[0151] When it is determined that the charge corresponding to the reflected light is stored in the charge storage units CS4 and CS1, it is determined that there is an object at the distance (fifth distance) corresponding to the zone Z5, and the distance to the object is calculated using equation (1) (step S58). In this case, the charge ratio R in equation (1) is R = (Q1# - Q2) / (Q4 + Q1# - 2×Q2). Since step S59 is the same as step S19 in FIG. 5, its description is omitted.

[0152] As described above, in the distance image capturing apparatus 1 according to the second modification of the second embodiment, in the short-distance object measurement process, charges corresponding to the reflected light RL are distributed and stored in the charge storage units CS1 and CS2 (some of the charge storage units CS). In the long-distance object measurement process, charges corresponding to the reflected light RL are distributed and stored in the charge storage units CS3, CS4, and CS1 (the charge storage unit CS including charge storage units CS different from some of the charge storage units CS).

[0153] Thereby, the distance image capturing apparatus 1 according to the second modification of the second embodiment can measure the distance to an object very close by using a two-tap configuration (charge storage units CS1 and CS2) in the short-distance object measurement process, and can measure the distance to an object by using a three-tap configuration (charge storage units CS3, CS4, and CS1) in the long-distance object measurement process. Therefore, without performing the branching process as shown in step S34 of FIG. 10, the distance to an object at a long distance can be measured to a greater distance.

[0154] In the above-described embodiment, the case where the distribution process performed in the unit accumulation time UT is repeatedly executed the number of times corresponding to one frame has been described as an example. However, it is not limited to this.

[0155] Since the distribution process of the embodiment includes a discharge process that requires a predetermined time, when the distribution process of the embodiment is repeatedly executed, it takes more time than when the conventional distribution process is repeatedly executed. Therefore, the maximum number of times of the distribution process that can be executed in one frame may be less than that in the conventional case. As a countermeasure, the distribution process of the embodiment and the conventional distribution process may be mixed at a certain ratio in one frame.

[0156] Also, the time period for executing the emission process in zone Z3 may be arbitrarily set according to the environment in which the distance image capturing device 1 is applied. For example, regarding the relationship between zone Z3 and the third distance, when zone Z3 is 1 [ns], the third distance is about 15 [cm]. When zone Z3 is 100 [ns], the third distance is about 15 [m]. When zone Z3 is 200 [ns], the third distance is about 30 [m].

[0157] For example, consider a case where the distance image capturing device 1 is mounted on a moving body and used for the purpose of avoiding a collision with an object existing in the moving direction. In this case, for example, assume that a moving body traveling at about 50 [km / h] applies brakes 0.033 [seconds] after recognizing an object existing at a long distance. In this case, the coasting distance is about 0.5 "m", and the braking distance is about 14.1 [m], depending on the road surface conditions. That is, if it is possible to recognize whether an object exists at a distance of about 15 [m] or not, a collision with that object can be avoided. In this case, the upper limit value of the third distance and the lower limit value of the fourth distance are set to 15 [m]. Therefore, zone Z3 is set to about 100 [ns].

[0158] Applying this concept to a moving body traveling at about 70 [km / h], the coasting distance is about 0.6 "m", the braking distance is about 27.6 [m], and if it is possible to recognize whether an object exists at a distance of about 30 [m] or not, a collision with that object can be avoided. In this case, the upper limit value of the third distance and the lower limit value of the fourth distance are set to 30 [m]. Therefore, zone Z3 is set to about 200 [ns].

[0159] Also, in at least one of the above-described embodiments, when the unit accumulation process is repeatedly performed in frame units, based on the amount of charge accumulated in each of the charge accumulation units CS during the current one-frame period, the content of the unit accumulation process (the processing content in each of the short-distance object measurement process, the discharge process, and the long-distance object measurement process) in the next one-frame period may be determined. For example, the distance image processing unit 4 first determines whether an object exists at a long distance in the first frame according to the first embodiment. If an object exists at a long distance, in the next frame, the distance to the object existing at a long distance is measured according to the second embodiment. If no object exists at a long distance, in the next frame as well, it is determined whether an object exists at a distance according to the first embodiment. Thereby, depending on the situation, the processing content can be changed for each frame between the case where an object exists at a long distance and the case where no object exists, and it becomes possible to respond according to the situation.

[0160] In addition, in at least one of the above-described embodiments, the light source unit 2 may include a plurality of light source devices having different properties. The plurality of light source devices here only need to have different light intensities irradiated per unit area at least, and any light source may be used. As the light source, for example, an LED (Light Emitting Diode), a laser, a VCSEL, etc. can be applied.

[0161] The surface light source irradiates the light pulse PO uniformly into the space to be imaged. When the surface light source irradiates the space to be imaged, since the space can be irradiated uniformly, there is an advantage that the resolution in that space can be increased. On the other hand, it is necessary to irradiate the light pulse PO uniformly over a wide range. For this reason, if one tries to comply with the so-called eye-safe standard defined in the standards (for example, IEC 60825-1, JIS C6802, etc.) that regulate the safety of laser products, there is a demerit that the intensity of the irradiated light must be lowered. This is because eye safety is an index defined by the intensity of the light irradiated per unit area.

[0162] Therefore, when the intensity of the light irradiated using the surface light source is kept low, the intensity of the reflected light reflected by an object existing at a far distance becomes weak. For this reason, even if the number of sorting processes performed in the unit accumulation time UT per frame is increased, the signal (charge corresponding to the reflected light) is small and is likely to be buried in noise.

[0163] The dot light source irradiates the space to be imaged with the light pulse PO uniformly, for example, by irradiating a light pulse PO having a predetermined dot pattern. For example, the light from the light source can be made parallel light using a collimator lens or the like, and light having a dot pattern can be irradiated by passing the parallel light through a diffractive optical element (DOE). This is because one or both surfaces of the DOE are diffractive surfaces. Note that it is also possible to omit the collimator lens by devising the light source and the design (see, for example, International Publication No. 2020 / 066981).

[0164] In the above description, the case where the dot pattern is a lattice pattern has been exemplified, but the present invention is not limited to this. The dot pattern may be a regular pattern such as hexagonal, or may be a random or pseudo-random pattern.

[0165] When light is irradiated using a dot light source, the spatial resolution becomes low because the space is unevenly irradiated. On the other hand, when a dot light source is used, it is possible to increase the intensity of the light per dot compared to a surface light source within the range of the light intensity defined by eye safety. That is, when the light intensity per unit area is the same, the dot light source can increase the local light intensity compared to the surface light source.

[0166] For this reason, when a dot light source is used, the signal (charge corresponding to the reflected light) due to the intensity of the reflected light reflected by an object existing at a far distance can be made large enough not to be buried in noise. Then, it becomes possible to separate and detect the reflected signal from a far distance (charge corresponding to the reflected light reflected by an object existing at a far distance) from the noise.

[0167] Also, according to the range of the distance that can be measured, the density and intensity of the dot pattern may be adjusted. For example, by increasing the density of the dot pattern and approaching the surface light source, the spatial resolution is adjusted to be higher. Alternatively, considering the eye safety regulations, the density of the dot pattern may be set low so that the light intensity per unit area does not exceed the threshold value.

[0168] Also, only dot light sources may be used, or a surface light source and dot light sources may be combined and used. When combining a surface light source and dot light sources, the surface light source and the dot light sources may be irradiated alternately, or the surface light source and the dot light sources may be used together and irradiated simultaneously.

[0169] When the surface light source and the dot light sources are irradiated alternately, a light pulse is irradiated using the dot light source periodically, for example, every frame, every several frames, every single unit accumulation time UT, or every plurality of single unit accumulation times UT, and irradiation of a light pulse using the surface light source is performed at other irradiation timings.

[0170] As described above, in the distance image capturing device 1 of the embodiment, the light source unit 2 includes a surface light source and dot light sources. In the irradiation by the surface light source and the irradiation by the dot light sources, the light intensity per unit area is the same. In the irradiation by the surface light source and the irradiation by the dot light sources, the intensity of the light irradiated by the dot light sources is greater than the intensity of the light irradiated by the surface light source, and the light intensity per unit area is the same. The distance image processing unit 4 switches between the irradiation by the surface light source and the irradiation by the dot light sources to perform measurement. Thereby, in the distance image capturing device 1 of the embodiment, measurement can be switched according to the purpose, such as increasing the resolution so that the space can be measured uniformly or increasing the measurement accuracy of an object existing at a far distance.

[0171] In addition, in the distance image capturing device 1 of the embodiment, the distance image processing unit 4 controls so that the irradiation by the surface light source and the irradiation by the dot light source are periodically repeated. For example, the distance image processing unit 4 periodically causes the surface light source and the dot light source to emit light alternately in units such as for each frame, for every several frames, for each single unit accumulation time UT, or for every plurality of single unit accumulation times UT. Thereby, while maintaining the resolution so that the space can be uniformly measured, it is possible to improve the measurement accuracy of an object existing at a far distance.

[0172] Note that the configuration described in at least one of the above-described embodiments may be applied to other configurations. For example, a measurement for determining the presence or absence of a far-distance object according to the first embodiment and a measurement of the distance to the far-distance object according to the second embodiment may be combined and the measurement may be performed. When performing the measurement according to the first embodiment, the dot light source according to the embodiment may be used.

[0173] In addition, in the above-described embodiment, in the formula (3), it was premised that the time interval (Ta in FIG. 7) during which charges are accumulated in the charge accumulation unit CS1 in the near-distance object measurement process and the far-distance object measurement process is the same value. When the two are different values, it is necessary to make a case-by-case division according to whether the reflected light RL is received in either the near-distance object measurement process or the far-distance object measurement process, and calculate the corrected charge amount Q1#.

[0174] Whether the reflected light RL is received in the short-distance object measurement process or in the long-distance object measurement process can be determined by comparing the charge amount Q2 and the charge amount Q4 when driven at the timing shown in FIG. 7. Here, the charge amount Q2 is the charge amount accumulated in the charge accumulation unit CS2. The charge amount Q4 is the charge amount accumulated in the charge accumulation unit CS4. Specifically, when the charge amount Q2 > the charge amount Q4, it can be determined that the reflected light RL is received in the short-distance object measurement process. When the charge amount Q2 < the charge amount Q4, it can be determined that the reflected light RL is received in the long-distance object measurement process. In this case, it is assumed that the accumulation time Ta for accumulating charge in the charge accumulation unit CS2 in the short-distance object measurement process is the same as the accumulation time Ta for accumulating charge in the charge accumulation unit CS4 in the long-distance object measurement process. If they are not the same, when the accumulation times of the charge accumulation units CS2 and CS4 are made the same, the charge amount Q4# estimated to be accumulated in the charge accumulation unit CS4 is calculated using the following formula (6), and the calculated charge amount Q4# and the charge amount Q2 are compared. The charge amount Q2 is the charge amount accumulated in the charge accumulation unit CS2.

[0175] Q4# = Ta / Ta# × Q4 …(6) However, Ta is the time interval during which charge is accumulated in the charge accumulation unit CS2 in the short-distance object measurement process Ta# is the time interval during which charge is accumulated in the charge accumulation unit CS4 in the long-distance object measurement process Q4 is the charge amount accumulated in the charge accumulation unit CS4 Q4# is the charge amount estimated to be accumulated in the charge accumulation unit CS4 when the accumulation time is Ta

[0176] When the reflected light RL is received in the short-distance object measurement process, the corrected charge amount Q1# is calculated by the following formula (7).

[0177] Q1# = (Q1 - (Ta + Ta#) / Ta × Qg) + Qg …(7) However, Ta is the time interval during which charge is accumulated in the charge accumulation unit CS1 in the short-distance object measurement process Ta# is the time interval during which charges are accumulated in the charge accumulation unit CS1 in the long-distance object measurement process Q1 is the amount of charge accumulated in the charge accumulation unit CS1 Qg is the amount of charge corresponding to the external light component (the amount of charge when charges are accumulated for the accumulation time Ta)

[0178] On the other hand, when the reflected light RL is received in the long-distance object measurement process, the corrected charge amount Q1# is calculated by the following formula (8).

[0179] Q1# = (Q1 - (Ta + Ta#) / Ta × Qg) × Ta / Ta# + Qg …(8) However Ta is the time interval during which charges are accumulated in the charge accumulation unit CS1 in the short-distance object measurement process Ta# is the time interval during which charges are accumulated in the charge accumulation unit CS1 in the long-distance object measurement process Q1 is the amount of charge accumulated in the charge accumulation unit CS1 Qg is the amount of charge corresponding to the external light component (the amount of charge when charges are accumulated for the accumulation time Ta)

[0180] In formula (4), further case-by-case analysis is required. Specifically, it is necessary to conduct case-by-case analysis when the reflected light RL is received by the charge accumulation units CS1 and CS2 in the short-distance object measurement process, and when the reflected light RL is received by the charge accumulation units CS2 and CS3. Furthermore, it is necessary to conduct case-by-case analysis when the reflected light RL is received by the charge accumulation units CS4 and CS1, and when the reflected light RL is received by the charge accumulation units CS1 and CS2 in the long-distance object measurement process.

[0181] When the reflected light RL is received by the charge storage units CS2 and CS3 in the short-distance object measurement process, and when the reflected light RL is received by the charge storage units CS4 and CS1 in the long-distance object measurement process, it is possible to make a determination by comparing the charge amount Q3 and the charge amount Q4 when driven at the timing shown in FIG. 9. Here, the charge amount Q3 is the charge amount accumulated in the charge storage unit CS3. The charge amount Q4 is the charge amount accumulated in the charge storage unit CS4. Specifically, when the charge amount Q3 > the charge amount Q4, it can be determined that the reflected light RL has been received by the charge storage units CS2 and CS3 in the short-distance object measurement process. When the charge amount Q2 < the charge amount Q4, it can be determined that the reflected light RL has been received by the charge storage units CS4 and CS1 in the long-distance object measurement process. In this case, it is assumed that the accumulation time Ta for accumulating charge in the charge storage unit CS3 in the short-distance object measurement process is the same as the accumulation time Ta for accumulating charge in the charge storage unit CS4 in the long-distance object measurement process. If they are not the same, when the accumulation times of the charge storage units CS3 and CS4 are made the same, the charge amount Q4# estimated to be accumulated in the charge storage unit CS4 is calculated using the following formula (9), and the calculated charge amount Q4# is compared with the charge amount Q3. The charge amount Q3 is the charge amount accumulated in the charge storage unit CS3.

[0182] Q4# = Ta / Ta# × Q4 …(9) However, Ta is the time interval during which charge is accumulated in the charge storage unit CS3 in the short-distance object measurement process Ta# is the time interval during which charge is accumulated in the charge storage unit CS4 in the long-distance object measurement process Q4 is the charge amount accumulated in the charge storage unit CS4 Q4# is the charge amount estimated to be accumulated in the charge storage unit CS4 when the accumulation time is Ta

[0183] When the reflected light RL is received by the charge storage units CS2 and CS3 in the short-distance object measurement process, the corrected charge amount Q2# is calculated by the following formula (10).

[0184] Q2# = (Q2 - (Ta + Ta#) / Ta × Qg) + Qg …(10) However Ta is the time interval during which charge is accumulated in the charge accumulation section CS2 in the short-distance object measurement process Ta# is the time interval during which charge is accumulated in the charge accumulation section CS2 in the long-distance object measurement process Q2 is the amount of charge accumulated in the charge accumulation section CS2 Qg is the amount of charge corresponding to the external light component (the amount of charge when charge is accumulated for the accumulation time Ta)

[0185] When the reflected light RL is received by the charge accumulation sections CS4 and CS1 in the long-distance object measurement process, the corrected charge amount Q1# is calculated by the following equation (11).

[0186] Q1# = (Q1 - (Ta + Ta#) / Ta × Qg) × Ta / Ta# + Qg …(11) However Ta is the time interval during which charge is accumulated in the charge accumulation section CS1 in the short-distance object measurement process Ta# is the time interval during which charge is accumulated in the charge accumulation section CS1 in the long-distance object measurement process Q1 is the amount of charge accumulated in the charge accumulation section CS1 Qg is the amount of charge corresponding to the external light component (the amount of charge when charge is accumulated for the accumulation time Ta)

[0187] When the reflected light RL is received by the charge accumulation sections CS1 and CS2 in the short-distance object measurement process and when the reflected light RL is received by the charge accumulation sections CS1 and CS2 in the long-distance object measurement process, when driven at the timing shown in FIG. 9, it is considered that the charge amount Q3 and the charge amount Q4 are almost the same. That is, the charge amount Q3 = the charge amount Q4, or the charge amount Q3 ≈ the charge amount Q4. In this case, for example, once, the provisional charge amounts Q1## and Q2## are calculated by the following equation (12).

[0188] Q1## = (Q1 - (Ta + Ta#) / Ta × Qg) Q2## = (Q2 - (Ta + Ta#) / Ta × Qg) …(12) However, Ta is the time interval during which charges are accumulated in the charge accumulation units CS1 and CS2 in the short-distance object measurement process Ta# is the time interval during which charges are accumulated in the charge accumulation units CS1 and CS2 in the long-distance object measurement process Q1 is the amount of charge accumulated in the charge accumulation unit CS1 Q2 is the amount of charge accumulated in the charge accumulation unit CS2 Qg is the amount of charge corresponding to the external light component (the amount of charge when charges are accumulated for the accumulation time Ta)

[0189] In this case, it is assumed that the accumulation time Ta for accumulating charges in the charge accumulation unit CS3 in the short-distance object measurement process is the same as the accumulation time Ta for accumulating charges in the charge accumulation unit CS4 in the long-distance object measurement process. If they are not the same, as described above, the charge amount Q4# is calculated using equation (9), and the calculated charge amount Q4# and the charge amount Q3 are almost the same, resulting in charge amount Q3 = charge amount Q4#, or charge amount Q3 ≈ charge amount Q4#. The charge amount Q3 is the amount of charge accumulated in the charge accumulation unit CS3. The charge amount Q4# is the amount of charge estimated to be accumulated in the charge accumulation unit CS4 when the accumulation times of the charge accumulation units CS3 and CS4 are the same.

[0190] Next, the sum (Q1## + Q2##) of the provisional charge amounts Q1## and Q2## is compared with the threshold Th. The threshold Th is a value set according to the amount of charge corresponding to the boundary between the intensity of the reflected light from an object at a short distance and the intensity of the reflected light from an object at a long distance. That is, when the sum (Q1## + Q2##) is greater than the threshold Th, it can be determined that the reflected light RL is received by the charge accumulation units CS1 and CS2 in the short-distance object measurement process. On the other hand, when the sum (Q1## + Q2##) is less than or equal to the threshold Th, it can be determined that the reflected light RL is not received by the charge accumulation units CS1 and CS2 in the short-distance object measurement process, that is, the reflected light RL is received by the charge accumulation units CS1 and CS2 in the long-distance object measurement process.

[0191] When reflected light RL is received by charge storage units CS1 and CS2 in the short-distance object measurement process, corrected charge amounts Q1# and Q2# are calculated according to the following formula (13).

[0192] Q1#=(Q1-(Ta+Ta#) / Ta×Qg)+Qg Q2#=(Q2-(Ta+Ta#) / Ta×Qg)+Qg …(13) However, Ta is the time interval during which charge is accumulated in charge storage units CS1 and CS2 in the short-distance object measurement process Ta# is the time interval during which charge is accumulated in charge storage units CS1 and CS2 in the long-distance object measurement process Q1 is the charge amount accumulated in charge storage unit CS1 Q2 is the charge amount accumulated in charge storage unit CS2 Qg is the charge amount corresponding to the external light component (the charge amount when charge is accumulated for the accumulation time Ta)

[0193] When reflected light RL is received by charge storage units CS1 and CS2 in the long-distance object measurement process, corrected charge amounts Q1# and Q2# are calculated according to the following formula (14).

[0194] Q1#=(Q1-(Ta+Ta#) / Ta×Qg)×Ta / Ta#+Qg Q2#=(Q2-(Ta+Ta#) / Ta×Qg)×Ta / Ta#+Qg …(14) However, Ta is the time interval during which charge is accumulated in charge storage units CS1 and CS2 in the short-distance object measurement process Ta# is the time interval during which charge is accumulated in charge storage units CS1 and CS2 in the long-distance object measurement process Q1 is the charge amount accumulated in charge storage unit CS1 Q2 is the charge amount accumulated in charge storage unit CS2 Qg is the charge amount corresponding to the external light component (the charge amount when charge is accumulated for the accumulation time Ta)

[0195] Using the amount of charge calculated above, determine whether an object is in any of the zones Z (steps S23, S33, S34, S53) and calculate the distance to the object (steps S15, S17, S19, etc.). When it is determined that there is an object at a long distance, correct the distance using the charge ratio, the base distance, and the ratio of the accumulation times Ta and Ta#, and use the corrected distance as the distance to the object. Also, in the above example, the case of correcting the amount of charge accumulated in the accumulation time Ta# according to the accumulation time Ta is illustrated, but it is not limited to this. At least, the two amounts of charge to be compared may be the amounts of charge accumulated in a time corresponding to the same accumulation time. The amount of charge accumulated in the accumulation time Ta may be corrected according to the accumulation time Ta#.

[0196] Thus, when the time intervals Ta and Ta# are not the same, it is necessary to correct the signal amount according to the time of the sorting process (the time for accumulating charge). When correcting the signal amount, the noise may increase or decrease, or the calculation load may increase. For this reason, it is desirable to make the time intervals Ta and Ta# the same in the short-distance object measurement process and the long-distance object measurement process.

[0197] Also, when the time intervals Ta and Ta# are not the same, in step S33 of FIG. 10, the distance image processing unit 4 compares the amounts of charge Q3 and Q4 accumulated in the charge accumulation units CS3 and CS4. When the amount of charge Q3 > the amount of charge Q4, the distance image processing unit 4 proceeds to step S39. When the amount of charge Q3 < the amount of charge Q4, the distance image processing unit 4 proceeds to step S41. When the amounts of charge of both are substantially the same, that is, when the amount of charge Q3 = the amount of charge Q4 or the amount of charge Q3 ≒ the amount of charge Q4, the distance image processing unit 4 proceeds to step S34. In step S34, the distance image processing unit 4 once calculates the provisional amounts of charge Q1## and Q2## shown in equation (12), and determines whether the sum (Q1## + Q2##) is greater than the threshold Th. When the sum (Q1## + Q2##) is greater than the threshold Th, the distance image processing unit 4 proceeds to step S35. When the sum (Q1## + Q2##) is less than or equal to the threshold Th, the distance image processing unit 4 proceeds to step S37.

[0198] All or part of the distance image capturing device 1 and the distance image processing unit 4 in the above-described embodiments may be implemented by a computer. In that case, a program for realizing this function may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be read into a computer system and executed to realize it. Note that the "computer system" referred to here includes hardware such as an OS and peripheral devices. Further, the "computer-readable recording medium" refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, or a storage device such as a hard disk built into a computer system. Furthermore, the "computer-readable recording medium" also includes those that hold a program dynamically for a short time, such as a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line, and those that hold a program for a certain period of time, such as a volatile memory inside a computer system serving as a server or a client in that case. Also, the above program may be for realizing a part of the aforementioned functions, and may further be realizable in combination with a program already recorded in a computer system for realizing the aforementioned functions, or may be realized using a programmable logic device such as an FPGA.

[0199] As described above, the embodiments of the present invention have been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and designs and the like within the scope not departing from the gist of the present invention are also included.

Explanation of Reference Numerals

[0200] 1... Distance image capturing device 2... Light source unit 3... Light receiving unit 32... Distance image sensor 321... Pixel 323... Vertical scanning circuit 4... Distance image processing unit 41... Timing control unit 42... Distance calculation unit 43... Measurement control unit 44…Memory unit CS…Charge storage unit PO…Optical pulse

Claims

1. A light source unit that irradiates a measurement space where a subject exists with light pulses, a pixel including a photoelectric conversion element that generates charges corresponding to the incident light and three or more charge storage units that store the charges, and a pixel drive circuit that distributes and stores the charges to each of the charge storage units in the pixel at a predetermined timing synchronized with the irradiation of the light pulses, a charge discharge unit that discharges the charges generated by the photoelectric conversion element, a distance image processing unit that determines the measurement distance to the subject using the amount of charge stored in each of the charge storage units, comprising: The distance image processing unit performs a plurality of unit accumulation processes in one frame period, and determines the measurement distance to the subject using the amount of charge stored in each of the three or more charge storage units, In the unit accumulation process, a short-distance object measurement process in which charges corresponding to the reflected light, which is the light pulse reflected by the subject, are distributed and stored in some of the three or more charge storage units, a discharge process in which the charge discharge unit discharges the charges generated by the photoelectric conversion element, and a long-distance object measurement process in which charges corresponding to the reflected light are stored in charge storage units including charge storage units different from the some charge storage units are sequentially performed. A distance image imaging device.

2. The pixel is provided with a first charge storage unit, a second charge storage unit, a third charge storage unit, and a fourth charge storage unit, which are four of the charge storage units, The distance image processing unit is Charges corresponding to the reflected light reflected by the subject at the first distance are sequentially distributed and stored in the first charge storage unit and the second charge storage unit, Charges corresponding to the reflected light reflected by the subject at a second distance greater than the first distance are sequentially distributed and stored in the second charge storage unit and the third charge storage unit, The discharge process is performed at the time when the reflected light reflected by the subject between the second distance and a third distance greater than the second distance by a distance outside the measurement range is received, Controls such that charges corresponding to the reflected light reflected by the subject at a fourth distance greater than the third distance are stored in the fourth charge storage unit. The distance image imaging device according to Claim 1.

3. The pixel is provided with a first charge storage unit, a second charge storage unit, a third charge storage unit, and a fourth charge storage unit, which are four of the charge storage units, The distance image processing unit is The charge corresponding to the reflected light reflected by the subject at the first distance is sequentially distributed and stored in the first charge storage unit and the second charge storage unit. The charge corresponding to the reflected light reflected by the subject at a second distance greater than the first distance is sequentially distributed and stored in the second charge storage unit and the third charge storage unit. The discharge process is performed at a time when the reflected light reflected by the subject between the second distance and a third distance greater than the second distance by a distance outside the measurement range from the second distance is received. Control is performed such that the charge corresponding to the reflected light reflected by the subject at a fourth distance greater than the third distance is sequentially distributed and stored in the fourth charge storage unit and the first charge storage unit. The distance image capturing device according to claim 1.

4. The pixel is provided with a first charge storage unit, a second charge storage unit, a third charge storage unit, and a fourth charge storage unit, which are four of the charge storage units. The distance image processing unit The charge corresponding to the reflected light reflected by the subject at the first distance is sequentially distributed and stored in the first charge storage unit and the second charge storage unit. The discharge process is performed at a time when the reflected light reflected by the subject between the first distance and a third distance greater than the first distance by a distance outside the measurement range from the first distance is received. The charge corresponding to the reflected light reflected by the subject at a fourth distance greater than the third distance is sequentially distributed and stored in the third charge storage unit and the fourth charge storage unit. Control is performed such that the charge corresponding to the reflected light reflected by the subject at a fifth distance greater than the fourth distance is sequentially distributed and stored in the fourth charge storage unit and the first charge storage unit. The distance image capturing device according to claim 1.

5. The pixel is provided with a first charge storage unit, a second charge storage unit, a third charge storage unit, and a fourth charge storage unit, which are four of the charge storage units. The distance image processing unit The charge corresponding to the reflected light reflected by the subject at the first distance is sequentially distributed and stored in the first charge storage unit and the second charge storage unit. The charge corresponding to the reflected light reflected by the subject at a second distance greater than the first distance is sequentially distributed and stored in the second charge storage unit and the third charge storage unit. The discharge process is performed at a time when the reflected light reflected by the subject between the second distance and a third distance greater than the second distance by a distance outside the measurement range from the second distance is received. Charges corresponding to the reflected light reflected by the subject at a fourth distance greater than the third distance are sequentially distributed and stored in the fourth charge storage unit and the first charge storage unit. Control is performed such that charges corresponding to the reflected light reflected by the subject at a fifth distance greater than the fourth distance are sequentially distributed and stored in the first charge storage unit and the second charge storage unit. The distance image capturing device according to claim 1.

6. The pixel is provided with a first charge storage unit, a second charge storage unit, a third charge storage unit, and a fourth charge storage unit, which are four said charge storage units. The distance image processing unit Charges corresponding to the reflected light reflected by the subject at the first distance are sequentially distributed and stored in the first charge storage unit and the second charge storage unit. The discharge process is performed at a time when the reflected light reflected by the subject between the first distance and a third distance greater than the first distance by a distance outside the measurement range from the first distance is received. Charges corresponding to the reflected light reflected by the subject at a fourth distance greater than the third distance are sequentially distributed and stored in the third charge storage unit and the fourth charge storage unit. Control is performed such that charges corresponding to the reflected light reflected by the subject at a fifth distance greater than the fourth distance are sequentially distributed and stored in the fourth charge storage unit and the first charge storage unit. The distance image capturing device according to claim 1.

7. The distance image processing unit determines the processing content in each of the short-distance object measurement process, the discharge process, and the long-distance object measurement process in the next one-frame period based on the amount of charges stored in each of the three or more charge storage units in the current one-frame period. The distance image capturing device according to claim 1.

8. The light source unit includes a surface light source that uniformly irradiates the measurement space with the light pulse and a dot light source that non-uniformly irradiates the measurement space with the light pulse. In the irradiation by the surface light source and the irradiation by the dot light source, the intensity of the light irradiated by the dot light source is greater than the intensity of the light irradiated by the surface light source, and the intensity of the light per unit area is the same. The distance image processing unit performs measurement by switching between irradiation by the surface light source and irradiation by the dot light source. The distance image capturing device according to claim 1.

9. The distance image processing unit controls so that irradiation by the surface light source and irradiation by the dot light source are periodically repeated. The distance image capturing device according to claim 8.

10. A light source unit that irradiates a measurement space where a subject exists with an optical pulse, a photoelectric conversion element that generates an electric charge in response to the incident light, and a pixel including three or more charge storage units that store the electric charge, and a pixel driving circuit that distributes and stores the electric charge to each of the charge storage units in the pixel at a predetermined timing synchronized with the irradiation of the optical pulse, a light receiving unit having the same, a charge discharging unit that discharges the electric charge generated by the photoelectric conversion element, and a distance image processing unit that determines a measurement distance to the subject using the amount of electric charge stored in each of the charge storage units, a distance image capturing method by a distance image capturing device comprising: The distance image processing unit performs the unit accumulation process a plurality of times in one frame period, and determines the measurement distance to the subject using the amount of electric charge stored in each of the three or more charge storage units. In the unit accumulation process, a short-distance object measurement process in which an electric charge corresponding to the reflected light, which is the optical pulse reflected by the subject, is distributed and stored in some of the three or more charge storage units, a discharging process in which the charge discharging unit discharges the electric charge generated by the photoelectric conversion element, and a long-distance object measurement process in which an electric charge corresponding to the reflected light is stored in a charge storage unit including a charge storage unit different from the some charge storage units are sequentially performed. Distance image capturing method.

Images