Distance image capturing device and distance image capturing method

Abstract

To accurately calculate a distance even when a FPN fluctuates depending on a situation of a device.SOLUTION: A distance image capturing device comprises: a light source portion; a light receiving portion having a pixel circuit in which a plurality of pixels are arranged in a two-dimensional matrix shape, and a pixel driving circuit; and a distance calculation portion that calculates a distance to a subject on the basis of an amount of charges accumulated in each of charge accumulation portions. A frame cycle includes: an accumulation period in which reflected light of an optical pulse is accumulated in the charge accumulation portion; a first output period in which a first voltage value corresponding to the amount of charges accumulated in the charge accumulation portion in the accumulation period is output; an FPN measurement period in which a fixed pattern noise accumulated in the charge accumulation portion; a second output period in which a second voltage value corresponding to the amount of charges accumulated in the charge accumulation portion in the FPN measurement period is output; and a readout period in which a differential signal corresponding to a difference between the first voltage value and the second voltage value is output. The distance calculation portion calculates the distance to the subject using a signal value of the differential signal.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a distance image capturing device and a distance image capturing method. [Background technology]

[0002] Time-of-Flight (hereinafter referred to as "TOF") distance imaging devices have been developed that utilize the known speed of light to measure the distance between a measuring device and an object based on the time of flight of light in space (measurement space) (see, for example, Patent Document 1). When measurements are performed using such distance imaging devices, fixed pattern noise (FPN), an error caused by circuit configuration, may occur and be mixed into the accumulated signal. For this reason, one solution is to calculate the distance after performing a correction (FPN correction) by subtracting a pre-measured FPN from the accumulated signal during measurement. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 4235729 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the FPN may vary depending on the conditions of the range imaging device. For example, the FPN varies between immediately after the range imaging device is turned on and after a certain time has passed since the device was turned on due to differences in the temperature inside the device. Furthermore, when a measurement-related variable, such as the number of integrations, is changed, the FPN also varies due to changes in the amount of heat generated inside the device and the accumulation time. When the FPN varies, it becomes difficult to accurately calculate the distance using corrections based on a pre-measured FPN.

[0005] One possible solution to this problem is to measure the FPN in advance for each of several measurement conditions, such as the number of integrations, but this requires measuring for each of many measurement conditions, which is time-consuming. Furthermore, it requires memory capacity to store the many measured FPNs along with the measurement conditions, making this an unrealistic approach. Therefore, there is a risk that FPN correction will not function properly in modes where measurements are performed that frequently change the number of integrations, such as when performing AE (auto exposure).

[0006] The present invention has been made in light of the above-mentioned problems, and aims to provide a distance image capturing device and a distance image capturing method that can accurately calculate distance even when the FPN varies depending on the situation of the device. [Means for solving the problem]

[0007] The distance image pickup device of the present invention includes a light receiving unit having a light source unit that irradiates a subject with a light pulse, a pixel circuit in which a plurality of pixels are arranged in a two-dimensional matrix, each pixel having a photoelectric conversion element that generates a charge according to the incident light, a charge discharge unit that discharges the charge, and a plurality of charge storage units that accumulate the charge, and a pixel drive circuit that distributes and accumulates the charge in each of the charge storage units at an accumulation timing synchronized with the irradiation of the light pulse according to a frame period, and causes the charge discharge unit to discharge the charge during a period other than the accumulation timing, and calculates a distance to the subject based on the amount of charge accumulated in each of the charge storage units. the frame period includes an accumulation period in which reflected light of the light pulse is accumulated in the charge accumulation unit, a first output period in which a first voltage value corresponding to the amount of charge accumulated in the charge accumulation unit during the accumulation period is output, an FPN measurement period in which fixed pattern noise accumulated in the charge accumulation unit is measured, a second output period in which a second voltage value corresponding to the amount of charge accumulated in the charge accumulation unit during the FPN measurement period is output, and a readout period in which a differential signal corresponding to the difference between the first voltage value and the second voltage value is output, and the distance calculation unit calculates the distance to the subject using the signal value of the differential signal.

[0008] The distance image capturing method of the present invention includes a light receiving unit having a light source unit that irradiates a subject with a light pulse, a pixel circuit in which a plurality of pixels are arranged in a two-dimensional matrix, the pixel circuit including a light source unit that irradiates a subject with a light pulse, a photoelectric conversion element that generates a charge according to the incident light, a charge drain unit that drains the charge, and a plurality of charge storage units that accumulate the charge, and a pixel drive circuit that distributes and accumulates the charge in each of the charge storage units at an accumulation timing synchronized with the irradiation of the light pulse according to a frame period, and causes the charge drain unit to drain the charge during a period other than the accumulation timing, and a distance calculation unit that calculates a distance to the subject based on the amount of charge accumulated in each of the charge storage units. a frame period including an accumulation period in which reflected light of the light pulse is accumulated in the charge accumulation unit, a first output period in which a first voltage value corresponding to the amount of charge accumulated in the charge accumulation unit during the accumulation period is output, an FPN measurement period in which fixed pattern noise accumulated in the charge accumulation unit is measured, a second output period in which a second voltage value corresponding to the amount of charge accumulated in the charge accumulation unit during the FPN measurement period is output, and a readout period in which a differential signal corresponding to the difference between the first voltage value and the second voltage value is output, and the distance calculation unit calculates the distance to the subject using the signal value of the differential signal. [Effects of the Invention]

[0009] According to the present invention, even if the FPN varies depending on the state of the device, the distance can be calculated with high accuracy. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a block diagram showing a schematic configuration of a distance image capturing device 1 according to an embodiment. [Figure 2] FIG. 2 is a block diagram showing a schematic configuration of a range image sensor 32 according to the embodiment. [Figure 3] FIG. 3 is a circuit diagram showing an example of the configuration of a pixel 321 according to an embodiment. [Figure 4] FIG. 2 is a diagram illustrating a frame period according to an embodiment. [Figure 5]2A to 2C are diagrams for explaining processing performed by the distance image capturing device 1 of the embodiment. [Figure 6] 2A to 2C are diagrams for explaining processing performed by the distance image capturing device 1 of the embodiment. [Figure 7] 4 is a flowchart showing the flow of processing performed by the distance image capturing device 1 of the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, a distance image capturing device according to an embodiment will be described with reference to the drawings.

[0012] Fig. 1 is a block diagram showing the schematic configuration of a distance image capturing device according to an embodiment. The distance image capturing device 1 includes, for example, a light source unit 2, a light receiving unit 3, and a distance image processing unit 4. Fig. 1 also shows a subject OB, which is an object to which the distance is measured by the distance image capturing device 1.

[0013] The light source unit 2 irradiates a light pulse PO into a measurement target space in which the object OB exists, under the control of the distance image processor 4. The light source unit 2 is, for example, a surface-emitting semiconductor laser module such as a vertical cavity surface-emitting laser (VCSEL). The light source unit 2 includes a light source device 21 and a diffuser plate 22.

[0014] The light source device 21 is a light source that emits laser light in a near-infrared wavelength band (for example, a wavelength band of 850 nm to 940 nm) that becomes the light pulses PO to be irradiated onto the subject OB. The light source device 21 is, for example, a semiconductor laser light-emitting element. The light source device 21 emits pulsed laser light in response to control from the timing control unit 41.

[0015] The diffusion plate 22 is an optical component that diffuses the laser light in the near-infrared wavelength band emitted by the light source device 21 to the extent of the surface that is irradiated onto the subject OB. The pulsed laser light diffused by the diffusion plate 22 is emitted as a light pulse PO and is irradiated onto the subject OB.

[0016] The light receiving unit 3 receives reflected light RL of the light pulse PO reflected by an object OB, the distance of which is to be measured in the range image pickup device 1, and outputs a pixel signal corresponding to the received reflected light RL. The light receiving unit 3 includes a lens 31 and a range image sensor 32.

[0017] The lens 31 is an optical lens that guides the incident reflected light RL to the range image sensor 32. The lens 31 outputs the incident reflected light RL to the range image sensor 32 side, and causes the light to be received (incident) by pixels provided in the light receiving region of the range image sensor 32.

[0018] The range image sensor 32 is an imaging element used in the range image capturing device 1. The range image sensor 32 has a plurality of pixels arranged in a two-dimensional matrix. Each pixel of the range image sensor 32 has one photoelectric conversion element, a plurality of charge accumulation units corresponding to this one photoelectric conversion element, and components that distribute charge to each charge accumulation unit. In other words, the pixel is an imaging element with a distribution configuration in which charge is distributed and stored in a plurality of charge accumulation units.

[0019] The range image sensor 32 distributes the charges generated by the photoelectric conversion elements to the respective charge accumulation sections under the control of the timing control section 41. The range image sensor 32 also outputs pixel signals according to the amount of charge distributed to the charge accumulation sections. The range image sensor 32 has multiple pixels arranged in a two-dimensional matrix, and outputs pixel signals for one frame corresponding to each pixel.

[0020] The distance image processing unit 4 controls the distance image pickup device 1 and calculates the distance to the subject OB. The distance image processing unit 4 includes a timing control unit 41, a distance calculation unit 42, and a measurement control unit 43.

[0021] The timing control unit 41 controls the timing of outputting various control signals required for measurement in accordance with the control of the measurement control unit 43. The various control signals here include, for example, a signal for controlling the irradiation of the light pulse PO, a signal for distributing and accumulating the reflected light RL in multiple charge accumulation units, a signal for controlling the number of integrations per frame, etc. The number of integrations is the number of times the process of distributing and accumulating electric charge in the charge accumulation units is repeated.

[0022] The distance calculation unit 42 calculates the distance to the object OB based on the pixel signals output from the distance image sensor 32, and outputs the calculated distance. The distance calculation unit 42 calculates the delay time from when the light pulse PO is emitted until when the reflected light RL is received based on the amount of charge accumulated in the multiple charge accumulation units, and calculates the distance to the object OB according to the calculated delay time.

[0023] The measurement control unit 43 controls the timing control unit 41. For example, the measurement control unit 43 sets the number of integration times for one frame, and controls the timing control unit 41 so that imaging is performed according to the set contents.

[0024] With this configuration, in the distance image capturing device 1, the light source unit 2 irradiates a light pulse PO in the near-infrared wavelength band onto the subject OB, and the light receiving unit 3 receives the reflected light RL reflected by the subject OB, and the distance image processing unit 4 outputs distance information measuring the distance to the subject OB.

[0025] Although Figure 1 shows a distance image capturing device 1 configured such that the distance image processing unit 4 is provided inside the distance image capturing device 1, the distance image processing unit 4 may also be a component provided outside the distance image capturing device 1.

[0026] The configuration of the range image sensor 32 will now be described with reference to Fig. 2. Fig. 2 is a block diagram showing a schematic configuration of the imaging element (range image sensor 32) used in the range image capturing device 1 of the embodiment.

[0027] As shown in FIG. 2, the distance image sensor 32 includes, for example, a light receiving area 320 in which a plurality of pixels 321 are arranged, a control circuit 322, a vertical scanning circuit 323 having a distribution operation, a horizontal scanning circuit 324, and a pixel signal processing circuit 325.

[0028] The light receiving area 320 is an area in which a plurality of pixels 321 are arranged, and FIG. 2 shows an example in which the pixels are arranged in a two-dimensional matrix of 8 rows and 8 columns. The pixels 321 accumulate electric charges corresponding to the amount of light they receive. The control circuit 322 comprehensively controls the range image sensor 32. The control circuit 322 controls the operation of the components of the range image sensor 32 in accordance with instructions from, for example, the timing control unit 41 of the range image processing unit 4. Note that the components of the range image sensor 32 may be directly controlled by the timing control unit 41, in which case the control circuit 322 may be omitted.

[0029] The vertical scanning circuit 323 is a circuit that controls the pixels 321 arranged in the light receiving region 320 for each row in accordance with control from the control circuit 322. The vertical scanning circuit 323 outputs a voltage signal corresponding to the amount of charge accumulated in each charge accumulation unit CS of the pixels 321 to the pixel signal processing circuit 325. For example, the vertical scanning circuit 323 distributes and accumulates the charges converted by the photoelectric conversion elements in the charge accumulation units of the pixels 321 at accumulation timing synchronized with the irradiation of the light pulse PO. Furthermore, the vertical scanning circuit 323 discharges the charges converted by the photoelectric conversion elements from charge discharge units (drain gate transistors GD, described later) during a period different from the accumulation timing. In this case, the vertical scanning circuit 323 is an example of a "pixel driving circuit."

[0030] The pixel signal processing circuit 325 is a circuit that performs predetermined signal processing (e.g., noise suppression processing, A / D conversion processing, etc.) on the voltage signals output from the pixels 321 in each column to the corresponding vertical signal lines in accordance with control from the control circuit 322.

[0031] Horizontal scanning circuit 324 is a circuit that sequentially outputs signals output from pixel signal processing circuit 325 to horizontal signal lines in accordance with control from control circuit 322. As a result, pixel signals corresponding to the amount of charge accumulated for one frame are sequentially output to distance image processing unit 4 via the horizontal signal lines.

[0032] In the following description, it is assumed that the pixel signal processing circuit 325 performs A / D conversion processing and the pixel signals are digital signals.

[0033] The configuration of the pixel 321 will now be described with reference to Fig. 3. Fig. 3 is a circuit diagram showing an example of the configuration of the pixel 321 arranged in the light receiving region 320 of the range image sensor 32 of the embodiment. Fig. 3 shows an example of the configuration of one pixel 321 out of the multiple pixels 321 arranged in the light receiving region 320. The pixel 321 is an example of a configuration including four pixel signal readout units.

[0034] The pixel 321 includes one photoelectric conversion element PD, a drain gate transistor GD, and four pixel signal readout units RU that output voltage signals from corresponding output terminals O. Each pixel signal readout unit RU includes a readout gate transistor G, a floating diffusion FD, a charge storage capacitance C, a reset gate transistor RT, a source follower gate transistor SF, and a select gate transistor SL. In each pixel signal readout unit RU, the floating diffusion FD and the charge storage capacitance C form a charge storage unit CS.

[0035] 3, the four pixel signal readout units RU are distinguished from one another by adding a number from 1 to 4 after the symbol "RU" of each pixel signal readout unit RU. Similarly, the components of each pixel signal readout unit RU are distinguished from one another by adding a number representing the pixel signal readout unit RU after the symbol.

[0036] 3, the pixel signal readout unit RU1, which outputs a voltage signal from the output terminal O1, includes a readout gate transistor G1, a floating diffusion FD1, a charge storage capacitance C1, a reset gate transistor RT1, a source follower gate transistor SF1, and a select gate transistor SL1. In the pixel signal readout unit RU1, the floating diffusion FD1 and the charge storage capacitance C1 form a charge storage unit CS1. The pixel signal readout units RU2 to RU4 have a similar configuration.

[0037] The photoelectric conversion element PD is a buried photodiode that photoelectrically converts incident light to generate electric charges and accumulates the generated electric charges. The photoelectric conversion element PD may have any structure. For example, the photoelectric conversion element PD may be a PN photodiode having a structure in which a P-type semiconductor and an N-type semiconductor are joined together, or a PIN photodiode having a structure in which an I-type semiconductor is sandwiched between a P-type semiconductor and an N-type semiconductor. Furthermore, the photoelectric conversion element PD is not limited to a photodiode, and may be, for example, a photogate type photoelectric conversion element.

[0038] In pixel 321, the photoelectric conversion element PD photoelectrically converts incident light to generate electric charges, which are then distributed to each of the four charge accumulation units CS, and voltage signals corresponding to the amount of electric charge distributed are output to the pixel signal processing circuit 325.

[0039] The configuration of pixels arranged in the range image sensor 32 is not limited to the configuration including four pixel signal readout units RU as shown in Fig. 3, but may be any pixel configured to include multiple pixel signal readout units RU. In other words, the number of pixel signal readout units RU (charge accumulation units CS) provided in the pixels arranged in the range image sensor 32 may be two, three, five or more.

[0040] 3 shows an example in which the charge storage section CS is configured by a floating diffusion FD and a charge storage capacitance C. However, the charge storage section CS only needs to be configured by at least a floating diffusion FD, and the pixel 321 may not have a charge storage capacitance C.

[0041] 3 shows an example of a configuration including a drain gate transistor GD, but if there is no need to discard the charge accumulated (remaining) in the photoelectric conversion element PD, the pixel 321 may be configured without the drain gate transistor GD. The drain gate transistor GD is an example of a "charge discharging section."

[0042] Here, when measurements are performed using the range imaging device 1, a certain amount of fixed pattern noise (FPN) resulting from circuit characteristics, signal processing, etc. may be mixed into the accumulated signal. Furthermore, depending on the status of the range imaging device 1, for example, the magnitude of the FPN may vary between immediately after the range imaging device is started and after a certain time has passed since the start-up.

[0043] To address this issue, in this embodiment, a period for measuring the FPN (an FPN measurement period in this embodiment) is provided in addition to a normal exposure period (an accumulation period in this embodiment) in one frame period. By providing an accumulation period and an FPN measurement period in one frame period, it is possible to measure the FPN according to the current situation in the distance image pickup device 1, and to correct the accumulated signal using the measured FPN. Even if the FPN fluctuates, the accumulated signal can be corrected using the FPN after the fluctuation, making it possible to calculate the distance with high accuracy.

[0044] Any method can be used to subtract the FPN from the accumulated signal. For example, in the depth image pickup device 1, CDS (Correlated Double Sampling), which is a method for reducing thermal noise that accompanies reading of the pixels 321, can be used.

[0045] Here, the frame period of the embodiment will be described with reference to Fig. 4. Fig. 4 is a diagram for explaining the frame period of the embodiment. Fig. 4 shows the breakdown of the processes to be driven per frame as the frame period and a timing chart corresponding to the breakdown.

[0046] This diagram shows a timing chart of elements corresponding to the items "LI," "G1" to "G4," "Drain," "Reset," and "Sel." "LI" indicates the irradiation timing of the light pulse PO. When it is in the ON state, light is emitted, and when it is in the OFF state, light is not emitted. "G1" to "G4" indicate the drive timing of the readout gate transistors G1 to G4. When it is in the ON state, charge is accumulated, and when it is in the OFF state, charge is not accumulated. "Drain" indicates the drive timing of the drain gate transistor GD. When it is in the ON state, charge is discharged, and when it is in the OFF state, charge is not discharged. "Reset" indicates the drive timing of the reset gate transistor RT. When it is in the ON state, the charge accumulated in the charge storage unit CS is discharged and reset, and when it is in the OFF state, the charge is not reset and remains accumulated. "Sel" indicates the drive timing of the select gate transistor SL. When it is in the ON state, the pixel signal readout unit RU is selected, and when it is in the OFF state, it is not selected.

[0047] As shown in FIG. 4, a frame period is formed by, for example, driving an accumulation period, a first output period, a reset period, an FPN measurement period, a second output period, a readout period, and a reset period in order.

[0048] The accumulation period is a period during which a light pulse PO is irradiated, reflected light RL is received, and charge corresponding to the reflected light RL is accumulated. During the accumulation period, the drain gate transistor GD is turned off and the readout gate transistors G1 to G4 are sequentially turned on at a timing synchronized with the timing of irradiation of the light pulse PO. As a result, charge photoelectrically converted by the photoelectric conversion element PD while each of the readout gate transistors G1 to G4 is controlled to be on is accumulated in the charge accumulation units CS1 to CS4. Then, at the timing when the readout gate transistor G4 is turned off, the drain gate transistor GD is turned on to discharge the charge. As a result, the charge photoelectrically converted by the photoelectric conversion element PD is discarded via the drain gate transistor GD. During the accumulation period, this driving to accumulate reflected light RL accompanying light emission is repeated a predetermined number of times.

[0049] The first output period is a period during which accumulation signals corresponding to the charges accumulated in each charge accumulation unit CS during the accumulation period are stored in a storage unit (memory) included in the distance image pickup device 1. During the first output period, the selection gate transistors SL are turned on, and voltages (analog values) corresponding to the amounts of charges accumulated in the charge accumulation units CS1 to CS4 are output from the output terminals O1 to O4 to the pixel signal processing circuit 325. The pixel signal processing circuit 325 stores each of the analog values ​​output from the output terminals O1 to O4 in memory as accumulation signals.

[0050] The reset period is a period in which the reset gate transistor RT is turned on to discharge the charge stored in the charge storage section CS and perform resetting.

[0051] The FPN measurement period is a period during which only FPN is accumulated in the charge accumulation unit CS. During the FPN measurement period, for example, the light pulse PO is not irradiated and the drain gate transistor GD is always kept in an on state.

[0052] The second output period is a period during which FPN signals corresponding to the charges accumulated in each charge accumulation unit CS during the FPN measurement period are stored in a storage unit (memory) included in the distance image pickup device 1. In the second output period, as in the first output period, the selection gate transistors SL are turned on, and voltages (analog values) corresponding to the amounts of charges accumulated in the charge accumulation units CS1 to CS4 are output from the output terminals O1 to O4 to the pixel signal processing circuit 325. The pixel signal processing circuit 325 stores each of the analog values ​​output from the output terminals O1 to O4 in memory as an FPN signal.

[0053] The readout period is a period during which a differential signal obtained by subtracting the FPN signal from the accumulation signal is output. The pixel signal processing circuit 325 acquires the accumulation signal stored in memory during the first output period. The pixel signal processing circuit 325 acquires the FPN signal stored in memory during the second output period. The pixel signal processing circuit 325 converts the analog value obtained by subtracting the FPN signal from the accumulation signal into a digital value through A / D conversion processing, and outputs the digital value as the differential signal.

[0054] The differential signal output during the readout period is used to calculate the distance to the object OB. The distance D to the object OB can be calculated using, for example, the following formula (1). Formula (1) shows an example in which charges corresponding to external light are accumulated in the charge accumulation unit CS1, and charges corresponding to external light are accumulated in the charge accumulation units CS2 and CS3, and charges corresponding to reflected light RL are also accumulated across them.

[0055] D=(Q3-Q1) / (Q2+Q3-2Q1)×c / 2×Tw+c / 2×Tc…(1) however, D indicates the distance to the subject OB. Q1 represents a signal value obtained by subtracting FPN from the amount of charge stored in the charge storage unit CS1. Q2 represents a signal value obtained by subtracting FPN from the amount of charge stored in the charge storage unit CS2. Q3 represents a signal value obtained by subtracting FPN from the amount of charge stored in the charge storage unit CS3. c represents the speed of light. Tw indicates the time width during which the light pulse PO is irradiated. Tc indicates the time difference between when the light pulse PO is irradiated and when the read gate transistor G2 is set to the ON state.

[0056] Here, the relationship between the accumulation period and the FPN measurement period will be described with reference to Fig. 5. Fig. 5 is a diagram for explaining the processing performed by the range image pickup device 1 of the embodiment. Fig. 5 shows a breakdown of driving during the accumulation period and the FPN measurement period.

[0057] As shown in Fig. 5, the number of integrations and the number of light emissions during the accumulation period are set to the same number (here, X times). Also, "Drain" during the accumulation period, that is, the drive timing of the drain gate transistor GD, is set to the off state (Off) while any of the readout gate transistors G1 to G4 is set to the on state (GateOn). Otherwise, that is, while all of the readout gate transistors G1 to G4 are set to the off state, it is set to the on state (On). The signal during the accumulation period, that is, the signal value S stored in the memory during the first output period, is expressed, for example, by the following equation (2):

[0058] S=Sig+N FPN …(2) where S is the signal value corresponding to the amount of charge accumulated during the accumulation period. Sig is a signal value corresponding to the amount of light received by pixel 321 (the signal value that is originally desired to be obtained). N FPN is the signal value corresponding to fixed pattern noise (FPN)

[0059] On the other hand, the number of integrations during the FPN measurement period is set to the same number as during the accumulation period (here, X times). However, Gate is set to Off, and all of the readout gate transistors G1 to G4 are set to the off state. The number of light emissions during the FPN measurement period is 0 (zero). Also, "Drain" during the FPN measurement period, i.e., the drain gate transistor GD, is set to the on state (On). Here, all of the readout gate transistors G1 to G4 are set to the off state, and the drain gate transistor GD is set to the on state (On), so as not to accumulate charge corresponding to external light in the charge accumulation section CS. The signal during the FPN measurement period, i.e., the signal value N stored in the memory during the second output period, is expressed, for example, by the following equation (3):

[0060] N=N FPN …(3) However, N FPN is the signal value corresponding to fixed pattern noise (FPN)

[0061] As the final output signal, that is, the differential signal output during the readout period, (SN=Sig) can be output. Therefore, the signal value Sig corresponding to the amount of light received by the pixel 321, which is the signal that is originally desired, can be obtained.

[0062] As described above, in the frame cycle, the length of time in the FPN measurement period is set to be the same as the length of time in the accumulation period. This allows the amount of fixed pattern noise (N FPN ) and the amount of fixed pattern noise (N FPN ) can be made equal to the amount of charge (Sig+N FPN The amount of fixed pattern noise contained in FPN ) can be removed by a simple operation (subtraction) called (SN).

[0063] The relationship between driving in each period in a frame cycle and the signal value that is held or output will now be described with reference to Fig. 6. Fig. 6 is a diagram for explaining the processing performed by the range image pickup device 1 of the embodiment.

[0064] The vertical axis of Fig. 6 indicates each period in a frame cycle, namely, the accumulation period, the first output period, the reset period, the FPN measurement period, the second output period, the readout period, and the reset period. The vertical axis of Fig. 6 indicates each element that holds or outputs a signal as a result of driving each period in a frame cycle, namely, FD (floating diffusion), SH1 (shift register 1), SH2 (shift register 2), and output.

[0065] As shown in FIG. 6, during the accumulation period, the FD stores a charge amount (Sig+N FPN During the first output period, a voltage value V(Sig+N) corresponding to the signal value S is accumulated in SH1. FPN ) is retained.

[0066] During the FPN measurement period, the FD stores the charge (N FPN During the second output period, a voltage value V(N FPN ) is held. During the readout period, a voltage value V(Sig) is output as an output (for example, a differential signal after A / D conversion).

[0067] Strictly speaking, noise caused by reset driving accumulates in the FD during the reset period, but this is considered to be sufficiently small compared to the FPN, and therefore will not be discussed in this embodiment.

[0068] Here, the flow of processing performed by the range image pickup device 1 will be described with reference to Fig. 7. Fig. 7 is a flowchart showing the flow of processing performed by the range image pickup device 1 of the embodiment. Fig. 7 shows the flow of driving in a frame cycle.

[0069] Step S10: The distance image pickup device 1 performs driving during the accumulation period. The distance image pickup device 1 receives reflected light RL corresponding to the irradiation of the light pulse PO, and accumulates charge corresponding to the received reflected light RL in the charge accumulation section CS, thereby performing driving during the accumulation period. As a result, charge corresponding to the reflected light RL is distributed and accumulated in any two charge accumulation sections CS out of the charge accumulation sections CS1 to CS4 in the pixel 321. The distance image pickup device 1 repeatedly executes driving during such an accumulation period the number of times as many times as the integration number. As a result, each of the charge accumulation sections CS accumulates a charge amount (Sig+N FPN ) is accumulated. Step S11: The distance image pickup device 1 calculates the amount of charge (Sig+N FPN ) is stored in memory by being held in a shift register. Step S12: During the reset period, the range image pickup device 1 performs resetting by discharging the charges accumulated in the charge accumulation unit CS. Step S13: The distance image pickup device 1 performs driving during the FPN measurement period. The distance image pickup device 1 performs driving during the FPN measurement period by turning off all of the readout gate transistors G1 to G4 and turning on the drain gate transistor GD for the same time as the accumulation period. As a result, a charge amount (N FPN ) is accumulated. Step S14: The distance image pickup device 1 calculates the amount of charge (N FPN ) is stored in memory by being held in a shift register. Step S15: During the readout period, the range image pickup device 1 reads the voltage value V(Sig+N FPN ) to obtain the voltage value V(N FPN ) is subtracted and a differential signal V(Sig) is output. Step S16: During the reset period, the range image pickup device 1 performs resetting by discharging the charges accumulated in the charge accumulation unit CS. Step S17: The range image pickup device 1 calculates the distance using the difference signal V(Sig) output in step S15.

[0070] As described above, the distance image capturing device 1 of the embodiment includes a light source unit 2, a light receiving unit 3, and a distance calculation unit 42. The light source unit 2 irradiates a light pulse PO onto the object OB. The light receiving unit 3 includes a range image sensor 32 (pixel circuit) in which a plurality of pixels 321, each including a photoelectric conversion element PD, a drain gate transistor GD, and a plurality of charge accumulation units CS, are arranged in a two-dimensional matrix, and a vertical scanning circuit 323 (pixel drive circuit). The photoelectric conversion element PD generates charge in response to incident light. The drain gate transistor GD discharges the charge. The charge accumulation unit CS accumulates the charge. The vertical scanning circuit 323 distributes and accumulates the charge in each of the charge accumulation units CS at accumulation timings synchronized with the irradiation of the light pulse PO according to a frame period, and causes the drain gate transistor GD to discharge the charge during periods other than the accumulation timings. The distance calculation unit 42 calculates the distance to the object OB based on the amount of charge accumulated in each of the charge accumulation units CS. The frame period includes an accumulation period, a first output period, an FPN measurement period, a second output period, and a readout period. The accumulation period is a period during which reflected light RL of the light pulse PO is accumulated in the charge accumulation unit CS. The first output period is a period during which a voltage value (first voltage value) corresponding to the amount of charge accumulated in each charge accumulation unit CS during the accumulation period is output. The FPN measurement period is a period during which the light pulse PO is not emitted. The FPN measurement period is a period during which FPN (fixed pattern noise) is measured. The second output period is a period during which a voltage value (second voltage value) corresponding to the amount of charge accumulated in each charge accumulation unit CS during the FPN measurement period is output. The readout period is a period during which a differential signal corresponding to the difference between the first voltage value and the second voltage value is output. The distance calculation unit 42 calculates the distance to the object OB using the signal value of the differential signal. As a result, the range image capturing device 1 of the embodiment can measure FPN for each frame. Therefore, even if FPN varies depending on the device's conditions, the distance can be calculated accurately.

[0071] Furthermore, in the distance image pickup device 1 of the embodiment, the vertical scanning circuit 323 (pixel driving circuit) causes the drain gate transistor GD (charge discharging unit) to discharge charge during the FPN measurement period, without distributing and storing charge in each charge storage unit CS. This allows the distance image pickup device 1 of the embodiment to accurately measure the amount of noise corresponding to FPN.

[0072] Furthermore, in the distance image pickup device 1 of the embodiment, the accumulation period and the FPN measurement period are equal in length. As a result, in the distance image pickup device 1 of the embodiment, FPN that has entered the charge accumulation unit CS during the accumulation period can be accurately measured during the FPN measurement period. Furthermore, FPN that has entered the charge accumulation unit CS during the accumulation period can be removed by a simple calculation.

[0073] The range image capture device 1 and range image processing unit 4 in the above-described embodiment may be implemented in whole or in part by a computer. In this case, a program for implementing these functions may be recorded on a computer-readable recording medium and then loaded and executed by a computer system. Note that the term "computer system" as used herein includes hardware such as an OS and peripheral devices. Furthermore, the term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, as well as storage devices such as hard disks built into a computer system. Furthermore, the term "computer-readable recording medium" may also include devices that dynamically store programs for a short period of time, such as communication lines used when transmitting programs via networks such as the Internet or telephone lines, or devices that store programs for a fixed period of time, such as volatile memory within a computer system acting as a server or client. The program may also be designed to implement some of the above-described functions, or may be capable of implementing the above-described functions in combination with a program already stored in the computer system, or may be implemented using a programmable logic device such as an FPGA.

[0074] Although an embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and includes designs within the scope of the gist of the present invention. [Explanation of symbols]

[0075] 1...Distance image capturing device 2...Light source section 3...Light receiving section 32...Distance image sensor 321...pixel (pixel circuit) 4...Distance image processing section 42...Distance calculation section CS…Charge storage section PO...light pulse RL…Reflected light

Claims

1. a light source unit for irradiating a subject with a light pulse; a pixel circuit in which a plurality of pixels are arranged in a two-dimensional matrix, the pixel circuit comprising a photoelectric conversion element that generates a charge according to incident light, a charge draining section that drains the charge, and a plurality of charge storage sections that accumulate the charge; and a pixel drive circuit that distributes and accumulates the charge in each of the charge storage sections at an accumulation timing synchronized with the irradiation of the light pulse according to a frame period, and causes the charge draining section to drain the charge during a period other than the accumulation timing; a distance calculation unit that calculates a distance to the subject based on the amount of charge accumulated in each of the charge accumulation units; Equipped with The frame period is an accumulation period in which the reflected light of the optical pulse is accumulated in the charge accumulation unit; a first output period for outputting a first voltage value corresponding to an amount of charge accumulated in the charge accumulation section during the accumulation period; an FPN measurement period for measuring fixed pattern noise accumulated in the charge accumulation unit; a second output period for outputting a second voltage value corresponding to an amount of charge accumulated in the charge accumulation unit during the FPN measurement period; a readout period during which a differential signal corresponding to a difference between the first voltage value and the second voltage value is output; Including, the distance calculation unit calculates the distance to the subject by using a signal value of the difference signal. Range imaging device.

2. the pixel driving circuit causes the charge discharging section to discharge the charges without distributing and storing the charges in each of the charge accumulation sections during the FPN measurement period; 2. The distance imaging device according to claim 1.

3. The accumulation period and the FPN measurement period have equal lengths.

2. The distance imaging device according to claim 1.

4. a light receiving section having a light source section which irradiates a subject with a light pulse; a pixel circuit in which a plurality of pixels are arranged in a two-dimensional matrix, the pixel circuit comprising a photoelectric conversion element which generates a charge according to incident light, a charge drain section which drains the charge, and a plurality of charge storage sections which accumulate the charge; a pixel drive circuit which distributes and accumulates the charge in each of the charge storage sections at an accumulation timing synchronized with the irradiation of the light pulse according to a frame period, and causes the charge drain section to drain the charge in a period other than the accumulation timing; and a distance calculation section which calculates a distance to the subject based on an amount of charge accumulated in each of the charge storage sections, The frame period is an accumulation period in which the reflected light of the optical pulse is accumulated in the charge accumulation unit; a first output period for outputting a first voltage value corresponding to an amount of charge accumulated in the charge accumulation section during the accumulation period; an FPN measurement period for measuring fixed pattern noise accumulated in the charge accumulation unit; a second output period for outputting a second voltage value corresponding to an amount of charge accumulated in the charge accumulation unit during the FPN measurement period; a readout period during which a differential signal corresponding to a difference between the first voltage value and the second voltage value is output; Including, the distance calculation unit calculates the distance to the subject by using a signal value of the difference signal. A distance imaging method.

Images