Range image capturing device and range image capturing method

Abstract

The distance image capturing device includes: a light source unit; a light receiving unit having a pixel circuit and a pixel driving circuit in which a plurality of pixels are arranged in a two-dimensional matrix; and a distance calculation unit that calculates the distance to the subject on the basis of the amount of charge stored in each of the charge storage units. The frame period includes: an accumulation period in which the charge accumulation unit accumulates reflected light of the optical pulse; a first output period in which a first voltage value corresponding to the amount of charge stored in the charge storage unit during the storage period is output; during the FPN measurement period, the fixed pattern noise accumulated in the charge accumulation part is measured; a second output period for outputting a second voltage value corresponding to the amount of charge stored in the charge storage unit during the FPN measurement period; and a readout period in which a difference signal corresponding to the difference between the first voltage value and the second voltage value is output. The distance calculation unit calculates the distance to the subject using the signal value of the difference signal.

Description

[0001] This application claims priority based on Japanese Patent Application No. 2023-108745 filed on June 30, 2023, the contents of which are cited here. Technical Field

[0002] The invention relates to a distance image photographing device and a distance image photographing method. Background Art

[0003] Using the known speed of light, a time of flight (TOF) distance image capture device is implemented that measures the distance between a measuring device and an object based on the flight time of light in space (measurement space) (for example, refer to Japanese Patent No. 4235729).

[0004] When measuring with such a distance image camera, there is a case where fixed pattern noise (FPN) is generated as an error caused by circuit configuration, etc. and mixed into the accumulated signal. Therefore, there is a countermeasure such as subtracting the pre-measured FPN from the accumulated signal during measurement to perform correction (FPN correction) and then calculate the distance. Summary of the invention

[0005] However, there are cases where the FPN changes depending on the condition of the distance image camera. For example, the FPN changes due to the difference in temperature inside the device just after the distance image camera is turned on and after a certain period of time has passed since the start. In addition, when variables related to measurement, such as the number of accumulated times, are changed, the FPN also changes as the heat generation and accumulation time inside the device change. If the FPN changes, it is difficult to calculate the distance with good accuracy by using the correction of the pre-measured FPN.

[0006] As a countermeasure, it is possible to measure FPN in advance for each of a plurality of measurement conditions such as the cumulative number of times, but it is necessary to measure for each of a plurality of measurement conditions and this is time-consuming. In addition, since a memory capacity is required to store a plurality of measured FPNs together with the measurement conditions, this is not practical. Therefore, in a measurement mode where the cumulative number of times is frequently changed, such as when AE (automatic exposure, automatic exposure adjustment) is performed, there is a possibility that FPN correction will not function.

[0007] The present invention has been made based on the above-mentioned technical problems, and an object of the present invention is to provide a range image capturing device and a range image capturing method which can calculate the distance with high accuracy even when the FPN varies depending on the status of the device.

[0008] The range image camera device disclosed in the present invention comprises: a light source unit for irradiating a light pulse to a subject; a light receiving unit having a pixel circuit and a pixel driving circuit, wherein the pixel circuit is obtained by arranging a plurality of pixels in a two-dimensional matrix, the pixels having a photoelectric conversion element for generating charges corresponding to incident light, a charge discharge unit for discharging the charges, and a plurality of charge accumulation units for accumulating the charges, the pixel driving circuit distributing and accumulating the charges to each of the charge accumulation units at an accumulation timing synchronized with the irradiation of the light pulse according to a frame cycle, and causing the charge discharge unit to discharge the charges during a period other than the accumulation timing; and a distance calculation unit for calculating a distance based on the amount of charge accumulated in each of the charge accumulation units. Calculating the distance to the above-mentioned object; the above-mentioned frame period includes: during the accumulation period, the above-mentioned charge accumulation unit accumulates the reflected light of the above-mentioned light pulse; during the first output period, outputting the first voltage value corresponding to the amount of charge accumulated in the above-mentioned charge accumulation unit during the above-mentioned accumulation period; during the FPN measurement period, measuring the fixed pattern noise accumulated in the above-mentioned charge accumulation unit; during the second output period, outputting the second voltage value corresponding to the amount of charge accumulated in the above-mentioned charge accumulation unit during the above-mentioned FPN measurement period; and during the readout period, outputting a difference signal corresponding to the difference between the above-mentioned first voltage value and the above-mentioned second voltage value; the above-mentioned distance calculation unit calculates the above-mentioned distance to the above-mentioned object using the signal value of the above-mentioned difference signal.

[0009] The distance image capturing method disclosed in the present invention is a distance image capturing method performed by a distance image capturing device, the distance image capturing device comprising: a light source unit for irradiating a light pulse to a subject; a light receiving unit having a pixel circuit and a pixel driving circuit, the pixel circuit being a plurality of pixels arranged in a two-dimensional matrix, the pixels comprising a photoelectric conversion element for generating a charge corresponding to the incident light, a charge discharge unit for discharging the charge, and a plurality of charge accumulation units for accumulating the charge, the pixel driving circuit distributing and accumulating the charge to each of the charge accumulation units at an accumulation timing synchronized with the irradiation of the light pulse according to a frame cycle, and causing the charge discharge unit to discharge the charge during a period other than the accumulation timing; and a distance calculation unit, based on the distance calculation at each of the charge accumulation units, The distance to the above-mentioned object is calculated based on the amount of charge accumulated in the charge accumulation section; wherein the above-mentioned frame period includes: during an accumulation period, the above-mentioned charge accumulation section accumulates the reflected light of the above-mentioned light pulse; during a first output period, a first voltage value corresponding to the amount of charge accumulated in the above-mentioned charge accumulation section during the above-mentioned accumulation period is output; during an FPN measurement period, a fixed pattern noise accumulated in the above-mentioned charge accumulation section is measured; during a second output period, a second voltage value corresponding to the amount of charge accumulated in the above-mentioned charge accumulation section during the above-mentioned FPN measurement period is output; and during a readout period, a difference signal corresponding to the difference between the above-mentioned first voltage value and the above-mentioned second voltage value is output; the above-mentioned distance calculation section calculates the above-mentioned distance to the above-mentioned object using the signal value of the above-mentioned difference signal.

[0010] Effects of the Invention

[0011] According to the present invention, even when FPN varies depending on the status of the device, the distance can be calculated with high accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a block diagram showing a schematic configuration of the distance image capturing device 1 according to the embodiment.

[0013] Figure 2 : is a block diagram showing a schematic configuration of the distance image sensor 32 according to the embodiment.

[0014] Figure 3 3 is a circuit diagram showing an example of the configuration of the pixel 321 according to the embodiment.

[0015] Figure 4 It is a diagram illustrating a frame period of an embodiment.

[0016] Figure 5 It is a diagram for explaining the processing performed by the distance image capturing device 1 according to the embodiment.

[0017] Figure 6 It is a diagram for explaining the processing performed by the distance image capturing device 1 according to the embodiment.

[0018] Figure 7 1 is a flowchart showing the flow of processing performed by the distance image capturing device 1 according to the embodiment. DETAILED DESCRIPTION

[0019] Below, while referring to the attached Figure 1 The distance image capturing device according to the embodiment will be described.

[0020] Figure 1 1 is a block diagram showing a schematic configuration of a distance image pickup device according to an embodiment. The distance image pickup device 1 includes, for example, a light source unit 2, a light receiving unit 3, and a distance image processing unit 4. Figure 1 4 also shows a subject OB which is an object whose distance is measured by the distance image capturing apparatus 1 .

[0021] The light source unit 2 irradiates the space of the measurement target where the object OB exists with the light pulse PO according to the control from the distance image processing unit 4. The light source unit 2 is, for example, a surface emitting semiconductor laser module such as a vertical cavity surface emitting laser (VCSEL). The light source unit 2 includes a light source device 21 and a diffusion plate 22.

[0022] The light source device 21 is a light source that emits laser light in a near-infrared wavelength band (e.g., a wavelength band of 850nm to 940nm) as a light pulse PO irradiated to the subject OB. The light source device 21 is, for example, a semiconductor laser light emitting element. The light source device 21 emits pulsed laser light according to the control from the timing control unit 41.

[0023] The diffuser 22 is an optical component that diffuses the near-infrared laser light emitted by the light source device 21 to a size of a surface irradiated to the object OB. The pulsed laser light diffused by the diffuser 22 is emitted as a light pulse PO and irradiates the object OB.

[0024] The light receiving unit 3 receives reflected light RL of the light pulse PO reflected by the object OB for measuring the distance in the distance image capturing device 1 , and outputs a pixel signal corresponding to the received reflected light RL. The light receiving unit 3 includes a lens 31 and a distance image sensor 32 .

[0025] The lens 31 is an optical lens that guides the incident reflected light RL toward the distance image sensor 32. The lens 31 emits the incident reflected light RL toward the distance image sensor 32 and causes the pixels in the light receiving area of ​​the distance image sensor 32 to receive light (is incident on the pixels in the light receiving area of ​​the distance image sensor 32).

[0026] The distance image sensor 32 is an imaging element used in the distance image imaging device 1. The distance image sensor 32 has a plurality of pixels arranged in a two-dimensional matrix. In each pixel of the distance image sensor 32, there is provided a photoelectric conversion element, a plurality of charge accumulation parts corresponding to the photoelectric conversion element, and a component for distributing charges to each charge accumulation part. That is, the pixel is an imaging element with a distribution structure that distributes charges to a plurality of charge accumulation parts and causes them to accumulate.

[0027] The distance image sensor 32 distributes the charge generated by the photoelectric conversion element to each charge accumulation unit according to the control from the timing control unit 41. In addition, the distance image sensor 32 outputs a pixel signal corresponding to the amount of charge distributed to the charge accumulation unit. In the distance image sensor 32, a plurality of pixels are arranged in a two-dimensional matrix, and the pixel signal corresponding to one frame of each pixel is output.

[0028] The distance image processing unit 4 controls the distance image capturing device 1 to calculate the distance to the object OB. The distance image processing unit 4 includes a timing control unit 41 , a distance calculation unit 42 , and a measurement control unit 43 .

[0029] The timing control unit 41 controls the timing of outputting various control signals required for measurement according to the control of the measurement control unit 43. The various control signals here include, for example, a signal for controlling the irradiation of the light pulse PO, a signal for distributing the reflected light RL to a plurality of charge accumulation units to accumulate them, a signal for controlling the number of accumulations per frame, etc. The number of accumulations is the number of times the process of distributing the charges to the charge accumulation units and accumulating them is repeated.

[0030] The distance calculation unit 42 calculates the distance to the object OB based on the pixel signal output from the distance image sensor 32, and outputs the calculated distance. The distance calculation unit 42 calculates the delay time from the irradiation of the light pulse PO to the reception of the reflected light RL based on the amount of charge accumulated in the plurality of charge accumulation units, and calculates the distance to the object OB based on the calculated delay time.

[0031] The measurement control unit 43 controls the timing control unit 41. For example, the measurement control unit 43 sets the number of accumulations for one frame, and controls the timing control unit 41 so that the timing control unit 41 performs imaging according to the set contents.

[0032] With such a configuration, in the distance image camera device 1, the light receiving unit 3 receives the reflected light RL obtained by reflecting the near-infrared light pulse PO irradiated on the subject OB by the light source unit 2, and the distance image processing unit 4 outputs the distance information measuring the distance to the subject OB.

[0033] In addition, Figure 1 , the distance image capturing device 1 is shown to include the distance image processing unit 4 inside the distance image capturing device 1 , but the distance image processing unit 4 may be a component included outside the distance image capturing device 1 .

[0034] Here, use Figure 2 The configuration of the distance image sensor 32 will be described. Figure 2 This is a block diagram showing a schematic configuration of an image pickup element (range image sensor 32 ) used in the range image pickup device 1 according to the embodiment.

[0035] like Figure 2 As shown, the distance image sensor 32 includes, for example, a light receiving area 320 in which a plurality of pixels 321 are arranged, a control circuit 322 , a vertical scanning circuit 323 having a distribution operation, a horizontal scanning circuit 324 , and a pixel signal processing circuit 325 .

[0036] The light receiving area 320 is an area where a plurality of pixels 321 are arranged. Figure 2, an example of a two-dimensional matrix configuration of 8 rows and 8 columns is shown. The pixel 321 accumulates a charge corresponding to the amount of light received. The control circuit 322 controls the distance image sensor 32 in general. The control circuit 322 controls the operation of the components of the distance image sensor 32, for example, according to instructions from the timing control unit 41 of the distance image processing unit 4. In addition, it is also possible to configure such that the control of the components included in the distance image sensor 32 is directly performed by the timing control unit 41, in which case the control circuit 322 can also be omitted.

[0037] The vertical scanning circuit 323 is a circuit that controls the pixels 321 arranged in the light receiving area 320 for each row according to the control from the control circuit 322. The vertical scanning circuit 323 outputs a voltage signal corresponding to the amount of charge accumulated in the charge accumulation section CS of each pixel 321 to the pixel signal processing circuit 325. For example, the vertical scanning circuit 323 distributes the charge converted by the photoelectric conversion element at the accumulation timing synchronized with the irradiation of the light pulse PO to the charge accumulation section of the pixel 321 and accumulates them. In addition, the vertical scanning circuit 323 discharges the charge converted by the photoelectric conversion element from the charge discharge section (drain gate transistor GD described later) during a period different from the accumulation timing. In this case, the vertical scanning circuit 323 is an example of a "pixel driving circuit".

[0038] The pixel signal processing circuit 325 is a circuit that performs predetermined signal processing (for example, noise suppression processing, A / D conversion processing, etc.) on the voltage signal output from the pixels 321 of each column to the corresponding vertical signal line under the control of the control circuit 322 .

[0039] The horizontal scanning circuit 324 sequentially outputs the signal output from the pixel signal processing circuit 325 to the horizontal signal line according to the control from the control circuit 322. Thus, the pixel signal corresponding to the charge amount accumulated for one frame is sequentially output to the image processing unit 4 via the horizontal signal line.

[0040] The following describes that the pixel signal processing circuit 325 performs A / D conversion processing and the pixel signal becomes a digital signal.

[0041] Here, use Figure 3 The structure of the pixel 321 will be described. Figure 3 1 is a circuit diagram showing an example of the configuration of a pixel 321 arranged in the light receiving area 320 of the distance image sensor 32 according to the embodiment. Figure 3 2 shows an example of the configuration of one pixel 321 among the plurality of pixels 321 arranged in the light receiving region 320. The pixel 321 is an example of a configuration including four pixel signal readout units.

[0042] The pixel 321 includes one photoelectric conversion element PD, a drain-gate transistor GD, and four pixel signal readout units RU that output voltage signals from corresponding output terminals O. The pixel signal readout units RU each include a readout gate transistor G, a floating diffusion unit FD, a charge storage capacitor C, a reset gate transistor RT, a source follower gate transistor SF, and a selection gate transistor SL. In each pixel signal readout unit RU, the floating diffusion unit FD and the charge storage capacitor C constitute a charge storage unit CS.

[0043] In addition, Figure 3 In the figure, each pixel signal readout unit RU is distinguished by assigning a number from "1" to "4" after the label "RU" of the four pixel signal readout units RU. Similarly, for each component of the four pixel signal readout units RU, the pixel signal readout units RU corresponding to each component are distinguished by indicating the number representing each pixel signal readout unit RU after the label.

[0044] exist Figure 3 In the pixel 321 shown, the pixel signal readout section RU1 that outputs a voltage signal from the output terminal O1 includes a readout gate transistor G1, a floating diffusion section FD1, a charge storage capacitor C1, a reset gate transistor RT1, a source follower gate transistor SF1, and a selection gate transistor SL1. In the pixel signal readout section RU1, the floating diffusion section FD1 and the charge storage capacitor C1 constitute a charge storage section CS1. The pixel signal readout sections RU2 to RU4 also have the same configuration.

[0045] The photoelectric conversion element PD is an implantable photodiode that generates electric charge by photoelectrically converting incident light and accumulates the generated electric charge. The structure of the photoelectric conversion element PD can be arbitrary. The photoelectric conversion element PD can be, for example, a PN photodiode having a structure in which a P-type semiconductor and an N-type semiconductor are joined, or a PIN photodiode having a structure in which an I-type semiconductor is sandwiched between a P-type semiconductor and an N-type semiconductor. In addition, the photoelectric conversion element PD is not limited to a photodiode, and can be, for example, a photoelectric conversion element of a photogate type.

[0046] In the pixel 321 , charges generated by the photoelectric conversion element PD performing photoelectric conversion on incident light are distributed to the four charge accumulation sections CS, and respective voltage signals corresponding to the charge amounts of the distributed charges are output to the pixel signal processing circuit 325 .

[0047] The configuration of the pixels arranged in the distance image sensor 32 is not limited to Figure 3The structure with four pixel signal readout sections RU as shown in the figure can be any pixel as long as it has a structure with multiple pixel signal readout sections RU. That is, the number of pixel signal readout sections RU (charge accumulation sections CS) provided in the pixel arranged in the distance image sensor 32 can be two, three, or five or more.

[0048] In addition, Figure 3 The pixel 321 of the structure shown shows an example in which the charge storage portion CS is formed of a floating diffusion portion FD and a charge storage capacitor C. However, the charge storage portion CS only needs to be formed of at least the floating diffusion portion FD, and the pixel 321 may not include the charge storage capacitor C.

[0049] In addition, Figure 3 The pixel 321 of the structure shown shows an example of a structure including a drain-gate transistor GD, but when it is not necessary to discard the charge accumulated (remaining) in the photoelectric conversion element PD, a structure without the drain-gate transistor GD is also possible. The drain-gate transistor GD is an example of a "charge discharge unit".

[0050] Here, if the distance image camera 1 is used for measurement, a certain fixed pattern noise (FPN) may be mixed into the accumulated signal due to circuit characteristics, signal processing, etc. Furthermore, depending on the condition of the distance image camera 1, for example, the size of the FPN may change immediately after the distance image camera is started and after a certain period of time has passed since the start.

[0051] As a countermeasure, in the present embodiment, in addition to the normal exposure period (accumulation period in the present embodiment), a period for measuring FPN (FPN measurement period in the present embodiment) is also provided during one frame period. By providing the accumulation period and the FPN measurement period during one frame period, the FPN corresponding to the current state of the distance image camera device 1 can be measured, and the accumulated signal can be corrected using the measured FPN. Even when the FPN changes, the accumulated signal can be corrected using the changed FPN, so the distance can be calculated with good accuracy.

[0052] As a method of subtracting FPN from the accumulated signal, any method may be adopted. For example, in the range image pickup device 1 , CDS (Correlated Double Sampling) may be used as a method for reducing thermal noise associated with reading out the pixel 321 .

[0053] Here, use Figure 4 The frame period of the embodiment will be described. Figure 4is a diagram illustrating a frame period of an embodiment. Figure 4 , the details of the process driven per frame and a timing chart corresponding to the details are shown as a frame cycle.

[0054] In this figure, a timing diagram of the elements corresponding to each item of "LI", "G1" to "G4", "Drain", "Reset" and "Sel" is shown. "LI" represents the irradiation timing of the light pulse PO. If it is set to the on state, light is irradiated, and if it is set to the off state, it becomes a state of not irradiating light. "G1" to "G4" represent the drive timing of the read gate transistors G1 to G4. If it is set to the on state, charge is accumulated, and if it is set to the off state, it becomes a state of not accumulating charge. "Drain" represents the drive timing of the drain gate transistor GD. If it is set to the on state, charge is discharged, and if it is set to the off state, it becomes a state of not discharging charge. "Reset" represents the drive timing of the reset gate transistor RT. If it is set to the on state, the charge accumulated in the charge accumulation part CS is reset by discharge, and if it is set to the off state, it is not reset and maintains the accumulated state. "Sel" represents the drive timing of the selection gate transistor SL. If it is set to the on state, the pixel signal readout part RU is selected, and if it is set to the off state, it becomes a non-selected state.

[0055] like Figure 4 As shown, the frame period is constituted by, for example, sequentially driving an accumulation period, a first output period, a reset period, an FPN measurement period, a second output period, a readout period, and a reset period.

[0056] The accumulation period is a period during which a light pulse PO is irradiated, and its reflected light RL is received to accumulate charges corresponding to the reflected light RL. During the accumulation period, at a timing synchronized with the timing of the irradiated light pulse PO, the drain gate transistor GD is set to an off state, and the read gate transistors G1 to G4 are sequentially set to an on state. Thus, the charges photoelectrically converted by the photoelectric conversion element PD during the period during which the read gate transistors G1 to G4 are respectively controlled to be in an on state are accumulated in the charge accumulation units CS1 to CS4. Furthermore, at the timing of setting the read gate transistor G4 to an off state, the drain gate transistor GD is set to an on state to discharge the charges. Thus, the charges photoelectrically converted by the photoelectric conversion element PD are discarded via the drain gate transistor GD. During the accumulation period, such a drive to accumulate the reflected light RL accompanying the light emission is repeatedly performed for a predetermined cumulative number of times.

[0057] The first output period is a period during which the accumulated signals corresponding to the charges accumulated in the respective charge accumulation sections CS during the accumulation period are stored in the storage section (memory) provided in the distance image pickup device 1. During the first output period, the selection gate transistor SL is turned on, and the voltage (analog value) corresponding to the amount of charge accumulated in the charge accumulation sections CS1 to CS4 is output from the output terminals O1 to O4 to the pixel signal processing circuit 325. The pixel signal processing circuit 325 stores the analog values ​​output from the output terminals O1 to O4 as accumulated signals in the memory.

[0058] The reset period is a period in which the reset gate transistor RT is turned on to discharge the charge accumulated in the charge accumulation portion CS to perform resetting.

[0059] The FPN measurement period is a period during which only the FPN is accumulated in the charge accumulation unit CS. During the FPN measurement period, for example, the light pulse PO is not irradiated, and the drain-gate transistor GD is always turned on.

[0060] The second output period is a period in which the FPN signals corresponding to the charges accumulated in the charge accumulation section CS during the FPN measurement period are stored in the storage section (memory) provided in the distance image pickup device 1. In the second output period, as in the first output period, the selection gate transistor SL is turned on, and the voltage (analog value) corresponding to the amount of charge accumulated in the charge accumulation sections CS1 to CS4 is output from the output terminals O1 to O4 to the pixel signal processing circuit 325. The pixel signal processing circuit 325 stores the analog values ​​output from the output terminals O1 to O4 as FPN signals in the memory.

[0061] The readout period is a period during which a difference signal obtained by subtracting the FPN signal from the accumulation signal is output. The pixel signal processing circuit 325 obtains the accumulation signal stored in the memory during the first output period. The pixel signal processing circuit 325 obtains the FPN signal stored in the memory during the second output period. The pixel signal processing circuit 325 converts the analog value obtained by subtracting the FPN signal from the accumulation signal into a digital value through A / D conversion processing as a difference signal.

[0062] The difference signal output during the readout period is used to calculate the distance to the object OB. The distance D to the object OB can be calculated, for example, using the following formula (1). Formula (1) shows an example in which charges corresponding to external light are accumulated in the charge accumulation section CS1, charges corresponding to external light are accumulated in the charge accumulation sections CS2 and CS3, and charges corresponding to the reflected light RL are accumulated across CS2 and CS3.

[0063] D=(Q3-Q1) / (Q2+Q3-2Q1)×c / 2×Tw+c / 2×Tc…(1)

[0064] in,

[0065] D represents the distance to the object OB.

[0066] Q1 represents a signal value obtained by removing FPN from the amount of charge accumulated in the charge accumulation unit CS1.

[0067] Q2 represents a signal value obtained by removing FPN from the amount of charge accumulated in the charge accumulation unit CS2.

[0068] Q3 represents a signal value obtained by removing FPN from the amount of charge accumulated in the charge accumulation unit CS3.

[0069] c represents the speed of light.

[0070] Tw represents the time width of the irradiated light pulse PO.

[0071] Tc represents the time difference from the irradiation of the light pulse PO to the time when the readout gate transistor G2 is set to the on state.

[0072] Here, use Figure 5 The relationship between the accumulation period and the FPN measurement period will be described. Figure 5 1 is a diagram for explaining the processing performed by the distance image capturing device 1 according to the embodiment. Figure 5 The details of the driving during the accumulation period and the FPN measurement period are shown in FIG.

[0073] like Figure 5 As shown, the cumulative number of times and the number of times of light emission during the accumulation period are set to the same number of times (here, X times). In addition, regarding the "drain" during the accumulation period, that is, the drive timing of the drain gate transistor GD, during the period when one of the read gate transistors G1 to G4 is set to the on state (Gate On), the drive timing of the drain gate transistor GD is set to the off state (Off). Otherwise, that is, during the period when all the read gate transistors G1 to G4 are set to the off state, they are set to the on state (On). The signal during the accumulation period, that is, the signal value S stored in the memory during the first output period, is expressed by, for example, the following formula (2).

[0074] S=Sig+N FPN …(2)

[0075] in,

[0076] S is a signal value corresponding to the amount of charge accumulated during the accumulation period.

[0077] Sig is a signal value corresponding to the amount of light received by the pixel 321 (the signal value originally desired to be obtained).

[0078] N FPN This is the signal value equivalent to fixed pattern noise (FPN)

[0079] On the other hand, the cumulative number of times during the FPN measurement period is set to the same number as the accumulation period (here X times). Among them, the gate (Gate) is set to be cut off (Off), and all of the read gate transistors G1 to G4 are set to the off state. The number of light emissions during the FPN measurement period is 0 (zero). In addition, the "drain" during the FPN measurement period, that is, the drain gate transistor GD, is set to the on state (On). Here, all of the read gate transistors G1 to G4 are set to the off state, and the drain gate transistor GD is set to the on state (On) to prevent the charge equivalent to the external light from accumulating in the charge accumulation unit CS. The signal during the FPN measurement period, that is, the signal value N stored in the memory during the second output period, is expressed by the following formula (3), for example.

[0080] N=N FPN …(3)

[0081] Among them, N FPN This is the signal value equivalent to fixed pattern noise (FPN)

[0082] As the final output signal, that is, the difference signal output during the readout period, (S-N=Sig) can be outputted. Therefore, a signal value Sig corresponding to the amount of light received by the pixel 321, which is the originally intended signal, can be obtained.

[0083] As described above, in the frame cycle, the length of the time in the FPN measurement period is set to be the same as the length of the time in the accumulation period. Thus, the noise amount (N) of the fixed pattern noise accumulated in the charge accumulation unit CS during the accumulation period can be reduced. FPN ) and the noise amount (N FPN Therefore, the charge amount (Sig+N) accumulated in the charge accumulation unit CS during the accumulation period can be set to the same amount. FPN ) contains the fixed pattern noise (N FPN ) can be removed by a simple operation (subtraction) such as (S-N).

[0084] Here, use Figure 6 The relationship between driving in each period of the frame cycle and the signal value to be held or output will be described. Figure 6 It is a diagram for explaining the processing performed by the distance image capturing device 1 according to the embodiment.

[0085] Figure 6 The vertical axis of shows an accumulation period, a first output period, a reset period, an FPN measurement period, a second output period, a readout period, and a reset period as respective periods in a frame cycle. Figure 6 The horizontal axis shows the result of driving each period in the frame cycle, and FD (floating diffusion), SH1 (shift register 1), SH2 (shift register 2), and output are shown as elements for holding or outputting signals.

[0086] like Figure 6 As shown, during the accumulation period, a charge amount (Sig+N FPN During the first output period, a voltage value V(Sig+N) corresponding to the signal value S is maintained in SH1. FPN ).

[0087] During the FPN measurement period, a charge amount (N) corresponding to the signal value N is accumulated in FD. FPN During the second output period, a voltage value V (N) corresponding to the signal value N is maintained in SH2. FPN During the readout period, a voltage value V(Sig) is output as an output (for example, a difference signal after A / D conversion).

[0088] In addition, during the reset period, strictly speaking, noise caused by the reset drive is accumulated in the FD, but since it can be considered to be sufficiently small compared with FPN, it is omitted in the present embodiment.

[0089] Here, use Figure 7 The flow of processing performed by the distance image capturing device 1 will be described. Figure 7 1 is a flowchart showing the flow of processing performed by the distance image capturing device 1 according to the embodiment. Figure 7 The driving process in the frame period is shown in FIG.

[0090] Step S10: The distance image camera device 1 drives during the accumulation period. The distance image camera device 1 receives the reflected light RL corresponding to the irradiation of the light pulse PO, and drives during the accumulation period by causing the charge accumulation unit CS to accumulate charges corresponding to the reflected light RL of the received light. Thus, charges corresponding to the reflected light RL are allocated to and accumulated in two of the charge accumulation units CS1 to CS4 in the pixel 321. The distance image camera device 1 repeatedly performs such driving during the accumulation period for a cumulative number of times. Thus, in each of the charge accumulation units CS, charges corresponding to the charge amount (Sig+N FPN ) equivalent charge.

[0091] Step S11: The distance image capturing device 1 outputs the charge amount (Sig+N FPN ) is held in a shift register and thereby stored in memory.

[0092] Step S12: During the reset period, the distance image capturing device 1 is reset by discharging the charge accumulated in the charge accumulation section CS.

[0093] Step S13: The distance image camera 1 is driven during the FPN measurement period. The distance image camera 1 is driven during the FPN measurement period by turning off all the readout gate transistors G1 to G4 and turning on the drain gate transistor GD at the same time as during the accumulation period. As a result, a charge corresponding to the charge amount (N FPN ) equivalent charge.

[0094] Step S14: The distance image capturing device 1 outputs the charge amount (N) accumulated in each charge accumulation section CS during the second output period. FPN ) is held in the shift register and thus stored in the memory.

[0095] Step S15: During the readout period, the distance image pickup device 1 reads the voltage value V(Sig+N FPN ) minus the voltage value V(N) held in step S14 FPN ) and the resulting difference signal V(Sig) is output.

[0096] Step S16: During the reset period, the distance image capturing device 1 discharges the charge accumulated in the charge accumulation section CS to thereby perform reset.

[0097] Step S17 : The distance image capturing device 1 calculates the distance using the difference signal V(Sig) outputted in step S15 .

[0098] As described above, the distance image pickup device 1 of the embodiment includes a light source unit 2, a light receiving unit 3, and a distance calculation unit 42. The light source unit 2 irradiates the light pulse PO to the object OB. The light receiving unit 3 includes a distance image sensor 32 (pixel circuit) and a vertical scanning circuit 323 (pixel driving circuit). The distance image sensor 32 (pixel circuit) is configured in a two-dimensional matrix with a plurality of pixels 321 including a photoelectric conversion element PD, a drain-gate transistor GD, and a plurality of charge accumulation units CS. The photoelectric conversion element PD generates a charge corresponding to the incident light. The drain-gate transistor GD discharges the charge. The charge accumulation unit CS accumulates the charge. The vertical scanning circuit 323 distributes and accumulates the charge to each charge accumulation unit CS at the accumulation timing synchronized with the irradiation of the light pulse PO according to the frame cycle, and causes the drain-gate transistor GD to discharge the charge during a period other than the accumulation timing. The distance calculation unit 42 calculates the distance to the object OB based on the amount of charge accumulated in each of the charge accumulation units CS. The frame period includes an accumulation period, a first output period, an FPN measurement period, a second output period, and a readout period. The accumulation period is a period during which the charge accumulation unit CS accumulates the reflected light RL of the light pulse PO. The first output period is a period during which a voltage value (first voltage value) corresponding to the amount of charge accumulated in each of the charge accumulation units CS during the accumulation period is output. The FPN measurement period is a period during which the light pulse PO is not irradiated. The FPN measurement period is a period during which FPN (fixed pattern noise) is measured. The second output period is a period during which a voltage value (second voltage value) corresponding to the amount of charge accumulated in each of the charge accumulation units CS during the FPN measurement period is output. The readout period is a period during which a difference signal corresponding to the difference between the first voltage value and the second voltage value is output. The distance calculation unit 42 calculates the distance to the object OB using the signal value of the difference signal. Thus, in the distance image capture device 1 of the embodiment, FPN can be measured per frame. Therefore, even when the FPN varies according to the status of the device, the distance can be calculated with high accuracy.

[0099] In addition, in the distance image pickup device 1 of the embodiment, the vertical scanning circuit 323 (pixel driving circuit) does not distribute and accumulate charges to each charge accumulation unit CS during the FPN measurement period, and causes the drain gate transistor GD (charge discharge unit) to discharge the charges. As a result, in the distance image pickup device 1 of the embodiment, the noise amount corresponding to FPN can be measured with good accuracy.

[0100] In addition, in the distance image camera device 1 of the embodiment, the lengths of the accumulation period and the FPN measurement period are equal. Therefore, in the distance image camera device 1 of the embodiment, the FPN mixed into the charge accumulation unit CS during the accumulation period can be measured with good accuracy during the FPN measurement period. In addition, the FPN mixed into the charge accumulation unit CS during the accumulation period can be removed by simple calculation.

[0101] The range image camera device 1 and the range image processing unit 4 of the above-mentioned embodiment may be implemented in whole or in part by a computer. In this case, a program for implementing the function may be recorded in a computer-readable recording medium, and the program recorded in the recording medium may be read and executed by the computer system. In addition, the "computer system" mentioned here includes hardware such as OS or peripheral devices. In addition, the "computer-readable recording medium" refers to a removable medium such as a floppy disk, a magneto-optical disk, a ROM, a CD-ROM, or a storage device such as a hard disk built into the computer system. Furthermore, the "computer-readable recording medium" may also include a medium that dynamically retains the program for a short period of time, such as a communication line when the program is sent via a network such as the Internet or a communication line such as a telephone line, or a medium that retains the program for a certain period of time, such as a volatile memory inside a computer system that is a server or client in this case. In addition, the above-mentioned program may be a program for implementing a part of the above-mentioned function, and may be a program that can realize the above-mentioned function by combining with a program already recorded in the computer system, or may be a program implemented using a programmable logic device such as an FPGA.

[0102] As mentioned above, although the embodiment of the present invention is described in detail with reference to the drawings, the specific configuration is not limited to the embodiment, and also includes designs and the like within the scope that do not depart from the gist of the present invention.

[0103] Industrial Applicability

[0104] According to the present invention, even when FPN varies depending on the status of the device, the distance can be calculated with high accuracy.

[0105] Description of Reference Numerals

[0106] 1…Distance image camera

[0107] 2…Light source

[0108] 3…Light receiving part

[0109] 32…Distance Image Sensor

[0110] 321...Pixels (Pixel Circuit)

[0111] 4…Distance image processing unit

[0112] 42…Distance calculation unit

[0113] CS…Charge storage unit

[0114] PO…Light Pulse

[0115] RL…Reflected light

Claims

1. A distance image camera device, wherein: have: A light source unit irradiates a light pulse to a subject; A light receiving unit having a pixel circuit and a pixel driving circuit, wherein the pixel circuit is obtained by arranging a plurality of pixels in a two-dimensional matrix, the pixels having a photoelectric conversion element for generating charges corresponding to incident light, a charge discharge unit for discharging the charges, and a plurality of charge accumulation units for accumulating the charges, the pixel driving circuit distributing and accumulating the charges to each of the charge accumulation units at an accumulation timing synchronized with the irradiation of the light pulse according to a frame cycle, and causing the charge discharge units to discharge the charges during a period other than the accumulation timing; as well as a distance calculation unit that calculates the distance to the subject based on the amount of charge accumulated in each of the charge accumulation units; The above frame period includes: During the accumulation period, the charge accumulation unit accumulates the reflected light of the light pulse; During the first output period, a first voltage value corresponding to the amount of charge accumulated in the charge accumulation section during the accumulation period is output; During the FPN measurement, the fixed pattern noise accumulated in the charge accumulation section is measured; outputting a second voltage value corresponding to the amount of charge accumulated in the charge accumulation section during the FPN measurement period during the second output period; and During the readout period, a difference signal corresponding to the difference between the first voltage value and the second voltage value is output; The distance calculation unit calculates the distance to the object using a signal value of the difference signal.

2. The distance image camera device according to claim 1, wherein: The pixel driving circuit does not distribute and accumulate the charges in the respective charge accumulation sections during the FPN measurement period, and causes the charge discharge section to discharge the charges.

3. The distance image camera device according to claim 1, wherein: The accumulation period and the FPN measurement period are equal in length.

4. A distance image capturing method, which is a distance image capturing method performed by a distance image capturing device, the distance image capturing device comprising: a light source unit for irradiating a light pulse to a subject; a light receiving unit having a pixel circuit and a pixel driving circuit, the pixel circuit being a plurality of pixels arranged in a two-dimensional matrix, the pixels comprising a photoelectric conversion element for generating a charge corresponding to incident light, a charge discharge unit for discharging the charge, and a plurality of charge accumulation units for accumulating the charge, the pixel driving circuit distributing and accumulating the charge to each of the charge accumulation units at an accumulation timing synchronized with the irradiation of the light pulse according to a frame cycle, and causing the charge discharge unit to discharge the charge during a period other than the accumulation timing; and a distance calculation unit that calculates the distance to the subject based on the amount of charge accumulated in each of the charge accumulation units; wherein, The above frame period includes: During the accumulation period, the charge accumulation unit accumulates the reflected light of the light pulse; During the first output period, a first voltage value corresponding to the amount of charge accumulated in the charge accumulation section during the accumulation period is output; During the FPN measurement, the fixed pattern noise accumulated in the charge accumulation section is measured; outputting a second voltage value corresponding to the amount of charge accumulated in the charge accumulation section during the FPN measurement period during the second output period; and During the readout period, a difference signal corresponding to the difference between the first voltage value and the second voltage value is output; The distance calculation unit calculates the distance to the object using a signal value of the difference signal.

Images