Photoelectric conversion device and photoelectric conversion system having photoelectric conversion device

Abstract

Provided is a photoelectric conversion device characterized by including: a photoelectric conversion unit for generating a photon detection signal by avalanche multiplication; a counter circuit for counting the photon detection signal output from the photoelectric conversion unit; a first period included in one frame; and a second period included in the one frame and not overlapping with the first period after the first period, wherein a value counted by the counter circuit on the basis of one photon detection signal in the first period is larger than a value counted by the counter circuit on the basis of one photon detection signal in the second period.

Description

Photoelectric conversion device and photoelectric conversion system having the photoelectric conversion device

[0001] The present invention relates to a photoelectric conversion device and a photoelectric conversion system having the photoelectric conversion device.

[0002] 2. Description of the Related Art Photoelectric conversion devices that utilize avalanche multiplication and are capable of detecting weak light at the single photon level are known.

[0003] JP 2023-039400 A

[0004] In the invention described in Patent Document 1, the amount of incident light is calculated from the count value and exposure time information, and the exposure time is controlled for each pixel. With this configuration, there is a concern that image quality may deteriorate due to the occurrence of LED flicker.

[0005] One aspect of the present invention is characterized in that it comprises a photoelectric conversion unit that generates a photon detection signal by avalanche multiplication, a counter circuit that counts the photon detection signals output from the photoelectric conversion unit, a first period included in one frame, and a second period included in the one frame that comes after the first period and does not overlap with the first period, wherein the value counted by the counter circuit based on one of the photon detection signals in the first period is greater than the value counted by the counter circuit based on one of the photon detection signals in the second period.

[0006] According to the present invention, deterioration in image quality can be reduced by adjusting the sensitivity of the photoelectric conversion device and appropriately counting the number of incident photons.

[0007] 1 is a schematic diagram of a photoelectric conversion device according to an embodiment; FIG. 2 is a schematic diagram of a PD substrate of a photoelectric conversion device according to an embodiment; FIG. 3 is a schematic diagram of a circuit board of a photoelectric conversion device according to an embodiment; FIG. 4 is a configuration example of a pixel circuit of a photoelectric conversion device according to an embodiment; FIG. 5 is a schematic diagram showing driving of a pixel circuit of a photoelectric conversion device according to an embodiment; FIG. 6 is a diagram explaining the principle of the present invention; FIG. 7 is a block diagram of a pixel of a photoelectric conversion device according to a first embodiment; FIG. 8 is a diagram showing a configuration example of a pixel circuit of a photoelectric conversion device according to the first embodiment; FIG. 9 is a timing chart of a photoelectric conversion device according to the first embodiment; FIG. 10 is a diagram showing a configuration example of a pixel circuit of a photoelectric conversion device according to a second embodiment; FIG. 11 is a timing chart of a photoelectric conversion device according to the second embodiment; FIG. 12 is a diagram showing a configuration example of a pixel circuit of a photoelectric conversion device according to a modification of the third embodiment; FIG. 13 is a timing chart of a photoelectric conversion device according to a modification of the third embodiment; FIG. 14 is a diagram showing a configuration example of a pixel circuit of a photoelectric conversion device according to a modification of the third embodiment; FIG. 15 is a timing chart of a photoelectric conversion device according to a modification of the third embodiment; FIG. 16 is a diagram showing a configuration example of a pixel circuit of a photoelectric conversion device according to a modification of the fourth embodiment; FIG. 17 is a timing chart of a photoelectric conversion device according to a modification of the fourth embodiment. FIG. 10 is a diagram showing an example of the configuration of a pixel circuit of a photoelectric conversion device according to a modified example of the fifth embodiment. FIG. 11 is a timing chart of a photoelectric conversion device according to a modified example of the fifth embodiment. FIG. 12 is a functional block diagram of a photoelectric conversion system according to a sixth embodiment. FIG. 13 is a schematic diagram according to a seventh embodiment. FIG. 14 is a schematic diagram according to the seventh embodiment. FIG. 15 is a functional block diagram of a photoelectric conversion system according to an eighth embodiment. FIG. 16 is a functional block diagram of a photoelectric conversion system according to a ninth embodiment. FIG. 17 is a functional block diagram of a photoelectric conversion system according to the ninth embodiment. FIG. 18 is a functional block diagram of a photoelectric conversion system according to a tenth embodiment. FIG. 19 is a functional block diagram of a photoelectric conversion system according to an eleventh embodiment. FIG. 20 is a functional block diagram of a photoelectric conversion system according to a twelfth embodiment. FIG. 21 is a functional block diagram of a photoelectric conversion system according to the twelfth embodiment.

[0008] The embodiments shown below are intended to embody the technical concept of the present invention and are not intended to limit the present invention. The size and positional relationship of components shown in each drawing may be exaggerated for clarity. In the following description, the same components may be designated by the same reference numerals and their description may be omitted.

[0009] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the following description, terms indicating specific directions or positions (for example, "upper," "lower," "right," "left," and other terms including these terms) will be used as necessary. The use of these terms is intended to facilitate understanding of the embodiments with reference to the drawings, and the meaning of these terms does not limit the technical scope of the present invention.

[0010] In this specification, a planar view refers to a view from a direction perpendicular to the light incident surface of the semiconductor layer. A cross-sectional view refers to a surface perpendicular to the light incident surface of the semiconductor layer. When the light incident surface of the semiconductor layer is a rough surface when viewed microscopically, the planar view is defined based on the light incident surface of the semiconductor layer when viewed macroscopically.

[0011] In the following description, the anode of the avalanche photodiode (APD) is set to a fixed potential, and the signal is extracted from the cathode side. Therefore, the first conductivity type semiconductor region in which charges of the same polarity as the signal charges are the majority carriers is an N-type semiconductor region, and the second conductivity type semiconductor region in which charges of the opposite polarity to the signal charges are the majority carriers is a P-type semiconductor region.

[0012] The present invention also applies when the cathode of the APD is set to a fixed potential and the signal is extracted from the anode side. In this case, the first conductivity type semiconductor region, which has charges of the same polarity as the signal charges as the majority carriers, is a P-type semiconductor region, and the second conductivity type semiconductor region, which has charges of the opposite polarity to the signal charges as the majority carriers, is an N-type semiconductor region. The following describes a case where one node of the APD is set to a fixed potential, but the potentials of both nodes may fluctuate.

[0013] In this specification, when the term "impurity concentration" is simply used, it means the net impurity concentration minus the amount compensated by impurities of the opposite conductivity type. In other words, "impurity concentration" refers to the net doping concentration. A region where the concentration of P-type added impurities is higher than the concentration of N-type added impurities is a P-type semiconductor region. Conversely, a region where the concentration of N-type added impurities is higher than the concentration of P-type added impurities is an N-type semiconductor region.

[0014] A configuration common to each embodiment of a photoelectric conversion device and a driving method thereof according to the present invention will be described with reference to FIGS. 1 to 5. FIG.

[0015] 1 is a diagram showing the configuration of a stacked photoelectric conversion device according to an embodiment of the present invention. The photoelectric conversion device 100 is configured by stacking and electrically connecting two chips: a sensor chip 11 and a circuit chip 21. A pixel region 12 is arranged on the sensor chip 11, and a circuit region 22 that processes signals detected in the pixel region 12 is arranged on the circuit chip 21.

[0016] 2 is a layout diagram of the sensor chip 11. Pixels 101, each having a photoelectric conversion unit 102 including an APD, are arranged in an array of multiple rows and multiple columns to form a pixel region 12. Typically, the pixels 101 are pixels for forming an image, but when used for TOF (Time of Flight), they do not necessarily need to form an image. In other words, the pixels 101 may be pixels for measuring the time and amount of light that arrives.

[0017] 3 is a configuration diagram of the circuit chip 21. It has a signal processing unit 103 that processes the charges photoelectrically converted by the photoelectric conversion unit 102 in FIG. 2, a readout circuit 112, a control pulse generation unit 115, a horizontal scanning circuit unit 111, signal lines 113, and a vertical scanning circuit unit 110.

[0018] The photoelectric conversion unit 102 in FIG. 2 and the signal processing unit 103 in FIG. 3 are electrically connected via connection wiring provided for each pixel.

[0019] The vertical scanning circuit section 110 receives the control pulses supplied from the control pulse generating section 115 and supplies the control pulses to each pixel. The vertical scanning circuit section 110 uses logic circuits such as a shift register and an address decoder.

[0020] The signal output from the photoelectric conversion unit 102 of each pixel is processed by the signal processing unit 103. The signal processing unit 103 is provided with a counter, a memory, etc., and digital values ​​are held in the memory.

[0021] The horizontal scanning circuit unit 111 inputs a control pulse to the signal processing unit 103 to sequentially select each column in order to read out the signal from the memory of each pixel in which the digital signal is held.

[0022] A signal is output to the signal line 113 from the signal processing unit 103 of the pixel selected by the vertical scanning circuit unit 110 for the selected column.

[0023] The signal output to the signal line 113 is output via an output circuit 114 to a recording unit or a signal processing unit outside the photoelectric conversion device 100 .

[0024] 2 , the pixels 100 may be arranged one-dimensionally in the pixel region 12. The function of the signal processing unit 103 does not necessarily need to be provided for each pixel 101, and for example, one signal processing unit 103 may be shared by multiple pixels 101, and signal processing may be performed sequentially.

[0025] Fig. 4 is an example of a block diagram including the equivalent circuits of Fig. 2 and Fig. 3. In Fig. 2, the photoelectric conversion unit 102 having the APD 201 is provided on the sensor chip 11, and the other members are provided on the circuit chip 21.

[0026] The APD 201 generates charge pairs in response to incident light through photoelectric conversion. A voltage VL (first voltage) is supplied to the anode of the APD 201. A voltage VH (second voltage) higher than the voltage VL supplied to the anode is supplied to the cathode of the APD 201. A reverse bias voltage is supplied to the anode and cathode so that the APD 201 performs avalanche multiplication. With such a voltage supplied, the charge generated by the incident light undergoes avalanche multiplication, generating an avalanche current.

[0027] When a reverse bias voltage is supplied, there are two modes: Geiger mode, in which the potential difference between the anode and cathode is greater than the breakdown voltage, and linear mode, in which the potential difference between the anode and cathode is close to or less than the breakdown voltage. An APD operating in Geiger mode is called a SPAD. For example, the voltage VL (first voltage) is -30 V, and the voltage VH (second voltage) is 1 V.

[0028] The quench element 202 is connected to a power supply that supplies a voltage VH and the APD 201. The quench element 202 has a function of converting a change in avalanche current generated in the APD 201 into a voltage signal. The quench element 202 functions as a load circuit (quench circuit) during signal multiplication by avalanche multiplication, and has the function of suppressing avalanche multiplication by suppressing the voltage supplied to the APD 201 (quench operation).

[0029] The signal processing unit 103 has a waveform shaping unit 210, a counter circuit 211, and a selection circuit 212. In this specification, the signal processing unit 103 may have any one of the waveform shaping unit 210, the counter circuit 211, and the selection circuit 212.

[0030] The waveform shaping unit 210 shapes the potential change of the cathode of the APD 201 obtained when photons are detected, and outputs a pulse signal. For example, an inverter circuit is used as the waveform shaping unit 210. While Fig. 4 shows an example in which one inverter is used as the waveform shaping unit 210, a circuit in which multiple inverters are connected in series, or another circuit that has a waveform shaping effect, may also be used.

[0031] The counter circuit 211 counts the pulse signal output from the waveform shaping unit 210 and holds the count value. When a control pulse pRES is supplied via a drive line 213, the signal held in the counter circuit 211 is reset.

[0032] 1 via a drive line 214 (not shown in FIG. 3 ) in FIG. 4 , and switches between electrical connection and disconnection between the counter circuit 211 and the signal line 113. The selection circuit 212 includes, for example, a buffer circuit for outputting a signal.

[0033] The electrical connection may be switched by disposing a switch such as a transistor between the quench element 202 and the APD 201 or between the photoelectric conversion unit 102 and the signal processing unit 103. Similarly, the supply of the voltage VH or the voltage VL to the photoelectric conversion unit 102 may be electrically switched using a switch such as a transistor.

[0034] In this embodiment, a configuration using the counter circuit 211 has been described. However, instead of the counter circuit 211, the photoelectric conversion device 100 may be configured to acquire pulse detection timing using a time-to-digital converter (hereinafter, referred to as TDC) and a memory. In this case, the generation timing of the pulse signal output from the waveform shaping unit 210 is converted into a digital signal by the TDC. To measure the timing of the pulse signal, a control pulse pREF (reference signal) is supplied to the TDC from the vertical scanning circuit unit 110 in FIG. 1 via a drive line. The TDC acquires, as a digital signal, a signal obtained when the input timing of the signal output from each pixel via the waveform shaping unit 210 is converted into a relative time based on the control pulse pREF.

[0035] 5A and 5B are diagrams showing the relationship between the operation of an APD and an output signal. Fig. 5A is a diagram showing the APD 201, quench element 202, and waveform shaping unit 210 of Fig. 2. Here, the input side of the waveform shaping unit 210 is referred to as node A, and the output side is referred to as node B. Fig. 5B shows the waveform change of node A in Fig. 5A, and Fig. 5C shows the waveform change of node B in Fig. 5A.

[0036] Between times t0 and t1, a potential difference of VH-VL is applied to the APD 201 in FIG. 5(a).

[0037] When a photon is incident at time t1, an avalanche multiplication current flows through the quench element 202, causing the voltage at node A to drop. As the voltage drop increases further and the potential difference applied to the APD 201 decreases, avalanche multiplication in the APD 201 stops, and the voltage level at node A no longer drops below a certain value. Thereafter, a current flows from voltage VL to node A to compensate for the voltage drop, and at time t3, node A settles to its original potential level.

[0038] At this time, the portion of the output waveform at node A that exceeds a certain threshold is waveform-shaped by waveform shaping section 210 and output as a signal at node B.

[0039] The principle of the present invention will be explained using Figure 6. Here, Figure 6(i) is a diagram showing the probability of detecting the first photon with respect to time under each light intensity condition and an example of a weighting coefficient for each period. Figure 6(ii) is a graph showing the probability of photon detection with respect to light intensity. Figure 6(iii) is a graph showing the number of counts of the photon detection signal with respect to light intensity. Here, the first photon detected within each detection period is called the first photon.

[0040] As shown in Figure 6(i), under high light conditions, the probability of detecting the first photon is higher the earlier the time, and the detection probability decreases as time passes. When the first half of the detection period is divided into the first period and the second half into the second period, there is a high probability that detection of the first photon will be completed during the first period. Under medium light conditions, the photon detection probability also decreases over time, but compared to high light conditions, the time dependence of the first photon detection probability is low, and there is a possibility that the first photon will be detected even during the second period. Under low light conditions, the detection probability of the first photon is almost independent of time.

[0041] In Fig. 6(ii), dashed line A indicates the probability of detecting the first photon in the first period for each light intensity, and dashed line B indicates the probability of detecting the first photon in the second period for each light intensity. The sum of dashed lines A and B, that is, the straight line A+B, indicates the probability of detecting the first photon in each light intensity regardless of the period.

[0042] As shown by dashed line A, in the first period, the probability of detecting a photon increases as the light intensity increases, eventually saturating under high light conditions. In particular, under low to medium light conditions, the probability increases exponentially with increasing light intensity. On the other hand, as shown by dashed line B, in the second period, the probability of detecting a photon increases as the light intensity increases under low to medium light conditions, but decreases under medium to high light conditions. This is because, under high light conditions, the first photon is often detected during the first period, while photons detected during the second period are those that arrive later than the first photon. The probability of detecting a first photon at each light intensity, as shown by the straight line A+B, which is the sum of dashed lines A and B, increases as the light intensity increases under low to medium light conditions, and converges to a constant value under medium to high light conditions.

[0043] Figure 6(iii) shows a graph of the count value at each light intensity. Similar to the detection probability of the first photon shown in Figure 6(ii), the second and subsequent photons are incident on the APD. That is, under low to medium light conditions, the photon count number increases as the light intensity increases, and under medium to high light conditions, the count value saturates as the light intensity increases. Under high light conditions, the increase in the count value and the increase in the number of incident photons no longer correspond one-to-one, and it can be said that the weight of one count in the count value increases.

[0044] In other words, under low to medium light conditions, the expected value of the number of detected photons depends on the light intensity, but under medium to high light conditions, the weight of the count value depends on the light intensity.

[0045] First Embodiment FIG. 7 shows a block diagram of one pixel of a photoelectric conversion device according to a first embodiment.

[0046] A voltage VL (first voltage) is supplied to the anode of the APD 301, and a voltage VH (second voltage) higher than the voltage VL supplied to the anode is supplied to the cathode of the APD 301. A reverse bias voltage that causes avalanche multiplication is applied to the APD 301.

[0047] The output signal Vc of the APD 301 is input to a waveform shaping circuit 303 in the subsequent stage. A count control circuit 304, a counter circuit 305, and a readout circuit 306 are further provided in the subsequent stage of the waveform shaping circuit 303, and the output signal of the readout circuit is output to an output line. Here, the count control circuit 304 is controlled by a control signal CTRL, and an enable signal EN and a reset signal RES are input to the counter circuit 305. A selection signal SEL is input to the readout circuit 306. The photoelectric conversion device according to this embodiment includes an APD that operates by a so-called passive recharge method, in which a fixed potential is input to the quench element 302.

[0048] FIG. 8 shows an example of the circuit configuration of the photoelectric conversion device according to the first embodiment.

[0049] An inverter circuit is provided as an example of the waveform shaping circuit 303 .

[0050] A control circuit 311 and an XOR circuit 321 are provided as the count control circuit 304, and T-flip-flop circuits 312, 313, 314, and 315 are provided as the counter circuit 305 to configure an asynchronous binary counter. The readout circuit 306 is a tri-state buffer. In the circuit configuration example shown in Figure 8, the control circuit 311 is used to selectively input a signal corresponding to photon detection to the least significant bit or the middle bit of the counter circuit 305, thereby adding different integer values ​​to the counter circuit 305 depending on the photon detection timing.

[0051] Fig. 9 shows a timing chart of the operation per exposure period corresponding to one frame of the photoelectric conversion device having the circuit shown in Fig. 8. Hereinafter, Low and High of the pulse signal will be represented as L and H, respectively.

[0052] If the period from time t0 to time t1 is defined as a first period, the control signal CTRL is controlled to H during the first period, and the weighting coefficient of the count value during this period is set to 4. The period from time t1 to time t3, which is after the first period and does not overlap with the first period in time, is defined as a second period, and the control signal CTRL is controlled to L. The weighting coefficient of the count value during this period is set to 1. After time t3, the signal VR and the control signal CTRL are driven repeatedly from time t0 to time t3 a predetermined number of times.

[0053] At time t0, the enable signal EN rises, starting exposure of the kth frame. At the same time, the signal VR is set to L to initialize the control circuit 311. After initialization, IN1 and IN2, which correspond to the outputs of the control circuit 311, are both L.

[0054] When a photon is incident on the APD 301 at time t2, the output signal Vc drops. The output signal Vc is shaped by the shaping circuit 303 and input to the control circuit 311, which latches IN1 to H. IN1 is input to the T-flip-flop 312, which is the least significant bit of the counter. That is, the count value is incremented by +1.

[0055] At time t3, the signal VR goes low. The control circuit 311 is initialized again, and both IN1 and IN2 go low. The control signal CTRL is high from time t3 to time T5.

[0056] When a photon is incident on the APD 301 at time t4, the output signal Vc drops and a pulse signal is input to the shaping circuit 303. At this time, IN2 is latched to H and input to the XOR circuit 321. The signal output from the T-flip-flop 313 is input to the other input terminal of the XOR circuit 321. That is, IN2 is counted by interrupting the middle bit of the counter, and the counter value is incremented by +4 for each IN2 signal.

[0057] Photons are also incident on the APD 301 at times t6 and t7, but the photons are not counted.

[0058] At time t10, the enable signal EN falls, and the exposure of the kth frame ends.

[0059] From time t11 to time t12, the selection signal SEL goes to H, and the signal held in the counter circuit 305 is read out of the pixel.

[0060] At time t13, a photon is incident on the APD 301, but since the enable signal EN is at L, the photon is not counted.

[0061] From time t14 to t15, the reset signal RES goes high, and the count value held by the counter circuit 305 is reset.

[0062] At time t16, the enable signal EN rises again, and exposure of the (k+1)th frame begins.

[0063] A configuration has been considered in which one frame is divided into multiple subframes and exposure is stopped depending on the count value for each subframe compared with a threshold value. However, this configuration has the potential for LED flicker when performing short exposures, and for output to become an abnormal value or signal information to be lost when the amount of light changes within a frame.

[0064] According to the present invention, a frame is divided into multiple subframes, and each subframe is further divided unevenly, and photons are counted with different weights depending on the period in which they are detected. This configuration eliminates the need for periodic exposure stop periods, reducing the effects of LED flicker and enabling appropriate photon counting to continue even if the amount of incident light changes within a frame.

[0065] Note that application of the present invention is not limited to cases where weighting coefficients are set uniformly for all pixels arranged in the pixel region 12 (pixel array). For example, when a first pixel group and a second pixel group are set in the pixel array, the lengths of the first and second periods or the combinations of weighting coefficients for each period may be different between the first pixel group and the second pixel group. The first pixel group may be located in the periphery of the pixel region, and the second pixel group may be located in the center of the pixel region.

[0066] 10 shows an example of the circuit configuration of a photoelectric conversion device according to a second embodiment. The circuit configuration is the same as that of the photoelectric conversion device according to the first embodiment, but the photoelectric conversion device according to this embodiment is characterized in that a clock signal VR is input to the quench element 302, thereby performing a so-called clock recharge operation. This type of driving has the effect of suppressing current consumption in high-light environments.

[0067] FIG. 11 shows a timing chart of the operation per frame of the photoelectric conversion device according to this embodiment.

[0068] At time t0, the enable signal EN rises, and exposure of the kth frame begins.

[0069] At time t0, the clock signal VR is set to L for a short period of time, connecting the cathode terminal of the APD 301 to the power supply voltage and putting the APD 301 into a standby state. In this embodiment, during the time from when the clock signal VR goes to L until it goes to L again, a maximum of one photon is detected regardless of the number of photons incident on the APD 301.

[0070] From time t0 to time t1, the control signal CTRL is controlled to H, and during this period the weighting coefficient of the count value is 4. From time t1 to time t3, the control signal CTRL is controlled to L, and during this period the weighting coefficient of the count value is 1. After time t3, the driving of the control signal CTRL from time t0 to time t3 is repeated a predetermined number of times.

[0071] When a photon is incident on the APD 301 at time t2, the output signal Vc drops. The output signal Vc is shaped by the shaping circuit 303 and input to the control circuit 311, causing IN1 to go high. IN1 is input to the T-flip-flop 312, which is the least significant bit of the counter. That is, the count value is incremented by +1.

[0072] At time t3, the clock signal VR goes low, and the cathode terminal of the APD 301 is connected to the power supply voltage and recharged. At the same time, the control circuit 311 is initialized, and IN1 and IN2 go low. When the signal VR returns to high, the APD 301 enters a standby state. The control signal CTRL is high from time t3 to time T5.

[0073] When a photon is incident on the APD 301 at time t4, the output signal Vc drops and a pulse signal is input to the shaping circuit 303. At this time, IN2 goes high and is input to the XOR circuit 321. The signal output from the T-flip-flop 313 is input to the other input terminal of the XOR circuit 321. That is, IN2 is counted by interrupting the middle bit of the counter, and the counter value is incremented by 4 for each IN2 signal.

[0074] Photons are also incident on the APD 301 at times t6 and t7, but Vc has dropped due to the avalanche multiplication caused by the photon incident at time t4, and no avalanche multiplication occurs due to the newly incident photon, so the counter value is not incremented.

[0075] At time t10, the enable signal EN falls, and the exposure of the kth frame ends.

[0076] From time t11 to time t12, the selection signal SEL goes to H, and the signal held in the counter circuit 305 is read out of the pixel.

[0077] At time t13, a photon is incident on the APD 301, but since the enable signal EN is at L, the photon is not counted.

[0078] From time t14 to time t15, the reset signal RES goes high, and the count value held by the counter circuit 305 is reset.

[0079] At time t16, the enable signal EN rises, and exposure of the (k+1)th frame begins.

[0080] 12 shows a timing chart according to a modification of the second embodiment. In this embodiment, the period from time t0 to time t1 and the period from time t1 to time t3 have the same length. That is, the period with a weighting factor of 4 and the period with a weighting factor of 1 are set to have the same length.

[0081] At time t0, the enable signal EN rises, starting exposure of the kth frame. At the same time, the signal VR is set to L to recharge the APD 301. When the signal VR goes to H, the APD 301 enters a standby state, and can detect up to one photon incident on the APD 301 until the signal VR is set to L again.

[0082] The control signal CTRL is controlled to H from time t0 to time t1, and the weighting coefficient of the count value during this period is 4. The control signal CTRL is controlled to L from time t1 to time t3, and the weighting coefficient of the count value during this period is 1. After time t3, the VR and control signal CTRL are driven from time t0 to time t3 a predetermined number of times.

[0083] When a photon is incident on the APD 301 at time t2, the output signal Vc drops. The output signal Vc is shaped by the shaping circuit 303 and input to the control circuit 311, causing IN1 to go high. IN1 is input to the T-flip-flop 312, which is the least significant bit of the counter. That is, the count value is incremented by +1.

[0084] At time t3, the signal VR goes low. The cathode terminal of the APD 301 is connected to the power supply voltage and recharged. When the signal VR returns to high, the APD 301 enters a standby state. The control signal CTRL is high from time t3 to time T5.

[0085] When a photon is incident on the APD 301 at time t4, the output signal Vc drops and a pulse signal is input to the shaping circuit 303. At this time, IN2 goes high and is input to the XOR circuit 321. The signal output from the T-flip-flop 313 is input to the other input terminal of the XOR circuit 321. That is, IN2 is counted by interrupting the middle bit of the counter, and the counter value is incremented by 4 for each IN2 signal.

[0086] Photons are also incident on the APD 301 at times t6 and t7. However, the output signal Vc drops due to avalanche multiplication caused by the incident photon at time t4 within the same recharge cycle, and a voltage not higher than the breakdown voltage is applied to the APD 301. Therefore, avalanche multiplication due to the incident photons at times t6 and t7 does not occur. The counter value does not increase due to photon detection.

[0087] At time t10, the enable signal EN falls, and the exposure of the kth frame ends.

[0088] From time t11 to time t12, the selection signal SEL goes to H, and the signal held in the counter circuit 305 is read out of the pixel.

[0089] At time t13, a photon is incident on the APD 301, but since the enable signal EN is at L, the photon is not counted.

[0090] From time t14 to t15, the reset signal RES goes high, and the count value held by the counter circuit 305 is reset.

[0091] At time t16, the enable signal EN rises again, and exposure of the (k+1)th frame begins.

[0092] In the case of clock recharge driving, photons are counted during a period when the weighting factor is 1 when "no photons are detected from time t0 to time t1, and photons are detected from time t1 to time t3." Therefore, even if the periods when the weighting factor is 4 and the periods when the weighting factor is 1 are set to be of equal length, the probability that photons are counted during a period when the weighting factor is 4 is higher than the probability that photons are counted during a period when the weighting factor is 1.

[0093] The length of the period for which each weighting factor is set and the weighting factors to be set are merely examples and are not limiting. The length of the period for which each weighting factor is set can be set appropriately depending on the weighting factor to be set.

[0094] In the third embodiment described below, weighted counting is performed by inputting different numbers of pulses to a counter circuit depending on the timing of photon detection, rather than by inserting a pulse into the middle bit of a counter as in the first embodiment. With this configuration, it is possible to reduce the circuit scale of the pixel circuit compared to the first embodiment.

[0095] In the first and second embodiments, the frame period is divided into two periods, and the count values ​​are weighted using two weighting coefficients corresponding to the two periods. In this embodiment, the frame period can be divided into, for example, three periods, and three weighting coefficients can also be used.

[0096] By appropriately setting the number of divisions of the frame period and the weighting coefficients, it is possible to improve the SNR and expand the dynamic range at the same time.

[0097] FIG. 13 shows an example of the circuit configuration of a photoelectric conversion device according to the third embodiment.

[0098] Similar to the photoelectric conversion devices according to the first and second embodiments, this device has a quench element 302 between a power supply and the cathode terminal of the APD 301, and has a waveform shaping circuit 303, an AND circuit 325, a counter circuit 305, and a readout circuit 306 at the downstream of the cathode terminal. An output signal from the waveform shaping circuit 303 is input to one terminal of the AND circuit 325, and a clock signal TCLK is input to the other terminal.

[0099] FIG. 14 shows a timing chart according to this embodiment.

[0100] At time t0, the enable signal EN rises, and exposure of the kth frame begins.

[0101] In this embodiment, one recharge period extends from time t0 to time t5 in Fig. 14. Each recharge period is further divided into four periods, and a weighting coefficient is assigned to each period. Here, the weighting coefficient represents "the total count number added to the counter within the relevant recharge period as a result of photons being received by the APD 301 at the timing of interest."

[0102] At time t1, the clock signal TCLK goes high. The clock signal TCLK defines the timing for switching the weighting coefficients, and if a photon has already been detected within the relevant recharge period and the output signal Vc has gone low, the count value of the counter circuit 305 is incremented by 1. Since no photon has been detected at time t1, no increment of the counter value occurs.

[0103] At time t2, a photon is incident on the APD 301, and the output signal Vc becomes L.

[0104] At time t3, the clock signal TCLK goes high, and the count value of the counter circuit 305 is incremented by one.

[0105] At time t4, the clock signal TCLK goes high again, and the count value of the counter circuit 305 is incremented by 1. That is, the count value is incremented by 2 in total from t1 to t4.

[0106] From time t4 to time t5, the enable signal EN is set to L, disabling the counter circuit 305. During this period, no photons incident on the APD 301 are detected.

[0107] After time t5, the operation from time t0 to time t5 is repeated.

[0108] In the recharge cycle starting at time t5, at time t6, a photon is incident on the APD 301. The clock signal TCLK goes high at times t7, t9, and t11, and the counter circuit 305 is incremented by 1 in synchronization with each rising edge of the clock signal TCLK. That is, the count value is incremented by 3 in total between t5 and t11.

[0109] Between time t16 and time t17, the shaping signal SEL goes high, and the signal held in the counter circuit 305 is read out of the pixel.

[0110] At time t18, a photon is incident on the APD 301, but since the enable signal EN is at L, the signal based on this photon is not counted.

[0111] From time t19 to time t20, the reset signal RES goes high, resetting the value held in the counter circuit 305.

[0112] At time t21, the enable signal EN rises, starting exposure of the (k+1)th frame.

[0113] When a photon detection signal is output during a first period, a natural number N is added to the counter circuit 305, and when a photon detection signal is output during a second period, a natural number M is added to the counter circuit 305. Furthermore, when a photon detection signal is output during a third period, a natural number L is added to the counter circuit. In this case, the natural number N is larger than the natural number M, and the natural number L is smaller than the natural number M. In this embodiment, a case where three types of weighting coefficients are set has been described, but the number of settings is not limited to three.

[0114] FIG. 15 shows a modified example of the circuit configuration of the photoelectric conversion device according to the third embodiment.

[0115] As a difference from the circuit shown in FIG. 13 , this configuration example includes a control circuit 322, a D flip-flop circuit 323, and a multiplexer 324. The D flip-flop circuit 323 maintains a history of whether or not a photon was detected in the immediately preceding recharge cycle. If no photons were detected in the immediately preceding recharge cycle, the count value for the current recharge cycle is not weighted and one count is added. This configuration reduces the possibility of counting noise as a photon detection result. This suppresses statistical variation in the count value, and is expected to improve the SNR, especially in low-light environments.

[0116] Here, clock signal BCLK is a clock signal for generating pulses of clock signal VR and clock signal TCLK. Furthermore, trigger signal TRIG is a pulse for selectively generating clock signal VR and clock signal TCLK from clock signal BCLK. When trigger signal TRIG is H and a pulse of clock signal BCLK is input, an inverted logic pulse is generated in clock signal VR. When trigger signal TRIG is L and a pulse of clock signal BCLK is input, a positive logic pulse is generated in clock signal TCLK.

[0117] Compared to transmitting the clock signal VR and the clock signal TCLK over separate signal lines, generating them using a common clock signal BCLK is less susceptible to the effects of skew, etc., and image quality is less likely to deteriorate even when there are a large number of pixels.

[0118] The control circuit 322 is configured, for example, by a combination of a NAND circuit and an AND circuit. The clock signal BCLK and the trigger signal TRIG are input to the input terminals of the NAND circuit. The clock signal BCLK and the inverted signal of the trigger signal TRIG are also input to the input terminals of the AND circuit.

[0119] FIG. 16 shows a timing chart of the configuration example shown in FIG.

[0120] The D flip-flop circuit 323 sets the signal STAT to H if a photon was detected in the immediately preceding recharge cycle, and to L if a photon was detected. When the signal STAT is L, if a photon is detected within the cycle of interest, the count value is incremented by 1 regardless of the timing. When the signal STAT is H, if a photon is detected within the cycle of interest, the counter is incremented by 3, 2, or 1 depending on the timing.

[0121] At time t0, the enable signal EN rises, starting exposure of the kth frame.

[0122] No photons are detected from time t0 to time t4. The signal STAT goes low during the recharge period from time t4.

[0123] At time t5, a photon is detected. Since the signal STAT is at L, the counter is incremented by 1 upon the occurrence of a photon detection event.

[0124] From time t9 to time t14, the signal STAT goes high because photons were detected in the immediately preceding recharge period (time t4 to time t9).

[0125] At time t11, a photon is detected. Since the signal STAT is H, the count value is incremented by one at times t12 and t13 when the clock signal TCLK rises, for a total of two.

[0126] From time t14 to time t18, a photon was detected in the immediately preceding recharge cycle (time t9 to time t14), so the signal STAT remains H. Thereafter, if a photon is detected in the cycle immediately preceding each recharge cycle, the signal STAT goes H, and if no photon is detected, the signal STAT goes L.

[0127] At time t15, the enable signal EN falls, and incident light thereafter is not counted.

[0128] At time t23, the enable signal EN rises, starting exposure of the (k+1)th frame. The signal STAT may be forced to L in accordance with the change of frames, or the level of the signal STAT for the first recharge period of the (k+1)th frame may be determined based on the photon detection result for the last recharge period of the kth frame.

[0129] In this embodiment, the method of weighting the counts was changed based on whether or not photons were detected in the immediately preceding recharge cycle, but the weighting method may also be changed based on whether or not photons were detected in a recharge cycle two or more cycles prior, or based on whether or not photons were detected in multiple past recharge cycles.

[0130] Fourth Embodiment FIG. 17 shows an example of the circuit configuration of a photoelectric conversion device according to a fourth embodiment.

[0131] A feature of the photoelectric conversion device according to this embodiment is that it controls whether or not to count up the pixel of interest depending on whether or not photons are detected in the surrounding pixels arranged around the pixel of interest. A predetermined value is added to the counter when photons are detected in both pixel of interest A and surrounding pixel B within the same recharge cycle. The surrounding pixels referenced to determine signal addition for the pixel of interest may be one or more neighboring pixels. Pixels arranged in an array may form a one- or two-dimensional network, and neighboring pixels may mutually reference the photon detection states. In the following description, pixels adjacent to the pixel of interest are referred to as neighboring pixels, but the relationship between the pixel of interest and the surrounding pixels is not limited to this.

[0132] The upper row in Figure 17 shows pixel of interest A, and the lower row shows surrounding pixel B. Pixel of interest A and surrounding pixel B are configured in the same way. That is, a quench element 302 is provided between the power supply and the cathode terminal of APD 301, and a waveform shaping circuit 303 is connected to the cathode terminal. Connected to the latter stage are a control circuit 311, T flip-flop circuits 312 to 315 and XOR circuit 321 that constitute a counter circuit 305, and a readout circuit 306. Corresponding elements of pixel of interest A and surrounding pixel B are assigned the same reference numeral, and the suffix "A" or "B" at the end of the reference numeral distinguishes which pixel the element belongs to.

[0133] The photoelectric conversion device according to this embodiment differs from the photoelectric conversion devices according to the first to third embodiments in that wiring is provided to interconnect the control circuit 311A ​​and the control circuit 311B. The control circuit 311A ​​controls the outputs of IN1A and IN2A based on the output signal VA of the pixel of interest A and the output signal VB of the peripheral pixel B. As an example, either IN1A or IN1B is latched to H depending on the timing when both VA and VB become H.

[0134] FIG. 18 shows a timing chart for pixel A and pixel B when the configuration of FIG. 17 is used.

[0135] At time t0, the enable signal EN rises, starting exposure of the kth frame. The period from time t0 to time t4 is one recharge period.

[0136] At time t2, a photon is detected by the APD 301 A. At this time, the pixel B surrounding the pixel A of interest does not detect a photon, and therefore does not add a signal to the counter circuit 305 .

[0137] At time t3, the APD 301B of peripheral pixel B also detects a photon. In response to the rising edge of the output VB of the waveform shaping circuit 303B, IN1A becomes H and a signal is added to the counter circuit 305. At time t3, the control signal CTRL is L, so the count value added is 1.

[0138] At time t5, APD 301B of peripheral pixel B detects a photon, and at time t6, APD 301A of target pixel A detects a photon. At time t6, the control signal CTRL is H, so IN2A goes H in response to the rising edge of output VB of 303B, and a signal is added to XOR circuit 321A. That is, the count value is incremented by 4.

[0139] By incorporating the presence or absence of photon detection from surrounding pixels into the count value of the pixel of interest, the effective sensitivity can be appropriately adjusted, ensuring gradation at higher light levels and expanding the dynamic range.

[0140] 19 shows an example of the circuit configuration of a photoelectric conversion device according to a fifth embodiment. In the photoelectric conversion device shown in this embodiment, when the count value of the counter circuit 305 exceeds a predetermined threshold, the weighting coefficient of the count value is changed.

[0141] 19, the photoelectric conversion device has a quench element 302 between a power supply and the cathode terminal of an APD 301, and a waveform shaping circuit 303 is connected to the cathode terminal. A count control circuit 304, a counter circuit 305, and a readout circuit 306 are arranged downstream of the waveform shaping circuit 303. The output of the counter circuit 305 is also input to a latch circuit 307, and the output signal STAT of the latch circuit 307 is input to the count control circuit 304.

[0142] FIG. 20 shows a timing chart when the photoelectric conversion device having the configuration shown in FIG. 19 is used.

[0143] At time t0, the enable signal EN rises, starting exposure of the kth frame. The period from time t0 to time t3 is one recharge period. When the control signal CTRL is H, 4 is added to the count value, and when the control signal CTRL is L, 1 is added to the count value. In this embodiment, the threshold value of the count value is set to m.

[0144] At time t1, the control signal CTRL falls from H to L. At this time, the output signal STAT is at L.

[0145] At time t2, a photon is incident on the APD 301, causing the output signal Vc to fall. Since the control signal CTRL is at L, the count value is incremented by one.

[0146] At time t4, a photon is detected by the APD 301, and the count value becomes m + 1. Since the count value exceeds the threshold value m, the output signal STAT switches from L to H.

[0147] In the recharge period after time t7, when the control signal CTRL is H, 2 is added to the count value, and when the control signal CTRL is L, 1 is added to the count value.

[0148] With this configuration, it is possible to expand the dynamic range while reducing the circuit scale of the photoelectric conversion device.

[0149] Sixth Embodiment A photoelectric conversion system using the photoelectric conversion device according to each of the above-described embodiments will be described with reference to Fig. 21. Fig. 21 is a block diagram showing a schematic configuration of the photoelectric conversion system according to this embodiment.

[0150] The processing device according to the sixth embodiment includes a control unit 401 , a timing adjustment unit 402 , an image acquisition unit 403 , a readout unit 404 , a gain adjustment unit 405 , a nonlinear correction unit 406 , a defect correction unit 407 , a data compression unit 408 , and a memory unit 409 .

[0151] The image acquisition unit 403 is, for example, the pixel 101 in the pixel region 12 shown in Fig. 2, and the readout unit 404 is, for example, the signal processing unit 103 in Fig. 3. The timing adjustment unit 402 is, for example, the control pulse generation unit 115 in Fig. 3. The control unit 401 may be a control unit inside the photoelectric conversion device, or may be located outside the photoelectric conversion device. The image acquisition unit 403 is controlled by the timing adjustment unit 402, which is controlled by the control unit 401.

[0152] The image data generated by the image acquisition unit is subjected to correction processing and then input to the storage unit 409. Note that the order of the correction processing is not limited to the order shown in FIG.

[0153] The gain adjustment unit 405 is provided between the readout unit 404 and the nonlinear correction unit 406 and applies a digital gain to the image data generated by the image acquisition unit 403. Data for image correction often contains decimal values, but if the image output is an integer, quantization error can reduce the correction accuracy. By applying a gain to the image data in advance, the effects of quantization error can be suppressed and the correction accuracy can be improved. If the quantization error can be suppressed to ¼ of the single-photon signal level or less, the corrected image will appear visually natural. Therefore, it is desirable that the digital gain applied to the image data be, for example, 4 times or more.

[0154] The nonlinear correction unit 406 is disposed between the gain adjustment unit 405 and the defect correction unit 407, and corrects the image data under the control of the control unit 401. As described in the above embodiments, when the image acquisition unit 403 is a photon-counting detector, the optical response often becomes nonlinear due to the influence of dead time. When affected by a nonlinear optical response, correction based on a linear response may result in overcorrection. Therefore, by performing nonlinear correction on the image data prior to the calculation processing in the defect correction unit 407, overcorrection can be prevented, and appropriate nonlinear correction can be performed according to the drive timing. This nonlinear correction is performed using, for example, a LookUp Table.

[0155] The defect correction unit 407 corrects the data of defective pixels contained in the image data. As a specific example, the output value of the defective pixel is extracted and the position information and output value of the defective pixel are identified. There are methods for replacing the output value with the average value or median value of the outputs of the pixels surrounding the identified defective pixel, and methods for dividing the estimated defective image data.

[0156] The data compression unit 408 compresses the corrected image data. The photoelectric conversion device according to the present invention generates a huge amount of image data corresponding to a high dynamic range. By providing the data compression unit 408, the data can be compressed before being stored in the subsequent storage unit 409.

[0157] The storage unit 409 is a storage unit that holds at least a part of the image data generated in the previous stage. Specifically, the image data is stored in a memory such as an SRAM, a DRAM, or a nonvolatile memory.

[0158] As described above, according to this embodiment, a photoelectric conversion system can be realized to which the photoelectric conversion device shown in any of the above-described embodiments is applied.

[0159] Seventh Embodiment The effects of the present invention will be further described with reference to Figures 22A and 22B. In the following description, the case where addition is performed according to each of the above-described embodiments will be referred to as a weighted counting method.

[0160] In the conventional clock recharge method shown in Figure 22A, the maximum estimated number of incident photons is determined by the number of Recharge CLKs (clock signals TCLK). That is, the greater the number of Recharge CLKs, the wider the dynamic range. On the other hand, power consumption increases in proportion to the number of Recharge CLKs, so there is a trade-off between dynamic range and power consumption.

[0161] When the exposure period is T and the pulse interval of Recharge CLK is Δtr, the number of Recharge CLKs is expressed as T / Δtr, and therefore it can be considered that the dynamic range is determined by T / Δtr.

[0162] In contrast, as shown in FIG. 22B, in the weighted counting method, the number of incident photons is estimated from the timing at which the photons are incident. Therefore, the maximum estimated number of incident photons is determined by T / Δtw. Here, Δtw corresponds to the first period in the first embodiment and is the period during which an integer value is added to the counter circuit. In other words, the maximum estimated number of incident photons is no longer dependent on the recharge CLK interval Δtr, and there is no longer a trade-off between dynamic range and power consumption. In this case, the weighting coefficient is preferably Δtr / Δtw.

[0163] Specific numerical examples are shown below. In the conventional method, when the exposure period T is 1024 and Δtr is 1, the maximum number of avalanches is 1024, and the maximum number of detected incident photons at this time is 1024. On the other hand, in the weighted count method, when the exposure period T is 1024, Δtr is 4, and Δtw is 1, the maximum number of avalanches is 256. If addition is made to the counter circuit so that the number of counts corresponding to photons incident during Δtw becomes Δtr / Δtw, or 4, the maximum estimated number of incident photons at this time is 1024. In other words, it is possible to achieve an equivalent dynamic range while reducing the power consumption associated with recharging to one-fourth of that of the conventional method.

[0164] Here, the weighting coefficient Δtr / Δtw is preferably set because it allows the saturation counts to be equal under driving conditions where both methods have equivalent dynamic ranges. Setting these conditions makes it possible to reduce spurious signals during nonlinear correction. However, because power consumption depends only on Δtr and not on the weighting coefficient, it is not necessary to satisfy the above relationship from the perspective of reducing power consumption.

[0165] Eighth Embodiment A photoelectric conversion system according to this embodiment will be described with reference to Fig. 23. Fig. 23 is a block diagram showing a schematic configuration of the photoelectric conversion system according to this embodiment.

[0166] The photoelectric conversion device described in the above embodiment can be applied to various photoelectric conversion systems, such as digital still cameras, digital camcorders, surveillance cameras, copiers, fax machines, mobile phones, vehicle-mounted cameras, and observation satellites.

[0167] Also included in the photoelectric conversion system is a camera module equipped with an optical system such as a lens and an imaging device. Figure 23 shows a block diagram of a digital still camera as an example of such a system.

[0168] 23 includes an image pickup device 1004, which is an example of a photoelectric conversion device, and a lens 1002 that forms an optical image of a subject on the image pickup device 1004. The photoelectric conversion system further includes an aperture 1003 that adjusts the amount of light passing through the lens 1002, and a barrier 1001 that protects the lens 1002. The lens 1002 and the aperture 1003 form an optical system that focuses light on the image pickup device 1004. The image pickup device 1004 is a photoelectric conversion device according to any of the above embodiments, and converts the optical image formed by the lens 1002 into an electrical signal.

[0169] The photoelectric conversion system includes a signal processing unit 1007, which is an image generation unit that generates an image by processing an output signal output from an imaging device 1004. The signal processing unit 1007 performs various corrections and compressions as necessary to output image data. The signal processing unit 1007 may be formed in the same semiconductor layer as the imaging device 1004, or may be formed in a different semiconductor layer from the imaging device 1004. Furthermore, the imaging device 1004 and the signal processing unit 1007 may be formed in the same semiconductor layer.

[0170] The photoelectric conversion system further includes a memory unit 1010 for temporarily storing image data, and an external interface unit (external I / F unit) 1013 for communicating with an external computer or the like. The photoelectric conversion system further includes a recording medium 1012 such as a semiconductor memory for recording or reading out image data, and a recording medium control interface unit (recording medium control I / F unit) 1011 for recording or reading out data from the recording medium 1012. The recording medium 1012 may be built into the photoelectric conversion system or may be detachable.

[0171] The photoelectric conversion system further includes an overall control / calculation unit 1009 that performs various calculations and controls the entire digital still camera, and a timing generation unit 1008 that outputs various timing signals to the image capture device 1004 and the signal processing unit 1007. Here, timing signals and the like may be input from outside, and the photoelectric conversion system only needs to include at least the image capture device 1004 and the signal processing unit 1007 that processes the output signal output from the image capture device 1004.

[0172] The imaging device 1004 outputs an imaging signal to the signal processing unit 1007. The signal processing unit 1007 performs predetermined signal processing on the imaging signal output from the imaging device 1004 and outputs image data. The signal processing unit 1007 generates an image using the imaging signal.

[0173] As described above, according to this embodiment, it is possible to realize a photoelectric conversion system to which the photoelectric conversion device (image pickup device) according to any one of the above embodiments is applied.

[0174] Ninth Embodiment A photoelectric conversion system and a moving object according to this embodiment will be described with reference to Fig. 24A and Fig. 24B, which are diagrams showing the configurations of the photoelectric conversion system and the moving object according to this embodiment.

[0175] FIG. 24A illustrates an example of a photoelectric conversion system for an in-vehicle camera. The photoelectric conversion system 2300 includes an image capture device 2310. The image capture device 2310 is the photoelectric conversion device described in any of the above embodiments. The photoelectric conversion system 2300 also includes an image processing unit 2312 that performs image processing on multiple pieces of image data acquired by the image capture device 2310. The photoelectric conversion system 2300 also includes a parallax acquisition unit 2314 that calculates parallax (phase difference between parallax images) from the multiple pieces of image data acquired by the photoelectric conversion system 2300. The photoelectric conversion system 2300 also includes a distance acquisition unit 2316 that calculates the distance to an object based on the calculated parallax, and a collision determination unit 2318 that determines whether or not there is a possibility of a collision based on the calculated distance. Here, the parallax acquisition unit 2314 and the distance acquisition unit 2316 are examples of distance information acquisition means that acquire distance information to an object. That is, distance information may be acquired using not only the phase difference but also ToF (Time Of Flight) technology. The collision determination unit 2318 may determine the possibility of a collision using any of these distance information. The distance information acquisition means may be realized by dedicated hardware or a software module. Furthermore, it may be realized by an FPGA (Field Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), or the like, or a combination thereof.

[0176] The photoelectric conversion system 2300 is connected to a vehicle information acquisition device 2320 and can acquire vehicle information such as vehicle speed, yaw rate, and steering angle. The photoelectric conversion system 2300 is also connected to a control ECU 2330, which is a control device (control unit) that outputs a control signal to generate a braking force to the vehicle based on the determination result of the collision determination unit 2318. The photoelectric conversion system 2300 is also connected to an alarm device 2340 that issues an alarm to the driver based on the determination result of the collision determination unit 2318. For example, if the determination result of the collision determination unit 2318 indicates a high possibility of collision, the control ECU 2330 applies the brakes, releases the accelerator, suppresses engine output, or performs other vehicle control to avoid a collision and mitigate damage.

[0177] The alarm device 2340 warns the user by sounding an alarm, displaying alarm information on the screen of a car navigation system, or vibrating the seat belt or steering wheel.

[0178] In this embodiment, the surroundings of the vehicle, for example, the front or rear, are imaged by the photoelectric conversion system 2300. Fig. 24B shows the photoelectric conversion system when imaging the area in front of the vehicle (imaging range 2350). A vehicle information acquisition device 2320 sends instructions to the photoelectric conversion system 2300 or the imaging device 2310. This configuration can further improve the accuracy of distance measurement.

[0179] Although the above describes an example of control to prevent collision with other vehicles, the present invention can also be applied to control of automatic driving by following other vehicles, control of automatic driving to prevent deviation from a lane, etc. Furthermore, the photoelectric conversion system is not limited to vehicles such as the subject vehicle, but can be applied to moving bodies (moving devices) such as ships, aircraft, and industrial robots. In addition, the present invention can be applied not only to moving bodies but also to a wide range of devices that use object recognition, such as intelligent transport systems (ITS).

[0180] Tenth Embodiment A photoelectric conversion system according to this embodiment will be described with reference to Fig. 25. Fig. 25 is a block diagram showing an example of the configuration of a range image sensor that is a photoelectric conversion system.

[0181] 25, the range image sensor 1401 is configured to include an optical system 1402, a photoelectric conversion device 1403, an image processing circuit 1404, a monitor 1405, and a memory 1406. The range image sensor 1401 can obtain a range image according to the distance to the subject by receiving light (modulated light or pulsed light) that is projected toward the subject from a light source device 1411 and reflected from the surface of the subject.

[0182] The optical system 1402 is configured to have one or more lenses, and guides image light (incident light) from an object to the photoelectric conversion device 1403 , forming an image on the light receiving surface (sensor portion) of the photoelectric conversion device 1403 .

[0183] The photoelectric conversion device 1403 is the photoelectric conversion device described in the above-mentioned embodiment, and a distance signal indicating the distance determined from the light receiving signal output from the photoelectric conversion device 1403 is supplied to the image processing circuit 1404.

[0184] The image processing circuit 1404 performs image processing to construct a distance image based on the distance signal supplied from the photoelectric conversion device 1403. The distance image (image data) obtained by this image processing is then supplied to a monitor 1405 for display, or supplied to a memory 1406 for storage (recording).

[0185] In the range image sensor 1401 configured in this way, by applying the above-described photoelectric conversion device, it is possible to obtain, for example, a more accurate range image as the pixel characteristics improve.

[0186] Eleventh Embodiment A photoelectric conversion system of this embodiment will be described with reference to Fig. 26. Fig. 26 is a diagram showing an example of the schematic configuration of an endoscopic surgery system, which is the photoelectric conversion system of this embodiment.

[0187] 26 shows an operator (doctor) 1131 performing surgery on a patient 1132 on a patient bed 1133 using an endoscopic surgery system 1103. As shown in the figure, the endoscopic surgery system 1103 is composed of an endoscope 1100, a surgical tool 1110, and a cart 1134 on which various devices for endoscopic surgery are mounted.

[0188] The endoscope 1100 is composed of a lens barrel 1101, a region of a predetermined length from the tip of which is inserted into a body cavity of a patient 1132, and a camera head 1102 connected to the base end of the lens barrel 1101. In the illustrated example, the endoscope 1100 is configured as a so-called rigid lens barrel having a rigid lens barrel 1101, but the endoscope 1100 may also be configured as a so-called flexible lens barrel having a flexible lens barrel.

[0189] An opening into which an objective lens is fitted is provided at the tip of the lens barrel 1101. A light source device 1203 is connected to the endoscope 1100, and light generated by the light source device 1203 is guided to the tip of the lens barrel 1101 by a light guide extending inside the lens barrel 1101, and is irradiated via the objective lens toward an observation target inside a body cavity of a patient 1132. The endoscope 1100 may be a direct-viewing endoscope, an oblique-viewing endoscope, or a side-viewing endoscope.

[0190] An optical system and a photoelectric conversion device are provided inside the camera head 1102, and light reflected from an observation object (observation light) is focused onto the photoelectric conversion device by the optical system. The observation light is photoelectrically converted by the photoelectric conversion device to generate an electrical signal corresponding to the observation light, i.e., an image signal corresponding to the observation image. The photoelectric conversion device may be the photoelectric conversion device described in the above-described embodiment. The image signal is transmitted to a camera control unit (CCU) 1135 as RAW data.

[0191] The CCU 1135 is configured with a CPU (CentraL Processing Unit), a GPU (Graphics Processing Unit), etc., and comprehensively controls the operations of the endoscope 1100 and the display device 1136. Furthermore, the CCU 1135 receives an image signal from the camera head 1102 and performs various types of image processing on the image signal, such as development processing (demosaic processing), to display an image based on the image signal.

[0192] Under the control of the CCU 1135 , the display device 1136 displays an image based on the image signal that has been subjected to image processing by the CCU 1135 .

[0193] The light source device 1203 is configured from a light source such as an LED (Light Emitting Diode), and supplies the endoscope 1100 with irradiation light when photographing an operation site or the like.

[0194] The input device 1137 is an input interface for the endoscopic surgery system 1103. A user can input various information and instructions to the endoscopic surgery system 1103 via the input device 1137.

[0195] The treatment tool control device 1138 controls the driving of the energy treatment tool 1112 for cauterizing tissue, incising, sealing blood vessels, or the like.

[0196] The light source device 1203, which supplies illumination light to the endoscope 1100 when photographing the surgical site, can be configured from a white light source formed, for example, by an LED, a laser light source, or a combination of these. When the white light source is configured from a combination of RGB laser light sources, the output intensity and output timing of each color (each wavelength) can be controlled with high precision, allowing the light source device 1203 to adjust the white balance of the captured image. In this case, it is also possible to time-share images corresponding to each RGB by irradiating the object of observation with laser light from each RGB laser light source in a time-division manner and controlling the drive of the image sensor of the camera head 1102 in synchronization with the irradiation timing. This method allows color images to be obtained without providing a color filter to the image sensor.

[0197] Furthermore, the light source device 1203 may be controlled to change the intensity of light it outputs at predetermined time intervals. By controlling the driving of the image sensor of the camera head 1102 in synchronization with the timing of the change in light intensity to acquire images in a time-division manner and combining the images, it is possible to generate an image with a high dynamic range that is free from so-called blocked-up shadows and blown-out highlights.

[0198] The light source device 1203 may also be configured to supply light in a predetermined wavelength band corresponding to special light observation. Special light observation utilizes, for example, the wavelength dependency of light absorption in body tissue. Specifically, by irradiating light with a narrower band than the light irradiated during normal observation (i.e., white light), a predetermined tissue, such as blood vessels on the surface of the mucous membrane, can be photographed with high contrast.

[0199] Alternatively, special light observation may involve fluorescence observation, in which an image is obtained from fluorescence generated by irradiating excitation light onto a body tissue. Fluorescence observation may involve irradiating excitation light onto a body tissue and observing the fluorescence from the body tissue, or may involve locally injecting a reagent such as indocyanine green (ICG) into the body tissue and irradiating the body tissue with excitation light corresponding to the fluorescent wavelength of the reagent to obtain a fluorescent image. The light source device 1203 may be configured to supply narrowband light and / or excitation light compatible with such special light observation.

[0200] Twelfth Embodiment A photoelectric conversion system according to this embodiment will be described with reference to Fig. 27A and Fig. 27B. Fig. 27A is a diagram showing an example of the configuration of glasses 1600 (smart glasses) which are the photoelectric conversion system.

[0201] The eyeglasses 1600 have a photoelectric conversion device 1602. The photoelectric conversion device 1602 is the photoelectric conversion device described in the above-mentioned 12th embodiment. A display device including a light-emitting device such as an OLED or LED may be provided on the back side of the lens 1601. The photoelectric conversion device 1602 may be one or more. A combination of multiple types of photoelectric conversion devices may also be used. The arrangement position of the photoelectric conversion device 1602 is not limited to that shown in FIG. 27A .

[0202] The glasses 1600 further include a control device 1603. The control device 1603 functions as a power source that supplies power to the photoelectric conversion device 1602 and the display device. The control device 1603 also controls the operations of the photoelectric conversion device 1602 and the display device. The lens 1601 is formed with an optical system for focusing light onto the photoelectric conversion device 1602.

[0203] FIG. 27B illustrates glasses 1610 (smart glasses) according to one application example. The glasses 1610 include a control device 1612, which is equipped with a photoelectric conversion device corresponding to the photoelectric conversion device 1602 and a display device. A lens 1611 includes an optical system for projecting light emitted from the photoelectric conversion device and the display device, and an image is projected onto the lens 1611. The control device 1612 functions as a power source for supplying power to the photoelectric conversion device and the display device, and controls the operation of the photoelectric conversion device and the display device. The control device may include a gaze detection unit for detecting the gaze of the wearer. Infrared light may be used for gaze detection. The infrared light emitter emits infrared light toward the eyeball of a user gazing at a displayed image. An imaging unit with a light-receiving element detects the reflected light of the emitted infrared light from the eyeball, thereby obtaining an image of the eyeball. By including a reduction unit for reducing light from the infrared light emitter to the display unit in a planar view, degradation of image quality is reduced.

[0204] The gaze of the user relative to the displayed image is detected from an image of the eyeball obtained by capturing infrared light. Any known method can be used for gaze detection using the image of the eyeball. As an example, a gaze detection method based on the Purkinje image formed by reflection of irradiated light on the cornea can be used.

[0205] More specifically, gaze detection processing is performed based on the pupil-corneal reflex method, which calculates a gaze vector representing the direction (rotation angle) of the eyeball based on the pupil image and Purkinje image included in the captured image of the eyeball, thereby detecting the user's gaze.

[0206] The display device of this embodiment may have a photoelectric conversion device having a light receiving element, and may control the image displayed on the display device based on information on the user's line of sight from the photoelectric conversion device.

[0207] Specifically, the display device determines a first field of view area where the user gazes and a second field of view area other than the first field of view area based on the line-of-sight information. The first field of view area and the second field of view area may be determined by a control device of the display device, or may be determined by an external control device and received. In the display area of ​​the display device, the display resolution of the first field of view area may be controlled to be higher than the display resolution of the second field of view area. In other words, the resolution of the second field of view area may be lower than that of the first field of view area.

[0208] The display area may include a first display area and a second display area different from the first display area, and a high-priority area may be determined from the first display area and the second display area based on line-of-sight information. The first field of view area and the second field of view area may be determined by a control device of the display device, or may be determined by an external control device and received. The resolution of the high-priority area may be controlled to be higher than the resolution of areas other than the high-priority area. In other words, the resolution of an area with a relatively low priority may be lowered.

[0209] Note that AI may be used to determine the first field of view area or the area with high priority. The AI ​​may be a model configured to estimate the angle of gaze and the distance to an object in the line of sight from the image of the eyeball, using as training data an image of the eyeball and the direction in which the eyeball in the image was actually looking. The AI ​​program may be included in the display device, the photoelectric conversion device, or an external device. If included in the external device, it is transmitted to the display device via communication.

[0210] When display control is performed based on visual recognition detection, the present invention is preferably applied to smart glasses that further include a photoelectric conversion device for capturing images of the outside world. The smart glasses can display captured external information in real time.

[0211] The above-described embodiments can be modified as appropriate without departing from the scope of the technical concept. Furthermore, examples in which part of the configuration of one embodiment is added to another embodiment, or examples in which part of the configuration of another embodiment is replaced with another embodiment, are also included in the embodiments of the present invention.

[0212] The disclosure of the present embodiment also includes the following configurations and methods.

[0213] (Configuration 1) A photoelectric conversion device comprising: a photoelectric conversion unit that generates a photon detection signal by avalanche multiplication; and a counter circuit that counts the photon detection signals output from the photoelectric conversion unit. The photoelectric conversion device comprises a first period included in one frame; and a second period included in the one frame, the second period being after the first period and not overlapping with the first period. The photoelectric conversion device is characterized in that a value counted by the counter circuit based on one of the photon detection signals during the first period is greater than a value counted by the counter circuit based on one of the photon detection signals during the second period.

[0214] (Configuration 2) The photoelectric conversion device according to Configuration 1, wherein when the photoelectric conversion unit outputs a photon detection signal in the first period, a natural number N is added to the counter circuit, and when the photoelectric conversion unit outputs a photon detection signal in the second period, a natural number M is added to the counter circuit, and the natural number N is greater than the natural number M.

[0215] (Configuration 3) The photoelectric conversion device according to configuration 2, wherein the natural number M is 1.

[0216] (Configuration 4) The photoelectric conversion device according to any one of configurations 1 to 3, wherein the first period and the second period are adjacent in time.

[0217] (Configuration 5) The photoelectric conversion device according to any one of configurations 1 to 4, wherein a clock signal is input to recharge the photoelectric conversion unit and put it into a standby state.

[0218] (Configuration 6) The photoelectric conversion device according to configuration 5, wherein the first period and the second period occur after the input of the clock signal.

[0219] (Configuration 7) The photoelectric conversion device according to any one of configurations 1 to 6, wherein the length of the first period is shorter than the length of the second period.

[0220] (Configuration 8) The photoelectric conversion device according to Configuration 2, wherein when the photoelectric conversion unit outputs a photon detection signal during a third period, a natural number L is added to the counter circuit, and the natural number L is smaller than the natural number M.

[0221] (Configuration 9) The photoelectric conversion device according to configuration 2, wherein a value according to the photon detection result of the previous recharge period is added.

[0222] (Configuration 10) The photoelectric conversion device according to configuration 2, wherein a value according to the photon detection results of the surrounding pixels is added.

[0223] (Configuration 11) The photoelectric conversion device according to Configuration 2, wherein if a photon is detected during the first period, the natural number M is not added to the counter circuit even if a photon is detected during the second period.

[0224] (Configuration 12) The photoelectric conversion device according to any one of configurations 1 to 11, wherein the first period and the second period each detect up to one photon.

[0225] (Configuration 13) The photoelectric conversion device according to configuration 2, wherein the natural number N or the natural number M is set according to the count value of the counter circuit.

[0226] (Configuration 14) The photoelectric conversion device according to any one of configurations 1 to 13, wherein the lengths of the first period and the second period are set in accordance with the count value of the counter circuit.

[0227] (Configuration 15) The photoelectric conversion device according to Configuration 2, wherein the length of the first period and the length of the second period, or the combination of the natural number N and the natural number M, is made different between the kth frame and the k+1th frame.

[0228] (Configuration 16) The photoelectric conversion device according to Configuration 2, wherein the photoelectric conversion unit has a pixel array in which a plurality of pixels are arranged in a plurality of rows and a plurality of columns, the pixel array having a first pixel group and a second pixel group, the first pixel group and the second pixel group having different lengths of the first period and the second period, or different combinations of the natural number N and the natural number M.

[0229] (Configuration 17) A photoelectric conversion system comprising: the photoelectric conversion device according to any one of configurations 1 to 16; and a signal processing unit that generates an image using a signal output from the photoelectric conversion device.

[0230] (Configuration 18) The photoelectric conversion system according to configuration 17, wherein nonlinear correction is performed.

[0231] (Configuration 19) A moving body including the photoelectric conversion device according to any one of Configurations 1 to 16, characterized in that the moving body has a control unit that controls movement of the moving body using a signal output by the photoelectric conversion device.

[0232] The present invention is not limited to the above-described embodiments, and various modifications and variations can be made without departing from the spirit and scope of the present invention. Therefore, the following claims are appended to apprise the public of the scope of the present invention.

[0233] This application claims priority based on Japanese Patent Application No. 2023-108682 filed on June 30, 2023 and Japanese Patent Application No. 2024-102829 filed on June 26, 2024, the entire contents of which are incorporated herein by reference.

[0234] 301 Avalanche photodiode 302 Quench element 303 Waveform shaping circuit 305 Counter circuit 311 Control circuit

Claims

1. A photoelectric conversion device comprising: a photoelectric conversion unit which generates a photon detection signal by avalanche multiplication; a counter circuit which counts the photon detection signals output from the photoelectric conversion unit; a first period included in one frame; and a second period included in the one frame, which is after the first period and does not overlap with the first period, wherein the value counted by the counter circuit based on one of the photon detection signals during the first period is greater than the value counted by the counter circuit based on one of the photon detection signals during the second period.

2. The photoelectric conversion device according to claim 1, characterized in that when the photoelectric conversion unit outputs a photon detection signal during the first period, a natural number N is added to the counter circuit, and when the photoelectric conversion unit outputs a photon detection signal during the second period, a natural number M is added to the counter circuit, and the natural number N is greater than the natural number M.

3. The photoelectric conversion device according to claim 2, wherein the natural number M is 1.

4. The photoelectric conversion device according to claim 1, wherein the first period and the second period are adjacent in time.

5. The photoelectric conversion device according to claim 1, further comprising an input of a clock signal which recharges said photoelectric conversion section and places said photoelectric conversion section in a standby state.

6. The photoelectric conversion device according to claim 5, wherein the first period and the second period occur after the input of the clock signal.

7. The photoelectric conversion device according to claim 1, wherein the length of the first period is shorter than the length of the second period.

8. The photoelectric conversion device according to claim 2, characterized in that when the photoelectric conversion unit outputs a photon detection signal during a third period, a natural number L is added to the counter circuit, and the natural number L is smaller than the natural number M.

9. The photoelectric conversion device according to claim 2, further comprising an additional value according to the photon detection result in the previous recharge period.

10. The photoelectric conversion device according to claim 2, further comprising an additional value according to the photon detection results of surrounding pixels.

11. The photoelectric conversion device according to claim 2, characterized in that if a photon is detected during the first period, the natural number M is not added to the counter circuit even if a photon is detected during the second period.

12. The photoelectric conversion device according to claim 1, wherein said first period and said second period each detect up to one photon.

13. The photoelectric conversion device according to claim 2, wherein the natural number N or the natural number M is set in accordance with the count value of the counter circuit.

14. The photoelectric conversion device according to claim 1, wherein the lengths of the first period and the second period are set in accordance with the count value of the counter circuit.

15. The photoelectric conversion device according to claim 2, characterized in that the length of the first period and the length of the second period, or the combination of the natural number N and the natural number M, is different between the kth frame and the k+1th frame.

16. The photoelectric conversion device according to claim 2, characterized in that the photoelectric conversion unit has a pixel array in which a plurality of pixels are arranged in a plurality of rows and a plurality of columns, the pixel array has a first pixel group and a second pixel group, and the first pixel group and the second pixel group have mutually different lengths of the first period and the second period, or different combinations of the natural number N and the natural number M.

17. A photoelectric conversion system comprising: a photoelectric conversion device according to any one of claims 1 to 16; and a signal processing unit that generates an image using a signal output by the photoelectric conversion device.

18. The photoelectric conversion system according to claim 17, further comprising a nonlinear correction.

19. A moving body including a photoelectric conversion device according to any one of claims 1 to 16, characterized in that the moving body has a control unit that controls the movement of the moving body using a signal output by the photoelectric conversion device.

Images