Photoelectric converter and photoelectric conversion system

Abstract

To provide a photoelectric conversion device that has equalized power consumption in a frame.SOLUTION: A photoelectric conversion device has a plurality of pixels each including a photodiode performing the avalanche multiplication, and a signal processing unit that generates signals based on photons detected by the photodiodes. The signal processing unit has a control circuit, a counter, and an illuminance determination unit. The control circuit is connected with the photodiodes and a generation circuit generating pulse signals, and according to the pulse signals, controls the photodiode in a first state where it does not electrically connect one node of the photodiode to a predetermined electrical potential, and a second state where it electrically connects one node to the predetermined electrical potential. The counter counts the number of periods in which the avalanche multiplication occurs in a plurality of periods in the first state. The illuminance determination unit determines whether a count value of the counter reaches a threshold. The signal processing unit has means that sets the frequency of the pulse signals according to a result of the determination made by the illuminance determination unit.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] The present invention relates to structures of a photoelectric conversion apparatus and a photoelectric conversion system.

Background Art

[0002] Photoelectric conversion apparatuses having a pixel array in which pixels each including a plurality of avalanche photodiodes (APDs) are arranged in a plurality of rows and a plurality of columns are known.

[0003] Patent Document 1 discloses an APD, a quench circuit connected to the APD, a signal control circuit to which a signal output from the APD is input, and a pulse generation circuit connected to the quench circuit and the signal control circuit.

[0004] Non-Patent Document 1 discloses a technique of counting signals originating from single photons output from an APD, and recording a saturation time and stopping the operation of the APD when a counter is saturated.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Non-Patent Documents

[0006]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0007] When the technology described in Non-Patent Document 1 is applied to the photoelectric conversion device described in Patent Document 1, if the count reaches a threshold (for example, the count value at which the counter saturates) in the first half of the frame, the operation of the APD is stopped in the first half of the frame, resulting in power consumption occurring only in the first half of the frame. Therefore, there is a problem in that power consumption is uneven between the first and second halves of the frame, and power consumption cannot be leveled within the frame. [Means for solving the problem]

[0008] One aspect of the present invention is a photoelectric converter comprising a plurality of pixels including a photodiodes that perform avalanche multiplication, and a signal processing unit that generates a signal based on photons detected by the photodiodes, wherein the signal processing unit comprises a control circuit, a counter, and an illuminance determination unit, the control circuit is connected to the photodiodes and a generation circuit that generates a pulse signal, and controls the photodiodes to a first state in which one node is not electrically connected to a predetermined potential and a second state in which the one node is electrically connected to the predetermined potential, the counter counts the number of periods in which avalanche multiplication occurs among a plurality of periods in the first state, the illuminance determination unit determines whether the count value of the counter has reached a threshold, and the signal processing unit has means for setting the frequency of the pulse signal according to the result of the determination by the illuminance determination unit.

[0009] Another aspect of the present invention is a photoelectric converter comprising a plurality of pixels including a photodiodes that perform avalanche multiplication, and a signal processing unit that generates a signal based on photons detected by the photodiodes, wherein the signal processing unit comprises a control circuit, a counter, and an illuminance determination unit, the control circuit is connected to the photodiodes and a generation circuit that generates a pulse signal, and controls the photodiodes to a first state in which one node is not electrically connected to a predetermined potential and a second state in which the one node is electrically connected to the predetermined potential, the counter counts the number of periods in which avalanche multiplication occurs among a plurality of periods in the first state, the illuminance determination unit determines whether the count value of the counter has reached a threshold, and the signal processing unit has means for setting the timing at which the pulse signal starts to oscillate according to the result of the determination by the illuminance determination unit. [Effects of the Invention]

[0010] According to the present invention, it is possible to provide a photoelectric conversion device that equalizes power consumption within a frame. [Brief explanation of the drawing]

[0011] [Figure 1] This diagram shows the configuration of a photoelectric conversion device according to an embodiment. [Figure 2] This is an example of the arrangement of the sensor substrate of the photoelectric conversion device according to the embodiment. [Figure 3] This is an example of the arrangement of the circuit board of a photoelectric conversion device according to an embodiment. [Figure 4] This is a block diagram including the equivalent circuit of the photoelectric conversion element of the photoelectric conversion device according to the embodiment. [Figure 5] This figure shows the relationship between the operation of the APD of the photoelectric converter according to the embodiment and the output signal. [Figure 6] This figure shows an example of the configuration of a signal processing circuit according to the first embodiment. [Figure 7] This is the output format of the counter circuit and time counter circuit according to the first embodiment. [Figure 8] This is a timing chart illustrating the operation of the signal processing circuit according to the first embodiment. [Figure 9] This is a schematic diagram illustrating how power consumption is leveled. [Figure 10] This is a diagram illustrating a configuration example of the signal processing circuit according to the second embodiment. [Figure 11] This is a timing chart illustrating the operation of the signal processing circuit according to the third embodiment. [Figure 12] This is a schematic diagram of control signals P_CLK0 / 1 and average power consumption. [Figure 13] This is a simplified diagram illustrating the concept of a configuration example of the signal processing circuit according to the fourth embodiment. [Figure 14] This is a simplified diagram illustrating the concept of a configuration example of the signal processing circuit according to the fifth embodiment. [Figure 15] This is a diagram illustrating a configuration example of the signal processing circuit according to the sixth embodiment. [Figure 16] This is the output format of the signal processing circuit according to the sixth embodiment. [Figure 17] This is a functional block diagram of the photoelectric conversion system according to the seventh embodiment. [Figure 18] This is a functional block diagram of the photoelectric conversion system according to the eighth embodiment. [Figure 19] This is a functional block diagram of the photoelectric conversion system according to the ninth embodiment. [Figure 20] This is a functional block diagram of the photoelectric conversion system according to the tenth embodiment. [Figure 21] This is a functional block diagram of the photoelectric conversion system according to the eleventh embodiment. [Modes for Carrying Out the Invention]

[0012] The embodiments shown below are intended to embody the technical concept of the present invention and do not limit it. The size and positional relationships of the components shown in each drawing may be exaggerated for clarity of explanation. In the following description, identical components may be given the same number and their explanation may be omitted.

[0013] Figures 1 to 5 illustrate the common configuration of the photoelectric converter in each embodiment. The photoelectric converter has SPAD pixels including an avalanche diode (APD). The conductivity type of the charge used as the signal charge among the charge pairs generated in the avalanche diode is called the first conductivity type. The first conductivity type refers to a conductivity type in which the majority carriers are charges of the same polarity as the signal charge. The conductivity type opposite to the first conductivity type is called the second conductivity type. Below, we will describe an example in which the signal charge is an electron, the first conductivity type is N-type, and the second conductivity type is P-type, but the signal charge may be a hole, the first conductivity type may be P-type, and the second conductivity type may be N-type.

[0014] When the signal charge is an electron, the signal is read from the cathode of the APD; however, when the signal charge is a hole, the signal is read from the anode of the APD. Therefore, the relationship between the cathode and anode of the APD is reversed.

[0015] In this specification, "planar view" refers to viewing the semiconductor layer on which the photoelectric conversion element described later is located from a direction perpendicular to the light incident surface. A cross-section refers to a surface perpendicular to the light incident surface of the semiconductor layer on which the photoelectric conversion element is located. If the light incident surface of the semiconductor layer is rough when viewed microscopically, the planar view is defined based on the light incident surface of the semiconductor layer as viewed macroscopically.

[0016] In the following explanation, the anode of the APD is set to a fixed potential, and the signal is taken from the cathode side. Therefore, the semiconductor region of the first conductivity type, which has majority carriers of the same polarity as the signal charge, is an N-type semiconductor region, and the semiconductor region of the second conductivity type, which has majority carriers of charges with a different polarity than the signal charge, is a P-type semiconductor region. Note that the present invention also applies when the cathode of the APD is set to a fixed potential and the signal is taken from the anode side. In this case, the semiconductor region of the first conductivity type, which has majority carriers of the same polarity as the signal charge, is a P-type semiconductor region, and the semiconductor region of the second conductivity type, which has majority carriers of charges with a different polarity than the signal charge, is an N-type semiconductor region. The following explanation describes the case where one node of the APD is set to a fixed potential, but the potentials of both nodes may fluctuate.

[0017] In this specification, when the term "impurity concentration" is used, it refers to the net impurity concentration after subtracting the amount compensated by reverse-conductivity impurities. In other words, "impurity concentration" refers to the NET doping concentration. The region where the doping concentration of P-type impurities is higher than that of N-type impurities is a P-type semiconductor region. Conversely, the region where the doping concentration of N-type impurities is higher than that of P-type impurities is an N-type semiconductor region.

[0018] First, we will describe the configuration common to each embodiment.

[0019] Figure 1 shows the configuration of a photoelectric converter 100 in an embodiment of the present invention. In the following description, the case in which the photoelectric converter 100 is a stacked type photoelectric converter will be used as an example. That is, a photoelectric converter configured by stacking and electrically connecting two substrates, a sensor substrate 11 and a circuit substrate 21, will be described as an example. However, the photoelectric converter is not limited to this. For example, it may be a photoelectric converter in which the configuration included in the sensor substrate 11 and the configuration included in the circuit substrate are arranged on a common semiconductor layer, as described below. In the following description, a photoelectric converter in which the configuration included in the sensor substrate 11 and the configuration included in the circuit substrate are arranged on a common semiconductor layer will also be called a non-stacked photoelectric converter.

[0020] The sensor substrate 11 has a first semiconductor layer having a photoelectric conversion element 102 (described later) and a first wiring structure. The circuit board 21 has a second semiconductor layer having circuits such as a signal processing circuit 103 (described later) and a second wiring structure. The photoelectric conversion device 100 is constructed by stacking the second semiconductor layer, the second wiring structure, the first wiring structure, and the first semiconductor layer in that order.

[0021] Figure 1 illustrates a back-illuminated photoelectric converter in which light is incident from the first surface, and the circuit board is located on the second surface, which is opposite to the first surface. In the case of a non-stacked photoelectric converter, the side on which the transistors of the signal processing circuit are located is called the second surface. In the case of a back-illuminated photoelectric converter, the first surface, which is opposite to the second surface of the semiconductor layer, is the light incident surface. In the case of a front-illuminated photoelectric converter, the second surface of the semiconductor layer is the light incident surface.

[0022] In the following description, the sensor substrate 11 and the circuit board 21 are explained using diced chips, but they are not limited to chips. For example, each substrate may be a wafer. Furthermore, each substrate may be stacked in wafer form and then diced, or the chips may be stacked and bonded together after being formed into chips.

[0023] The sensor board 11 is provided with a pixel region 12, and the circuit board 21 is provided with a circuit region 22 for processing signals detected in the pixel region 12.

[0024] Figure 2 shows an example of the arrangement of the sensor substrate 11. Pixels 101, each having a photoelectric conversion element 102 including an avalanche photodiode (hereinafter referred to as APD), are arranged in a two-dimensional array in a planar view, forming a pixel region 12.

[0025] Pixel 101 is typically a pixel used to form an image, but when used in TOF (Time of Flight), it does not necessarily have to form an image. In other words, pixel 101 may be a pixel used to measure the time and amount of light that arrives.

[0026] Figure 3 is a diagram of the circuit board 21. It includes a signal processing circuit 103 for processing the charge photoelectrically converted by the photoelectric conversion element 102 in Figure 2, a readout circuit 112, a control pulse generation unit 115, a horizontal scanning circuit unit 111, a signal line 113, and a vertical scanning circuit unit 110.

[0027] The photoelectric conversion element 102 in Figure 2 and the signal processing circuit 103 in Figure 3 are electrically connected via connection wiring provided for each pixel.

[0028] The vertical scanning circuit section 110 receives control pulses supplied from the control pulse generation section 115 and supplies control pulses to each pixel. Logic circuits such as a shift register and an address decoder are used in the vertical scanning circuit section 110.

[0029] The control pulse generation unit 115 includes a signal generation unit 215 that generates a control signal P_CLK for a switch, which will be described later. The signal generation unit 215 generates a pulse signal to control the switch, as will be described later. For example, as shown in Figure 4(a), the signal generation unit 215 may generate a common control signal P_CLK for multiple pixels in a pixel region, or as shown in Figure 4(b), it may generate a control signal P_CLK for each pixel. When generating a common pulse signal P_CLK, at least one of the period, number of pulses, and pulse width of the pulse signal P_EXP, which controls the exposure period, is generated in common and corresponds to the exposure period. When controlling the control signal P_CLK for each pixel, the signal can be generated using both the input signal P_CLK_IN output from the control pulse generation unit 115 and the signal P_EXP that controls the exposure period. The control pulse generation unit 115 preferably has, for example, a frequency divider circuit. This makes control simpler and reduces the increase in the number of elements.

[0030] The signal output from the photoelectric conversion element 102 of the pixel is processed by the signal processing circuit 103. The signal processing circuit 103 is equipped with a counter and memory, and digital values ​​are stored in the memory.

[0031] The horizontal scanning circuit unit 111 inputs control pulses to the signal processing circuit 103 to sequentially select each column in order to read the signal from the memory of each pixel in which the digital signal is held.

[0032] For the selected column, a signal is output from the signal processing circuit 103 of the pixel selected by the vertical scanning circuit unit 110 to the signal line 113.

[0033] The signal output to signal line 113 is output via output circuit 114 to an external recording unit or signal processing unit of the photoelectric converter 100.

[0034] In Figure 2, the arrangement of photoelectric conversion elements in the pixel region may be one-dimensional. Furthermore, it is possible to obtain the effects of the present invention even with just one pixel, and the present invention is included even in the case of one pixel. However, the power consumption reduction effect of this embodiment is easier to obtain with a photoelectric conversion device having multiple pixels. The function of the signal processing unit does not necessarily need to be provided for each photoelectric conversion element; for example, one signal processing unit may be shared by multiple photoelectric conversion elements, and signal processing may be performed sequentially.

[0035] As shown in Figures 2 and 3, multiple signal processing circuits 103 are arranged in the region that overlaps with the pixel region 12 in a plan view. Then, the vertical scanning circuit section 110, the horizontal scanning circuit section 111, the readout circuit 112, the output circuit 114, and the control pulse generation section 115 are arranged so as to overlap between the edge of the sensor substrate 11 and the edge of the pixel region 12 in a plan view. In other words, the sensor substrate 11 has a pixel region 12 and a non-pixel region arranged around the pixel region 12. Then, the vertical scanning circuit section 110, the horizontal scanning circuit section 111, the readout circuit 112, the output circuit 114, and the control pulse generation section 115 are arranged in the region that overlaps with the non-pixel region in a plan view.

[0036] Note that the arrangement of the signal line 113, the read circuit 112, and the output circuit 114 is not limited to Figure 3. For example, the signal line 113 may be arranged extending in the row direction, and the read circuit 112 may be placed at the end of the signal line 113.

[0037] Figure 4 is an example of a block diagram including the equivalent circuits of Figures 2 and 3. Figure 4(a) is an example in which the signal generation unit 215 is provided in common for multiple pixels, and Figure 4(b) is an example in which the control signal P_CLK can be controlled for each pixel.

[0038] In Figure 4, the photoelectric conversion element 102 having the APD201 is provided on the sensor substrate 11, and the other components are provided on the circuit board 21.

[0039] The APD201 generates charge pairs corresponding to incident light through photoelectric conversion. One of the two nodes of the APD201 is connected to a control line to which a drive voltage VL (first voltage) is supplied. The other node of the APD201 is connected to a control line to which a drive voltage VH (second voltage), which is higher than the voltage VL supplied to the anode, is supplied. In Figure 4, one node of the APD201 is the anode, and the other node of the APD is the cathode. A reverse bias voltage is supplied to the anode and cathode of the APD201 so that the APD201 performs avalanche multiplication. By supplying such a voltage, the charge generated by the incident light undergoes avalanche multiplication, and an avalanche current is generated.

[0040] Furthermore, when a reverse bias voltage is supplied, there are two modes: Geiger mode, in which the anode and cathode operate with a potential difference greater than the breakdown voltage, and linear mode, in which the anode and cathode operate with a potential difference near or below the breakdown voltage.

[0041] An APD operating in Geiger mode is called a SPAD. For example, the voltage VL (first voltage) is -30V and the voltage VH (second voltage) is 1V. The APD201 may operate in linear mode or Geiger mode. In the case of a SPAD, the potential difference is larger compared to a linear mode APD, and the voltage withstand effect is more pronounced, so it is preferable to use a SPAD.

[0042] Switch 202 is connected to the control line to which the drive voltage VH is supplied and to the APD201. Switch 202 is connected to one of the nodes, either the anode or the cathode, of the APD. Switch 202 switches the potential difference between the anode and cathode of the APD between a first potential difference that enables avalanche multiplication and a second potential difference that does not enable avalanche multiplication. Hereafter, switching from the second potential difference to the first potential difference will be referred to as turning switch 202 on, and switching from the first potential difference to the second potential difference will be referred to as turning switch 202 off. Switch 202 functions as a quench element. When the signal is multiplied by avalanche multiplication, switch 202 functions as a load circuit (quench circuit) and suppresses the voltage supplied to the APD201, thereby suppressing avalanche multiplication (quench operation). Furthermore, switch 202 has the function of returning the voltage supplied to APD201 to the drive voltage VH by supplying current to compensate for the voltage drop caused by the quench operation (recharge operation). In other words, switch 202 functions as a control circuit that controls the occurrence of avalanche multiplication in APD201.

[0043] The switch 202 can be configured, for example, with a MOS transistor, and Figure 4 shows the case where the switch 202 is a PMOS transistor. The control signal P_CLK for the switch 202, supplied from the signal generation unit 215, is applied to the gate electrode of the MOS transistor constituting the switch 202. In this embodiment, the on and off states of the switch 202 are controlled by controlling the voltage applied to the gate electrode of the switch 202.

[0044] The signal processing circuit 103 includes a waveform shaping unit 210, a counter circuit 211, and a selection circuit 212. In Figure 4, the signal processing circuit 103 includes a waveform shaping unit 210, a counter circuit 211, and a selection circuit 212, but in this specification, the signal processing circuit 103 only needs to have at least one of the waveform shaping unit 210, the counter circuit 211, and the selection circuit 212.

[0045] The waveform shaping unit 210 shapes the cathode potential change of the APD201 obtained during photon detection and outputs a pulse signal. The input node of the waveform shaping unit 210 is denoted as nodeA, and the output node as nodeB. The waveform shaping unit 210 changes the output potential from nodeB depending on whether the input potential to nodeA is above or below a predetermined value. For example, in Figure 5, when the input potential to nodeA becomes a high potential above a judgment threshold, the output potential from nodeB becomes low level. When the input potential to nodeA becomes a potential below the judgment threshold, the output potential from nodeB becomes high level. For example, an inverter circuit can be used as the waveform shaping unit 210. Figure 4 shows an example using one inverter as the waveform shaping unit 210, but a circuit with multiple inverters connected in series may be used, or other circuits that have a waveform shaping effect may be used.

[0046] While it is possible to perform quench and recharge operations using switch 202 in response to avalanche multiplication in APD201, the output signal may not be recognized depending on the timing of photon detection. For example, consider the case where avalanche multiplication occurs in APD, causing node A to become low level and triggering a recharge operation. Generally, the judgment threshold of the waveform shaping unit 210 is set to a potential higher than the potential difference at which avalanche multiplication occurs in APD. When a photon is incident while the potential of node A is lower than the judgment threshold due to the recharge operation and at a potential where avalanche multiplication is possible in APD, avalanche multiplication occurs in APD and the voltage of node A decreases. In other words, because the potential of node A decreases at a voltage lower than the judgment threshold, the output potential from node B does not change despite the detection of a photon. Therefore, even though avalanche multiplication is occurring, it is not recognized as a signal. In particular, under high illumination conditions, photons enter continuously in short periods, making it difficult to recognize them as a signal. As a result, even at high illumination levels, there is a tendency for the actual number of incident photons to deviate from the output signal.

[0047] In contrast, by applying the control signal P_CLK to switch 202 and switching it on and off, it becomes possible to determine the signal even when photons enter the APD continuously in a short period of time. Figure 5 illustrates an example where the control signal P_CLK is a pulse signal with a repeating period. In other words, Figure 5 illustrates a configuration in which the on / off state of switch 202 is switched at a predetermined clock frequency. However, the effect of suppressing the increase in power consumption of the photoelectric converter can be obtained even if the pulse signal is not a repeating period signal.

[0048] The counter circuit 211 counts the pulse signal output from the waveform shaping unit 210 and holds the count value. When the control pulse pRES is supplied via the drive line 213, the signal held by the counter circuit 211 is reset.

[0049] The selection circuit 212 receives a control pulse pSEL from the vertical scanning circuit section 110 in Figure 3 via the drive line 214 (not shown in Figure 3) in Figure 4, which switches the electrical connection between the counter circuit 211 and the signal line 113. The selection circuit 212 includes, for example, a buffer circuit for outputting a signal. The output signal OUT shown in Figure 4 is the signal output from the pixel.

[0050] A switch such as a transistor may be placed between switch 202 and APD201, or between photoelectric conversion element 102 and signal processing circuit 103, to switch the electrical connection. Similarly, the supply of voltage VH or voltage VL to the photoelectric conversion element 102 may be electrically switched using a switch such as a transistor.

[0051] In this embodiment, a configuration using a counter circuit 211 is shown. However, instead of the counter circuit 211, a photoelectric converter 100 may be used that acquires pulse detection timing using a Time to Digital Converter (TDC) and memory. In this case, the generation timing of the pulse signal output from the waveform shaping unit 210 is converted into a digital signal by the TDC. A control pulse pREF (reference signal) is supplied to the TDC via a drive line from the vertical scanning circuit unit 110 in Figure 1 to measure the timing of the pulse signal. The TDC acquires the signal as a digital signal when the input timing of the signal output from each pixel via the waveform shaping unit 210 is relative to the control pulse pREF.

[0052] As shown in Figure 4(b), the signal generation unit 215 may be provided for each pixel. In Figure 4(b), the circuit diagrams from the waveform shaping unit 210 onwards and the signal generation unit shown in Figure 4(a) are omitted. Note that the signal generation unit 215 in Figure 4(a) is assumed to be present for each pixel. In Figure 4(b), a logic circuit is taken up within the pixel to determine whether or not to supply a pulse signal to the switch 202. The logic circuit receives the signal P_EXP which controls the exposure period and the input signal P_CLK_IN which controls the control signal P_CLK. An inverted signal is then output. For example, if the signal P_EXP which controls the exposure period is low level and the input signal P_CLK_IN is low level, a high-level signal is output from the control signal P_CLK. In other words, the switch is turned off. Also, if the signal P_EXP which controls the exposure period is high level and the input signal P_CLK_IN is high level, a low-level signal is output from the control signal P_CLK. In other words, the switch is turned on. Furthermore, if either the exposure period control signal P_EXP or the input signal P_CLK_IN is at a low level, a high-level signal is output as the control signal P_CLK. In other words, switch 202 is turned off. It is preferable to control the switch for each pixel in this manner. Note that when using the circuit diagram in Figure 4(b), as will be explained in the second embodiment described later, when the exposure period P becomes low, the control signal P_CLK remains at a high level. In other words, the switch is turned off.

[0053] Figure 5 schematically shows the relationship between the switch control signal P_CLK, the potential of node A, the potential of node B, and the output signal. In this embodiment, when the control signal P_CLK is high level, the drive voltage VH is less likely to be supplied to the APD, and when the control signal P_CLK is low level, the drive voltage VH is supplied to the APD. A high level control signal P_CLK is, for example, 1V, and a low level control signal P_CLK is, for example, 0V. When the control signal P_CLK is high level, the switch is off, and when the control signal P_CLK is low level, the switch is on. The resistance value of the switch when the control signal P_CLK is high level is higher than the resistance value of the switch when the control signal P_CLK is low level. When the control signal P_CLK is high level, even if avalanche multiplication occurs in the APD, recharge operation is less likely to occur, so the potential supplied to the APD becomes a potential below the breakdown voltage of the APD. Therefore, the avalanche multiplication operation in the APD stops.

[0054] As shown in Figure 4, it is preferable to configure the switch 202 with a single transistor and perform both the quench operation and the recharge operation with that single transistor. This makes it possible to reduce the number of circuits compared to when the quench operation and the recharge operation are performed with different circuit elements. In particular, when each pixel has a counter circuit and reads out the SPAD signal for each pixel, it is preferable to reduce the circuit area used for the switch in order to accommodate the counter circuit, and the effect of configuring the switch 202 with a single transistor becomes significant.

[0055] At time t1, the control signal P_CLK changes from high to low, the switch turns on, and the APD recharge operation begins. This causes the cathode potential of the APD to transition to a high level. Then, the potential difference between the anode and cathode of the APD becomes a state where avalanche multiplication is possible. The cathode potential is the same as that of node nodeA. Therefore, when the cathode potential transitions from low to high, at time t2, the potential of node nodeA becomes greater than or equal to the judgment threshold. At this time, the pulse signal output from node B inverts, going from high to low. Subsequently, a potential difference of drive voltage VH - drive voltage VL is applied to APD201. The control signal P_CLK becomes high, and the switch turns off.

[0056] Next, at time t3, when a photon is incident on APD201, avalanche multiplication occurs in APD201, and the cathode voltage drops. In other words, the voltage at node A drops. As the voltage drop becomes larger and the voltage difference applied to APD201 decreases, the avalanche multiplication of APD201 stops as at time t2, and the voltage level at node A stops dropping below a certain value. If the voltage at node A falls below the judgment threshold while the voltage at node A is dropping, the voltage at node B changes from a low level to a high level. In other words, the portion of the output waveform at node A that exceeds the judgment threshold is waveform shaped by the waveform shaping unit 210 and output as a signal at node B. This is then counted by the counter circuit, and the count value of the counter signal output from the counter circuit increases by 1 LSB.

[0057] Although photons are incident on the APD between time t3 and time t4, the switch is off, and the voltage applied to APD201 is not a potential difference that allows for avalanche multiplication. Therefore, the voltage level of node A does not exceed the threshold.

[0058] At time t4, the control signal P_CLK changes from high to low, and the switch turns on. Consequently, current flows through node A to compensate for the voltage drop from the drive voltage VH, and the voltage at node A returns to its original voltage level. At this time, at time t5, the voltage at node A exceeds the threshold, so the pulse signal at node B inverts, changing from high to low.

[0059] At time t6, node A settles back to its original voltage level, and the control signal P_CLK changes from a low level to a high level. Therefore, the switch is turned off. Subsequently, as explained from time t1 to time t6, the potentials of each node and signal line change in response to the control signal P_CLK and the incidence of photons.

[0060] The following describes the photoelectric conversion device for each embodiment.

[0061] <First Embodiment> Figure 6 shows an example of the configuration of the signal processing circuit 103 according to the first embodiment.

[0062] The signal processing circuit 103 includes a counter circuit 211, a selection circuit 212, a selection circuit 226, a photon pulse generation unit 220, a frequency divider circuit 221, a time counter circuit 222, an illuminance determination unit 223, and a selector 227.

[0063] The photon pulse generation unit 220 includes a switch 202 and a waveform shaping unit 210. The illuminance determination unit 223 includes a threshold count detection circuit 224 and a comparison circuit 225.

[0064] The counter circuit 211 counts the pulse signals output from the photon pulse generation unit 220. The counter circuit 211 resets the count value in response to the supply of control pulses pRES from the control pulse generation unit 115 via the vertical scanning circuit unit 110 and drive line 213. The counter circuit 211 also starts counting on the rising edge of the drive signal CNTEN supplied from the control pulse generation unit 115 via the vertical scanning circuit unit 110 and drive line 228, and ends counting on the falling edge. In other words, the drive signal CNTEN is a signal that controls the exposure time. The counter circuit 211 is, for example, an 8-bit counter.

[0065] The threshold count detection circuit 224 detects when the count value of the counter circuit 211 reaches a predetermined threshold and notifies the time counter circuit 222, the comparison circuit 225, and the selector 227. In the first embodiment, the predetermined threshold is set to 255, which is the countable saturation value of the 8-bit counter, but the threshold may also be an intermediate value such as 127 or 63.

[0066] The time counter circuit 222 is a counter that measures exposure time. It starts counting the exposure time from the start of exposure for one frame, and stops counting when the threshold count detection circuit 224 notifies that the count value of the counter circuit 211 has reached the saturation value within the duration of that frame. If it is not notified that the count value of the counter circuit 211 has reached the saturation value within the duration of one frame, it does not stop counting for that frame. The time counter circuit 222 also resets the count value in response to the supply of control pulse pRES from the control pulse generation unit 115 via the vertical scanning circuit unit 110 and the drive line 213. Furthermore, the time counter circuit 222 starts counting on the rising edge of the drive signal CNTEN supplied from the control pulse generation unit 115 via the vertical scanning circuit unit 110 and the drive line 228, and stops counting on the falling edge. In other words, the drive signal CNTEN is a signal that controls the exposure time. The time counter circuit 222 is, for example, a 14-bit counter that counts the clock edges of the time clock (TCLK) supplied from outside the signal processing circuit 103 via the control line 229.

[0067] In Figure 6, the signal processing circuit 103 is configured to have a time counter circuit 222, but this configuration is not limited to this, as long as the time when the count value of the counter circuit 211 reaches the saturation value can be obtained within the period of one frame. For example, the signal processing circuit 103 may have a latch circuit, and a time code may be constantly input to the latch circuit from outside the signal processing circuit 103. In this case, when the latch circuit is notified that the count value of the counter circuit 211 has reached the saturation value within the period of one frame, the latch circuit latches the time code. Alternatively, for example, the counter circuit 211 may count the pulse signals output from the photon pulse generation unit 220 until the count value of the counter circuit 211 reaches the saturation value within the period of one frame. In this case, when the count value of the counter circuit 211 reaches the saturation value within the period of one frame, the counter circuit 211 is reconfigured into a latch circuit, and the time code is latched in the latch circuit. As a result, the counter circuit 211 and the latch circuit are shared, so the circuit size can be reduced.

[0068] The comparison circuit 225 compares the timing at which the control signal CTIME0 or CTIME1 is input with the timing at which the count value of the counter circuit 211 reaches its saturation value. The control signals CTIME0 and CTIME1 are supplied from the control pulse generation unit 115 via the vertical scanning circuit unit 110 and control lines 230 and 231, and the timing at which the count value of the counter circuit 211 reaches its saturation value is notified by the threshold count detection circuit 224. The control signals CTIME0 or CTIME1 to be compared are selected by the selection circuit 226. The comparison circuit 225 compares whether the timing at which the control signal CTIME selected by the selection circuit 226 is input or the timing at which the count value of the counter circuit 211 reaches its saturation value comes first, and notifies the frequency divider circuit 221, the counter circuit 211, and the selection circuit 226. If the count value of the counter circuit 211 does not reach its saturation value, nothing is notified by the threshold count detection circuit 224. The comparison circuit 225 determines that the timing at which the control signal CTIME selected by the selection circuit 226 is input is earlier than the timing at which the count value of the counter circuit 211 reaches the saturation value.

[0069] The illuminance determination unit 223 determines the illuminance according to the comparison result of the comparison circuit 225. If the timing at which the count value of the counter circuit 211 reaches the saturation value is earlier than the timing at which the control signal CTIME selected by the selection circuit 226 is input, it is determined to be high illuminance; otherwise, it is determined to be low illuminance.

[0070] The frequency divider circuit 221 divides the control signal P_CLK supplied from the signal generation unit 215 via the vertical scanning circuit unit 110 and the drive line 232, according to the comparison result notified by the comparison circuit 225. If the illuminance determination unit 223 determines from the comparison result of the comparison circuit 225 that the illuminance is low, the frequency divider circuit 221 does not divide the control signal P_CLK; if it determines that the illuminance is high, it divides the control signal P_CLK by 2 and supplies it to the photon pulse generation unit 220. The frequency division ratio can be designed to a value suitable for the system, and may be 4 division or 8 division, or a configuration with multiple frequency division ratios may be used. In the first embodiment, the comparison circuit 225 stores the comparison result in a buffer or the like, and the frequency division ratio is switched when the control pulse pRES is supplied to the frequency divider circuit 221 at the beginning of the next frame, but the configuration is not limited to this. For example, the frequency division ratio may be switched in real time in the middle of a frame.

[0071] The selection circuit 226 selects either control signal CTIME0 or control signal CTIME1 supplied from the control pulse generation unit 115 via the vertical scanning circuit unit 110 and control lines 230 and 231, according to the comparison result notified by the comparison circuit 225. In the first embodiment, control signal CTIME0 is used as the control signal for low illuminance, and control signal CTIME1 is used as the control signal for high illuminance. If the illuminance determination unit 223 determines from the comparison result of the comparison circuit 225 that the illuminance is low, the selection circuit 226 selects control signal CTIME0 and supplies it to the comparison circuit 225. Similarly, if the illuminance determination unit 223 determines from the comparison result of the comparison circuit 225 that the illuminance is high, the selection circuit 226 selects control signal CTIME1 and supplies it to the comparison circuit 225. In the first embodiment, the comparison circuit 225 stores the comparison result in a buffer or the like, and switches the control signal CTIME at the timing when the control pulse pRES is supplied to the selection circuit 226 at the beginning of the next frame, but the configuration is not limited to this. For example, it's also acceptable to configure the system to switch the control signal CTIME in real time during a frame.

[0072] If the selector 227 is notified by the threshold count detection circuit 224 that the count value of the counter circuit 211 has reached the saturation value within one frame period, it selects the output of the time counter circuit 222 and supplies it to the selection circuit 212. If the selector 227 is not notified that the count value of the counter circuit 211 has reached the saturation value within one frame period, it selects the output of the counter circuit 211 and supplies it to the selection circuit 212.

[0073] The selection circuit 212 switches the electrical connection between the selector 227 and the signal line 113 using control pulses pSEL supplied from the control pulse generation unit 115 via the vertical scanning circuit unit 110 and the drive line 214. The selection circuit 212 includes, for example, a buffer circuit for outputting a signal.

[0074] Figure 7 shows the output formats of the counter circuit 211 and the time counter circuit 222 according to the first embodiment. Figure 7(a) shows the output format 701 of the counter circuit 211, and Figure 7(b) shows the output format 702 of the time counter circuit 222.

[0075] The most significant bit of the output format 701 of the counter circuit 211 is provided with a counter flag 7011 that indicates whether the data is an output of the counter circuit 211 or a time counter circuit 222 in a subsequent processing stage (not shown). In the first embodiment, 0 indicates the counter circuit 211, and 1 indicates the time counter circuit 222. The counter flag 7011 is stored as 0.

[0076] The output format 701 of the counter circuit 211 includes a P_CLK flag 7012 that indicates whether or not the control signal P_CLK was divided by 2 in a subsequent processing step (not shown). In the first embodiment, 0 indicates no division, and 1 indicates division by 2. The P_CLK flag 7012 stores either a value of 0 or 1. The division ratio can be designed to be a value suitable for the system, and may be 4-fold or 8-fold. Furthermore, the P_CLK flag 7012 may be extended to multiple bits to accommodate multiple division ratios.

[0077] In order to make the output bus width of the counter circuit 211 and the time counter circuit 222 16 bits, the output format 701 of the counter circuit 211 is provided with a 6-bit reserve area 7013, and in the first embodiment, 0 is stored in the reserve area 7013.

[0078] The lower 8 bits of the output format 701 of the counter circuit 211 are provided with a region 7014 for storing the count value.

[0079] The counter circuit 211 adds a fixed value counter flag 7011 and a reserve area 7013. The counter circuit 211 further adds 0 to the P_CLK flag 7012 if the illuminance determination unit 223 determines that the illuminance is low, based on the comparison result of the comparison circuit 225, and adds 1 to the P_CLK flag 7012 if the illuminance determination unit 223 determines that the illuminance is high. This forms the output format 701 of the counter circuit 211.

[0080] In the first embodiment, the comparison circuit 225 stores the comparison result in a buffer or the like, and switches the division ratio when the control pulse pRES is supplied to the frequency divider circuit 221 at the beginning of the next frame. However, the configuration is not limited to this. For example, the division ratio may be switched in real time in the middle of a frame. In that case, a flag indicating that the division ratio was switched in the middle of a frame and information indicating the time when the division ratio was switched are added to the output format 701 of the counter circuit 211.

[0081] The most significant bit of the output format 702 of the time counter circuit 222 is provided with a counter flag 7021 that indicates in a subsequent processing stage (not shown) whether the data is an output of the counter circuit 211 or the time counter circuit 222. In the first embodiment, 0 indicates the counter circuit 211, and 1 indicates the time counter circuit 222, and 1 is stored in the counter flag 7021.

[0082] The output format 702 of the time counter circuit 222 includes a P_CLK flag 7022 that indicates whether or not the control signal P_CLK was divided by 2 in a subsequent processing step (not shown). In the first embodiment, 0 indicates no division and 1 indicates division by 2, and the P_CLK flag 7022 stores either a value of 0 or 1. The division ratio can be designed to be a value suitable for the system, and may be 4-fold or 8-fold. Furthermore, the P_CLK flag 7022 may be extended to multiple bits to accommodate multiple division ratios.

[0083] The lower 14 bits of the output format 702 of the time counter circuit 222 are provided with a region 7023 for storing the time count value.

[0084] The time counter circuit 222 adds a fixed value counter flag 7021. The time counter circuit 222 further adds 0 to the P_CLK flag 7022 if the illuminance determination unit 223 determines that the illuminance is low, based on the comparison result of the comparison circuit 225, and adds 1 to the P_CLK flag 7022 if the illuminance determination unit 223 determines that the illuminance is high. This forms the output format 702 of the time counter circuit 222.

[0085] In the first embodiment, the comparison circuit 225 stores the comparison result in a buffer or the like, and switches the frequency division ratio when a control pulse pRES is supplied to the frequency divider circuit 221 at the beginning of the next frame. However, the configuration is not limited to this. For example, the frequency division ratio may be switched in real time in the middle of a frame. In that case, a flag indicating that the frequency division ratio was switched in the middle of a frame and information indicating the time when the frequency division ratio was switched are added to the output format 702 of the time counter circuit 222.

[0086] In the subsequent processing (not shown), the counter flags 7011 and 7021 of the received data are referenced, and if each counter flag is 0, it is recognized as the output of the counter circuit 211, and if it is 1, it is recognized as the output of the time counter circuit 222.

[0087] If the received data is the output of the counter circuit 211, the subsequent processing (not shown) further refers to the P_CLK flag 7012 and corrects the count value 7014 according to its value. If the P_CLK flag 7012 is 0, the control signal P_CLK is not divided, and the count value 7014 is treated as the number of photons per frame without correction. If the P_CLK flag 7012 is 1, the control signal P_CLK is divided by 2, and the count value 7014 is doubled and treated as the number of photons per frame. When the control signal P_CLK is divided by 2, the counting speed of the counter circuit 211 becomes halved compared to the case of no division, even at the same illuminance, so the count value 7014 is doubled to correct it.

[0088] If the received data is the output of the time counter circuit 222, the subsequent processing (not shown) further refers to the P_CLK flag 7022 and calculates the number of photons per frame from the time count value 7023 according to its value. If the P_CLK flag 7022 is 0, the control signal P_CLK is not divided. In that case, the number of photons per frame CNT is calculated from the saturation value S of the counter circuit 211, the count value FT from the time counter circuit 222 for one frame period, and the count value ST from the time counter circuit 222 until the saturation value S is reached, using the following formula. CNT=S×FT / ST…(Formula 1)

[0089] On the other hand, if the P_CLK flag 7022 is 1, the control signal P_CLK is divided by 2. In that case, the number of photons per frame CNT is calculated from the saturation value S of the counter circuit 211, the count value FT of the time counter circuit 222 for one frame period, and the count value ST of the time counter circuit 222 until the saturation value S is reached, using the following formula. CNT=S×FT / (ST / 2)…(Formula 2)

[0090] When the control signal P_CLK is divided by 2, it takes twice as long to reach the saturation value S, even at the same illuminance, compared to the case without division. Therefore, the time ST to reach the saturation value S is halved to compensate.

[0091] Figure 8 is a timing chart showing the operation of the signal processing circuit 103 according to the first embodiment.

[0092] At time T0, the first frame begins.

[0093] At time T1, a control pulse pRES is supplied from the control pulse generation unit 115 via the vertical scanning circuit unit 110 and the drive line 213, and the count values ​​of the counter circuit 211 and the time counter circuit 222 are reset. The count values ​​of the counter circuit 211 and the time counter circuit 222 are set to 0.

[0094] At time T2, the drive signal CNTEN supplied from the control pulse generation unit 115 via the vertical scanning circuit unit 110 and drive line 228 rises, and the counting of the counter circuit 211 and the time counter circuit 222, i.e., exposure begins. In the first frame, the frequency divider circuit 221 supplies the control signal P_CLK to the photon pulse generation unit 220 without dividing it.

[0095] At time T3, the count value of the counter circuit 211 reaches the saturation value, and the threshold count detection circuit 224 detects the saturation. The time counter circuit 222 is notified by the threshold count detection circuit 224 that the count value of the counter circuit 211 has reached the saturation value, and stops counting.

[0096] At time T4, the comparison circuit 225 is supplied with the control signal CTIME0 selected by the selection circuit 226. At this point, the threshold count detection circuit 224 has detected saturation, so the timing at which the count value of the counter circuit 211 reaches the saturation value is earlier than the timing at which the control signal CTIME0 is input, and the illuminance determination unit 223 determines that the first frame is high illuminance. The frequency divider circuit 221 divides the control signal P_CLK by 2 if the illuminance determination unit 223 determines that the illuminance is high based on the comparison result of the comparison circuit 225. The selection circuit 226 selects the control signal CTIME1 if the illuminance determination unit 223 determines that the illuminance is high based on the comparison result of the comparison circuit 225.

[0097] If the illuminance determination unit 223 determines that the illuminance is high, the frequency divider circuit 221 divides the control signal P_CLK by 2 in the next frame, and the selection circuit 226 selects the control signal CTIME1. On the other hand, if the illuminance determination unit 223 determines that the illuminance is low, the frequency divider circuit 221 does not divide the control signal P_CLK in the next frame, and the selection circuit 226 selects the control signal CTIME0. If the frequency divider circuit 221 divides the control signal P_CLK by 2, the counting speed of the counter circuit 211 becomes halved, even if the illuminance is the same. Therefore, the period during which the control signal CTIME, which indicates the timing for comparison by the comparison circuit 225, is input also needs to be doubled. For example, if the time from the rising edge of the drive signal CNTEN to the falling edge of the control signal CTIME0 is CmpT, then the time from the rising edge of the drive signal CNTEN to the falling edge of the control signal CTIME1 is set to 2 × CmpT.

[0098] In the first embodiment, the comparison circuit 225 stores the comparison result in a buffer or the like, and switches the division ratio when the control pulse pRES is supplied to the frequency divider circuit 221 at the beginning of the next frame. Therefore, the control signal P_CLK, which has been divided by 2, is actually used from the second frame onwards. However, it is also acceptable to configure the system to switch the division ratio in real time during the first frame.

[0099] At time T5, the drive signal CNTEN supplied from the control pulse generation unit 115 via the vertical scanning circuit unit 110 and drive line 228 falls, ending the exposure and transitioning to readout. In other words, the period from time T2 to time T5 is the exposure period for the first frame.

[0100] At time T6, the selection circuit 212 is supplied with a control pulse pSEL from the control pulse generation unit 115 via the vertical scanning circuit unit 110 and the drive line 214. At this point, the threshold count detection circuit 224 detects that the count value of the counter circuit 211 has reached saturation. As a result, the output format 702 of the time counter circuit 222 selected by the selector 227 is output to the signal line 113.

[0101] At time T7, the first frame ends and the second frame begins.

[0102] At time T8, a control pulse pRES is supplied from the control pulse generation unit 115 via the vertical scanning circuit unit 110 and the drive line 213, and the count values ​​of the counter circuit 211 and the time counter circuit 222 are reset. The count values ​​of the counter circuit 211 and the time counter circuit 222 are set to 0. In addition, the illuminance determination result (high illuminance) of the first frame is set in the frequency divider circuit 221, and the frequency divider circuit 221 outputs the control signal P_CLK divided by 2. The comparison result of the comparison circuit 225 and the detection result of the threshold count detection circuit 224 are cleared.

[0103] At time T9, the drive signal CNTEN supplied from the control pulse generation unit 115 via the vertical scanning circuit unit 110 and drive line 228 rises, and the counter circuit 211 and time counter circuit 222 begin counting, i.e., exposure begins. The illuminance determination unit 223 determines that the first frame is high illuminance, so in the second frame, the frequency divider circuit 221 divides the control signal P_CLK by 2 and supplies it to the photon pulse generation unit 220.

[0104] At time T10, the comparison circuit 225 is supplied with the control signal CTIME1 selected by the selection circuit 226. At this point, the threshold count detection circuit 224 has not detected saturation of the count value of the counter circuit 211. The timing at which the count value of the counter circuit 211 reaches the saturation value is later than the timing at which the control signal CTIME1 is input, and the illuminance determination unit 223 determines that the second frame is low illuminance. The frequency divider circuit 221 does not divide the control signal P_CLK if the illuminance determination unit 223 determines from the comparison result of the comparison circuit 225 that the illuminance is low. The selection circuit 226 selects the control signal CTIME0 if the illuminance determination unit 223 determines from the comparison result of the comparison circuit 225 that the illuminance is low illuminance.

[0105] In the first embodiment, the comparison circuit 225 stores the comparison result in a buffer or the like, and switches the division ratio when the control pulse pRES is supplied to the frequency divider circuit 221 at the beginning of the next frame. Therefore, the control signal P_CLK, which is not actually divided by 2, is used from the third frame onwards. However, it is also acceptable to configure the system to switch the division ratio in real time during the second frame.

[0106] At time T11, the count value of the counter circuit 211 reaches the saturation value, and the threshold count detection circuit 224 detects the saturation. The time counter circuit 222 is notified by the threshold count detection circuit 224 that the count value of the counter circuit 211 has reached the saturation value, and stops counting.

[0107] At time T12, the drive signal CNTEN supplied from the control pulse generation unit 115 via the vertical scanning circuit unit 110 and drive line 228 falls, ending the exposure and transitioning to readout. In other words, the period from time T9 to time T12 becomes the exposure period for the second frame.

[0108] At time T13, the selection circuit 212 is supplied with a control pulse pSEL from the control pulse generation unit 115 via the vertical scanning circuit unit 110 and the drive line 214. At this point, the threshold count detection circuit 224 has detected saturation of the count value of the counter circuit 211, and as a result, the output format 702 of the time counter circuit 222 selected by the selector 227 is output to the signal line 113.

[0109] At time T14, the second frame ends and the third frame begins.

[0110] At time T15, a control pulse pRES is supplied from the control pulse generation unit 115 via the vertical scanning circuit unit 110 and the drive line 213, and the count values ​​of the counter circuit 211 and the time counter circuit 222 are reset. The count values ​​of the counter circuit 211 and the time counter circuit 222 are set to 0. In addition, the illuminance determination result (low illuminance) of the second frame is set in the frequency divider circuit 221, and the frequency divider circuit 221 outputs the control signal P_CLK without dividing the frequency. The comparison result of the comparison circuit 225 and the detection result of the threshold count detection circuit 224 are cleared.

[0111] At time T16, the drive signal CNTEN supplied from the control pulse generation unit 115 via the vertical scanning circuit unit 110 and drive line 228 rises, and the counter circuit 211 and time counter circuit 222 begin counting, i.e., exposure begins. The illuminance determination unit 223 determines that the second frame is low illuminance, so in the third frame, the frequency divider circuit 221 supplies the control signal P_CLK to the photon pulse generation unit 220 without dividing it.

[0112] At time T17, the comparison circuit 225 is supplied with the control signal CTIME0 selected by the selection circuit 226. At this point, the threshold count detection circuit 224 has not detected saturation of the count value of the counter circuit 211, and the illuminance determination unit 223 determines that the third frame is low illuminance. The frequency divider circuit 221 does not divide the control signal P_CLK based on the comparison result of the comparison circuit 225, as it did for the second frame. The selection circuit 226 selects the control signal CTIME0 based on the comparison result of the comparison circuit 225, as it did for the second frame.

[0113] At time T18, the drive signal CNTEN supplied from the control pulse generation unit 115 via the vertical scanning circuit unit 110 and drive line 228 falls, the counter circuit 211 and time counter circuit 222 stop counting, exposure ends and the process moves to readout. In other words, the period from time T16 to time T18 becomes the exposure period for the third frame.

[0114] At time T19, the selection circuit 212 is supplied with a control pulse pSEL from the control pulse generation unit 115 via the vertical scanning circuit unit 110 and the drive line 214. At this point, the threshold count detection circuit 224 has not detected saturation of the count value of the counter circuit 211, and as a result, the output format 701 of the counter circuit 211 selected by the selector 227 is output to the signal line 113.

[0115] At time T20, the third frame ends.

[0116] Furthermore, even though the illuminance determination unit 223 determines that the first frame is high illuminance, the frequency divider circuit 221 supplies the control signal P_CLK to the photon pulse generation unit 220 without dividing it. Also, even though the illuminance determination unit 223 determines that the second frame is low illuminance, the frequency divider circuit 221 divides the control signal P_CLK by 2 and supplies it to the photon pulse generation unit 220. In the first embodiment, the comparison circuit 225 stores the comparison result in a buffer or the like, and switches the division ratio at the timing when the control pulse pRES is supplied to the frequency divider circuit 221 at the beginning of the next frame. Therefore, this state occurs when switching from low illuminance to high illuminance, or from high illuminance to low illuminance. However, probabilistically, low illuminance and high illuminance frames tend to be consecutive. Also, if the division ratio is switched in real time in the middle of a frame, this state will not occur.

[0117] In a high-illumination frame, if the frequency divider circuit 221 divides the control signal P_CLK by 2, the counting speed of the counter circuit 211 becomes half, even at the same illuminance. Therefore, compared to when the control signal P_CLK is not divided, the time it takes for the threshold count detection circuit 224 to detect saturation becomes twice as long.

[0118] On the other hand, if the frequency divider circuit 221 divides the control signal P_CLK by 2 in a high-illumination frame, the frequency of recharge operation is halved, even at the same illuminance. Compared to when the control signal P_CLK is not divided, the frequency of current (recharge current) due to voltage drop during quench operation and the frequency of avalanche current flow are also halved. As a result, when the frequency divider circuit 221 divides the control signal P_CLK by 2, power consumption can be leveled out within one frame period compared to when the control signal P_CLK is not divided.

[0119] Figures 9(a) and 9(b) illustrate how power consumption is leveled out within a single frame.

[0120] Figure 9(a) shows the exposure time and average power consumption when the first embodiment is not applied. ExpT is the exposure period until the count value of the counter circuit 211 saturates, and P is the rise height of the average power consumption during the exposure period. Since the count value of the counter circuit 211 saturates in the first half of the frame period, power consumption is concentrated in the first half of the frame period.

[0121] Figure 9(b) shows the exposure time and average power consumption when the first embodiment is applied and the frequency divider circuit 221 divides the control signal P_CLK by 2. The exposure period until the count value of the counter circuit 211 saturates is 2 × ExpT, and the rise height of the average power consumption during the exposure period is 2 / P. Since the count value of the counter circuit 211 saturates in the latter half of the frame period, power consumption is not concentrated in the first half of the frame. In other words, it can be seen that power consumption is leveled out within one frame.

[0122] In the first embodiment described above, the frequency divider circuit 221 divides the control signal P_CLK in high-illumination frames to input a low-frequency control signal P_CLK, and in low-illumination frames to input a high-frequency control signal P_CLK, thereby leveling out power consumption within a single frame.

[0123] <Second Embodiment> Figure 10 shows an example of the configuration of the signal processing circuit 103 according to the second embodiment.

[0124] The difference from the first embodiment shown in Figure 6 is that the frequency divider circuit 221 is replaced by a selection circuit 234.

[0125] The selection circuit 234 selects one of several control signals P_CLK0 / 1 supplied from the signal generation unit 215 via the vertical scanning circuit unit 110 and drive lines 232 and 233, according to the comparison result notified by the comparison circuit 225. Based on the comparison result from the comparison circuit 225, the selection circuit 234 selects control signal P_CLK0 if the illuminance determination unit 223 determines that the illuminance is low, and selects control signal P_CLK1 if it determines that the illuminance is high. The selected control signal P_CLK is supplied to the photon pulse generation unit 220. The signal generation unit 215 generates control signal P_CLK0, which corresponds to the control signal P_CLK in the first embodiment, and control signal P_CLK1, which is a division of P_CLK by two. The division ratio can be designed to a value suitable for the system, and may be a division by four or eight, or it may be configured to have multiple division ratios, i.e., the signal generation unit 215 may output three or more types of control signals P_CLK. In the second embodiment, the comparison circuit 225 stores the comparison result in a buffer or the like, and switches the control signal P_CLK at the timing when the control pulse pRES is supplied to the selection circuit 234 at the beginning of the next frame, but the configuration is not limited to this. For example, the control signal P_CLK may be switched in real time in the middle of a frame.

[0126] The output formats of the counter circuit 211 and the time counter circuit 222 according to the second embodiment are the same as those of the first embodiment shown in Figure 7.

[0127] The timing chart showing the operation of the signal processing circuit 103 according to the second embodiment is substantially the same as that of the first embodiment shown in Figure 8. The differences from Figure 8 are that there are two control signal lines that can input the control signal P_CLK to the signal processing circuit instead of one, and the output of the frequency divider circuit becomes the output of the selection circuit.

[0128] As described above, in the second embodiment, in a high-illumination frame, the selection circuit 234 selects the control signal P_CLK1, which is obtained by dividing the control signal P_CLK0 by two, thereby leveling the power consumption within one frame.

[0129] <Third Embodiment> The configuration example of the signal processing circuit 103 according to the third embodiment is the same as that of the second embodiment shown in Figure 10.

[0130] The selection circuit 234 selects one of several control signals P_CLK0 / 1 supplied from the signal generation unit 215 via the vertical scanning circuit unit 110 and drive lines 232 and 233, according to the comparison result notified by the comparison circuit 225. Based on the comparison result from the comparison circuit 225, the selection circuit 234 selects control signal P_CLK0 if the illuminance determination unit 223 determines that the illuminance is low, and selects control signal P_CLK1 if it determines that the illuminance is high. The selected control signal P_CLK is supplied to the photon pulse generation unit 220. The signal generation unit 215 generates control signal P_CLK0 which oscillates throughout the entire frame period, and control signal P_CLK1 which oscillates from the middle of the frame period onward.

[0131] The output formats of the counter circuit 211 and time counter circuit 222 according to the third embodiment are the same as those of the first embodiment shown in Figure 7, but there are fields with different meanings, so the differences will be explained.

[0132] The output format 701 of the counter circuit 211 includes a P_CLK flag 7012. The P_CLK flag 7012 indicates whether a control signal P_CLK0 that oscillates over the entire frame period is selected for a subsequent processing step (not shown), or whether a control signal P_CLK1 that oscillates from the middle of the frame period onward is selected. In the third embodiment, 0 indicates control signal P_CLK0 and 1 indicates control signal P_CLK1, and the P_CLK flag 7012 stores either a value of 0 or 1.

[0133] The output format 702 of the time counter circuit 222 includes a P_CLK flag 7022. The P_CLK flag 7022 indicates whether a control signal P_CLK0 that oscillates over the entire frame period is selected for a subsequent processing step (not shown), or whether a control signal P_CLK1 that oscillates from the middle of the frame period onward is selected. In the third embodiment, 0 indicates control signal P_CLK0 and 1 indicates control signal P_CLK1, and the P_CLK flag 7022 stores either a value of 0 or 1.

[0134] In the subsequent processing (not shown), the counter flags 7011 and 7021 of the received data are referenced, and if each counter flag is 0, it is recognized as the output of the counter circuit 211, and if it is 1, it is recognized as the output of the time counter circuit 222.

[0135] If the received data is the output of the counter circuit 211, the subsequent processing (not shown) further refers to the P_CLK flag 7012 and corrects the count value 7014 according to its value. If the P_CLK flag 7012 is 0, the control signal P_CLK is control signal P_CLK0 which oscillates throughout the entire frame period, and the count value 7014 is treated as the number of photons per frame without correction. If the P_CLK flag 7012 is 1, the control signal P_CLK is control signal P_CLK1 which oscillates from the middle of the frame period onward. If the oscillation of control signal P_CLK1 starts when half of the exposure period has elapsed, the count value 7014 is doubled and treated as the number of photons per frame. This is because the count period of the counter circuit 211 is halved even at the same illuminance. Note that the oscillation of control signal P_CLK1 does not have to start at half the exposure period onward, but can start from the middle of the frame period onward.

[0136] If the received data is the output of the time counter circuit 222, the subsequent processing (not shown) further refers to the P_CLK flag 7022 and calculates the number of photons per frame from the time count value 7023 according to its value. If the P_CLK flag 7022 is 0, the control signal P_CLK is the control signal P_CLK0 that oscillates for the entire duration of one frame. In that case, the number of photons per frame CNT is calculated from the saturation value S of the counter circuit 211, the count value FT from the time counter circuit 222 for one frame period, and the count value ST from the time counter circuit 222 until the saturation value S is reached, using the above (Equation 1).

[0137] On the other hand, if the P_CLK flag 7022 is 1, the control signal P_CLK is control signal P_CLK1, which oscillates from the middle of the frame period onward. In that case, assuming that the oscillation of control signal P_CLK1 starts when half of the exposure period has elapsed, the number of photons per frame CNT is calculated using the following formula. Here, S is the saturation value of the counter circuit 211, FT is the count value of the time counter circuit 222 for the frame period, and ST is the count value of the time counter circuit 222 until the saturation value S is reached. CNT=S×FT / (ST-(FT / 2))…(Formula 3)

[0138] The fact that the control signal P_CLK1 starts oscillating when half the exposure period has elapsed means that the count value (ST) calculated by the time counter circuit 222 until the saturation value S is reached includes the non-counting time (FT / 2) of the half-frame period, so that amount is subtracted. Note that the start of oscillation of the control signal P_CLK1 does not have to be limited to half the exposure period; it can start anytime after the middle of one frame period.

[0139] Figure 11 is a timing chart showing the operation of the signal processing circuit 103 according to the third embodiment.

[0140] At time T0, the first frame begins.

[0141] At time T1, a control pulse pRES is supplied from the control pulse generation unit 115 via the vertical scanning circuit unit 110 and the drive line 213, and the count values ​​of the counter circuit 211 and the time counter circuit 222 are reset. The count values ​​of the counter circuit 211 and the time counter circuit 222 are set to 0.

[0142] At time T2, the drive signal CNTEN supplied from the control pulse generation unit 115 via the vertical scanning circuit unit 110 and drive line 228 rises, and the counting of the counter circuit 211 and the time counter circuit 222, i.e., exposure begins. In the first frame, the selection circuit 234 selects the control signal P_CLK0 which oscillates for the entire duration of one frame and supplies it to the photon pulse generation unit 220.

[0143] At time T3, the count value of the counter circuit 211 reaches the saturation value, and the threshold count detection circuit 224 detects the saturation. The time counter circuit 222 is notified by the threshold count detection circuit 224 that the count value of the counter circuit 211 has reached the saturation value, and stops counting.

[0144] At time T4, the comparison circuit 225 is supplied with the control signal CTIME0 selected by the selection circuit 226. At this point, the threshold count detection circuit 224 detects that the count value of the counter circuit 211 has reached saturation. The timing at which the count value of the counter circuit 211 reaches the saturation value is earlier than the timing at which the control signal CTIME0 is input, and the illuminance determination unit 223 determines that the first frame is high illuminance. Based on the comparison result of the comparison circuit 225, the selection circuit 234 selects the control signal P_CLK1, which will be oscillated from the middle of the frame period onward, if the illuminance determination unit 223 has determined that the illuminance is high. Based on the comparison result of the comparison circuit 225, the selection circuit 226 selects the control signal CTIME1 if the illuminance determination unit 223 has determined that the illuminance is high.

[0145] At time T5, the control signal P_CLK1, which oscillates from the middle of the frame period onward, begins to oscillate.

[0146] If the illuminance determination unit 223 determines that the illuminance is high, in the next frame, the selection circuit 234 selects the control signal P_CLK1 which oscillates from the middle of the frame period onward, and the selection circuit 226 selects the control signal CTIME1. On the other hand, if the illuminance determination unit 223 determines that the illuminance is low, in the next frame, the selection circuit 234 selects the control signal P_CLK0 which oscillates throughout the entire frame period, and the selection circuit 226 selects the control signal CTIME0. This is because the count start times of the counter circuit 211 are different for control signals P_CLK0 and P_CLK1, so the timing at which the control signal CTIME, which indicates the timing at which the comparison circuit 225 performs the comparison, must also be different.

[0147] Therefore, as shown in Figure 11, the time from when the drive signal CNTEN rises to the falling edge of the control signal CTIME0 and the time from when the control signal P_CLK1 starts oscillating to the falling edge of the control signal CTIME1 are both defined as CmpT.

[0148] In the third embodiment, the comparison circuit 225 stores the comparison result in a buffer or the like, and switches the control signal P_CLK at the timing when the control pulse pRES is supplied to the selection circuit 234 at the beginning of the next frame. Therefore, the control signal P_CLK1, which actually oscillates from the middle of the frame period onwards, is used from the second frame onwards. However, it is also acceptable to configure the system to switch the control signal P_CLK in real time during the first frame.

[0149] At time T6, the drive signal CNTEN supplied from the control pulse generation unit 115 via the vertical scanning circuit unit 110 and the drive line 228 falls, ending the exposure and transitioning to readout. In other words, the period from time T2 to time T6 is the exposure period for the first frame.

[0150] At time T7, the selection circuit 212 is supplied with a control pulse pSEL from the control pulse generation unit 115 via the vertical scanning circuit unit 110 and the drive line 214. At this point, the threshold count detection circuit 224 has detected saturation of the count value of the counter circuit 211, and as a result, the output format 702 of the time counter circuit 222 selected by the selector 227 is output to the signal line 113.

[0151] At time T8, the first frame ends and the second frame begins.

[0152] At time T9, a control pulse pRES is supplied from the control pulse generation unit 115 via the vertical scanning circuit unit 110 and the drive line 213, and the count values ​​of the counter circuit 211 and the time counter circuit 222 are reset. The count values ​​of the counter circuit 211 and the time counter circuit 222 are set to 0. In addition, the illuminance determination result (high illuminance) of the first frame is set in the selection circuit 234, and the selection circuit 234 selects and outputs the control signal P_CLK1 that oscillates from the middle of the frame period onward. Then, the comparison result of the comparison circuit 225 and the detection result of the threshold count detection circuit 224 are cleared.

[0153] At time T10, the drive signal CNTEN supplied from the control pulse generation unit 115 via the vertical scanning circuit unit 110 and drive line 228 rises, and the time counter circuit 222 starts counting. However, since the illuminance determination unit 223 determined that the first frame was high illuminance, the selection circuit 234 in the second frame selects the control signal P_CLK1 which oscillates from the middle of the frame period onward and supplies it to the photon pulse generation unit 220. Therefore, the counting of the counter circuit 211, i.e., exposure, does not yet start at time T10.

[0154] At time T11, the control signal P_CLK1, which oscillates from the middle of the frame period onward, begins to oscillate. This triggers the counting of the counter circuit 211, i.e., the start of exposure.

[0155] At time T12, the comparison circuit 225 is supplied with the control signal CTIME1 selected by the selection circuit 226. At this point, the threshold count detection circuit 224 has not detected saturation of the count value of the counter circuit 211. The timing at which the count value of the counter circuit 211 reaches the saturation value is later than the timing at which the control signal CTIME1 is input, and the illuminance determination unit 223 determines that the second frame is low illuminance. Based on the comparison result of the comparison circuit 225, the selection circuit 234 selects the control signal P_CLK0 to be oscillated for the entire frame period if the illuminance determination unit 223 determines that the illuminance is low. Based on the comparison result of the comparison circuit 225, the selection circuit 226 selects the control signal CTIME0 if the illuminance determination unit 223 determines that the illuminance is low.

[0156] In other words, the relative position within the first frame of the timing at which the control signal P_CLK starts oscillating is different from the relative position within the second frame of the timing at which the control signal P_CLK starts oscillating is different. The relative position within each frame of the timing at which the control signal P_CLK starts oscillating refers to the ratio of the period from the start of each frame period to the start of the control signal P_CLK oscillating to the period of each frame. For example, even if the period from the start of the frame to the timing at which the control signal P_CLK starts oscillating is the same in both the first and second frames, the relative position will be different if the length of the first frame and the length of the second frame are different.

[0157] In the third embodiment, the comparison circuit 225 stores the comparison result in a buffer or the like, and switches the control signal P_CLK at the timing when the control pulse pRES is supplied to the selection circuit 234 at the beginning of the next frame. Therefore, the control signal P_CLK0 that actually oscillates for the entire duration of one frame is used starting from the third frame. However, it is also acceptable to configure the system to switch the control signal P_CLK in real time during the second frame.

[0158] At time T13, the count value of the counter circuit 211 reaches the saturation value, and the threshold count detection circuit 224 detects the saturation. The time counter circuit 222 is notified by the threshold count detection circuit 224 that the count value of the counter circuit 211 has reached the saturation value, and stops counting.

[0159] At time T14, the drive signal CNTEN supplied from the control pulse generation unit 115 via the vertical scanning circuit unit 110 and the drive line 228 falls, ending the exposure and transitioning to readout. In other words, the period from time T11 to time T14 becomes the exposure period for the second frame.

[0160] At time T15, the selection circuit 212 is supplied with a control pulse pSEL from the control pulse generation unit 115 via the vertical scanning circuit unit 110 and the drive line 214. At this point, the threshold count detection circuit 224 detects that the count value of the counter circuit 211 has reached saturation. As a result, the output format 702 of the time counter circuit 222 selected by the selector 227 is output to the signal line 113.

[0161] At time T16, the second frame ends and the third frame begins.

[0162] At time T17, a control pulse pRES is supplied from the control pulse generation unit 115 via the vertical scanning circuit unit 110 and the drive line 213, and the count values ​​of the counter circuit 211 and the time counter circuit 222 are reset. The count values ​​of the counter circuit 211 and the time counter circuit 222 are set to 0. In addition, the illuminance determination result (low illuminance) of the first frame is set in the selection circuit 234, and the selection circuit 234 selects and outputs the control signal P_CLK0 that oscillates for the entire duration of one frame. Then, the comparison result of the comparison circuit 225 and the detection result of the threshold count detection circuit 224 are cleared.

[0163] At time T18, the drive signal CNTEN supplied from the control pulse generation unit 115 via the vertical scanning circuit unit 110 and drive line 228 rises, and the counter circuit 211 and time counter circuit 222 begin counting, i.e., exposure begins. The illuminance determination unit 223 determines that the second frame is low illuminance, so in the third frame, the selection circuit 234 selects the control signal P_CLK0 that oscillates throughout the entire frame period and supplies it to the photon pulse generation unit 220.

[0164] At time T19, the comparison circuit 225 is supplied with the control signal CTIME0 selected by the selection circuit 226. At this point, the threshold count detection circuit 224 has not detected saturation of the count value of the counter circuit 211, and the illuminance determination unit 223 determines that the third frame is low illuminance. Based on the comparison result of the comparison circuit 225, the selection circuit 234 selects the control signal P_CLK0 to oscillate for the entire frame period, continuing from the second frame. Based on the comparison result of the comparison circuit 225, the selection circuit 226 selects the control signal CTIME0, continuing from the second frame.

[0165] At time T20, the control signal P_CLK1, which oscillates from the middle of the frame period onward, begins to oscillate.

[0166] At time T21, the drive signal CNTEN supplied from the control pulse generation unit 115 via the vertical scanning circuit unit 110 and drive line 228 falls, the counter circuit 211 and time counter circuit 222 stop counting, exposure ends and the process moves to readout. In other words, the period from time T18 to time T21 becomes the exposure period for the third frame.

[0167] At time T22, the selection circuit 212 is supplied with a control pulse pSEL from the control pulse generation unit 115 via the vertical scanning circuit unit 110 and the drive line 214. At this point, the threshold count detection circuit 224 has not detected saturation of the count value of the counter circuit 211. As a result, the output format 701 of the counter circuit 211 selected by the selector 227 is output to the signal line 113.

[0168] At time T23, the third frame ends.

[0169] Furthermore, even though the illuminance determination unit 223 determines that the first frame is high illuminance, the selection circuit 234 selects the control signal P_CLK0 that oscillates throughout the entire frame period and supplies it to the photon pulse generation unit 220. Similarly, even though the illuminance determination unit 223 determines that the second frame is low illuminance, the selection circuit 234 selects the control signal P_CLK1 that oscillates from the middle of the frame period onward and supplies it to the photon pulse generation unit 220. In the third embodiment, the comparison circuit 225 stores the comparison result in a buffer or the like, and switches the control signal P_CLK at the timing when the control pulse pRES is supplied to the frequency divider circuit 221 at the beginning of the next frame. As a result, this state occurs when switching from low illuminance to high illuminance, or from high illuminance to low illuminance. However, probabilistically, low illuminance and high illuminance frames tend to be consecutive. Also, if the control signal P_CLK is switched in real time in the middle of a frame, this state will not occur.

[0170] Figures 12(a) and (b) show conceptual diagrams of the control signals P_CLK0 and P_CLK1, and the average power consumption, within a single frame.

[0171] Figure 12(a) shows the exposure time and average power consumption when the control signal P_CLK0 is selected. The count value of the counter circuit 211 saturates in the first half of the frame period. Therefore, power consumption due to recharge current and avalanche current is concentrated in the first half of the frame period.

[0172] Figure 12(b) shows the exposure time and average power consumption when the control signal P_CLK1 is selected. The count value of the counter circuit 211 saturates in the latter half of the frame period. As a result, power consumption due to recharge current and avalanche current is concentrated in the latter half of the frame period.

[0173] If the illuminance determination unit 223 determines that the illuminance is high and all pixels select the control signal P_CLK1, the power consumption of all pixels will be concentrated in the latter half of the frame period, and power consumption cannot be equalized within a single frame. Therefore, the selection circuit 234 pre-classifies the pixels into a group that selects either the control signal P_CLK0 or the control signal P_CLK1, and a group that always uses the control signal P_CLK0. By designating 50% of the total number of pixels as the group that selects either the control signal P_CLK0 or the control signal P_CLK1 using the selection circuit 234, and the remaining 50% as the group that always uses the control signal P_CLK0, power consumption can be equalized within a single frame.

[0174] The methods for classifying pixels are diverse and not limited to any particular method, including single-row, multi-row, column-by-column, multi-column, checkerboard, and random distribution. Furthermore, the pixel classification ratio is not limited to the 50% example.

[0175] As described above, in the third embodiment, in a high-illumination frame, the selection circuit 234 selects a control signal P_CLK1 that oscillates only in the latter half of the frame period for predetermined pixels. For the remaining pixels, the control signal P_CLK0, which corresponds to the control signal P_CLK in the first embodiment, is used. This allows for a mix of pixels in the screen where power consumption is concentrated in the first half of the frame period and pixels where power consumption is concentrated in the latter half of the frame period, thereby leveling out power consumption within a single frame.

[0176] <Fourth Embodiment> In the first embodiment, the frequency division ratio of the control signal P_CLK could be switched on a per-pixel basis. In contrast, the fourth embodiment is configured to switch the frequency division ratio of the control signal P_CLK on a per-pixel basis. For the sake of simplicity, the configuration described here uses a per-4-pixel unit to switch the frequency division ratio, but it could be, for example, a per-16-pixel unit, and is not limited to a per-4-pixel unit.

[0177] Figure 13 is a simplified diagram illustrating the conceptual configuration of a signal processing circuit according to the fourth embodiment. Only the blocks necessary for explaining the fourth embodiment are shown, and the parts explained in the first embodiment are omitted.

[0178] 1031, 1032, 1033, and 1034 are signal processing circuits that share frequency divider circuit 221 and selection circuit 226, respectively. These four pixels may be used to form a Bayer array. Furthermore, the Bayer array is just one example; depending on the number of pixels, they may be grouped in units of multiple rows or multiple columns, and it is not necessary to group them uniformly in the same units across the entire screen.

[0179] The outputs of the illuminance determination units 2231, 2232, 2233, and 2234 for each pixel are supplied to the frequency divider circuit 221 and the selection circuit 226 via the AND circuit 235. If each illuminance determination unit outputs 1 when it determines high illuminance and 0 when it determines low illuminance, then if all four pixels are determined to be in high illuminance, the frequency divider circuit 221 divides the control signal P_CLK by 2, and the selection circuit 226 selects the control signal CTIME1. Each illuminance determination unit may output 0 when it determines high illuminance and 1 when it determines low illuminance, and the AND circuit 235 may be composed of other logic circuits such as an OR circuit.

[0180] 2201, 2202, 2203, and 2204 are photon pulse generation units for each pixel, and are supplied with a common control signal P_CLK that has been divided by the frequency divider circuit 221. In addition, each illuminance determination unit is supplied with a common control signal CTIME that has been selected by the selection circuit 226.

[0181] When adjacent pixels use control signals P_CLK with different division ratios, if all of those adjacent pixels saturate quickly, the difference in gradation may appear in the image as steps or flickering. In the fourth embodiment, such steps and flickering can be suppressed by using a control signal P_CLK with a common division ratio for multiple pixel units. In addition, since the division circuit 221 and the selection circuit 226 can be shared by two or more pixels, the circuit size can be reduced.

[0182] <Fifth Embodiment> In the first to fourth embodiments, the time counter circuit 222 was configured to be provided for each pixel. In contrast, in the fifth embodiment, the time counter circuit 222 is shared across multiple pixels. For the sake of simplicity, the configuration described here uses a 4-pixel unit to share the time counter circuit 222, but it could be, for example, 16 pixels, and is not limited to 4 pixels.

[0183] Figure 14 is a simplified diagram illustrating the concept of a signal processing circuit configuration example according to the fifth embodiment. Only the blocks necessary for explaining the fifth embodiment are shown, and the parts explained in the first embodiment are omitted.

[0184] Pixels 1031, 1032, 1033, and 1034 are signal processing circuits that share the time counter circuit 222. These four pixels may form a Bayer array. Furthermore, the Bayer array is just one example; the time counter circuit 222 may also be shared on a row-by-row, multi-row, column-by-column, or multi-column basis.

[0185] For example, a color filter may be omitted only from pixel 101 connected to the signal processing circuit 1031, while red, green, and blue color filters are implemented on the other three pixels. In this case, the counter circuit 2111 of the signal processing circuit 1031 connected to pixel 101, which does not have a color filter, will saturate faster than the counter circuits 2112, 2113, and 2114 of the other three pixels.

[0186] When the threshold count detection circuit 224 detects that the count value of the counter circuit 2111 has reached the saturation value, it notifies the time counter circuit 222 and the other three pixel counter circuits 2112, 2113, and 2114.

[0187] Similar to the first embodiment, the time counter circuit 222 stops counting when it is notified by the threshold count detection circuit 224 that the count value of the counter circuit 2111 has reached a saturation value within the period of one frame. The other three pixel counter circuits 2112, 2113, and 2114 also stop counting when they are notified by the threshold count detection circuit 224 that the count value of the counter circuit 2111 has reached a saturation value within the period of one frame.

[0188] In the configuration shown in the fifth embodiment, the counter circuits 2112, 2113, and 2114 connected to the three pixels having color filters will not saturate before counter circuit 2111. Therefore, the output format of the signal processing circuits 1032, 1033, and 1034 will be 701 in Figure 7. In the fifth embodiment, the configuration is such that counter circuit 2111 saturates earlier than the counter circuits 2112, 2113, and 2114 of the other three pixels, but the configuration is not limited to this. For example, the configuration may be such that if any of the counter circuits saturates, the time counter circuit 222 and the counter circuits of the other three pixels are stopped.

[0189] Furthermore, the time counter circuit 222 is, for example, a 14-bit counter, and therefore has a large circuit size. If it is implemented in the signal processing circuit 103 of each pixel, the pixel size will increase. On the other hand, in the fifth embodiment, the time counter circuit 222 can be shared by two or more pixels, so the circuit size can be reduced, thereby reducing the pixel size and increasing the resolution.

[0190] <Sixth Embodiment> Figure 15 shows an example of the configuration of the signal processing circuit 103 according to the sixth embodiment.

[0191] The signal processing circuit 103 includes a counter circuit 211, a selection circuit 212, a photon pulse generation unit 220, a frequency divider circuit 221, a check circuit 239, and a connection circuit 240.

[0192] The photon pulse generation unit 220 and the counter circuit 211 are as described in the first embodiment.

[0193] The check circuit 239 refers to the control signal P_TIME supplied from the control pulse generation unit 115, and at each timing when the control signal P_TIME is input, it refers to the count value of the counter circuit 211. If the count value of the counter circuit 211 reaches a predetermined threshold, the counter circuit 211 is stopped. In the sixth embodiment, the control signal P_TIME0 is input when 1 / 8 of the frame period has elapsed, the control signal P_TIME1 is input when 1 / 4 of the frame period has elapsed, and the control signal P_TIME2 is input when 1 / 2 of the frame period has elapsed. In the sixth embodiment, the predetermined threshold is set to the midpoint value (127) of the count value of the 8-bit counter, but the timing at which the control signal P_TIME is input and the predetermined threshold are not limited to this.

[0194] The check circuit 239 further outputs the timing at which the count value of the counter circuit 211 exceeds the threshold as 3-bit data to the frequency divider circuit 221 and the connection circuit 240. If the count value of the counter circuit 211 exceeds the threshold of 127 at the timing when the control signal P_TIME0 is input, 3b'111 is output. If the count value of the counter circuit 211 exceeds the threshold of 127 at the timing when the control signal P_TIME1 is input, 3b'110 is output. If the count value of the counter circuit 211 exceeds the threshold of 127 at the timing when the control signal P_TIME2 is input, 3b'100 is output. Also, if the count value of the counter circuit 211 does not exceed the threshold of 127 at the timing when the control signal P_TIME2 is input, 3b'000 is output.

[0195] The frequency divider circuit 221 divides the control signal P_CLK supplied from the signal generation unit 215 via the vertical scanning circuit unit 110 and drive line 232 in accordance with the 3-bit timing information notified by the check circuit 239. In the sixth embodiment, the division ratio is set to 2, but the division ratio can be designed to a value suitable for the system, and may be 4 or 8 division, or a configuration with multiple division ratios may be used. If the frequency divider circuit 221 has not divided the control signal P_CLK in the frame, when 3'b111 is notified by the check circuit 239, it determines that the frame is high-illumination and divides the control signal P_CLK by 2, and does not divide under other conditions. On the other hand, if the frequency divider circuit 221 has divided the control signal P_CLK by 2 in the frame, when 3'b111 or 3'b110 is notified by the check circuit 239, it determines that the frame is high-illumination and continues to divide the control signal P_CLK by 2. Under other conditions, the control signal P_CLK is not divided. That is, if the frequency divider circuit 221 divides the control signal P_CLK by 2, the counting speed of the counter circuit 211 becomes 1 / 2 even at the same illuminance. Therefore, high illuminance is determined not only when 1 / 8 of the frame period has elapsed, but also when 1 / 4 of the frame period has elapsed and the count value of the counter circuit 211 reaches the threshold. In the sixth embodiment, the check circuit 239 stores the check result in a buffer or the like, and the division ratio is switched when the control pulse pRES is supplied to the frequency divider circuit 221 at the beginning of the next frame. However, it is also acceptable to switch the division ratio in real time in the middle of a frame, for example.

[0196] The connecting circuit 240 connects the 3-bit timing information from the check circuit 239 and the 8-bit count value from the counter circuit 211 and supplies them to the selection circuit 212. In the sixth embodiment, since the count value of the counter circuit 211 may exceed the threshold at the timing checked by the check circuit 239, it is necessary to output the count value of the counter circuit 211 even if the count value of the counter circuit 211 exceeds the threshold.

[0197] The selection circuit 212 is as described in the first embodiment.

[0198] Figure 16 shows the output format 801 of the signal processing circuit 103 according to the sixth embodiment.

[0199] The most significant bit of the output format 801 is provided with a P_CLK flag 8011 that indicates whether or not the control signal P_CLK was divided by 2 in a subsequent processing step (not shown). In the sixth embodiment, 0 indicates no division and 1 indicates division by 2, and the P_CLK flag 8011 stores either a value of 0 or 1. The division ratio can be designed to be a value suitable for the system, and may be 4-fold or 8-fold. Furthermore, the P_CLK flag 8011 may be extended to multiple bits to accommodate multiple division ratios.

[0200] The output format 801 includes a 3-bit area 8012 for storing timing information of the check circuit 239.

[0201] The lower 8 bits of the output format 801 are provided with a region 8013 for storing the count value of the counter circuit 211.

[0202] The check circuit 239 determines that the frame is high-illumination if the timing information is 3'b111, and adds 1 to the P_CLK flag 8011. Furthermore, if the timing information is 3'b110 and the control signal P_CLK is divided by 2 in the frame, it determines that the frame is high-illumination and adds 1 to the P_CLK flag 8011. In all other cases, it determines that the frame is low-illumination and adds 0 to the P_CLK flag 8011. This forms the output format 801.

[0203] In the sixth embodiment, the check circuit 239 stores the check result in a buffer or the like, and switches the frequency division ratio when the control pulse pRES is supplied to the frequency divider circuit 221 at the beginning of the next frame. However, the configuration is not limited to this. For example, the frequency division ratio may be switched in real time in the middle of a frame. In that case, a flag indicating that the frequency division ratio was switched in the middle of a frame and information indicating the time of the switch are added to the output format 801.

[0204] The subsequent processing (not shown) calculates the number of photons per frame from the received data.

[0205] First, the P_CLK flag 8011 of the received data is referenced, and the count value 8013 is corrected according to its value. If the P_CLK flag 8011 is 0, the control signal P_CLK is not divided, and the count value 8013 is not corrected. If the P_CLK flag 8011 is 1, the control signal P_CLK is divided by 2, and the count value 8013 is doubled. When the control signal P_CLK is divided by 2, the counting speed of the counter circuit 211 becomes halved compared to the case of no division, even at the same illuminance, so the count value 8013 is doubled to correct it.

[0206] Here, the uncorrected count value 8013 when the P_CLK flag 8011 is 0 and the corrected count value 8013 when the P_CLK flag 8011 is 1 are uniformly defined as the corrected value C. In a subsequent process (not shown), the number of photons per frame is calculated from the corrected value C and the timing information 8012 of the received data.

[0207] When timing information 8012 is 3'b111, the count value of counter circuit 211 exceeds the threshold 127 after 1 / 8 of the frame period has elapsed. The number of photons per frame CNT is calculated from the correction value C using the following formula. CNT=C×8…(Formula 4)

[0208] When timing information 8012 is 3'b110, the count value of counter circuit 211 exceeds the threshold 127 when 1 / 4 of the frame period has elapsed. The number of photons per frame CNT is calculated from the correction value C using the following formula. CNT=C×4…(Formula 5)

[0209] When timing information 8012 is 3'b100, the count value of counter circuit 211 exceeds the threshold 127 when half of the frame period has elapsed. The number of photons per frame CNT is calculated from the correction value C using the following formula. CNT=C×2…(Formula 6)

[0210] When the timing information 8012 is 3'b111, the count value of the counter circuit 211 does not exceed the threshold of 127 during one frame period, and the correction value C becomes the number of photons CNT per frame.

[0211] In high-illumination frames, if the frequency divider circuit 221 divides the control signal P_CLK by 2, the counting speed of the counter circuit 211 becomes 1 / 2, even at the same illuminance. Therefore, the time it takes for the threshold count detection circuit 224 to detect saturation becomes twice as long compared to when the control signal P_CLK is not divided. On the other hand, if the frequency divider circuit 221 divides the control signal P_CLK by 2, the frequency of recharge operations becomes 1 / 2, even at the same illuminance. Compared to when the control signal P_CLK is not divided, the frequency of recharge current and avalanche current flow also becomes 1 / 2. As a result, when the frequency divider circuit 221 divides the control signal P_CLK by 2, power consumption can be leveled within one frame compared to when the control signal P_CLK is not divided.

[0212] As described above, according to the sixth embodiment, in a high-illumination frame, the frequency divider circuit 221 divides the control signal P_CLK by two, thereby leveling the power consumption within one frame.

[0213] <Seventh Embodiment> The photoelectric conversion system according to this embodiment will be described with reference to Figure 17. Figure 17 is a block diagram showing the schematic configuration of the photoelectric conversion system according to this embodiment.

[0214] The photoelectric conversion devices described in the first to sixth embodiments above are applicable to various photoelectric conversion systems. Examples of applicable photoelectric conversion systems include digital still cameras, digital camcorders, surveillance cameras, photocopiers, fax machines, mobile phones, in-vehicle cameras, and observation satellites. Camera modules, which include optical systems such as lenses and imaging devices, are also included in photoelectric conversion systems. Figure 17 shows a block diagram of a digital still camera as an example of these.

[0215] The photoelectric conversion system illustrated in Figure 17 includes an imaging device 1004, which is an example of a photoelectric conversion device, and a lens 1002 that forms an optical image of a subject onto the imaging device 1004. Furthermore, it has an aperture 1003 for varying the amount of light passing through the lens 1002, and a barrier 1001 for protecting the lens 1002. The lens 1002 and aperture 1003 are an optical system that focuses light onto the imaging device 1004. The imaging device 1004 is a photoelectric conversion device of any of the above embodiments, which converts the optical image formed by the lens 1002 into an electrical signal.

[0216] The photoelectric conversion system also includes a signal processing unit 1007, which is an image generation unit that generates an image by processing the output signal output from the imaging device 1004. The signal processing unit 1007 performs various corrections and compressions as needed and outputs the image data. The signal processing unit 1007 may be formed on the semiconductor substrate on which the imaging device 1004 is mounted, or it may be formed on a semiconductor substrate separate from the imaging device 1004.

[0217] The photoelectric conversion system further includes a memory unit 1010 for temporarily storing image data, and an external interface unit (external I / F unit) 1013 for communicating with an external computer or the like. Furthermore, the photoelectric conversion system includes a recording medium 1012 such as a semiconductor memory for recording or reading imaging data, and a recording medium control interface unit (recording medium control I / F unit) 1011 for recording or reading data from the recording medium 1012. The recording medium 1012 may be built into the photoelectric conversion system or may be detachable.

[0218] Furthermore, the photoelectric conversion system includes an overall control / calculation unit 1009 that controls various calculations and the entire digital still camera, and a timing generation unit 1008 that outputs various timing signals to the imaging device 1004 and the signal processing unit 1007. Here, the timing signals and the like may be input from an external source, and the photoelectric conversion system only needs to have at least the imaging device 1004 and the signal processing unit 1007 that processes the output signals output from the imaging device 1004.

[0219] The imaging device 1004 outputs the imaging signal to the signal processing unit 1007. The signal processing unit 1007 performs predetermined signal processing on the imaging signal output from the imaging device 1004 and outputs image data. The signal processing unit 1007 generates an image using the imaging signal.

[0220] Thus, according to this embodiment, a photoelectric conversion system can be realized by applying a photoelectric conversion device (imaging device) of any of the above embodiments.

[0221] <Eighth Embodiment> The photoelectric conversion system and mobile unit of this embodiment will be described with reference to Figure 18. Figure 18 is a diagram showing the configuration of the photoelectric conversion system and mobile unit of this embodiment.

[0222] Figure 18(a) shows an example of a photoelectric conversion system related to an in-vehicle camera. The photoelectric conversion system 1300 has an imaging device 1310. The imaging device 1310 is a photoelectric conversion device as described in any of the embodiments above. The photoelectric conversion system 1300 has an image processing unit 1312 that performs image processing on a plurality of image data acquired by the imaging device 1310, and a parallax acquisition unit 1314 that calculates parallax (phase difference of parallax images) from a plurality of image data acquired by the photoelectric conversion system 1300. The photoelectric conversion system 1300 also has a distance acquisition unit 1316 that calculates the distance to an object based on the calculated parallax, and a collision determination unit 1318 that determines whether or not there is a possibility of collision based on the calculated distance. Here, the parallax acquisition unit 1314 and the distance acquisition unit 1316 are examples of distance information acquisition means that acquire distance information to an object. That is, distance information is information related to parallax, defocus amount, distance to an object, etc. The collision determination unit 1318 may use any of this distance information to determine the possibility of a collision. The distance information acquisition means may be implemented by specially designed hardware or by a software module. It may also be implemented by an FPGA (Field Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), or a combination thereof.

[0223] The photoelectric conversion system 1300 is connected to the vehicle information acquisition device 1320 and can acquire vehicle information such as vehicle speed, yaw rate, and steering angle. The photoelectric conversion system 1300 is also connected to the control ECU 1330, which is a control unit that outputs a control signal to generate braking force on the vehicle based on the judgment result of the collision judgment unit 1318. The photoelectric conversion system 1300 is also connected to the warning device 1340, which issues a warning to the driver based on the judgment result of the collision judgment unit 1318. For example, if the collision judgment result of the collision judgment unit 1318 indicates a high probability of collision, the control ECU 1330 performs vehicle control to avoid a collision or mitigate damage by applying the brakes, releasing the accelerator, or suppressing engine output. The warning device 1340 warns the user by sounding an alarm, displaying warning information on a screen such as a car navigation system, or vibrating the seat belt or steering wheel.

[0224] In this embodiment, the photoelectric conversion system 1300 images the area around the vehicle, for example, in front of or behind it. Figure 18(b) shows the photoelectric conversion system when imaging the area in front of the vehicle (imaging range 1350). The vehicle information acquisition device 1320 sends instructions to the photoelectric conversion system 1300 or the imaging device 1310. This configuration can further improve the accuracy of distance measurement.

[0225] The above example illustrates control to prevent collisions with other vehicles, but it can also be applied to control systems that automatically follow other vehicles or automatically drive to prevent vehicles from straying from their lanes. Furthermore, the photoelectric conversion system can be applied not only to vehicles such as the vehicle itself, but also to mobile objects (mobile devices) such as ships, aircraft, or industrial robots. In addition, it can be applied not only to mobile objects but also to a wide range of devices that utilize object recognition, such as intelligent transportation systems (ITS).

[0226] <Ninth Embodiment> The photoelectric conversion system of this embodiment will be described with reference to Figure 19. Figure 19 is a block diagram showing an example configuration of a distance image sensor, which is the photoelectric conversion system of this embodiment.

[0227] As shown in Figure 19, the distance image sensor 401 is configured to include an optical system 407, a photoelectric converter 408, an image processing circuit 404, a monitor 405, and a memory 406. The distance image sensor 401 receives light (modulated light or pulsed light) that is projected from a light source device 409 toward the subject and reflected from the surface of the subject, thereby acquiring a distance image corresponding to the distance to the subject.

[0228] The optical system 407 is composed of one or more lenses and guides the image light (incident light) from the subject to the photoelectric converter 408, where it forms an image on the light-receiving surface (sensor part) of the photoelectric converter 408.

[0229] The photoelectric converter 408 is one of the photoelectric converters from each of the embodiments described above, and a distance signal indicating the distance, which is determined from the light received signal output from the photoelectric converter 408, is supplied to the image processing circuit 404.

[0230] The image processing circuit 404 performs image processing to construct a distance image based on the distance signal supplied from the photoelectric converter 408. The distance image (image data) obtained through this image processing is then supplied to the monitor 405 for display or supplied to the memory 406 for storage (recording).

[0231] With the distance image sensor 401 configured in this way, by applying the photoelectric conversion device described above, the characteristics of the pixels are improved, and for example, more accurate distance images can be acquired.

[0232] <Tenth Embodiment> The photoelectric conversion system of this embodiment will be described with reference to Figure 20. Figure 20 is a diagram showing an example of a schematic configuration of an endoscopic surgical system, which is the photoelectric conversion system of this embodiment.

[0233] Figure 20 illustrates a surgeon (physician) 1131 performing surgery on a patient 1132 on a patient bed 1133 using an endoscopic surgical system 1150. As shown in the figure, the endoscopic surgical system 1150 consists of an endoscope 1100, surgical instruments 1110, and a cart 1134 equipped with various devices for endoscopic surgery.

[0234] The endoscope 1100 consists of a barrel 1101, the tip of which is inserted into the body cavity of the patient 1132 for a predetermined length, and a camera head 1102 connected to the base end of the barrel 1101. In the illustrated example, the endoscope 1100 is shown as a so-called rigid endoscope having a rigid barrel 1101, but the endoscope 1100 may also be configured as a so-called flexible endoscope having a flexible barrel.

[0235] An opening into which an objective lens is fitted is provided at the tip of the endoscope tube 1101. A light source device 1203 is connected to the endoscope 1100, and the light generated by the light source device 1203 is guided to the tip of the endoscope tube by a light guide extending inside the endoscope tube 1101, and is irradiated through the objective lens towards the object to be observed inside the body cavity of the patient 1132. The endoscope 1100 may be a straight-viewing endoscope, an oblique-viewing endoscope, or a side-viewing endoscope.

[0236] The camera head 1102 contains an optical system and a photoelectric converter. Reflected light from the object being observed (observation light) is focused by the optical system into the photoelectric converter. The photoelectric converter converts the observation light into electrical signals, generating an electrical signal corresponding to the observation light, i.e., an image signal corresponding to the observed image. The photoelectric converter can be any of the photoelectric converters described in the embodiments described above. The image signal is transmitted as RAW data to the camera control unit (CCU) 1135.

[0237] The CCU1135 consists of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and other components, and comprehensively controls the operation of the endoscope 1100 and the display device 1136. Furthermore, the CCU1135 receives an image signal from the camera head 1102 and performs various image processing operations on that image signal, such as development processing (demosaic processing), to display the image based on that image signal.

[0238] The display device 1136 displays an image based on an image signal that has been processed by the CCU 1135, under control from the CCU 1135.

[0239] The light source device 1203 consists of a light source such as an LED (Light Emitting Diode) and supplies illumination light to the endoscope 1100 when photographing the surgical area, etc.

[0240] The input device 1137 is an input interface for the endoscopic surgical system 1150. The user can input various types of information and instructions to the endoscopic surgical system 1150 via the input device 1137.

[0241] The treatment instrument control device 1138 controls the driving of the energy treatment instrument 1112 for purposes such as tissue cauterization, incision, or blood vessel sealing.

[0242] The light source device 1203, which supplies illumination light to the endoscope 1100 when photographing the surgical area, can be composed of, for example, an LED, a laser light source, or a combination thereof. When the white light source is composed of a combination of RGB laser light sources, the output intensity and output timing of each color (each wavelength) can be controlled with high precision, so the white balance of the captured image can be adjusted in the light source device 1203. In this case, it is also possible to capture images corresponding to each of the RGB colors in time-division by irradiating the observation target with laser light from each of the RGB laser light sources in time-division and controlling the drive of the image sensor of the camera head 1102 in synchronization with the irradiation timing. According to this method, a color image can be obtained without providing a color filter on the image sensor.

[0243] Furthermore, the light source device 1203 may be controlled to change the intensity of the light it outputs at predetermined time intervals. By controlling the drive of the image sensor of the camera head 1102 in synchronization with the timing of the change in light intensity, images can be acquired in time-division order, and these images can be combined to generate high dynamic range images without so-called black crushing and white clipping.

[0244] Furthermore, the light source device 1203 may be configured to supply light in a predetermined wavelength band corresponding to special light observation. In special light observation, for example, the wavelength dependence of light absorption in body tissue is utilized. Specifically, by irradiating with narrowband light compared to the irradiation light used during normal observation (i.e., white light), predetermined tissues such as blood vessels on the surface of mucosa can be imaged with high contrast. Alternatively, in special light observation, fluorescence observation may be performed to obtain an image from fluorescence generated by irradiation with excitation light. In fluorescence observation, excitation light can be irradiated onto body tissue and fluorescence from the body tissue can be observed, or a reagent such as indocyanine green (ICG) can be injected into body tissue and excitation light corresponding to the fluorescence wavelength of the reagent can be irradiated onto the body tissue to obtain a fluorescence image. The light source device 1203 may be configured to supply narrowband light and / or excitation light corresponding to such special light observation.

[0245] <Embodiment 11> The photoelectric conversion system of this embodiment will be described using Figures 21(a) and (b). Figure 21(a) illustrates the eyeglasses 1600 (smart glasses), which are the photoelectric conversion system of this embodiment. The eyeglasses 1600 have a photoelectric conversion device 1602. The photoelectric conversion device 1602 is the photoelectric conversion device described in each of the embodiments described above. In addition, a display device including a light-emitting device such as an OLED or LED may be provided on the back side of the lens 1601. There may be one or more photoelectric conversion devices 1602. In addition, multiple types of photoelectric conversion devices may be used in combination. The arrangement position of the photoelectric conversion device 1602 is not limited to that shown in Figure 21(a).

[0246] The eyeglasses 1600 further include a control device 1603. The control device 1603 functions as a power source that supplies power to the photoelectric converter 1602 and the display device. The control device 1603 also controls the operation of the photoelectric converter 1602 and the display device. The lens 1601 has an optical system formed therein for focusing light onto the photoelectric converter 1602.

[0247] Figure 21(b) illustrates a pair of glasses 1610 (smart glasses) relating to one application example. The glasses 1610 have a control device 1612, which is equipped with a photoelectric converter equivalent to a photoelectric converter 1602 and a display device. The lens 1611 has an optical system formed therein for projecting light emitted from the photoelectric converter in the control device 1612 and from the display device, and an image is projected onto the lens 1611. The control device 1612 functions as a power supply that provides power to the photoelectric converter and the display device, and also controls the operation of the photoelectric converter and the display device. The control device may have a gaze detection unit that detects the wearer's gaze. Gaze detection may use infrared light. The infrared light emitter emits infrared light towards the user's eyeball that is fixated on the displayed image. An imaging unit having a light-receiving element detects the reflected light from the eyeball of the emitted infrared light, thereby obtaining an image of the eyeball. By having a reduction means that reduces the light from the infrared light emitter to the display unit in planar view, the deterioration of image quality is reduced.

[0248] The user's gaze towards the displayed image is detected from an image of the eyeball obtained by imaging with infrared light. Any known method can be applied to gaze detection using an image of the eyeball. For example, a gaze detection method based on the Purkinje image obtained by the reflection of the irradiated light from the cornea can be used.

[0249] More specifically, gaze detection processing is performed based on the pupil-corneal reflection method. Using the pupil-corneal reflection method, a gaze vector representing the orientation (rotation angle) of the eyeball is calculated based on the pupil image and Purkinje image contained in the captured image of the eyeball, thereby detecting the user's gaze.

[0250] The display device of this embodiment includes a photoelectric converter having a light-receiving element, and may control the display image of the display device based on the user's gaze information from the photoelectric converter.

[0251] Specifically, the display device determines a first field of view that the user is fixated on, and a second field of view other than the first field of view, based on gaze information. The first and second field of view may be determined by the control device of the display device, or they may be determined by an external control device and received. Within the display area of ​​the display device, the display resolution of the first field of view may be controlled to be higher than that of the second field of view. In other words, the resolution of the second field of view may be lower than that of the first field of view.

[0252] Furthermore, the display area may have a first display area and a second display area different from the first display area, and a higher priority area may be determined from the first and second display areas based on line-of-sight information. The first and second field-of-sight areas may be determined by the control device of the display device, or they may be determined by an external control device and received. The resolution of the higher priority area may be controlled to be higher than the resolution of the areas other than the higher priority area. In other words, the resolution of areas with relatively lower priority may be set lower.

[0253] AI may be used to determine the first field of view area and the areas with higher priority. The AI ​​may be a model configured to estimate the angle of gaze and the distance to the target object at the end of the line of sight from the image of the eye, using the image of the eye and the direction the eye was actually looking in that image as training data. The AI ​​program may be installed in the display device, the photoelectric converter, or an external device. If installed in an external device, it will be transmitted to the display device via communication.

[0254] When display control is based on visual detection, this method is preferably applicable to smart glasses that further include a photoelectric converter for capturing images of the surrounding environment. The smart glasses can display the captured external information in real time.

[0255] [Modified Embodiment] The present invention is not limited to the embodiments described above and can be modified in various ways.

[0256] For example, examples in which some configurations of one embodiment are added to other embodiments, or in which some configurations of other embodiments are replaced, are also included as embodiments of the present invention.

[0257] Furthermore, the photoelectric conversion systems shown in the seventh and eighth embodiments above are merely examples of photoelectric conversion systems to which the photoelectric conversion device can be applied, and the photoelectric conversion systems to which the photoelectric conversion device of the present invention can be applied are not limited to the configurations shown in Figures 15 to 16. The same applies to the ToF system shown in the ninth embodiment, the endoscope shown in the tenth embodiment, and the smart glasses shown in the eleventh embodiment.

[0258] It should be noted that the above embodiments are merely examples of how the present invention can be implemented, and the technical scope of the present invention should not be interpreted as being limited by them. In other words, the present invention can be implemented in various forms without departing from its technical concept or its main features.

[0259] Furthermore, this disclosure includes the following components.

[0260] (Configuration 1) A photoelectric conversion device comprising a plurality of pixels including a photodiode that performs avalanche multiplication, and a signal processing unit that generates a signal based on photons detected by the photodiode, wherein the signal processing unit comprises a control circuit, a counter, and an illuminance determination unit, the control circuit is connected to the photodiode and a generation circuit that generates a pulse signal, and controls one node of the photodiode between a first state in which it is not electrically connected to a predetermined potential and a second state in which it is electrically connected to the predetermined potential, the counter counts the number of periods in which avalanche multiplication occurs among a plurality of periods in the first state, the illuminance determination unit determines whether the count value of the counter has reached a threshold, and the signal processing unit has means for setting the frequency of the pulse signal according to the result of the determination by the illuminance determination unit.

[0261] (Configuration 2) A photoelectric conversion device comprising a plurality of pixels including a photodiode that performs avalanche multiplication, and a signal processing unit that generates a signal based on photons detected by the photodiode, wherein the signal processing unit comprises a control circuit, a counter, and an illuminance determination unit, the control circuit is connected to the photodiode and a generation circuit that generates a pulse signal, and controls one node of the photodiode between a first state in which it is not electrically connected to a predetermined potential and a second state in which it is electrically connected to the predetermined potential, the counter counts the number of periods in which avalanche multiplication occurs among a plurality of periods in the first state, the illuminance determination unit determines whether the count value of the counter has reached a threshold, and the signal processing unit has means for setting the timing at which the pulse signal starts to oscillate according to the result of the determination by the illuminance determination unit.

[0262] (Configuration 3) The photoelectric conversion device according to Configuration 1 or Configuration 2, characterized in that the signal processing unit has an acquisition unit for acquiring exposure time.

[0263] (Configuration 4) The photoelectric conversion device according to Configuration 3, characterized in that the acquisition unit is a time counter for measuring exposure time.

[0264] (Configuration 5) The photoelectric conversion device according to Configuration 3, characterized in that the acquisition unit acquires timing information by referring to the determination result of the illuminance determination unit.

[0265] (Configuration 6) The photoelectric conversion device according to Configuration 3, characterized in that the acquisition unit acquires a time code indicating the exposure time.

[0266] (Configuration 7) The photoelectric conversion device according to Configuration 1 or 2, characterized in that the means is a selection circuit that selects one of the plurality of pulse signals generated by the generation circuit.

[0267] (Configuration 8) The photoelectric converter according to Configuration 7, characterized in that, in the first frame, the relative position within the first frame of the timing at which the oscillation of the pulse signal begins is different from the relative position within the second frame of the timing at which the oscillation of the pulse signal begins in the second frame.

[0268] (Configuration 9) The photoelectric converter according to Configuration 7, wherein the plurality of pixels are arranged in multiple rows and multiple columns to form a pixel array having a first pixel group and a second pixel group, and the selection circuit sets the timing for starting the oscillation of the pulse signal supplied to the control circuit that controls the pixels included in the first pixel group, and the timing for starting the oscillation of the pulse signal supplied to the control circuit that controls the pixels included in the second pixel group.

[0269] (Configuration 10) The photoelectric conversion device according to Configuration 9, characterized in that the relative position within a frame at the timing when the oscillation of the pulse signal starts in the first pixel group is different from the relative position within the frame at the timing when the oscillation of the pulse signal starts in the second pixel group.

[0270] (Configuration 11) The photoelectric conversion apparatus according to Configuration 1, wherein the means is a circuit that frequency-divides the pulse signal generated by the generation circuit.

[0271] (Configuration 12) The photoelectric conversion apparatus according to any one of Configurations 1 to 11, wherein the counter stops counting when a count value reaches the threshold value.

[0272] (Configuration 13) The photoelectric conversion apparatus according to any one of Configurations 1 to 12, wherein the threshold value is a countable saturation value of the counter.

[0273] (Configuration 14) The photoelectric conversion apparatus according to any one of Configurations 1 to 13, wherein the counter is shared by two or more pixels included in the plurality of pixels.

[0274] (Configuration 15) The photoelectric conversion apparatus according to any one of Configurations 1 to 14, wherein the means is shared by two or more pixels included in the plurality of pixels, and the common pulse signal is selected for the two or more pixels.

[0275] (Configuration 16) The photoelectric conversion apparatus according to any one of Configurations 1 to 15, wherein the means changes the set pulse signal from a high-frequency first pulse signal to a low-frequency second pulse signal.

[0276] (Configuration 17) A photoelectric conversion system comprising the photoelectric conversion apparatus according to any one of Configurations 1 to 16, the photoelectric conversion system comprising: a light emitting unit that emits light to be detected by the photoelectric conversion apparatus; and a calculation unit that calculates a distance using a digital signal held in the photoelectric conversion apparatus.

[0277] (Configuration 18) A moving body comprising: the photoelectric conversion apparatus according to any one of Configurations 1 to 16; an acquisition unit that acquires distance information to an object based on a signal from the photoelectric conversion apparatus; and a control unit that controls the moving body based on the distance information. [Explanation of Symbols]

[0278] 102 Photoelectric conversion element 202 Control Circuit 211 Counter Circuit 220 Photon pulse generation unit 223 Illuminance judgment section

Claims

1. A photoelectric converter comprising a plurality of pixels including a photodiodes that perform avalanche multiplication, and a signal processing unit that generates a signal based on photons detected by the photodiodes, The signal processing unit comprises a control circuit, a counter, and an illuminance determination unit. The control circuit is connected to the photodiode and a generation circuit that generates a pulse signal, and controls the photodiode between a first state in which one node is not electrically connected to a predetermined potential and a second state in which one node is electrically connected to the predetermined potential, The counter counts the number of periods in which avalanche multiplication occurred among the multiple periods in the first state, The illuminance determination unit determines whether the count value of the counter has reached a threshold, The photoelectric converter is characterized in that the signal processing unit has means for setting the frequency of the pulse signal according to the result of the determination by the illuminance determination unit.

2. A photoelectric converter comprising a plurality of pixels including a photodiodes that perform avalanche multiplication, and a signal processing unit that generates a signal based on photons detected by the photodiodes, The signal processing unit comprises a control circuit, a counter, and an illuminance determination unit. The control circuit is connected to the photodiode and a generation circuit that generates a pulse signal, and controls the photodiode between a first state in which one node is not electrically connected to a predetermined potential and a second state in which one node is electrically connected to the predetermined potential, The counter counts the number of periods in which avalanche multiplication occurred among the multiple periods in the first state, The illuminance determination unit determines whether the count value of the counter has reached a threshold, The photoelectric converter is characterized in that the signal processing unit has means for setting the timing at which the pulse signal starts oscillating according to the result of the determination by the illuminance determination unit.

3. The photoelectric conversion apparatus according to claim 1 or 2, characterized in that the signal processing unit has an acquisition unit for acquiring exposure time.

4. The photoelectric conversion apparatus according to claim 3, characterized in that the acquisition unit is a time counter for measuring exposure time.

5. The photoelectric conversion device according to claim 3, characterized in that the acquisition unit acquires timing information that refers to the determination result of the illuminance determination unit.

6. The photoelectric conversion apparatus according to claim 3, characterized in that the acquisition unit acquires a time code indicating the exposure time.

7. The photoelectric conversion device according to claim 1, characterized in that the means is a selection circuit that selects one of the plurality of pulse signals generated by the generation circuit.

8. The photoelectric conversion device according to claim 2, characterized in that the means is a selection circuit that selects one of the plurality of pulse signals generated by the generation circuit.

9. The photoelectric converter according to claim 8, characterized in that, in the first frame, the relative position within the first frame of the timing at which the oscillation of the pulse signal begins is different from the relative position within the second frame of the timing at which the oscillation of the pulse signal begins in the second frame.

10. The plurality of pixels are arranged in multiple rows and multiple columns, forming a pixel array having a first pixel group and a second pixel group. The photoelectric converter according to claim 8, characterized in that the selection circuit sets the timing for starting the oscillation of the pulse signal supplied to the control circuit that controls the pixels included in the first pixel group, and the timing for starting the oscillation of the pulse signal supplied to the control circuit that controls the pixels included in the second pixel group.

11. The photoelectric converter according to claim 10, characterized in that the relative position within a frame at the timing when the oscillation of the pulse signal starts in the first pixel group is different from the relative position within the frame at the timing when the oscillation of the pulse signal starts in the second pixel group.

12. The photoelectric conversion device according to claim 1, characterized in that the means is a circuit that divides the frequency of the pulse signal generated by the generation circuit.

13. The photoelectric conversion device according to claim 1 or 2, characterized in that the counter stops counting when the count value reaches the threshold.

14. The photoelectric conversion apparatus according to claim 1 or 2, characterized in that the threshold is a countable saturation value of the counter.

15. The photoelectric conversion device according to claim 1 or 2, characterized in that the counter is shared by two or more pixels included in the plurality of pixels.

16. The photoelectric conversion device according to claim 1 or 2, characterized in that the means is shared by two or more pixels included in the plurality of pixels, and a common pulse signal is selected for the two or more pixels.

17. The photoelectric conversion device according to claim 1 or 2, characterized in that the means changes the set pulse signal from a high-frequency first pulse signal to a low-frequency second pulse signal.

18. The photoelectric conversion device according to claim 1 or 2, characterized in that in each of the plurality of periods in the first state, the number of avalanche multiplications occurring in the photodiode is only once.

19. In each of the plurality of periods in the first state, avalanche multiplication occurs in the photodiode, causing the potential of one node to transition from a first level to a second level. The photoelectric conversion device according to claim 1 or 2, characterized in that the control circuit causes the one node to transition to the second state, thereby causing the potential of the one node to transition from the second level to the first level.

20. A photoelectric conversion system having the photoelectric conversion device described in claim 1 or claim 2, A light-emitting unit that emits light detected by the aforementioned photoelectric conversion device, A photoelectric conversion system characterized by having a calculation means for calculating distance using a digital signal held in the photoelectric conversion device.

21. It is a mobile object, A photoelectric conversion device according to claim 1 or claim 2, An acquisition means for acquiring distance information to an object based on a signal from the aforementioned photoelectric converter, A mobile body characterized by having control means for controlling the mobile body based on the distance information.

Images