Photoelectric conversion device and photoelectric conversion system

Abstract

To provide a photoelectric conversion device capable of widening a dynamic range and securing a function of low power consumption even under an environment of high illuminance.SOLUTION: A photoelectric conversion device comprises a photodiode which performs avalanche amplification, a recharge circuit, and a control signal generation circuit which generates a control signal for controlling the recharge circuit. The photoelectric conversion device also comprises a count pulse generation circuit for generating a pulse signal from an output signal of the photodiode and a counter for counting the pulse signal outputted from the count pulse generation circuit. The control signal generation circuit is capable of outputting the control signal having a first cycle and a control signal having a second cycle, which is a cycle longer than the first cycle, on the basis of a clock signal and a mask signal. In a case where a count value of the counter reaches a threshold, an output signal of the control signal generation circuit can be switched from the control signal having the first cycle to the control signal having the second cycle.SELECTED DRAWING: Figure 9

Description

[Technical field]

[0001] The present invention relates to a photoelectric conversion device and a photoelectric conversion system. [Background technology]

[0002] There is known a photodetector using an avalanche photodiode (APD) that can detect weak light at the single photon level by utilizing avalanche (electron avalanche) multiplication. The APD forms a high electric field region (avalanche multiplication section) by a first semiconductor region of a first conductivity type with the same polarity as the signal charge, and a second semiconductor region of a second conductivity type with a different polarity from the signal charge.

[0003] Patent Document 1 suppresses power consumption by controlling a MOS transistor that functions as a passive quench circuit. Specifically, at the start of an exposure period, the MOS transistor is turned on to set a period in which the transistor is always in operation (idle period 0%). Next, when the counter value reaches a threshold value N1, a period is set in which the MOS transistor is turned off, and the operation period is halved (idle period 50%). Next, when the counter value reaches a threshold value N2, the operation period is reduced to 1 / 4 (idle period 75%). With this configuration, the number of times avalanche multiplication is performed is reduced in the case of high illuminance. This makes it possible to reduce the power consumption of the pixel. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2020-123846 A Summary of the Invention [Problem to be solved by the invention]

[0005] By using the configuration of Patent Document 1, it is possible to reduce power consumption. However, since Patent Document 1 uses a passive quench circuit, in a high-illuminance environment, the relationship between the illuminance and the count value may be reversed, and if the illuminance is too high, no signal may be generated. Therefore, an appropriate count value cannot be obtained when the illuminance is high, and the dynamic range becomes narrow.

[0006] Therefore, an object of the present invention is to provide a photoelectric conversion device that can widen the dynamic range as compared with Patent Document 1 even in a high-illuminance environment and ensures a low power consumption function. [Means for solving the problem]

[0007] A photoelectric conversion device according to the present invention comprises a photodiode that performs avalanche multiplication, a recharge circuit that switches between a first state in which the photodiode is electrically connected to a power supply and a second state in which the photodiode is not electrically connected to the power supply, and a control signal generation circuit that generates a signal to control the recharge circuit, a count pulse generation circuit that generates a pulse signal from an output signal from the photodiode, and a counter that counts the pulse signal output from the count pulse generation circuit, wherein the control signal generation circuit is configured to be able to output the control signal having a first period and the control signal having a second period that is longer than the first period, based on a clock signal and a mask signal, and wherein when the count value of the counter reaches a threshold value, the output signal of the control signal generation circuit is configured to be switched from the control signal having the first period to the control signal having the second period. Effect of the Invention

[0008] According to the present invention, it is possible to provide a photoelectric conversion device that can widen the dynamic range compared to Patent Document 1 even in a high-illuminance environment and ensures low power consumption. [Brief description of the drawings]

[0009] [Figure 1] 1 is a diagram showing a configuration of a photoelectric conversion device; [Diagram 2] Sensor board layout example [Diagram 3] Circuit board layout example [Figure 4] Block diagram including equivalent circuit of photoelectric conversion element [Diagram 5] A diagram showing the relationship between APD operation and output signal [Figure 6] Block diagrams and equivalent circuit diagrams of pixel circuits in each embodiment [Figure 7] FIG. 13 is a diagram showing the relationship between illuminance and count value in each mode. [Figure 8] A diagram explaining the issues with the clock recharge driving circuit [Figure 9] FIG. 1 is a block diagram showing a configuration of a photoelectric conversion device according to a first embodiment. [Figure 10] FIG. 1 is a detailed diagram showing a configuration of a photoelectric conversion device according to a first embodiment; [Figure 11] 1 is a timing chart for explaining the operation of the photoelectric conversion device according to the first embodiment; [Figure 12] FIG. 1 is a diagram illustrating a correction concept of a photoelectric conversion device according to a first embodiment. [Figure 13] 11 is a timing chart for explaining the operation of the photoelectric conversion device according to the second embodiment; [Figure 14] FIG. 13 is a block diagram showing the configuration of a photoelectric conversion device according to a third embodiment. [Figure 15] Block diagram of a photoelectric conversion system according to a fourth embodiment. [Figure 16] Block diagram of a photoelectric conversion system according to a fifth embodiment. [Figure 17] Block diagram of a photoelectric conversion system according to a sixth embodiment. [Figure 18] Block diagram of a photoelectric conversion system according to a seventh embodiment. [Figure 19] FIG. 13 is a diagram showing a specific example of a photoelectric conversion system according to an eighth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] The embodiments described below are intended to embody the technical ideas of the present invention, and are not intended to limit the present invention. The size and positional relationship of the components shown in each drawing may be exaggerated to clarify the description. In the following description, the same configurations may be given the same numbers and the description may be omitted. Furthermore, the configurations described in each embodiment may be mutually substituted or combined with the configurations described in other embodiments, as long as there is no technical problem.

[0011] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. In the following description, terms indicating specific directions or positions (for example, "upper", "lower", "right", "left" and other terms including these terms) will be used as necessary. The use of these terms is for the purpose of facilitating understanding of the invention with reference to the drawings, and the technical scope of the present invention is not limited by the meaning of these terms.

[0012] In the following description, connections between elements of a circuit may be mentioned. In this case, unless otherwise specified, the elements of interest are treated as being connected to each other even if there is another element between them.

[0013] In the following description, the anode of the avalanche photodiode (APD) is set to a fixed potential, and a signal is taken from the cathode side. Therefore, the first conductive type semiconductor region having charges of the same polarity as the signal charge as the majority carrier is an N-type semiconductor region, and the second conductive type semiconductor region having charges of a different polarity from the signal charge as the majority carrier is a P-type semiconductor region. Note that the cathode of the APD may be set to a fixed potential, and the signal may be taken from the anode side. In this case, the first conductive type semiconductor region having charges of the same polarity as the signal charge as the majority carrier is a P-type semiconductor region, and the second conductive type semiconductor region having charges of a different polarity from the signal charge as the majority carrier is an N-type semiconductor region. In the following, a case where one node of the APD is set to a fixed potential will be described, but the potentials of both nodes may fluctuate.

[0014] FIG. 1 is a diagram showing the configuration of a stacked photoelectric conversion device 100. The photoelectric conversion device 100 is configured by stacking and electrically connecting two substrates, a sensor substrate 11 and a circuit substrate 21. The sensor substrate 11 has a first semiconductor layer having a photoelectric conversion element 102 described later, and a first wiring structure. The circuit substrate 21 has a second semiconductor layer having circuits such as a signal processing portion 103 described later, and a second wiring structure. The photoelectric conversion device 100 is configured by stacking the second semiconductor layer, the second wiring structure, the first wiring structure, and the first semiconductor layer in this order. The photoelectric conversion device described in each embodiment is a back-illuminated photoelectric conversion device in which light is incident from the second surface and a circuit substrate is disposed on the first surface.

[0015] In the following, the sensor substrate 11 and the circuit substrate 21 will be described as diced chips, but are not limited to chips. For example, each substrate may be a wafer. Also, each substrate may be stacked in a wafer state and then diced, or each chip may be stacked and bonded after being chipped from the wafer state.

[0016] A pixel region 12 is disposed on the sensor substrate 11, and a circuit region 22 that processes signals detected in the pixel region 12 is disposed on the circuit substrate 21.

[0017] 2 is a diagram showing an example of the arrangement of the sensor substrate 11. Pixels 101, each having a photoelectric conversion element 102 including an APD, are arranged in a two-dimensional array in a plan view to form a pixel region 12.

[0018] The pixel 101 is typically a pixel for forming an image, but may additionally be provided with a Time of Flight (TOF) function.

[0019] 3 is a configuration diagram of the circuit board 21. It has a signal processing unit 103 that processes electric charges photoelectrically converted by the photoelectric conversion element 102 in FIG. 2, a readout circuit 112, a control pulse generating unit 115, a horizontal scanning circuit unit 111, a signal line 113, and a vertical scanning circuit unit 110.

[0020] The photoelectric conversion element 102 in FIG. 2 and the signal processing unit 103 in FIG. 3 are electrically connected via connection wiring provided for each pixel.

[0021] The vertical scanning circuit section 110 receives a control pulse supplied from a control pulse generating section 115 and supplies the control pulse to each pixel. The vertical scanning circuit section 110 uses logic circuits such as a shift register and an address decoder.

[0022] The signal output from the photoelectric conversion element 102 of the pixel is processed by a signal processing unit 103. The signal processing unit 103 is provided with a counter, a memory, etc., and digital values ​​are written and held in the memory.

[0023] The horizontal scanning circuit unit 111 inputs a control pulse for sequentially selecting each column to the signal processing unit 103 in order to read out the signal from the memory of each pixel in which the digital signal is held.

[0024] A signal is output to the signal line 113 from the signal processing unit 103 of the pixel selected by the vertical scanning circuit unit 110 for the selected column.

[0025] The signal output to the signal line 113 is output via an output circuit 114 to a recording unit or a signal processing unit outside the photoelectric conversion device 100 .

[0026] 2, the photoelectric conversion elements in the pixel region may be arranged one-dimensionally. The function of the signal processing unit does not necessarily need to be provided for each of the photoelectric conversion elements, and for example, one signal processing unit may be shared by a plurality of photoelectric conversion elements to perform signal processing sequentially.

[0027] 2 and 3, a plurality of signal processing units 103 are arranged in a region overlapping the pixel region 12 in a planar view. A vertical scanning circuit unit 110, a horizontal scanning circuit unit 111, a readout circuit 112, an output circuit 114, and a control pulse generating unit 115 are arranged so as to overlap between an end of the sensor substrate 11 and an end of the pixel region 12 in a planar view. In other words, the sensor substrate 11 has the pixel region 12 and a non-pixel region arranged around the pixel region 12. A vertical scanning circuit unit 110, a horizontal scanning circuit unit 111, a readout circuit 112, an output circuit 114, and a control pulse generating unit 115 are arranged in a region overlapping the non-pixel region in a planar view.

[0028] Fig. 4 is an example of a block diagram including the equivalent circuits of Fig. 2 and Fig. 3. Fig. 4 shows a block diagram of a photoelectric conversion device having a general APD.

[0029] In FIG. 4, a photoelectric conversion element 102 having an APD 201 is provided on a sensor substrate 11, and other members are provided on a circuit substrate 21.

[0030] The APD 201 generates charge pairs according to the incident light by photoelectric conversion. A voltage VL (first voltage) is supplied to the anode of the APD 201. A voltage VH (second voltage) higher than the voltage VL supplied to the anode is supplied to the cathode of the APD 201. A reverse bias voltage that causes the APD 201 to perform avalanche multiplication is supplied to the anode and cathode. With such a voltage supplied, the charges generated by the incident light undergo avalanche multiplication, generating an avalanche current.

[0031] When a reverse bias voltage is supplied, there are two modes: Geiger mode, in which the potential difference between the anode and cathode is greater than the breakdown voltage, and linear mode, in which the potential difference between the anode and cathode is close to or less than the breakdown voltage.

[0032] An APD operated in Geiger mode is called a SPAD (Single Photon Avalanche Diode). For example, the voltage VL (first voltage) is −30 V, and the voltage VH (second voltage) is 3 V. The APD 201 may be operated in either the linear mode or the Geiger mode.

[0033] The quench element 202 is connected to a power supply that supplies a voltage VH and the APD 201. The quench element 202 functions as a load circuit (quench circuit) during signal multiplication by avalanche multiplication, and has a function of suppressing avalanche multiplication by suppressing the voltage supplied to the APD 201 (quench operation). The quench element 202 also has a function of returning the voltage supplied to the APD 201 to the voltage VH by flowing a current equivalent to the voltage drop caused by the quench operation (recharge operation).

[0034] The signal processing unit 103 has a waveform shaping unit 210, a counter 211, and a selection circuit 212. In this specification, it is sufficient that the signal processing unit 103 has any one of the waveform shaping unit 210, the counter 211, and the selection circuit 212.

[0035] The waveform shaping unit 210 shapes the potential change of the cathode of the APD 201 obtained when a photon is detected, and outputs a pulse signal. For example, an inverter circuit is used as the waveform shaping unit 210. Although an example in which one inverter is used as the waveform shaping unit 210 is shown in Fig. 4, a circuit in which a plurality of inverters are connected in series may be used, or another circuit having a waveform shaping effect may be used.

[0036] The counter 211 counts the number (number of times) of pulse signals output from the waveform shaping unit 210 and holds the count value. When a control pulse pRES is supplied via the drive line 213, the signal held in the counter 211 is reset.

[0037] A control pulse pSEL is supplied to the selection circuit 212 from the vertical scanning circuit unit 110 in Fig. 3 via a drive line 214 (not shown in Fig. 3) in Fig. 4, and switches between electrical connection and non-connection between the counter 211 and the signal line 113. The selection circuit 212 includes, for example, a buffer circuit for outputting a signal.

[0038] The electrical connection may be switched by disposing a switch such as a transistor between the quench element 202 and the APD 201 or between the photoelectric conversion element 102 and the signal processing unit 103. Similarly, the supply of the voltage VH or the voltage VL to the photoelectric conversion element 102 may be electrically switched using a switch such as a transistor.

[0039] FIG. 5 is a diagram showing a schematic diagram of the relationship between the operation of an APD and an output signal.

[0040] Fig. 5(a) is a diagram excerpting the APD 201, quench element 202, and waveform shaping unit 210 of Fig. 4. Here, the input side of the waveform shaping unit 210 is designated as VC, and the output side is designated as VO. Fig. 5(b) shows the voltage of VC in Fig. 5(a), and Fig. 5(c) shows the signal of VO in Fig. 5(a).

[0041] Between time t0 and time t1, a potential difference of VH-VL is applied to the APD 201 in FIG. 5(a). When a photon is incident on the APD 201 at time t1, avalanche multiplication occurs in the APD 201, an avalanche multiplication current flows through the quench element 202, and the voltage of VC drops. When the voltage drop amount becomes larger and the potential difference applied to the APD 201 becomes smaller, the avalanche multiplication of the APD 201 stops as shown at time t2, and the voltage level of VC does not drop more than a certain value. After that, between time t2 and time t3, a current that compensates for the voltage drop from the voltage VL flows through VC, and at time t3, VC is stabilized to the original potential level. At this time, the part of the output waveform of VC that exceeds a certain threshold is shaped by the waveform shaping unit 210 and output as a signal at VO.

[0042] The arrangement of the signal lines 113, the readout circuits 112, and the output circuits 114 is not limited to that shown in Fig. 3. For example, the signal lines 113 may be arranged to extend in the row direction, and the readout circuits 112 may be arranged at the ends of the signal lines 113.

[0043] (First embodiment) (Passive recharge circuit) FIG. 6(a) is a block diagram and an equivalent circuit of a passive recharge pixel circuit as a comparative example. The gate of the transistor that is the quench element 202 connected to the cathode side of the APD 201 and the output of the logic circuit 221 (OR circuit) are the same node. The logic circuit 221 is configured so that the signals ENB and STOP can be input. The signal ENB is at an L level (first level) during the exposure period T, and at an H level (second level) during the other periods. Note that, although the L level is defined as the first level and the H level is defined as the second level here, if the conductivity type of the MOS transistor is different, the relationship will be reversed, so the H level may also be referred to as the first level and the L level as the second level.

[0044] When the signal STOP is at L level and the signal ENB is at L level, the transistor of the quench element 202 turns on and enters a recharge state, and the APD 201 enters a standby state in which avalanche multiplication is possible after a predetermined period. In the case of other signal patterns, the quench element 202 turns off and the APD 201 enters a non-standby state in which avalanche multiplication is not possible. The waveform shaping unit 210 is composed of an inverter, and the counter 211 is provided with an 11-bit flip-flop. For example, when the counter 211 reaches a maximum count value of 2047, the signal STOP transitions from L level to H level, and the transistor of the quench element 202 turns off. As a result, even if the signal ENB becomes L level, the quench element 202 continues to be off, and the non-standby state of the APD 201 is maintained.

[0045] The right diagram in Fig. 7(a) shows the VC voltage and VO signal in Fig. 6(a). In the case of low illuminance, the interval between photon incidence is long, so there is enough time for the VC voltage to be recharged and become high after the photon incidence, after the voltage becomes low. In the case of the right diagram in Fig. 7(a), three pulses can be counted corresponding to three photons. On the other hand, in the case of high illuminance, the interval between photon incidence is short, so the VC voltage remains low and does not return to a high state, so it takes time to cross the decision threshold from below to above. In the case of the right diagram in Fig. 7(a), it is shown that only three pulses are counted, even though about 20 photons are incident. In other words, photon count leakage occurs.

[0046] As a result, in the case of the passive recharge configuration shown in FIG. 6(a), a situation may occur in which the same count value is obtained at low and high illuminance, as shown in FIG. 7(a). Also, if photon incidence occurs more frequently, the voltage of VC remains low and does not exceed the decision threshold from below. In this case, the voltage of VO remains high and no signal is generated. That is, in the case of the passive recharge configuration shown in FIG. 6(a), the appropriate count value cannot be obtained at high illuminance, resulting in a narrow dynamic range.

[0047] (Clock recharge circuit) 6(b) is a block diagram and an equivalent circuit diagram of a pixel circuit that performs clock recharge driving. A signal Pctrl and a signal STOP can be input to the input of a logic circuit 221 (OR circuit). The signal Pctrl is a clock signal having Nc pulse signals during the exposure period T.

[0048] In Fig. 6(b), when the signal STOP is at L level and the signal Pctrl is at L level, the transistor 202 turns on and recharges. For other signal patterns, the transistor 202 turns off. That is, the transistor functions as a recharge circuit that switches between a first state in which the APD and the power supply are electrically connected and a second state in which the APD and the power supply are not electrically connected, and charges the APD each time. The signal Pctrl serves as a control signal that controls the recharge circuit.

[0049] For example, when the signal Pctrl is at H level, the transistor is off after recharging, and thus the avalanche multiplication is possible in the standby state. In other words, a circuit (transistor) is provided between the APD 201 and a power supply (voltage VH), and the circuit controls switching between a first state in which the APD 201 and the power supply are electrically connected, and a second state in which the APD 201 and the power supply are not electrically connected. In this case, the first state is the recharge state, and the second state is the standby state.

[0050] A logic circuit 222 (an AND circuit with the logic of one input inverted) is provided between the cathode of the APD 201 and the counter 211, and is configured to be able to receive the signals Pctrl and VC. The logic circuit 222 receives the inverted output of VC, and VC transitions from H level to L level in response to the incidence of a photon. In this case, when the signal Pctrl is at H level, the output of the logic circuit 222 is at H level. Thereafter, when the signal Pctrl transitions from H level to L level, the output of the logic circuit 222 is at L level, and an output signal is generated.

[0051] The right diagram in FIG. 7(b) shows the signals Pctrl, the voltage of VC, and the signal VO in FIG. 6(b).

[0052] First, in the case of low illuminance, when a photon is incident while the signal Pctrl is at H level (standby state), the voltage of VC drops and transitions to L level. Since the signal Pctrl is at H level, the output from VO becomes H level, and when the signal Pctrl subsequently becomes L level, the output from VO changes from H level to L level, and a signal is generated.

[0053] On the other hand, in the case of high illuminance, when the signal Pctrl is at H level (standby state), even if photons are frequently incident, as long as the signal Pctrl is at H level, VO, which has become H level, will maintain H level. Next, when the signal Pctrl transitions from H level to L level, the output from VO transitions from H level to L level, and one signal is generated.

[0054] In this way, the clock recharge circuit eliminates the situation where the count value is the same at low illuminance and high illuminance, or where no signal is generated, which occurs in the case of passive recharge. In other words, the clock recharge circuit has the advantage that the count value at low illuminance does not become larger than the count value at high illuminance. The signal output from the logic circuit 222 is output from the counter 211 to the outside via the memory 240. It is possible to appropriately set whether to count at the falling edge of the pulse output from the logic circuit 222 or at the rising edge of the previous pulse. With this configuration, it is possible to obtain an appropriate count value at high illuminance, and to widen the dynamic range, compared to the passive quench circuit.

[0055] (Clock recharge circuit with pause period) 8(a) and (b) are diagrams showing a method of generating a clock signal Pctrl used in clock recharge driving. FIG. 8(a) shows a clock signal Pctrl generated by inverting the clock signal PCLK. The clock signal Pctrl in FIG. 8(a) is a signal used during a 0% idle period. On the other hand, FIG. 8(b) shows a clock signal Pctrl generated by masking every other clock signal PCLK with a mask circuit and inverting it. The clock signal Pctrl in FIG. 8(b) is a signal used during a 50% idle period. Although not shown, if the first signal is not masked by the clock signal PCLK and the second to fourth signals are masked, the signal will be used during a 75% idle period.

[0056] FIG. 8(c) is a timing chart assuming a 50% idle period. It is assumed that, among signals 801 to 804, signals 802 and 804 are masked to set the idle period to 50%. However, the period from the falling edge of signal 811 generated from signal 801 to the falling edge of signal 813 generated from signal 803 is a waiting period after recharging. Therefore, in FIG. 8(c), avalanche multiplication occurs even by photons that enter during the mask period, the voltage of VC drops, VO rises, and the counter counts by 1. In other words, unless some ingenuity is taken, the light that enters the masked period will also be counted, making it impossible to set the idle period appropriately.

[0057] FIG. 9 is a block diagram of a photoelectric conversion device provided to solve the above problem.

[0058] The photoelectric conversion device includes an APD 201 and a count pulse generating circuit 310 that generates a pulse signal from an output signal of the APD 201. The pulse signal output from the count pulse generating circuit 310 is counted by a counter 311.

[0059] In the mask signal generation circuit 340, a plurality of types of signal patterns for masking the clock signal PCLK are generated, and a mask signal selection circuit 350 selects which of these signals is to be output to the subsequent stage.

[0060] The count control circuit 330 is a circuit that generates a signal that controls whether or not the signal counted by the counter 311 is output from the count pulse generation circuit 310 .

[0061] FIG. 10 is a diagram for explaining the block diagram of FIG. 9 in detail, and FIG. 11 is a timing chart corresponding to the block diagram of FIG.

[0062] 10 is a circuit that generates a mask signal P1 corresponding to a 0% pause period, a mask signal P2 corresponding to a 50% pause period, and a mask signal P3 corresponding to a 75% pause period. The mask signal generation circuit 340 is provided in the signal processing unit 103, and generates a mask signal using a signal input from outside the signal processing unit 103. Specifically, the mask signal generation circuit 340 may generate a mask signal using a frequency divider circuit.

[0063] The mask signal selection circuit 350 selects which of the signals P1 to P3 is to be output. The signal output from the mask signal selection circuit 350 is referred to as Ppmsk. For convenience, P1 to P3 may be referred to as mask signals, and Ppmsk, which is a signal based on P1 to P3, may be referred to as a mask signal. As will be described later, P1 is a mask signal that maintains an L level and does not actually have the function of masking the PCLK signal. Therefore, P2 and P3, and Ppmsk corresponding to P2 and P3 may be referred to as mask signals.

[0064] At the start of the exposure period, the P1 signal is output from the mask signal selection circuit 350. Next, when the count value of the counter 311 reaches the threshold value N1, the mask signal selection circuit 350 selects and outputs the P2 signal. Next, when the count value of the counter 311 reaches the threshold value N2, the mask signal selection circuit 350 selects and outputs the P3 signal.

[0065] The photodiode control circuit 320 includes a PMOS transistor that performs recharging, and a control signal generation circuit 322 that outputs Pctrl that is input to the gate of the PMOS transistor. This control signal generation circuit 322 is composed of a NAND circuit with one input inverted, and is configured to receive PCLK and Ppmsk as inputs. In other words, the control signal generation circuit 322 is a control circuit that generates control signals with different periods based on a clock signal and a mask signal.

[0066] (0% downtime) 11, during a 0% idle period, a mask signal P1 that is always at L level is input as Ppmsk to the control signal generation circuit 322. At time t1, since PCLK is at L level, Pctrl, which is the output from the control signal generation circuit 322, is at L level. Therefore, the PMOS transistor 321, which is a recharge circuit, is turned on, and a reverse bias voltage is applied to the APD 201 to an extent that avalanche amplification is possible. Next, when PCLK transitions from H level to L level, Pctrl transitions to H level, and the PMOS transistor 321 is turned off and enters a standby state.

[0067] When a photon is incident on the standby state at time t11, VC drops from H level to L level, and an L level signal is input to the count pulse generation circuit 310. As a result, the output VO from the count pulse generation circuit 310 transitions from L level to H level. The counter 311 counts at the timing when the output VO rises. Although a photon is incident on the standby state again after time t11, the PMOS transistor 321 remains in the off state and VC is already at L level, so avalanche multiplication does not occur and VC remains at L level.

[0068] At time t12, when PCLK transitions from L level to H level, Pctrl transitions from H level to L level, the PMOS transistor 321 turns on, the APD 201 is recharged, and VC transitions from L level to H level. Because VC transitions from L level to H level, the output from the count pulse generating circuit 310 transitions from H level to L level.

[0069] Here, when the idle period is 0%, the selection circuit 331 is configured to output the output from the logic circuit 334 as Pcmsk. Pcmsk is a signal input to the count pulse generation circuit 310, and is a signal that controls the signal output from the count pulse generation circuit 310 to the counter 311. In that sense, Pcmsk is called a count control signal, and the count control circuit 330 can also be said to be a circuit that generates a count control signal.

[0070] When the idle period is 0%, Ppmsk maintains the L level, so that the outputs of D latch 332 and D latch 333 are L level at both the rising and falling edges of PCLK, and the output from logic circuit 334 is L level. As a result, Pcmsk is maintained at the L level, and a signal that is an inversion of the output signal from VC is output from count pulse generation circuit 310. As will be described later in the explanation of time t27, when Ppmsk is H level, even if VC transitions from H level to L level, output VO from count pulse generation circuit 310 is maintained at the L level, so counter 311 does not count.

[0071] (50% downtime) When the count value of the counter 311 reaches the threshold value N1, the mask signal selection circuit 350 selects and outputs the signal P2. Specifically, at the timing of time t2, Ppmsk is replaced from the signal corresponding to P1 to the signal corresponding to P2. Unlike P1, P2 is a signal consisting of a combination of L level and H level. The period when P2 is at H level is a mask period, and Pctrl is set not to turn on the PMOS transistor 321 even if PCLK is at H level during the mask period. Here, the pause period of 50% means that the mask signal P2 is set to mask 50% of the clock signals constituting PCLK. In FIG. 11, P2 is set to be at H level so as to alternately mask the clock signals constituting PCLK.

[0072] At time t21, when P2 transitions from H level to L level, Ppmsk also transitions from H level to L level. At time t22, when PCLK transitions from L level to H level, Pctrl, which is the output from the control signal generation circuit 322, transitions from H level to L level. This causes the APD 201 to be recharged.

[0073] At time t23, PCLK transitions from L level to H level, but since P2 and Ppmsk are H level, Pctrl remains at H level, APD 201 is not recharged, and VC also remains at L level. In other words, P2 and Ppmsk function as mask signals, and the period of Pctrl becomes longer. Specifically, if the period of Pctrl with a 0% idle period is the first period, the period of Pctrl with a 50% idle period is the second period that is longer than the first period.

[0074] On the other hand, because an H level is input to the D latches 332 and 333, an H level signal is output to the logic circuit 334 at the rising and falling edges of PCLK, and the output signal Pcmsk from the selection circuit 331 transitions from an L level to an H level. Since VC is at an L level at time t23, VO transitions from an H level to an L level.

[0075] At time t24, Ppmsk transitions from H level to L level, and the inputs to D latches 332 and 333 become L level. Thereafter, at the timing when PCLK rises, an L level signal is output from D latch 332 to logic circuit 334, and at the timing when PCLK falls, an L level signal is output from D latch 333 to logic circuit 334. As a result, at time t25, Pcmsk transitions from H level to L level.

[0076] At time t26, Pcmsk transitions from L level to H level, similarly to time t23. When a photon is incident at time t27, VC transitions to L level, but since Pcmsk is H level, VO remains at L level and the counter 311 does not count. In this way, the count control circuit 330 performs control so that a signal is not output from the count pulse generation circuit even if a photon is incident during the mask period. Specifically, the count control circuit 330 generates the Pcmsk signal based on the mask signal (P1, P2, P3) that generates Pctrl input to the PMOS transistor 321 of the photodiode control circuit 320. Also, based on the Pcmsk signal, it controls the output signal from the count pulse generation circuit 310. This prevents light that entered during the period when the Pctrl signal was masked from being counted, and allows the pause period to be set appropriately.

[0077] (75% downtime) When the count value of the counter 311 reaches the threshold value N2, the mask signal selection circuit 350 selects and outputs the signal P3. Specifically, at the timing of time t3, Ppmsk is replaced from the signal corresponding to P2 to the signal corresponding to P3. The mask signal P3 has a longer clock signal period than the mask signal P2. As a result, the period of Pctrl with a 75% idle period is longer than the period of Pctrl with a 50% idle period. Specifically, if the period of Pctrl with a 50% idle period is a second period, the period of Pctrl with a 75% idle period is a third period that is longer than the second period.

[0078] As with P2, the period when P3 is at H level is a mask period, and Pctrl is set so as not to turn on the PMOS transistor 321 even if PCLK is at H level during the mask period. Here, a 75% pause period means that the mask signal P3 is set so as to mask 75% of the clock signals that make up PCLK. In Fig. 11, three consecutive signals among the clock signals that make up PCLK are masked, and the signal of Pctrl is generated using the next signal.

[0079] At time t31, similar to times t23 and t26, Pcmsk transitions from L level to H level, and the mask period begins. Also, at time t32, similar to time t25, Pcmsk transitions from H level to L level, and the mask period ends.

[0080] (Correction process) 12 is a diagram showing the relationship between the number of incident photons on the APD and the count number POUT of counter 311. Until POUT reaches threshold N1, the relationship between the number of incident photons and POUT has a slope a. Ideally, if counter 311 counts all photons incident on the APD without exception, the slope a is 1 and the number of photons PN1 and threshold N1 are equal.

[0081] When the count number POUT reaches the threshold N1, the pause period is 50%. Therefore, the slope of the count number POUT with respect to the number of incident photons is a / 2. When the count number POUT reaches the threshold N2, the pause period is 75%. Therefore, the slope of the count number POUT with respect to the number of incident photons is a / 4.

[0082] The correction process of the count number POUT is performed when the count value Nx is greater than the threshold value N1, and the slope of the count POUT with respect to the number of incident photons is corrected to match the slope a for count values ​​in a range different from the slope a. The correction unit that performs the correction process may be a signal processing circuit provided in the photoelectric conversion device, or may be a signal processing circuit provided outside the photoelectric conversion device.

[0083] (effect) According to the configuration of this embodiment, the dynamic range can be expanded, and the number of times avalanche multiplication is performed when high-intensity light is incident can be reduced, thereby reducing power consumption.

[0084] Furthermore, according to the configuration of this embodiment, counting of light that enters during the period when the Pctrl signal is masked is suppressed, and the pause period can also be set appropriately.

[0085] Second Embodiment This embodiment differs from the above-described embodiment in that during a 0% idle period, the selection circuit 331 outputs PCLK as Pcmsk, rather than the output from the logic circuit 334. The other configurations are the same as those of the first embodiment.

[0086] In the embodiment shown in FIG. 11, Pcmsk was maintained at the L level during the 0% pause period from time t1 to time t2, but in FIG. 13, PCLK is set to Pcmsk. At time t11, Pctrl becomes the L level, the PMOS transistor 321 is turned on, and the recharge state is entered. However, as shown in FIG. 13, if photons are incident at the timing of recharge, a situation may occur in which VC does not become the H level. This phenomenon may occur especially in the case of high illuminance. If VC does not become the H level, VO does not become the L level, and photon count leakage may occur. On the other hand, according to the present embodiment, the signal of Pcmsk is configured to be input to the count pulse generating circuit 310. Therefore, when Pcmsk transitions from the L level to the H level, VO transitions from the H level to the L level even if VC is at the L level, which has the advantage of reducing count leakage.

[0087] According to the configuration of this embodiment, the dynamic range can be expanded, and the number of times avalanche multiplication is performed when high-intensity light is incident can be reduced, thereby reducing power consumption.

[0088] Furthermore, according to the configuration of this embodiment, counting of light that enters during the period when the Pctrl signal is masked is suppressed, and the pause period can also be set appropriately.

[0089] Furthermore, according to the configuration of this embodiment, it is possible to reduce count omissions in the case of high illuminance.

[0090] (Third embodiment) This embodiment is different in that the mask signal generating circuit 340 is not provided in each signal processing unit 103 provided for each pixel, but is provided in common to a plurality of pixel circuits 410. The other configurations are the same as those of the above embodiment.

[0091] 14 shows an example in which a mask signal generation circuit 340 is provided in common to the pixel circuits 410 arranged in each row. If the mask signal generation circuit 340 is provided in each pixel circuit 410, the circuit size increases and the space for arranging other circuits such as a counter becomes narrow. According to this embodiment, it is possible to suppress the increase in the circuit size.

[0092] According to the configuration of this embodiment, the dynamic range can be expanded, and the number of times avalanche multiplication is performed when high-intensity light is incident can be reduced, thereby reducing power consumption.

[0093] Furthermore, according to the configuration of this embodiment, counting of light that enters during the period when the Pctrl signal is masked is suppressed, and the pause period can also be set appropriately.

[0094] Furthermore, according to the configuration of this embodiment, it is possible to suppress an increase in the circuit scale.

[0095] (Modification) The embodiment described in this specification does not necessarily have to be configured to gradually extend the idle period within a certain exposure period. For example, in the first mode, it is possible to adopt a drive that does not provide an idle period by using only the mask signal P1, and in the second mode, it is possible to adopt a drive that provides an idle period by using the mask signals P1 to P3. Specifically, it is possible to perform the drive in the first mode during the first exposure period, and the drive in the second mode during the second exposure period following the first exposure period.

[0096] Furthermore, in the above embodiment, three types of mask signals are used, but any types of mask signals may be used as long as they are two or more.

[0097] (Fourth embodiment) The photoelectric conversion system according to this embodiment will be described with reference to Fig. 15. Fig. 15 is a block diagram showing a schematic configuration of the photoelectric conversion system according to this embodiment.

[0098] The photoelectric conversion device described in the above embodiment is applicable to various photoelectric conversion systems. Examples of the applicable photoelectric conversion system include digital still cameras, digital camcorders, security cameras, copiers, fax machines, mobile phones, vehicle-mounted cameras, and observation satellites. In addition, a camera module having an optical system such as a lens and an imaging device is also included in the photoelectric conversion system. FIG. 15 illustrates a block diagram of a digital still camera as an example of these.

[0099] 15 includes an image pickup device 1004, which is an example of a photoelectric conversion device, and a lens 1002 that forms an optical image of a subject on the image pickup device 1004. The photoelectric conversion system further includes an aperture 1003 for varying the amount of light passing through the lens 1002, and a barrier 1001 for protecting the lens 1002. The lens 1002 and the aperture 1003 form an optical system that focuses light on the image pickup device 1004. The image pickup device 1004 is a photoelectric conversion device according to any one of the above embodiments, and converts the optical image formed by the lens 1002 into an electrical signal.

[0100] The photoelectric conversion system also has a signal processing unit 1007 which is an image generating unit that generates an image by processing an output signal output from the imaging device 1004. The signal processing unit 1007 performs various corrections and compression as necessary to output image data. The signal processing unit 1007 may be formed in a semiconductor layer in which the imaging device 1004 is provided, or may be formed in a semiconductor layer different from that of the imaging device 1004. Moreover, the imaging device 1004 and the signal processing unit 1007 may be formed in the same semiconductor layer.

[0101] The photoelectric conversion system further includes a memory unit 1010 for temporarily storing image data, and an external interface unit (external I / F unit) 1013 for communicating with an external computer or the like. The photoelectric conversion system further includes a recording medium 1012 such as a semiconductor memory for recording or reading out imaging data, and a recording medium control interface unit (recording medium control I / F unit) 1011 for recording or reading out data on the recording medium 1012. The recording medium 1012 may be built into the photoelectric conversion system, or may be removable.

[0102] The photoelectric conversion system further includes an overall control / calculation unit 1009 that performs various calculations and controls the entire digital still camera, and a timing generation unit 1008 that outputs various timing signals to the image capture device 1004 and the signal processing unit 1007. Here, the timing signals and the like may be input from outside, and the photoelectric conversion system only needs to include at least the image capture device 1004 and the signal processing unit 1007 that processes the output signal output from the image capture device 1004.

[0103] The imaging device 1004 outputs an imaging signal to a signal processing unit 1007. The signal processing unit 1007 performs predetermined signal processing on the imaging signal output from the imaging device 1004, and outputs image data. The signal processing unit 1007 generates an image using the imaging signal.

[0104] In this way, according to this embodiment, it is possible to realize a photoelectric conversion system to which the photoelectric conversion device (imaging device) according to any one of the above embodiments is applied.

[0105] Fifth embodiment The photoelectric conversion system and the moving object of this embodiment will be described with reference to Fig. 16. Fig. 16 is a diagram showing the configuration of the photoelectric conversion system and the moving object of this embodiment.

[0106] FIG. 16(a) shows an example of a photoelectric conversion system related to an in-vehicle camera. The photoelectric conversion system 2300 has an imaging device 2310. The imaging device 2310 is the photoelectric conversion device described in any of the above embodiments. The photoelectric conversion system 2300 has an image processing unit 2312 that performs image processing on a plurality of image data acquired by the imaging device 2310. The photoelectric conversion system 2300 also has a parallax acquisition unit 2314 that calculates parallax (phase difference of parallax images) from a plurality of image data acquired by the photoelectric conversion system 2300. The photoelectric conversion system 2300 also has a distance acquisition unit 2316 that calculates a distance to an object based on the calculated parallax, and a collision determination unit 2318 that determines whether or not there is a possibility of collision based on the calculated distance. Here, the parallax acquisition unit 2314 and the distance acquisition unit 2316 are examples of distance information acquisition means that acquire distance information to an object. That is, the distance information is information on the parallax, the defocus amount, the distance to the object, and the like. The collision determination unit 2318 may determine the possibility of a collision using any of these pieces of distance information. The distance information acquisition means may be realized by dedicated hardware, or may be realized by a software module. In addition, it may be realized by a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or the like, or may be realized by a combination of these.

[0107] The photoelectric conversion system 2300 is connected to a vehicle information acquisition device 2320, and can acquire vehicle information such as vehicle speed, yaw rate, and steering angle. The photoelectric conversion system 2300 is also connected to a control ECU 2330, which is a control device (control unit) that outputs a control signal to generate a braking force for the vehicle based on the judgment result of the collision judgment unit 2318. The photoelectric conversion system 2300 is also connected to an alarm device 2340 that issues an alarm to the driver based on the judgment result of the collision judgment unit 2318. For example, when the judgment result of the collision judgment unit 2318 indicates that there is a high possibility of a collision, the control ECU 2330 performs vehicle control to avoid a collision and reduce damage by applying the brakes, releasing the accelerator, suppressing engine output, etc. The alarm device 2340 warns the user by sounding an alarm, displaying alarm information on the screen of a car navigation system, etc., or vibrating the seat belt or steering wheel.

[0108] In this embodiment, the surroundings of the vehicle, for example, the front or rear, are imaged by the photoelectric conversion system 2300. Fig. 16(b) shows a photoelectric conversion system for imaging the area in front of the vehicle (imaging range 2350). A vehicle information acquisition device 2320 sends an instruction to the photoelectric conversion system 2300 or the imaging device 2310. This configuration can further improve the accuracy of distance measurement.

[0109] Although the above describes an example of control to prevent collision with other vehicles, the present invention can also be applied to control of automatic driving by following other vehicles, control of automatic driving to prevent deviation from lanes, etc. Furthermore, the photoelectric conversion system is not limited to vehicles such as the vehicle itself, but can be applied to moving bodies (moving devices) such as ships, aircraft, and industrial robots. In addition, the present invention can be applied not only to moving bodies, but also to a wide range of devices that use object recognition, such as intelligent transport systems (ITS).

[0110] Sixth embodiment The photoelectric conversion system of this embodiment will be described with reference to Fig. 17. Fig. 17 is a block diagram showing an example of the configuration of a range image sensor which is the photoelectric conversion system.

[0111] 17, the distance image sensor 401 is configured to include an optical system 402, a photoelectric conversion device 403, an image processing circuit 404, a monitor 405, and a memory 406. The distance image sensor 401 can obtain a distance image according to the distance to the subject by receiving light (modulated light or pulsed light) that is projected from a light source device 411 toward the subject and reflected by the surface of the subject.

[0112] The optical system 402 is configured to have one or more lenses, and guides image light (incident light) from a subject to the photoelectric conversion device 403 , forming an image on the light receiving surface (sensor portion) of the photoelectric conversion device 403 .

[0113] As the photoelectric conversion device 403 , the photoelectric conversion device described in the above-mentioned embodiment is applied, and a distance signal indicating a distance determined from a light receiving signal output from the photoelectric conversion device 403 is supplied to the image processing circuit 404 .

[0114] The image processing circuit 404 performs image processing to construct a distance image based on the distance signal supplied from the photoelectric conversion device 403. The distance image (image data) obtained by this image processing is then supplied to a monitor 405 for display, or supplied to a memory 406 for storage (recording).

[0115] In the range image sensor 401 configured in this way, by applying the above-described photoelectric conversion device, it is possible to obtain, for example, a more accurate range image as the pixel characteristics improve.

[0116] Seventh embodiment The photoelectric conversion system of this embodiment will be described with reference to Fig. 18. Fig. 18 is a diagram showing an example of a schematic configuration of an endoscopic surgery system which is the photoelectric conversion system of this embodiment.

[0117] 18 shows a state in which an operator (doctor) 1131 is performing surgery on a patient 1132 on a patient bed 1133 using an endoscopic surgery system 1103. As shown in the figure, the endoscopic surgery system 1103 is composed of an endoscope 1100, a surgical tool 1110, and a cart 1134 on which various devices for endoscopic surgery are mounted.

[0118] The endoscope 1100 is composed of a lens barrel 1101, a region of a predetermined length from the tip of which is inserted into a body cavity of a patient 1132, and a camera head 1102 connected to the base end of the lens barrel 1101. In the illustrated example, the endoscope 1100 is configured as a so-called rigid lens barrel having a rigid lens barrel 1101, but the endoscope 1100 may be configured as a so-called flexible lens barrel having a flexible lens barrel.

[0119] An opening into which an objective lens is fitted is provided at the tip of the lens barrel 1101. A light source device 1203 is connected to the endoscope 1100, and light generated by the light source device 1203 is guided to the tip of the lens barrel 1101 by a light guide extending inside the lens barrel 1101, and is irradiated via the objective lens toward an observation target in a body cavity of a patient 1132. The endoscope 1100 may be a direct-viewing endoscope, an oblique-viewing endoscope, or a side-viewing endoscope.

[0120] An optical system and a photoelectric conversion device are provided inside the camera head 1102, and reflected light (observation light) from an observation target is collected by the optical system onto the photoelectric conversion device. The observation light is photoelectrically converted by the photoelectric conversion device to generate an electrical signal corresponding to the observation light, i.e., an image signal corresponding to an observation image. The photoelectric conversion device may be any of the photoelectric conversion devices described in the above-mentioned embodiments. The image signal is transmitted to a camera control unit (CCU) 1135 as RAW data.

[0121] The CCU 1135 is configured with a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), etc., and performs overall control of the operations of the endoscope 1100 and the display device 1136. Furthermore, the CCU 1135 receives an image signal from the camera head 1102, and performs various types of image processing on the image signal, such as development processing (demosaic processing), for displaying an image based on the image signal.

[0122] Under the control of the CCU 1135 , the display device 1136 displays an image based on an image signal that has been subjected to image processing by the CCU 1135 .

[0123] The light source device 1203 is composed of a light source such as an LED (Light Emitting Diode), and supplies the endoscope 1100 with irradiation light when photographing an operation site or the like.

[0124] The input device 1137 is an input interface for the endoscopic surgery system 1103. A user can input various information and instructions to the endoscopic surgery system 1103 via the input device 1137.

[0125] The treatment tool control device 1138 controls the driving of the energy treatment tool 1112 for cauterizing tissue, incising, sealing blood vessels, or the like.

[0126] The light source device 1203 that supplies irradiation light to the endoscope 1100 when photographing the surgical site can be composed of a white light source composed of, for example, an LED, a laser light source, or a combination of these. When the white light source is composed of a combination of RGB laser light sources, the output intensity and output timing of each color (each wavelength) can be controlled with high precision, so that the white balance of the captured image can be adjusted in the light source device 1203. In this case, it is also possible to capture images corresponding to each of the RGB colors in a time-division manner by irradiating the observation target with laser light from each of the RGB laser light sources in a time-division manner and controlling the driving of the image sensor of the camera head 1102 in synchronization with the irradiation timing. According to this method, a color image can be obtained without providing a color filter to the image sensor.

[0127] Furthermore, the light source device 1203 may be controlled to change the intensity of the light it outputs at predetermined time intervals. By controlling the driving of the image sensor of the camera head 1102 in synchronization with the timing of the change in the light intensity to acquire images in a time-division manner and synthesizing the images, it is possible to generate an image with a high dynamic range that is free of so-called blocked-up shadows and blown-out highlights.

[0128] The light source device 1203 may be configured to supply light in a predetermined wavelength band corresponding to the special light observation. In the special light observation, for example, the wavelength dependency of light absorption in body tissue is utilized. Specifically, a predetermined tissue such as blood vessels on the mucous membrane surface is photographed with high contrast by irradiating light in a narrower band than the irradiation light (i.e., white light) in normal observation. Alternatively, the special light observation may be a fluorescent observation in which an image is obtained by fluorescence generated by irradiating excitation light. In the fluorescent observation, it is possible to irradiate excitation light to the body tissue and observe the fluorescence from the body tissue, or to locally inject a reagent such as indocyanine green (ICG) into the body tissue and irradiate the body tissue with excitation light corresponding to the fluorescent wavelength of the reagent to obtain a fluorescent image. The light source device 1203 may be configured to supply narrow band light and / or excitation light corresponding to such special light observation.

[0129] Eighth embodiment The photoelectric conversion system of this embodiment will be described with reference to FIG. 19. FIG. 19(a) is a diagram showing an example of the configuration of glasses 1600 (smart glasses) which are a photoelectric conversion system. The glasses 1600 have a photoelectric conversion device 1602. The photoelectric conversion device 1602 is the photoelectric conversion device described in the above-mentioned twelfth embodiment. A display device including a light-emitting device such as an OLED or LED may be provided on the back side of the lens 1601. The photoelectric conversion device 1602 may be one or more. A combination of multiple types of photoelectric conversion devices may also be used. The arrangement position of the photoelectric conversion device 1602 is not limited to that shown in FIG. 19(a).

[0130] The glasses 1600 further include a control device 1603. The control device 1603 functions as a power source that supplies power to the photoelectric conversion device 1602 and the display device. The control device 1603 also controls the operations of the photoelectric conversion device 1602 and the display device. The lens 1601 is formed with an optical system for focusing light on the photoelectric conversion device 1602.

[0131] FIG. 19(b) illustrates glasses 1610 (smart glasses) according to one application example. The glasses 1610 have a control device 1612, and the control device 1612 is equipped with a photoelectric conversion device corresponding to the photoelectric conversion device 1602 and a display device. The lens 1611 is formed with an optical system for projecting light emitted from the photoelectric conversion device in the control device 1612 and the display device, and an image is projected onto the lens 1611. The control device 1612 functions as a power source for supplying power to the photoelectric conversion device and the display device, and controls the operation of the photoelectric conversion device and the display device. The control device may have a line of sight detection unit that detects the line of sight of the wearer. Infrared light may be used for detecting the line of sight. The infrared light emission unit emits infrared light toward the eyeball of a user gazing at a display image. An imaging unit having a light receiving element detects the reflected light of the emitted infrared light from the eyeball, thereby obtaining an image of the eyeball. By having a reduction means for reducing light from the infrared light emission unit to the display unit in a planar view, deterioration of image quality is reduced.

[0132] The gaze of the user with respect to the displayed image is detected from an image of the eyeball obtained by capturing infrared light. Any known method can be applied to gaze detection using the image of the eyeball. As an example, a gaze detection method based on a Purkinje image formed by reflection of irradiated light on the cornea can be used.

[0133] More specifically, a gaze detection process based on the pupil-corneal reflex method is performed. Using the pupil-corneal reflex method, a gaze vector that indicates the direction (rotation angle) of the eyeball is calculated based on the pupil image and the Purkinje image included in the captured image of the eyeball, thereby detecting the user's gaze.

[0134] The display device of this embodiment may have a photoelectric conversion device having a light receiving element, and may control the display image of the display device based on information on the user's line of sight from the photoelectric conversion device.

[0135] Specifically, the display device determines a first field of view area to which the user gazes and a second field of view area other than the first field of view area based on the line-of-sight information. The first field of view area and the second field of view area may be determined by a control device of the display device, or may be received from an external control device. In the display area of ​​the display device, the display resolution of the first field of view area may be controlled to be higher than the display resolution of the second field of view area. In other words, the resolution of the second field of view area may be lower than that of the first field of view area.

[0136] The display area may have a first display area and a second display area different from the first display area, and a high priority area may be determined from the first display area and the second display area based on line-of-sight information. The first field of view area and the second field of view area may be determined by a control device of the display device, or may be received from an external control device. The resolution of the high priority area may be controlled to be higher than the resolution of areas other than the high priority area. In other words, the resolution of an area with a relatively low priority may be lowered.

[0137] In addition, AI may be used to determine the first field of view area and the area with high priority. The AI ​​may be a model configured to estimate the angle of the line of sight and the distance to an object at the end of the line of sight from the image of the eyeball, using the image of the eyeball and the direction in which the eyeball in the image was actually looking as teacher data. The AI ​​program may be included in the display device, the photoelectric conversion device, or an external device. If included in the external device, it is transmitted to the display device via communication.

[0138] When display control is performed based on visual recognition detection, the present invention is preferably applicable to smart glasses further including a photoelectric conversion device for capturing an image of the outside world. The smart glasses can display captured outside information in real time.

[0139] The above-described embodiments can be modified as appropriate without departing from the technical concept. In addition, an example in which a part of the configuration of any of the embodiments is added to another embodiment, or an example in which a part of the configuration of any of the embodiments is replaced with another embodiment, is also included in the embodiments of the present invention.

[0140] The disclosure of this embodiment also includes the following configurations and methods.

[0141] (Configuration 1) a photodiode for avalanche multiplication; a photodiode control circuit including: a recharge circuit that switches between a first state in which the photodiode is electrically connected to a power supply and a second state in which the photodiode is not electrically connected to the power supply; and a control signal generation circuit that generates a signal that controls the recharge circuit; a count pulse generating circuit for generating a pulse signal from an output signal from the photodiode; a counter that counts the pulse signal output from the count pulse generating circuit, the control signal generation circuit is configured to be capable of outputting the control signal having a first period and the control signal having a second period that is longer than the first period, based on a clock signal and a mask signal; A photoelectric conversion device characterized in that, when the count value of the counter reaches a threshold value, the output signal of the control signal generation circuit is switched from a control signal having the first period to a control signal having the second period.

[0142] (Configuration 2) a count control circuit that generates a count control signal to be input to the count pulse generation circuit; 2. The photoelectric conversion device according to configuration 1, wherein the count control circuit generates the count control signal based on the clock signal and the mask signal.

[0143] (Configuration 3) The count pulse generating circuit includes: outputting a signal corresponding to an output signal from the photodiode to the counter when the count control signal is at a first level; The photoelectric conversion device according to configuration 2, characterized in that when the count control signal is at a second level different from the first level, a signal corresponding to the output signal from the photodiode is not output to the counter.

[0144] (Configuration 4) The count control circuit has a selection circuit, 4. The photoelectric conversion device according to configuration 2 or 3, wherein the selection circuit is configured to be able to select between outputting the count control signal and outputting the clock signal.

[0145] (Configuration 5) 5. The photoelectric conversion device according to configuration 4, wherein the selection circuit is configured to output the clock signal before the counter reaches the threshold value.

[0146] (Configuration 6) the control signal generation circuit is configured to be capable of outputting the control signal having the second period and the control signal having a third period that is longer than the second period, based on the clock signal and the mask signal; A photoelectric conversion device described in any one of configurations 1 to 5, characterized in that when the count value of the counter reaches another threshold value that is higher than the threshold value, the output signal of the control signal generation circuit is switched from a control signal having the second period to a control signal having the third period.

[0147] (Configuration 7) The photoelectric conversion device according to any one of configurations 1 to 6, further comprising a correction unit that corrects the count value of the counter based on the first period and the second period when the count value of the counter reaches the threshold value.

[0148] (Configuration 8) The photoelectric conversion device according to any one of configurations 1 to 7, further comprising a mask signal generating circuit that generates the mask signal.

[0149] (Configuration 9) a plurality of pixels, each of the plurality of pixels including the photodiode, the photodiode control circuit, the count pulse generation circuit, and the counter; 9. The photoelectric conversion device according to configuration 8, wherein the mask signal generation circuit is provided for each of the plurality of pixels.

[0150] (Configuration 10) a plurality of pixels, each of the plurality of pixels including the photodiode, the photodiode control circuit, the count pulse generation circuit, and the counter; 9. The photoelectric conversion device according to configuration 8, wherein the mask signal generation circuit is provided commonly to the plurality of pixels.

[0151] (Configuration 11) A photoelectric conversion device according to any one of configurations 1 to 10; a signal processing unit that generates an image using a signal output from the photoelectric conversion device.

[0152] (Configuration 12) A moving object including the photoelectric conversion device according to any one of configurations 1 to 10, A moving body comprising: a control unit that controls the movement of the moving body using a signal output from the photoelectric conversion device. [Explanation of symbols]

[0153] 201 Avalanche Photodiode (APD) 310 Count pulse generating circuit 311 Counter 320 Photodiode control circuit 321 PMOS transistor 322 Control signal generation circuit 330 Count control circuit 331 Selection circuit 332 D-Latch 333 D Latch 340 Mask signal generating circuit 350 Mask signal selection circuit

Claims

1. a photodiode for avalanche multiplication; a photodiode control circuit including: a recharge circuit that switches between a first state in which the photodiode is electrically connected to a power supply and a second state in which the photodiode is not electrically connected to the power supply; and a control signal generation circuit that generates a signal that controls the recharge circuit; a count pulse generating circuit for generating a pulse signal from an output signal from the photodiode; a counter that counts the pulse signal output from the count pulse generating circuit, the control signal generation circuit is configured to be capable of outputting the control signal having a first period and the control signal having a second period that is longer than the first period, based on a clock signal and a mask signal; A photoelectric conversion device characterized in that, when the count value of the counter reaches a threshold value, the output signal of the control signal generation circuit is switched from a control signal having the first period to a control signal having the second period.

2. a count control circuit that generates a count control signal to be input to the count pulse generation circuit; 2. The photoelectric conversion device according to claim 1, wherein the count control circuit generates the count control signal based on the clock signal and the mask signal.

3. The count pulse generating circuit includes: outputting a signal corresponding to an output signal from the photodiode to the counter when the count control signal is at a first level; 3. The photoelectric conversion device according to claim 2, wherein the device is configured not to output a signal corresponding to an output signal from the photodiode to the counter when the count control signal is at a second level different from the first level.

4. The count control circuit has a selection circuit, 3. The photoelectric conversion device according to claim 2, wherein the selection circuit is configured to be able to select between outputting the count control signal and outputting the clock signal.

5. 5. The photoelectric conversion device according to claim 4, wherein the selection circuit is configured to output the clock signal before the counter reaches the threshold value.

6. the control signal generation circuit is configured to be capable of outputting the control signal having the second period and the control signal having a third period that is longer than the second period, based on the clock signal and the mask signal; The photoelectric conversion device described in claim 1, characterized in that when the count value of the counter reaches another threshold value that is higher than the threshold value, the output signal of the control signal generation circuit is configured to be switched from a control signal having the second period to a control signal having the third period.

7. 2. The photoelectric conversion device according to claim 1, further comprising a correction unit that corrects the count value of the counter based on the first period and the second period when the count value of the counter reaches the threshold value.

8. 2. The photoelectric conversion device according to claim 1, further comprising a mask signal generating circuit that generates the mask signal.

9. a plurality of pixels, each of the plurality of pixels including the photodiode, the photodiode control circuit, the count pulse generation circuit, and the counter; 9. The photoelectric conversion device according to claim 8, wherein the mask signal generating circuit is provided for each of the plurality of pixels.

10. a plurality of pixels, each of the plurality of pixels including the photodiode, the photodiode control circuit, the count pulse generation circuit, and the counter; 9. The photoelectric conversion device according to claim 8, wherein the mask signal generating circuit is provided in common to the plurality of pixels.

11. The photoelectric conversion device according to claim 1 ; a signal processing unit that generates an image using a signal output from the photoelectric conversion device.

12. A moving object comprising the photoelectric conversion device according to claim 1, A moving body comprising: a control unit that controls the movement of the moving body using a signal output from the photoelectric conversion device.

Images