Photoelectric conversion device and light detection system

Abstract

To provide a photoelectric conversion device capable of controlling pulse signals in a specific manner when there are multiple exposure periods.SOLUTION: The photoelectric conversion device includes an avalanche photodiode including an anode and a cathode, and a switch connected to one of the anode and the cathode, to which a pulse signal is input. A value obtained by dividing the number of pulse signals during a first exposure period by the first exposure period and a value obtained by dividing the number of pulse signals in a second exposure period having a length different from the length of the first exposure period by the second exposure period are different.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] The present invention relates to a photoelectric conversion device that performs photoelectric conversion and a photodetection system.

Background Art

[0002] A photoelectric conversion device including a pixel array formed such that pixels including a plurality of avalanche photodiodes (APDs) are arranged in a two-dimensional array in a plane is known. In each pixel, in the PN junction region in the semiconductor region, the photo charges caused by single photons cause avalanche multiplication.

[0003] Patent Document 1 discloses that a pixel having an APD includes an APD, a quench circuit connected to the APD, a signal control circuit to which a signal output from the APD is input, and a pulse generation circuit connected to the quench circuit and the signal control circuit. The pulse generation circuit controls the on / off of the quench circuit. Further, it is disclosed that the output signal is reset for each pulse signal and a pulse signal corresponding to the input photons is output even under high brightness.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] Patent Document 1 does not disclose anything regarding the number or period of pulse signals within the exposure period when the exposure periods are different. There is room to consider controlling the pulse signal in relation to the exposure period in Patent Document 1.

Means for Solving the Problems

[0006] A photoelectric conversion device according to one embodiment includes an avalanche photodiode including an anode and a cathode, and a switch connected to one of the anode and the cathode to which a pulse signal is input. A value obtained by dividing the number of pulse signals in the first exposure period by the first exposure period and multiplying by the first exposure period is different from a value obtained by dividing the number of pulse signals in a second exposure period having a length different from the length of the first exposure period by the second exposure period and multiplying by the first exposure period.

Effect of the Invention

[0007] According to the present invention, it is possible to provide a photoelectric conversion device capable of controlling specific pulse signals when there are a plurality of exposure periods.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Modes for Carrying Out the Invention

[0009] The following embodiments are for embodying the technical idea of the present invention and do not limit the present invention. The sizes and positional relationships of the members shown in each drawing may be exaggerated for clarity of explanation. In the following description, the same configurations may be denoted by the same numbers and the description thereof may be omitted.

[0010] Using FIGS. 1 to 4, the configurations common to the photoelectric conversion devices in each embodiment will be described. The photoelectric conversion device has an SPAD pixel including an avalanche diode. The conductivity type of the charge used as the signal charge among the charge pairs generated in the avalanche diode is called the first conductivity type. The first conductivity type refers to the conductivity type in which charges having the same polarity as the signal charge are majority carriers. Also, the conductivity type opposite to the first conductivity type is called the second conductivity type. Hereinafter, an example in which the signal charge is an electron, the first conductivity type is N-type, and the second conductivity type is P-type will be described, but the signal charge may be a hole, the first conductivity type may be P-type, and the second conductivity type may be N-type.

[0011] When the signal charge is an electron, the signal is read out from the cathode of the APD. When the signal charge is a hole, the signal is read out from the anode of the APD. Therefore, the cathode and anode of the APD have an inverse relationship.

[0012] In this specification, "planar view" refers to viewing from a direction perpendicular to the light incident surface of the semiconductor layer on which the photoelectric conversion element described later is disposed. Further, the cross section refers to a plane in a direction perpendicular to the light incident surface of the semiconductor layer on which the photoelectric conversion element is disposed. When the light incident surface of the semiconductor layer is a rough surface when viewed microscopically, the planar view is defined based on the light incident surface of the semiconductor layer when viewed macroscopically.

[0013] First, the configuration common to each embodiment will be described.

[0014] FIG. 1 is a diagram showing the configuration of a photoelectric conversion device 100 according to an embodiment of the present invention. Hereinafter, the case where the photoelectric conversion device 100 is a stacked photoelectric conversion device will be described as an example. That is, a photoelectric conversion device configured by laminating two substrates, a sensor substrate 11 and a circuit substrate 21, and electrically connecting them will be described as an example. However, the photoelectric conversion device is not limited to this. For example, a photoelectric conversion device in which the configuration included in the sensor substrate 11 and the configuration included in the circuit substrate are arranged on a common semiconductor layer may be used. Hereinafter, a photoelectric conversion device in which the configuration included in the sensor substrate 11 and the configuration included in the circuit substrate are arranged on a common semiconductor layer is also referred to as a non-stacked photoelectric conversion device.

[0015] The sensor substrate 11 has a first semiconductor layer having a photoelectric conversion element 102 to be described later and a first wiring structure. The circuit substrate 21 has a second semiconductor layer having circuits such as a signal processing circuit 103 to be described later and a second wiring structure. The photoelectric conversion device 100 is configured by laminating the second semiconductor layer, the second wiring structure, the first wiring structure, and the first semiconductor layer in this order.

[0016] FIG. 1 illustrates a back-illuminated type photoelectric conversion device in which light is incident from a first surface and a circuit board is disposed on a second surface which is the surface opposite to the first surface. In the case of a non-laminated photoelectric conversion device, the surface on which the transistors of the signal processing circuit are disposed is referred to as the second surface. In the case of a back-illuminated type photoelectric conversion device, the first surface opposite to the second surface of the semiconductor layer serves as the light incident surface. In the case of a front-illuminated type photoelectric conversion device, the second surface of the semiconductor layer serves as the light incident surface.

[0017] Hereinafter, the sensor substrate 11 and the circuit substrate 21 will be described as diced chips, but are not limited to chips. For example, each substrate may be a wafer. Further, each substrate may be diced after being laminated in a wafer state, or each chip may be laminated and joined after being chipped.

[0018] A pixel region 12 is disposed on the sensor substrate 11, and a circuit region 22 for processing the signal detected in the pixel region 12 is disposed on the circuit substrate 21.

[0019] FIG. 2 is a diagram showing an arrangement example of the sensor substrate 11. Pixels 101 each having a photoelectric conversion element 102 including an avalanche photodiode (hereinafter, APD) are arranged in a two-dimensional array in a plan view to form the pixel region 12.

[0020] The pixel 101 is typically a pixel for forming an image, but when used for TOF (Time of Flight), it does not necessarily have to form an image. That is, the pixel 101 may be a pixel for measuring the arrival time and the amount of light of the light.

[0021] FIG. 3 is a configuration diagram of the circuit substrate 21. It has a signal processing circuit 103, a readout circuit 112, a control pulse generation unit 115, a horizontal scanning circuit unit 111, signal lines 113, and a vertical scanning circuit unit 110 that process the charges photoelectrically converted by the photoelectric conversion element 102 in FIG. 2.

[0022] The photoelectric conversion element 102 in FIG. 2 and the signal processing circuit 103 in FIG. 3 are electrically connected via connection wirings provided for each pixel.

[0023] The vertical scanning circuit section 110 receives the control pulses supplied from the control pulse generation section 115 and supplies the control pulses to each pixel. Logic circuits such as a shift register and an address decoder are used for the vertical scanning circuit section 110.

[0024] The control pulse generation section 115 includes a signal generation section 215 that generates a control signal P_CLK for a switch described later. The signal generation section 215 generates a pulse signal for controlling the switch as will be described later. For example, as shown in FIG. 4(a), the signal generation section 215 may commonly generate the control signal P_CLK for a plurality of pixels in the pixel region, or as shown in FIG. 4(b), the control signal P_CLK may be generated for each pixel. When the pulse signal P_CLK is commonly generated, at least one of the period, the number of pulses, and the pulse width of the signal P_EXP pulse signal for controlling the exposure period is commonly generated in correspondence with the exposure period. Also, when the control signal P_CLK is controlled for each pixel, a signal can be generated using both the input signal P_CLK_IN output from the control pulse generation section 115 and the signal P_EXP for controlling the exposure period. The control pulse generation section 115 preferably has, for example, a frequency division circuit. Thereby, it becomes possible to control simply and increase in the number of elements can be reduced.

[0025] The signal output from the photoelectric conversion element 102 of the pixel is processed by the signal processing circuit 103. The signal processing circuit 103 is provided with a counter, a memory, etc., and digital values are held in the memory.

[0026] The horizontal scanning circuit section 111 inputs a control pulse for sequentially selecting each column to the signal processing circuit 103 in order to read a signal from the memory of each pixel in which the digital signal is held.

[0027] For the selected column, a signal is output from the signal processing circuit 103 of the pixel selected by the vertical scanning circuit section 110 to the signal line 113.

[0028] The signal output to the signal line 113 is output to a recording unit or a signal processing unit outside the photoelectric conversion device 100 via the output circuit 114.

[0029] In FIG. 2, the arrangement of the photoelectric conversion elements in the pixel region may be arranged in a one-dimensional manner. Also, even if there is only one pixel, the effects of the present invention can be obtained, and the case of one pixel is also included in the present invention. However, in the case of a photoelectric conversion device having a plurality of pixels, the effect of reducing power consumption of the present embodiment can be easily obtained. The functions of the signal processing unit do not necessarily have to be provided one by one for all the photoelectric conversion elements. For example, one signal processing unit may be shared by a plurality of photoelectric conversion elements, and signal processing may be performed sequentially.

[0030] As shown in FIGS. 2 and 3, a plurality of signal processing circuits 103 are arranged in a region overlapping the pixel region 12 in a plan view. And in a plan view, the vertical scanning circuit unit 110, the horizontal scanning circuit unit 111, the reading circuit 112, the output circuit 114, and the control pulse generation unit 115 are arranged so as to overlap between the end of the sensor substrate 11 and the end of the pixel region 12. In other words, the sensor substrate 11 has a pixel region 12 and a non-pixel region arranged around the pixel region 12. And in a region overlapping the non-pixel region in a plan view, the vertical scanning circuit unit 110, the horizontal scanning circuit unit 111, the reading circuit 112, the output circuit 114, and the control pulse generation unit 115 are arranged.

[0031] Note that the arrangement of the signal line 113, the arrangement of the reading circuit 112, and the arrangement of the output circuit 114 are not limited to FIG. 3. For example, the signal line 113 may be arranged to extend in the row direction, and the reading circuit 112 may be arranged at the end where the signal line 113 extends.

[0032] FIG. 4 is an example of a block diagram including the equivalent circuits of FIGS. 2 and 3. FIG. 4(a) is an example in which the signal generation unit 215 is provided in common for a plurality of pixels, and FIG. 4(b) is an example in which the control signal P_CLK can be controlled for each pixel.

[0033] In FIG. 4, the photoelectric conversion element 102 having the APD 201 is provided on the sensor substrate 11, and the other members are provided on the circuit substrate 21.

[0034] The APD 201 generates charge pairs corresponding to incident light by photoelectric conversion. One of the two nodes of the APD 201 is connected to a power supply line to which a drive voltage VL (first voltage) is supplied. The other of the two nodes of the APD 201 is connected to a power supply line to which a drive voltage VH (second voltage) higher than the voltage VL supplied to the anode is supplied. In FIG. 4, one node of the APD 201 is the anode, and the other node of the APD is the cathode. A reverse bias voltage is supplied to the anode and cathode of the APD 201 so that the APD 201 performs an avalanche multiplication operation. By supplying such a voltage, the charges generated by the incident light cause avalanche multiplication, and an avalanche current is generated.

[0035] In addition, when a reverse bias voltage is supplied, there are a Geiger mode in which the potential difference between the anode and the cathode operates at a potential difference greater than the breakdown voltage, and a linear mode in which the potential difference between the anode and the cathode operates at a voltage difference near or below the breakdown voltage.

[0036] An APD operating in the Geiger mode is called a SPAD. For example, the voltage VL (first voltage) is -30 V, and the voltage VH (second voltage) is 1 V. The APD 201 may operate in the linear mode or the Geiger mode. In the case of a SPAD, the potential difference is larger than that of a linear-mode APD, and the effect of voltage resistance is remarkable. Therefore, it is preferably a SPAD.

[0037] Switch 202 is connected to a power supply line to which a drive voltage VH is supplied and one of the anode and cathode nodes of APD 201. Then, switch 202 switches the resistance value between APD 201 and the power supply line to which the drive voltage VH is supplied. Here, it is preferable to change the resistance value by 10 times or more, and more preferably by 100 times or more, when switching the resistance value. Hereinafter, when the resistance value becomes low, it is also referred to as the on state of switch 202, and when the resistance value becomes high, it is also referred to as the off state of switch 202. Switch 202 functions as a quench element. Switch 202 functions as a load circuit (quench circuit) during signal multiplication by avalanche multiplication, and has a function of suppressing the voltage supplied to APD 201 to suppress avalanche multiplication (quench operation). Further, switch 202 has a function of returning the voltage supplied to APD 201 to the drive voltage VH by flowing a current corresponding to the voltage drop during the quench operation (recharge operation).

[0038] Switch 202 can be constituted by, for example, a MOS transistor. In FIG. 4, the case where switch 202 is a PMOS transistor is shown. The control signal P_CLK of switch 202 supplied from the signal generation unit 215 is applied to the gate electrode of the MOS transistor constituting switch 202. In the present embodiment, the on and off states of switch 202 are controlled by controlling the voltage applied to the gate electrode of switch 202.

[0039] The signal processing circuit 103 includes a waveform shaping unit 210, a counter circuit 211, and a selection circuit 212. In FIG. 4, the signal processing circuit 103 includes a waveform shaping unit 210, a counter circuit 211, and a selection circuit 212. However, in this specification, the signal processing circuit 103 only needs to include at least any one of the waveform shaping unit 210, the counter circuit 211, and the selection circuit 212.

[0040] The waveform shaping unit 210 shapes the potential change of the cathode of the APD 201 obtained at the time of photon detection and outputs a pulse signal. Let the input-side node of the waveform shaping unit 210 be nodeA and the output-side node be nodeB. The waveform shaping unit 210 changes the output potential from nodeB according to whether the input potential to nodeA is equal to or higher than a predetermined value or lower than that. For example, in FIG. 5, when the input potential to nodeA becomes a high potential equal to or higher than the determination threshold value, the output potential from nodeB becomes a low level. And when the input potential to nodeA becomes a potential lower than the determination threshold value, the output potential from nodeB becomes a high level. As the waveform shaping unit 210, for example, an inverter circuit is used. In FIG. 4, an example using one inverter as the waveform shaping unit 210 is shown, but a circuit in which a plurality of inverters are connected in series may be used, or other circuits having a waveform shaping effect may be used.

[0041] It is possible to perform a quench operation and a recharge operation using the switch 202 according to the avalanche multiplication in the APD 201, but there are cases where it is not determined as an output signal depending on the photon detection timing. For example, assume a case where avalanche multiplication occurs in the APD and the input potential to nodeA becomes a low level, and the recharge operation is being performed. Generally, the determination threshold value of the waveform shaping unit 210 is set to a potential higher than the potential difference at which avalanche multiplication occurs in the APD. When photons are incident when the potential of nodeA is lower than the determination threshold value due to the recharge operation and the potential at which avalanche multiplication can occur in the APD, avalanche multiplication occurs in the APD and the voltage of nodeA drops. That is, since the potential of nodeA drops at a voltage lower than the determination threshold value, the output potential from nodeB does not change even though photons are being detected. Therefore, even though avalanche multiplication is occurring, it is not determined as a signal. In particular, under high illuminance, since photons enter continuously in a short period, it becomes difficult to be determined as a signal. As a result, even though the illuminance is high, the actual number of incident photons and the output signal are likely to deviate.

[0042] On the other hand, by applying the control signal P_CLK to the switch 202 to switch the switch 202 between on and off, it becomes possible to determine as a signal even when photons continuously enter the APD in a short time. In FIG. 5, an example in which the control signal P_CLK is a pulse signal with a repetition period will be described. In other words, FIG. 5 describes a form in which the on / off of the switch 202 is switched at a predetermined clock frequency. However, the effect of suppressing an increase in power consumption of the photoelectric conversion device can also be obtained even if the pulse signal is not a signal with a repetition period.

[0043] The counter circuit 211 counts the pulse signal output from the waveform shaping unit 210 and holds the count value. Further, when the control pulse pRES is supplied via the drive line 213, the signal held in the counter circuit 211 is reset.

[0044] The selection circuit 212 is supplied with the control pulse pSEL from the vertical scanning circuit unit 110 in FIG. 3 via the drive line 214 (not shown in FIG. 3) in FIG. 4, and switches the electrical connection and disconnection between the counter circuit 211 and the signal line 113. The selection circuit 212 includes, for example, a buffer circuit for outputting a signal. The output signal OUT shown in FIG. 4 is a signal output from the pixel.

[0045] A switch such as a transistor may be arranged between the switch 202 and the APD 201 or between the photoelectric conversion element 102 and the signal processing circuit 103 to switch the electrical connection. Similarly, the supply of the voltage VH or the voltage VL supplied to the photoelectric conversion element 102 may be electrically switched using a switch such as a transistor.

[0046] In this embodiment, a configuration using the counter circuit 211 is shown. However, instead of the counter circuit 211, a photoelectric conversion device 100 that acquires pulse detection timing using a time-to-digital converter (hereinafter referred to as TDC) and a memory may be used. At this time, the generation timing of the pulse signal output from the waveform shaping unit 210 is converted into a digital signal by the TDC. A control pulse pREF (reference signal) is supplied to the TDC from the vertical scanning circuit unit 110 in FIG. 1 via a driving line for measuring the timing of the pulse signal. The TDC acquires, as a digital signal, a signal when the input timing of the signal output from each pixel via the waveform shaping unit 210 is a relative time with reference to the control pulse pREF.

[0047] Note that, as shown in FIG. 4(b), the signal generation unit 215 may be provided for each pixel. In FIG. 4(b), the circuits after the waveform shaping unit 210 shown in FIG. 4(a) and the diagram of the signal generation unit are both omitted. Note that the signal generation unit 215 in FIG. 4(a) is assumed to be for each pixel. FIG. 4(b) shows a logic circuit within the pixel that determines whether to supply a pulse signal to the switch 202. An input signal P_CLK_IN for controlling the signal P_EXP that controls the exposure period and the control signal P_CLK is input to the logic circuit. Then, an inverted signal is output. For example, when the signal P_EXP that controls the exposure period is at a low level and the input signal P_CLK_IN is at a low level, a high-level signal is output from the control signal P_CLK. That is, the switch is turned off. Also, when the signal P_EXP that controls the exposure period is at a high level and the input signal P_CLK_IN is at a high level, a low-level signal is output from the control signal P_CLK. That is, the switch is turned on. Also, when one of the signal P_EXP that controls the exposure period and the input signal P_CLK_IN is at a low level, a high-level signal is output as the control signal P_CLK. That is, the switch 202 is turned off. It is preferable to control the switch for each pixel in this way. When using the circuit diagram of FIG. 4(b), as described in Embodiment 2 to be described later, when the exposure period P becomes a low level, the control signal P_CLK is maintained at a high level. That is, the switch is turned off.

[0048] FIG. 5 is a diagram schematically showing the relationship between the control signal P_CLK of the switch, the potential of node nodeA, the potential of node nodeB, and the output signal. In the present embodiment, when the control signal P_CLK is at a high level, it becomes a state where it is difficult to supply the driving voltage VH to the APD, and when the control signal P_CLK is at a low level, the driving voltage VH is supplied to the APD. The control signal P_CLK being at a high level is, for example, 1V, and the control signal P_CLK being at a low level is, for example, 0V. When the control signal P_CLK is at a high level, the switch is turned off, and when the control signal P_CLK is at a low level, the switch is turned on. The resistance value of the switch when the control signal P_CLK is at a high level is higher than the resistance value of the switch when the control signal P_CLK is at a low level. When the control signal P_CLK is at a high level, even if avalanche multiplication occurs in the APD, the recharge operation is difficult to perform, so the potential supplied to the APD becomes a potential below the breakdown voltage of the APD. Therefore, the avalanche multiplication operation in the APD stops.

[0049] As shown in FIG. 4, it is preferable that the switch 202 is composed of one transistor and the quenching operation and the recharge operation are performed by one transistor. Thereby, it becomes possible to reduce the number of circuits as compared with the case where the quenching operation and the recharge operation are performed by different circuit elements. In particular, when each pixel has a counter circuit and reads out the signal of the SPAD for each pixel, it is preferable to reduce the circuit area used for the switch for arranging the counter circuit, and the effect of configuring the switch 202 with one transistor becomes remarkable.

[0050] At time t1, the control signal P_CLK changes from high level to low level, the switch turns on, and the recharge operation of the APD starts. As a result, the potential of the cathode of the APD transitions to a high level. Then, the potential difference between the potentials applied to the anode and cathode of the APD becomes a state where avalanche multiplication is possible. The potential of the cathode is the same as that of node A. Therefore, when the potential of the cathode transitions from low level to high level, the potential of node A at time t2 becomes equal to or higher than the determination threshold value. At this time, the pulse signal output from node B is inverted and changes from high level to low level. Thereafter, a potential difference of drive voltage VH - drive voltage VL is applied to the APD201. The control signal P_CLK becomes high level, and the switch turns off.

[0051] Next, at time t3, when photons are incident on the APD201, avalanche multiplication occurs in the APD201, an avalanche multiplication current flows through the switch 202, and the voltage of the cathode drops. That is, the voltage of node A drops. When the voltage drop amount becomes even larger and the voltage difference applied to the APD201 becomes smaller, the avalanche multiplication of the APD201 stops like at time t2, and the voltage level of node A no longer drops below a certain value. When the voltage of node A becomes lower than the determination threshold value while the voltage of node A is dropping, the voltage of node B changes from low level to high level. That is, the portion of the output waveform that exceeds the determination threshold value at node A is waveform-shaped by the waveform shaping unit 210 and output as a signal at node B. Then, it is counted by the counter circuit, and the count value of the counter signal output from the counter circuit increases by 1 LSB.

[0052] Photons are incident on the APD between time t3 and time t4, but since the switch is off and the applied voltage to the APD201 is not a potential difference that allows avalanche multiplication, the voltage level of node A does not exceed the determination threshold value.

[0053] At time t4, the control signal P_CLK changes from high level to low level, and the switch turns on. Along with this, a current that compensates for the voltage drop flows from the driving voltage VL to node nodeA, and the voltage of node nodeA transitions to the original voltage level. At this time, since the voltage of node nodeA becomes equal to or higher than the determination threshold at time t5, the pulse signal of node nodeB is inverted and changes from high level to low level.

[0054] At time t6, node nodeA stabilizes at the original voltage level, and the control signal P_CLK changes from low level to high level. Therefore, the switch turns off. Thereafter, as described from time t1 to time t6, the potentials of each node, signal line, etc. change according to the control signal P_CLK and the incidence of photons.

[0055] Hereinafter, the photoelectric conversion devices of the respective embodiments will be described.

[0056] <Embodiment 1> FIGS. 6(a), (b), and (c) are timing chart diagrams showing the relationship between the exposure period P and the control signal P_CLK in Embodiment 1. FIG. 6(a) is a diagram showing the pulse signal of the control signal P_CLK in the case of the exposure period P1. FIG. 6(b) is a diagram showing the pulse signal of the control signal P_CLK in the case of the exposure period P2, which has a longer exposure period than the exposure period P1. FIG. 6(c) is a diagram showing the pulse signal of the control signal P_CLK in the case of the exposure period P3, which has a longer exposure period than the exposure period P2.

[0057] In this embodiment, the exposure period P is, for example, the period during which the mechanical shutter or the electronic shutter is open, and the non-exposure period is, for example, the period during which the mechanical shutter or the electronic shutter is closed. Also, the exposure period P may be defined by adjusting the bias applied to the APD 201 to change the availability of photon signal acquisition. The exposure period P refers to the period during which the APD 201 is in an operable state and the APD and the signal processing circuit are in a state where they can read out signals. Here, the state where the APD and the signal processing circuit can read out signals refers to the state where the APD can be avalanche multiplied. During this time, it can also be said that the counter circuit is operating. At this time, the period of the quenching operation of the APD that is in an off state based on the incidence of photons is part of the operable state. On the other hand, the period during which light is blocked by a shutter or the like, or the period during which the APD is controlled not to be avalanche multiplied regardless of the presence or absence of photon incidence, is the non-exposure period.

[0058] As shown in FIGS. 6(a) to 6(c), in this embodiment, the pulse signal of the control signal P_CLK is changed in accordance with the exposure period P. Assume a second exposure period that is a different period from the first exposure period. The control signal is controlled such that the value obtained by dividing the number of control signals P_CLK in the first exposure period by the first exposure period and multiplying by the first exposure period is different from the value obtained by dividing the number of control signals P_CLK in the second exposure period by the second exposure period and multiplying by the first exposure period. In this embodiment, since the switch 202 is a PMOS transistor, the number of control signals P_CLK refers to the number of pulse offs within a predetermined exposure period.

[0059] Also, when comparing per unit time, the control signal P_CLK is controlled such that the average frequency of the control signal P_CLK in the first exposure period is different from the average frequency of the control signal P_CLK in a second exposure period different from the first exposure period. Here, the average frequency of the control signal P_CLK in the exposure period refers to the frequency obtained by averaging the pulse signals within the exposure period so as to be uniform. For example, when pulse signals are densely arranged in the first half of the exposure period and no pulse signals are arranged in the second half, the frequency adjusted so that pulse signals are arranged on average over the entire period is referred to as the average frequency of the control signal P_CLK in the exposure period. And in the present embodiment, when comparing the average frequencies between the first exposure period and the second exposure period, they are controlled to be different. Note that the unit time is the time during which at least two pulse signals of the control signal P_CLK enter.

[0060] In FIG. 6, even if there is a change in the exposure period P, the pulse signal of the control signal P_CLK is controlled so that the number of pulses of the pulse signal in the exposure period P is the same. Assuming that the number of pulses of the control signal P_CLK in the exposure period P1 shown in FIG. 6(a) is N. The number of pulses of the control signal P_CLK is also set to N in the exposure period P2 shown in FIG. 6(b). Furthermore, the number of pulses of the control signal P_CLK is also set to N in the exposure period P3 shown in FIG. 6(c).

[0061] With the above configuration, an increase in power consumption generated in each pixel can be prevented. Also, when the signal processing unit includes a counter circuit, it becomes possible to realize clock driving while maintaining an appropriate count upper limit value.

[0062] Hereinafter, details will be described while comparing with the comparative form of FIG. 7. FIGS. 7(a), (b), and (c) are timing chart diagrams showing the relationship between the exposure period P and the control signal P_CLK in the comparative form. In FIG. 7, for the configurations similar to those in FIG. 6, the same reference numerals as those in FIG. 6 are given and the description thereof is omitted.

[0063] In the comparative form shown in FIG. 7, even when the exposure period changes, the period of the control signal P_CLK remains the same. In the present embodiment, since the switch 202 is a PMOS transistor, the period of the control signal P_CLK refers to the period from the falling edge of the pulse to the next falling edge. For example, whether it is the exposure period P1 or the exposure period P2, the period of the control signal P_CLK is the same. Therefore, in the comparative form, when the exposure period P changes, the number of pulses of the pulse signal in the exposure period P changes. In this form, depending on the exposure period, the power consumption may increase unintentionally. In the subsequent signal processing circuit, even if the clock frequency increases, the result is not reflected, and there is a possibility that useless power consumption occurs. For example, when the signal processing circuit is a counter circuit, the clock frequency at a value larger than the readable counter upper limit value is not read as a signal. That is, at a value larger than the counter upper limit value, even if the signal is read by performing the quench operation and the recharge operation, the count value is not added in the counter circuit. Therefore, at a value larger than the counter upper limit value, useless power consumption occurs. On the other hand, if the clock frequency is increased too much, the number of photons that can be read may decrease, and the dynamic range may decrease.

[0064] On the contrary, as described above, by controlling the number of pulses of the pulse signal of the control signal P_CLK according to the exposure period, useless power consumption can be reduced.

[0065] As shown in FIG. 6, the period of the control signal P_CLK changes according to the exposure period. For example, the period of the control signal P_CLK in the exposure period P1 is shorter than that of the control signal P_CLK in the exposure period P2. In this way, in FIG. 6, even when the exposure period changes such as the exposure period P1, the exposure period P2, and the exposure period P3, the number of pulses of the control signal P_CLK is always set to N times. Thereby, an increase in power consumption can be reduced.

[0066] The number of pulses during the exposure period can be set to any value. When the subsequent signal processing circuit is a counter circuit, for example, it is preferably set to the count upper limit value of the counter circuit. Thereby, it is possible to suppress the generation of unnecessary power consumption while suppressing the reduction of the dynamic range.

[0067] It is preferable that the first pulse width in the exposure period P1 is the same as the first pulse width in the exposure period P2. Here, the first pulse width refers to the period during which the switch 202 is on by the control signal P_CLK. In the present embodiment, since the switch 202 is a PMOS transistor, the first pulse width refers to the period during which the control signal P_CLK is at the low level (first level). As described above, during the period when the control signal P_CLK is at the high level (second level), the switch is off, and it is a period in which the recharge operation is difficult to be performed by the APD. In this specification, the period during which the control signal P_CLK maintains the first level state is described as the "first pulse width", and the period during which the control signal P_CLK maintains the second level state is referred to as the "second pulse width". In FIG. 6, the first pulse width in the exposure period P1 is the same as the first pulse width in the exposure period P2, but the second pulse width in the exposure period P1 is different from the second pulse width in the exposure period P2.

[0068] Conversely, during the period when P_CLK is at the low level, the switch is turned on, and it is a period in which the recharge operation is performed by the APD. When the period during which the control signal P_CLK is at the low level becomes long, there is a possibility that the recharge operation is performed multiple times. As described with reference to FIG. 5, when the potential of the node nodeA drops in a state where the potential of the node nodeA is lower than the determination threshold value, the signal of the node B may not be inverted and may not be read out correctly as a signal value. Therefore, by making the period during which the recharge operation is performed by the APD constant even when the period of the pulse signal changes, the recharge operation is not performed multiple times during the period when the control signal P_CLK is at the low level, and the detection leakage of photons can be reduced.

[0069] FIG. 8 is an example of a flowchart of the operation of the photoelectric conversion device in the present embodiment.

[0070] In step S1, the exposure period and the number of pulses of the control signal P_CLK are set. Here, the clock frequency of the control signal P_CLK is set. In step S2, shooting is started. In step S3, it is determined whether to change the exposure period. Whether to change the exposure period can be determined based on the information obtained from the image taken previously (information of the previous frame). If the information obtained from the image is too bright according to the information of the previous frame, the exposure period is shortened, and if the information obtained from the image is too dark, the exposure period is lengthened. In addition to this, the exposure period can also be switched manually or automatically.

[0071] If it is determined to change in step S3, it is determined in step S4 whether to increase the setting of the exposure period. If it is determined to change the exposure period in step S3, the process proceeds to step S4, and if it is determined not to change the exposure period in step 3, the process proceeds to step S.

[0072] If it is determined in step S4 to increase the setting of the exposure period, the average clock frequency of the control signal P_CLK within the exposure period is lowered in step S5. At this time, control is performed so that the number of pulses of the control signal P_CLK in the exposure period does not change before and after changing the exposure period. If it is determined in step S4 not to increase (to shorten) the setting of the exposure period, the average clock frequency of the control signal P_CLK within the exposure period is set high in step S6. Also in this case, control is performed so that the number of pulses of the control signal P_CLK before changing the exposure period and the number of pulses of the control signal P_CLK in the exposure period do not change.

[0073] After step S5 or step S6, proceed to step S7 to determine whether to end the shooting. If it is determined in step S7 that the exposure period has ended, end the shooting in step S8. If it is determined in step S7 that the shooting is not ended, return to step S3 and repeat steps S3 to S7. Then, if it is determined in step S7 that the shooting is ended, end the shooting in step S8.

[0074] It is possible to operate according to the flowchart as described above.

[0075] Fig. 9 shows a modified example of the timing chart. In Fig. 6, the pulse signal P_CLK was input at a constant period, but it is not limited to this. As shown in Figs. 9(a) to 9(c), the pulse signal P_CLK may be input while changing the pulse period. Note that in Figs. 9(a) to (c), the period is changed for each pulse signal, but the period may be changed for every three or four or more pulse signals.

[0076] Also, as shown in Figs. 9(d) to 9(f), during the same period as the exposure period P1, the number of pulses in the exposure period P2 and the number of pulses in the exposure period P3 may be the same as the number of pulses in the exposure period P1. That is, the control signal P_CLK may be continuously supplied to the switch in the first half of the exposure period P2, and the control signal P_CLK may not be supplied in the second half of the exposure period P2. Even in such a case, if the number of pulses in the exposure period P2 is the same as the number of pulses in the exposure period P1, the effects of this embodiment can be obtained. Similarly, if the number of pulses in the exposure period P3 is the same as the number of pulses in the exposure period P1, the effects of this embodiment can be obtained.

[0077] <Embodiment 2> Figures 10(a), (b), and (c) are timing chart diagrams showing the relationship between the exposure period P and the control signal P_CLK in Embodiment 2. Figure 10(a) is a diagram showing the pulse of the control signal P_CLK in the case of the exposure period P1. Figure 10(b) is a diagram showing the pulse signal of the control signal P_CLK in the case of the exposure period P2, which has a longer exposure period than P1. Figure 10(c) is a diagram showing the pulse signal of the control signal P_CLK in the case of the exposure period P3, which is longer than P2.

[0078] The photoelectric conversion device of this embodiment is different from Embodiment 1 in that the pulse signal of the control signal P_CLK stops being generated at the timing when changing from the exposure period to the non-exposure period. Except for this point and the matters described below, since it is substantially the same as Embodiment 1, the same components as those in Embodiment 1 may be denoted by the same reference numerals and the description may be omitted.

[0079] In this embodiment, at the end timing of the exposure period, the control signal P_CLK is stopped at the low level. That is, the switch is kept in the off state. And at the start timing of the exposure period, the control signal P_CLK is changed to the high level.

[0080] During the non-exposure period, since photons do not enter the APD, there is no need to control the on / off of the switch. Therefore, by not changing the control signal P_CLK that controls the on / off of the switch, the power consumption caused by the on / off of the switch can be suppressed.

[0081] The end timing of the exposure period may be controlled to be synchronized with the shutter.

[0082] Also, as shown in Figure 10, although it is preferable to stop the pulse signal of the control signal P_CLK at the same time as the end timing of the exposure period for power consumption reduction, it may be slightly deviated from the end timing of the exposure period.

[0083] Also according to this embodiment, similar to Embodiment 1, an increase in power consumption can be suppressed. Further, compared to Embodiment 1, the number of times the switch is turned on and off can be reduced, so that it becomes possible to further suppress the power consumption of the photoelectric conversion device.

[0084] <Embodiment 3> Correction of the count value Nct obtained by the photoelectric conversion device according to Embodiment 1 or Embodiment 2 will be described with reference to FIGS. 11 to 15.

[0085] FIG. 11 is a diagram showing the relationship between the number of incident photons Nph and the count value Nct in each pixel of the photoelectric conversion device. Here, Nph is the number of photons actually incident on each pixel per exposure period T (number of incident photons). Further, Nct is the count value of the pulse signal output from the waveform shaping unit 210 and counted by the counter circuit 211. Here, assuming that the frequency of P_CLK is f, the number of pulses of P_CLK within the exposure period T is equal to f×T.

[0086] When the switch 202 of each pixel is controlled by a periodic pulse, the count value Nct of each pixel has characteristics like curve A in FIG. 11. That is, the count value Nct increases as the number of incident photons Nph increases and is counted with the number of pulses f×T as the upper limit. Since the recharge operation is performed once for each pulse of P_CLK, only one photon can be counted per cycle of P_CLK. Therefore, when a plurality of photons are incident per cycle of P_CLK, photons after the second one in each cycle are not counted. Therefore, in a high-luminance environment, the number of photons that are not counted even though they are incident on the pixel increases, and the count value Nct and the actual number of incident photons Nph deviate.

[0087] Therefore, in this embodiment, a correction is performed to convert the count value Nct to a value corresponding to the actual number of incident photons Nph. This correction is performed by a correction circuit 118 connected to the circuit board 21. The correction circuit 118 may be outside the photoelectric conversion unit 100 as shown in FIG. 12, or may be inside the photoelectric conversion unit 100 (for example, the signal processing circuit 103). Note that 119 in FIG. 12 is an external output circuit. The relationship between Nct and Nph is shown by the following equation based on the natural logarithm. Nct = f × T × (1 - exp(-Nph / (f × T))) Equation (1) In other words, in this correction circuit 118, when the count value Nct, the frequency f of the pulse signal, and the length T of the exposure period are explanatory variables, and the number of incident photons Nph is the target variable, the explanatory variables and the target variable are described by a relational expression based on the natural logarithm.

[0088] The corrected count value Nct can be represented by the dotted line B in FIG. 11. By performing this correction, the count value A that exhibits non-linear characteristics with respect to the number of incident photons can be corrected to the count value B corresponding to the number of incident photons having linearity.

[0089] Here, since the correction formula is determined by the value of f × T, even if the value of the exposure period T is determined for each different P_CLK frequency f, as long as the combination of f × T is constant, the value of the count value Nct with respect to the number of incident photons Nph does not change.

[0090] Also, when two types of P_CLK frequencies are mixed during one exposure period, it can be corrected by the following formula. Nct = f1 × T1 × (1 - exp(-Nph1 / (f1 × T1))) + f2 × T2 × (1 - exp(-Nph2 / (f2 × T2))) Equation (2) At this time, T1 is the period during which the pulse signal operates at the first frequency f1, T2 is the period during which the pulse signal operates at the second frequency f2, and T1 + T2 is the exposure period T. Also, Nph1 is the number of incident photons in T1, and Nph2 is the number of incident photons in T2. Each of Nph1 and Nph2 is represented by the following formula. Nph1 = Nph × T1 / (T1 + T2) Equation (3) Nph2 = Nph × T2 / (T1 + T2), Equation (4) In this way, the number of incident photons is determined by the ratio of the exposure period at each frequency to the total exposure period T.

[0091] Here, consider the case where multiple frequencies are mixed during the exposure period as shown in FIG. 13. For example, as shown in FIG. 13(a), the pulse signal input in the first half of the exposure period may have a frequency of f1, and the pulse signal input in the second half may have a frequency of f2, and the pulse signals may be input in groups for each frequency. Also, as shown in FIG. 13(b), two pulses with different frequencies may follow each other alternately, and the pulse signals with frequencies f1 and f2 may follow each other alternately one by one. FIGS. 13(a) and (b) show examples of the case where pulse signals of two types of frequencies are mixed during the exposure period, but pulse signals of three or more types of frequencies may also be mixed.

[0092] In this way, by mixing pulse signals of multiple types of frequencies during the exposure period, the slope of the count value at high illuminance becomes larger compared to the case where only a low-frequency pulse signal is input. Therefore, the harmony of the count value is maintained even at high illuminance, and the dynamic range can be expanded compared to the case where there is only one type of frequency of the pulse signal.

[0093] Furthermore, when n types of P_CLK frequencies are mixed during the exposure period, the correction formula is expressed by the following formula. n is a natural number of 2 or more. Nct = f1 × T1 × (1 - exp(-Nph1 / (f1 × T1))) + f2 × T2 × (1 - exp(-Nph2 / (f2 × T2))) + ··· + f n-1 × T n-1 × (1 - exp(-Nph n―1 / (f n-1 × T n-1 ))) + f n × T n × (1 - exp(-Nph n / (f n × T n ))), Equation (5) The sum of the periods during which the pulse signals of respective frequencies are input is equal to the exposure period T. Also, the number of incident photons during the period when the pulse signal of each frequency is input is expressed by the following formula. Nph1 = Nph × T1 / (T1 + T2 + ··· + T n―1 + T n ) Equation (6) Nph2 = Nph × T2 / (T1 + T2 + ··· + T n―1 + T n ) Equation (7) Nph n―1 = Nph × T n―1 / (T1 + T2 + ··· + T n―1 + T n ) Equation (8) Nph n = Nph × T n / (T1 + T2 + ··· + T n―1 + T n ) Equation (9) The number of incident photons is determined by the ratio of the exposure period at each frequency to the total exposure period T.

[0094] Note that the correction performed by the correction circuit 118 is not limited to performing the correction according to the above formula for each count value each time. For example, the correction circuit 118 may have a three-dimensional table of combinations of the exposure period T, the frequency f of the pulse signal, and the count value Nct. It is possible to estimate the number of incident photons Nph by selecting the value closest to the measured value from the values in the table possessed by the correction circuit 118. At this time, the values in the table are set by a relational expression based on the natural logarithm according to the combination of f and T in the same manner as the above formula.

[0095] The correction for the count value is not limited to this, and the correction steps may be reduced by performing other corrections simultaneously with this correction in the correction circuit 118. For example, for display purposes, so-called γ correction for adjusting the brightness of an image formed based on the count value may be performed.

[0096] <Embodiment 4> FIG. 14 is a block diagram showing the configuration of the light detection system 1200 according to the present embodiment. The light detection system 1200 of the present embodiment includes a photoelectric conversion device 1204. Here, as the photoelectric conversion device 1204, any of the photoelectric conversion devices described in the above embodiments can be applied. The light detection system 1200 can be used, for example, as an imaging system. Specific examples of the imaging system include a digital still camera, a digital camcorder, a surveillance camera, and the like. In FIG. 14, an example of a digital still camera is shown as the light detection system 1200.

[0097] The light detection system 1200 shown in FIG. 1 includes a photoelectric conversion device 1204, a lens 1202 that forms an optical image of a subject on the photoelectric conversion device 1204, a diaphragm 1203 for variably controlling the amount of light passing through the lens 1202, and a barrier 1201 for protecting the lens 1202. The lens 1202 and the diaphragm 1203 are an optical system that condenses light on the photoelectric conversion device 1204. A shutter is disposed between the diaphragm 1203 and the photoelectric conversion device 1204. The exposure period of the photoelectric conversion device is controlled by opening and closing the shutter.

[0098] The light detection system 1200 has a signal processing unit 1205 that processes an output signal output from the photoelectric conversion device 1204. The signal processing unit 1205 performs signal processing operations of performing various corrections and compressions on the input signal and outputting the result as necessary. The light detection system 1200 further includes a buffer memory unit 1206 for temporarily storing image data, and an external interface unit (external I / F unit) 1209 for communicating with an external computer or the like. Furthermore, the light detection system 1200 includes a recording medium 1211 such as a semiconductor memory for recording or reading imaging data, and a recording medium control interface unit (recording medium control I / F unit) 1210 for recording or reading from the recording medium 1211. The recording medium 1211 may be built into the light detection system 1200 or may be detachable. Also, the communication between the recording medium control I / F unit 1210 and the recording medium 1211 and the communication from the external I / F unit 1209 may be performed wirelessly.

[0099] Furthermore, the light detection system 1200 includes an overall control and arithmetic unit 1208 that performs various operations and controls the entire digital still camera, and a timing generation unit 1207 that outputs various timing signals to the photoelectric conversion device 1204 and the signal processing unit 1205. Here, the timing signals and the like may be input from the outside, and the light detection system 1200 may have at least the photoelectric conversion device 1204 and a signal processing unit 1205 that processes the output signal output from the photoelectric conversion device 1204. As described in the fourth embodiment, the timing generation unit 1207 may be mounted on the photoelectric conversion device. The overall control and arithmetic unit 1208 and the timing generation unit 1207 may be configured to implement part or all of the control functions of the photoelectric conversion device 1204.

[0100] The photoelectric conversion device 1204 outputs an image signal to the signal processing unit 1205. The signal processing unit 1205 performs predetermined signal processing on the image signal output from the photoelectric conversion device 1204 and outputs image data. Further, the signal processing unit 1205 generates an image using the image signal. Further, the signal processing unit 1205 may perform a distance measurement operation on the signal output from the photoelectric conversion device 1204. Note that the signal processing unit 1205 and the timing generation unit 1207 may be mounted on the photoelectric conversion device. That is, the signal processing unit 1205 and the timing generation unit 1207 may be provided on the substrate on which the pixels are arranged, or may be provided on a separate substrate. By configuring an imaging system using the photoelectric conversion device of each of the above-described embodiments, an imaging system capable of acquiring higher-quality images can be realized.

[0101] <Embodiment 5> FIG. 15 is a block diagram showing a configuration example of a distance image sensor, which is an electronic device using the photoelectric conversion device described in the foregoing embodiment.

[0102] As shown in FIG. 15, the distance image sensor 401 includes an optical system 407, a photoelectric conversion device 408, an image processing circuit 404, a monitor 405, and a memory 406. The distance image sensor 401 can obtain a distance image corresponding to the distance to the subject by projecting light from the light source device 409 toward the subject and receiving the light (modulated light or pulsed light) reflected by the surface of the subject.

[0103] The optical system 407 is configured to have one or more lenses, guide image light (incident light) from the subject to the photoelectric conversion device 408, and form an image on the light receiving surface (sensor unit) of the photoelectric conversion device 408.

[0104] As the photoelectric conversion device 408, the photoelectric conversion devices of the above-described embodiments are applicable, and a distance signal indicating the distance obtained from the light reception signal output from the photoelectric conversion device 408 is supplied to the image processing circuit 404.

[0105] The image processing circuit 404 performs image processing for constructing a distance image based on the distance signal supplied from the photoelectric conversion device 408. Then, the distance image (image data) obtained by the image processing is supplied to the monitor 405 for display or supplied to the memory 406 for storage (recording).

[0106] In the distance image sensor 401 configured as described above, by applying the above-described photoelectric conversion device, it is possible to obtain a more accurate distance image, for example, as the characteristics of the pixels are improved.

[0107] <Embodiment 6> The technology according to the present disclosure (this technology) can be applied to various products. For example, the technology according to the present disclosure may be applied to an endoscopic surgery system.

[0108] FIG. 16 is a diagram showing an example of a schematic configuration of an endoscopic surgery system to which the technology according to the present disclosure (this technology) can be applied.

[0109] In FIG. 16, a state where an operator (doctor) 1131 is performing a surgery on a patient 1132 on a patient bed 1133 using an endoscopic surgery system 1003 is illustrated. As shown, the endoscopic surgery system 1003 includes an endoscope 1100, a surgical instrument 1110, and a cart 1134 equipped with various devices for endoscopic surgery.

[0110] The endoscope 1100 includes a lens barrel 1101 whose region of a predetermined length from the tip is inserted into the body cavity of the patient 1132, and a camera head 1102 connected to the base end of the lens barrel 1101. In the illustrated example, an endoscope 1100 configured as a so-called rigid endoscope having a rigid lens barrel 1101 is shown, but the endoscope 1100 may be configured as a so-called flexible endoscope having a flexible lens barrel.

[0111] An opening in which an objective lens is fitted is provided at the tip of the lens barrel 1101. A light source device 1203 is connected to the endoscope 1100, and the light generated by the light source device 1203 is guided to the tip of the lens barrel by a light guide extending inside the lens barrel 1101 and irradiated toward an observation target in the body cavity of the patient 1132 through the objective lens. Note that the endoscope 1100 may be a direct vision endoscope, a forward oblique endoscope, or a side vision endoscope.

[0112] An optical system and a photoelectric conversion device are provided inside the camera head 1102, and the reflected light (observation light) from the observation target is condensed onto the photoelectric conversion device by the optical system. The observation light is photoelectrically converted by the photoelectric conversion device, and an electrical signal corresponding to the observation light, that is, an image signal corresponding to the observation image, is generated. As the photoelectric conversion device, the photoelectric conversion devices described in the foregoing embodiments can be used. The image signal is transmitted as RAW data to a camera control unit (CCU) 1135.

[0113] The CCU 1135 is composed of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), etc., and comprehensively controls the operations of the endoscope 1100 and the display device 1136. Further, the CCU 1135 receives an image signal from the camera head 1102, and performs various image processes for displaying an image based on the image signal, such as development processing (demosaicing processing).

[0114] The display device 1136 displays an image based on the image signal on which image processing has been performed by the CCU 1135 under the control of the CCU 1135.

[0115] The light source device 1203 is composed of a light source such as an LED (Light Emitting Diode), and supplies irradiation light for photographing the surgical site, etc. to the endoscope 1100.

[0116] The input device 1137 is an input interface for the endoscope surgical system 1003. The user can input various information and give instructions to the endoscope surgical system 1003 via the input device 1137.

[0117] The treatment instrument control device 1138 controls the driving of the energy treatment instrument 1112 for tissue cauterization, incision, or blood vessel sealing, etc.

[0118] The light source device 1203 that supplies irradiation light for photographing the surgical site with the endoscope 1100 can be configured from, for example, an LED, a laser light source, or a white light source composed of a combination thereof. When a white light source is configured by a combination of RGB laser light sources, since the output intensity and output timing of each color (each wavelength) can be controlled with high precision, the white balance of the captured image can be adjusted in the light source device 1203. Further, in this case, the laser light from each of the RGB laser light sources is irradiated to the observation target in a time-division manner, and by controlling the driving of the imaging element of the camera head 1102 in synchronization with the irradiation timing, it is also possible to capture images corresponding to each of RGB in a time-division manner. According to this method, a color image can be obtained without providing a color filter on the imaging element.

[0119] Further, the driving of the light source device 1203 may be controlled so as to change the intensity of the output light at predetermined time intervals. By controlling the driving of the imaging element of the camera head 1102 in synchronization with the timing of the change in the intensity of the light and acquiring images in a time-division manner and synthesizing the images, it is possible to generate a high-dynamic range image without so-called black crush and white clip.

[0120] Further, the light source device 1203 may be configured to be able to supply light in a predetermined wavelength band corresponding to special light observation. In special light observation, for example, the wavelength dependence of light absorption in body tissue is utilized. Specifically, by irradiating light in a narrow band compared to the irradiation light (i.e., white light) during normal observation, a predetermined tissue such as blood vessels in the mucosal surface layer can be photographed with high contrast. Alternatively, in special light observation, fluorescence observation may be performed to obtain an image by fluorescence generated by irradiating excitation light. In fluorescence observation, it is possible to irradiate the body tissue with excitation light and observe the fluorescence from the body tissue, or to locally inject a reagent such as indocyanine green (ICG) into the body tissue and irradiate the body tissue with excitation light corresponding to the fluorescence wavelength of the reagent to obtain a fluorescence image. The light source device 1203 can be configured to be able to supply such narrow-band light and / or excitation light corresponding to special light observation.

[0121] <Embodiment 7> The optical detection system and the moving body according to this embodiment will be described with reference to FIGS. 17 and 18. FIG. 17 is a schematic diagram showing a configuration example of the optical detection system and the moving body according to this embodiment. FIG. 18 is a flowchart showing the operation of the optical detection system according to this embodiment. In this embodiment, an in-vehicle camera is shown as an example of the optical detection system.

[0122] FIG. 17 shows an example of a vehicle system and an optical detection system that performs imaging mounted thereon. The optical detection system 1301 includes a photoelectric conversion device 1302, an image preprocessing unit 1315, an integrated circuit 1303, and an optical system 1314. The optical system 1314 forms an optical image of a subject on the photoelectric conversion device 1302. The photoelectric conversion device 1302 converts the optical image of the subject formed by the optical system 1314 into an electrical signal. The photoelectric conversion device 1302 is any of the photoelectric conversion devices of the above-described embodiments. The image preprocessing unit 1315 performs predetermined signal processing on the signal output from the photoelectric conversion device 1302. The function of the image preprocessing unit 1315 may be incorporated in the photoelectric conversion device 1302. At least two sets of the optical system 1314, the photoelectric conversion device 1302, and the image preprocessing unit 1315 are provided in the optical detection system 1301, and the output from each set of the image preprocessing units 1315 is input to the integrated circuit 1303.

[0123] The integrated circuit 1303 is an integrated circuit for an imaging system application, and includes an image processing unit 1304 including a memory 1305, an optical distance measurement unit 1306, a distance measurement calculation unit 1307, an object recognition unit 1308, and an abnormality detection unit 1309. The image processing unit 1304 performs image processing such as development processing and defect correction on the output signal of the image preprocessing unit 1315. The memory 1305 stores the primary storage of the captured image and the defect positions of the captured pixels. The optical distance measurement unit 1306 performs focusing and distance measurement of the subject. The distance measurement calculation unit 1307 calculates distance measurement information from a plurality of image data acquired by a plurality of photoelectric conversion devices 1302. The object recognition unit 1308 recognizes subjects such as cars, roads, signs, and people. When the abnormality detection unit 1309 detects an abnormality in the photoelectric conversion device 1302, it reports the abnormality to the main control unit 1313.

[0124] The integrated circuit 1303 may be implemented by dedicatedly designed hardware, by software modules, or by a combination thereof. It may also be implemented by an FPGA (Field Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), etc., or by a combination thereof.

[0125] The main control unit 1313 comprehensively controls the operations of the optical detection system 1301, the vehicle sensor 1310, the control unit 1320, etc. It is also possible to adopt a method in which the optical detection system 1301, the vehicle sensor 1310, and the control unit 1320 do not have a main control unit 1313 and each has a communication interface, and each performs transmission and reception of control signals via a communication network (for example, the CAN standard).

[0126] The integrated circuit 1303 has a function of receiving a control signal from the main control unit 1313 or transmitting a control signal and a set value to the photoelectric conversion device 1302 by its own control unit.

[0127] The optical detection system 1301 is connected to the vehicle sensor 1310 and can detect the driving state of the host vehicle such as vehicle speed, yaw rate, and steering angle, as well as the state of the external environment of the host vehicle and other vehicles / obstacles. The vehicle sensor 1310 is also a distance information acquisition means for acquiring distance information to an object. In addition, the optical detection system 1301 is connected to a driving support control unit 1311 that performs various driving supports such as automatic steering, automatic cruise, and collision prevention functions. In particular, regarding the collision determination function, based on the detection results of the optical detection system 1301 and the vehicle sensor 1310, the collision estimation and the presence or absence of a collision with other vehicles / obstacles are determined. Thereby, avoidance control when a collision is estimated and activation of a safety device at the time of a collision are performed.

[0128] In addition, the light detection system 1301 is also connected to an alarm device 1312 that alerts the driver based on the determination result of the collision determination unit. For example, when the collision determination unit determines that there is a high possibility of a collision, the main control unit 1313 performs vehicle control to avoid the collision and reduce the damage, such as applying the brakes, returning the accelerator, and suppressing the engine output. The alarm device 1312 warns the user by sounding an alarm such as a sound, displaying alarm information on the display screen of a display unit such as a car navigation system or a meter panel, or applying vibration to the seat belt or the steering wheel.

[0129] In this embodiment, the light detection system 1301 captures an image of the surroundings of the vehicle, for example, the front or the rear. FIG. 17(b) shows an example of the arrangement of the light detection system 1301 when imaging the front of the vehicle with the light detection system 1301.

[0130] The two photoelectric conversion devices 1302 are arranged in front of the vehicle 1300. Specifically, when the center line with respect to the advancing / retreating direction or the outer shape (for example, the vehicle width) of the vehicle 1300 is regarded as the axis of symmetry, and the two photoelectric conversion devices 1302 are arranged symmetrically with respect to the axis of symmetry, it is preferable for acquiring distance information between the vehicle 1300 and the object to be photographed and determining the possibility of a collision. In addition, the photoelectric conversion device 1302 is preferably arranged so as not to obstruct the driver's view when the driver visually recognizes the situation outside the vehicle 1300 from the driver's seat. The alarm device 1312 is preferably arranged so as to easily enter the driver's field of view.

[0131] Next, the fault detection operation of the photoelectric conversion device 1302 in the light detection system 1301 will be described with reference to FIG. 18. The fault detection operation of the photoelectric conversion device 1302 is performed according to steps S1410 to S1480 shown in FIG. 18.

[0132] Step S1410 is a step of setting the photoelectric conversion device 1302 at startup. That is, settings for the operation of the photoelectric conversion device 1302 are transmitted from the outside of the light detection system 1301 (for example, the main control unit 1313) or from inside the light detection system 1301, and the imaging operation and the fault detection operation of the photoelectric conversion device 1302 are started.

[0133] Next, in step S1420, a pixel signal is acquired from the effective pixels. Also, in step S1430, an output value from the failure detection pixels provided for failure detection is acquired. These failure detection pixels include a photoelectric conversion element, just like the effective pixels. A predetermined voltage is written to this photoelectric conversion element. The failure detection pixels output a signal corresponding to the voltage written to this photoelectric conversion element. Note that steps S1420 and S1430 may be reversed.

[0134] Next, in step S1440, a comparison is made between the output expected value of the failure detection pixels and the output value from the actual failure detection pixels. As a result of the comparison in step S1440, if the output expected value and the actual output value match, the process proceeds to step S1450, where it is determined that the imaging operation is being performed normally, and the processing steps proceed to step S1460. In step S1460, the pixel signals of the scanning line are transmitted to the memory 1305 for primary storage. Thereafter, the process returns to step S1420 to continue the failure detection operation. On the other hand, as a result of the comparison in step S1440, if the output expected value and the actual output value do not match, the processing steps proceed to step S1470. In step S1470, it is determined that there is an abnormality in the imaging operation, and an alarm is reported to the main control unit 1313 or the alarm device 1312. The alarm device 1312 causes the display unit to display that an abnormality has been detected. Thereafter, in step S1480, the photoelectric conversion device 1302 is stopped, and the operation of the light detection system 1301 is terminated.

[0135] Note that in this embodiment, an example in which the flowchart is looped for each row is illustrated, but the flowchart may be looped for every plurality of rows, or the failure detection operation may be performed for each frame. The alarm reporting in step S1470 may be notified outside the vehicle via a wireless network.

[0136] In addition, in this embodiment, control for not colliding with other vehicles has been described, but it is also applicable to control for automatically driving while following other vehicles, control for automatically driving so as not to deviate from the lane, and the like. Furthermore, the light detection system 1301 can be applied not only to vehicles such as automobiles, but also to moving bodies (moving devices) such as ships, airplanes, or industrial robots. In addition, it can be applied not only to moving bodies, but also to devices that widely utilize object recognition, such as advanced road traffic systems (ITS).

[0137] The photoelectric conversion device of the present invention may further be configured to be able to acquire various information such as distance information.

[0138] <Embodiment 8> FIG. 19(a) illustrates glasses 1600 (smart glasses) according to one application example. The glasses 1600 have a photoelectric conversion device 1602. The photoelectric conversion device 1602 is the photoelectric conversion device described in each of the above embodiments. Also, a display device including a light emitting device such as an OLED or an LED may be provided on the back side of the lens 1601. There may be one or a plurality of photoelectric conversion devices 1602. Also, a plurality of types of photoelectric conversion devices may be combined and used. The arrangement position of the photoelectric conversion device 1602 is not limited to FIG. 19(a).

[0139] The glasses 1600 further include a control device 1603. The control device 1603 functions as a power supply that supplies power to the photoelectric conversion device 1602 and the above display device. Also, the control device 1603 controls the operations of the photoelectric conversion device 1602 and the display device. An optical system for condensing light onto the photoelectric conversion device 1602 is formed in the lens 1601.

[0140] FIG. 19(b) illustrates glasses 1610 (smart glasses) according to one application example. The glasses 1610 have a control device 1612, and a photoelectric conversion device corresponding to the photoelectric conversion device 1602 and a display device are mounted on the control device 1612. An optical system for projecting light emission from the photoelectric conversion device and the display device formed in the control device 1612 and an image is projected onto the lens 1611. The control device 1612 functions as a power source for supplying power to the photoelectric conversion device and the display device, and controls the operations of the photoelectric conversion device and the display device. The control device may have a line-of-sight detection unit for detecting the wearer's line of sight. Infrared rays may be used for detecting the line of sight. The infrared light emitting unit emits infrared light to the eyeball of the user who is gazing at the display image. An imaging unit having a light receiving element detects the reflected light of the emitted infrared light from the eyeball, and thus an imaging image of the eyeball is obtained. By having a reduction means for reducing the light from the infrared light emitting unit to the display unit in a plan view, a decrease in image quality is reduced.

[0141] The user's line of sight with respect to the display image is detected from the imaging image of the eyeball obtained by imaging infrared light. Any known method can be applied to the line-of-sight detection using the imaging image of the eyeball. As an example, a line-of-sight detection method based on a Purkinje image by reflection of irradiation light on the cornea can be used.

[0142] More specifically, a line-of-sight detection process based on the pupil corneal reflex method is performed. Using the pupil corneal reflex method, a line-of-sight vector representing the orientation (rotation angle) of the eyeball is calculated based on the image of the pupil and the Purkinje image included in the imaging image of the eyeball, and thus the user's line of sight is detected.

[0143] The display device of the present embodiment has a photoelectric conversion device having a light receiving element, and may control the display image of the display device based on the user's line-of-sight information from the photoelectric conversion device.

[0144] Specifically, the display device determines a first viewing area that the user is gazing at and a second viewing area other than the first viewing area based on the line-of-sight information. The first viewing area and the second viewing area may be determined by the control device of the display device, or the display device may receive those determined by an external control device. In the display area of the display device, the display resolution of the first viewing area may be controlled to be higher than that of the second viewing area. That is, the resolution of the second viewing area may be made lower than that of the first viewing area.

[0145] Further, the display area has a first display area and a second display area different from the first display area, and an area with a higher priority may be determined from the first display area and the second display area based on the line-of-sight information. The first viewing area and the second viewing area may be determined by the control device of the display device, or the display device may receive those determined by an external control device. The resolution of the area with a higher priority may be controlled to be higher than that of the area other than the area with a higher priority. That is, the resolution of the area with a relatively lower priority may be made lower.

[0146] Note that AI may be used to determine the first viewing area or the area with a higher priority. AI may be a model configured to estimate the angle of the line of sight and the distance to the target at the tip of the line of sight from the image of the eyeball using the image of the eyeball and the direction in which the eyeball in the image is actually looking as teacher data. The AI program may be possessed by the display device, the photoelectric conversion device, or an external device. When an external device possesses it, it is transmitted to the display device via communication.

[0147] When performing display control based on visual recognition detection, it is preferably applicable to smart glasses further having a photoelectric conversion device that images the outside. The smart glasses can display the captured external information in real time.

[0148] <Other Embodiments> As described above, each embodiment has been described, but the present invention is not limited to these embodiments, and various changes and modifications are possible. Also, each embodiment is applicable to each other.

Explanation of Reference Numerals

[0149] 201 APD 103 signal processing circuit 202 switch 215 signal generation unit P exposure period

Claims

1. an avalanche photodiode including an anode and a cathode; a switch connected to one of the anode and the cathode and receiving a pulse signal; A photoelectric conversion device characterized in that a value obtained by dividing the number of the pulse signals in a first exposure period by the first exposure period is different from a value obtained by dividing the number of the pulse signals in a second exposure period having a length different from that of the first exposure period by the second exposure period.

2. 2. The photoelectric conversion device according to claim 1, wherein the pulse signal is a signal having a repetitive period.

3. 3. The photoelectric conversion device according to claim 1, wherein the number of the pulse signals in the first exposure period is the same as the number of the pulse signals in the second exposure period.

4. the first exposure period is shorter than the second exposure period; 4. The photoelectric conversion device according to claim 1, wherein a period of the pulse signal in the first exposure period is shorter than a period of the pulse signal in the second exposure period.

5. 5. The photoelectric conversion device according to claim 1, wherein a first pulse width of the first level of the pulse signal in the first exposure period is the same as a first pulse width of the first level of the pulse signal in the second exposure period.

6. 6. The photoelectric conversion device according to claim 5, wherein a second pulse width of the second level of the pulse signal in the first exposure period is different from a second pulse width of the second level of the pulse signal in the second exposure period.

7. 7. The photoelectric conversion device according to claim 1, wherein the switch functions as a quenching element.

8. the switch is a MOS transistor; one node of the switch is connected to the cathode of the avalanche photodiode; The other node of the switch is connected to a power supply line to which a drive voltage is applied, 8. The photoelectric conversion device according to claim 1, wherein the pulse signal is supplied to a gate electrode of the switch.

9. A signal processing circuit having a waveform shaping unit and a counter circuit, the waveform shaping unit is connected to one of the anode and the cathode; 9. The photoelectric conversion device according to claim 1, wherein a signal output from the waveform shaping section is input to the counter circuit.

10. 10. The photoelectric conversion device according to claim 9, wherein the number of the pulse signals in the first exposure period is equal to an upper limit value of the counter circuit.

11. A correction circuit is provided.

11. The photoelectric conversion device according to claim 9, wherein a count value output from the counter circuit is input to the correction circuit.

12. The photoelectric conversion device according to claim 11, wherein the correction circuit aims to describe the explanatory variables and the objective variable by a relational equation based on natural logarithms when the count value, the frequency of the pulse signal in the first exposure period, and the length of the first exposure period are explanatory variables and the number of incident photons is an objective variable.

13. The correction circuit, when the count value is input, assumes that Nct, the number of incident photons is Nph, the frequency of the pulse signal in the first exposure period is f, and the first exposure period is T. Nct=f×T×(1-exp(-Nph / (f×T))) 13. The photoelectric conversion device according to claim 11, wherein the number of incident photons Nph satisfies the relationship:

14. The correction circuit detects that the pulse signal has n kinds of frequencies, The first frequency of the pulse signal is f １ , the second frequency is f ２ , the n-1th frequency is f ｎ－１ , the nth frequency is f ｎ (n is a natural number equal to or greater than 2), The frequency of the pulse signal is f １ The period T １ , the frequency of the pulse signal is f ２ The period T ２ , the frequency of the pulse signal is f ｎ－１ The period T ｎ－１ , the frequency of the pulse signal is f ｎ The period T ｎ With respect to the input of the count value, Nct=f １ ×T １ ×(1-exp(-Nph １ / (f １ ×T １ )))+f ２ ×T ２ ×(1-exp(-Nph ２ / (f ２ ×T ２ )))+・・・+f ｎ－１ ×T ｎ－１ ×(1-exp(-Nph ｎ―１ / (f ｎ－１ ×T ｎ－１ )))+f ｎ ×T ｎ ×(1-exp(-Nph ｎ / (f ｎ ×T ｎ ))) 12. The photoelectric conversion device according to claim 11, wherein the number of incident photons Nph satisfies the following relationship:

15. 14. The photoelectric conversion device according to claim 1, wherein the pulse signal is not input to the switch during a period other than the first exposure period and the second exposure period.

16. 15. The photoelectric conversion device according to claim 1, wherein a plurality of the avalanche photodiodes are arranged in a two-dimensional array in a plan view.

17. The photoelectric conversion device according to claim 1 , and a signal processing unit that processes a signal output from the photoelectric conversion device.

18. The photoelectric conversion device according to claim 1 , a signal processing unit that processes a signal output from the photoelectric conversion device; a shutter for controlling incidence of light into the photoelectric conversion device; The optical detection system, wherein the first exposure period and the second exposure period are periods during which the shutter is open.

19. The photoelectric conversion device according to claim 1 , a distance information acquisition means for acquiring distance information to an object from distance measurement information based on a signal from the photoelectric conversion device, A moving body further comprising a control means for controlling the moving body based on the distance information.

Images