Photoelectric converter, light detection system

Abstract

This provides a photoelectric converter that can control specific pulse signals when there are multiple exposure periods. [Solution] The avalanche photodiode includes an anode and a cathode, and a switch connected to one of the nodes, the anode and the cathode, to which a pulse signal is input, wherein the value obtained by dividing the number of pulse signals in a first exposure period by the first exposure period is different from the value obtained by dividing the number of pulse signals in a second exposure period having a different length from the first exposure period by the second exposure period.

Description

[Technical Field]

[0001] The present invention relates to a photoelectric conversion device and a photodetection system. [Background technology]

[0002] A photoelectric conversion device is known that includes a pixel array formed such that pixels containing multiple avalanche photodiodes (APDs) are arranged in a two-dimensional array in a planar manner. In each pixel, in the PN junction region within the semiconductor region, the photocharge caused by a single photon undergoes avalanche multiplication.

[0003] Patent Document 1 discloses a pixel having an APD, comprising the APD, a quench circuit connected to the APD, a signal control circuit that receives the signal output from the APD, and a pulse generation circuit connected to the quench circuit and the signal control circuit. The pulse generation circuit controls the on / off state of the quench circuit. It also discloses resetting the output signal for each pulse signal to output a pulse signal corresponding to the input photon even under high brightness conditions. [Prior art documents] [Patent Documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2020-123847 [Overview of the Initiative] [Problems that the invention aims to solve]

[0005] Patent Document 1 does not disclose anything regarding the number or period of pulse signals within the exposure period when the exposure period differs. Patent Document 1 leaves room for consideration of controlling the pulse signal in relation to the exposure period. [Means for solving the problem]

[0006] A photoelectric conversion device according to one embodiment comprises an avalanche photodiode including an anode and a cathode, and a switch connected to one of the nodes, the anode and the cathode, to which a pulse signal is input, wherein the value obtained by dividing the number of pulse signals in a first exposure period by the first exposure period and multiplying by the first exposure period is different from the value obtained by dividing the number of pulse signals in a second exposure period having a different length from the first exposure period by the second exposure period and multiplying by the first exposure period. [Effects of the Invention]

[0007] According to the present invention, it is possible to provide a photoelectric conversion device that can control specific pulse signals when there are multiple exposure periods. [Brief explanation of the drawing]

[0008] [Figure 1] Diagram showing the configuration of a photoelectric converter. [Figure 2] Example of sensor board layout [Figure 3] Circuit board layout example [Figure 4] Block diagram including the equivalent circuit of the photoelectric conversion element [Figure 5] This diagram shows the relationship between APD operation and output signals. [Figure 6] Timing chart of control pulses of the photoelectric converter according to Embodiment 1 [Figure 7] Timing chart of control pulses of photoelectric converters in comparative configurations. [Figure 8] A flowchart illustrating an example of the operating mode of the photoelectric converter according to Embodiment 1. [Figure 9] Timing chart of control pulses of a photoelectric converter according to a modified example of Embodiment 1. [Figure 10] Timing chart of control pulses of the photoelectric converter according to Embodiment 2 [Figure 11] This figure shows the relationship between the number of incident photons per exposure period and the count value of the photoelectric conversion device according to Embodiment 3. [Figure 12]Arrangement example of the circuit board of the photoelectric conversion device according to Embodiment 3 [Figure 13] Timing chart of the control pulse of the photoelectric conversion device according to Embodiment 3 [Figure 14] Block diagram of the light detection system of Embodiment 4 [Figure 15] Block diagram of the light detection system of Embodiment 5 [Figure 16] Block diagram of the light detection system of Embodiment 6 [Figure 17] Block diagram of the light detection system of Embodiment 7 [Figure 18] Flowchart of the light detection system of Embodiment 7 [Figure 19] Diagram showing a specific example of the electronic device of Embodiment 8

Embodiments for Carrying Out the Invention

[0009] The following embodiments are for embodying the technical idea of the present invention and do not limit the present invention. The sizes and positional relationships of the members shown in each drawing may be exaggerated for clarity of explanation. In the following description, the same components may be denoted by the same reference numerals and the description thereof may be omitted.

[0010] Using FIGS. 1 to 4, the configuration common to the photoelectric conversion devices in each embodiment will be described. The photoelectric conversion device has a SPAD pixel including an avalanche diode. The conductivity type of the charge used as the signal charge among the charge pairs generated in the avalanche diode is called the first conductivity type. The first conductivity type refers to the conductivity type in which charges having the same polarity as the signal charge are majority carriers. Also, the conductivity type opposite to the first conductivity type is called the second conductivity type. Hereinafter, an example in which the signal charge is an electron, the first conductivity type is N-type, and the second conductivity type is P-type will be described, but the signal charge may be a hole, the first conductivity type may be P-type, and the second conductivity type may be N-type.

[0011] When the signal charge is an electron, the signal is read from the cathode of the APD; however, when the signal charge is a hole, the signal is read from the anode of the APD. Therefore, the relationship between the cathode and anode of the APD is reversed.

[0012] In this specification, "planar view" refers to viewing the semiconductor layer on which the photoelectric conversion element described later is located from a direction perpendicular to the light incident surface. A cross-section refers to a surface perpendicular to the light incident surface of the semiconductor layer on which the photoelectric conversion element is located. If the light incident surface of the semiconductor layer is rough when viewed microscopically, the planar view is defined based on the light incident surface of the semiconductor layer as viewed macroscopically.

[0013] First, we will describe the configuration common to each embodiment.

[0014] Figure 1 shows the configuration of a photoelectric converter 100 according to an embodiment of the present invention. In the following description, the case in which the photoelectric converter 100 is a stacked type photoelectric converter will be used as an example. That is, a photoelectric converter configured by stacking and electrically connecting two substrates, a sensor substrate 11 and a circuit substrate 21, will be described as an example. However, the photoelectric converter is not limited to this. For example, it may be a photoelectric converter in which the configuration included in the sensor substrate 11 and the configuration included in the circuit substrate are arranged on a common semiconductor layer, as described below. In the following description, a photoelectric converter in which the configuration included in the sensor substrate 11 and the configuration included in the circuit substrate are arranged on a common semiconductor layer will also be called a non-stacked photoelectric converter.

[0015] The sensor substrate 11 has a first semiconductor layer having a photoelectric conversion element 102 (described later) and a first wiring structure. The circuit board 21 has a second semiconductor layer having circuits such as a signal processing circuit 103 (described later) and a second wiring structure. The photoelectric conversion device 100 is constructed by stacking the second semiconductor layer, the second wiring structure, the first wiring structure, and the first semiconductor layer in that order.

[0016] Figure 1 illustrates a back-illuminated photoelectric converter in which light is incident from the first surface, and the circuit board is located on the second surface, which is opposite to the first surface. In the case of a non-stacked photoelectric converter, the side on which the transistors of the signal processing circuit are located is called the second surface. In the case of a back-illuminated photoelectric converter, the first surface, which is opposite to the second surface of the semiconductor layer, is the light incident surface. In the case of a front-illuminated photoelectric converter, the second surface of the semiconductor layer is the light incident surface.

[0017] In the following description, the sensor substrate 11 and the circuit board 21 are explained using diced chips, but they are not limited to chips. For example, each substrate may be a wafer. Furthermore, each substrate may be stacked in wafer form and then diced, or the chips may be stacked and bonded together after being formed into chips.

[0018] The sensor board 11 is provided with a pixel region 12, and the circuit board 21 is provided with a circuit region 22 for processing signals detected in the pixel region 12.

[0019] Figure 2 shows an example of the arrangement of the sensor substrate 11. Pixels 101, each having a photoelectric conversion element 102 including an avalanche photodiode (hereinafter referred to as APD), are arranged in a two-dimensional array in a planar view, forming a pixel region 12.

[0020] Pixel 101 is typically a pixel used to form an image, but when used in TOF (Time of Flight), it does not necessarily have to form an image. In other words, pixel 101 may be a pixel used to measure the time and amount of light that arrives.

[0021] Figure 3 is a diagram of the circuit board 21. It includes a signal processing circuit 103 for processing the charge photoelectrically converted by the photoelectric conversion element 102 in Figure 2, a readout circuit 112, a control pulse generation unit 115, a horizontal scanning circuit unit 111, a signal line 113, and a vertical scanning circuit unit 110.

[0022] The photoelectric conversion element 102 in Figure 2 and the signal processing circuit 103 in Figure 3 are electrically connected via connection wiring provided for each pixel.

[0023] The vertical scanning circuit section 110 receives control pulses supplied from the control pulse generation section 115 and supplies control pulses to each pixel. Logic circuits such as a shift register and an address decoder are used in the vertical scanning circuit section 110.

[0024] The control pulse generation unit 115 includes a signal generation unit 215 that generates a control signal P_CLK for a switch, which will be described later. The signal generation unit 215 generates a pulse signal to control the switch, as will be described later. For example, as shown in Figure 4(a), the signal generation unit 215 may generate a common control signal P_CLK for multiple pixels in a pixel region, or as shown in Figure 4(b), it may generate a control signal P_CLK for each pixel. When generating a common pulse signal P_CLK, at least one of the period, number of pulses, and pulse width of the pulse signal P_EXP that controls the exposure period is matched to the exposure period when it is generated in common. When controlling the control signal P_CLK for each pixel, the signal can be generated using both the input signal P_CLK_IN output from the control pulse generation unit 115 and the signal P_EXP that controls the exposure period. The control pulse generation unit 115 preferably has, for example, a frequency divider circuit. This makes control simpler and reduces the increase in the number of elements.

[0025] The signal output from the photoelectric conversion element 102 of the pixel is processed by the signal processing circuit 103. The signal processing circuit 103 is equipped with a counter and memory, and digital values ​​are stored in the memory.

[0026] The horizontal scanning circuit unit 111 inputs control pulses to the signal processing circuit 103 to sequentially select each column in order to read the signal from the memory of each pixel in which the digital signal is held.

[0027] For the selected column, a signal is output from the signal processing circuit 103 of the pixel selected by the vertical scanning circuit unit 110 to the signal line 113.

[0028] The signal output to signal line 113 is output via output circuit 114 to an external recording unit or signal processing unit of the photoelectric converter 100.

[0029] In Figure 2, the arrangement of photoelectric conversion elements in the pixel region may be one-dimensional. Furthermore, it is possible to obtain the effects of the present invention even with just one pixel, and the present invention is included even in the case of one pixel. However, the power consumption reduction effect of this embodiment is easier to obtain with a photoelectric conversion device having multiple pixels. The function of the signal processing unit does not necessarily need to be provided for each photoelectric conversion element; for example, one signal processing unit may be shared by multiple photoelectric conversion elements, and signal processing may be performed sequentially.

[0030] As shown in Figures 2 and 3, multiple signal processing circuits 103 are arranged in the region that overlaps with the pixel region 12 in a plan view. Then, the vertical scanning circuit section 110, the horizontal scanning circuit section 111, the readout circuit 112, the output circuit 114, and the control pulse generation section 115 are arranged so as to overlap between the edge of the sensor substrate 11 and the edge of the pixel region 12 in a plan view. In other words, the sensor substrate 11 has a pixel region 12 and a non-pixel region arranged around the pixel region 12. Then, the vertical scanning circuit section 110, the horizontal scanning circuit section 111, the readout circuit 112, the output circuit 114, and the control pulse generation section 115 are arranged in the region that overlaps with the non-pixel region in a plan view.

[0031] Note that the arrangement of the signal line 113, the read circuit 112, and the output circuit 114 is not limited to Figure 3. For example, the signal line 113 may be arranged extending in the row direction, and the read circuit 112 may be placed at the end of the signal line 113.

[0032] Figure 4 is an example of a block diagram including the equivalent circuits of Figures 2 and 3. Figure 4(a) is an example in which the signal generation unit 215 is provided in common for multiple pixels, and Figure 4(b) is an example in which the control signal P_CLK can be controlled for each pixel.

[0033] In Figure 4, the photoelectric conversion element 102 having the APD201 is provided on the sensor substrate 11, and the other components are provided on the circuit board 21.

[0034] The APD201 generates charge pairs corresponding to incident light through photoelectric conversion. One of the two nodes of the APD201 is connected to a power line that supplies a drive voltage VL (first voltage). The other node of the APD201 is connected to a power line that supplies a drive voltage VH (second voltage), which is higher than the voltage VL supplied to the anode. In Figure 4, one node of the APD201 is the anode, and the other node of the APD is the cathode. The anode and cathode of the APD201 are supplied with a reverse bias voltage that causes the APD201 to perform avalanche multiplication. By supplying such a voltage, the charge generated by the incident light undergoes avalanche multiplication, and an avalanche current is generated.

[0035] Furthermore, when a reverse bias voltage is supplied, there are two modes: Geiger mode, in which the device operates with a potential difference between the anode and cathode greater than the breakdown voltage, and linear mode, in which the device operates with a potential difference between the anode and cathode near or below the breakdown voltage.

[0036] An APD operating in Geiger mode is called a SPAD. For example, the voltage VL (first voltage) is -30V and the voltage VH (second voltage) is 1V. The APD201 may operate in linear mode or Geiger mode. In the case of a SPAD, the potential difference is larger compared to a linear mode APD, and the voltage withstand effect is more pronounced, so it is preferable to use a SPAD.

[0037] Switch 202 is connected to the power line to which the drive voltage VH is supplied and to one of the nodes, either the anode or cathode, of the APD201. Switch 202 switches the resistance between the APD201 and the power line to which the drive voltage VH is supplied. Here, switching the resistance value preferably means changing the resistance value by 10 times or more, and more preferably by 100 times or more. Hereafter, when the resistance value becomes low, it will be referred to as switching 202 on, and when the resistance value becomes high, it will be referred to as switching 202 off. Switch 202 functions as a quench element. When the signal is multiplied by avalanche multiplication, switch 202 functions as a load circuit (quench circuit) and suppresses the voltage supplied to the APD201, thereby suppressing avalanche multiplication (quench operation). Switch 202 also has the function of returning the voltage supplied to the APD201 to the drive voltage VH by flowing the current that was lost due to the voltage drop during the quench operation (recharge operation).

[0038] The switch 202 can be configured, for example, with a MOS transistor, and Figure 4 shows the case where the switch 202 is a PMOS transistor. The control signal P_CLK for the switch 202, supplied from the signal generation unit 215, is applied to the gate electrode of the MOS transistor constituting the switch 202. In this embodiment, the on and off states of the switch 202 are controlled by controlling the voltage applied to the gate electrode of the switch 202.

[0039] The signal processing circuit 103 includes a waveform shaping unit 210, a counter circuit 211, and a selection circuit 212. In Figure 4, the signal processing circuit 103 includes a waveform shaping unit 210, a counter circuit 211, and a selection circuit 212, but in this specification, the signal processing circuit 103 only needs to have at least one of the waveform shaping unit 210, the counter circuit 211, and the selection circuit 212.

[0040] The waveform shaping unit 210 shapes the cathode potential change of the APD201 obtained during photon detection and outputs a pulse signal. The input node of the waveform shaping unit 210 is denoted as nodeA, and the output node as nodeB. The waveform shaping unit 210 changes the output potential from nodeB depending on whether the input potential to nodeA is above or below a predetermined value. For example, in Figure 5, when the input potential to nodeA becomes a high potential above a judgment threshold, the output potential from nodeB becomes low level. When the input potential to nodeA becomes a potential below the judgment threshold, the output potential from nodeB becomes high level. For example, an inverter circuit can be used as the waveform shaping unit 210. Figure 4 shows an example using one inverter as the waveform shaping unit 210, but a circuit with multiple inverters connected in series may be used, or other circuits that have a waveform shaping effect may be used.

[0041] While it is possible to perform quench and recharge operations using switch 202 in response to avalanche multiplication in APD201, the output signal may not be recognized depending on the timing of photon detection. For example, consider the case where avalanche multiplication occurs in APD, the input potential to node A becomes low, and a recharge operation is performed. Generally, the judgment threshold of the waveform shaping unit 210 is set to a potential higher than the potential difference at which avalanche multiplication occurs in APD. When a photon is incident while the potential of node A is lower than the judgment threshold due to the recharge operation and the potential is at which avalanche multiplication is possible in APD, avalanche multiplication occurs in APD and the voltage of node A decreases. In other words, because the potential of node A decreases at a voltage lower than the judgment threshold, the output potential from node B does not change despite the detection of a photon. Therefore, even though avalanche multiplication is occurring, it is not recognized as a signal. In particular, under high illumination conditions, photons enter continuously in a short period of time, making it difficult to recognize as a signal. As a result, even at high illumination levels, there is a tendency for the actual number of incident photons to deviate from the output signal.

[0042] In contrast, by applying the control signal P_CLK to switch 202 and switching it on and off, it becomes possible to determine the signal even when photons enter the APD continuously in a short period of time. Figure 5 illustrates an example where the control signal P_CLK is a pulse signal with a repeating period. In other words, Figure 5 illustrates a configuration in which the on / off state of switch 202 is switched at a predetermined clock frequency. However, the effect of suppressing the increase in power consumption of the photoelectric converter can be obtained even if the pulse signal is not a repeating period signal.

[0043] The counter circuit 211 counts the pulse signal output from the waveform shaping unit 210 and holds the count value. When the control pulse pRES is supplied via the drive line 213, the signal held by the counter circuit 211 is reset.

[0044] The selection circuit 212 receives a control pulse pSEL from the vertical scanning circuit section 110 in Figure 3 via the drive line 214 (not shown in Figure 3) in Figure 4, which switches the electrical connection between the counter circuit 211 and the signal line 113. The selection circuit 212 includes, for example, a buffer circuit for outputting a signal. The output signal OUT shown in Figure 4 is the signal output from the pixel.

[0045] A switch such as a transistor may be placed between switch 202 and APD201, or between photoelectric conversion element 102 and signal processing circuit 103, to switch the electrical connection. Similarly, the supply of voltage VH or voltage VL to the photoelectric conversion element 102 may be electrically switched using a switch such as a transistor.

[0046] In this embodiment, a configuration using a counter circuit 211 is shown. However, instead of the counter circuit 211, a photoelectric converter 100 may be used that acquires pulse detection timing using a Time to Digital Converter (TDC) and memory. In this case, the generation timing of the pulse signal output from the waveform shaping unit 210 is converted into a digital signal by the TDC. A control pulse pREF (reference signal) is supplied to the TDC via a drive line from the vertical scanning circuit unit 110 in Figure 1 to measure the timing of the pulse signal. The TDC acquires the signal as a digital signal when the input timing of the signal output from each pixel via the waveform shaping unit 210 is relative to the control pulse pREF.

[0047] As shown in Figure 4(b), the signal generation unit 215 may be provided for each pixel. In Figure 4(b), the circuit diagrams from the waveform shaping unit 210 onwards and the signal generation unit shown in Figure 4(a) are omitted. Note that the signal generation unit 215 in Figure 4(a) is assumed to be present for each pixel. In Figure 4(b), a logic circuit is taken up within the pixel to determine whether or not to supply a pulse signal to the switch 202. The logic circuit receives the signal P_EXP which controls the exposure period and the input signal P_CLK_IN which controls the control signal P_CLK. An inverted signal is then output. For example, if the signal P_EXP which controls the exposure period is low level and the input signal P_CLK_IN is low level, a high-level signal is output from the control signal P_CLK. In other words, the switch is turned off. Also, if the signal P_EXP which controls the exposure period is high level and the input signal P_CLK_IN is high level, a low-level signal is output from the control signal P_CLK. In other words, the switch is turned on. Furthermore, if either the exposure period control signal P_EXP or the input signal P_CLK_IN is at a low level, a high-level signal is output as the control signal P_CLK. In other words, switch 202 is turned off. It is preferable to control the switch for each pixel in this way. Note that when using the circuit diagram in Figure 4(b), as will be explained in Embodiment 2 below, when the exposure period P becomes low, the control signal P_CLK remains at a high level. In other words, the switch is turned off.

[0048] Figure 5 schematically shows the relationship between the switch control signal P_CLK, the potential of node A, the potential of node B, and the output signal. In this embodiment, when the control signal P_CLK is high level, the drive voltage VH is less likely to be supplied to the APD, and when the control signal P_CLK is low level, the drive voltage VH is supplied to the APD. A high level control signal P_CLK is, for example, 1V, and a low level control signal P_CLK is, for example, 0V. When the control signal P_CLK is high level, the switch is off, and when the control signal P_CLK is low level, the switch is on. The resistance value of the switch when the control signal P_CLK is high level is higher than the resistance value of the switch when the control signal P_CLK is low level. When the control signal P_CLK is high level, even if avalanche multiplication occurs in the APD, recharge operation is less likely to occur, so the potential supplied to the APD becomes a potential below the breakdown voltage of the APD. Therefore, the avalanche multiplication operation in the APD stops.

[0049] As shown in Figure 4, it is preferable to configure the switch 202 with a single transistor and perform both the quench operation and the recharge operation with that single transistor. This makes it possible to reduce the number of circuits compared to when the quench operation and the recharge operation are performed with different circuit elements. In particular, when each pixel has a counter circuit and the SPAD signal is read out for each pixel, it is preferable to reduce the circuit area used for the switch in order to accommodate the counter circuit, and the effect of configuring the switch 202 with a single transistor becomes significant.

[0050] At time t1, the control signal P_CLK changes from high to low, the switch turns on, and the APD recharge operation begins. This causes the cathode potential of the APD to transition to a high level. Then, the potential difference between the anode and cathode of the APD becomes a state where avalanche multiplication is possible. The cathode potential is the same as that of node nodeA. Therefore, when the cathode potential transitions from low to high, at time t2, the potential of node nodeA becomes greater than or equal to the judgment threshold. At this time, the pulse signal output from node B inverts, going from high to low. Subsequently, a potential difference of drive voltage VH - drive voltage VL is applied to APD201. The control signal P_CLK becomes high, and the switch turns off.

[0051] Next, at time t3, when a photon is incident on APD201, avalanche multiplication occurs in APD201, an avalanche multiplication current flows through switch 202, and the cathode voltage drops. In other words, the voltage at node A drops. As the voltage drop becomes larger and the voltage difference applied to APD201 decreases, the avalanche multiplication of APD201 stops as at time t2, and the voltage level at node A stops dropping below a certain value. If the voltage at node A falls below the judgment threshold while the voltage at node A is dropping, the voltage at node B changes from a low level to a high level. In other words, the portion of the output waveform at node A that exceeds the judgment threshold is waveform shaped by the waveform shaping unit 210 and output as a signal at node B. This is then counted by the counter circuit, and the count value of the counter signal output from the counter circuit increases by 1 LSB.

[0052] Although photons are incident on the APD between time t3 and time t4, the switch is off, and the voltage applied to APD201 is not a potential difference that allows for avalanche multiplication. Therefore, the voltage level of node A does not exceed the threshold.

[0053] At time t4, the control signal P_CLK changes from high level to low level, and the switch turns on. Consequently, current flows through node A to compensate for the voltage drop from the drive voltage VL, and the voltage at node A returns to its original voltage level. At this time, at time t5, the voltage at node A exceeds the threshold, so the pulse signal at node B inverts, changing from high level to low level.

[0054] At time t6, node A settles back to its original voltage level, and the control signal P_CLK changes from a low level to a high level. Therefore, the switch is turned off. Subsequently, as explained from time t1 to time t6, the potentials of each node and signal line change in response to the control signal P_CLK and the incidence of photons.

[0055] The following describes the photoelectric conversion device for each embodiment.

[0056] <Embodiment 1> Figures 6(a), (b), and (c) are timing charts showing the relationship between the exposure period P and the control signal P_CLK in Embodiment 1. Figure 6(a) shows the pulse signal of the control signal P_CLK in the case of exposure period P1. Figure 6(b) shows the pulse signal of the control signal P_CLK in the case of exposure period P2, which is longer than exposure period P1. Figure 6(c) shows the pulse signal of the control signal P_CLK in the case of exposure period P3, which is longer than exposure period P2.

[0057] In this embodiment, the exposure period P is, for example, the period during which the mechanical shutter or electronic shutter is open, and the non-exposure period is, for example, the period during which the mechanical shutter or electronic shutter is closed. The exposure period P may also be defined by adjusting the bias applied to the APD201 to change whether or not photon signals can be acquired. The exposure period P refers to the period during which the APD201 is operational and the APD and signal processing circuit are able to read signals. Here, the state in which the APD and signal processing circuit are able to read signals refers to the state in which the APD is capable of avalanche multiplication. During this time, the counter circuit may also be operating. At this time, the period of quench operation of the APD, which is in an off state based on photon incidence, is part of the operational state. On the other hand, the period during which light is blocked by a shutter or the like, or the period during which the APD is controlled so as not to be avalanche multiplied regardless of the presence or absence of photon incidence, is the non-exposure period.

[0058] As shown in Figures 6(a) to 6(c), in this embodiment, the pulse signal of the control signal P_CLK is changed in accordance with the exposure period P. A first exposure period and a second exposure period which is a different period from the first exposure period are assumed. The control signal is controlled so that the value obtained by dividing the number of control signals P_CLK in the first exposure period by the first exposure period and multiplying it by the first exposure period is different from the value obtained by dividing the number of control signals P_CLK in the second exposure period by the second exposure period and multiplying it by the first exposure period. In this embodiment, since switch 202 is a PMOS transistor, the number of control signals P_CLK refers to the number of pulses that are turned off within a predetermined exposure period.

[0059] Furthermore, the control signal P_CLK is controlled so that, when compared per unit time, the average frequency of the control signal P_CLK in the first exposure period is different from the average frequency of the control signal P_CLK in the second exposure period, which is different from the first exposure period. Here, the average frequency of the control signal P_CLK in the exposure period refers to the frequency obtained by averaging the pulse signals within the exposure period so that they are evenly distributed. For example, if pulse signals are densely distributed in the first half of the exposure period and no pulse signals are distributed in the second half, the frequency adjusted so that pulse signals are evenly distributed throughout the entire period is called the average frequency of the control signal P_CLK in the exposure period. In this embodiment, the control is performed so that the average frequencies are different when comparing the average frequencies of the first exposure period and the second exposure period. Note that a unit time is the time in which at least two pulse signals of the control signal P_CLK are present.

[0060] In Figure 6, the pulse signal of the control signal P_CLK is controlled so that the number of pulses in the pulse signal remains the same even if the exposure period P changes. Let's assume that the number of pulses in the control signal P_CLK during exposure period P1, as shown in Figure 6(a), is N. In exposure period P2, as shown in Figure 6(b), the number of pulses in the control signal P_CLK is also set to N. Furthermore, in exposure period P3, as shown in Figure 6(c), the number of pulses in the control signal P_CLK is also set to N.

[0061] The above configuration prevents an increase in power consumption at each pixel. Furthermore, if the signal processing unit has a counter circuit, it becomes possible to achieve clock driving while maintaining an appropriate upper limit for counting.

[0062] The details will be explained below in comparison with the comparative configuration shown in Figure 7. Figures 7(a), (b), and (c) are timing charts showing the relationship between the exposure period P and the control signal P_CLK in the comparative configuration. In Figure 7, components similar to those in Figure 6 are given the same reference numerals as in Figure 6, and their explanation is omitted.

[0063] In the comparison configuration shown in Figure 7, the period of the control signal P_CLK remains the same even when the exposure period changes. In this embodiment, since switch 202 is a PMOS transistor, the period of the control signal P_CLK refers to the period from one falling edge of a pulse to the next. For example, the period of the control signal P_CLK is the same for both exposure period P1 and exposure period P2. Therefore, in the comparison configuration, when the exposure period P changes, the number of pulses in the pulse signal during exposure period P changes. In this configuration, power consumption may unintentionally increase depending on the exposure period. In the subsequent signal processing circuit, even if the clock frequency is increased, the result may not be reflected, and unnecessary power consumption may be consumed. For example, if the signal processing circuit is a counter circuit, clock frequencies greater than the upper limit of the readable counter value are not read out as a signal. In other words, even if quench and recharge operations are performed to read the signal at values ​​greater than the upper limit of the counter value, the count value is not added in the counter circuit. Therefore, unnecessary power consumption occurs at values ​​greater than the upper limit of the counter value. On the other hand, if the clock frequency is made too high, the number of readable photons decreases, which may reduce the dynamic range.

[0064] In contrast, as described above, unnecessary power consumption can be reduced by controlling the number of pulses in the control signal P_CLK according to the exposure period.

[0065] As shown in Figure 6, the period of the control signal P_CLK changes according to the exposure period. For example, the period of the control signal P_CLK during exposure period P1 is shorter than that of the control signal P_CLK during exposure period P2. Thus, in Figure 6, even when the exposure period changes, such as during exposure periods P1, P2, and P3, the number of pulses of the control signal P_CLK is always N. This reduces the increase in power consumption.

[0066] The number of pulses during the exposure period can be set to any value. If the subsequent signal processing circuit is a counter circuit, it is preferable to set it to, for example, the upper limit of the counter circuit's count. This suppresses the reduction in dynamic range while suppressing the generation of unnecessary power consumption.

[0067] Preferably, the first pulse width in exposure period P1 and the first pulse width in exposure period P2 are the same. Here, the first pulse width refers to the period during which the switch 202 is ON due to the control signal P_CLK. In this embodiment, since the switch 202 is a PMOS transistor, the first pulse width refers to the period during which the control signal P_CLK is at a low level (first level). As described above, the period during which the control signal P_CLK is at a high level (second level) is when the switch is OFF, and the APD is less likely to perform a recharge operation. In this specification, the period during which the control signal P_CLK maintains the state of the first level is described as the "first pulse width," and the period during which the control signal P_CLK maintains the state of the second level is described as the "second pulse width." In Figure 6, the first pulse width in exposure period P1 and the first pulse width in exposure period P2 are the same, but the second pulse width in exposure period P1 and the second pulse width in exposure period P2 are different.

[0068] Conversely, during periods when P_CLK is low, the switch is turned on, and the APD performs a recharge operation. If the period during which the control signal P_CLK is low is long, multiple recharge operations may occur. As explained using Figure 5, when the potential of node A is lower than the judgment threshold, the potential of node A drops, and the signal at node B does not invert, making it possible that the signal value cannot be read correctly. Therefore, by setting the period during which the APD performs a recharge operation to a constant period even when the pulse signal period changes, multiple recharge operations are not performed during periods when the control signal P_CLK is low, and the detection of missed photons can be reduced.

[0069] Figure 8 is an example of a flowchart showing the operation of the photoelectric converter in this embodiment.

[0070] In step S1, the exposure period and the number of pulses for the control signal P_CLK are set. Here, the clock frequency of the control signal P_CLK is set. In step S2, shooting begins. In step S3, it is decided whether or not to change the exposure period. Whether or not to change the exposure period can be determined by the information obtained from the previously taken image (information from the previous frame). If the information obtained from the previous frame is too bright, the exposure period is shortened; if the information obtained from the image is too dark, the exposure period is lengthened. In addition, the exposure period can also be switched manually or automatically.

[0071] If a change is made in step S3, step S4 determines whether or not to lengthen the exposure period. If the determination in step S3 is to change the exposure period, the process proceeds to step S4; if the determination in step S3 is not to change the exposure period, the process proceeds to step S.

[0072] If step S4 determines that the exposure period should be lengthened, step S5 lowers the average clock frequency of the control signal P_CLK during the exposure period. At this time, the control is made so that the number of pulses of the control signal P_CLK during the exposure period does not change before and after the exposure period is changed. If step S4 determines that the exposure period should not be lengthened (shortened), step S6 sets the average clock frequency of the control signal P_CLK during the exposure period to a higher value. Even in this case, the control is made so that the number of pulses of the control signal P_CLK before the exposure period is changed and the number of pulses of the control signal P_CLK during the exposure period do not change.

[0073] After step S5 or step S6, the process proceeds to step S7 to determine whether or not to end the shooting. If it is determined in step S7 to end the exposure period, the shooting is terminated in step S8. If it is determined in step S7 not to end the shooting, the process returns to step S3 and repeats from step S3 to step S7. If it is determined in step S7 to end the shooting, the shooting is terminated in step S8.

[0074] It is possible to operate using the flowchart described above.

[0075] Figure 9 shows a modified version of the timing chart. In Figure 6, the pulse signal P_CLK was input at a constant period, but this is not the only option. As shown in Figures 9(a) to 9(c), the pulse signal P_CLK may be input while changing the pulse period. Note that in Figures 9(a) to (c), the period is changed for each pulse signal, but the period may also be changed for every three or four or more pulse signals.

[0076] Furthermore, as shown in Figures 9(d) to 9(f), the number of pulses in exposure period P2 and exposure period P3 may be the same as the number of pulses in exposure period P1, even if they are the same as the number of pulses in exposure period P1. In other words, the control signal P_CLK may not be continuously supplied to the switch during the first half of exposure period P2, nor may it be supplied during the second half of exposure period P2. Even in such cases, the effects of this embodiment can be obtained if the number of pulses in exposure period P2 and exposure period P1 are the same. Similarly, the effects of this embodiment can be obtained if the number of pulses in exposure period P3 and exposure period P1 are the same.

[0077] <Embodiment 2> Figures 10(a), (b), and (c) are timing charts showing the relationship between the exposure period P and the control signal P_CLK in Embodiment 2. Figure 10(a) shows the pulse signal of the control signal P_CLK in the case of exposure period P1. Figure 10(b) shows the pulse signal of the control signal P_CLK in the case of exposure period P2, which is longer than exposure period P1. Figure 10(c) shows the pulse signal of the control signal P_CLK in the case of exposure period P3, which is longer than exposure period P2.

[0078] The photoelectric conversion device of this embodiment differs from Embodiment 1 in that it stops generating a pulse signal of the control signal P_CLK at the timing of the transition from the exposure period to the non-exposure period. Except for this point and the matters described below, it is substantially the same as Embodiment 1, so the same reference numerals are used for components similar to those in Embodiment 1, and their descriptions may be omitted.

[0079] In this embodiment, the control signal P_CLK is stopped at a low level at the end of the exposure period; in other words, the switch remains in the off state. Then, at the start of the exposure period, the control signal P_CLK is changed to a high level.

[0080] During non-exposure periods, no photons enter the APD, eliminating the need to control the on / off state of the switch. Therefore, by keeping the control signal P_CLK that controls the on / off state of the switch constant, power consumption caused by the on / off switching can be suppressed.

[0081] The timing of the end of the exposure period may be controlled to synchronize with the shutter.

[0082] Furthermore, as shown in Figure 10, it is preferable to stop the pulse signal of the control signal P_CLK at the same time as the end of the exposure period in order to reduce power consumption, but it is also acceptable to stop it slightly later than the end of the exposure period.

[0083] Similar to Embodiment 1, this embodiment can suppress an increase in power consumption. Furthermore, since the number of times the switch is turned on and off can be reduced compared to Embodiment 1, it is possible to further suppress the power consumption of the photoelectric converter.

[0084] <Embodiment 3> The correction of the count value Nct obtained by the photoelectric conversion device according to Embodiment 1 or Embodiment 2 will be explained using Figures 11 to 15.

[0085] Figure 11 shows the relationship between the number of incident photons Nph and the count value Nct at each pixel of the photoelectric converter. Here, Nph is the number of photons actually incident on each pixel per exposure period T (number of incident photons). Nct is the count value of the pulse signal output from the waveform shaping unit 210, which is counted by the counter circuit 211. Here, if the frequency of P_CLK is f, the number of P_CLK pulses during the exposure period T is equal to f × T.

[0086] When the switch 202 of each pixel is controlled by periodic pulses, the count value Nct of each pixel exhibits characteristics similar to curve A in Figure 11. That is, the count value Nct increases with increasing number of incident photons Nph, and is counted up to the number of pulses f × T. Since there is one recharge operation per P_CLK pulse, only one photon can be counted per P_CLK period. Therefore, if multiple photons are incident per P_CLK period, the second and subsequent photons in each period are not counted. Consequently, in high-illumination environments, the number of photons that are incident on a pixel but not counted increases, and the count value Nct and the actual number of incident photons Nph diverge.

[0087] Therefore, in this embodiment, a correction is performed to convert the count value Nct to the equivalent of the actual incident photon number Nph. This correction is performed by a correction circuit 118 connected to the circuit board 21. The correction circuit 118 may be located outside the photoelectric conversion unit 100, as shown in Figure 12, or it may be located inside the photoelectric conversion unit 100 (for example, in the signal processing circuit 103). Note that 119 in Figure 12 is an external output circuit. The relationship between Nct and Nph is shown by the following equation based on the natural logarithm. Nct=f×T×(1-exp(-Nph / (f×T))) Equation (1) In other words, in this correction circuit 118, when the count value Nct, the pulse signal frequency f, and the exposure period length T are used as explanatory variables and the incident photon number Nph is used as the objective variable, the relationship between the explanatory variables and the objective variable is described by a relationship based on the natural logarithm.

[0088] The corrected count value Nct can be represented by the dotted line B in Figure 11. By performing this correction, the count value A, which exhibits a nonlinear characteristic with respect to the number of incident photons, can be corrected to the count value B, which corresponds to the number of incident photons and exhibits linearity.

[0089] Here, the correction formula is determined by the value of f × T. Therefore, even if a value for the exposure period T is set for each of the different P_CLK frequencies f, the value of the count Nct relative to the number of incident photons Nph will not change as long as the combination of f × T is constant.

[0090] Furthermore, if two different P_CLK frequencies are present during a single exposure period, this can be corrected using the following formula. Nct=f1×T1×(1-exp(-Nph1 / (f1×T1)))+f2×T2×(1-exp(-Nph2 / (f2×T2))) Equation (2) In this case, T1 is the period during which the pulse signal operates at a first frequency f1, T2 is the period during which the pulse signal operates at a second frequency f2, and T1 + T2 is the exposure period T. Also, Nph1 is the number of incident photons at T1, and Nph2 is the number of incident photons at T2. Nph1 and Nph2 are expressed by the following equations. Nph1=Nph×T1 / (T1+T2) Equation (3) Nph2 = Nph × T2 / (T1 + T2) Equation (4) Thus, the number of incident photons is determined by the ratio of the exposure period at each frequency to the total exposure period T.

[0091] Here, consider the case where a plurality of frequencies are mixed during the exposure period as shown in FIG. 13. For example, as shown in FIG. 13(a), the frequency of the pulse signal input in the first half of the exposure period may be f1, and the frequency of the pulse signal input in the second half may be f2, and the pulse signals may be input together for each frequency. Also, as shown in FIG. 13(b), two pulses with different frequencies may be in a state of alternating continuously, or the pulse signals of frequency f1 and frequency f2 may alternate one by one. FIGS. 13(a) and (b) show examples of the case where pulse signals of two types of frequencies are mixed during the exposure period, but pulse signals of three or more types of frequencies may also be mixed.

[0092] Thus, by mixing pulse signals of a plurality of types of frequencies during the exposure period, the slope of the count value at high illuminance becomes larger compared to the case where only a low-frequency pulse signal is input. Therefore, the harmony of the count value is maintained even at high illuminance, and the dynamic range can be expanded compared to the case where the frequency of the pulse signal is one type.

[0093] Furthermore, when n types of P_CLK frequencies are mixed during the exposure period, the correction formula is expressed by the following formula. n is a natural number of 2 or more. Nct = f1 × T1 × (1 - exp(-Nph1 / (f1 × T1))) + f2 × T2 × (1 - exp(-Nph2 / (f2 × T2))) + ··· + f n-1 × T n-1 × (1 - exp(-Nph n―1 / (f n-1 × T n-1 ))) + f n × T n × (1 - exp(-Nph n / (f n × T n ))) Equation (5) The sum of the periods during which pulse signals of each frequency are input is equal to the exposure period T. Furthermore, the number of incident photons during the period when pulse signals of each frequency are input is expressed by the following formula. Nph1 = Nph × T1 / (T1 + T2 + ... + T n―1 +T n ) Formula (6) Nph2 = Nph × T2 / (T1 + T2 + ... + T n―1 +T n ) Formula (7) Nph n―1 =Nph×T n―1 / (T1+T2+···+T n―1 +T n ) Formula (8) Nph n =Nph×T n / (T1+T2+···+T n―1 +T n ) Formula (9) The number of incident photons is determined by the ratio of the exposure period at each frequency to the total exposure period T.

[0094] Furthermore, the correction performed by the correction circuit 118 is not limited to performing the correction using the above formula for each count value each time. For example, the correction circuit 118 may have a three-dimensional table of combinations of exposure period T, pulse signal frequency f, and count value Nct. The incident photon number Nph can be estimated by selecting the value closest to the measured value from the values ​​in the table held by the correction circuit 118. In this case, the values ​​in the table are set by a relational expression based on the natural logarithm corresponding to the combination of f and T, similar to the formula above.

[0095] The correction to the count value is not limited to this, and the correction step can be reduced by performing other corrections simultaneously with this correction in the correction circuit 118. For example, in preparation for display, so-called gamma correction may be performed to adjust the brightness of the image formed based on the count value.

[0096] <Embodiment 4> Figure 14 is a block diagram showing the configuration of the light detection system 1200 according to this embodiment. The light detection system 1200 of this embodiment includes a photoelectric converter 1204. Here, the photoelectric converter 1204 can be any of the photoelectric converters described in the embodiments described above. The light detection system 1200 can be used, for example, as an imaging system. Specific examples of imaging systems include digital still cameras, digital camcorders, and surveillance cameras. In Figure 14, an example of a digital still camera is shown as the light detection system 1200.

[0097] The light detection system 1200 shown in Figure 1 includes a photoelectric converter 1204, a lens 1202 that forms an optical image of the subject onto the photoelectric converter 1204, an aperture 1203 for varying the amount of light passing through the lens 1202, and a barrier 1201 for protecting the lens 1202. The lens 1202 and aperture 1203 form an optical system that focuses light onto the photoelectric converter 1204. A shutter is positioned between the aperture 1203 and the photoelectric converter 1204. The exposure period of the photoelectric converter is controlled by opening and closing the shutter.

[0098] The photodetection system 1200 has a signal processing unit 1205 that processes the output signal output from the photoelectric converter 1204. The signal processing unit 1205 performs signal processing operations that perform various corrections and compressions on the input signal as needed before outputting it. The photodetection system 1200 further has a buffer memory unit 1206 for temporarily storing image data, and an external interface unit (external I / F unit) 1209 for communicating with an external computer or the like. Furthermore, the photodetection system 1200 has a recording medium 1211 such as a semiconductor memory for recording or reading imaging data, and a recording medium control interface unit (recording medium control I / F unit) 1210 for recording or reading from the recording medium 1211. The recording medium 1211 may be built into the photodetection system 1200 or may be detachable. In addition, communication from the recording medium control I / F unit 1210 to the recording medium 1211 and communication from the external I / F unit 1209 may be performed wirelessly.

[0099] Furthermore, the photodetection system 1200 includes an overall control and calculation unit 1208 that performs various calculations and controls the entire digital still camera, and a timing generation unit 1207 that outputs various timing signals to the photoelectric converter 1204 and the signal processing unit 1205. Here, the timing signals may be input from an external source, and the photodetection system 1200 only needs to include at least the photoelectric converter 1204 and the signal processing unit 1205 that processes the output signals output from the photoelectric converter 1204. As described in the fourth embodiment, the timing generation unit 1207 may be mounted on the photoelectric converter. The overall control and calculation unit 1208 and the timing generation unit 1207 may be configured to perform some or all of the control functions of the photoelectric converter 1204.

[0100] The photoelectric converter 1204 outputs an image signal to the signal processing unit 1205. The signal processing unit 1205 performs predetermined signal processing on the image signal output from the photoelectric converter 1204 and outputs image data. The signal processing unit 1205 also generates an image using the image signal. The signal processing unit 1205 may also perform distance measurement calculations on the signal output from the photoelectric converter 1204. The signal processing unit 1205 and the timing generation unit 1207 may be mounted on the photoelectric converter. In other words, the signal processing unit 1205 and the timing generation unit 1207 may be provided on the substrate on which the pixels are arranged, or they may be provided on a separate substrate. By configuring an imaging system using the photoelectric converters of each embodiment described above, an imaging system capable of acquiring higher quality images can be realized.

[0101] <Embodiment 5> Figure 15 is a block diagram showing an example configuration of a distance image sensor, which is an electronic device utilizing the photoelectric conversion device described in the above embodiment.

[0102] As shown in Figure 15, the distance image sensor 401 is configured to include an optical system 407, a photoelectric converter 408, an image processing circuit 404, a monitor 405, and a memory 406. The distance image sensor 401 receives light (modulated light or pulsed light) that is projected from a light source device 409 toward the subject and reflected from the surface of the subject, thereby acquiring a distance image corresponding to the distance to the subject.

[0103] The optical system 407 is composed of one or more lenses and guides the image light (incident light) from the subject to the photoelectric converter 408, where it forms an image on the light-receiving surface (sensor part) of the photoelectric converter 408.

[0104] The photoelectric converter 408 is one of the photoelectric converters from each of the embodiments described above, and a distance signal indicating the distance obtained from the light received signal output from the photoelectric converter 408 is supplied to the image processing circuit 404.

[0105] The image processing circuit 404 performs image processing to construct a distance image based on the distance signal supplied from the photoelectric converter 408. The distance image (image data) obtained through this image processing is then supplied to the monitor 405 for display or supplied to the memory 406 for storage (recording).

[0106] With the distance image sensor 401 configured in this way, by applying the photoelectric conversion device described above, the characteristics of the pixels are improved, and for example, more accurate distance images can be acquired.

[0107] <Embodiment 6> The technology disclosed herein (the Technology) can be applied to a variety of products. For example, the Technology disclosed herein may be applied to an endoscopic surgical system.

[0108] Figure 16 shows an example of a schematic configuration of an endoscopic surgical system to which the technology described herein (the technology) may be applied.

[0109] Figure 16 illustrates a surgeon (physician) 1131 performing surgery on a patient 1132 on a patient bed 1133 using an endoscopic surgical system 1003. As shown in the figure, the endoscopic surgical system 1003 consists of an endoscope 1100, surgical instruments 1110, and a cart 1134 equipped with various devices for endoscopic surgery.

[0110] The endoscope 1100 consists of a barrel 1101, the tip of which is inserted into the body cavity of the patient 1132 for a predetermined length, and a camera head 1102 connected to the base end of the barrel 1101. In the illustrated example, the endoscope 1100 is shown as a so-called rigid endoscope having a rigid barrel 1101, but the endoscope 1100 may also be configured as a so-called flexible endoscope having a flexible barrel.

[0111] An opening into which an objective lens is fitted is provided at the tip of the endoscope tube 1101. A light source device 1203 is connected to the endoscope 1100, and the light generated by the light source device 1203 is guided to the tip of the endoscope tube by a light guide extending inside the endoscope tube 1101, and is irradiated through the objective lens towards the object to be observed inside the body cavity of the patient 1132. The endoscope 1100 may be a straight-viewing endoscope, an oblique-viewing endoscope, or a side-viewing endoscope.

[0112] The camera head 1102 contains an optical system and a photoelectric converter. Reflected light from the object being observed (observation light) is focused by the optical system into the photoelectric converter. The photoelectric converter converts the observation light into electrical signals, generating an electrical signal corresponding to the observation light, i.e., an image signal corresponding to the observed image. The photoelectric converter can be any of the photoelectric converters described in the embodiments described above. The image signal is transmitted as RAW data to the camera control unit (CCU) 1135.

[0113] The CCU1135 consists of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and other components, and comprehensively controls the operation of the endoscope 1100 and the display device 1136. Furthermore, the CCU1135 receives an image signal from the camera head 1102 and performs various image processing operations on that image signal, such as development processing (demosaic processing), to display the image based on that image signal.

[0114] The display device 1136, under control from the CCU 1135, displays an image based on an image signal that has been processed by the CCU 1135.

[0115] The light source device 1203 consists of a light source such as an LED (Light Emitting Diode) and supplies illumination light to the endoscope 1100 when photographing the surgical area, etc.

[0116] The input device 1137 is an input interface for the endoscopic surgical system 1003. The user can input various types of information and instructions to the endoscopic surgical system 1003 via the input device 1137.

[0117] The treatment instrument control device 1138 controls the driving of the energy treatment instrument 1112 for purposes such as tissue cauterization, incision, or blood vessel sealing.

[0118] The light source device 1203, which supplies illumination light to the endoscope 1100 when photographing the surgical area, can be composed of, for example, an LED, a laser light source, or a combination thereof. When the white light source is composed of a combination of RGB laser light sources, the output intensity and output timing of each color (each wavelength) can be controlled with high precision, so the white balance of the captured image can be adjusted in the light source device 1203. In this case, it is also possible to capture images corresponding to each of the RGB colors in time-division by irradiating the observation target with laser light from each of the RGB laser light sources in time-division and controlling the drive of the image sensor of the camera head 1102 in synchronization with the irradiation timing. According to this method, a color image can be obtained without providing a color filter on the image sensor.

[0119] Furthermore, the light source device 1203 may be controlled to change the intensity of the light it outputs at predetermined time intervals. By controlling the drive of the image sensor of the camera head 1102 in synchronization with the timing of the change in light intensity, images can be acquired in time-division order, and these images can be combined to generate high dynamic range images without so-called black crushing and white clipping.

[0120] Furthermore, the light source device 1203 may be configured to supply light in a predetermined wavelength band corresponding to special light observation. In special light observation, for example, the wavelength dependence of light absorption in body tissue is utilized. Specifically, by irradiating with narrowband light compared to the irradiation light used during normal observation (i.e., white light), predetermined tissues such as blood vessels on the surface of mucosa can be imaged with high contrast. Alternatively, in special light observation, fluorescence observation may be performed to obtain an image from fluorescence generated by irradiation with excitation light. In fluorescence observation, excitation light can be irradiated onto body tissue and fluorescence from the body tissue can be observed, or a reagent such as indocyanine green (ICG) can be injected into body tissue and excitation light corresponding to the fluorescence wavelength of the reagent can be irradiated onto the body tissue to obtain a fluorescence image. The light source device 1203 may be configured to supply narrowband light and / or excitation light corresponding to such special light observation.

[0121] <Embodiment 7> The light detection system and mobile body of this embodiment will be described with reference to Figures 17 and 18. Figure 17 is a schematic diagram showing an example configuration of the light detection system and mobile body according to this embodiment. Figure 18 is a flowchart showing the operation of the light detection system according to this embodiment. In this embodiment, an example of an in-vehicle camera is shown as the light detection system.

[0122] Figure 17 shows an example of a vehicle system and an image-capturing photodetection system mounted thereon. The photodetection system 1301 includes a photoelectric converter 1302, an image preprocessing unit 1315, an integrated circuit 1303, and an optical system 1314. The optical system 1314 forms an optical image of the subject on the photoelectric converter 1302. The photoelectric converter 1302 converts the optical image of the subject formed by the optical system 1314 into an electrical signal. The photoelectric converter 1302 is one of the photoelectric converters in each of the embodiments described above. The image preprocessing unit 1315 performs predetermined signal processing on the signal output from the photoelectric converter 1302. The functions of the image preprocessing unit 1315 may be incorporated into the photoelectric converter 1302. The photodetection system 1301 is provided with at least two sets of optical systems 1314, photoelectric converters 1302, and image preprocessing units 1315, and the output from each set of image preprocessing units 1315 is input to the integrated circuit 1303.

[0123] The integrated circuit 1303 is an integrated circuit for imaging system applications and includes an image processing unit 1304 with memory 1305, an optical distance measuring unit 1306, a distance measurement calculation unit 1307, an object recognition unit 1308, and an anomaly detection unit 1309. The image processing unit 1304 performs image processing such as development and defect correction on the output signal of the image preprocessing unit 1315. The memory 1305 stores the primary storage of the captured image and the location of defects in the captured pixels. The optical distance measuring unit 1306 focuses on the subject and measures the distance. The distance measurement calculation unit 1307 calculates distance measurement information from multiple image data acquired by multiple photoelectric converters 1302. The object recognition unit 1308 recognizes subjects such as cars, roads, signs, and people. When the anomaly detection unit 1309 detects an anomaly in the photoelectric converter 1302, it alerts the main control unit 1313 to the anomaly.

[0124] The integrated circuit 1303 may be implemented by specially designed hardware, by a software module, or by a combination of these. It may also be implemented by an FPGA (Field Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), or a combination of these.

[0125] The main control unit 1313 coordinates and controls the operation of the light detection system 1301, vehicle sensor 1310, control unit 1320, etc. Alternatively, there may be no main control unit 1313, and the light detection system 1301, vehicle sensor 1310, and control unit 1320 may each have their own communication interfaces and send and receive control signals via a communication network (e.g., CAN standard).

[0126] The integrated circuit 1303 has the function of receiving control signals from the main control unit 1313 or transmitting control signals and set values ​​to the photoelectric converter 1302 via its own control unit.

[0127] The light detection system 1301 is connected to the vehicle sensor 1310 and can detect the vehicle's driving conditions, such as vehicle speed, yaw rate, and steering angle, as well as the external environment and the state of other vehicles and obstacles. The vehicle sensor 1310 also serves as a distance information acquisition means for acquiring distance information to objects. Furthermore, the light detection system 1301 is connected to the driver assistance control unit 1311, which performs various driver assistance functions such as automatic steering, automatic cruising, and collision avoidance. In particular, regarding the collision judgment function, it determines whether a collision with another vehicle or obstacle has occurred and estimates a collision based on the detection results of the light detection system 1301 and the vehicle sensor 1310. This enables avoidance control when a collision is estimated and activation of safety devices in the event of a collision.

[0128] Furthermore, the light detection system 1301 is also connected to a warning device 1312 that issues a warning to the driver based on the judgment result of the collision judgment unit. For example, if the collision judgment unit determines that there is a high probability of collision, the main control unit 1313 performs vehicle control to avoid a collision or mitigate damage by applying the brakes, releasing the accelerator, or suppressing engine output. The warning device 1312 warns the user by sounding an alarm, displaying warning information on a display screen such as the car navigation system or instrument panel, or vibrating the seat belt or steering wheel.

[0129] In this embodiment, the light detection system 1301 captures images of the area around the vehicle, for example, the front or rear. Figure 17(b) shows an example of the arrangement of the light detection system 1301 when it captures images of the area in front of the vehicle.

[0130] The two photoelectric converters 1302 are positioned in front of the vehicle 1300. Specifically, it is preferable for the two photoelectric converters 1302 to be positioned symmetrically with respect to the axis of symmetry, considering the center line of the vehicle 1300's direction of movement or external shape (e.g., vehicle width), when acquiring distance information between the vehicle 1300 and the object being photographed and determining the possibility of collision. Furthermore, it is preferable that the photoelectric converters 1302 are positioned so as not to obstruct the driver's field of view when the driver is visually observing the situation outside the vehicle 1300 from the driver's seat. The warning device 1312 is preferably positioned so as to be easily visible to the driver.

[0131] Next, the fault detection operation of the photoelectric converter 1302 in the photodetection system 1301 will be explained using Figure 18. The fault detection operation of the photoelectric converter 1302 is performed according to steps S1410 to S1480 shown in Figure 18.

[0132] Step S1410 is a step in which the photoelectric converter 1302 is configured for startup. Specifically, settings for the operation of the photoelectric converter 1302 are transmitted from outside the photodetection system 1301 (e.g., the main control unit 1313) or from inside the photodetection system 1301, and the imaging operation and fault detection operation of the photoelectric converter 1302 are started.

[0133] Next, in step S1420, a pixel signal is acquired from the active pixels. Also, in step S1430, an output value is acquired from a fault detection pixel provided for fault detection. This fault detection pixel, like the active pixels, is equipped with a photoelectric conversion element. A predetermined voltage is written to this photoelectric conversion element. The fault detection pixel outputs a signal corresponding to the voltage written to this photoelectric conversion element. Note that steps S1420 and S1430 may be reversed.

[0134] Next, in step S1440, a determination is made between the expected output value of the fault-detection pixel and the actual output value from the fault-detection pixel. If the determination in step S1440 shows that the expected output value and the actual output value match, the process proceeds to step S1450, where it is determined that the imaging operation is functioning normally, and the process moves to step S1460. In step S1460, the pixel signals of the scanned row are transmitted to the memory 1305 for temporary storage. After that, the process returns to step S1420, and the fault detection operation continues. On the other hand, if the determination in step S1440 shows that the expected output value and the actual output value do not match, the process proceeds to step S1470. In step S1470, it is determined that there is an abnormality in the imaging operation, and an alarm is issued to the main control unit 1313 or the alarm device 1312. The alarm device 1312 displays that an abnormality has been detected on its display unit. Subsequently, in step S1480, the photoelectric converter 1302 is stopped, and the operation of the photodetection system 1301 is terminated.

[0135] In this embodiment, an example is shown where the flowchart is looped every row, but the flowchart may be looped every multiple rows, or the fault detection operation may be performed every frame. The alarm in step S1470 may be notified to an external party via a wireless network.

[0136] Furthermore, although this embodiment describes control to avoid collisions with other vehicles, it can also be applied to control that automatically follows other vehicles or control that automatically drives without deviating from the lane. In addition, the light detection system 1301 can be applied not only to vehicles such as automobiles, but also to mobile objects (mobile devices) such as ships, aircraft, or industrial robots. Moreover, it can be applied not only to mobile objects, but also to a wide range of devices that utilize object recognition, such as intelligent transportation systems (ITS).

[0137] The photoelectric conversion device of the present invention may further be configured to acquire various types of information, such as distance information.

[0138] <Embodiment 8> Figure 19(a) illustrates a pair of glasses 1600 (smart glasses) according to one application example. The glasses 1600 have a photoelectric converter 1602. The photoelectric converter 1602 is the photoelectric converter described in each of the embodiments described above. In addition, a display device including a light-emitting device such as an OLED or LED may be provided on the back side of the lens 1601. There may be one or more photoelectric converters 1602. Furthermore, multiple types of photoelectric converters may be used in combination. The arrangement position of the photoelectric converter 1602 is not limited to that shown in Figure 19(a).

[0139] The eyeglasses 1600 further include a control device 1603. The control device 1603 functions as a power source that supplies power to the photoelectric converter 1602 and the display device. The control device 1603 also controls the operation of the photoelectric converter 1602 and the display device. The lens 1601 has an optical system formed therein for focusing light onto the photoelectric converter 1602.

[0140] Figure 19(b) illustrates a pair of glasses 1610 (smart glasses) according to one application example. The glasses 1610 have a control device 1612, which is equipped with a photoelectric converter equivalent to a photoelectric converter 1602 and a display device. The lens 1611 has an optical system formed therein for projecting light emitted from the photoelectric converter in the control device 1612 and from the display device, and an image is projected onto the lens 1611. The control device 1612 functions as a power supply that provides power to the photoelectric converter and the display device, and also controls the operation of the photoelectric converter and the display device. The control device may have a gaze detection unit that detects the wearer's gaze. Gaze detection may use infrared light. The infrared light emitter emits infrared light towards the eyeball of the user who is fixating on the displayed image. An imaging unit having a light-receiving element detects the reflected light from the eyeball of the emitted infrared light, thereby obtaining an image of the eyeball. By having a reduction means that reduces the light from the infrared light emitter to the display unit in planar view, the deterioration of image quality is reduced.

[0141] The user's gaze towards the displayed image is detected from an image of the eyeball obtained by imaging with infrared light. Any known method can be applied to gaze detection using an image of the eyeball. For example, a gaze detection method based on the Purkinje image obtained by the reflection of the irradiated light from the cornea can be used.

[0142] More specifically, gaze detection processing is performed based on the pupil-corneal reflection method. Using the pupil-corneal reflection method, a gaze vector representing the orientation (rotation angle) of the eyeball is calculated based on the pupil image and Purkinje image contained in the captured image of the eyeball, thereby detecting the user's gaze.

[0143] The display device of this embodiment includes a photoelectric converter having a light-receiving element, and may control the display image of the display device based on the user's gaze information from the photoelectric converter.

[0144] Specifically, the display device determines a first field of view that the user is fixated on, and a second field of view other than the first field of view, based on gaze information. The first and second field of view may be determined by the control device of the display device, or they may be determined by an external control device and received by the display device. Within the display area of ​​the display device, the display resolution of the first field of view may be controlled to be higher than the display resolution of the second field of view. In other words, the resolution of the second field of view may be lower than that of the first field of view.

[0145] Furthermore, the display area has a first display area and a second display area different from the first display area, and a higher priority area may be determined from the first and second display areas based on gaze information. The first and second view areas may be determined by the control device of the display device, or they may be determined by an external control device and received. The resolution of the higher priority area may be controlled to be higher than the resolution of the areas other than the higher priority area. In other words, the resolution of areas with relatively lower priority may be set lower.

[0146] AI may be used to determine the first field of view area and high-priority areas. The AI ​​may be a model configured to estimate the angle of gaze and the distance to the target object at the end of the line of sight from the image of the eye, using the image of the eye and the direction the eye was actually looking in that image as training data. The AI ​​program may be installed in the display device, the photoelectric converter, or an external device. If installed in an external device, it will be transmitted to the display device via communication.

[0147] When display control is based on visual detection, this method is preferably applicable to smart glasses that further include a photoelectric converter for capturing images of the surrounding environment. The smart glasses can display the captured external information in real time.

[0148] <Other Embodiments> Although the embodiments have been described above, the present invention is not limited to these embodiments, and various modifications and variations are possible. Furthermore, the embodiments are mutually applicable. [Explanation of symbols]

[0149] 201 APD 103 Signal Processing Circuit 202 Switch 215 Signal Generation Unit P exposure period

Claims

1. An avalanche photodiode including an anode and a cathode, A switch connected to one of the anodes and cathodes, to which a pulse signal is input, is provided. A photoelectric converter characterized in that the value obtained by dividing the number of pulse signals in the first exposure period by the first exposure period is different from the value obtained by dividing the number of pulse signals in the second exposure period, which has a different length from the first exposure period, by the second exposure period.

2. The photoelectric conversion device according to claim 1, characterized in that the pulse signal is a signal with a repeating period.

3. The photoelectric conversion device according to claim 1 or 2, characterized in that the number of pulse signals in the first exposure period is the same as the number of pulse signals in the second exposure period.

4. The first exposure period is shorter than the second exposure period. The photoelectric conversion apparatus according to any one of claims 1 to 3, characterized in that the period of the pulse signal during the first exposure period is shorter than the period of the pulse signal during the second exposure period.

5. The photoelectric conversion apparatus according to any one of claims 1 to 4, characterized in that the first pulse width of the first level of the pulse signal during the first exposure period is the same as the first pulse width of the first level of the pulse signal during the second exposure period.

6. The photoelectric conversion apparatus according to claim 5, characterized in that the second pulse width of the second level of the pulse signal during the first exposure period is different from the second pulse width of the second level of the pulse signal during the second exposure period.

7. The photoelectric converter according to any one of claims 1 to 6, characterized in that the switch functions as a quench element.

8. The aforementioned switch is a MOS transistor, One node of the switch is connected to the cathode of the avalanche photodiode. The other node of the switch is connected to a power line to which the drive voltage is applied. The photoelectric conversion device according to any one of claims 1 to 7, characterized in that the pulse signal is supplied to the gate electrode of the switch.

9. The signal processing circuit comprises a waveform shaping unit and a counter circuit. The waveform shaping unit is connected to one of the nodes, the anode and the cathode. The photoelectric converter according to any one of claims 1 to 8, characterized in that the signal output from the waveform shaping unit is input to the counter circuit.

10. The photoelectric conversion apparatus according to claim 9, characterized in that the number of pulse signals during the first exposure period is the same as the upper limit of the counter circuit.

11. It has a correction circuit, The photoelectric conversion device according to claim 9 or 10, characterized in that the count value output from the counter circuit is input to the correction circuit.

12. The photoelectric conversion apparatus according to claim 11, wherein the correction circuit aims to describe the relationship between the explanatory variables and the objective variable by a relational expression based on the natural logarithm, with the count value, the frequency of the pulse signal during the first exposure period, and the length of the first exposure period as explanatory variables and the number of incident photons as the objective variable.

13. The correction circuit, with the count value Nct, the number of incident photons Nph, the frequency of the pulse signal during the first exposure period f, and the first exposure period T, applies the following to the input of the count value. Nct=f×T×(1-exp(-Nph / (f×T))) The photoelectric conversion device according to claim 11 or 12, characterized in that it outputs an incident photon number Nph that satisfies the relationship.

14. The correction circuit is configured such that the pulse signal has n different frequencies. The first frequency of the pulse signal is f １ , the second frequency is f ２ , the (n-1)th frequency is f ｎ－１ , the nth frequency is f ｎ (Let n be a natural number greater than or equal to 2) The period during which the frequency of the pulse signal is f １ is T １ The period during which the frequency of the pulse signal is f ２ is T ２ The period during which the frequency of the pulse signal is f ｎ－１ is T ｎ－１ The period during which the frequency of the pulse signal is f ｎ is T ｎ Taking this as such, for the input of the count value Nct=f １ ×T １ ×(1-exp(-Nph １ / (f １ ×T １ )))+f ２ ×T ２ ×(1-exp(-Nph ２ / (f ２ ×T ２ )))+・・・+f ｎ－１ ×T ｎ－１ ×(1-exp(-Nph ｎ―１ / (f ｎ－１ ×T ｎ－１ )))+f ｎ ×T ｎ ×(1-exp(-Nph ｎ / (f ｎ ×T ｎ ))) The photoelectric conversion device according to claim 11, characterized in that it outputs an incident photon number Nph that satisfies the relationship.

15. The photoelectric converter according to any one of claims 1 to 13, characterized in that the pulse signal is not input to the switch during periods other than the first exposure period and the second exposure period.

16. The photoelectric conversion device according to any one of claims 1 to 14, characterized in that a plurality of avalanche photodiodes are arranged in a two-dimensional array in a planar view.

17. A photoelectric conversion device according to any one of claims 1 to 16, A photodetection system characterized by having a signal processing unit that processes the signal output by the photoelectric converter.

18. A photoelectric conversion device according to any one of claims 1 to 16, A signal processing unit that processes the signal output by the aforementioned photoelectric converter, The device comprises a shutter that controls the incidence of light onto the photoelectric converter, A light detection system characterized in that the first exposure period and the second exposure period are the periods during which the shutter is open.

19. A photoelectric conversion device according to any one of claims 1 to 16, A mobile body having distance information acquisition means for acquiring distance information to an object from distance measurement information based on a signal from the aforementioned photoelectric converter, A mobile body further comprising control means for controlling the mobile body based on the distance information.

Images