Photoelectric conversion device, photodetection system

Abstract

To provide a photoelectric conversion device capable of performing a specific pulse signal control in a case where there are a plurality of exposure periods.SOLUTION: A photoelectric conversion device includes: an avalanche photodiode that includes an anode and a cathode; a switch that is connected to a node of one of the anode and the cathode and a power supply line, to which a drive voltage is applied, and that is for switching a resistance value between the one node and the power supply line; and a signal generation unit that generates pulse signals for controlling the switching of the switch. A value obtained by dividing the number of the pulse signals during a first exposure period by the first exposure period is different from a value obtained by dividing the number of the pulse signals during a second exposure period, which is different in length from the first exposure period, by the second exposure period.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present invention relates to a photoelectric conversion device that performs photoelectric conversion, and a photodetection system. [Background technology]

[0002] There is known a photoelectric conversion device including a pixel array in which pixels each including a plurality of avalanche photodiodes (APDs) are arranged in a two-dimensional array on a plane. In each pixel, avalanche multiplication of photocharges caused by a single photon occurs in a PN junction region in a semiconductor region.

[0003] Patent Document 1 discloses a pixel having an APD, the APD, a quench circuit connected to the APD, a signal control circuit to which a signal output from the APD is input, and a pulse generation circuit connected to the quench circuit and the signal control circuit. The pulse generation circuit controls the on / off of the quench circuit. It also discloses that the output signal is reset for each pulse signal, and a pulse signal corresponding to the input photon is output even under high brightness. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2020-123847 A Summary of the Invention [Problem to be solved by the invention]

[0005] Patent Document 1 does not disclose anything about the number or cycles of pulse signals within an exposure period when the exposure period is different. Patent Document 1 leaves room for consideration of controlling pulse signals in relation to the exposure period. [Means for solving the problem]

[0006] A photoelectric conversion device according to one embodiment includes an avalanche photodiode including an anode and a cathode, a switch connected to one of the nodes of the anode and the cathode and a power line to which a drive voltage is applied, the switch switching a resistance value between the one node and the power line, and a signal generating unit generating a pulse signal for controlling the switching of the switch, wherein a value obtained by dividing the number of the pulse signals in a first exposure period by the first exposure period and multiplying the result by the first exposure period is different from a value obtained by dividing the number of the pulse signals in a second exposure period having a length different from that of the first exposure period by the second exposure period and multiplying the result by the first exposure period. Effect of the Invention

[0007] According to the present invention, it is possible to provide a photoelectric conversion device capable of controlling a specific pulse signal when there are a plurality of exposure periods. [Brief description of the drawings]

[0008] [Figure 1] 1 is a diagram showing a configuration of a photoelectric conversion device; [Diagram 2] Sensor board layout example [Diagram 3] Circuit board layout example [Figure 4] Block diagram including equivalent circuit of photoelectric conversion element [Diagram 5] A diagram showing the relationship between APD operation and output signal [Figure 6] FIG. 1 is a timing chart of control pulses of the photoelectric conversion device according to the first embodiment. [Figure 7] FIG. 11 is a timing chart of control pulses of a photoelectric conversion device according to a comparative example. [Figure 8] FIG. 1 is a flowchart illustrating an example of an operation mode of the photoelectric conversion device according to the first embodiment. [Figure 9] FIG. 11 is a timing chart of control pulses of a photoelectric conversion device according to a modification of the first embodiment. [Figure 10] FIG. 11 is a timing chart of control pulses of the photoelectric conversion device according to the second embodiment. [Figure 11]FIG. 13 is a diagram showing the relationship between the number of incident photons per exposure period and the count value of the photoelectric conversion device according to the third embodiment. [Figure 12] Arrangement example of circuit board of photoelectric conversion device according to embodiment 3 [Figure 13] FIG. 11 is a timing chart of control pulses of a photoelectric conversion device according to a third embodiment. [Figure 14] Block diagram of a light detection system according to the fourth embodiment [Figure 15] Block diagram of a light detection system according to the fifth embodiment [Figure 16] Block diagram of a light detection system according to the sixth embodiment [Figure 17] Block diagram of a light detection system according to the seventh embodiment [Figure 18] Flowchart of the light detection system of embodiment 7 [Figure 19] FIG. 13 is a diagram showing a specific example of an electronic device according to an eighth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] The following embodiments are intended to embody the technical ideas of the present invention, and are not intended to limit the present invention. The sizes and positional relationships of the components shown in the drawings may be exaggerated to clarify the description. In the following description, the same components may be designated by the same reference numerals, and the description may be omitted.

[0010] A configuration common to the photoelectric conversion devices in each embodiment will be described with reference to Figs. 1 to 4. The photoelectric conversion device has a SPAD pixel including an avalanche diode. The conductivity type of the charge used as the signal charge among the charge pairs generated in the avalanche diode is called the first conductivity type. The first conductivity type refers to a conductivity type in which the charge of the same polarity as the signal charge is the majority carrier. The conductivity type opposite to the first conductivity type is called the second conductivity type. In the following, an example will be described in which the signal charge is an electron, the first conductivity type is an N type, and the second conductivity type is a P type, but the signal charge may be a hole, the first conductivity type is a P type, and the second conductivity type is an N type.

[0011] When the signal charge is electrons, the signal is read out from the cathode of the APD, but when the signal charge is holes, the signal is read out from the anode of the APD. Therefore, the relationship between the cathode and anode of the APD is reversed.

[0012] In this specification, the term "planar view" refers to a view from a direction perpendicular to the light incident surface of the semiconductor layer on which the photoelectric conversion element described later is disposed. Also, the term "cross section" refers to a surface in a direction perpendicular to the light incident surface of the semiconductor layer on which the photoelectric conversion element is disposed. Note that, when the light incident surface of the semiconductor layer is a rough surface when viewed microscopically, the planar view is defined based on the light incident surface of the semiconductor layer when viewed macroscopically.

[0013] First, a configuration common to each embodiment will be described.

[0014] FIG. 1 is a diagram showing a configuration of a photoelectric conversion device 100 according to an embodiment of the present invention. In the following, a case where the photoelectric conversion device 100 is a stacked type photoelectric conversion device will be described as an example. That is, a photoelectric conversion device configured by stacking and electrically connecting two substrates, a sensor substrate 11 and a circuit substrate 21, will be described as an example. However, the photoelectric conversion device is not limited to this. For example, the photoelectric conversion device may be a photoelectric conversion device in which the configuration included in the sensor substrate 11 and the configuration included in the circuit substrate are arranged on a common semiconductor layer, as described below. In the following, a photoelectric conversion device in which the configuration included in the sensor substrate 11 and the configuration included in the circuit substrate are arranged on a common semiconductor layer is also referred to as a non-stacked photoelectric conversion device.

[0015] The sensor substrate 11 has a first semiconductor layer having a photoelectric conversion element 102 described later, and a first wiring structure. The circuit substrate 21 has a second semiconductor layer having circuits such as a signal processing circuit 103 described later, and a second wiring structure. The photoelectric conversion device 100 is configured by laminating the second semiconductor layer, the second wiring structure, the first wiring structure, and the first semiconductor layer in this order.

[0016] FIG. 1 illustrates a back-illuminated photoelectric conversion device in which light is incident from a first surface and a circuit board is disposed on a second surface opposite the first surface. In the case of a non-stacked photoelectric conversion device, the surface on which the transistors of the signal processing circuit are disposed is called the second surface. In the case of a back-illuminated photoelectric conversion device, the first surface opposite the second surface of the semiconductor layer is the light incident surface. In the case of a front-illuminated photoelectric conversion device, the second surface of the semiconductor layer is the light incident surface.

[0017] In the following, the sensor substrate 11 and the circuit substrate 21 will be described as diced chips, but are not limited to chips. For example, each substrate may be a wafer. Also, each substrate may be stacked in a wafer state and then diced, or each chip may be stacked and bonded after being chipped.

[0018] A pixel region 12 is disposed on the sensor substrate 11, and a circuit region 22 that processes signals detected in the pixel region 12 is disposed on the circuit substrate 21.

[0019] 2 is a diagram showing an example of the arrangement of the sensor substrate 11. Pixels 101 each having a photoelectric conversion element 102 including an avalanche photodiode (hereinafter, APD) are arranged in a two-dimensional array in a plan view to form a pixel region 12.

[0020] The pixel 101 is typically a pixel for forming an image, but when used for TOF (Time of Flight), it does not necessarily have to form an image. That is, the pixel 101 may be a pixel for measuring the time when light arrives and the amount of light.

[0021] 3 is a configuration diagram of the circuit board 21. It has a signal processing circuit 103 for processing electric charges photoelectrically converted by the photoelectric conversion element 102 in FIG. 2, a readout circuit 112, a control pulse generation unit 115, a horizontal scanning circuit unit 111, a signal line 113, and a vertical scanning circuit unit 110.

[0022] The photoelectric conversion element 102 in FIG. 2 and the signal processing circuit 103 in FIG. 3 are electrically connected via a connection wiring provided for each pixel.

[0023] The vertical scanning circuit section 110 receives a control pulse supplied from a control pulse generating section 115 and supplies the control pulse to each pixel. The vertical scanning circuit section 110 uses logic circuits such as a shift register and an address decoder.

[0024] The control pulse generating unit 115 has a signal generating unit 215 that generates a control signal P_CLK for a switch, which will be described later. The signal generating unit 215 generates a pulse signal for controlling the switch, as will be described later. For example, as shown in FIG. 4(a), the signal generating unit 215 may generate a control signal P_CLK in common for a plurality of pixels in the pixel region, or as shown in FIG. 4(b), the control signal P_CLK may be generated for each pixel. When the pulse signal P_CLK is generated in common, at least one of the period, the number of pulses, and the pulse width of the signal P_EXP pulse signal for controlling the exposure period is generated in common in correspondence with the exposure period. When the control signal P_CLK is controlled for each pixel, a signal can be generated using both the input signal P_CLK_IN output from the control pulse generating unit 115 and the signal P_EXP for controlling the exposure period. The control pulse generating unit 115 preferably has, for example, a frequency dividing circuit. This allows simple control and reduces an increase in the number of elements.

[0025] The signal output from the photoelectric conversion element 102 of the pixel is processed by the signal processing circuit 103. The signal processing circuit 103 is provided with a counter, a memory, etc., and the memory holds digital values.

[0026] The horizontal scanning circuit unit 111 inputs a control pulse for sequentially selecting each column to the signal processing circuit 103 in order to read out the signal from the memory of each pixel in which the digital signal is held.

[0027] A signal is output to the signal line 113 from the signal processing circuit 103 of the pixel selected by the vertical scanning circuit unit 110 for the selected column.

[0028] The signal output to the signal line 113 is output via an output circuit 114 to a recording unit or a signal processing unit outside the photoelectric conversion device 100 .

[0029] In Fig. 2, the photoelectric conversion elements in the pixel region may be arranged one-dimensionally. The effect of the present invention can be obtained even with only one pixel, and the case of one pixel is also included in the present invention, but the effect of reducing power consumption of this embodiment can be easily obtained with a photoelectric conversion device having multiple pixels. The function of the signal processing unit does not necessarily need to be provided for each photoelectric conversion element, and for example, one signal processing unit may be shared by multiple photoelectric conversion elements and signal processing may be performed sequentially.

[0030] 2 and 3, a plurality of signal processing circuits 103 are arranged in a region overlapping the pixel region 12 in a planar view. A vertical scanning circuit section 110, a horizontal scanning circuit section 111, a readout circuit 112, an output circuit 114, and a control pulse generating section 115 are arranged so as to overlap between an end of the sensor substrate 11 and an end of the pixel region 12 in a planar view. In other words, the sensor substrate 11 has the pixel region 12 and a non-pixel region arranged around the pixel region 12. A plurality of signal processing circuits 103 are arranged in a region overlapping the non-pixel region in a planar view. A vertical scanning circuit section 110, a horizontal scanning circuit section 111, a readout circuit 112, an output circuit 114, and a control pulse generating section 115 are arranged.

[0031] The arrangement of the signal lines 113, the readout circuits 112, and the output circuits 114 is not limited to that shown in Fig. 3. For example, the signal lines 113 may be arranged to extend in the row direction, and the readout circuits 112 may be arranged at the ends of the signal lines 113.

[0032] Fig. 4 is an example of a block diagram including the equivalent circuits of Fig. 2 and Fig. 3. Fig. 4(a) is an example in which the signal generating unit 215 is provided in common to a plurality of pixels, and Fig. 4(b) is an example in which the control signal P_CLK can be controlled for each pixel.

[0033] In FIG. 4, a photoelectric conversion element 102 having an APD 201 is provided on a sensor substrate 11, and other members are provided on a circuit substrate 21.

[0034] The APD201 generates a pair of electric charges according to the incident light by photoelectric conversion. One of the two nodes of the APD201 is connected to a power supply line to which a driving voltage VL (first voltage) is supplied. The other of the two nodes of the APD201 is connected to a power supply line to which a driving voltage VH (second voltage) higher than the voltage VL supplied to the anode is supplied. In FIG. 4, one node of the APD201 is an anode, and the other node of the APD is a cathode. A reverse bias voltage is supplied to the anode and cathode of the APD201 such that the APD201 performs an avalanche multiplication operation. By supplying such a voltage, the electric charges generated by the incident light undergo avalanche multiplication, and an avalanche current is generated.

[0035] When a reverse bias voltage is supplied, there are two modes: a Geiger mode in which the potential difference between the anode and cathode is greater than the breakdown voltage, and a linear mode in which the potential difference between the anode and cathode is close to or less than the breakdown voltage.

[0036] An APD operated in Geiger mode is called a SPAD. For example, the voltage VL (first voltage) is −30 V, and the voltage VH (second voltage) is 1 V. The APD 201 may be operated in either linear mode or Geiger mode. In the case of a SPAD, the potential difference is larger than that of a linear mode APD, and the effect of withstanding voltage is more pronounced, so a SPAD is preferable.

[0037] The switch 202 is connected to a power supply line to which the driving voltage VH is supplied and one of the anode and cathode nodes of the APD 201. The switch 202 switches the resistance value between the APD 201 and the power supply line to which the driving voltage VH is supplied. Here, switching the resistance value means preferably changing the resistance value by 10 times or more, and more preferably changing the resistance value by 100 times or more. Hereinafter, a lowering of the resistance value is also referred to as the switch 202 being turned on, and an increasing of the resistance value is also referred to as the switch 202 being turned off. The switch 202 functions as a quenching element. The switch 202 functions as a load circuit (quenching circuit) during signal multiplication by avalanche multiplication, and has a function of suppressing the voltage supplied to the APD 201 and suppressing the avalanche multiplication (quenching operation). The switch 202 also has a function of returning the voltage supplied to the APD 201 to the driving voltage VH by flowing a current equivalent to the voltage drop caused by the quenching operation (recharge operation).

[0038] 4 shows a case where the switch 202 is a PMOS transistor. A control signal P_CLK for the switch 202, which is supplied from the signal generating unit 215, is applied to a gate electrode of the MOS transistor that constitutes the switch 202. In this embodiment, the voltage applied to the gate electrode of the switch 202 is controlled to control the on and off of the switch 202.

[0039] The signal processing circuit 103 has a waveform shaping section 210, a counter circuit 211, and a selection circuit 212. In Fig. 4, the signal processing circuit 103 has the waveform shaping section 210, the counter circuit 211, and the selection circuit 212, but in this specification, it is sufficient that the signal processing circuit 103 has at least one of the waveform shaping section 210, the counter circuit 211, and the selection circuit 212.

[0040] The waveform shaping unit 210 shapes the potential change of the cathode of the APD 201 obtained when a photon is detected, and outputs a pulse signal. The node on the input side of the waveform shaping unit 210 is nodeA, and the node on the output side is nodeB. The waveform shaping unit 210 changes the output potential from the node nodeB depending on whether the input potential to the node nodeA is equal to or higher than a predetermined value or lower. For example, in FIG. 5, when the input potential to the node nodeA becomes a high potential equal to or higher than the judgment threshold, the output potential from the node nodeB becomes a low level. When the input potential to the node nodeA becomes a potential lower than the judgment threshold, the output potential from the node nodeB becomes a high level. For example, an inverter circuit is used as the waveform shaping unit 210. In FIG. 4, an example in which one inverter is used as the waveform shaping unit 210 is shown, but a circuit in which multiple inverters are connected in series may be used, or other circuits having a waveform shaping effect may be used.

[0041] It is possible to perform a quench operation and a recharge operation using the switch 202 according to the avalanche multiplication in the APD 201, but depending on the timing of photon detection, it may not be determined as an output signal. For example, assume that avalanche multiplication occurs in the APD, the input potential to the node nodeA becomes low level, and a recharge operation is performed. In general, the decision threshold of the waveform shaping unit 210 is set to a potential higher than the potential difference at which avalanche multiplication occurs in the APD. When a photon is incident when the potential of the node nodeA is lower than the decision threshold due to the recharge operation and the potential is such that avalanche multiplication is possible in the APD, avalanche multiplication occurs in the APD and the voltage of the nodeA drops. In other words, since the potential of the nodeA drops at a voltage lower than the decision threshold, the output potential from the node nodeB does not change even though a photon is detected. Therefore, even though avalanche multiplication occurs, it is not determined as a signal. In particular, under high illuminance, photons enter continuously in a short period of time, making it difficult to determine as a signal. As a result, even if the illuminance is high, the actual number of incident photons is likely to deviate from the output signal.

[0042] In contrast, by applying a control signal P_CLK to the switch 202 to switch the switch 202 on and off, it is possible to determine a signal even when photons enter the APD continuously in a short period of time. In Fig. 5, an example is described in which the control signal P_CLK is a pulse signal with a repetitive cycle. In other words, in Fig. 5, a form is described in which the switch 202 is switched on and off at a predetermined clock frequency. However, the effect of suppressing an increase in power consumption of the photoelectric conversion device can be obtained even if the pulse signal is not a signal with a repetitive cycle.

[0043] The counter circuit 211 counts the pulse signal output from the waveform shaping unit 210 and holds the count value. When a control pulse pRES is supplied via a drive line 213, the signal held in the counter circuit 211 is reset.

[0044] A control pulse pSEL is supplied to the selection circuit 212 from the vertical scanning circuit unit 110 in Fig. 3 via a drive line 214 (not shown in Fig. 3) in Fig. 4, and switches between electrical connection and non-connection between the counter circuit 211 and the signal line 113. The selection circuit 212 includes, for example, a buffer circuit for outputting a signal. An output signal OUT shown in Fig. 4 is a signal output from a pixel.

[0045] The electrical connection may be switched by disposing a switch such as a transistor between the switch 202 and the APD 201 or between the photoelectric conversion element 102 and the signal processing circuit 103. Similarly, the supply of the voltage VH or the voltage VL to the photoelectric conversion element 102 may be electrically switched using a switch such as a transistor.

[0046] In this embodiment, a configuration using the counter circuit 211 has been shown. However, instead of the counter circuit 211, a photoelectric conversion device 100 may be configured to acquire the pulse detection timing using a time-to-digital converter (hereinafter, TDC) and a memory. In this case, the generation timing of the pulse signal output from the waveform shaping unit 210 is converted into a digital signal by the TDC. To measure the timing of the pulse signal, a control pulse pREF (reference signal) is supplied to the TDC from the vertical scanning circuit unit 110 in FIG. 1 via a drive line. The TDC acquires, as a digital signal, a signal obtained by converting the input timing of the signal output from each pixel via the waveform shaping unit 210 into a relative time based on the control pulse pREF.

[0047] As shown in FIG. 4(b), the signal generating unit 215 may be provided for each pixel. In FIG. 4(b), the circuits after the waveform shaping unit 210 shown in FIG. 4(a) and the signal generating unit are omitted. It is assumed that the signal generating unit 215 in FIG. 4(a) is provided for each pixel. In FIG. 4(b), a logic circuit is provided within the pixel to determine whether or not to supply a pulse signal to the switch 202. A signal P_EXP that controls the exposure period and an input signal P_CLK_IN that controls the control signal P_CLK are input to the logic circuit. Then, an inverted signal is output. For example, when the signal P_EXP that controls the exposure period is at a low level and the input signal P_CLK_IN is at a low level, a high-level signal is output from the control signal P_CLK. That is, the switch is turned off. Also, when the signal P_EXP that controls the exposure period is at a high level and the input signal P_CLK_IN is at a high level, a low-level signal is output from the control signal P_CLK. That is, the switch is turned on. Furthermore, when either the signal P_EXP that controls the exposure period or the input signal P_CLK_IN is at a low level, a high-level signal is output as the control signal P_CLK. That is, the switch 202 is turned off. It is preferable to control the switch for each pixel in this manner. Note that, when the circuit diagram of FIG. 4(b) is used, as will be described in the second embodiment below, when the exposure period P becomes a low level, the control signal P_CLK is maintained at a high level. That is, the switch is turned off.

[0048] FIG. 5 is a diagram showing a schematic diagram of the relationship between the control signal P_CLK of the switch, the potential of the node nodeA, the potential of the node nodeB, and the output signal. In this embodiment, when the control signal P_CLK is at a high level, the drive voltage VH is not easily supplied to the APD, and when the control signal P_CLK is at a low level, the drive voltage VH is supplied to the APD. The high level of the control signal P_CLK is, for example, 1V, and the low level of the control signal P_CLK is, for example, 0V. When the control signal P_CLK is at a high level, the switch is turned off, and when the control signal P_CLK is at a low level, the switch is turned on. The resistance value of the switch when the control signal P_CLK is at a high level is higher than the resistance value of the switch when the control signal P_CLK is at a low level. When the control signal P_CLK is at a high level, even if avalanche multiplication occurs in the APD, a recharge operation is not easily performed, so the potential supplied to the APD is a potential equal to or lower than the breakdown voltage of the APD. Therefore, the avalanche multiplication operation in the APD stops.

[0049] As shown in Fig. 4, it is preferable to configure switch 202 with one transistor, and to perform quenching and recharging operations with one transistor. This makes it possible to reduce the number of circuits compared to when the quenching and recharging operations are performed with different circuit elements. In particular, when each pixel has a counter circuit and the SPAD signal is read out for each pixel, it is preferable to reduce the circuit area used for the switch in order to place the counter circuit, and the effect of configuring switch 202 with one transistor becomes significant.

[0050] At time t1, the control signal P_CLK changes from high level to low level, the switch is turned on, and the recharge operation of the APD is started. This causes the potential of the cathode of the APD to transition to high level. Then, the potential difference between the potentials applied to the anode and cathode of the APD becomes a state in which avalanche multiplication is possible. The potential of the cathode is the same as that of the node nodeA. Therefore, when the potential of the cathode transitions from low level to high level, the potential of the node nodeA becomes equal to or higher than the judgment threshold value at time t2. At this time, the pulse signal output from the node nodeB is inverted and goes from high level to low level. After that, the potential difference of the drive voltage VH-drive voltage VL is applied to the APD 201. The control signal P_CLK becomes high level, and the switch is turned off.

[0051] Next, at time t3, when a photon is incident on the APD 201, avalanche multiplication occurs in the APD 201, an avalanche multiplication current flows through the switch 202, and the voltage of the cathode drops. That is, the voltage of the node nodeA drops. When the amount of voltage drop becomes larger and the voltage difference applied to the APD 201 becomes smaller, the avalanche multiplication of the APD 201 stops as at time t2, and the voltage level of the node nodeA does not drop by more than a certain value. When the voltage of the node nodeA becomes lower than the decision threshold while the voltage of the node nodeA is dropping, the voltage of the node nodeB goes from low level to high level. That is, the part of the output waveform at the node nodeA that exceeds the decision threshold is waveform-shaped by the waveform shaping unit 210 and output as a signal at the nodeB. Then, it is counted by the counter circuit, and the count value of the counter signal output from the counter circuit increases by 1 LSB.

[0052] Between time t3 and time t4, photons are incident on the APD. However, since the switch is off and the voltage applied to the APD 201 does not have a potential difference that allows avalanche multiplication, the voltage level of the node A does not exceed the decision threshold.

[0053] At time t4, the control signal P_CLK changes from high to low, turning the switch on. As a result, a current flows to node A to compensate for the voltage drop from the drive voltage VL, and the voltage of node A transitions to its original voltage level. At this time, the voltage of node A becomes equal to or higher than the decision threshold at time t5, so the pulse signal of node B is inverted and goes from high to low.

[0054] At time t6, node A is stabilized to the original voltage level, and the control signal P_CLK goes from low to high. Therefore, the switch is turned off. After this, the potentials of the nodes and signal lines change in response to the control signal P_CLK and the incidence of photons, as described from time t1 to time t6.

[0055] The photoelectric conversion devices of the respective embodiments will be described below.

[0056] <Embodiment 1> 6(a), (b), and (c) are timing charts showing the relationship between the exposure period P and the control signal P_CLK in embodiment 1. Fig. 6(a) is a diagram showing the pulse signal of the control signal P_CLK in the case of an exposure period P1. Fig. 6(b) is a diagram showing the pulse signal of the control signal P_CLK in the case of an exposure period P2 that is longer than the exposure period P1. Fig. 6(c) is a diagram showing the pulse signal of the control signal P_CLK in the case of an exposure period P3 that is longer than the exposure period P2.

[0057] In this embodiment, the exposure period P is, for example, a period during which the mechanical shutter or electronic shutter is open, and the non-exposure period is, for example, a period during which the mechanical shutter or electronic shutter is closed. The exposure period P may be defined by adjusting the bias applied to the APD 201 to change whether or not a photon signal can be acquired. The exposure period P refers to a period during which the APD 201 is in an operable state and the APD and the signal processing circuit are in a state in which the signal can be read out. Here, the state in which the APD and the signal processing circuit are in a state in which the APD is capable of avalanche multiplication. During this period, it can also be said that the counter circuit is in operation. At this time, the period during which the APD is in a quench operation, which is a state in which the APD is turned off based on the incidence of photons, is part of the operable state. On the other hand, the period during which the APD is shielded by a shutter or the like, or the period during which the APD is controlled so as not to be avalanche multiplied regardless of the presence or absence of the incidence of photons, is a non-exposure period.

[0058] As shown in FIG. 6(a) to FIG. 6(c), in this embodiment, the pulse signal of the control signal P_CLK is changed in accordance with the exposure period P. A first exposure period and a second exposure period that is a period different from the first exposure period are assumed. The control signal is controlled so that a value obtained by dividing the number of control signals P_CLK in the first exposure period by the first exposure period and multiplying the result by the first exposure period is different from a value obtained by dividing the number of control signals P_CLK in the second exposure period by the second exposure period and multiplying the result by the first exposure period. In this embodiment, since the switch 202 is a PMOS transistor, the number of control signals P_CLK refers to the number of off pulses in a predetermined exposure period.

[0059] Moreover, the control signal P_CLK is controlled so that the average frequency of the control signal P_CLK in the first exposure period is different from the average frequency of the control signal P_CLK in the second exposure period, which is different from the first exposure period, when compared per unit time. Here, the average frequency of the control signal P_CLK in the exposure period refers to a frequency obtained by averaging the pulse signals in the exposure period so that they are uniform. For example, when pulse signals are densely arranged in the first half of the exposure period and no pulse signals are arranged in the second half, the frequency adjusted so that the pulse signals are evenly arranged over the entire period is called the average frequency of the control signal P_CLK in the exposure period. In this embodiment, the average frequency is controlled to be different when comparing the average frequency in the first exposure period and the second exposure period. Note that the unit time is the time during which at least two pulse signals of the control signal P_CLK are included.

[0060] In Fig. 6, the pulse signal of the control signal P_CLK is controlled so that the number of pulses of the pulse signal in the exposure period P remains the same even if there is a change in the exposure period P. Suppose the number of pulses of the control signal P_CLK in the exposure period P1 shown in Fig. 6(a) is N. The number of pulses of the control signal P_CLK is also N in the exposure period P2 shown in Fig. 6(b). Furthermore, the number of pulses of the control signal P_CLK is also N in the exposure period P3 shown in Fig. 6(c).

[0061] The above configuration can prevent an increase in power consumption in each pixel. In addition, if the signal processing unit has a counter circuit, it is possible to realize clock driving that maintains an appropriate upper count limit.

[0062] Details will be described below in comparison with the comparative embodiment in Fig. 7. Fig. 7(a), (b), and (c) are timing charts showing the relationship between the exposure period P and the control signal P_CLK in the comparative embodiment. In Fig. 7, the same components as in Fig. 6 are given the same reference numerals and descriptions thereof will be omitted.

[0063] In the comparative embodiment shown in FIG. 7, the period of the control signal P_CLK remains the same even if the exposure period changes. In this embodiment, since the switch 202 is a PMOS transistor, the period of the control signal P_CLK refers to the period from the falling edge of a pulse to the next falling edge. For example, the period of the control signal P_CLK remains the same in both the case of the exposure period P1 and the case of the exposure period P2. Therefore, in the comparative embodiment, when the exposure period P changes, the number of pulses of the pulse signal in the exposure period P changes. In this embodiment, depending on the exposure period, there is a possibility that power consumption may increase unintentionally. Even if the clock frequency increases in the signal processing circuit at the subsequent stage, the result is not reflected, and there is a possibility that power consumption may be wasted. For example, when the signal processing circuit is a counter circuit, a clock frequency at a value larger than the readable counter upper limit value is not read as a signal. In other words, even if a quench operation and a recharge operation are performed at a value larger than the counter upper limit value to read a signal, the counter circuit does not add the count value. Therefore, at a value larger than the counter upper limit value, power consumption is wasted. On the other hand, if the clock frequency is made too high, the number of readable photons may decrease, and the dynamic range may decrease.

[0064] In response to this, as described above, by controlling the number of pulses of the pulse signal of the control signal P_CLK in accordance with the exposure period, it is possible to reduce unnecessary power consumption.

[0065] As shown in Fig. 6, the period of the control signal P_CLK changes depending on the exposure period. For example, the period of the control signal P_CLK in the exposure period P1 is shorter than the period of the control signal P_CLK in the exposure period P2. In this way, in Fig. 6, even if the exposure period changes to the exposure period P1, the exposure period P2, and the exposure period P3, the number of pulses of the control signal P_CLK is always N. This makes it possible to reduce the increase in power consumption.

[0066] The number of pulses in the exposure period can be set to any value. If the downstream signal processing circuit is a counter circuit, it is preferable to set the number of pulses to, for example, the upper count limit of the counter circuit. This makes it possible to suppress the decrease in the dynamic range while suppressing the generation of unnecessary power consumption.

[0067] It is preferable that the first pulse width in the exposure period P1 and the first pulse width in the exposure period P2 are the same. Here, the first pulse width refers to a period during which the switch 202 is turned on by the control signal P_CLK. In this embodiment, since the switch 202 is a PMOS transistor, the first pulse width refers to a period during which the control signal P_CLK is at a low level (first level). As described above, during a period during which the control signal P_CLK is at a high level (second level), the switch is turned off, and the APD is unlikely to perform a recharge operation. In this specification, the period during which the control signal P_CLK maintains the first level state is described as the "first pulse width," and the period during which the control signal P_CLK maintains the second level state is described as the "second pulse width." In FIG. 6, the first pulse width in the exposure period P1 and the first pulse width in the exposure period P2 are the same, but the second pulse width in the exposure period P1 and the second pulse width in the exposure period P2 are different.

[0068] Conversely, during the period when P_CLK is at a low level, the switch is on and the APD performs a recharge operation. If the period when the control signal P_CLK is at a low level becomes long, there is a possibility that the recharge operation will be performed multiple times. As explained using FIG. 5, when the potential of node A is lower than the decision threshold, the potential of node A drops, and the signal of node B is not inverted and cannot be read out correctly as a signal value. Therefore, by making the period during which the APD performs a recharge operation a constant period even if the cycle of the pulse signal changes, multiple recharge operations will not be performed during the period when the control signal P_CLK is at a low level, and it is possible to reduce missed photon detection.

[0069] FIG. 8 is an example of a flowchart of the operation of the photoelectric conversion device in this embodiment.

[0070] In step S1, the exposure period and the number of pulses of the control signal P_CLK are set. Here, the clock frequency of the control signal P_CLK is set. In step S2, shooting begins. In step S3, it is decided whether or not to change the exposure period. Whether or not to change the exposure period can be determined based on information obtained from the image captured previously (information from the previous frame). If the information obtained from the image from the previous frame is too bright, the exposure period is shortened, and if the information obtained from the image is too dark, the exposure period is lengthened. In addition, the exposure period can also be switched manually or automatically.

[0071] If it is determined in step S3 that the exposure period should be changed, then in step S4 it is determined whether or not to lengthen the exposure period setting. If it is determined in step S3 that the exposure period should be changed, then the process proceeds to step S4, and if it is determined in step S3 that the exposure period should not be changed, then the process proceeds to step S.

[0072] If it is determined in step S4 that the exposure period is to be set longer, the average clock frequency of the control signal P_CLK during the exposure period is lowered in step S5. At this time, control is performed so that the number of pulses of the control signal P_CLK during the exposure period does not change before and after the exposure period is changed. If it is determined in step S4 that the exposure period is not to be set longer (shortened), the average clock frequency of the control signal P_CLK during the exposure period is set higher in step S6. Even in this case, control is performed so that the number of pulses of the control signal P_CLK before the exposure period is changed and the number of pulses of the control signal P_CLK during the exposure period do not change.

[0073] After step S5 or step S6, the process proceeds to step S7, where it is determined whether or not to end the shooting. If it is determined in step S7 that the exposure period is to end, the shooting is ended in step S8. If it is determined in step S7 that the shooting is not to end, the process returns to step S3, and steps S3 to S7 are repeated. Then, if it is determined in step S7 that the shooting is to end, the shooting is ended in step S8.

[0074] It is possible to operate according to the above flowchart.

[0075] A modified example of the timing chart is shown in Fig. 9. In Fig. 6, the pulse signal P_CLK is input at a constant cycle, but this is not limiting. As shown in Figs. 9(a) to 9(c), the pulse signal P_CLK may be input with a varying pulse cycle. Note that, although the cycle is varied for each pulse signal in Figs. 9(a) to 9(c), the cycle may be varied for every three or four or more pulse signals.

[0076] 9(d) to 9(f), in the same period as exposure period P1, the number of pulses in exposure period P2 and the number of pulses in exposure period P3 may be the same as the number of pulses in exposure period P1. That is, the control signal P_CLK may be continuously supplied to the switch in the first half of exposure period P2, and the control signal P_CLK may not be supplied in the second half of exposure period P2. Even in such a case, the effect of this embodiment can be obtained as long as the number of pulses in exposure period P2 is the same as the number of pulses in exposure period P1. Similarly, the effect of this embodiment can be obtained as long as the number of pulses in exposure period P3 is the same as the number of pulses in exposure period P1.

[0077] <Embodiment 2> 10(a), (b), and (c) are timing charts showing the relationship between the exposure period P and the control signal P_CLK in embodiment 2. Fig. 10(a) is a diagram showing the pulse of the control signal P_CLK in the case of an exposure period P1. Fig. 10(b) is a diagram showing the pulse signal of the control signal P_CLK in the case of an exposure period P2 that is longer than the exposure period P1. Fig. 10(c) is a diagram showing the pulse signal of the control signal P_CLK in the case of an exposure period P3 that is longer than the exposure period P2.

[0078] The photoelectric conversion device of this embodiment differs from embodiment 1 in that it does not generate a pulse signal of the control signal P_CLK at the timing when the exposure period changes to the non-exposure period. Since this point and other points other than those described below are substantially the same as embodiment 1, the same reference numerals are used for the same configurations as embodiment 1, and the description thereof may be omitted.

[0079] In this embodiment, the control signal P_CLK is stopped at a low level at the end of the exposure period, i.e., the switch is kept in an off state, and then the control signal P_CLK is changed to a high level at the start of the exposure period.

[0080] During the non-exposure period, since no photons are incident on the APD, there is no need to control the on / off of the switch. Therefore, by not changing the control signal P_CLK that controls the on / off of the switch, it is possible to suppress the power consumption caused by turning the switch on and off.

[0081] The end timing of the exposure period may be controlled so as to be synchronized with the shutter.

[0082] Also, as shown in FIG. 10, it is preferable to stop the pulse signal of the control signal P_CLK simultaneously with the end timing of the exposure period in order to reduce power consumption, but it may be slightly shifted from the end timing of the exposure period.

[0083] According to this embodiment, it is possible to suppress an increase in power consumption, similarly to embodiment 1. Moreover, compared to embodiment 1, it is possible to reduce the number of times the switch is turned on and off, so that it is possible to further suppress the power consumption of the photoelectric conversion device.

[0084] <Embodiment 3> The correction of the count value Nct obtained by the photoelectric conversion device according to the first or second embodiment will be described with reference to FIGS.

[0085] 11 is a diagram showing the relationship between the number of incident photons Nph and the count value Nct in each pixel of the photoelectric conversion device. Here, Nph is the number of photons (number of incident photons) that actually enter each pixel per exposure period T. Also, Nct is the count value of the pulse signal output from the waveform shaping unit 210, counted by the counter circuit 211. Here, if the frequency of P_CLK is f, the number of pulses of P_CLK within the exposure period T is equal to f×T.

[0086] When the switch 202 of each pixel is controlled by a periodic pulse, the count value Nct of each pixel has characteristics like the curve A in FIG. 11. That is, the count value Nct increases with an increase in the number of incident photons Nph, and is counted with the number of pulses f×T as the upper limit. Since one recharge operation is performed per P_CLK pulse, only one photon can be counted per P_CLK period. Therefore, if multiple photons are incident per P_CLK period, the second and subsequent photons in each period are not counted. Therefore, in a high-illumination environment, the number of photons that are not counted even if they are incident on a pixel increases, and the count value Nct and the actual number of incident photons Nph diverge.

[0087] Therefore, in this embodiment, a correction is performed to convert the count value Nct into a value equivalent to the actual number of incident photons Nph. This correction is performed by a correction circuit 118 connected to the circuit board 21. The correction circuit 118 may be located outside the photoelectric conversion unit 100 as shown in FIG. 12, or may be located inside the photoelectric conversion unit 100 (for example, the signal processing circuit 103). Note that 119 in FIG. 12 is an external output circuit. The relationship between Nct and Nph is expressed by the following equation based on natural logarithms. Nct=f×T×(1-exp(-Nph / (f×T))) Equation (1)

[0088] In other words, in this correction circuit 118, when the count value Nct, the frequency f of the pulse signal, and the length T of the exposure period are used as explanatory variables and the number of incident photons Nph is used as an objective variable, the explanatory variables and the objective variable are described by a relational equation based on natural logarithms.

[0089] The corrected count value Nct can be represented by the dotted line B in Figure 11. By performing this correction, the count value A, which shows nonlinear characteristics with respect to the number of incident photons, can be corrected to the count value B, which is linear and equivalent to the number of incident photons.

[0090] Here, the correction formula is determined by the value of f×T. Therefore, even if the value of the exposure period T is determined for each different P_CLK frequency f, as long as the combination is such that f×T is constant, the value of the count value Nct relative to the number of incident photons Nph will not change.

[0091] Furthermore, when two types of P_CLK frequencies are mixed during one exposure period, correction can be made using the following formula. Nct=f1×T1×(1-exp(-Nph1 / (f1×T1)))+f2×T2×(1-exp(-Nph2 / (f2×T2))) Equation (2)

[0092] In this case, T1 is the period during which the pulse signal operates at the first frequency f1, T2 is the period during which the pulse signal operates at the second frequency f2, and T1+T2 is the exposure period T. In addition, Nph1 is the number of incident photons in T1, and Nph2 is the number of incident photons in T2. ​​Nph1 and Nph2 are each expressed by the following formulas. Nph1=Nph×T1 / (T1+T2) Equation (3) Nph2=Nph×T2 / (T1+T2) Equation (4)

[0093] Thus, the number of incident photons is determined by the ratio of the exposure period at each frequency to the total exposure period T.

[0094] Here, consider a case where multiple frequencies are mixed within an exposure period as shown in Fig. 13. For example, as shown in Fig. 13(a), pulse signals may be input in groups of frequencies, such that the frequency of the pulse signal input in the first half of the exposure period is f1 and the frequency of the pulse signal input in the second half is f2. Alternatively, as shown in Fig. 13(b), two pulses of different frequencies may alternate, with one pulse each of pulse signals of frequency f1 and frequency f2. Figs. 13(a) and 13(b) show an example where pulse signals of two different frequencies are mixed within an exposure period, but pulse signals of three or more different frequencies may be mixed.

[0095] In this way, by mixing pulse signals of multiple frequencies within the exposure period, the slope of the count value at high illuminance becomes larger compared to when only low-frequency pulse signals are input. Therefore, the gradation of the count value is maintained even at high illuminance, and the dynamic range can be expanded compared to when the pulse signal frequency is one type.

[0096] Furthermore, when n types of P_CLK frequencies are mixed within an exposure period, the correction formula is expressed by the following formula, where n is a natural number of 2 or more. Nct=f1×T1×(1-exp(-Nph1 / (f1×T1)))+f2×T2×(1-exp(-Nph2 / (f2×T2)))+···+f n-1 ×T n-1×(1-exp(-Nph n―1 / (f n-1 ×T n-1 )))+f n ×T n ×(1-exp(-Nph n / (f n ×T n ))) Equation (5)

[0097] The sum of the periods during which the pulse signals of each frequency are input is equal to the exposure period T. In addition, the number of incident photons during the period during which the pulse signals of each frequency are input is expressed by the following formula. Nph1 = Nph × T1 / (T1 + T2 + + T n―1 +T n ) Formula (6) Nph2 = Nph × T2 / (T1 + T2 + + T n―1 +T n ) Formula (7) Nph n―1 = Nph × T n―1 / (T1+T2+···+T n―1 +T n ) Formula (8) Nph n = Nph × T n / (T1+T2+···+T n―1 +T n ) Formula (9)

[0098] The number of incident photons is determined by the ratio of the exposure period at each frequency to the total exposure period T.

[0099] The correction performed by the correction circuit 118 is not limited to the correction according to the above formula performed for each count value. For example, the correction circuit 118 may have a three-dimensional table of combinations of the exposure period T, the frequency f of the pulse signal, and the count value Nct. The number of incident photons Nph can be roughly calculated by selecting the value closest to the measured value from the values ​​in the table held by the correction circuit 118. In this case, the values ​​in the table are set by a relational expression based on the natural logarithm corresponding to the combination of f and T, similar to the above formula.

[0100] The correction to the count value is not limited to this, and the correction steps may be reduced by performing other corrections simultaneously with this correction in the correction circuit 118. For example, in preparation for display, so-called gamma correction may be performed to adjust the brightness of an image formed based on the count value.

[0101] <Embodiment 4> FIG. 14 is a block diagram showing a configuration of a light detection system 1200 according to this embodiment. The light detection system 1200 according to this embodiment includes a photoelectric conversion device 1204. Here, any of the photoelectric conversion devices described in the above embodiments can be applied to the photoelectric conversion device 1204. The light detection system 1200 can be used as, for example, an imaging system. Specific examples of the imaging system include a digital still camera, a digital camcorder, and a surveillance camera. FIG. 14 shows an example of a digital still camera as the light detection system 1200.

[0102] 1 includes a photoelectric conversion device 1204, a lens 1202 that forms an optical image of a subject on the photoelectric conversion device 1204, an aperture 1203 that varies the amount of light passing through the lens 1202, and a barrier 1201 that protects the lens 1202. The lens 1202 and the aperture 1203 form an optical system that focuses light on the photoelectric conversion device 1204. A shutter is disposed between the aperture 1203 and the photoelectric conversion device 1204. The exposure period of the photoelectric conversion device is controlled by opening and closing the shutter.

[0103] The light detection system 1200 has a signal processing unit 1205 that processes an output signal output from the photoelectric conversion device 1204. The signal processing unit 1205 performs signal processing operations such as performing various corrections and compression on an input signal as necessary and outputting the signal. The light detection system 1200 further has a buffer memory unit 1206 for temporarily storing image data, and an external interface unit (external I / F unit) 1209 for communicating with an external computer or the like. The light detection system 1200 further has a recording medium 1211 such as a semiconductor memory for recording or reading out imaging data, and a recording medium control interface unit (recording medium control I / F unit) 1210 for recording or reading out the imaging data on the recording medium 1211. The recording medium 1211 may be built in the light detection system 1200 or may be removable. In addition, communication between the recording medium control I / F unit 1210 and the recording medium 1211 and communication from the external I / F unit 1209 may be performed wirelessly.

[0104] Furthermore, the light detection system 1200 has an overall control / calculation unit 1208 that performs various calculations and controls the entire digital still camera, and a timing generation unit 1207 that outputs various timing signals to the photoelectric conversion device 1204 and the signal processing unit 1205. Here, the timing signals and the like may be input from the outside, and the light detection system 1200 may have at least the photoelectric conversion device 1204 and the signal processing unit 1205 that processes the output signal output from the photoelectric conversion device 1204. As described in the fourth embodiment, the timing generation unit 1207 may be mounted on the photoelectric conversion device. The overall control / calculation unit 1208 and the timing generation unit 1207 may be configured to implement a part or all of the control functions of the photoelectric conversion device 1204.

[0105] The photoelectric conversion device 1204 outputs an image signal to the signal processing unit 1205. The signal processing unit 1205 performs a predetermined signal processing on the image signal output from the photoelectric conversion device 1204 and outputs image data. The signal processing unit 1205 generates an image using the image signal. The signal processing unit 1205 may perform distance measurement calculation on the signal output from the photoelectric conversion device 1204. The signal processing unit 1205 and the timing generating unit 1207 may be mounted on the photoelectric conversion device. That is, the signal processing unit 1205 and the timing generating unit 1207 may be provided on a substrate on which pixels are arranged, or may be provided on a different substrate. By configuring an imaging system using the photoelectric conversion device of each of the above-mentioned embodiments, an imaging system capable of acquiring a better quality image can be realized.

[0106] <Embodiment 5> FIG. 15 is a block diagram showing an example of the configuration of a range image sensor, which is an electronic device that uses the photoelectric conversion device described in the above embodiment.

[0107] 15, the distance image sensor 401 is configured to include an optical system 407, a photoelectric conversion device 408, an image processing circuit 404, a monitor 405, and a memory 406. The distance image sensor 401 can obtain a distance image according to the distance to the subject by receiving light (modulated light or pulsed light) that is projected from a light source device 409 toward the subject and reflected from the surface of the subject.

[0108] The optical system 407 is configured to have one or more lenses, and guides image light (incident light) from a subject to a photoelectric conversion device 408 , forming an image on the light receiving surface (sensor portion) of the photoelectric conversion device 408 .

[0109] As the photoelectric conversion device 408 , the photoelectric conversion device of each of the above-mentioned embodiments is applied, and a distance signal indicating a distance determined from a light reception signal output from the photoelectric conversion device 408 is supplied to the image processing circuit 404 .

[0110] The image processing circuit 404 performs image processing to construct a distance image based on the distance signal supplied from the photoelectric conversion device 408. The distance image (image data) obtained by this image processing is then supplied to a monitor 405 for display, or supplied to a memory 406 for storage (recording).

[0111] In the range image sensor 401 configured in this way, by applying the above-described photoelectric conversion device, it is possible to obtain, for example, a more accurate range image as the pixel characteristics improve.

[0112] <Embodiment 6> The technology according to the present disclosure (the present technology) can be applied to various products. For example, the technology according to the present disclosure may be applied to an endoscopic surgery system.

[0113] FIG. 16 is a diagram showing an example of a schematic configuration of an endoscopic surgery system to which the technology according to the present disclosure (the present technology) can be applied.

[0114] 16 shows a state in which an operator (doctor) 1131 is performing surgery on a patient 1132 on a patient bed 1133 using an endoscopic surgery system 1003. As shown in the figure, the endoscopic surgery system 1003 is composed of an endoscope 1100, a surgical tool 1110, and a cart 1134 on which various devices for endoscopic surgery are mounted.

[0115] The endoscope 1100 is composed of a lens barrel 1101, a region of a predetermined length from the tip of which is inserted into a body cavity of a patient 1132, and a camera head 1102 connected to the base end of the lens barrel 1101. In the illustrated example, the endoscope 1100 is configured as a so-called rigid lens barrel having a rigid lens barrel 1101, but the endoscope 1100 may be configured as a so-called flexible lens barrel having a flexible lens barrel.

[0116] An opening into which an objective lens is fitted is provided at the tip of the lens barrel 1101. A light source device 1203 is connected to the endoscope 1100, and light generated by the light source device 1203 is guided to the tip of the lens barrel 1101 by a light guide extending inside the lens barrel 1101, and is irradiated via the objective lens toward an observation target in a body cavity of a patient 1132. The endoscope 1100 may be a direct-viewing endoscope, an oblique-viewing endoscope, or a side-viewing endoscope.

[0117] An optical system and a photoelectric conversion device are provided inside the camera head 1102, and reflected light (observation light) from an observation target is collected by the optical system onto the photoelectric conversion device. The observation light is photoelectrically converted by the photoelectric conversion device to generate an electrical signal corresponding to the observation light, i.e., an image signal corresponding to an observation image. The photoelectric conversion device described in each of the above-mentioned embodiments can be used as the photoelectric conversion device. The image signal is transmitted to a camera control unit (CCU) 1135 as RAW data.

[0118] The CCU 1135 is configured with a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), etc., and performs overall control of the operations of the endoscope 1100 and the display device 1136. Furthermore, the CCU 1135 receives an image signal from the camera head 1102, and performs various types of image processing on the image signal, such as development processing (demosaic processing), for displaying an image based on the image signal.

[0119] Under the control of the CCU 1135 , the display device 1136 displays an image based on an image signal that has been subjected to image processing by the CCU 1135 .

[0120] The light source device 1203 is composed of a light source such as an LED (Light Emitting Diode), and supplies the endoscope 1100 with irradiation light when photographing an operation site or the like.

[0121] The input device 1137 is an input interface for the endoscopic surgery system 1003. A user can input various information and instructions to the endoscopic surgery system 1003 via the input device 1137.

[0122] The treatment tool control device 1138 controls the driving of the energy treatment tool 1112 for cauterizing tissue, incising, sealing blood vessels, or the like.

[0123] The light source device 1203 that supplies irradiation light to the endoscope 1100 when photographing the surgical site can be composed of a white light source composed of, for example, an LED, a laser light source, or a combination of these. When the white light source is composed of a combination of RGB laser light sources, the output intensity and output timing of each color (each wavelength) can be controlled with high precision, so that the white balance of the captured image can be adjusted in the light source device 1203. In this case, it is also possible to capture images corresponding to each of the RGB colors in a time-division manner by irradiating the observation target with laser light from each of the RGB laser light sources in a time-division manner and controlling the driving of the image sensor of the camera head 1102 in synchronization with the irradiation timing. According to this method, a color image can be obtained without providing a color filter to the image sensor.

[0124] Furthermore, the light source device 1203 may be controlled to change the intensity of the light it outputs at predetermined time intervals. By controlling the driving of the image sensor of the camera head 1102 in synchronization with the timing of the change in the light intensity to acquire images in a time-division manner and synthesizing the images, it is possible to generate an image with a high dynamic range that is free of so-called blocked-up shadows and blown-out highlights.

[0125] The light source device 1203 may be configured to supply light in a predetermined wavelength band corresponding to the special light observation. In the special light observation, for example, the wavelength dependency of light absorption in body tissue is utilized. Specifically, a predetermined tissue such as blood vessels on the mucous membrane surface is photographed with high contrast by irradiating light in a narrower band than the irradiation light (i.e., white light) in normal observation. Alternatively, the special light observation may be a fluorescent observation in which an image is obtained by fluorescence generated by irradiating excitation light. In the fluorescent observation, it is possible to irradiate excitation light to the body tissue and observe the fluorescence from the body tissue, or to locally inject a reagent such as indocyanine green (ICG) into the body tissue and irradiate the body tissue with excitation light corresponding to the fluorescent wavelength of the reagent to obtain a fluorescent image. The light source device 1203 may be configured to supply narrow band light and / or excitation light corresponding to such special light observation.

[0126] <Embodiment 7> The light detection system and the moving body of this embodiment will be described with reference to Fig. 17 and Fig. 18. Fig. 17 is a schematic diagram showing a configuration example of the light detection system and the moving body according to this embodiment. Fig. 18 is a flow diagram showing the operation of the light detection system according to this embodiment. In this embodiment, an example of an in-vehicle camera is shown as the light detection system.

[0127] FIG. 17 shows an example of a vehicle system and a photodetection system mounted thereon for capturing images. The photodetection system 1301 includes a photoelectric conversion device 1302, an image preprocessing unit 1315, an integrated circuit 1303, and an optical system 1314. The optical system 1314 forms an optical image of a subject on the photoelectric conversion device 1302. The photoelectric conversion device 1302 converts the optical image of the subject formed by the optical system 1314 into an electrical signal. The photoelectric conversion device 1302 is any of the photoelectric conversion devices according to the above-mentioned embodiments. The image preprocessing unit 1315 performs predetermined signal processing on the signal output from the photoelectric conversion device 1302. The function of the image preprocessing unit 1315 may be incorporated in the photoelectric conversion device 1302. The light detection system 1301 is provided with at least two sets of an optical system 1314 , a photoelectric conversion device 1302 , and an image pre-processing unit 1315 , and the output from each set of image pre-processing units 1315 is input to the integrated circuit 1303 .

[0128] The integrated circuit 1303 is an integrated circuit for use in an imaging system, and includes an image processing unit 1304 including a memory 1305, an optical distance measuring unit 1306, a distance measuring calculation unit 1307, an object recognition unit 1308, and an abnormality detection unit 1309. The image processing unit 1304 performs image processing such as development processing and defect correction on the output signal of the image pre-processing unit 1315. The memory 1305 temporarily stores the captured image and stores the defective positions of the captured pixels. The optical distance measuring unit 1306 performs focusing and distance measurement of the subject. The distance measuring calculation unit 1307 calculates distance measurement information from multiple image data acquired by multiple photoelectric conversion devices 1302. The object recognition unit 1308 recognizes subjects such as cars, roads, signs, and people. When the abnormality detection unit 1309 detects an abnormality in the photoelectric conversion device 1302, it notifies the main control unit 1313 of the abnormality.

[0129] The integrated circuit 1303 may be realized by dedicated hardware, a software module, or a combination of these. It may also be realized by a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or the like, or a combination of these.

[0130] The main control unit 1313 supervises and controls the operations of the light detection system 1301, the vehicle sensor 1310, the control unit 1320, etc. It is also possible to adopt a method in which the main control unit 1313 is not provided, and the light detection system 1301, the vehicle sensor 1310, and the control unit 1320 each have their own communication interface and transmit and receive control signals via a communication network (for example, CAN standard).

[0131] The integrated circuit 1303 has a function of receiving a control signal from the main control unit 1313 or transmitting a control signal or a set value to the photoelectric conversion device 1302 by its own control unit.

[0132] The optical detection system 1301 is connected to a vehicle sensor 1310, and can detect the vehicle's driving state, such as vehicle speed, yaw rate, and steering angle, as well as the state of the environment outside the vehicle, other vehicles, and obstacles. The vehicle sensor 1310 is also a distance information acquisition means for acquiring distance information to an object. The optical detection system 1301 is also connected to a driving assistance control unit 1311 that performs various driving assistance functions, such as automatic steering, automatic cruising, and collision prevention functions. In particular, the collision determination function determines whether or not a collision with another vehicle or obstacle has occurred based on the detection results of the optical detection system 1301 and the vehicle sensor 1310. This allows for avoidance control when a collision is estimated, and activation of a safety device when a collision occurs.

[0133] The light detection system 1301 is also connected to an alarm device 1312 that issues an alarm to the driver based on the result of the determination by the collision determination unit. For example, if the collision determination unit determines that there is a high possibility of a collision, the main control unit 1313 performs vehicle control to avoid a collision and reduce damage by applying the brakes, releasing the accelerator, suppressing engine output, etc. The alarm device 1312 warns the user by sounding an alarm, displaying alarm information on a display screen such as a car navigation system or a meter panel, vibrating the seat belt or steering wheel, etc.

[0134] In this embodiment, the surroundings of the vehicle, for example, the front or rear, are photographed by the light detection system 1301. Fig. 17(b) shows an example of the arrangement of the light detection system 1301 when the light detection system 1301 photographs the area in front of the vehicle.

[0135] The two photoelectric conversion devices 1302 are disposed in front of the vehicle 1300. Specifically, if the center line of the vehicle 1300 relative to its forward / backward direction or outer shape (for example, vehicle width) is regarded as an axis of symmetry, and the two photoelectric conversion devices 1302 are disposed in line symmetry with respect to the axis of symmetry, this is preferable for obtaining distance information between the vehicle 1300 and an object to be photographed and for determining the possibility of a collision. In addition, the photoelectric conversion devices 1302 are preferably disposed so as not to obstruct the driver's field of vision when the driver visually checks the situation outside the vehicle 1300 from the driver's seat. The alarm device 1312 is preferably disposed so as to be easily within the driver's field of vision.

[0136] Next, the operation of detecting a fault in the photoelectric conversion device 1302 in the light detection system 1301 will be described with reference to Fig. 18. The operation of detecting a fault in the photoelectric conversion device 1302 is performed in accordance with steps S1410 to S1480 shown in Fig. 18.

[0137] Step S1410 is a step for performing startup settings of the photoelectric conversion device 1302. That is, settings for the operation of the photoelectric conversion device 1302 are transmitted from outside the photodetection system 1301 (e.g., the main control unit 1313) or from inside the photodetection system 1301, and the image capturing operation and fault detection operation of the photoelectric conversion device 1302 are started.

[0138] Next, in step S1420, pixel signals are obtained from the valid pixels. In addition, in step S1430, output values ​​are obtained from failure detection pixels provided for failure detection. These failure detection pixels have photoelectric conversion elements, just like the valid pixels. A predetermined voltage is written to these photoelectric conversion elements. The failure detection pixels output signals corresponding to the voltage written to these photoelectric conversion elements. Note that steps S1420 and S1430 may be reversed.

[0139] Next, in step S1440, a judgment is made as to whether the output expected value of the fault detection pixel corresponds to the output value from the actual fault detection pixel. If the result of the judgment in step S1440 indicates that the output expected value and the actual output value correspond, the process proceeds to step S1450, it is judged that the imaging operation is performed normally, and the process proceeds to step S1460. In step S1460, the pixel signal of the scanning row is sent to the memory 1305 and temporarily stored. Thereafter, the process returns to step S1420, and the fault detection operation is continued. On the other hand, if the result of the judgment in step S1440 indicates that the output expected value and the actual output value do not correspond, the process proceeds to step S1470. In step S1470, it is judged that there is an abnormality in the imaging operation, and an alarm is issued to the main control unit 1313 or the alarm device 1312. The alarm device 1312 displays the detection of the abnormality on the display unit. Thereafter, in step S1480, the photoelectric conversion device 1302 is stopped, and the operation of the photodetection system 1301 is terminated.

[0140] In this embodiment, the flowchart is looped for each line, but the flowchart may be looped for each set of lines, or the fault detection operation may be performed for each frame. The issuance of the alarm in step S1470 may be notified to the outside of the vehicle via a wireless network.

[0141] In addition, in this embodiment, the control to prevent collision with other vehicles has been described, but the control can also be applied to automatic driving control to follow other vehicles, automatic driving control not to go out of the lane, etc. Furthermore, the light detection system 1301 is not limited to vehicles such as automobiles, but can be applied to moving bodies (moving devices) such as ships, aircraft, and industrial robots. In addition, the light detection system 1301 is not limited to moving bodies, but can be applied to a wide range of devices that use object recognition, such as intelligent transport systems (ITS).

[0142] The photoelectric conversion device of the present invention may further be configured to be capable of acquiring various types of information such as distance information.

[0143] <Embodiment 8> FIG. 19(a) illustrates glasses 1600 (smart glasses) according to one application example. The glasses 1600 have a photoelectric conversion device 1602. The photoelectric conversion device 1602 is the photoelectric conversion device described in each of the above embodiments. A display device including a light-emitting device such as an OLED or LED may be provided on the back side of the lens 1601. The photoelectric conversion device 1602 may be one or more. A combination of multiple types of photoelectric conversion devices may also be used. The arrangement position of the photoelectric conversion device 1602 is not limited to that shown in FIG. 19(a).

[0144] The glasses 1600 further include a control device 1603. The control device 1603 functions as a power source that supplies power to the photoelectric conversion device 1602 and the display device. The control device 1603 also controls the operations of the photoelectric conversion device 1602 and the display device. The lens 1601 is formed with an optical system for focusing light on the photoelectric conversion device 1602.

[0145] FIG. 19(b) illustrates glasses 1610 (smart glasses) according to one application example. The glasses 1610 have a control device 1612, and the control device 1612 is equipped with a photoelectric conversion device corresponding to the photoelectric conversion device 1602 and a display device. The lens 1611 is formed with an optical system for projecting light emitted from the photoelectric conversion device in the control device 1612 and the display device, and an image is projected onto the lens 1611. The control device 1612 functions as a power source for supplying power to the photoelectric conversion device and the display device, and controls the operation of the photoelectric conversion device and the display device. The control device may have a line of sight detection unit that detects the line of sight of the wearer. Infrared light may be used for detecting the line of sight. The infrared light emission unit emits infrared light to the eyeball of a user gazing at a display image. An imaging unit having a light receiving element detects the reflected light of the emitted infrared light from the eyeball, thereby obtaining an image of the eyeball. By having a reduction means for reducing light from the infrared light emission unit to the display unit in a planar view, deterioration of image quality is reduced.

[0146] The gaze of the user with respect to the displayed image is detected from an image of the eyeball obtained by capturing infrared light. Any known method can be applied to gaze detection using the image of the eyeball. As an example, a gaze detection method based on a Purkinje image formed by reflection of irradiated light on the cornea can be used.

[0147] More specifically, the gaze detection process is performed based on the pupil-corneal reflex method. Using the pupil-corneal reflex method, a gaze vector that indicates the direction (rotation angle) of the eyeball is calculated based on the pupil image and the Purkinje image included in the captured image of the eyeball, thereby detecting the user's gaze.

[0148] The display device of this embodiment may have a photoelectric conversion device having a light receiving element, and may control the display image of the display device based on information about the user's line of sight from the photoelectric conversion device.

[0149] Specifically, the display device determines a first field of view area to which the user gazes and a second field of view area other than the first field of view area based on the line-of-sight information. The first field of view area and the second field of view area may be determined by a control device of the display device, or may be received from an external control device. In the display area of ​​the display device, the display resolution of the first field of view area may be controlled to be higher than the display resolution of the second field of view area. In other words, the resolution of the second field of view area may be lower than that of the first field of view area.

[0150] The display area may have a first display area and a second display area different from the first display area, and a high priority area may be determined from the first display area and the second display area based on line-of-sight information. The first field of view area and the second field of view area may be determined by a control device of the display device, or may be received from an external control device. The resolution of the high priority area may be controlled to be higher than the resolution of areas other than the high priority area. In other words, the resolution of an area with a relatively low priority may be lowered.

[0151] In addition, AI may be used to determine the first field of view area and the area with high priority. The AI ​​may be a model configured to estimate the angle of the line of sight and the distance to an object at the end of the line of sight from the image of the eyeball, using the image of the eyeball and the direction in which the eyeball in the image was actually looking as teacher data. The AI ​​program may be included in the display device, the photoelectric conversion device, or an external device. If included in the external device, it is transmitted to the display device via communication.

[0152] When display control is performed based on visual recognition detection, the present invention is preferably applicable to smart glasses further including a photoelectric conversion device for capturing an image of the outside world. The smart glasses can display captured outside information in real time.

[0153] <Other embodiments> Although the embodiments have been described above, the present invention is not limited to these embodiments, and various changes and modifications are possible. In addition, the embodiments are mutually applicable. [Explanation of symbols]

[0154] 201 APD 103 Signal Processing Circuit 202 Switch 215 Signal Generator P Exposure period

Claims

1. an avalanche photodiode including an anode and a cathode; a switch connected to one of the nodes of the anode and the cathode and a power supply line to which a drive voltage is applied, for switching a resistance value between the one node and the power supply line; A signal generating unit that generates a pulse signal for controlling switching of the switch; a signal processing circuit having a counter circuit connected to the avalanche photodiode; A first exposure period and a second exposure period longer than the first exposure period can be set as an exposure period which is a period from the start of counting to the end of counting in the counter circuit, The photoelectric conversion device is characterized in that the signal generating unit generates the pulse signals so that a value obtained by dividing the number of the pulse signals in the second exposure period by the second exposure period is smaller than a value obtained by dividing the number of the pulse signals in the first exposure period by the first exposure period.

2. 2. The photoelectric conversion device according to claim 1, wherein the pulse signal is a signal having a repetitive period.

3. 3. The photoelectric conversion device according to claim 1, wherein the number of the pulse signals in the first exposure period is the same as the number of the pulse signals in the second exposure period.

4. A photoelectric conversion device described in any one of claims 1 to 3, characterized in that the period of the pulse signal during the first exposure period is shorter than the period of the pulse signal during the second exposure period.

5. a first pulse width of the first level of the pulse signal in the first exposure period is equal to a first pulse width of the first level of the pulse signal in the second exposure period; 5. The photoelectric conversion device according to claim 1, wherein the first level of the pulse signal is a level at which the switch is turned on.

6. a second pulse width of the second level of the pulse signal in the first exposure period is different from a second pulse width of the second level of the pulse signal in the second exposure period; 6. The photoelectric conversion device according to claim 5, wherein the second level of the pulse signal is a level at which the switch is turned off.

7. 7. The photoelectric conversion device according to claim 1, wherein the switch functions as a quenching element.

8. the switch is a MOS transistor; one node of the switch is connected to the cathode of the avalanche photodiode; the other node of the switch is connected to the power supply line; 8. The photoelectric conversion device according to claim 1, wherein the pulse signal is supplied to a gate electrode of the switch.

9. The signal processing circuit has a waveform shaping unit, the waveform shaping unit is connected to one of the anode and the cathode; 9. The photoelectric conversion device according to claim 1, wherein a signal output from the waveform shaping section is input to the counter circuit.

10. 10. The photoelectric conversion device according to claim 1, wherein the number of the pulse signals in the first exposure period is equal to an upper limit value of the counter circuit.

11. A correction circuit is provided.

11. The photoelectric conversion device according to claim 1, wherein a count value output from the counter circuit is input to the correction circuit.

12. The correction circuit calculates the count value, the frequency of the pulse signal in the first exposure period, and the length of the first exposure period as explanatory variables, and calculates the number of incident photons as a response variable, The photoelectric conversion device according to claim 11 , wherein the explanatory variables and the response variable are described by a relational expression based on a natural logarithm.

13. The correction circuit, when the count value is input, assumes that Nct, the number of incident photons is Nph, the frequency of the pulse signal in the first exposure period is f, and the first exposure period is T. Nct=f×T×(1-exp(-Nph / (f×T))) 13. The photoelectric conversion device according to claim 11, wherein the number of incident photons Nph satisfies the relationship:

14. The correction circuit detects that the pulse signal has n kinds of frequencies, The first frequency of the pulse signal is f １ , the second frequency is f ２ , the n-1th frequency is f ｎ－１ , the nth frequency is f ｎ (n is a natural number equal to or greater than 2), The frequency of the pulse signal is f １ The period T １ , the frequency of the pulse signal is f ２ The period T ２ , the frequency of the pulse signal is f ｎ－１ The period T ｎ－１ , the frequency of the pulse signal is f ｎ The period T ｎ With respect to the input of the count value, Nct=f １ ×T １ ×(1-exp(-Nph １ / (f １ ×T １ )))+f ２ ×T ２ ×(1-exp(-Nph ２ / (f ２ ×T ２ )))+・・・+f ｎ－１ ×T ｎ－１ ×(1-exp(-Nph ｎ―１ / (f ｎ－１ ×T ｎ－１ )))+f ｎ ×T ｎ ×(1-exp(-Nph ｎ / (f ｎ ×T ｎ ))) 12. The photoelectric conversion device according to claim 11, wherein the number of incident photons Nph satisfies the following relationship:

15. 15. The photoelectric conversion device according to claim 1, wherein the pulse signal is not input to the switch during a period other than the exposure period.

16. 16. The photoelectric conversion device according to claim 1, wherein a plurality of the avalanche photodiodes are arranged in a two-dimensional array in a plan view.

17. The photoelectric conversion device according to claim 1 , and a signal processing unit that processes a signal output from the photoelectric conversion device.

18. The photoelectric conversion device according to claim 1 , a signal processing unit that processes a signal output from the photoelectric conversion device; a shutter for controlling incidence of light into the photoelectric conversion device; The optical detection system, wherein the first exposure period and the second exposure period are periods during which the shutter is open.

19. The photoelectric conversion device according to claim 1 , a distance information acquisition means for acquiring distance information to an object from distance measurement information based on a signal from the photoelectric conversion device, A moving body further comprising a control means for controlling the moving body based on the distance information.

Images