Photoelectric conversion device, photoelectric conversion system, and mobile object

Abstract

This photoelectric conversion device comprises: an avalanche photodiode; a recharge circuit to which a first control signal is input; a first gating circuit to which a second control signal is input; and a second gating circuit to which a third control signal is input. When the first gating circuit is in an off state, the first gating circuit outputs a signal based on an output signal of the avalanche photodiode. When the second gating circuit is in an off state, the second gating circuit outputs a signal based on the output signal of the avalanche photodiode. A period during which the recharge circuit is brought into an on state by the first control signal includes a time point when the first gating circuit changes from an off state to an on state and a time point when the second gating circuit changes from an on state to an off state.

Description

Photoelectric conversion device, photoelectric conversion system, and moving object

[0001] The present disclosure relates to a photoelectric conversion device, a photoelectric conversion system, and a moving object.

[0002] Patent Document 1 discloses a photoelectric conversion device in which an output signal from an avalanche photodiode (APD) is input to two circuit units, and detection signals output from the respective circuit units are input to respective counting units. The photoelectric conversion device of Patent Document 1 is disclosed to gate the detection signals output from the respective circuit units and input the gated signals to the respective counting units at different timings.

[0003] Japanese Patent Laid-Open No. 2024-160505

[0004] In Patent Document 1, when one photon is detected by the APD, the detection signal is input to each counting unit at different timings, but there are cases where it is desired to input and hold signals of different photons detected by one APD in the respective counting units. Patent Document 1 does not discuss the case of detecting and holding different photons incident on the APD.

[0005] One aspect of the photoelectric conversion device of the present disclosure includes an avalanche photodiode, a recharge circuit to which a periodic first control signal is input and which controls the recharge operation of the avalanche photodiode, and a first gating circuit and a second gating circuit connected to the avalanche photodiode, wherein the first gating circuit is input to control whether or not to output a signal based on the output signal of the avalanche photodiode, and the second gating circuit is input to control whether or not to output a signal based on the output signal of the avalanche photodiode, and when the first gating circuit is in the off state, the first gating circuit outputs a signal based on the output signal of the avalanche photodiode, and when the second gating circuit is in the off state, the second gating circuit outputs a signal based on the output signal of the avalanche photodiode, and the period during which the recharge circuit is in the ON state by the first control signal includes the time when the first gating circuit changes from the OFF state to the ON state and the time when the second gating circuit changes from the ON state to the OFF state.

[0006] According to this disclosure, it is possible to provide a photoelectric converter capable of counting and holding different photons incident on an APD in each counting unit, and a photoelectric converter system and mobile body using such a photoelectric converter.

[0007] This is a schematic diagram of a photoelectric converter according to an embodiment. This is a schematic diagram of a pixel substrate of a photoelectric converter according to an embodiment. This is a schematic diagram of a circuit board of a photoelectric converter according to an embodiment. This is an example of the configuration of a photoelectric converter according to an embodiment. This is a circuit diagram of a pixel of a photoelectric converter according to the first embodiment. This is a timing diagram showing the operation of a pixel of a photoelectric converter according to the first embodiment. This is a timing diagram showing the operation of a pixel of a photoelectric converter according to the second embodiment. This is a timing diagram showing the operation of a pixel of a photoelectric converter according to the third embodiment. This is a timing diagram showing the operation of a pixel of a photoelectric converter according to the fourth embodiment. This is a circuit diagram of a pixel of a photoelectric converter according to the fifth embodiment. This is a timing diagram showing the operation of a pixel of a photoelectric converter according to the fifth embodiment. This is a timing diagram showing the operation of a pixel of a photoelectric converter according to the sixth embodiment. This is a diagram showing an example of the operation of a pixel of a photoelectric converter according to the seventh embodiment. This is a functional block diagram of a photoelectric conversion system according to the eighth embodiment. This is a functional block diagram of a photoelectric conversion system according to the ninth embodiment. This is a functional block diagram of a photoelectric conversion system according to the tenth embodiment. This is a functional block diagram of a photoelectric conversion system according to the eleventh embodiment. This is a functional block diagram of a photoelectric conversion system according to the eleventh embodiment. This is a functional block diagram of a photoelectric conversion system according to the twelfth embodiment. This is a functional block diagram of the photoelectric conversion system according to the twelfth embodiment. This is a functional block diagram of the photoelectric conversion system according to the thirteenth embodiment. This is a functional block diagram of the photoelectric conversion system according to the fourteenth embodiment. This is a functional block diagram of the photoelectric conversion system according to the fourteenth embodiment.

[0008] The following embodiments are intended to embody the technical concept of this disclosure and do not limit it. The size and positional relationships of the components shown in each drawing may be exaggerated for clarity. In the following description, identical components may be given the same number and their description may be omitted.

[0009] Embodiments of this disclosure will be described in detail below with reference to the drawings. In the following description, terms indicating specific directions or positions (e.g., "up," "down," "right," "left," and other terms including these terms) will be used as needed. The use of these terms is for the purpose of facilitating the understanding of embodiments with reference to the drawings, and the technical scope is not limited by the meaning of these terms.

[0010] In this specification, a plan view refers to viewing the semiconductor layer on which the avalanche photodiode (avalanche photodiode, APD) is arranged from a direction perpendicular to the light incident surface. A cross-section refers to the surface of the semiconductor layer perpendicular to the light incident surface. If the light incident surface of the semiconductor layer is rough when viewed microscopically, the plan view is defined based on the light incident surface of the semiconductor layer when viewed macroscopically.

[0011] In the following explanation, the anode of the APD is assumed to be at a fixed potential, and the signal is taken from the cathode side. Also, in the following explanation, the signal charge is an electron. Alternatively, the cathode of the APD may be assumed to be at a fixed potential, and the signal may be taken from the anode side. In this case, the signal charge will be a hole. The following explanation describes the case where one node of the APD is at a fixed potential, but the potentials of both nodes may fluctuate.

[0012] Furthermore, in the following embodiments, connections between circuit elements may be described. In this case, even if another element is interposed between the elements of interest, unless otherwise specified, the elements of interest will be treated as connected. For example, suppose element A is connected to one node of a capacitive element C having multiple nodes, and element B is connected to the other node. Even in such a case, elements A and B will be treated as connected unless otherwise specified.

[0013] In the following, elements and circuits with similar functions are given the same designation, and may be distinguished by adding different alphabetical subscripts at the end. For example, they may be described as gating circuits 218a and 218b. When it is not necessary to explain them separately, the subscripts such as a, b, etc., are omitted and the common parts are explained.

[0014] In the embodiments described below, the imaging device will be described as an example of a photoelectric conversion device. However, each embodiment is not limited to the imaging device and can be applied to other examples of photoelectric conversion devices. Other examples of photoelectric conversion devices include distance measuring devices (devices for measuring distance using focus detection or TOF (Time Of Flight)), photometric devices (devices for measuring the amount of incident light, etc.).

[0015] The common configuration of the photoelectric converter 100 in each embodiment will be explained using Figures 1 to 4. The photoelectric converter has SPAD pixels including an APD.

[0016] Figure 1 is a schematic diagram of a photoelectric converter 100, showing the configuration of a stacked type photoelectric converter 100. The photoelectric converter 100 includes a sensor substrate (first substrate) and a circuit board (second substrate) stacked on top of each other, and the sensor substrate and the circuit board are electrically connected to each other. However, the photoelectric converter is not limited to this. For example, it may be a non-stacked type photoelectric converter in which the components included in the sensor substrate and the components included in the circuit board are arranged on a common semiconductor layer, as described below.

[0017] The photoelectric conversion device in this embodiment is a back-illuminated type photoelectric conversion device in which light is incident from the first surface of the sensor substrate and a circuit board is arranged on the second surface of the sensor substrate. The sensor substrate has a first semiconductor layer 11 having a photoelectric conversion element described later and a first wiring structure. The circuit board has a second semiconductor layer 21 having a circuit such as a signal processing unit described later and a second wiring structure. The second semiconductor layer 21, the second wiring structure, the first wiring structure, and the first semiconductor layer 11 are stacked in that order to constitute the photoelectric conversion device 100.

[0018] In the following description, the sensor substrate and the circuit board may be, but are not limited to, diced chips. For example, each substrate may be a wafer. Furthermore, each substrate may be stacked in wafer form and then diced, or it may be made into chips and then stacked and bonded together. The sensor substrate has a pixel region 12, and the circuit board has a circuit region 22 that processes the signals detected by the pixel region 12.

[0019] Figure 2 shows an example of the arrangement of the first semiconductor layer 11 of the sensor substrate. Multiple pixels 101 each contain an APD 201 and are arranged in a two-dimensional array in a planar view, forming a pixel region 12.

[0020] Pixel 101 is typically a pixel for forming an image, but when used in Time of Flight (TOF), it is not necessarily required to form an image. That is, pixel 101 may be a pixel for measuring the time and amount of light that arrives.

[0021] Figure 3 shows an example of the arrangement of the second semiconductor layer 21 on the circuit board. The circuit board has a circuit region 22 on which the signal processing circuit 103 is located, a vertical scanning circuit 110, a readout circuit 112, a horizontal scanning circuit 111, a control pulse generation circuit 115, an adder 118, an output circuit 114, a scan line 116, and an output line 113. In a plan view, the circuit region 22 in Figure 3 is located in the region that overlaps with the pixel region 12 in Figure 2. Furthermore, in a plan view, the vertical scanning circuit 110, readout circuit 112, horizontal scanning circuit 111, control pulse generation circuit 115, adder 118, and output circuit 114 are located so as to overlap with the region between the edge of the sensor board in Figure 2 and the edge of the pixel region 12. In other words, the sensor board has a pixel region 12 and a non-pixel region located around the pixel region 12. Then, in the area that overlaps with the non-pixel area in a plan view, the vertical scanning circuit 110, readout circuit 112, horizontal scanning circuit 111, control pulse generation circuit 115, adder 118, and output circuit 114 are arranged.

[0022] The signal processing circuit 103 is electrically connected to the pixels 101 via connecting wires provided for each pixel 101, and is arranged in a two-dimensional array in a planar view, similar to the pixels 101. The signal processing circuit 103 includes a counting unit (memory) that holds a count value based on photons incident on the pixels 101.

[0023] The vertical scanning circuit 110 receives control pulses supplied from the control pulse generation circuit 115 and supplies control pulses to the signal processing circuit 103 corresponding to the pixels 101 of each row via the scan lines 116. The vertical scanning circuit 110 may be composed of logic circuits such as a shift register and an address decoder. The scan lines 116 may be composed of multiple drive lines.

[0024] The control pulse generation circuit 115 has a signal generation unit that generates the first control signal VR, the second control signal VG, and the third control signal VG2, which will be described later. The signal generation unit generates pulse signals that control the recharge circuit 202, the gating circuit 218a (first gating circuit), and the gating circuit 218b (second gating circuit), as will be described later. For example, the signal generation unit may generate the control signals in common for multiple pixels in the pixel region, or it may generate the control signals for each pixel. The control pulse generation circuit 115 preferably has, for example, a frequency divider circuit. This makes it possible to control simply and reduces the increase in the number of elements.

[0025] The readout circuit 112 acquires the signal output from the memory of the signal processing circuit 103 for each row via the output line 113. The signals output from the memory are accumulated by the adder 118, and the output signal is output to an external signal processing circuit (signal processing device) of the photoelectric converter 100 via the output circuit 114. The readout circuit 112 may also have the function of a signal processing circuit that performs correction of the count value, etc. Furthermore, although the example below describes an example in which the adder 118 is located outside the circuit area 22, a circuit corresponding to the adder 118 may be located inside the pixel 101. In addition, the adder 118 may be located after the output circuit 114 and in a signal processing device located outside the photoelectric converter 100.

[0026] The horizontal scanning circuit 111 receives control pulses from the control pulse generation circuit 115 and sequentially outputs the signal values ​​of each column in the readout circuit 112 to the output circuit 114.

[0027] Furthermore, the arrangement of photoelectric conversion elements in the pixel region 12 may be arranged in a one-dimensional manner. Also, the effects of this disclosure can be achieved even in a configuration with only one pixel 101, and a configuration with one pixel 101 can also be included in this disclosure. The signal processing circuit 103 does not necessarily need to be provided for each APD 201; for example, one signal processing circuit 103 may be shared by multiple APDs 201, and signal processing may be performed sequentially.

[0028] Figure 4 shows an example configuration illustrating the relationship between the signal supplied from the vertical scanning circuit 110 and the circuit region 22 and the pixel region 12. In the pixel region 12, a color filter may be placed at each pixel. The color filters can be arranged, for example, in a Bayer array. In Figure 3, the vertical scanning circuit 110 is arranged along one side of the pixel region 12, but as shown in Figure 4, the vertical scanning circuit 110 may be arranged on the right and left sides of the pixel region 12 in a plan view.

[0029] Each pixel is supplied with a first control signal VR, a second control signal VG, and a third control signal VG2 from the vertical scanning circuit 110, as described later. Furthermore, the control lines for each control signal are connected so that a common control signal VR, second control signal VG, and third control signal VG2 are supplied to multiple pixels arranged in a single row. In the vertical scanning circuit 110, each control signal is transmitted to each control line using binary partitioning.

[0030] [First Embodiment] The photoelectric conversion device of the first embodiment will be described with reference to Figures 5 to 10.

[0031] Figure 5 is a circuit diagram of a pixel in this embodiment, and Figure 6 is a timing diagram showing the operation of a pixel in this embodiment.

[0032] The pixel in this embodiment includes an APD 201, a recharge circuit 202, a waveform shaping circuit 210, a gating circuit 218, a counting unit (memory) 211, and a selection circuit 212. In this embodiment, the APD 201 is located in the pixel area 12, and the other circuits are located in the circuit area 22.

[0033] The APD201 is a photodiode that generates charge pairs corresponding to incident light through photoelectric conversion. A voltage VL (first voltage) is supplied to the anode of the APD201. A voltage VH (second voltage), which is higher than the voltage VL supplied to the anode, is supplied to the cathode of the APD201, and a reverse bias voltage is applied to the APD201 that causes avalanche multiplication. When such voltages are supplied, the charge generated by the incident light on the APD201 undergoes avalanche multiplication, and an avalanche current is generated.

[0034] When a reverse bias voltage is supplied to the APD201, it can operate in Geiger mode or linear mode. In Geiger mode, the APD201 is operated by applying a voltage greater than the breakdown voltage to the anode and cathode. In linear mode, the APD201 is operated with a potential difference between the anode and cathode that is near or below the breakdown voltage. An APD operating in Geiger mode is called a SPAD. For example, the SPAD functions with a voltage VL of -30V and a voltage VH of 1V.

[0035] The recharge circuit 202 is connected between the power supply that provides voltage VH and the cathode terminal of the APD 201. The recharge circuit 202 is controlled by a periodically input clock signal (first control signal) VR, which drives the APD 201 to clock recharge. One period refers to, for example, the time from when the first control signal VR transitions from high level to low level to when the first control signal VR transitions from low level to high level and then to low level. The first control signal VR is input to the recharge circuit 202 via a drive line 215. The drive line 215 is one of the scan lines 116 in Figure 3. In this embodiment, the first control signal VR is supplied from the control pulse generation circuit 115, but this is not the only option. For example, in Figures 3 and 4, the control pulse generation circuit 115 may supply the control signal that forms the basis of the first control signal VR, and the inverted first control signal VR via an inverter may be input to the recharge circuit 202.

[0036] The recharge circuit 202 controls the recharge operation of the APD 201. The first control signal VR controls the APD 201 to either a state in which avalanche multiplication is possible (second state) or a recharge state in which avalanche multiplication is controlled (first state). In the first state, the recharge circuit 202 is ON, and the cathode of the APD 201 is connected to voltage VH. In the second state, the recharge circuit 202 is OFF, so the cathode of the APD 201 and voltage VH are not connected. In other words, the resistance between the cathode of the APD 201 and voltage VH is higher compared to the charged state. When a photon is incident on the APD 201 in the second state, avalanche multiplication occurs, and the voltage at the cathode of the APD 201 drops. Subsequently, when the clock signal VR is input to the recharge circuit 202, the recharge circuit 202 turns on, the voltage is charged, and the cathode voltage of the APD 201 returns to its initial state.

[0037] The recharge circuit 202 can be constructed using a MOS transistor, as shown in Figure 5. The following explanation will use a PMOS transistor as an example, but the recharge circuit 202 may also be an NMOS transistor. When the recharge circuit 202 is a PMOS transistor, it is ON when the signal level input to the gate is low (hereinafter referred to as L level), and OFF when the signal level input to the gate is high (hereinafter referred to as H level). When the recharge circuit 202 is an NMOS transistor, it is ON when the signal level input to the gate is H level, and OFF when the signal level input to the gate is L level.

[0038] The output signal from the APD 201 is input to the subsequent waveform shaping circuit 210. The output node VC of the APD 201 is the input node of the waveform shaping circuit 210. The waveform shaping circuit 210 shapes the potential change of the cathode of the APD 201 during photon detection and outputs a pulse signal to node VO. For example, an inverter circuit can be used as the waveform shaping circuit 210. Figure 5 shows an example in which one inverter circuit is used as the waveform shaping circuit 210, but a circuit in which multiple inverters are connected in series may be used, or other circuits that have a waveform shaping effect may be used.

[0039] A gating circuit 218a (first gating circuit) and a gating circuit 218b (second gating circuit) are connected downstream of the waveform shaping circuit 210. The gating circuit 218a controls whether or not to output an output signal indicating the detection of a photon in the APD 201, based on the signal from the waveform shaping circuit 210 and a second control signal VG supplied via the drive line 216. The gating circuit 218b controls whether or not to output an output signal indicating the detection of a photon in the APD 201, based on the signal from the waveform shaping circuit 210 and a third control signal VG2 supplied via the drive line 217.

[0040] The gating circuit 218 includes, for example, an AND gate. One input node of the AND gate is connected to the output node VO from the waveform shaping circuit 210, and the other input node of the AND gate is connected to the control line of the second control signal VG or the third control signal VG2. The gating circuit 218 performs a logical AND operation on the input signal. The gating circuit 218a outputs a signal based on the signal from APD 201 and the result of the operation based on the second control signal VG to the next stage. The gating circuit 218b also outputs a signal based on the signal from APD 201 and the result of the operation based on the third control signal VG2 to the next stage. When the gating circuit 218 is in the off state, the gating circuit 218 outputs a signal based on the signal from APD 201.

[0041] In this embodiment, the gating circuit 218a is in an off state when node VO is at a high level and the second control signal VG is at a high level, and is in an on state when node VO and at least one of the second control signal VG are at a low level. In other words, the period when the second control signal VG is at a low level is a period during which the gating circuit is blocked so that even if a photon is incident on the APD 201, it is not transmitted to the memory 211a. When the gating circuit 218a is in an off state after a photon is incident on the APD 201, the output signal P1 becomes high level and its value is held in the memory 211a. Also, the gating circuit 218b is in an off state when node VO is at a high level and the third control signal VG2 is at a high level, and is in an on state when node VO and at least one of the third control signal VG2 are at a low level. When the gating circuit 218b is in an off state after a photon is incident on the APD 201, the output signal P2 becomes high level and its value is held in the memory 211b.

[0042] The second control signal VG and the third control signal VG2 may be different, but it is preferable that they have the same period. The period of the second control signal VG refers to the period from the time of transition from L level to H level to the time of transition from H level to L level and then to H level again. The same applies to the period of the third control signal VG2. It is preferable that the first control signal VR, the second control signal VG, and the third control signal VG2 have the same period. This makes it possible to keep the period from the recharge of the APD by the first control signal VR to the retention of the value in memory 211a by the second control signal VG, and the period from the retention of the value in memory 211a to the retention of the value in memory 211b by the third control signal VG2 constant. Therefore, detection signals of different photons can be stored in each memory 211 at the correct timing.

[0043] The memory 211 holds and outputs a value based on a detection signal indicating detection of incidence of a photon on the APD 201. Specifically, the memory 211a (first memory) holds the H level at the timing when the output signal P1 transmitted from the gating circuit 218a transitions from the L level to the H level. The memory 211b (second memory) holds the H level at the timing when the output signal P2 from the gating circuit 218b transitions from the L level to the H level. After a lapse of a predetermined period (for example, one frame period), the memory 211a outputs an output signal O1 to the output line 113 via the selection circuit 212, and the memory 211b outputs an output signal O2 via the selection circuit 212. Then, after a lapse of a predetermined period (for example, one frame period), a reset signal pRES for resetting the memory 211 is input via the drive line 213. A selection signal pSEL for selecting the memory 211 is input to the selection circuit 212 via the drive line 214. The memory 211 only needs to be capable of holding at least 1-bit data, and may be capable of holding 2 bits or more of data. The memory 211 may also output a detection signal indicating detection of incidence of a photon on the APD 201 as a count value. Specifically, the memory 211a may count and hold the number of times the output signal P1 transmitted from the gating circuit 218a transitions from the L level to the H level as a count value for a predetermined period (for example, one frame period). The memory 211b may count and hold the number of times the output signal P2 transmitted from the gating circuit 218b transitions from the L level to the H level as a count value for a predetermined period (for example, one frame period). The drive line 213 and the drive line 214 are one type of the scanning lines 116 in FIG. 3.

[0044] In the present embodiment, the case where there are two memories 211 and two gating circuits 218 is described as an example, but three or more memories 211 and three or more gating circuits 218 may be included.

[0045] Figure 6 is a timing diagram showing the operation of pixels in the photoelectric converter of this embodiment. In this embodiment, the period from the time when the reset signal pRES changes from a high level to a low level to the time when the reset signal pRES changes from a high level to a low level again corresponds to one frame period. The reset timing of the memory 211 may be common to all pixels, or it may be performed sequentially for each pixel row.

[0046] One frame period, when the photoelectric converter is an imaging device, can be the period it takes to capture one image. Alternatively, one frame period can be the period during which a vertical scanning circuit scans from the first row of pixels to the last row of pixels in a pixel area. In other words, one frame period is the period from when the pulse signal, which is the vertical synchronization signal, reaches a high level once until it reaches a high level again. Here, it is not necessary to scan all the pixels in all rows from the first row to the last row during one frame period. For example, if some rows are skipped during scanning, one frame period is defined as the period from scanning in one direction from a certain row until the last row is scanned. For example, consider a case where the pixel area has pixels from the first row to the nth row, and scanning starts from the second row and goes up to the (n-1)th row. In this case, the period during which scanning is done from the second row to the (n-1)th row is one frame period. The next period during which scanning is done from the second row to the (n-1)th row is the next one frame period. Furthermore, if a row is scanned while omitting some rows, and then the omitted rows are scanned again, the period until the omitted rows have been scanned may be considered as one frame.

[0047] The first control signal VR is periodically input to the recharge circuit 202. In this embodiment, the period during which the first control signal VR is at an L level is the period during which the APD 201 is in a recharge state.

[0048] In the present embodiment, from the time point when the first control signal VR transitions from the H level to the L level to the next time point when the first control signal VR transitions from the L level to the H level, the second control signal VG includes a period in which it goes from the L level to the H level and then to the L level. That is, the period during which the first control signal VR is in the ON state overlaps with the period during which the first gating circuit 218a, which is the period during which the second control signal VG is at the H level, is in the OFF state. The period in which the first control signal VR is at the L level and the second control signal VG is at the H level is a period during which the gating circuit 218a can perform exposure.

[0049] As described above, in the present embodiment, the period during which the second control signal VG is at the H level is included within the period during which the first control signal VR is at the L level. Thereby, the period during which the second control signal VG is at the H level can be set to a passive recharge state, and a state is achieved in which photons are incident, detected, and then recharged.

[0050] Between the time point when the first control signal VR becomes the L level and the next time point when the first control signal VR becomes the H level, the third control signal VG2 includes a time point at which it transitions from the L level to the H level. That is, the period during which the first control signal VR is at the L level and the time point at which the third control signal VG2 is at the H level partially overlap. In FIG. 6, the period during which the first control signal VR is at the L level and the period during which the third control signal VG2 is at the H level partially overlap. At this time, the period during which the second control signal VG is at the H level and the period during which the third control signal VG2 is at the H level are offset from each other. That is, after the second control signal VG transitions from the H level to the L level, the third control signal VG2 transitions from the L level to the H level. This makes it possible to adjust so that signals based on different photons are held in the memory 211a and the memory 211b, respectively. The period in which the first control signal VR is at the L level and the third control signal VG2 is at the H level is a period during which the gating circuit 218b can perform exposure. Further, after the first control signal VR transitions from the L level to the H level, the period until a photon first enters the APD is also included in the period during which the gating circuit 218b can perform exposure.

[0051] It is preferable to bring the end timing of the period during which the gating circuit 218a can expose and the start timing of the period during which the gating circuit 218b can expose as close together as possible. For example, it is preferable that the difference between the time when the second control signal VG transitions from H level to L level and the time when the third control signal VG2 transitions from L level to H level is within the range of the period during which the second control signal VG is at an H level or the period during which the third control signal VG2 is at an H level. In this way, by bringing the time when the gating circuit 218a turns on and the time when the gating circuit 218b turns off as close together as possible, the occurrence of periods during which the signal cannot be detected can be reduced.

[0052] When a photon is incident during the period when the first control signal VR is at an L level, the output node VO from the waveform shaping circuit 210 changes from an L level to an H level. When node VO is at an H level and the second control signal VG is at an H level, the output signal P1 from the gating circuit 218a transitions from an L level to an H level. Also, when node VO is at an H level and the third control signal VG2 is at an H level, the output signal P2 from the gating circuit 218b transitions from an L level to an H level.

[0053] At time t1, the reset signal pRES input to memories 211a and 211b changes from high level to low level, and one frame period begins. At time t1, the first control signal VR changes from high level to low level, and the recharge circuit 202 enters the recharge state. Note that the point at which the first control signal VR changes from high level to low level may be before the point at which the reset signal pRES transitions from high level to low level, as long as it is within the period when the reset signal pRES is at high level. Also, the point at which the first control signal VR changes from high level to low level may be after the point at which the reset signal pRES transitions from high level to low level.

[0054] At time t2, the second control signal VG changes from L level to H level.

[0055] At time t3, when a photon is incident on APD201, node VO transitions from L level to H level. At this time, the second control signal VG is at H level, so at time t3, the output signal P1 from gating circuit 218a transitions from L level to H level. Then, the output signal O1 from memory 211a becomes H level.

[0056] At time t4, node VO transitions from high level to low level. This is because the first control signal VR is at the low level, and the recharge circuit 202 is in a passive recharge state. Also at time t4, the output signal P1 from the gating circuit 218a transitions from high level to low level. The output signal O1 of memory 211a remains at the high level.

[0057] At time t5, the second control signal VG transitions from high level to low level.

[0058] At time t6, the third control signal VG2 transitions from L level to H level. The time when the second control signal VG transitions from H level to L level and the time when the third control signal VG2 transitions from L level to H level are brought close together. For example, in this embodiment, the period during which the first control signal VR is at the L level includes the time when the second control signal VG transitions to the L level and the time when the third control signal VG2 transitions to the H level. By bringing the time when the second control signal VG transitions to the L level and the time when the third control signal VG2 transitions to the H level closer together in this way, it is possible to reduce the occurrence of periods during which the signal cannot be detected, as will be described later. It is more preferable that the difference between the time when the second control signal VG transitions from H level to L level and the time when the third control signal VG2 transitions from L level to H level is within the period during which the second control signal VG is at the H level and within the period during which the third control signal VG2 is at the H level.

[0059] At time t7, the first control signal VR transitions from L level to H level, and the recharge circuit 202 turns off.

[0060] At time t8, when a photon is incident on APD201, node VO transitions from L level to H level. This is the timing when the first photon is incident on APD201 after the first control signal VR has transitioned from L level to H level, and the third control signal VG2 is at H level, so at time t8, the output signal P2 from gating circuit 218b transitions from L level to H level. Then, the output signal O2 from memory 211b becomes H level.

[0061] At time t9, the third control signal VG2 transitions from high level to low level, and the output signal P2 from the gating circuit 218b transitions from high level to low level. The output signal O2 from the memory 211b remains at high level.

[0062] At time t10, the first control signal VR transitions from high level to low level, and the recharge circuit 202 enters the recharge state. Consequently, node VO transitions from high level to low level.

[0063] Between time t10 and time t11, the count values ​​held in memory 211a and 211b are output to the output lines, respectively.

[0064] At time t11, the reset signal pRES transitions from L level to H level, and memories 211a and 211b are reset. As a result, output signals O1 and O2 transition from H level to L level.

[0065] At time t12, the reset signal pRES transitions from high level to low level. In Figure 6, the period from time t1 to time t12 constitutes one frame period.

[0066] According to this embodiment, the period during which the signal based on the first control signal VR is effective includes the period during which the second control signal VG is effective, and further includes the start timing for when the third control signal VG2 becomes effective. This allows the APD 201 to be recharged immediately after the signal is stored in the memory 211a. Since the memory 211b can detect the next photon incident on the APD 201, it is possible to reduce the occurrence of periods during which the signal from the next photon incident on the APD 201 cannot be detected.

[0067] [Second Embodiment] The photoelectric conversion device of the second embodiment will be described with reference to Figure 7. Figure 7 is a timing diagram showing the operation of pixels in the photoelectric conversion device of this embodiment.

[0068] The photoelectric conversion device in this embodiment differs from the first embodiment in that the transition of the second control signal VG from H level to L level and the transition of the third control signal VG2 from L level to H level occur simultaneously. Aside from this point and the points described below, the embodiment is substantially the same as the first embodiment, and therefore, further explanation is omitted.

[0069] The operation from time t1 to time t4 is the same as in the first embodiment.

[0070] At time t5, the second control signal VG transitions from H level to L level, and the third control signal VG2 transitions from L level to H level.

[0071] The operation after time t7 is the same as in the first embodiment.

[0072] In this embodiment, the gating circuit 218b is turned off at the same time that the gating circuit 218a is turned on. This allows the period during which memory 211b can detect photons to begin simultaneously with the end of the period during which memory 211a can detect photons. Therefore, the period during which a signal based on the next photon incident on the APD 201 cannot be detected can be made shorter than in the first embodiment, and the occurrence of periods during which a signal cannot be detected can be further reduced.

[0073] [Third Embodiment] The photoelectric conversion device of the third embodiment will be described with reference to Figure 8. Figure 8 is a timing diagram showing the operation of pixels in the photoelectric conversion device of this embodiment.

[0074] The photoelectric conversion device in this embodiment differs from the first embodiment in that the timing of the transition of the first control signal VR from L level to H level, the timing of the transition of the second control signal VG from H level to L level, and the timing of the transition of the third control signal VG2 from L level to H level are the same. Aside from this point and the points described below, the embodiment is substantially the same as the first embodiment, and therefore, further explanation is omitted.

[0075] The operation from time t1 to time t4 is the same as in the first embodiment.

[0076] At time t5, the first control signal VR transitions from L level to H level, the second control signal VG transitions from H level to L level, and the third control signal VG2 transitions from L level to H level.

[0077] The operation after time t8 is the same as in the first embodiment.

[0078] According to this embodiment, the period during which no signal can be detected can be made shorter than in the first embodiment, and the occurrence of periods during which no signal can be detected can be further reduced. Furthermore, if the period during which the first control signal VR is at the H level and the period during which the third control signal VG2 is at the H level overlap for a long time, multiple avalanche multiplications may occur during the exposure period of the memory 211b. In this case, since the second and subsequent avalanche multiplications do not contribute to the count, unnecessary power consumption may occur. In this embodiment, the transition point of the first control signal VR from the L level to the H level and the transition point of the third control signal VG2 from the L level to the H level are simultaneous, so it is possible to reduce the occurrence of unnecessary power consumption compared to the second embodiment.

[0079] [Fourth Embodiment] The photoelectric conversion device of the fourth embodiment will be described with reference to Figure 9. Figure 9 is a timing diagram showing the operation of pixels in the photoelectric conversion device of this embodiment.

[0080] The photoelectric conversion device in this embodiment differs from the first embodiment in that the period during which the first control signal VR is at an L level includes the periods during which the second control signal VG is at an H level and the period during which the third control signal VG2 is at an H level. Aside from this point and the points described below, it is substantially the same as the first embodiment, and therefore the explanation is omitted.

[0081] At time t1, the first control signal VR transitions from H level to L level, and at time t9, the first control signal VR transitions from L level to H level. During the period from time t1 to time t9, the second control signal VG transitions from L level to H level and then from H level to L level. Also during the period from time t1 to time t9, the third control signal VG2 transitions from L level to H level and then from H level to L level.

[0082] Furthermore, the transition of the second control signal VG from H level to L level occurs simultaneously with the transition of the third control signal VG2 from L level to H level.

[0083] According to this embodiment, the period during which no signal can be detected can be made shorter than in the first embodiment, and the occurrence of periods during which no signal can be detected can be further reduced.

[0084] In Figure 9, the second control signal VG transitions to the L level and the third control signal VG2 transitions to the H level at time t5. However, similar to the first embodiment, the third control signal VG2 may transition from the L level to the H level after the second control signal VG transitions to the L level. Even in this case, it is possible to reduce the occurrence of periods during which the signal cannot be detected.

[0085] [Fifth Embodiment] The photoelectric converter of the fifth embodiment will be described with reference to Figures 10 and 11. Figure 10 is a circuit diagram of a pixel in the photoelectric converter of this embodiment, and Figure 11 is a timing diagram showing the operation of a pixel in the photoelectric converter of this embodiment.

[0086] The photoelectric conversion device in this embodiment differs from the third embodiment in that a mask circuit 219 is connected between the waveform shaping circuit 210 and the gating circuit 218b. Aside from this point and the points described below, it is substantially the same as the third embodiment, and therefore the description is omitted.

[0087] The mask circuit 219 is a circuit that masks the third control signal VG2 during the period when node VO is at the H level, when the second control signal VG spans from the H level to the L level during that period. The mask circuit 219 is composed of an SR latch circuit, an inverter circuit, and an AND circuit, but is not limited to these.

[0088] The mask circuit 219 is connected to receive a signal based on the signal from APD 201 and the second control signal VG. The second control signal VG is input to the inverter circuit and becomes an inverted signal. Then, the signal obtained by inverting node VO, which is the signal based on the signal from APD 201, and the second control signal VG is input to the SR latch circuit. The signal from the SR latch circuit is input to the inverter circuit, inverted, and the output signal SRQB is output. The output signal SRQB and the fourth control signal VG2_IN are input to the gating circuit 218b and function as the third control signal VG2.

[0089] The pixel driving timing of this embodiment will be described with reference to Figure 11.

[0090] At time t1, the reset signal pRES input to memories 211a and 211b changes from a high level to a low level, and one frame period begins. At time t1, the first control signal VR changes from a high level to a low level.

[0091] At time t2, the second control signal VG transitions from L level to H level. At time t2, the inverted signal of the second control signal VG is input to the SR latch circuit, and the output signal SRQB transitions from H level to L level.

[0092] At time t3, when a photon is incident on APD201, node VO transitions from L level to H level. At this time, the first control signal VR is at L level and the second control signal VG is at H level, so at time t3, the output signal P1 from gating circuit 218a transitions from L level to H level. Then, the output signal O1 from memory 211a becomes H level.

[0093] At time t4, node VO transitions from high level to low level, and the output signal P1 from the gating circuit 218a transitions from high level to low level. This is because the first control signal VR is at low level and APD 201 is in a recharge state, resulting in a passive recharge state.

[0094] At time t5, when a photon is incident on APD201, node VO transitions from L level to H level. At this time, the first control signal VR is at L level and the second control signal VG is at H level, so at time t5, the output signal P1 from gating circuit 218a becomes H level. However, since memory circuit 211a is already at H level, it remains at H level.

[0095] At time t6, the second control signal VG transitions from high level to low level, and the output signal P1 transitions from high level to low level. Also at time t6, the input signal VG2_IN transitions from low level to high level.

[0096] At time t7, the first control signal VR transitions from L level to H level, and node VO transitions from H level to L level. This is because, since the first control signal VR was at L level, APD201 was in a recharge state, a passive recharge state. As node VO transitions from H level to L level, signal SRQB transitions from L level to H level. As a result, the third control signal VG2 transitions from L level to H level.

[0097] At time t8, when a photon is incident on APD 201, node VO transitions from L level to H level. This is the timing when the first photon is incident on APD after the first control signal VR has transitioned from L level to H level, and the third control signal VG2 is at H level, so at time t8, the output signal P2 from gating circuit 218b transitions from L level to H level. Then, the output signal O2 from memory 211b becomes H level.

[0098] At time t9, the third control signal VG2 transitions from high level to low level, and the output signal P2 from the gating circuit 218b transitions from high level to low level. The output signal O2 from the memory 211b remains at high level.

[0099] At time t10, the first control signal VR transitions from high level to low level, and node VO transitions from high level to low level.

[0100] Between time t10 and time t11, the count values ​​held in memory 211a and 211b are output to the output lines, respectively.

[0101] At time t11, the reset signal pRES transitions from L level to H level, and memories 211a and 211b are reset. As a result, output signals O1 and O2 transition from H level to L level.

[0102] At time t12, the reset signal pRES transitions from high level to low level. The period from time t1 to time t8 constitutes one frame period.

[0103] According to this embodiment, the gating circuit 218b starts the exposure period after the dead time has ended between the incidence of a photon incident during the exposure period of memory 211a and the time until the next photon can be detected. This prevents memory 211b from erroneously detecting a photon incident during the exposure period of memory 211a.

[0104] [Sixth Embodiment] The photoelectric conversion device of the sixth embodiment will be described with reference to Figure 12. Figure 12 is a timing diagram showing the operation of pixels in the photoelectric conversion device of this embodiment.

[0105] The photoelectric converter in this embodiment differs from the first embodiment in that the second control signal VG is at the H level so that it is contained within the periods of the first control signal VR being at the L level and the H level, and the third control signal VG2 changes from the L level to the H level at the same time that the next first control signal VR changes from the H level to the L level. In other words, the period during which the first control signal VR transitions to the L level twice (the period during which it is in the ON state twice) includes the period during which the second control signal VG is at the H level and the period during which the third control signal VG2 is at the H level. Apart from this point and the points described below, the embodiment is substantially the same as the first embodiment, and therefore the explanation is omitted.

[0106] The process from time t1 to time t3 is the same as in the first embodiment.

[0107] In this embodiment, at time t4, the first control signal VR transitions from L level to H level. As a result, node VO transitions from H level to L level, and output signal P1 also transitions from H level to L level.

[0108] At time t5, the first control signal VR transitions from H level to L level, and the second control signal VG transitions from H level to L level. Thus, in this embodiment, the period during which the second control signal VG is at H level contains periods during which the first control signal is at L level and periods during which it is at H level. Also at time t5, the third control signal VG2 transitions from L level to H level.

[0109] At time t6, when a photon is incident on APD 201, node VO transitions from L level to H level. At this time, the first photon is incident on APD after the first control signal VR has transitioned from H level to L level, and the third control signal VG2 is at H level, so at time t6, the output signal P2 from gating circuit 218b transitions from L level to H level. The output signal O2 from memory 211b also transitions from L level to H level.

[0110] At time t7, the third control signal VG2 transitions from high level to low level.

[0111] At time t8, the first control signal VR transitions from L level to H level, and node VO transitions from H level to L level. Then, the output signal P2 from the gating circuit 218b transitions from H level to L level.

[0112] At time t9, the reset signal pRES transitions from L level to H level, and memories 211a and 211b are reset. As a result, output signals O1 and O2 transition from H level to L level.

[0113] At time t10, the reset signal pRES transitions from high level to low level. In this embodiment, the period from time t1 to time t10 constitutes one frame period.

[0114] When the exposure period of memory 211a is long, multiple avalanche multiplications may occur, potentially generating wasted power. However, according to this embodiment, the generation of wasted power can be suppressed by setting the first control signal VR to the L level twice. At this time, since the timing of the transition of the second control signal VG from the H level to the L level and the timing of the transition of the third control signal VG2 from the L level to the H level are close together, the occurrence of periods during which the signal cannot be detected can be reduced.

[0115] [Seventh Embodiment] Figure 13 is a diagram showing the processing flow in the photoelectric conversion system of the seventh embodiment. The photoelectric conversion system of this embodiment includes a control unit 301, a timing adjustment unit 302, an image acquisition unit 303, a readout unit 304, a nonlinear correction & image synthesis unit 305, a white balance adjustment & demosaicing unit 306, a tone mapping unit 307, and a storage unit 308.

[0116] The photoelectric conversion system of this embodiment can use the photoelectric conversion device described in the first to sixth embodiments.

[0117] The image acquisition unit 303 is, for example, an APD 201, and the reading unit 304 is, for example, located after the memory 211. The control unit 301 may be an internal control unit of the photoelectric converter or it may be external to the photoelectric converter. The image acquisition unit 303 is controlled by a timing adjustment unit 302 controlled by the control unit 301. The image data generated by the image acquisition unit is input to the storage unit 308 after correction processing. Note that the order of the correction processing is not limited to the order shown in Figures 17A and 17B.

[0118] The nonlinear correction and image synthesis unit 305 is positioned between the readout unit 304 and the white balance adjustment and demosaicing unit 306, and corrects the image data under the control of the control unit 301. As described in each of the embodiments above, when the image acquisition unit 303 is a photon counting type detector, the optical response often becomes nonlinear due to the influence of dead time. When affected by a nonlinear optical response, correction based on a linear response may result in overcorrection. Therefore, overcorrection can be prevented by performing nonlinear correction on the image data, and appropriate nonlinear correction can be implemented according to the drive timing. This nonlinear correction is performed, for example, using a lookup table.

[0119] In the nonlinear correction and image synthesis unit 305, the flow shown in Figure 13 is converted to the number of incident photons.

[0120] The white balance adjustment and demosaicing section 306 and the tone mapping section 307 are standard processes.

[0121] The storage unit 308 is a storage unit that holds at least a portion of the image data generated in the preceding stage. Specifically, the image data is stored in a memory such as SRAM, DRAM, or non-volatile memory.

[0122] Thus, according to this embodiment, a photoelectric conversion system can be realized by applying the photoelectric conversion device shown in any of the embodiments described above.

[0123] [Eighth Embodiment] The photoelectric conversion system according to this embodiment will be described with reference to Figure 14. Figure 14 is a block diagram showing the schematic configuration of the photoelectric conversion system according to this embodiment.

[0124] The photoelectric conversion devices described in the first to sixth embodiments are applicable to various photoelectric conversion systems. Examples of applicable photoelectric conversion systems include digital still cameras, digital camcorders, surveillance cameras, photocopiers, fax machines, mobile phones, in-vehicle cameras, and observation satellites. Camera modules, which include optical systems such as lenses and imaging devices, are also included in photoelectric conversion systems. Figure 14 shows a block diagram of a digital still camera as an example of these.

[0125] The photoelectric conversion system in Figure 14 includes an imaging device 1004, which is an example of a photoelectric conversion device; a lens 1002 that forms an optical image of a subject onto the imaging device 1004; an aperture 1003 for varying the amount of light passing through the lens 1002; and a barrier 1001 for protecting the lens 1002. The lens 1002 and the aperture 1003 are an optical system that focuses light onto the imaging device 1004. The imaging device 1004 is a photoelectric conversion device according to one of the above embodiments, and converts the optical image formed by the lens 1002 into an electrical signal.

[0126] The photoelectric conversion system also includes a signal processing unit 1007, which is an image generation unit that generates an image by processing the output signal output from the imaging device 1004. The signal processing unit 1007 performs various corrections and compressions as needed and outputs the image data.

[0127] The photoelectric conversion system further includes a memory unit 1010 for temporarily storing image data, and an external interface unit (external I / F unit) 1013 for communicating with an external computer or the like. Furthermore, the photoelectric conversion system includes a recording medium 1012 such as a semiconductor memory for recording or reading imaging data, and a recording medium control interface unit (recording medium control I / F unit) 1011 for recording or reading data from the recording medium 1012. The recording medium 1012 may be built into the photoelectric conversion system or may be detachable.

[0128] Furthermore, the photoelectric conversion system includes an overall control and calculation unit 1009 that controls various calculations and the entire digital still camera, and a timing generation unit 1008 that outputs various timing signals to the imaging device 1004 and the signal processing unit 1007. Here, the timing signals and the like may be input from an external source, and the photoelectric conversion system only needs to have at least the imaging device 1004 and the signal processing unit 1007 that processes the output signals output from the imaging device 1004.

[0129] The imaging device 1004 outputs the imaging signal to the signal processing unit 1007. The signal processing unit 1007 performs predetermined signal processing on the imaging signal output from the imaging device 1004 and outputs image data. The signal processing unit 1007 generates an image using the imaging signal.

[0130] Thus, according to this embodiment, a photoelectric conversion system can be realized by applying the photoelectric conversion device of any of the above embodiments.

[0131] [Ninth Embodiment] The photoelectric conversion system of this embodiment will be described with reference to Figure 15. Figure 15 is a block diagram showing an example configuration of a distance image sensor, which is the photoelectric conversion system of this embodiment.

[0132] As shown in Figure 15, the distance image sensor 401 is configured to include an optical system 402, a photoelectric converter 403, an image processing circuit 404, a monitor 405, and a memory 406. The distance image sensor 401 receives light (modulated light or pulsed light) that is projected from the light source device 411 toward the subject and reflected from the surface of the subject, thereby acquiring a distance image corresponding to the distance to the subject.

[0133] The optical system 402 is composed of one or more lenses and guides the image light (incident light) from the subject to the photoelectric converter 403, where it forms an image on the light-receiving surface (sensor part) of the photoelectric converter 403.

[0134] The photoelectric converter 403 is one of the photoelectric converters from each of the embodiments described above, and a distance signal indicating the distance obtained from the light received signal output from the photoelectric converter 403 is supplied to the image processing circuit 404.

[0135] The image processing circuit 404 performs image processing to construct a distance image based on the distance signal supplied from the photoelectric converter 403. The distance image (image data) obtained through this image processing is then supplied to the monitor 405 for display or supplied to the memory 406 for storage (recording).

[0136] With the distance image sensor 401 configured in this way, by applying the photoelectric conversion device described above, the characteristics of the pixels are improved, and for example, more accurate distance images can be acquired.

[0137] [Tenth Embodiment] The photoelectric conversion system of this embodiment will be described with reference to Figure 16. Figure 16 is a diagram showing an example of a schematic configuration of an endoscopic surgical system which is the photoelectric conversion system of this embodiment.

[0138] Figure 16 illustrates a surgeon (physician) 1131 performing surgery on a patient 1132 on a patient bed 1133 using an endoscopic surgical system 1103. As shown in the figure, the endoscopic surgical system 1103 consists of an endoscope 1100, surgical instruments 1110, and a cart 1134 equipped with various devices for endoscopic surgery.

[0139] The endoscope 1100 consists of a barrel 1101, the tip of which is inserted into the body cavity of the patient 1132 for a predetermined length, and a camera head 1102 connected to the base end of the barrel 1101. In the illustrated example, the endoscope 1100 is shown as a so-called rigid endoscope having a rigid barrel 1101, but the endoscope 1100 may also be configured as a so-called flexible endoscope having a flexible barrel.

[0140] An opening into which an objective lens is fitted is provided at the tip of the endoscope tube 1101. A light source device 1203 is connected to the endoscope 1100, and the light generated by the light source device 1203 is guided to the tip of the endoscope tube by a light guide extending inside the endoscope tube 1101, and is irradiated through the objective lens towards the object to be observed inside the body cavity of the patient 1132. The endoscope 1100 may be a straight-viewing endoscope, an oblique-viewing endoscope, or a side-viewing endoscope.

[0141] The camera head 1102 contains an optical system and a photoelectric converter. Reflected light from the object being observed (observation light) is focused by the optical system into the photoelectric converter. The photoelectric converter converts the observation light into electrical signals, generating an electrical signal corresponding to the observation light, i.e., an image signal corresponding to the observed image. The photoelectric converter can be any of the photoelectric converters (imaging devices) described in the embodiments described above. The image signal is transmitted as RAW data to the camera control unit (CCU) 1135.

[0142] The CCU 1135 is composed of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and other components, and comprehensively controls the operation of the endoscope 1100 and the display device 1136. Furthermore, the CCU 1135 receives an image signal from the camera head 1102 and performs various image processing operations on the image signal, such as development processing (demosaic processing), to display an image based on the image signal.

[0143] The display device 1136 displays an image based on an image signal that has been processed by the CCU 1135, under control from the CCU 1135.

[0144] The light source device 1203 consists of a light source such as an LED (Light Emitting Diode) and supplies illumination light to the endoscope 1100 when photographing the surgical area, etc.

[0145] The input device 1137 is an input interface for the endoscopic surgical system 1103. The user can input various types of information and instructions to the endoscopic surgical system 1103 via the input device 1137.

[0146] The treatment instrument control device 1138 controls the driving of the energy treatment instrument 1112 for purposes such as tissue cauterization, incision, or blood vessel sealing.

[0147] The light source device 1203 that supplies illumination light to the endoscope 1100 when photographing the surgical area can be composed of, for example, an LED, a laser light source, or a combination thereof. When the white light source is composed of a combination of RGB laser light sources, the output intensity and output timing of each color (each wavelength) can be controlled with high precision, so that the white balance of the captured image can be adjusted in the light source device 1203. In this case, it is also possible to capture images corresponding to each of the RGB colors in time-division by irradiating the observation target with laser light from each of the RGB laser light sources in time-division and controlling the drive of the image sensor of the camera head 1102 in synchronization with the irradiation timing. According to this method, a color image can be obtained without providing a color filter on the image sensor.

[0148] Furthermore, the light source device 1203 may be controlled to change the intensity of the light it outputs at predetermined time intervals. By controlling the drive of the image sensor of the camera head 1102 in synchronization with the timing of the change in light intensity, images can be acquired in time-division order, and these images can be combined to generate high dynamic range images without so-called black crushing and white clipping.

[0149] Furthermore, the light source device 1203 may be configured to supply light in a predetermined wavelength band corresponding to special light observation. In special light observation, for example, the wavelength dependence of light absorption in body tissue is utilized. Specifically, by irradiating with narrowband light compared to the irradiation light used during normal observation (i.e., white light), predetermined tissues such as blood vessels on the surface of mucosa can be imaged with high contrast. Alternatively, in special light observation, fluorescence observation may be performed to obtain an image from fluorescence generated by irradiation with excitation light. In fluorescence observation, excitation light can be irradiated onto body tissue and fluorescence from the body tissue can be observed, or a reagent such as indocyanine green (ICG) can be injected into body tissue and excitation light corresponding to the fluorescence wavelength of the reagent can be irradiated onto the body tissue to obtain a fluorescence image. The light source device 1203 may be configured to supply narrowband light and / or excitation light corresponding to such special light observation.

[0150] [Eleventh Embodiment] The photoelectric conversion system of this embodiment will be described with reference to Figures 17A and 17B. Figure 17A illustrates the eyeglasses 1600 (smart glasses), which are the photoelectric conversion system of this embodiment. The eyeglasses 1600 have a photoelectric conversion device 1602. The photoelectric conversion device 1602 is the photoelectric conversion device (imaging device) described in each of the embodiments above. In addition, a display device including a light-emitting device such as an OLED or LED may be provided on the back side of the lens 1601. There may be one or more photoelectric conversion devices 1602. In addition, multiple types of photoelectric conversion devices may be used in combination. The arrangement position of the photoelectric conversion device 1602 is not limited to Figure 17A.

[0151] The eyeglasses 1600 further include a control device 1603. The control device 1603 functions as a power source that supplies power to the photoelectric converter 1602 and the display device. The control device 1603 also controls the operation of the photoelectric converter 1602 and the display device. The lens 1601 has an optical system formed therein for focusing light onto the photoelectric converter 1602.

[0152] Figure 17B illustrates a pair of glasses 1610 (smart glasses) in one application example. The glasses 1610 have a control device 1612, which is equipped with a photoelectric converter corresponding to a photoelectric converter 1602 and a display device. The lens 1611 has an optical system formed therein for projecting light emitted from the photoelectric converter in the control device 1612 and from the display device, and an image is projected onto the lens 1611. The control device 1612 functions as a power supply that provides power to the photoelectric converter and the display device, and also controls the operation of the photoelectric converter and the display device. The control device may have a gaze detection unit that detects the wearer's gaze. Gaze detection may use infrared light. The infrared light emitter emits infrared light towards the eyeball of the user who is fixating on the displayed image. An imaging unit having a light-receiving element detects the reflected light from the eyeball of the emitted infrared light, thereby obtaining an image of the eyeball. By having a reduction means that reduces the light from the infrared light emitter to the display unit in planar view, the deterioration of image quality is reduced.

[0153] The user's gaze towards a displayed image is detected from an image of the eyeball obtained by imaging with infrared light. Any known method can be applied to gaze detection using an image of the eyeball. As an example, a gaze detection method based on the Purkinje image obtained by the reflection of the irradiated light from the cornea can be used.

[0154] More specifically, gaze detection processing is performed based on the pupil-corneal reflection method. Using the pupil-corneal reflection method, a gaze vector representing the orientation (rotation angle) of the eyeball is calculated based on the pupil image and Purkinje image contained in the captured image of the eyeball, thereby detecting the user's gaze.

[0155] The display device of this embodiment includes a photoelectric converter having a light-receiving element, and may control the display image of the display device based on the user's gaze information from the photoelectric converter.

[0156] Specifically, the display device determines a first field of view that the user is fixated on, and a second field of view other than the first field of view, based on gaze information. The first and second field of view may be determined by the control device of the display device, or they may be determined by an external control device and received by the display device. Within the display area of ​​the display device, the display resolution of the first field of view may be controlled to be higher than the display resolution of the second field of view. In other words, the resolution of the second field of view may be lower than that of the first field of view.

[0157] Furthermore, the display area has a first display area and a second display area different from the first display area, and a higher priority area may be determined from the first and second display areas based on gaze information. The first and second view areas may be determined by the control device of the display device, or they may be determined by an external control device and received. The resolution of the higher priority area may be controlled to be higher than the resolution of the areas other than the higher priority area. In other words, the resolution of areas with relatively lower priority may be set lower.

[0158] AI may be used to determine the first field of view area and the areas with high priority. The AI ​​may be a model configured to estimate the angle of line of sight and the distance to the target object at the end of the line of sight from the image of the eye, using the image of the eye and the direction the eye was actually looking in the image as training data. The AI ​​program may be installed in the display device, the photoelectric converter, or an external device. If installed in an external device, it will be transmitted to the display device via communication.

[0159] When display control is based on visual detection, this method is preferably applicable to smart glasses that further include a photoelectric converter for capturing images of the surrounding environment. The smart glasses can display the captured external information in real time.

[0160] [Twelfth Embodiment] The photoelectric conversion device and photoelectric conversion system described above may be applied to electronic devices such as so-called smartphones and tablets.

[0161] Figures 18A and 18B show an example of an electronic device 1500 equipped with a photoelectric converter. Figure 18A shows the front side of the electronic device 1500, and Figure 18B shows the back side of the electronic device 1500.

[0162] As shown in Figure 18A, a display 1510 for displaying images is positioned in the center of the surface of the electronic device 1500. Along the upper edge of the surface of the electronic device 1500, front cameras 1521 and 1522 using photoelectric converters, an IR light source 1530 that emits infrared light, and a visible light source 1540 that emits visible light are positioned.

[0163] Furthermore, as shown in Figure 18B, rear cameras 1551 and 1552, which use photoelectric converters, an IR light source 1560 that emits infrared light, and a visible light source 1570 that emits visible light are arranged along the upper edge of the back of the electronic device 1500.

[0164] In the electronic device 1500 configured in this way, by applying the photoelectric converter described above, it is possible to capture, for example, higher-quality images. The photoelectric converter can also be applied to other electronic devices such as infrared sensors, distance measuring sensors using active infrared light sources, security cameras, and personal or biometric authentication cameras. This can improve the accuracy and performance of these electronic devices.

[0165] [13th Embodiment] Figure 19 is a block diagram of the X-ray CT apparatus in this embodiment. As described above, the photoelectric converters in the first to sixth embodiments are applicable to the detector of the X-ray CT apparatus. The X-ray CT apparatus 30 in this embodiment has an X-ray generating unit 310, a wedge 311, a collimator 312, an X-ray detection unit 320, a top plate 330, and a rotating frame 340. Furthermore, the X-ray CT apparatus 30 includes a high-voltage generator 350, a data acquisition device (DAS: Data Acquisition System) 351, a signal processing unit 352, a display unit 353, and a control unit 354.

[0166] The X-ray generating unit 310 is composed of, for example, a vacuum tube that generates X-rays. The vacuum tube of the X-ray generating unit 310 is supplied with high voltage and filament current from the high voltage generator 350. X-rays are generated when thermionic electrons are irradiated from the cathode (filament) to the anode (target).

[0167] The wedge 311 is a filter that adjusts the amount of X-rays irradiated from the X-ray generator 310. The wedge 311 attenuates the amount of X-rays so that the X-rays irradiated from the X-ray generator 310 to the subject have a predetermined distribution. The collimator 312 is made up of a lead plate or the like that narrows the irradiation range of the X-rays that have passed through the wedge 311. The X-rays generated in the X-ray generator 310 are formed into a cone beam shape via the collimator 312 and irradiated onto the subject on the top plate 330.

[0168] The X-ray detection unit 320 is configured using the photoelectric conversion device described in the first to sixth embodiments above. The X-ray detection unit 320 detects the X-rays that have passed through the subject from the X-ray generation unit 310 and outputs a signal corresponding to the X-ray amount as DAS351.

[0169] The rotating frame 340 is annular in shape and is configured to be rotatable. Inside the rotating frame 340, the X-ray generating unit 310 (wedge 311, collimator 312) and the X-ray detection unit 320 are arranged facing each other. The X-ray generating unit 310 and the X-ray detection unit 320 are rotatable together with the rotating frame 340.

[0170] The high-voltage generator 350 includes a boost circuit and outputs a high voltage to the X-ray generation unit 310. The DAS 351 includes an amplification circuit and an A / D conversion circuit and outputs the signal from the X-ray detection unit 320 as digital data to the signal processing unit 352.

[0171] The signal processing unit 352 includes a CPU (Central Processing Unit), ROM (Read Only Memory), and RAM (Random Access Memory), and is capable of performing image processing on digital data. The display unit 353 includes a flat-panel display device, and is capable of displaying X-ray images. The control unit 354 includes a CPU, ROM, RAM, etc., and controls the operation of the entire X-ray CT apparatus 30.

[0172] [14th Embodiment] The photoelectric conversion system and mobile body of this embodiment will be described with reference to Figures 20A and 20B. Figures 20A and 20B are diagrams showing the configuration of the photoelectric conversion system and mobile body of this embodiment.

[0173] Figure 20A shows an example of a photoelectric conversion system related to an in-vehicle camera. The photoelectric conversion system 2300 has an imaging device 2310. The imaging device 2310 is a photoelectric conversion device as described in any of the embodiments above. The photoelectric conversion system 2300 has an image processing unit 2312 that performs image processing on a plurality of image data acquired by the imaging device 2310. The photoelectric conversion system 2300 also has a parallax acquisition unit 2314 that calculates parallax (phase difference of parallax images) from a plurality of image data acquired by the photoelectric conversion system 2300. Furthermore, the photoelectric conversion system 2300 has a distance acquisition unit 2316 that calculates the distance to an object based on the calculated parallax, and a collision determination unit 2318 that determines whether or not there is a possibility of collision based on the calculated distance. Here, the parallax acquisition unit 2314 and the distance acquisition unit 2316 are examples of distance information acquisition means that acquire distance information to an object. In other words, distance information may be acquired not only by phase difference but also by ToF (Time Of Flight) technology. The collision determination unit 2318 may use any of this distance information to determine the possibility of a collision. The distance information acquisition means may be implemented by specially designed hardware or by a software module. It may also be implemented by FPGA (Field Programmable Gate Array) or ASIC (Application Specific Integrated Circuit), or by a combination thereof.

[0174] The photoelectric conversion system 2300 is connected to the vehicle information acquisition device 2320 and can acquire vehicle information such as vehicle speed, yaw rate, and steering angle. The photoelectric conversion system 2300 is also connected to the control ECU 2330, which is a control device (control unit) that outputs a control signal to generate braking force on the vehicle based on the judgment result of the collision judgment unit 2318. The photoelectric conversion system 2300 is also connected to the warning device 2340, which issues a warning to the driver based on the judgment result of the collision judgment unit 2318. For example, if the collision judgment unit 2318 determines that there is a high probability of collision, the control ECU 2330 performs vehicle control to avoid a collision or mitigate damage by applying the brakes, releasing the accelerator, or suppressing engine output. The warning device 2340 warns the user by sounding an alarm, displaying warning information on a screen such as a car navigation system, or vibrating the seat belt or steering wheel.

[0175] In this embodiment, the photoelectric conversion system 2300 images the area around the vehicle, for example, in front of or behind it. Figure 20B shows the photoelectric conversion system when imaging the area in front of the vehicle (imaging range 2350). The vehicle information acquisition device 2320 sends instructions to the photoelectric conversion system 2300 or the imaging device 2310. This configuration can further improve the accuracy of distance measurement.

[0176] The above example illustrates control to prevent collisions with other vehicles, but it can also be applied to control systems that automatically follow other vehicles or automatically prevent vehicles from straying from their lanes. Furthermore, the photoelectric conversion system can be applied not only to vehicles such as the vehicle itself, but also to mobile objects (mobile devices) such as ships, aircraft, or industrial robots. In addition, it can be applied not only to mobile objects but also to a wide range of devices that utilize object recognition, such as intelligent transportation systems (ITS).

[0177] The embodiments described above are merely examples of how the Disclosure may be implemented, and the technical scope of the Disclosure should not be limited by them. In other words, the Disclosure can be implemented in various ways without departing from its technical concept or its main features.

[0178] The above-disclosed embodiment includes the following configuration.

[0179] (Configuration 1) The system comprises an avalanche photodiode, a recharge circuit to which a periodic first control signal is input and which controls the recharge operation of the avalanche photodiode, and a first gating circuit and a second gating circuit connected to the avalanche photodiode, wherein the first gating circuit is input to control whether or not to output a signal based on the output signal of the avalanche photodiode, the second gating circuit is input to control whether or not to output a signal based on the output signal of the avalanche photodiode, when the first gating circuit is in the off state, the first gating circuit outputs a signal based on the output signal of the avalanche photodiode, when the second gating circuit is in the off state, the second gating circuit outputs a signal based on the output signal of the avalanche photodiode, The photoelectric converter is characterized in that the period during which the recharge circuit is turned on by the first control signal includes the time when the first gating circuit changes from an off state to an on state and the time when the second gating circuit changes from an on state to an off state.

[0180] (Configuration 2) The photoelectric converter according to Configuration 1, characterized in that the difference between the time when the first gating circuit changes from an off state to an on state and the time when the second gating circuit changes from an on state to an off state is within the range of the period during which the first gating circuit is in an off state or the period during which the third control signal is in an off state.

[0181] (Configuration 3) The photoelectric conversion device according to Configuration 1 or Configuration 2, characterized in that the time at which the first gating circuit changes from the off state to the on state and the time at which the second gating circuit changes from the on state to the off state are simultaneous.

[0182] (Configuration 4) The photoelectric conversion device according to any one of Configurations 1 to 3, characterized in that the difference between the time when the recharge circuit changes from an ON state to an OFF state and the time when the second gating circuit changes from an ON state to an OFF state is within the period during which the first gating circuit is in an OFF state or the period during which the second gating circuit is in an OFF state.

[0183] (Configuration 5) The photoelectric conversion device according to any one of Configurations 1 to 4, characterized in that the time at which the recharge circuit changes from an ON state to an OFF state is simultaneous with the time at which the second gating circuit changes from an ON state to an OFF state.

[0184] (Configuration 6) The photoelectric conversion device according to any one of Configurations 1 to 5, characterized in that the period during which the recharge circuit is ON includes the period during which the second gating circuit is OFF.

[0185] (Configuration 7) The photoelectric conversion device according to any one of Configurations 1 to 6, characterized in that it has a mask circuit to which the second control signal and a signal based on the output signal of the avalanche photodiode are input and which is connected to the second gating circuit.

[0186] (Configuration 8) The photoelectric conversion device according to Configuration 7, characterized in that the mask circuit generates the third control signal which is input to the second gating circuit.

[0187] (Configuration 9) The photoelectric conversion device according to Configuration 7 or 8, characterized in that the mask circuit includes an SR latch circuit.

[0188] (Configuration 10) A photoelectric converter according to any one of Configurations 1 to 9, characterized by having a first memory for holding the output of the first gating circuit and a second memory for holding the output of the second gating circuit.

[0189] (Configuration 11) The photoelectric converter according to Configuration 10, characterized in that it has a first memory, a second memory, and a selection circuit connected to an output line.

[0190] (Configuration 12) The photoelectric conversion device according to any one of Configurations 1 to 11, characterized in that the periods of the second control signal and the third control signal are the same.

[0191] (Configuration 13) The photoelectric conversion device according to any one of Configurations 1 to 12, characterized in that the period during which the recharge circuit is ON includes a period during which the first gating circuit is OFF and a period during which the second gating circuit is OFF.

[0192] (Configuration 14) The photoelectric conversion device according to any one of Configurations 1 to 12, characterized in that the period during which the recharge circuit is turned on twice includes a period during which the first gating circuit is turned off and a period during which the second gating circuit is turned off.

[0193] (Configuration 15) A photoelectric conversion device according to any one of Configurations 1 to 4 or Configurations 6 to 13, characterized in that, in one frame period, the recharge circuit turns on from the off state before the first gating circuit turns off from the on state.

[0194] (Configuration 16) The photoelectric conversion device according to Configuration 15, characterized in that, within one frame period, the period in which the recharge circuit goes from an ON state to an OFF state and then to an ON state again includes a period in which the second gating circuit goes from an ON state to an OFF state.

[0195] (Configuration 17) A photoelectric conversion system characterized by comprising a photoelectric conversion device described in any one of Configurations 1 to 16, and a signal processing unit that generates an image using the signal output by the photoelectric conversion device.

[0196] (Configuration 18) A mobile body comprising a photoelectric converter according to any one of Configurations 1 to 16, characterized in that it has a control unit that controls the movement of the mobile body using signals output by the photoelectric converter.

[0197] The present invention is not limited to the embodiments described above, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are attached to make the scope of the invention public.

[0198] This application claims priority based on Japanese Patent Application No. 2025-067953, filed on 17 April 2025, and all of its contents are incorporated herein by reference.

[0199] 201 Photodiode 202 Recharge circuit 210 Waveform shaping circuit 218 Gating circuit 211 Counting unit (memory) 113 Output line

Claims

1. A photoelectric converter comprising: an avalanche photodiode; a recharge circuit to which a periodic first control signal is input and which controls the recharge operation of the avalanche photodiode; and a first gating circuit and a second gating circuit connected to the avalanche photodiode, wherein the first gating circuit is input to control whether or not to output a signal based on the output signal of the avalanche photodiode; the second gating circuit is input to control whether or not to output a signal based on the output signal of the avalanche photodiode; when the first gating circuit is in the off state, the first gating circuit outputs a signal based on the output signal of the avalanche photodiode; when the second gating circuit is in the off state, the second gating circuit outputs a signal based on the output signal of the avalanche photodiode; and the period during which the recharge circuit is in the ON state due to the first control signal includes the time when the first gating circuit changes from the OFF state to the ON state and the time when the second gating circuit changes from the ON state to the OFF state.

2. The photoelectric converter according to claim 1, characterized in that the difference between the time when the first gating circuit changes from an off state to an on state and the time when the second gating circuit changes from an on state to an off state is within the range of the period during which the first gating circuit is in an off state or the period during which the third control signal is in an off state.

3. The photoelectric conversion device according to claim 1, characterized in that the point at which the first gating circuit changes from the off state to the on state and the point at which the second gating circuit changes from the on state to the off state occur simultaneously.

4. The photoelectric conversion device according to claim 1, characterized in that the difference between the time when the recharge circuit changes from an ON state to an OFF state and the time when the second gating circuit changes from an ON state to an OFF state is within the period during which the first gating circuit is OFF or the period during which the second gating circuit is OFF.

5. The photoelectric conversion device according to claim 1, characterized in that the moment when the recharge circuit changes from the ON state to the OFF state and the moment when the second gating circuit changes from the ON state to the OFF state occur simultaneously.

6. The photoelectric conversion device according to claim 1, characterized in that the period during which the recharge circuit is ON includes the period during which the second gating circuit is OFF.

7. The photoelectric conversion device according to claim 1, characterized in that it has a mask circuit to which the second control signal and a signal based on the output signal of the avalanche photodiode are input and which is connected to the second gating circuit.

8. The photoelectric conversion device according to claim 7, characterized in that the mask circuit generates the third control signal that is input to the second gating circuit.

9. The photoelectric conversion device according to claim 8, characterized in that the mask circuit includes an SR latch circuit.

10. The photoelectric conversion device according to claim 1, characterized in that it has a first memory for holding the output of the first gating circuit and a second memory for holding the output of the second gating circuit.

11. The photoelectric conversion device according to claim 9, characterized in that it has a first memory for holding the output of the first gating circuit and a second memory for holding the output of the second gating circuit.

12. The photoelectric converter according to claim 10, characterized in that it has a selection circuit connected to the first memory, the second memory, and the output line.

13. The photoelectric converter according to claim 1, characterized in that the periods of the second control signal and the third control signal are the same.

14. The photoelectric conversion device according to claim 1, characterized in that the period during which the recharge circuit is ON includes a period during which the first gating circuit is OFF and a period during which the second gating circuit is OFF.

15. The photoelectric conversion device according to claim 1, characterized in that the period during which the recharge circuit is turned on twice includes a period during which the first gating circuit is turned off and a period during which the second gating circuit is turned off.

16. The photoelectric conversion device according to claim 1, characterized in that, during one frame period, the recharge circuit switches from the off state to the on state before the first gating circuit switches from the on state to the off state.

17. The photoelectric conversion device according to claim 16, characterized in that, within one frame period, the period in which the recharge circuit goes from an ON state to an OFF state and then turns ON again includes a period in which the second gating circuit goes from an ON state to an OFF state.

18. A photoelectric conversion system characterized by comprising: a photoelectric conversion device according to claim 1; and a signal processing unit that generates an image using a signal output by the photoelectric conversion device.

19. A mobile body comprising a photoelectric converter according to claim 1, characterized in that it has a control unit that controls the movement of the mobile body using signals output by the photoelectric converter.

Images