Photoelectric conversion device, photoelectric conversion system, and mobile body

Abstract

This photoelectric conversion device includes a photodiode for performing avalanche multiplication, a recharge circuit to which a periodic first control signal is input, at least two gating circuits, and at least one mask circuit. An output of the mask circuit is input to any one of the at least two gating circuits. The period during which the recharge circuit is turned on by the first control signal overlaps a period during which a second control signal at a first level is input to the first gating circuit and a period during which a third control signal at a second level is input to the second gating circuit, and a period during which the second control signal at the first level is input to the first gating circuit and a period during which the third control signal at the second level is input to the second gating circuit are offset from one another.

Description

Photoelectric conversion device, photoelectric conversion system, and mobile body

[0001] The present disclosure relates to a photoelectric conversion device, a photoelectric conversion system, and a mobile body.

[0002] Patent Document 1 discloses a photoelectric conversion device in which an output signal from a photodiode that performs avalanche multiplication is input to two circuit units, and detection signals output from each circuit unit are gated.

[0003] Japanese Patent Application Laid-Open No. 2024-160505

[0004] However, there is room for expanding the dynamic range in the photoelectric conversion device described in Patent Document 1.

[0005] According to the present invention, it is possible to provide a photoelectric conversion device capable of expanding the dynamic range with respect to Patent Document 1.

[0006] According to one disclosure of the present specification, a photoelectric conversion device includes a photodiode that performs avalanche multiplication, a recharge circuit to which a periodic first control signal is input and that controls the recharge operation of the photodiode, at least two gating circuits connected to the photodiode and controlling whether to output an output signal of the photodiode, and at least one mask circuit to which an output from at least any one of the at least two gating circuits and the first control signal are input. The output of the mask circuit is input to any one of the at least two gating circuits. The at least two gating circuits include a first gating circuit and a second gating circuit. A period during which the recharge circuit is in an on state by the first control signal overlaps with a period during which a second control signal of a first level is input so that a signal is output from the first gating circuit, and a period during which a third control signal of a second level is input so that a signal is output from the second gating circuit. A period during which the second control signal of the first level is input to the first gating circuit and a period during which the third control signal of the second level is input to the second gating circuit are out of alignment.

[0007] According to the present invention, it is possible to provide a photoelectric conversion device capable of expanding the dynamic range compared to Patent Document 1, and a photoelectric conversion system and mobile body using such a photoelectric conversion device.

[0008] This is a schematic diagram of a photoelectric converter according to an embodiment. This is a schematic diagram of a pixel substrate of a photoelectric converter according to an embodiment. This is a schematic diagram of a circuit board of a photoelectric converter according to an embodiment. This is an example of the configuration of a photoelectric converter according to an embodiment. This is a block diagram of a photoelectric converter according to the first embodiment. This is a circuit diagram of a pixel of a photoelectric converter according to the first embodiment. This is a drive timing diagram of a pixel according to the first embodiment. This is a drive timing diagram of a pixel according to the first embodiment. This is a diagram showing the output from the memory at each illuminance according to the first embodiment. This is a diagram showing the processing flow for estimating the number of incident photons at each pixel according to the first embodiment. This is a diagram showing the effect of setting the number of exposures appropriately for the photoelectric converter according to the first embodiment. This is a diagram showing the effect of setting the number of exposures appropriately for the photoelectric converter according to the first embodiment. This is a circuit diagram of a pixel of a photoelectric converter according to a second embodiment. This is a drive timing diagram of a pixel according to a second embodiment. This is a diagram relating to the dynamic range of the photoelectric converter for a pixel according to a second embodiment. This is a circuit diagram of a pixel of a photoelectric converter according to a third embodiment. This is a drive timing diagram of a pixel according to a third embodiment. This is a diagram relating to the dynamic range of the photoelectric converter for a pixel according to a third embodiment. This is a circuit diagram of a pixel in a photoelectric converter according to the fourth embodiment. This is a circuit diagram of a pixel in a photoelectric converter according to the fifth embodiment. This is a functional block diagram of a photoelectric conversion system according to the sixth embodiment. This is a functional block diagram of a photoelectric conversion system according to the seventh embodiment. This is a functional block diagram of a photoelectric conversion system according to the eighth embodiment. This is a functional block diagram of a photoelectric conversion system according to the ninth embodiment. This is a schematic diagram of a photoelectric conversion system according to the tenth embodiment. This is a schematic diagram of a photoelectric conversion system according to the tenth embodiment. This is a schematic diagram of a photoelectric conversion system according to the eleventh embodiment. This is a schematic diagram of a photoelectric conversion system according to the eleventh embodiment. This is a functional block diagram of a photoelectric conversion system according to the twelfth embodiment. This is a functional block diagram of a photoelectric conversion system according to the thirteenth embodiment. This is a functional block diagram of a photoelectric conversion system according to the thirteenth embodiment. This is a circuit diagram of a pixel in a photoelectric converter according to a comparative example. This is a drive timing diagram of a pixel according to a comparative example.This figure shows the output from memory at various illuminance levels for the comparative example.

[0009] The embodiments shown below are intended to embody the technical concept of the present invention and do not limit it. The size and positional relationships of the components shown in each drawing may be exaggerated for clarity of explanation. In the following description, identical components may be given the same number and their explanation may be omitted.

[0010] Embodiments of the present invention will be described in detail below with reference to the drawings. In the following description, terms indicating specific directions or positions (e.g., "up," "down," "right," "left," and other terms including these terms) will be used as needed. The use of these terms is for the purpose of facilitating the understanding of embodiments with reference to the drawings, and the technical scope is not limited by the meaning of these terms.

[0011] In this specification, a plan view refers to viewing the semiconductor layer on which the avalanche photodiode (avalanche photodiode, APD) is arranged from a direction perpendicular to the light incident surface. A cross-sectional view refers to the surface of the semiconductor layer perpendicular to the light incident surface. If the light incident surface of the semiconductor layer is rough when viewed microscopically, the plan view is defined based on the light incident surface of the semiconductor layer when viewed macroscopically.

[0012] In the following explanation, the anode of the APD is set to a fixed potential, and the signal is taken from the cathode side. Therefore, the semiconductor region of the first conductivity type, which has majority carriers of the same polarity as the signal charge, is an N-type semiconductor region, and the semiconductor region of the second conductivity type, which has majority carriers of charges with a different polarity than the signal charge, is a P-type semiconductor region. Note that the present invention also applies when the cathode of the APD is set to a fixed potential and the signal is taken from the anode side. In this case, the semiconductor region of the first conductivity type, which has majority carriers of the same polarity as the signal charge, is a P-type semiconductor region, and the semiconductor region of the second conductivity type, which has majority carriers of charges with a different polarity than the signal charge, is an N-type semiconductor region. The following explanation describes the case where one node of the APD is set to a fixed potential, but the potentials of both nodes may fluctuate.

[0013] Furthermore, in the following embodiments, connections between circuit elements may be described. In this case, even if another element is interposed between the elements of interest, unless otherwise specified, the elements of interest will be treated as connected. For example, suppose element A is connected to one node of a capacitive element C having multiple nodes, and element B is connected to the other node. Even in such a case, elements A and B will be treated as connected unless otherwise specified.

[0014] In the following, elements and circuits with similar functions are given the same designation, and are sometimes distinguished by adding different alphabetical subscripts at the end. For example, gating circuits 204a and 204b. When it is not necessary to explain them separately, the subscripts such as a, b, etc. are omitted and the common parts are explained.

[0015] In the embodiments described below, the imaging device will be described as an example of a photoelectric conversion device. However, each embodiment is not limited to the imaging device and can be applied to other examples of photoelectric conversion devices. Other examples of photoelectric conversion devices include distance measuring devices (devices for measuring distance using focus detection or TOF (Time Of Flight)), photometric devices (devices for measuring the amount of incident light, etc.).

[0016] The common configuration of the photoelectric converter 100 in each embodiment will be explained using Figures 1 to 4. The photoelectric converter has SPAD pixels including an APD.

[0017] Figure 1 is a schematic diagram of a photoelectric converter 100, showing the configuration of a stacked type photoelectric converter 100. The photoelectric converter 100 includes a sensor substrate (first substrate) and a circuit board (second substrate) stacked on top of each other, and the sensor substrate and the circuit board are electrically connected to each other. However, the photoelectric converter is not limited to this. For example, it may be a photoelectric converter in which the components included in the sensor substrate and the components included in the circuit board are arranged on a common semiconductor layer, as described below.

[0018] The photoelectric conversion device in this embodiment is a back-illuminated type photoelectric conversion device in which light is incident from the first surface of the sensor substrate and a circuit board is arranged on the second surface of the sensor substrate. The sensor substrate has a first semiconductor layer 11 having a photoelectric conversion element described later and a first wiring structure. The circuit board has a second semiconductor layer 21 having a circuit such as a signal processing unit described later and a second wiring structure. The second semiconductor layer 21, the second wiring structure, the first wiring structure, and the first semiconductor layer 11 are stacked in that order to constitute the photoelectric conversion device 100.

[0019] In the following description, the sensor substrate and the circuit board may be, but are not limited to, diced chips. For example, each substrate may be a wafer. Furthermore, each substrate may be stacked in wafer form and then diced, or it may be made into chips and then stacked and bonded together. The sensor substrate has a pixel region 12, and the circuit board has a circuit region 22 that processes the signals detected by the pixel region 12.

[0020] Figure 2 shows an example of the arrangement of the first semiconductor layer 11 of the sensor substrate. Multiple pixels 101 each contain an APD 201 and are arranged in a two-dimensional array in a planar view, forming a pixel region 12.

[0021] Pixel 101 is typically a pixel for forming an image, but when used in Time of Flight (TOF), it is not necessarily required to form an image. That is, pixel 101 may be a pixel for measuring the time and amount of light that arrives.

[0022] Figure 3 shows an example of the arrangement of the second semiconductor layer 21 on the circuit board. The circuit board has a circuit region 22 on which the signal processing circuit 103 is located, a vertical scanning circuit 110, a readout circuit 112, a horizontal scanning circuit 111, a control pulse generation circuit 115, an adder 118, an output circuit 114, a scan line 116, and an output line 113. In a plan view, the circuit region 22 is located in the region that overlaps with the pixel region 12 in Figure 2. Furthermore, in a plan view, the vertical scanning circuit 110, readout circuit 112, horizontal scanning circuit 111, control pulse generation circuit 115, adder 118, and output circuit 114 are located so as to overlap with the region between the edge of the sensor board in Figure 2 and the edge of the pixel region 12. In other words, the sensor board has a pixel region 12 and a non-pixel region arranged around the pixel region 12. Then, in the area that overlaps with the non-pixel area in a plan view, the vertical scanning circuit 110, readout circuit 112, horizontal scanning circuit 111, control pulse generation circuit 115, adder 118, and output circuit 114 are arranged.

[0023] The signal processing circuit 103 is electrically connected to the pixels 101 via connecting wires provided for each pixel 101, and is arranged in a two-dimensional array in a planar view, similar to the pixels 101. The signal processing circuit 103 includes a memory that holds a count value based on photons incident on the pixels 101.

[0024] The vertical scanning circuit 110 receives control pulses supplied from the control pulse generation circuit 115 and supplies control pulses to the signal processing circuit 103 corresponding to the pixels 101 of each row via the scan line 116. The vertical scanning circuit 110 may be composed of logic circuits such as a shift register and an address decoder.

[0025] The control pulse generation circuit 115 has a signal generation unit that generates the first control signal VR, the second control signal VG, and the third control signal VG2, which will be described later. The signal generation unit generates pulse signals that control the recharge circuit 202, the gating circuit 204a (first gating circuit), and the gating circuit 204b (second gating circuit), as will be described later. For example, the signal generation unit may generate the control signals in common for multiple pixels in the pixel region, or it may generate the control signals for each pixel. The control pulse generation circuit 115 preferably has, for example, a frequency divider circuit. This makes it possible to control simply and reduces the increase in the number of elements.

[0026] The readout circuit 112 acquires the signal output from the memory of the signal processing circuit 103 for each row via the output line 113. The signals output from the memory are accumulated by the adder 118, and the output signal is output to an external signal processing circuit (signal processing device) of the photoelectric converter 100 via the output circuit 114. The readout circuit 112 may also have the function of a signal processing circuit that performs correction of the count value, etc. Furthermore, although the example below describes an example in which the adder 118 is located outside the circuit area 22, a circuit corresponding to the adder 118 may be located inside the pixel 101. In addition, the adder 118 may be located after the output circuit 114 and in a signal processing device located outside the photoelectric converter 100.

[0027] The horizontal scanning circuit 111 receives control pulses from the control pulse generation circuit 115 and sequentially outputs the signal values ​​of each column in the readout circuit 112 to the output circuit 114.

[0028] In Figure 2, the arrangement of photoelectric conversion elements in the pixel region 12 may be arranged in a one-dimensional manner. Furthermore, the effects of this disclosure can be achieved even in a configuration with only one pixel 101, and a configuration with one pixel 101 can also be included in this disclosure. The signal processing circuit 103 does not necessarily need to be provided for each APD 201; for example, one signal processing circuit 103 may be shared by multiple APDs 201, and signal processing may be performed sequentially.

[0029] Figure 4 shows an example configuration illustrating the relationship between the signal supplied from the vertical scanning circuit 110 and the circuit region 22 and the pixel region 12. In the pixel region 12, a color filter may be placed at each pixel. The color filters can be arranged, for example, in a Bayer array. In Figure 3, the vertical scanning circuit 110 is arranged along one side of the pixel region 12, but as shown in Figure 4, the vertical scanning circuit 110 may be arranged on the right and left sides of the pixel region 12 in a plan view.

[0030] Each pixel is supplied with a first control signal VR, a second control signal VG, and a third control signal VG2 from the vertical scanning circuit 110, as described later. The control lines for each control signal are connected so that a common first control signal VR, second control signal VG, and third control signal VG2 are supplied to multiple pixels arranged in a single row. In the vertical scanning circuit 110, each control signal is transmitted to each control line using binary partitioning.

[0031] [First Embodiment] The photoelectric conversion device of the first embodiment will be described using Figures 5 to 10B.

[0032] Figure 5 is a block diagram of the pixel in this embodiment, and Figure 6 is a circuit diagram of the pixel in this embodiment.

[0033] The pixels of this embodiment include an avalanche multiplication photodiode (APD) 201, a recharge circuit 202, a waveform shaping circuit 203, a gating circuit 204, a memory 206, and a mask circuit 210.

[0034] The APD201 is a photodiode that generates charge pairs corresponding to incident light through photoelectric conversion. A voltage VL (first voltage) is supplied to the anode of the APD201. A voltage VH (second voltage), which is higher than the voltage VL supplied to the anode, is supplied to the cathode of the APD201, and a reverse bias voltage is applied to the APD201 that causes avalanche multiplication. When such voltages are supplied, the charge generated by the incident light on the APD201 undergoes avalanche multiplication, and an avalanche current is generated.

[0035] When a reverse bias voltage is supplied to the APD201, it can operate in Geiger mode or linear mode. In Geiger mode, the APD201 is operated by applying a voltage greater than the breakdown voltage to the anode and cathode. In linear mode, the APD201 is operated with a potential difference between the anode and cathode that is near or below the breakdown voltage. An APD operating in Geiger mode is called a SPAD. For example, the SPAD functions with a voltage VL of -30V and a voltage VH of 1V.

[0036] The recharge circuit 202 is connected between the power supply that provides voltage VH and the cathode terminal of the APD 201. The recharge circuit 202 is controlled by a periodically input clock signal (first control signal) VR, which drives the APD 201 to clock recharge. The recharge circuit 202 controls the recharge operation of the APD 201. The clock signal VR controls the APD 201 between a state in which avalanche multiplication is possible (second state) and a recharge state (first state) in which avalanche multiplication is controlled. In the first state, the recharge circuit 202 is ON, and the cathode of the APD 201 is connected to voltage VH. In the second state, the recharge circuit 202 is OFF, so there is no connection between the cathode of the APD 201 and voltage VH. In other words, the resistance between the cathode of the APD 201 and voltage VH is higher compared to the charged state. When a photon is incident on APD201 in the second state, avalanche multiplication occurs, and the cathode voltage of APD201 drops. Subsequently, the recharge circuit 202 is turned on when the clock signal VR is input to it, the voltage is charged, and the cathode voltage of APD201 returns to its initial state.

[0037] The recharge circuit 202 can be constructed using a MOS transistor, as shown in Figure 6. Below, we will describe an example where the recharge circuit 202 uses an NMOS transistor, but the recharge circuit 202 may also use a PMOS transistor. In that case, the signal level at which the PMOS transistor's gate is ON is reversed compared to the case where the recharge circuit 202 uses an NMOS transistor. That is, the recharge circuit is ON when the signal level input to the gate is low, and OFF when the signal level input to the gate is high.

[0038] The output signal from APD 201 is input to the subsequent waveform shaping circuit 203. The waveform shaping circuit 203 shapes the potential change of the cathode of APD 201 during photon detection and outputs a pulse signal to node node A. For example, an inverter circuit can be used as the waveform shaping circuit 203. Figure 5 shows an example where one inverter circuit is used as the waveform shaping circuit 203, but a circuit with multiple inverters connected in series may be used, or other circuits that have a waveform shaping effect may be used.

[0039] Gating circuits 204a and 204b are connected downstream of the waveform shaping circuit 203. Gating circuit 204a controls whether or not to output an output signal indicating the detection of a photon in the APD 201 based on the signal from the waveform shaping circuit 203 and the second control signal VG. Gating circuit 204b controls whether or not to output an output signal indicating the detection of a photon in the APD 201 based on the signal from the waveform shaping circuit 203 and the third control signal VG2. Gating circuit 204 includes, for example, an AND gate. One input node of the AND gate is connected to node node A, and the other input node of the AND gate is connected to the control line of the second control signal VG or the third control signal VG2. Gating circuit 204 performs a logical AND operation on the input signal. Gating circuit 204 outputs the result of the operation based on the signal from the APD 201 and the second control signal VG or the third control signal VG2 to the downstream stage.

[0040] Memory 206 counts the detection signal indicating the detection of photon incidence on the APD 201 and outputs the count result as a count value. Specifically, memory 206a (first memory) counts the number of times the output signal P1 transmitted from the gating circuit 204a transitions from a low level (hereinafter referred to as L level) to a high level (hereinafter referred to as H level) as the count value for a predetermined period (e.g., 1 frame period). Memory 206b (second memory) also counts the number of times the output signal P2 from the gating circuit 204b transitions from L level to H level as the count value for a predetermined period (e.g., 1 frame period). After the predetermined period (e.g., 1 frame period) has elapsed, memory 206a outputs output signal O1 to output line 113a, and memory 206b outputs output signal O2 to output line 113b. After the predetermined period (e.g., 1 frame period) has elapsed, a reset signal RES is input to reset memory 206.

[0041] In this embodiment, we will describe the case where the memories 206a and 206b are memories capable of holding 1 bit of data, but the memories may be memories capable of holding 2 bits or more of data.

[0042] The mask circuit 210 receives the output from at least one of the gating circuits 204 included in a single pixel and the first control signal VR as input, and is connected so that the output signal is input to one of the gating circuits 204 included in a single pixel. In Figure 6, two gating circuits 204, gating circuit 204a and gating circuit 204b, are arranged in a single pixel, and the output signal P1 of gating circuit 204a is connected to be input to the mask circuit 210. Also, the output signal of the mask circuit 210 is connected to be input to gating circuit 204b. The mask circuit 210 can control the driving of the other gating circuits 204 when an output is detected in any one of the multiple gating circuits 204 included in a single pixel. Specifically, it is possible to control the exposure period of the other gating circuits 204. This makes it possible to control the output of signals to the memory 206 connected to the gating circuits 204 according to the illuminance incident on the pixel.

[0043] In this embodiment, the mask circuit 210 is composed of a circuit including an SR latch, but the mask circuit 210 is not limited thereto.

[0044] During the period when the output signal from the / Q terminal of the mask circuit 210 is at the L level, even if the high-level third control signal VG2 is input to the gating circuit 204b, the output signal P2 from the gating circuit 204b remains at the L level. That is, the gating circuit 204b is in the non-exposure period. Thereby, at high illuminance, only the gating circuit 204a functions as the exposure period, and counting is performed only by the memory 206a.

[0045] In FIG. 6, there is one mask circuit 210 for one pixel, but there may be a plurality of mask circuits 210.

[0046] It is preferable that the number of at least two gating circuits is M (M is a natural number of 2 or more), and the number of mask circuits 210 is N (N is a natural number and N < M). And it is preferable that the number of times the recharge circuit 202 is turned on by the first control signal VR in one period is O times (O is a natural number and O < M).

[0047] FIGS. 7A to 7C are drive timing diagrams of pixels in the photoelectric conversion device of this embodiment. FIG. 7A is a drive timing diagram of pixels at low illuminance, FIG. 7B is a drive timing diagram of pixels at medium illuminance, and FIG. 7C is a drive timing diagram of pixels at high illuminance. In this embodiment, the period from the time when the reset signal RES changes from the H level to the L level to the next time when the reset signal RES changes from the H level to the L level corresponds to one frame period. The reset timing of the memory 206 may be performed commonly for all pixels, or may be sequentially performed for each pixel row.

[0048] When the photoelectric conversion device is an imaging device, one frame period can be the period for capturing one image. Also, for example, one frame period is the period during which the vertical scanning circuit scans from the pixels in the first row arranged in the pixel region to the pixels in the last row. That is, the period from when the pulse signal, which is the vertical synchronization signal, becomes high level once until it becomes high level again is one frame period. Here, it is not necessary to scan the pixels in all rows from the pixels in the first row to the pixels in the last row within one frame period. For example, when skipping some rows among all rows and scanning, the period from a certain row to the end of scanning in one direction until the last row is scanned is defined as one frame period. For example, assume that the pixel region has pixels from the first row to the nth row and starts scanning from the second row and scans up to the (n - 1)th row. In this case, the period from the second row to the (n - 1)th row is one frame period. And then the period for scanning from the second row to the (n - 1)th row next is the next frame period. Also, when scanning the skipped rows after skipping a certain row, the period until the skipped rows are scanned completely may be defined as one frame period.

[0049] The first control signal VR is periodically input to the recharge circuit 202. In this embodiment, the period during which the first control signal VR is at the H level is the period during which the APD 201 is in the recharge state.

[0050] The second control signal VG has a period at the H level (first level) from the time when the first control signal VR becomes the L level until the time when the first control signal VR becomes the H level next. That is, the period during which the first control signal VR is at the L level and the period during which the second control signal VG is at the H level partially overlap. The period from the time when the first control signal VR changes from the H level to the L level until the time when the second control signal VG changes from the H level to the L level is the period during which the gating circuit 204a can be exposed.

[0051] The third control signal VG2 has a period of being at an H level (second level) from the time the first control signal VR becomes L level until the time the first control signal VR becomes H level again. In other words, the period when the first control signal VR is at an L level and the period when the third control signal VG2 is at an H level partially overlap. The period from the time the first control signal VR changes from an H level to an L level until the time the third control signal VG2 changes from an H level to an L level is the period during which the gating circuit 204b can be exposed.

[0052] The period during which the second control signal VG is at a high level is different from the period during which the third control signal VG2 is at a high level. In other words, the period during which gating circuit 204a can be exposed is different from the period during which gating circuit 204b can be exposed. This makes it possible to adjust the signal retention in memory 206a and the signal retention in memory 206b.

[0053] It is preferable that the period during which gating circuit 204b can be exposed is longer than the period during which gating circuit 204a can be exposed. By appropriately changing the exposure period of each gating circuit in this way, it is possible to obtain a characteristic that maximizes the dynamic range.

[0054] The periods of the first control signal VR, the second control signal VG, and the third control signal VG2 can be made the same.

[0055] It is preferable that the number of times the recharge circuit 202 is turned on by the first control signal VR in one cycle is less than the number of at least two gating circuits 204 contained in one pixel. In Figure 6, the number of times the recharge circuit 202 is turned on by the first control signal VR in one cycle is 1, which is less than the number of two gating circuits 204a and 204b contained in one pixel. As a result, the mask circuit becomes effective and the output of the gating circuit 204b is controlled.

[0056] When a photon is incident during the period when the first control signal VR is at an L level, the output node nodeA from the waveform shaping circuit 203 changes from an L level to an H level. When node nodeA is at an H level and the second control signal VG is at an H level, the output from the gating circuit 204a transitions from an L level to an H level. Furthermore, when node nodeA is at an H level and the third control signal VG2 is at an H level, the output from the gating circuit 204b transitions from an L level to an H level.

[0057] First, let's explain the pixel driving timing diagram in low light conditions shown in Figure 7A.

[0058] At time t1, the reset signal RES input to memories 206a and 206b changes from a high level to a low level, and one frame period begins.

[0059] At time t2, when a photon is incident on APD201, node nodeA transitions from L level (third level) to H level (fourth level). At this time, the first control signal VR is at L level and the third control signal VG2 is at H level, so at time t2, the output signal P2 from gating circuit 202b transitions from L level to H level. Then, the output signal O2 from memory 206b becomes H level.

[0060] At time t3, when the first control signal VR changes from L level to H level, node nodeA transitions from H level to L level. As a result, the output signal P2 transitions from H level to L level.

[0061] At time t4, a photon is incident, and node nodeA becomes high. Since the third control signal VG2 is high, the output signal P2 of the gating circuit 204b becomes high. The output signal O2 from memory 206b has maintained a high level since time t2 and remains high at time t4.

[0062] At time t5, the first control signal VR reaches the H level.

[0063] At time t6, a photon is incident, and node nodeA becomes high. Since the third control signal VG2 is high, the output signal P2 of the gating circuit 204b becomes high. The output signal O2 from memory 206b has maintained a high level since time t2 and remains high at time t6.

[0064] At time t7, when the third control signal VG2 changes from high level to low level, node nodeA transitions from high level to low level. As a result, the output signal P2 transitions from high level to low level.

[0065] At time t8, the reset signal RES is input to memories 206a and 206b. In Figure 7A, the output signal O1 of memory 206a is at a low level and is maintained, while the output signal O2 of memory 206b changes from a high level to a low level. The period from time t1 to time t8 constitutes one frame period.

[0066] Next, we will explain the pixel timing diagram at medium illumination, as shown in Figure 7B.

[0067] In Figure 7B, the period from time t1 to time t8 is one frame period.

[0068] At time t1, the reset signal RES input to memories 206a and 206b changes from a high level to a low level, and one frame period begins.

[0069] At time t2, when a photon is incident, node nodeA becomes H level. At time t2, the second control signal VG is H level, so the output signal P1 from gating circuit 204a becomes H level. Then, the output signal O1 from memory 206a becomes H level. Also, when an H level signal P1 from gating circuit 204a is input to mask circuit 210, the output signal from the / Q terminal of mask circuit 210 transitions from H level (fifth level) to L level (sixth level). During the period when the output signal from the / Q terminal of mask circuit 210 is L level, the gating circuit 204b is masked so that exposure does not occur. Therefore, even when node nodeA is H level and the third control signal VG2 is H level, the output signal P2 from gating circuit 204b remains at L level.

[0070] When the second control signal VG reaches the L level, the exposure period of the gating circuit 204a ends, and the output signal P1 from the gating circuit 204a transitions from the H level to the L level.

[0071] At time t3, when the first control signal VR transitions from L level to H level, the output signal from the / Q terminal of the mask circuit 210 also transitions from L level to H level.

[0072] When a photon is incident at time t4, node nodeA becomes high. At time t4, the third control signal VG2 is high, so the output signal P2 from the gating circuit 204b becomes high. Then, the output signal O2 from memory 206b becomes high.

[0073] At time t5, when the third control signal VG2 changes from high level to low level, node nodeA transitions from high level to low level. As a result, the output signal P2 transitions from high level to low level.

[0074] At time t6, a photon is incident, and node nodeA becomes high. Since the third control signal VG2 is high, the output signal P2 of the gating circuit 204b becomes high. The output signal O2 from memory 206b has maintained a high level since time t2 and remains high at time t6.

[0075] At time t7, when the third control signal VG2 changes from high level to low level, node nodeA transitions from high level to low level. As a result, the output signal P2 transitions from high level to low level.

[0076] At time t8, the reset signal RES is input to memories 206a and 206b, and the output signals O1 and O2 of memories 206a and 206b become L level.

[0077] Next, we will explain the pixel timing diagram at high illumination levels shown in Figure 7C.

[0078] In Figure 7C, the period from time t1 to time t9 is one frame period.

[0079] At time t1, the reset signal RES input to memories 206a and 206b changes from a high level to a low level, and one frame period begins.

[0080] At time t2, when a photon is incident, node nodeA becomes H level. At time t2, the second control signal VG is H level, so the output signal P1 from gating circuit 204a changes from L level to H level. Then, the output signal O1 from memory 206a becomes H level. Also, when an H level signal P1 from gating circuit 204a is input to mask circuit 210, the output signal from the / Q terminal of mask circuit 210 transitions from H level to L level. During the period when the output signal from the / Q terminal of mask circuit 210 is L level, the gating circuit 204b is masked so that exposure does not occur. Therefore, even when node nodeA is H level and the third control signal VG2 is H level, the output signal P2 from gating circuit 204b remains at L level.

[0081] When the second control signal VG reaches the L level, the exposure period of the gating circuit 204a ends, and the output signal P1 from the gating circuit 204a transitions from the H level to the L level.

[0082] At time t3, when the first control signal VR transitions from L level to H level, the output signal from the / Q terminal of the mask circuit 210 also transitions from L level to H level.

[0083] When a photon is incident at time t4, node A becomes H level. Since the second control signal VG is H level, the output signal P1 from gating circuit 204a changes from L level to H level. When an H level signal P1 from gating circuit 204a is input to mask circuit 210, the output signal from the / Q terminal of mask circuit 210 transitions from H level to L level. During the period when the output signal from the / Q terminal of mask circuit 210 is L level, the gating circuit 204b is masked so that exposure does not occur. Therefore, even when node A is H level and the third control signal VG2 is H level, the output signal P2 from gating circuit 204b remains at L level.

[0084] At time t5, when the first control signal VR changes from L level to H level, node nodeA transitions from H level to L level. As a result, the output signal from the / Q terminal of the mask circuit 210 transitions from L level to H level.

[0085] At time t6, a photon is incident, and node nodeA becomes high. Since the second control signal VG is high, the output signal P1 of the gating circuit 204a becomes high. The output signal O1 from memory 206a has maintained a high level since time t2 and remains high at time t6.

[0086] At time t7, when the first control signal VR changes from L level to H level, node nodeA transitions from H level to L level. As a result, the output signal from the / Q terminal of the mask circuit 210 transitions from L level to H level.

[0087] At time t8, a photon is incident, and node nodeA becomes high. Since the second control signal VG is high, the output signal P1 of the gating circuit 204a becomes high. The output signal O1 from memory 206a has maintained a high level since time t2 and remains high at time t8.

[0088] At time t9, the reset signal RES is input to memories 206a and 206b, and the output signal O1 of memory 206a changes from a high level to a low level.

[0089] Figure 8 shows the sum of K 1-bit frames for the outputs from memory 206a and 206b at each illuminance level. The vertical axis represents the output (logarithmic), and the horizontal axis represents the number of incident photons (logarithmic). The solid line shows N_out1, the value obtained by summing K 1-bit frames for the output from memory 206a, and the dashed line shows N_out2, the value obtained by summing K 1-bit frames for the output from memory 206b. Region A represents the output (logarithmic) against the number of incident photons (logarithmic) at low illuminance. Region B represents the output (logarithmic) against the number of incident photons (logarithmic) at medium illuminance. Region C represents the output (logarithmic) against the number of incident photons (logarithmic) at high illuminance. K is the value at which the sum of K 1-bit frames saturates. K can be an integer greater than or equal to 2.

[0090] At low illumination levels, counting occurs only in memory 206b and not in memory 206a. At medium illumination levels, counting occurs in both memory 206a and 206b. At high illumination levels, counting occurs only in memory 206a and not in memory 206b.

[0091] The number of incident photons can be reconstructed using region A until values ​​N_out1 and N_out2 saturate. In region B, value N_out2 saturates, so the number of incident photons can be reconstructed using value N_out1. In region C, value N_out1 saturates, but value N_out2 decreases, so the number of incident photons can be reconstructed using value N_out2.

[0092] According to this embodiment, in region C, the number of incident photons can be reconstructed using the value N_out2, thus expanding the dynamic range compared to the comparative example described later. The comparative example shown in Figures 25 to 27 will be used to explain this further.

[0093] Figure 25 is a block diagram of the pixels in the comparative example, Figure 26 is a drive timing diagram of the pixels in the comparative example, and Figure 27 is a diagram showing the sum of K 1-bit frames for the outputs from the memories 206a and 206b for each illuminance in the comparative example. The photoelectric converter in the comparative example differs from the first embodiment in that a configuration corresponding to a mask circuit is not provided for the pixels. For other configurations, the same reference numerals are used for configurations similar to those in the first embodiment and their descriptions are omitted.

[0094] Figure 26 illustrates the pixel timing diagram in the comparative example.

[0095] In Figure 26, the period from time t1 to time t7 is one frame period.

[0096] At time t1, the reset signal RES input to memories 206a and 206b changes from a high level to a low level, and one frame period begins.

[0097] When a photon is incident at time t2, node nodeA becomes high. At time t2, the second control signal VG is high, so the output signal P1 from the gating circuit 204a becomes high. Then, the output signal O1 from the memory 206a becomes high.

[0098] At time t3, node nodeA is at the H level and the third control signal VG2 is at the H level, so the output signal P2 from the gating circuit 204b changes from L level to H level, and the output signal O2 from the memory 206b changes from L level to H level.

[0099] At time t4, the first control signal VR changes from L level to H level, and node nodeA changes from H level to L level. Then, the output signal P2 of the gating circuit 204b changes from H level to L level.

[0100] When a photon is incident at time t5, the third control signal VG2 is at the H level, so the output signal P2 of the gating circuit 204b changes from the L level to the H level. However, the output signal O2 of the memory 206b remains at the H level.

[0101] When a photon is incident at time t6, the third control signal VG2 is at the H level, so the output signal P2 of the gating circuit 204b changes from the L level to the H level. However, the output signal O2 of the memory 206b remains at the H level.

[0102] At time t7, the reset signal RES is input to memories 206a and 206b, and the output signals O1 and O2 of memories 206a and 206b become L level.

[0103] Figure 27 shows the sum of K 1-bit frames for the output from memory 206a and 206b at each illuminance level. In the comparative example, the number of incident photons can be reconstructed using region A until values ​​N_out1 and N_out2 saturate. In region B, since value N_out2 saturates, the number of incident photons can be reconstructed using value N_out1. However, in the comparative example, in the illuminance region higher than region B, values ​​N_out1 and N_out2 saturate, so the number of incident photons cannot be reconstructed. Therefore, only a dynamic range up to the range of region B can be secured.

[0104] According to this embodiment, a mask circuit 210 is provided for each pixel, and the exposure period of the gating circuit 204b is adjusted. As a result, the number of incident photons can be restored in region C using the value N_out2, thereby expanding the dynamic range by the range of region C compared to the comparative example.

[0105] Figure 9 shows the processing flow for estimating the number of incident photons in each pixel according to this embodiment. In this embodiment, the number of incident photons can be restored by performing the processing flow shown in Figure 9.

[0106] The frame period is started. In step S801, it is determined whether the value N_out2 is saturated. If the value N_out2 is not saturated, the process proceeds to step S802; if the value N_out2 is saturated, the process proceeds to step S803.

[0107] In step S803, it is determined that this is the illuminance of region B, and the value N_out1 is converted to the number of incident photons.

[0108] In step S802, it is determined whether the value N_out1 is saturated. If the value N_out1 is not saturated, the process proceeds to step S805; if the value N_out1 is saturated, the process proceeds to step S804.

[0109] In step S804, it is determined that the illuminance is in region C, and the value N_out2 is converted to the number of incident photons.

[0110] In step S805, it is determined that the illuminance is in region A, and the value N_out2 is converted to the number of incident photons.

[0111] In steps S803, S804, and S805, the number of incident photons can be reconstructed from the output count value using a lookup table or the like.

[0112] As shown in Figure 10A, it is preferable to appropriately set the number of first control signal VR, second control signal VG, and third control signal VG2 in one frame. Specifically, when K 1-bit frames are added at medium illumination, it is preferable to set the first control signal VR, second control signal VG, and third control signal VG2 in the frame so that all memories 206a and 206b contained in the pixels do not become saturated. As shown in Figure 10B, if a region D occurs where values ​​N_out1 and N_out2 become saturated, it becomes difficult to recover the number of incident photons from the count value.

[0113] According to this embodiment, by including the mask circuit 210, it is possible to expand the dynamic range compared to the comparative example.

[0114] [Second Embodiment] The photoelectric converter of the second embodiment will be described with reference to Figures 11, 12A, and 12B. The photoelectric converter of this embodiment differs from the first embodiment in that it has three gating circuits 204 and three memories 206. Aside from this point and the points described below, it is substantially the same as the first embodiment, and therefore the explanation will be omitted.

[0115] Figure 11 shows the circuit diagram of the pixel in this embodiment. Gating circuits 204a, 204b, and 204c are connected downstream of the waveform shaping circuit 203. The gating circuit 204c controls whether or not to output an output signal indicating the detection of a photon in the APD 201, based on the signal from the waveform shaping circuit 203, the fourth control signal VG3, and the output from the / Q terminal of the mask circuit 210.

[0116] In this embodiment, the first control signal VR is input twice in one cycle. One cycle refers to the period from the falling edge of the second first control signal VR to the falling edge of the next second first control signal. However, it is not limited to this, and the first control signal VR may be input once in one cycle.

[0117] Memory 206c counts the number of times the output signal P3 from the gating circuit 204c transitions from a low level to a high level as a count value over a predetermined period (e.g., one frame period). After the predetermined period (e.g., one frame period) has elapsed, memory 206c outputs the output signal O3 to the output line 113c. Then, after the predetermined period (e.g., one frame period) has elapsed, a reset signal RES is input to reset memory 206.

[0118] Figure 12A is a diagram showing the pixel drive timing in the photoelectric conversion device of this embodiment.

[0119] At time t1, the reset signal RES input to memories 206a, 206b, and 206c changes from a high level to a low level, and one frame period begins.

[0120] At time t2, when a photon is incident on APD 201, node nodeA transitions from L level to H level. Since the second control signal VG is at H level, at time t2 the output signal P1 from the gating circuit 202a transitions from L level to H level, and the signal O1 output from memory 206a becomes H level.

[0121] At time t3, a photon is incident. Since the third control signal VG2 is at the H level, the output signal P2 of the gating circuit 204b becomes H level, and the output signal O2 from memory 206b transitions from L level to H level. At this time, the H level of the output signal P2 is also input to the mask circuit 210, and the output signal from the / Q terminal of the mask circuit 210 changes from H level to L level.

[0122] At time t4, when the first control signal VR changes from L level to H level, the output signal from the / Q terminal of the mask circuit 210 also changes from L level to H level.

[0123] At time t5, a photon is incident. Since the third control signal VG2 is at the H level, the output signal P2 of the gating circuit 204b becomes H level, and the output signal O2 from the memory 206b transitions from L level to H level. At this time, the H level of the output signal P2 is also input to the mask circuit 210, and the output signal from the / Q terminal of the mask circuit 210 changes from H level to L level.

[0124] At time t6, when the first control signal VR changes from L level to H level, the output signal from the / Q terminal of the mask circuit 210 also changes from L level to H level.

[0125] At time t7, a photon is incident. Since the fourth control signal VG3 is at the H level, the output signal P3 of the gating circuit 204c becomes H level, and the output signal O3 from memory 206c transitions from L level to H level.

[0126] At time t8, the reset signal RES is input to memories 206a, 206b, and 206c. In Figure 12A, the output signals O1 of memory 206a, O2 of memory 206b, and O3 of memory 206c change from a high level to a low level. The period from time t1 to time t8 constitutes one frame period.

[0127] Figure 12B shows the sum of K 1-bit frames for the outputs from memory 206a, 206b, and 206c at each illuminance level. The dashed line shows the value N_out3, which is the sum of K 1-bit frames for the output from memory 206c.

[0128] According to this embodiment, since the illuminance output in region D can be restored, it is possible to expand the dynamic range compared to the first embodiment.

[0129] [Third Embodiment] The photoelectric conversion device of the third embodiment will be described with reference to Figures 13, 14A, and 14B. The photoelectric conversion device according to this embodiment differs from the second embodiment in that one pixel has two mask circuits 210a and 210b. Aside from this point and the points described below, it is substantially the same as the second embodiment, and therefore the description will be omitted.

[0130] Figure 13 shows the circuit diagram of the pixels in this embodiment. The mask circuit 210a and the mask circuit 210b control the memory 206c. The mask circuit 210a also controls the memory 206b.

[0131] Figure 14A is a diagram showing the pixel drive timing in the photoelectric converter of this embodiment, and Figure 14B is a diagram showing the sum of K 1-bit frames for the outputs from memory 206a, 206b, and 206c at each illuminance.

[0132] At time t1, the reset signal RES input to memories 206a, 206b, and 206c changes from a high level to a low level, and one frame period begins.

[0133] At time t2, when a photon is incident on APD 201, node nodeA transitions from L level to H level. Because the second control signal VG is at H level, at time t2 the output signal P1 from gating circuit 202a transitions from L level to H level, and the output signal O1 from memory 206a becomes H level. At time t2, the output signals from the / Q terminals of mask circuits 210a and 210b change from H level to L level.

[0134] At time t3, when the first control signal VR changes from L level to H level, the output signals from the / Q terminals of the mask circuits 210a and 210b also change from L level to H level.

[0135] At time t4, a photon is incident. Since the third control signal VG2 is at the H level, the output signal P2 of the gating circuit 204b becomes H level, and the output signal O2 from the memory 206b transitions from L level to H level. At this time, the H level of the output signal P2 is also input to the mask circuit 210b, and the output signal from the / Q terminal of the mask circuit 210b changes from H level to L level.

[0136] At time t5, a photon is incident. Since the fourth control signal VG3 is at the H level, the output signal P3 of the gating circuit 204c becomes H level, and the output signal O3 from memory 206c transitions from L level to H level.

[0137] At time t6, the reset signal RES is input to memories 206a, 206b, and 206c. In Figure 14A, the output signals O1 of memory 206a, O2 of memory 206b, and O3 of memory 206c change from H level to L level. The period from time t1 to time t6 constitutes one frame period.

[0138] Figure 14B is a diagram showing the sum of K 1-bit frames for the outputs from memories 206a, 206b, and 206c at each illuminance level. According to this embodiment, the illuminance output in region E can be restored, making it possible to expand the dynamic range compared to the second embodiment.

[0139] [Fourth Embodiment] The photoelectric converter of the fourth embodiment will be described with reference to Figure 15. Figure 15 is a diagram showing the circuit diagram of a pixel in this embodiment. The photoelectric converter of this embodiment differs from the first embodiment in that one pixel has a selection circuit 250, output signals O1 and O2 from memories 206a and 206b are input to the selection circuit 250, and output from the selection circuit 250 to the output line 113. Except for this point and the points described below, it is substantially the same as the first embodiment, so the explanation will be omitted.

[0140] According to this embodiment, the number of output lines 113 can be reduced while expanding the dynamic range, similar to the first embodiment.

[0141] [Fifth Embodiment] The photoelectric converter of the fifth embodiment will be described with reference to Figure 16. Figure 16 is a diagram showing the circuit diagram of the pixels of this embodiment. The photoelectric converter of this embodiment differs from the first embodiment in that the output signal O1 from memory 206a is input to the count circuit 208a, and the output signal O2 from memory 206b is input to the count circuit 208b. Except for this point and the points described below, it is substantially the same as the first embodiment, so the explanation will be omitted.

[0142] According to this embodiment, since signal integration can be performed in each count circuit 208, memory located outside the pixels or sensors becomes unnecessary. Therefore, a reduction in memory outside the pixels and sensors can be expected.

[0143] [Sixth Embodiment] Figure 17 is a diagram showing the processing flow in the photoelectric conversion system of the sixth embodiment. The photoelectric conversion system of this embodiment includes a control unit 301, a timing adjustment unit 302, an image acquisition unit 303, a readout unit 304, a nonlinear correction & image synthesis unit 305, a white balance adjustment & demosaicing unit 306, a tone mapping unit 307, and a storage unit 308.

[0144] The photoelectric conversion system of this embodiment can use the photoelectric conversion device described in the first to fifth embodiments.

[0145] The image acquisition unit 303 is, for example, an APD 101, and the reading unit 304 is, for example, located after the memory 206. The control unit 301 may be an internal control unit of the photoelectric converter or it may be external to the photoelectric converter. The image acquisition unit 303 is controlled by a timing adjustment unit 302 controlled by the control unit 301. The image data generated by the image acquisition unit is input to the storage unit 308 after correction processing. Note that the order of the correction processing is not limited to the order shown in Figure 17.

[0146] The nonlinear correction and image synthesis unit 305 is positioned between the readout unit 304 and the white balance adjustment and demosaicing unit 306, and corrects the image data under the control of the control unit 301. As described in each of the embodiments above, when the image acquisition unit 303 is a photon counting type detector, the optical response often becomes nonlinear due to the influence of dead time. When affected by a nonlinear optical response, correction based on a linear response may result in overcorrection. Therefore, overcorrection can be prevented by performing nonlinear correction on the image data, and appropriate nonlinear correction can be implemented according to the drive timing. This nonlinear correction is performed, for example, using a lookup table.

[0147] In the nonlinear correction and image synthesis unit 305, the number of incident photons is converted from the flow shown in Figure 9.

[0148] The white balance adjustment and demosaicing section 306 and the tone mapping section 307 are standard processes.

[0149] The storage unit 308 is a storage unit that holds at least a portion of the image data generated in the preceding stage. Specifically, the image data is stored in a memory such as SRAM, DRAM, or non-volatile memory.

[0150] Thus, according to this embodiment, a photoelectric conversion system can be realized by applying the photoelectric conversion device shown in any of the embodiments described above.

[0151] [Seventh Embodiment] The photoelectric conversion system according to this embodiment will be described with reference to Figure 18. Figure 18 is a block diagram showing the schematic configuration of the photoelectric conversion system according to this embodiment.

[0152] The photoelectric conversion devices described in the first to fifth embodiments are applicable to various photoelectric conversion systems. Examples of applicable photoelectric conversion systems include digital still cameras, digital camcorders, surveillance cameras, photocopiers, fax machines, mobile phones, in-vehicle cameras, and observation satellites. Camera modules, which include optical systems such as lenses and imaging devices, are also included in photoelectric conversion systems. Figure 18 shows a block diagram of a digital still camera as an example of these.

[0153] The photoelectric conversion system shown in Figure 18 includes an imaging device 1004, which is an example of a photoelectric conversion device; a lens 1002 that forms an optical image of a subject onto the imaging device 1004; an aperture 1003 for varying the amount of light passing through the lens 1002; and a barrier 1001 for protecting the lens 1002. The lens 1002 and the aperture 1003 are an optical system that focuses light onto the imaging device 1004. The imaging device 1004 is a photoelectric conversion device according to one of the above embodiments, and converts the optical image formed by the lens 1002 into an electrical signal.

[0154] The photoelectric conversion system also includes a signal processing unit 1007, which is an image generation unit that generates an image by processing the output signal output from the imaging device 1004. The signal processing unit 1007 performs various corrections and compressions as needed and outputs the image data. The signal processing unit 1007 may be formed on the semiconductor substrate on which the imaging device 1004 is mounted, or it may be formed on a semiconductor substrate separate from the imaging device 1004. Alternatively, the imaging device 1004 and the signal processing unit 1007 may be formed on the same semiconductor substrate.

[0155] The photoelectric conversion system further includes a memory unit 1010 for temporarily storing image data, and an external interface unit (external I / F unit) 1013 for communicating with an external computer or the like. Furthermore, the photoelectric conversion system includes a recording medium 1012 such as a semiconductor memory for recording or reading imaging data, and a recording medium control interface unit (recording medium control I / F unit) 1011 for recording or reading data from the recording medium 1012. The recording medium 1012 may be built into the photoelectric conversion system or may be detachable.

[0156] Furthermore, the photoelectric conversion system includes an overall control and calculation unit 1009 that controls various calculations and the entire digital still camera, and a timing generation unit 1008 that outputs various timing signals to the imaging device 1004 and the signal processing unit 1007. Here, the timing signals and the like may be input from an external source, and the photoelectric conversion system only needs to have at least the imaging device 1004 and the signal processing unit 1007 that processes the output signals output from the imaging device 1004.

[0157] The imaging device 1004 outputs the imaging signal to the signal processing unit 1007. The signal processing unit 1007 performs predetermined signal processing on the imaging signal output from the imaging device 1004 and outputs image data. The signal processing unit 1007 generates an image using the imaging signal.

[0158] Thus, according to this embodiment, a photoelectric conversion system can be realized by applying the photoelectric conversion device of any of the above embodiments.

[0159] [Eighth Embodiment] The photoelectric conversion system of this embodiment will be described with reference to Figure 19. Figure 19 is a block diagram showing an example configuration of a distance image sensor, which is the photoelectric conversion system of this embodiment.

[0160] As shown in Figure 19, the distance image sensor 401 is configured to include an optical system 402, a photoelectric converter 403, an image processing circuit 404, a monitor 405, and a memory 406. The distance image sensor 401 receives light (modulated light or pulsed light) that is projected from a light source device 411 toward the subject and reflected from the surface of the subject, thereby acquiring a distance image corresponding to the distance to the subject.

[0161] The optical system 402 is composed of one or more lenses and guides the image light (incident light) from the subject to the photoelectric converter 403, where it forms an image on the light-receiving surface (sensor part) of the photoelectric converter 403.

[0162] The photoelectric converter 403 is one of the photoelectric converters from each of the embodiments described above, and a distance signal indicating the distance obtained from the light received signal output from the photoelectric converter 403 is supplied to the image processing circuit 404.

[0163] The image processing circuit 404 performs image processing to construct a distance image based on the distance signal supplied from the photoelectric converter 403. The distance image (image data) obtained through this image processing is then supplied to the monitor 405 for display or supplied to the memory 406 for storage (recording).

[0164] With the distance image sensor 401 configured in this way, by applying the photoelectric conversion device described above, the characteristics of the pixels are improved, and for example, more accurate distance images can be acquired.

[0165] [Ninth Embodiment] The photoelectric conversion system of this embodiment will be described with reference to Figure 20. Figure 20 is a diagram showing an example of a schematic configuration of an endoscopic surgical system which is the photoelectric conversion system of this embodiment.

[0166] Figure 20 illustrates a surgeon (physician) 1131 performing surgery on a patient 1132 on a patient bed 1133 using an endoscopic surgical system 1103. As shown in the figure, the endoscopic surgical system 1103 consists of an endoscope 1100, surgical instruments 1110, and a cart 1134 equipped with various devices for endoscopic surgery.

[0167] The endoscope 1100 consists of a barrel 1101, the tip of which is inserted into the body cavity of the patient 1132 for a predetermined length, and a camera head 1102 connected to the base end of the barrel 1101. In the illustrated example, the endoscope 1100 is shown as a so-called rigid endoscope having a rigid barrel 1101, but the endoscope 1100 may also be configured as a so-called flexible endoscope having a flexible barrel.

[0168] An opening into which an objective lens is fitted is provided at the tip of the endoscope tube 1101. A light source device 1203 is connected to the endoscope 1100, and the light generated by the light source device 1203 is guided to the tip of the endoscope tube by a light guide extending inside the endoscope tube 1101, and is irradiated through the objective lens towards the object to be observed inside the body cavity of the patient 1132. The endoscope 1100 may be a straight-viewing endoscope, an oblique-viewing endoscope, or a side-viewing endoscope.

[0169] The camera head 1102 contains an optical system and a photoelectric converter. Reflected light from the object being observed (observation light) is focused by the optical system into the photoelectric converter. The photoelectric converter converts the observation light into electrical signals, generating an electrical signal corresponding to the observation light, i.e., an image signal corresponding to the observed image. The photoelectric converter can be any of the photoelectric converters (imaging devices) described in the embodiments described above. The image signal is transmitted as RAW data to the camera control unit (CCU) 1135.

[0170] The CCU 1135 is composed of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and other components, and comprehensively controls the operation of the endoscope 1100 and the display device 1136. Furthermore, the CCU 1135 receives an image signal from the camera head 1102 and performs various image processing operations on the image signal, such as development processing (demosaic processing), to display an image based on the image signal.

[0171] The display device 1136 displays an image based on an image signal that has been processed by the CCU 1135, under control from the CCU 1135.

[0172] The light source device 1203 consists of a light source such as an LED (Light Emitting Diode) and supplies illumination light to the endoscope 1100 when photographing the surgical area, etc.

[0173] The input device 1137 is an input interface for the endoscopic surgical system 1103. The user can input various types of information and instructions to the endoscopic surgical system 1103 via the input device 1137.

[0174] The treatment instrument control device 1138 controls the driving of the energy treatment instrument 1112 for purposes such as tissue cauterization, incision, or blood vessel sealing.

[0175] The light source device 1203 that supplies illumination light to the endoscope 1100 when photographing the surgical area can be composed of, for example, an LED, a laser light source, or a combination thereof. When the white light source is composed of a combination of RGB laser light sources, the output intensity and output timing of each color (each wavelength) can be controlled with high precision, so that the white balance of the captured image can be adjusted in the light source device 1203. In this case, it is also possible to capture images corresponding to each of the RGB colors in time-division by irradiating the observation target with laser light from each of the RGB laser light sources in time-division and controlling the drive of the image sensor of the camera head 1102 in synchronization with the irradiation timing. According to this method, a color image can be obtained without providing a color filter on the image sensor.

[0176] Furthermore, the light source device 1203 may be controlled to change the intensity of the light it outputs at predetermined time intervals. By controlling the drive of the image sensor of the camera head 1102 in synchronization with the timing of the change in light intensity, images can be acquired in time-division order, and these images can be combined to generate high dynamic range images without so-called black crushing and white clipping.

[0177] Furthermore, the light source device 1203 may be configured to supply light in a predetermined wavelength band corresponding to special light observation. In special light observation, for example, the wavelength dependence of light absorption in body tissue is utilized. Specifically, by irradiating with narrowband light compared to the irradiation light used during normal observation (i.e., white light), predetermined tissues such as blood vessels on the surface of mucosa can be imaged with high contrast. Alternatively, in special light observation, fluorescence observation may be performed to obtain an image from fluorescence generated by irradiation with excitation light. In fluorescence observation, excitation light can be irradiated onto body tissue and fluorescence from the body tissue can be observed, or a reagent such as indocyanine green (ICG) can be injected into body tissue and excitation light corresponding to the fluorescence wavelength of the reagent can be irradiated onto the body tissue to obtain a fluorescence image. The light source device 1203 may be configured to supply narrowband light and / or excitation light corresponding to such special light observation.

[0178] [Tenth Embodiment] The photoelectric conversion system of this embodiment will be described with reference to Figures 21A and 21B. Figure 21A illustrates the eyeglasses 1600 (smart glasses), which are the photoelectric conversion system of this embodiment. The eyeglasses 1600 have a photoelectric conversion device 1602. The photoelectric conversion device 1602 is the photoelectric conversion device (imaging device) described in each of the embodiments above. In addition, a display device including a light-emitting device such as an OLED or LED may be provided on the back side of the lens 1601. There may be one or more photoelectric conversion devices 1602. In addition, multiple types of photoelectric conversion devices may be used in combination. The arrangement position of the photoelectric conversion device 1602 is not limited to Figure 21A.

[0179] The eyeglasses 1600 further include a control device 1603. The control device 1603 functions as a power source that supplies power to the photoelectric converter 1602 and the display device. The control device 1603 also controls the operation of the photoelectric converter 1602 and the display device. The lens 1601 has an optical system formed therein for focusing light onto the photoelectric converter 1602.

[0180] Figure 21B illustrates a pair of glasses 1610 (smart glasses) according to one application example. The glasses 1610 have a control device 1612, which is equipped with a photoelectric converter corresponding to a photoelectric converter 1602 and a display device. The lens 1611 has an optical system formed therein for projecting light emitted from the photoelectric converter in the control device 1612 and from the display device, and an image is projected onto the lens 1611. The control device 1612 functions as a power supply that provides power to the photoelectric converter and the display device, and also controls the operation of the photoelectric converter and the display device. The control device may have a gaze detection unit that detects the wearer's gaze. Gaze detection may use infrared light. The infrared light emitter emits infrared light towards the eyeball of the user who is fixating on the displayed image. An imaging unit having a light-receiving element detects the reflected light from the eyeball of the emitted infrared light, thereby obtaining an image of the eyeball. By having a reduction means that reduces the light from the infrared light emitter to the display unit in planar view, the deterioration of image quality is reduced.

[0181] The user's gaze towards the displayed image is detected from an image of the eyeball obtained by imaging with infrared light. Any known method can be applied to gaze detection using an image of the eyeball. For example, a gaze detection method based on the Purkinje image obtained by the reflection of the irradiated light from the cornea can be used.

[0182] More specifically, gaze detection processing is performed based on the pupil-corneal reflection method. Using the pupil-corneal reflection method, a gaze vector representing the orientation (rotation angle) of the eyeball is calculated based on the pupil image and Purkinje image contained in the captured image of the eyeball, thereby detecting the user's gaze.

[0183] The display device of this embodiment includes a photoelectric converter having a light-receiving element, and may control the display image of the display device based on the user's gaze information from the photoelectric converter.

[0184] Specifically, the display device determines a first field of view that the user is fixated on, and a second field of view other than the first field of view, based on gaze information. The first and second field of view may be determined by the control device of the display device, or they may be determined by an external control device and received by the display device. Within the display area of ​​the display device, the display resolution of the first field of view may be controlled to be higher than the display resolution of the second field of view. In other words, the resolution of the second field of view may be lower than that of the first field of view.

[0185] Furthermore, the display area has a first display area and a second display area different from the first display area, and a higher priority area may be determined from the first and second display areas based on gaze information. The first and second view areas may be determined by the control device of the display device, or they may be determined by an external control device and received. The resolution of the higher priority area may be controlled to be higher than the resolution of the areas other than the higher priority area. In other words, the resolution of areas with relatively lower priority may be set lower.

[0186] AI may be used to determine the first field of view area and the areas with high priority. The AI ​​may be a model configured to estimate the angle of line of sight and the distance to the target object at the end of the line of sight from the image of the eye, using the image of the eye and the direction the eye was actually looking in the image as training data. The AI ​​program may be installed in the display device, the photoelectric converter, or an external device. If installed in an external device, it will be transmitted to the display device via communication.

[0187] When display control is based on visual detection, this method is preferably applicable to smart glasses that further include a photoelectric converter for capturing images of the surrounding environment. The smart glasses can display the captured external information in real time.

[0188] [Eleventh Embodiment] The photoelectric conversion device and photoelectric conversion system described above may be applied to electronic devices such as so-called smartphones and tablets.

[0189] Figures 22A and 22B show an example of an electronic device 1500 equipped with a photoelectric converter. Figure 22A shows the front side of the electronic device 1500, and Figure 22B shows the back side of the electronic device 1500.

[0190] As shown in Figure 22A, a display 1510 for displaying images is positioned in the center of the surface of the electronic device 1500. Along the upper edge of the surface of the electronic device 1500, front cameras 1521 and 1522 using photoelectric converters, an IR light source 1530 that emits infrared light, and a visible light source 1540 that emits visible light are positioned.

[0191] Furthermore, as shown in Figure 22B, rear cameras 1551 and 1552, which use photoelectric converters, an IR light source 1560 that emits infrared light, and a visible light source 1570 that emits visible light are arranged along the upper edge of the back of the electronic device 1500.

[0192] In the electronic device 1500 configured in this way, by applying the photoelectric converter described above, it is possible to capture, for example, higher-quality images. The photoelectric converter can also be applied to other electronic devices such as infrared sensors, distance measuring sensors using active infrared light sources, security cameras, and personal or biometric authentication cameras. This can improve the accuracy and performance of these electronic devices.

[0193] [Twelfth Embodiment] Figure 23 is a block diagram of the X-ray CT apparatus in this embodiment. As described above, the photoelectric converters in the first to fifth embodiments are applicable to the detector of the X-ray CT apparatus. The X-ray CT apparatus 30 in this embodiment has an X-ray generating unit 310, a wedge 311, a collimator 312, an X-ray detection unit 320, a top plate 330, and a rotating frame 340. Furthermore, the X-ray CT apparatus 30 includes a high-voltage generator 350, a data acquisition device (DAS: Data Acquisition System) 351, a signal processing unit 352, a display unit 353, and a control unit 354.

[0194] The X-ray generating unit 310 is composed of, for example, a vacuum tube that generates X-rays. The vacuum tube of the X-ray generating unit 310 is supplied with high voltage and filament current from the high voltage generator 350. X-rays are generated when thermionic electrons are irradiated from the cathode (filament) to the anode (target).

[0195] The wedge 311 is a filter that adjusts the amount of X-rays irradiated from the X-ray generator 310. The wedge 311 attenuates the amount of X-rays so that the X-rays irradiated from the X-ray generator 310 to the subject have a predetermined distribution. The collimator 312 is made up of a lead plate or the like that narrows the irradiation range of the X-rays that have passed through the wedge 311. The X-rays generated in the X-ray generator 310 are formed into a cone beam shape via the collimator 312 and irradiated onto the subject on the top plate 330.

[0196] The X-ray detection unit 320 is configured using the semiconductor device described in the second and third embodiments above. The X-ray detection unit 320 detects the X-rays that have passed through the subject from the X-ray generation unit 310 and outputs a signal corresponding to the X-ray amount as DAS351.

[0197] The rotating frame 340 is annular in shape and is configured to be rotatable. Inside the rotating frame 340, the X-ray generating unit 310 (wedge 311, collimator 312) and the X-ray detection unit 320 are arranged facing each other. The X-ray generating unit 310 and the X-ray detection unit 320 are rotatable together with the rotating frame 340.

[0198] The high-voltage generator 350 includes a boost circuit and outputs a high voltage to the X-ray generation unit 310. The DAS 351 includes an amplification circuit and an A / D conversion circuit and outputs the signal from the X-ray detection unit 320 as digital data to the signal processing unit 352.

[0199] The signal processing unit 352 includes a CPU (Central Processing Unit), ROM (Read Only Memory), and RAM (Random Access Memory), and is capable of performing image processing on digital data. The display unit 353 includes a flat-panel display device, and is capable of displaying X-ray images. The control unit 354 includes a CPU, ROM, RAM, etc., and controls the operation of the entire X-ray CT apparatus 30.

[0200] [13th Embodiment] The photoelectric conversion system and mobile body of this embodiment will be described with reference to Figures 24A and 24B. Figures 24A and 24B are diagrams showing the configuration of the photoelectric conversion system and mobile body of this embodiment.

[0201] Figure 24A shows an example of a photoelectric conversion system related to an in-vehicle camera. The photoelectric conversion system 2300 has an imaging device 2310. The imaging device 2310 is a photoelectric conversion device as described in any of the embodiments above. The photoelectric conversion system 2300 has an image processing unit 2312 that performs image processing on a plurality of image data acquired by the imaging device 2310. The photoelectric conversion system 2300 also has a parallax acquisition unit 2314 that calculates parallax (phase difference of parallax images) from a plurality of image data acquired by the photoelectric conversion system 2300. Furthermore, the photoelectric conversion system 2300 has a distance acquisition unit 2316 that calculates the distance to an object based on the calculated parallax, and a collision determination unit 2318 that determines whether or not there is a possibility of collision based on the calculated distance. Here, the parallax acquisition unit 2314 and the distance acquisition unit 2316 are examples of distance information acquisition means that acquire distance information to an object. In other words, distance information may be acquired not only by phase difference but also by ToF (Time Of Flight) technology. The collision determination unit 2318 may use any of this distance information to determine the possibility of a collision. The distance information acquisition means may be implemented by specially designed hardware or by a software module. It may also be implemented by FPGA (Field Programmable Gate Array) or ASIC (Application Specific Integrated Circuit), or by a combination thereof.

[0202] The photoelectric conversion system 2300 is connected to the vehicle information acquisition device 2320 and can acquire vehicle information such as vehicle speed, yaw rate, and steering angle. The photoelectric conversion system 2300 is also connected to the control ECU 2330, which is a control device (control unit) that outputs a control signal to generate braking force on the vehicle based on the judgment result of the collision judgment unit 2318. The photoelectric conversion system 2300 is also connected to the warning device 2340, which issues a warning to the driver based on the judgment result of the collision judgment unit 2318. For example, if the collision judgment unit 2318 determines that there is a high probability of collision, the control ECU 2330 performs vehicle control to avoid a collision or mitigate damage by applying the brakes, releasing the accelerator, or suppressing engine output. The warning device 2340 warns the user by sounding an alarm, displaying warning information on a screen such as a car navigation system, or vibrating the seat belt or steering wheel.

[0203] In this embodiment, the photoelectric conversion system 2300 images the area around the vehicle, for example, in front of or behind it. Figure 24B shows the photoelectric conversion system when imaging the area in front of the vehicle (imaging range 2350). The vehicle information acquisition device 2320 sends instructions to the photoelectric conversion system 2300 or the imaging device 2310. This configuration can further improve the accuracy of distance measurement.

[0204] The above example illustrates control to prevent collisions with other vehicles, but it can also be applied to control systems that automatically follow other vehicles or automatically prevent vehicles from straying from their lanes. Furthermore, the photoelectric conversion system can be applied not only to vehicles such as the vehicle itself, but also to mobile objects (mobile devices) such as ships, aircraft, or industrial robots. In addition, it can be applied not only to mobile objects but also to a wide range of devices that utilize object recognition, such as intelligent transportation systems (ITS).

[0205] The embodiments described above are merely examples of how the present invention can be implemented, and the technical scope of the present invention should not be interpreted as being limited by them. In other words, the present invention can be implemented in various ways without departing from its technical concept or its main features.

[0206] The present invention is not limited to the embodiments described above, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are attached to make the scope of the invention public.

[0207] This application claims priority based on Japanese Patent Application No. 2024-218474, filed on December 13, 2024, and all of its contents are incorporated herein by reference.

[0208] 201 Photodiode 202 Recharge circuit 203 Waveform shaping circuit 204 Gating circuit 206 Memory 210 Mask circuit 113 Output line

Claims

1. A photoelectric converter comprising: a photodiode that performs avalanche multiplication; a recharge circuit to which a periodic first control signal is input and which controls the recharge operation of the photodiode; at least two gating circuits connected to the photodiode and which control whether or not to output the output signal of the photodiode; and at least one mask circuit to which the output from at least one of the at least two gating circuits and the first control signal are input, wherein the output of the mask circuit is input to one of the at least two gating circuits, and the at least two gating circuits include a first gating circuit and a second gating circuit, and the period during which the recharge circuit is ON due to the first control signal overlaps with the period during which a first level second control signal is input to the first gating circuit and the period during which a second level third control signal is input to the second gating circuit, and the period during which the first level second control signal is input to the first gating circuit and the period during which the second level third control signal is input to the second gating circuit are separated.

2. The photoelectric converter according to claim 1, characterized in that the periods of the first control signal, the second control signal, and the third control signal are the same.

3. The photoelectric conversion device according to claim 1 or 2, characterized in that the output from the mask circuit is input to the second gating circuit, and the period during which the second level third control signal is input to the second gating circuit is longer than the period during which the first level second control signal is input to the first gating circuit.

4. The photoelectric converter according to any one of claims 1 to 3, characterized in that the number of times the recharge circuit is turned on by the first control signal in one cycle is less than the number of the at least two gating circuits.

5. The photoelectric converter according to any one of claims 1 to 4, characterized in that the number of at least two gating circuits is M (where M is a natural number of 2 or more), the number of mask circuits is N (where N is a natural number and N < M), and the number of times the recharge circuit is turned on by the first control signal in one period is O (where O is a natural number and O < M).

6. The photoelectric conversion device according to any one of claims 1 to 5, characterized in that the value based on the signal output from the second gating circuit may be smaller than the value based on the signal output from the first gating circuit.

7. The photoelectric conversion device according to any one of claims 1 to 6, characterized in that the mask circuit is composed of a circuit including an SR latch.

8. The photoelectric converter according to any one of claims 1 to 7, further comprising a first memory into which the output of the first gating circuit is input, and a second memory into which the output of the second gating circuit is input.

9. The photoelectric converter according to claim 8, characterized in that it has a first memory, a second memory, and a selection circuit connected to an output line.

10. The photoelectric converter according to claim 8 or 9, characterized in that a counting circuit for counting the output from the first memory is connected between the first memory and the output line.

11. The photoelectric conversion device according to any one of claims 1 to 10, characterized in that the output of the photodiode transitions from a third level to a fourth level in response to the incidence of a photon onto the photodiode, and the output of the first gating circuit transitions from a fifth level signal to a sixth level signal when a fourth level signal is input to the first gating circuit from the photodiode and a second control signal of the first level is input to the first gating circuit.

12. A photoelectric conversion system characterized by comprising: a photoelectric conversion device according to any one of claims 1 to 11; and a signal processing unit that generates an image using a signal output by the photoelectric conversion device.

13. A mobile body comprising a photoelectric converter according to any one of claims 1 to 11, characterized in that it has a control unit that controls the movement of the mobile body using signals output by the photoelectric converter.

Images