Photoelectric conversion device and apparatus including photoelectric conversion device

Abstract

The present application provides a photoelectric conversion device and an apparatus including the same. The photoelectric conversion device includes an avalanche photodiode, an output holding circuit configured to hold a light-receiving signal based on an output of the avalanche photodiode, a first logic circuit input with the light-receiving signal held in the output holding circuit and a first reference signal indicating a first weight amount, and a first accumulation circuit configured to hold a first count value obtained by accumulating an output of the first logic circuit. The first accumulation circuit resets the first count value for each first period. The first period is divided into a plurality of second periods. The first weight amount changes for each second period. The output holding circuit resets the light-receiving signal for each second period.

Description

Technical Field

[0001] This disclosure relates to photoelectric conversion devices. Background Technology

[0002] In "Time-Domain Correlation Imaging and Its Applications" (The transactions of the Institute of Electrical Engineers of Japan, a publication of Sensors and Micromachines Society, Volume 129, No. 5, pp. 129-137, May 1, 2009), Shigeru Ando and Kaiyo Rai disclose a time-correlation image sensor. The pixel circuit of the time-correlation image sensor in "Time-Domain Correlation Imaging and Its Applications" includes photodiodes that generate photocurrent and multiple capacitors, each storing photocarriers. Therefore, the time-correlation image sensor in "Time-Domain Correlation Imaging and Its Applications" can realize time-correlation imaging.

[0003] In the time-correlated image sensors illustrated in "Time-Domain Correlated Imaging and Its Applications", signal acquisition under various illumination conditions is sometimes required. Summary of the Invention

[0004] The present disclosure aims to provide a photoelectric conversion device capable of acquiring signals under various illuminance conditions.

[0005] According to one aspect of this disclosure, a photoelectric conversion device is provided, comprising: an avalanche photodiode; an output holding circuit configured to hold a light-receiving signal based on the output of the avalanche photodiode; a first logic circuit wherein the light-receiving signal held in the output holding circuit and a first reference signal indicating a first weight are input to the first logic circuit; and a first accumulation circuit configured to hold a first count value obtained by accumulating the output of the first logic circuit. The first accumulation circuit resets the first count value for each first time period. The first time period is divided into a plurality of second time periods. The first weight is changed for each second time period. The output holding circuit resets the light-receiving signal for each second time period.

[0006] According to one aspect of this disclosure, a photoelectric conversion device is provided, comprising: an avalanche photodiode; a memory configured to hold two or more two-bit light quantity values ​​generated by a light-receiving signal based on the output of the avalanche photodiode; a conversion unit wherein the light quantity values ​​held in the memory and a first reference signal indicating a first weight are input to the conversion unit, the conversion unit outputting a conversion signal by calculation based on the light quantity values ​​and the first weight; and a first accumulation circuit configured to hold a first count value obtained by accumulating the conversion signal. A first time period for the first accumulation circuit to accumulate the first count value is divided into a plurality of second time periods. The first weight changes for each second time period. The light quantity values ​​are generated based on the light-receiving signal input in one of the second time periods.

[0007] The features of this disclosure will become apparent from the following description of embodiments with reference to the accompanying drawings. The following description of the embodiments is illustrated by way of example. Attached Figure Description

[0008] Figure 1 This is a schematic diagram showing the overall configuration of the photoelectric conversion device according to the first embodiment.

[0009] Figure 2 This is a schematic block diagram illustrating an example configuration of a sensor substrate according to the first embodiment.

[0010] Figure 3 This is a schematic block diagram illustrating an example configuration of a circuit board according to the first embodiment.

[0011] Figure 4 This is a diagram illustrating the relationship between the main frame time period and the subframe time period according to the first embodiment, as well as the temporal variation of the weight.

[0012] Figure 5 This is a schematic block diagram illustrating an example configuration of a pixel of the photoelectric conversion unit and the pixel signal processing unit according to the first embodiment.

[0013] Figure 6 This is a diagram illustrating the operation of the avalanche photodiode according to the first embodiment.

[0014] Figure 7 This is a diagram illustrating the pixel configuration according to the first embodiment.

[0015] Figure 8 This is a timing diagram illustrating the pixel driving method according to the first embodiment.

[0016] Figure 9 This is a timing diagram illustrating the pixel driving method according to the first embodiment.

[0017] Figure 10 This is a timing diagram illustrating the pixel driving method according to the first embodiment.

[0018] Figure 11 This is a diagram illustrating the pixel configuration according to the second embodiment.

[0019] Figure 12 This is a timing diagram illustrating the pixel driving method according to the second embodiment.

[0020] Figure 13 This is a diagram illustrating the pixel configuration according to a third embodiment.

[0021] Figure 14 This is a diagram illustrating the pixel configuration according to the fourth embodiment.

[0022] Figure 15 This is a timing diagram illustrating the pixel driving method according to the fourth embodiment.

[0023] Figure 16 This is a block diagram illustrating a schematic configuration of the device according to the fifth embodiment.

[0024] Figure 17A and Figure 17B This is a block diagram illustrating a schematic configuration of the device according to the sixth embodiment. Detailed Implementation

[0025] In the following description, embodiments will be illustrated with reference to the accompanying drawings. In the embodiments described below, the camera device will be primarily described as an example of a photoelectric conversion device. However, the photoelectric conversion devices to which the techniques of the various embodiments are applicable are not limited to camera devices and can be other devices. For example, the techniques of the various embodiments can also be applied to ranging devices (e.g., focus detection devices or devices for measuring distances using time-of-flight (ToF)) and photometers (devices for measuring the amount of incident light).

[0026] Note that the conductivity types of the transistors described in the embodiments below are merely examples, and the conductivity types of the transistors are not limited to those described in the embodiments. The conductivity types described in the embodiments can be appropriately changed, and the potentials of the gate, source, or drain of the transistor can be appropriately changed accordingly. For example, in a transistor operating as a switch, when the conductivity type changes, the low and high levels of the potential supplied to the gate are reversed relative to those described in the embodiments.

[0027] In the following embodiments, connections between components in a circuit may be described. In this case, the components of interest are considered connected to each other even when they are located between components of interest, unless otherwise stated. For example, component A is connected to one node of a capacitor C with multiple nodes, and component B is connected to another node of capacitor C. Even in this case, components A and B are considered connected to each other, unless otherwise stated. First Embodiment

[0028] Before describing the photoelectric conversion device of this embodiment, the principles of time-correlated image sensors and event-based sensors will be schematically described.

[0029] The time-correlated image sensor includes a photodiode and a configuration for acquiring the signal output from the photodiode in multiple parts. The signal for each pixel used to generate the image is represented by the following expression (1).

[0030] f(x, y, t) is the brightness of pixel (x, y) at time t. Additionally, v is the velocity of pixel (x, y) (the time derivative of pixel (x, y)). It is the nabla operator (vector differential operator).

[0031] The exposure time in acquiring a single frame of an image is defined as T. Image g n (x, y) is represented by the following expression (2).

[0032] As shown in expression (2), the image g n (x, y) is obtained by multiplying the brightness f(x, y, t) by the complex number e. -inΔwt The reference signal is represented and the result is obtained by integrating the result over a one-frame time interval. Image g is captured. n (x, y) satisfies the following expression (3).

[0033] The second term on the left-hand side of expression (3) indicates the boundary value of the integral. Since expression (3) consists of multiple expressions based on different values ​​of n, it forms a simultaneous equation. Therefore, for example, the boundary value of the integral can be eliminated by solving the simultaneous equation using two images g0(x,y) and g1(x,y). The time-correlated image sensor can output an intensity image g0(x,y) consisting only of the real part and a complex correlation image g1(x,y). nThe real and imaginary parts of (x, y) (in the following text, the complex correlation image is also called the time-correlated signal). Therefore, by substituting the output signal of the time-correlated image sensor into the simultaneous equations of expression (3) and solving them, the velocity v (i.e., optical flow) in each pixel (x, y) can be obtained.

[0034] In the signal processing of a time-correlated image sensor, as shown in expression (2), it is necessary to calculate the integral over a frame time period. Therefore, the output timing of the correlated image is limited to a frame time period. In a time-correlated image sensor, the period of the reference signal and the period of the shutter opening time period are made consistent with each other. Therefore, the correlated image is output at a frequency corresponding to the period of the shutter opening time period.

[0035] Event-based sensors will now be outlined. Event-based sensors detect changes in brightness within their field of view and output an event signal whenever a change in brightness is detected. An event-based sensor may include, for example, multiple pixels arranged in a matrix. That is, the event signal is a signal associated with an event, and the event is a change in the brightness of a pixel. As an example, the event signal includes the time the event was detected, the position of the pixel that detected the event, and the change in the pixel value. The time of event detection can be measured based on the time indicated by the internal clock of the event-based sensor (event camera time).

[0036] Note that the time baseline for detecting an event can be reset as needed. Changes in pixel values ​​can be, for example, changes in brightness. These changes can be the magnitude of the change itself, or they can be information indicating whether the brightness change is positive or negative.

[0037] Event-based sensors output event signals when a brightness change occurs and do not output event signals when no brightness change occurs. In other words, event-based sensors output event signals asynchronously. Note that asynchronous output means outputting signals independently of time, on a pixel-by-pixel basis.

[0038] The operation of an event-based sensor is represented by the following expression (4).

[0039] In expression (4), Y(x, y, t) is the image at time t. Time t0 is the measurement start time. The image Y(x, y, t0) is the initial image stored at time t0. Typically, the image Y(x, y, t0) can be zero. ΔY is the threshold (absolute value of brightness change) for the event to occur. p(x, y, s i ) is the i-th event signal occurring in pixel (x, y), and p(x, y, s) is the event signal detected at the time of event detection. i The value of δ(ss) is either 1 or -1, depending on whether the brightness change is positive or negative.i ) is the Dirac delta function.

[0040] Event-based sensors can be configured to output time-correlated signals, such as time-correlated image sensors. When time t is the end time of a frame period, the output signal of a time-correlated image sensor can be represented by the following expressions (5) to (7) using the angular velocity ω (ω=2π / T).

[0041] In a time-correlated image sensor, charge based on the current output from the photodiode accumulates in a capacitor. The accumulated charge corresponds to the brightness. On the other hand, in an event-based sensor, a signal is output by quantizing the change in the current output from the photodiode. Therefore, in an event-based sensor, the output from the photodiode at time s can be divided into a constant term f(x, y, tT) with a constant value during the measurement period and a variable term δf(x, y, s) corresponding to the difference relative to the constant term. Therefore, f(x, y, s) is represented by the following expression (8).

[0042] Considering the properties of the reference signal, it satisfies the following expressions (9) and (10).

[0043] Using the relationship between expressions (9) and (10), expressions (5) through (7) can be rewritten as expressions using constant terms and variable terms. As a result, the following expressions (11) through (13) are obtained.

[0044] In an event-based sensor, as shown in the following expression (14), the current output by the photodiode of the time-correlated image sensor is converted into an event signal of the event-based sensor.

[0045] As a result, expressions (11) to (13) can be converted into the following expressions (15) to (17).

[0046] As shown in expressions (15) to (17), a time-dependent signal can be output by using an event signal generated during the period (period T) of signal acquisition.

[0047] Next, the configuration of the photoelectric conversion device according to this embodiment will be described. Figure 1This is a schematic diagram showing the overall configuration of the photoelectric conversion device 100 according to this embodiment. The photoelectric conversion device 100 includes a stacked sensor substrate 11 (first substrate) and a circuit substrate 21 (second substrate). The sensor substrate 11 and the circuit substrate 21 are electrically connected to each other. The sensor substrate 11 has a pixel region 12 on which a plurality of pixel circuits 101 are arranged to form a plurality of rows and a plurality of columns. The circuit substrate 21 includes a first circuit region 22 on which a plurality of pixel signal processing units 103 are arranged to form a plurality of rows and a plurality of columns, and a second circuit region 23 disposed outside the first circuit region 22. The second circuit region 23 may include circuitry for controlling the plurality of pixel signal processing units 103. The sensor substrate 11 has a light incident surface for receiving incident light and a connection surface opposite to the light incident surface. The sensor substrate 11 is connected to the circuit substrate 21 on the connection surface side. That is, the photoelectric conversion device 100 is a so-called back-illuminated type.

[0048] In this specification, the term "plan view" refers to a view taken from a direction perpendicular to the surface opposite the light incident surface. A section indicates a surface in a direction perpendicular to the surface opposite the light incident surface of the sensor substrate 11. Although the light incident surface may be rough when viewed microscopically, in this case, the plan view is defined with reference to the light incident surface when viewed macroscopically.

[0049] In the following description, sensor substrate 11 and circuit substrate 21 are diced chips, but sensor substrate 11 and circuit substrate 21 are not limited to chips. For example, sensor substrate 11 and circuit substrate 21 can be wafers. When sensor substrate 11 and circuit substrate 21 are diced chips, the photoelectric conversion device 100 can be manufactured by dicing after stacking in a wafer state, or by stacking after dicing.

[0050] Figure 2 This is a schematic block diagram illustrating an example arrangement of the sensor substrate 11. In the pixel region 12, a plurality of pixel circuits 101 are arranged to form a plurality of rows and columns. Each pixel circuit in the plurality of pixel circuits 101 includes a photoelectric conversion unit 102 in the substrate, the photoelectric conversion unit 102 including an avalanche photodiode (hereinafter referred to as an APD) as a photoelectric conversion element.

[0051] In the charge pairs generated in an APD, the conductivity type corresponding to the charge used as the signal charge is called the first conductivity type. The first conductivity type refers to a conductivity type where the majority carriers are charges with the same polarity as the signal charge. Furthermore, the conductivity type opposite to the first conductivity type (i.e., a conductivity type where the majority carriers are charges with a polarity different from the signal charge) is called the second conductivity type. In the APD described below, the anode of the APD is set to a fixed potential, and the signal is extracted from the cathode of the APD. Therefore, the semiconductor region of the first conductivity type is an N-type semiconductor region, and the semiconductor region of the second conductivity type is a P-type semiconductor region. Note that the cathode of the APD can also have a fixed potential, and the signal can be extracted from the anode of the APD. In this case, the semiconductor region of the first conductivity type is a P-type semiconductor region, and the semiconductor region of the second conductivity type is an N-type semiconductor region. Although the case where one node of the APD is set to a fixed potential is described below, the potentials of both nodes can vary.

[0052] Figure 3 This is a schematic block diagram showing an example configuration of the circuit board 21. The circuit board 21 has a first circuit region 22 on which a plurality of pixel signal processing units 103 are arranged to form a plurality of rows and a plurality of columns.

[0053] The circuit board 21 includes a vertical scanning circuit 110, a horizontal scanning circuit 111, a reading circuit 112, a pixel output signal line 113, an output circuit 114, a control signal generation unit 115, and a weight control unit 116. Figure 2 The multiple photoelectric conversion units 102 shown and Figure 3 The multiple pixel signal processing units 103 shown are electrically connected to each other via connection wiring provided for each pixel circuit 101.

[0054] The control signal generation unit 115 is a control circuit that generates control signals for driving the vertical scanning circuit 110, the horizontal scanning circuit 111, the readout circuit 112, and the weight control unit 116, and supplies the control signals to these units. As a result, the control signal generation unit 115 controls the driving timing of each unit, etc.

[0055] The vertical scanning circuit 110 supplies control signals to each of the plurality of pixel signal processing units 103 based on control signals supplied from the control signal generation unit 115. The vertical scanning circuit 110 supplies control signals for each row to the pixel signal processing unit 103 via drive lines provided for each row of the first circuit region 22. As will be described later, multiple drive lines may be provided for each row. Logic circuitry such as shift registers or address decoders may be used in the vertical scanning circuit 110. Therefore, the vertical scanning circuit 110 selects the row from which signals are to be output from the pixel signal processing unit 103.

[0056] The signal output from the photoelectric conversion unit 102 of the pixel circuit 101 is processed by the pixel signal processing unit 103. The pixel signal processing unit 103 acquires and holds a digital signal based on the pulses output from the APD included in the photoelectric conversion unit 102.

[0057] The weight control unit 116 controls the weighting coefficients (weights) given to the output signal from the APD in the pixel signal processing unit 103. The weight control unit 116 supplies reference signals including information related to the weights to each of the multiple pixel signal processing units 103.

[0058] The horizontal scanning circuit 111 supplies control signals to the readout circuit 112 based on control signals supplied from the control signal generation unit 115. Pixel signal processing units 103 are connected to the readout circuit 112 via pixel output signal lines 113 provided for each column of the first circuit region 22. Pixel output signal lines 113 in a column are shared by multiple pixel signal processing units 103 in the corresponding column. Each pixel output signal line 113 includes multiple wirings and has at least the function of outputting digital signals from the pixel signal processing units 103 to the readout circuit 112 and supplying control signals for selecting the column for output signals to the pixel signal processing units 103. The readout circuit 112 outputs signals to the processing device 400 via the output circuit 114 based on the control signals supplied from the control signal generation unit 115.

[0059] The processing device 400 performs signal processing on the signal output from the photoelectric conversion device 100. The processing device 400 can use the signal output from the photoelectric conversion device 100 to perform time-related imaging processing, such as optical flow calculation. This processing can be based, for example, on the expressions (1) to (17) above. The processing device 400 can be installed inside the photoelectric conversion device 100 or in a device in which the photoelectric conversion device 100 is installed.

[0060] The arrangement of photoelectric conversion units 102 in pixel region 12 can be one-dimensional. Furthermore, the function of pixel signal processing units 103 does not necessarily have to be provided in each of the pixel circuits 101 individually. For example, one pixel signal processing unit 103 can be shared by multiple pixel circuits 101. In this case, the pixel signal processing unit 103 sequentially processes the signals output from the photoelectric conversion units 102, thereby providing signal processing functionality to each pixel circuit 101.

[0061] like Figure 2 and Figure 3 As shown in the plan view, a first circuit region 22 having multiple pixel signal processing units 103 is arranged in a region overlapping with the pixel region 12. In the plan view, the vertical scanning circuit 110, horizontal scanning circuit 111, readout circuit 112, output circuit 114, control signal generation unit 115, and weight control unit 116 are arranged to overlap with the region between the edge of the sensor substrate 11 and the edge of the pixel region 12. In other words, the sensor substrate 11 includes the pixel region 12 and non-pixel regions arranged around the pixel region 12. In the circuit substrate 21, a second circuit region 23 having the vertical scanning circuit 110, horizontal scanning circuit 111, readout circuit 112, output circuit 114, control signal generation unit 115, and weight control unit 116 is arranged in a region overlapping with the non-pixel regions in the plan view.

[0062] Note that the arrangement of the pixel output signal line 113, the readout circuit 112, and the output circuit 114 are not limited to the following. Figure 3 The arrangement is shown. For example, the pixel output signal line 113 can extend in the row direction and can be shared by multiple pixel signal processing units 103 in the corresponding row. Readout circuits 112 can be configured to connect to the pixel output signal lines 113 of each row.

[0063] Figure 4 This is a graph illustrating the relationship between the main frame time period and the subframe time period according to the first embodiment, as well as the temporal variation of the weights. Figure 4 In the graph shown, the horizontal axis indicates time, and the vertical axis indicates the weight set by the weight control unit 116. For example... Figure 4As shown, the main frame period (first period), which serves as the exposure time for generating one frame, is divided into multiple sub-frame periods (second period). The weight control unit 116 sets weights such that the weights change each time a sub-frame period passes. The weights can be set based on a periodic function, in which time is a variable and the main frame period is a period. In other words, the periodic function has a different phase for each sub-frame period. The periodic function used to set the weights can be a sine function. By weighting according to the weights based on the sine function, a signal corresponding to expression (17) can be generated. The periodic function used to set the weights can be a cosine function. By weighting according to the weights based on the cosine function, a signal corresponding to expression (16) can be generated. As described later, a sub-frame period can be further divided into multiple micro-frame periods (third period).

[0064] Figure 5 This is a schematic block diagram illustrating an example configuration of a pixel of the photoelectric conversion unit 102 and the pixel signal processing unit 103 according to this embodiment. Figure 5 A more specific configuration example is schematically shown, including the connection relationship between the photoelectric conversion unit 102 arranged in the sensor substrate 11 and the pixel signal processing unit 103 arranged in the circuit board 21. Figure 5 middle, Figure 3 The drive lines between the vertical scanning circuit 110 and the pixel signal processing unit 103 are shown as drive lines 213 and 214.

[0065] The photoelectric conversion unit 102 includes an APD 201. The pixel signal processing unit 103 includes a quenching element 202, a waveform shaping unit 210, a counter circuit 211, and a selection circuit 212. The pixel signal processing unit 103 may include at least one of the waveform shaping unit 210, the counter circuit 211, and the selection circuit 212.

[0066] APD 201 generates charge pairs corresponding to incident light through photoelectric conversion. A voltage VL (first voltage) is supplied to the anode of APD 201. The cathode of APD 201 is connected to the first terminal of quenching element 202 and the input terminal of waveform shaping unit 210. A voltage VH (second voltage) higher than the voltage VL supplied to the anode is supplied to the cathode of APD 201. As a result, a reverse bias voltage that enables APD 201 to perform avalanche multiplication operation is supplied to the anode and cathode of APD 201. In APD 201 with the reverse bias voltage supplied, when incident light generates charge, the charge causes avalanche multiplication and generates an avalanche current.

[0067] The operating modes when a reverse bias voltage is supplied to the APD 201 include Geiger mode and linear mode. Geiger mode is the mode in which the potential difference between the anode and cathode is higher than the breakdown voltage, and linear mode is the mode in which the potential difference between the anode and cathode is close to or lower than the breakdown voltage.

[0068] An APD operating in Geiger mode is called a single-photon avalanche diode (SPAD). In this case, for example, the voltage VL (first voltage) is -30V and the voltage VH (second voltage) is 1V. The APD 201 can operate in either linear or Geiger mode. In the case of a SPAD, the potential difference becomes greater than that of an APD in linear mode, and the avalanche multiplication effect becomes significant, making the use of a SPAD possible.

[0069] When the signal is amplified by avalanche multiplication, the quenching element 202 acts as a load circuit (quenching circuit). The quenching element 202 suppresses the voltage supplied to the APD 201 and suppresses avalanche multiplication (quenching operation). Furthermore, the quenching element 202 returns the voltage supplied to the APD 201 to voltage VH by allowing a current corresponding to the voltage drop caused by the quenching operation to flow (recharge operation). The quenching element 202 can be, for example, a transistor.

[0070] Waveform shaping unit 210 shapes the potential change of the cathode of APD 201 obtained during photon detection and outputs a pulse signal. For example, an inverter circuit can be used as waveform shaping unit 210. Although Figure 5 An example of an inverter being used as waveform shaping unit 210 is shown, but waveform shaping unit 210 can be a circuit in which multiple inverters are connected in series, or it can be other circuits that have waveform shaping effects.

[0071] The counter circuit 211 counts the pulse signals output from the waveform shaping unit 210 and holds the digital signal indicating the count value. When a control signal is supplied from the vertical scanning circuit 110 via the drive line 213, the counter circuit 211 resets the signal held therein.

[0072] pass Figure 5 The drive line 214 shown, from Figure 3 The vertical scanning circuit 110 shown supplies a control signal to the selection circuit 212. In response to this control signal, the selection circuit 212 switches between electrical connection and non-connection of the counter circuit 211 and the pixel output signal line 113. The selection circuit 212 includes, for example, a buffer circuit for outputting a signal corresponding to the value held in the counter circuit 211.

[0073] exist Figure 5In the example, selection circuit 212 switches between electrical connection and non-connection of counter circuit 211 and pixel output signal line 113; however, the method of controlling the signal output to pixel output signal line 113 is not limited to this. For example, switches such as transistors can be arranged at nodes such as between quenching element 202 and APD 201 or between photoelectric conversion unit 102 and pixel signal processing unit 103, and the signal output to pixel output signal line 113 can be controlled by switching between electrical connection and non-connection. Alternatively, the signal output to pixel output signal line 113 can be controlled by changing the value of voltage VH or voltage VL supplied to photoelectric conversion unit 102 using switches such as transistors.

[0074] Figure 6 This is a diagram illustrating the operation of the APD 201 according to this embodiment. Figure 6 (A) is shown Figure 5 The diagram shows APD201, quenching element 202, and waveform shaping unit 210. (See diagram for reference.) Figure 6 As shown in (A), the connection node between the input terminals of APD 201, quenching element 202, and waveform shaping unit 210 is referred to as node A. Furthermore, as... Figure 6 As shown in (A), the output side of the waveform shaping unit 210 is referred to as node B.

[0075] Figure 6 (B) is shown Figure 6 The graph shows the time-varying potential of node A in (A). Figure 6 (C) is shown Figure 6 A graph showing the time-varying potential of node B in (A). During the time interval from time t0 to time t1, voltage VH-VL is applied to... Figure 6 In (A), APD 201 is used. When a photon is incident on APD 201 at time t1, avalanche multiplication occurs in APD 201. As a result, an avalanche current flows through quenching element 202, and the potential at node A drops. Thereafter, the potential drop increases further, and the voltage applied to APD 201 gradually decreases. Then, at time t2, avalanche multiplication in APD 201 stops. Therefore, the voltage level at node A does not drop below a certain constant value. Then, during the time interval from time t2 to time t3, a current compensating for the voltage drop flows from node VH to node A, and at time t3, node A stabilizes at its original potential.

[0076] In the above process, the potential of node B becomes high during the period when the potential of node A is below a certain threshold. In this way, the waveform of the potential drop of node A caused by the incident photon is shaped by the waveform shaping unit 210 and output as a pulse to node B.

[0077] Figure 7 This is a diagram illustrating the pixel configuration according to this embodiment. Figure 7 Showing more details Figure 5 The photoelectric conversion unit 102 and the pixel signal processing unit 103 are used in the process. In the following text, the element obtained by combining the photoelectric conversion unit 102 and the pixel signal processing unit 103 may be referred to as pixel 200. Figure 7 In, it has the same Figure 5 The components shown have the same function as those shown. Figure 5 The same reference numerals are used in the accompanying drawings, and the description of these elements may be omitted or simplified.

[0078] Pixel 200 includes an APD 201, a quenching element 202, a waveform shaping unit 210, a counter circuit 211, a NAND circuit 221, and a logic circuit 222. The quenching element 202 has a P-type MOS transistor 202a. The counter circuit 211 includes an output holding circuit 230, an AND circuit 241 (first logic circuit), and an accumulator circuit 251 (first accumulator circuit). The output holding circuit 230 is a circuit that holds the light-receiving signal based on the output of APD 201, and includes a selector circuit 231, a NOT circuit 232, and a trigger circuit 233.

[0079] Signal P_DECI_CLK is input to pixel 200 from weight control unit 116. Signals P_RCH_TRG, P_RCH_TRG2, P_RCH_TRG3 and P_RES are input to pixel 200 from vertical scan circuit 110.

[0080] The signal P_DECI_CLK is, for example, a pulse signal with a frequency of 1 MHz to 200 MHz, and indicates the weight (first weight) set by the weight control unit 116. As described above, the weight is determined by a periodic function of the main frame period being one cycle. The periodic function is, for example, a sine function or a cosine function. The weight control unit 116 generates the weight by discarding a portion of the pulse intervals from the clock signal input to the weight control unit 116. The number of pulses in a main frame period can be appropriately set according to the number of bits in the counter circuit 211. When the counter circuit 211 counts with 11 bits, the number of pulses in a main frame period is less than 2048.

[0081] The signal P_DECI_CLK is input to the first input terminal of NAND circuit 221, and the signal P_RCH_TRG is input to the second input terminal of NAND circuit 221. Signal P_RCH_TRG indicates the start of a subframe period. Signal P_RCH_TRG goes high at the start of the subframe period and then goes low. Then, signal P_RCH_TRG remains low until the end of the subframe. NAND circuit 221 outputs a signal obtained by inverting the logical product of signals P_DECI_CLK and P_RCH_TRG as signal PCLKB. The output terminal of NAND circuit 221 is connected to the gate of MOS transistor 202a. Voltage VH is supplied to the source of MOS transistor 202a. The drain of MOS transistor 202a is connected to the cathode of APD 201 and the input terminal of waveform shaping unit 210.

[0082] The PCLKB signal controls the timing of the recharge operation in APD 201. When both signals P_DECI_CLK and P_RCH_TRG go high, the PCLKB signal goes low. At this time, MOS transistor 202a is turned on, and a recharge operation is performed in APD 201. The recharge operation is performed once per subframe period.

[0083] The output terminal of waveform shaping unit 210 is connected to the first input terminal of selector circuit 231. The output terminal of selector circuit 231 is connected to the input terminal D of trigger circuit 233. Signal P_RCH_TRG2 is input to the reset terminal R of trigger circuit 233. Signal P_RCH_TRG2 controls the reset of the signal held in trigger circuit 233. Signal P_RCH_TRG3 is input to the clock input terminal of trigger circuit 233. Signal P_RCH_TRG3 controls the holding timing of the signal in trigger circuit 233. The output terminal Q of trigger circuit 233 is connected to the second input terminal of selector circuit 231, the input terminal of NOT circuit 232, and the second input terminal of AND circuit 241. The output terminal of NOT circuit 232 is connected to the control terminal of selector circuit 231.

[0084] When the output signal of the trigger circuit 233 is low, the selector circuit 231 selects the signal input to the first input terminal and outputs it. That is, the output signal of the waveform shaping unit 210 is input to the input terminal D of the trigger circuit 233. When the output signal of the trigger circuit 233 is high, the selector circuit 231 selects the signal input to the second input terminal and outputs it. That is, the output signal of the trigger circuit 233 is fed back and input to the input terminal D of the trigger circuit 233.

[0085] Logic circuit 222 outputs the logical product of the inverted values ​​of the input signals at the first and second input terminals. Signal P_DECI_CLK is input to the first input terminal of logic circuit 222, and signal P_RCH_TRG is input to the second input terminal. Logic circuit 222 outputs the logical product of the inverted values ​​of signals P_DECI_CLK and P_RCH_TRG as signal TCLK (first reference signal). The output terminal of logic circuit 222 is connected to the first input terminal of AND circuit 241. That is, signal TCLK is input to the first input terminal of AND circuit 241.

[0086] AND circuit 241 outputs the logical product of signal TCLK and the output signal of flip-flop circuit 233 to accumulator circuit 251. Accumulator circuit 251 counts the number of pulses by accumulating the pulses of the input signal. Accumulator circuit 251 holds the count value obtained thereby. In addition, signal P_RES is input to accumulator circuit 251. Signal P_RES goes high at the beginning of a main frame. As a result, the count value held in accumulator circuit 251 is reset.

[0087] Reference Figures 8 to 10 Describe the driving method for pixel 200. Figure 8 This is a timing diagram illustrating the driving method of pixel 200 according to this embodiment. Figure 8 The diagram schematically illustrates the configuration of the camera segment, main frame segment, and subframe segment, as well as the timing of each signal, the photon detection timing, and the transition of the count values.

[0088] The recording period is the operating period of the photoelectric conversion device 100, during which main frame recording is repeated. The recording period includes multiple main frame periods F1, F2, ... Each main frame period is an exposure period, and one frame is recorded for each main frame period. The signal P_RES goes high at the beginning of each main frame period and then goes low.

[0089] Each main frame time period is divided into multiple sub-frame time periods. Figure 8 The "Main Frame Period" indicates that the first main frame period F1 is divided into multiple sub-frame periods SF11, SF12, ... . Signals P_RCH_TRG and P_RCH_TRG2 go high at the beginning of each sub-frame period and then go low. The times at which signals P_RES, P_RCH_TRG, and P_RCH_TRG2 go low can be appropriately set within a range that does not affect operations such as signal counting.

[0090] Each subframe time period is divided into multiple microframe time periods. Figure 8The term "subframe segment" indicates that the first subframe segment SF11 is divided into multiple microframe segments MF11, MF12, ... . Figure 8 Within a "subframe segment," the lengths of multiple microframe segments MF11, MF12, ... are different from each other, but the lengths of multiple microframe segments MF11, MF12, ... are not necessarily different from each other. The lengths of multiple microframe segments MF11, MF12, ... can be the same. This also applies to microframe segments in other timing diagrams.

[0091] Figure 8 The pulse AP1 shown in "Photon Detection" indicates the timing of photon detection during subframe period SF11. Figure 8 In the example, photons were detected in the microframe period MF11.

[0092] The signal P_RCH_TRG3 goes high at the beginning of each microframe period and then goes low. Note that the time at which the signal P_RCH_TRG3 goes low can be appropriately set within a range that does not affect operations such as signal counting.

[0093] Figure 8 The "P_DECI_CLK" indicates multiple pulses of the signal P_DECI_CLK in microframe periods MF11 and MF12. In microframe period MF11, the signal P_DECI_CLK goes high at times t11, t12, t13, t14, ..., t1x, and then goes low. In microframe period MF12, the signal P_DECI_CLK goes high at times t21, t22, t23, t24, ..., t2x, and then goes low. The number of pulses or pulse density input in a microframe period is associated with the weight. The weight changes each time a subframe period passes.

[0094] Figure 8The "count value" indicates the time change of the count value (first count value) held in the accumulator circuit 251 during microframe periods MF11 and MF12. When the high-level signal P_RCH_TRG3 is input to the clock input terminal of the trigger circuit 233, the pulse generated by photon detection is held in the trigger circuit 233. The timing of the signal P_RCH_TRG3 going high is at the start time of the next microframe period MF12 after the photon-detected microframe period MF11. Therefore, the count value increases due to photon detection starting from the microframe period MF12 after the photon-detected microframe period MF11. That is, the count value does not change during the photon-detected microframe period MF11. Then, in the next microframe period MF12, the count value increases by 1 whenever the signal P_DECI_CLK goes high at times t21, t22, t23, t24, ..., t2x.

[0095] Figure 8 An example of setting the weights using a periodic function including the sine function sin(t) is shown. Since the weights are represented by the number of pulses, the function of the weights is set such that the range of the weights is non-negative. Here, it is assumed that the function of the weights is (sin(t)+1), and the range of the weights is zero to two. Furthermore, it is assumed that the bit width of the digital value indicating (sin(t)+1) is four bits. That is, the number of pulses in the signal P_DECI_CLK in a microframe period is zero to fifteen. In this case, assuming that the phase of the sine function is zero in the first subframe period SF11, then according to (sin(0)+1), the weight is 1. Since the number of pulses corresponding to this weight is eight, the count at time t2x of the microframe period MF12 is eight.

[0096] In the microframe period following microframe period MF12, the count value continues to increase. When signal P_RCH_TRG2 goes high at the start of the next subframe period SF12, trigger circuit 233 is reset and the increase of the count value stops. In this way, within a subframe period, accumulator circuit 251 counts pulses based on signal P_DECI_CLK from the microframe period following the microframe period after photon detection to the end of the subframe period.

[0097] When photons are detected within a subframe period, as illuminance increases, photons are detected in microframe periods closer to the beginning of the subframe period. In this case, the count value increases because the pulse counting based on the signal P_DECI_CLK continues for a longer period. Conversely, as illuminance decreases, photons are detected in microframe periods closer to the end of the subframe period. In this case, the count value decreases because the pulse counting based on the signal P_DECI_CLK continues for a shorter period. Therefore, it is possible to obtain a count value with a magnitude corresponding to the illuminance.

[0098] Figure 9 This is a timing diagram illustrating the driving method of pixel 200 according to this embodiment. Figure 8 This primarily illustrates the operations within subframe segment SF11. On the other hand, Figure 9 This mainly illustrates the operation of subframe period SF12 following subframe period SF11. (Omitted or simplified details will be omitted.) Figure 8 The description of common operations in the code.

[0099] Figure 9 The term "subframe period" indicates that the subframe period SF12 following subframe period SF11 is divided into multiple microframe periods MF21, MF22, ... Figure 9 The pulse AP2 shown in "Photon Detection" indicates the timing of photon detection during subframe period SF12. Figure 9 In the example, a photon is detected in microframe period MF21. Therefore, the count value begins to increase from the next microframe period MF22.

[0100] Figure 9 The "P_DECI_CLK" indicates multiple pulses of the signal P_DECI_CLK in microframe periods MF21 and MF22. In microframe period MF21, the signal P_DECI_CLK goes high at times t31, t32, t33, t34, t35, ..., t3y, and then goes low. In microframe period MF22, the signal P_DECI_CLK goes high at times t41, t42, t43, t44, ..., t4y, and then goes low.

[0101] During the microframe period MF21 in which photons are detected, the count value remains unchanged. Then, in the next microframe period MF22, the count value increases by 1 whenever the signal P_DECI_CLK goes high at times t41, t42, t43, t44, t45, ..., t4y.

[0102] The phase of the periodic function of the weights in subframe SF12 is different from the phase of the periodic function of the weights in subframe SF11. Therefore, Figure 9The number of pulses of signal P_DECI_CLK in microframe periods MF21 and MF22 shown are related to Figure 8 The number of pulses of signal P_DECI_CLK in microframe periods MF11 and MF12 shown is different.

[0103] For example, as in Figure 8 In this context, assume the weight is a function of (sin(t)+1) and the number of subframes is 50. In this case, the phase of the sine function in subframe time interval SF12 is 2π / 50. According to (sin(2π / 50)+1), the weight is approximately 1.115. Since the number of pulses corresponding to this weight is nine, the count at time t4y in microframe time interval MF22 is nine.

[0104] Figure 10 This is a timing diagram illustrating the driving method of pixel 200 according to this embodiment. Figure 8 and Figure 9 This primarily illustrates the operations within the main frame time slot F1. On the other hand, Figure 10 This primarily illustrates the operations within the main frame period F2, following main frame period F1. Omissions or simplifications will be noted. Figure 8 or Figure 9 The description of common operations in the code.

[0105] Figure 10 The term "main frame period" indicates that the main frame period F2 following the main frame period F1 is divided into multiple sub-frame periods SF21, SF22, ... Figure 10 The term "subframe period" indicates that the first subframe period SF21 in the main frame period F2 is divided into multiple microframe periods MF31, MF32, MF33, MF34, ...

[0106] Figure 10 The pulse AP3 shown in "Photon Detection" indicates the timing of photon detection during subframe segment SF21. Figure 10 In the example, a photon is detected in microframe period MF32. Therefore, the count value begins to increase from the next microframe period MF33. Furthermore, the count value continues to increase in the next microframe period MF34.

[0107] Figure 10The "P_DECI_CLK" indicates multiple pulses of the signal P_DECI_CLK in microframe periods MF32, MF33, and MF34. In microframe period MF32, the signal P_DECI_CLK goes high at times t51, t52, t53, t54, ..., t5x, and then goes low. In microframe period MF33, the signal P_DECI_CLK goes high at times t61, t62, t63, t64, ..., t6x, and then goes low. In microframe period MF34, the signal P_DECI_CLK goes high at times t71, t72, t73, t74, ..., t7x, and then goes low.

[0108] During the microframe period MF32 when photons are detected, the count value remains unchanged. Then, in the next microframe period MF33, the count value increases by 1 whenever the signal P_DECI_CLK goes high at times t61, t62, t63, t64, ..., t6x. Similarly, in the next microframe period MF34, the count value increases by 1 whenever the signal P_DECI_CLK goes high at times t71, t72, t73, t74, ..., t7x.

[0109] and Figure 8 Similarly, assume the weight is a function of (sin(t)+1), and the phase of the sine function in the first subframe time period SF21 is zero. According to (sin(0)+1), the weight is 1. Since the number of pulses corresponding to this weight is eight, the count at time t6x in microframe time period MF33 is eight. In addition, the count at time t7x in the next microframe time period MF34 is 16.

[0110] As described above, in this embodiment, a count value corresponding to the microframe in which photons are detected during a subframe period is generated. Typically, as the amount of incident light increases, photons are detected at earlier timing points, thus the generated count value has a value corresponding to the amount of incident light. Furthermore, in this embodiment, since the light-receiving signal held in the output hold circuit 230 is reset for each subframe period and photon detection is performed for each subframe period, saturation is unlikely. Therefore, in this embodiment, various light amounts can be handled. Therefore, according to this embodiment, a photoelectric conversion device capable of acquiring signals under various illuminance conditions is provided.

[0111] Furthermore, in the photoelectric conversion device of this embodiment, the output signal can be weighted based on a periodic function such as a trigonometric function. Therefore, the photoelectric conversion device of this embodiment is suitable for signal acquisition in time-correlated imaging, such as for calculating optical flow.

[0112] In the description of the operation of pixel 200, specific examples related to weighting are shown in some cases, such as expressions for functions of weights, bit widths of weights, and values ​​of weight weights. However, these examples are not particularly limited and can be changed appropriately. The number of subframe and microframe segments can also be set appropriately. Second Embodiment

[0113] In this embodiment, a variation of the circuitry for the pixel 200 of the first embodiment will be described. In this embodiment, the description of elements common to those in the first embodiment may be omitted or simplified.

[0114] Figure 11 This is a diagram illustrating the configuration of pixel 200 according to this embodiment. Figure 11 and Figure 7 The difference lies in that the signal P_DECI_CLK is input to the clock input terminal of the flip-flop circuit 233. Descriptions of other circuit configurations are omitted because... Figure 7 The circuit configuration is similar.

[0115] Figure 12 This is a timing diagram illustrating the driving method of pixel 200 according to this embodiment. The main frame time period, subframe time period, and microframe time period are... Figure 8 The main frame time period, subframe time period, and microframe time period are the same.

[0116] Figure 12 The "P_DECI_CLK" indicates multiple pulses of the signal P_DECI_CLK in microframe periods MF11 and MF12. In microframe period MF11, the signal P_DECI_CLK goes high at times t81, t82, t83, t84, ..., t8x, and then goes low. In microframe period MF12, the signal P_DECI_CLK goes high at times t91, t92, t93, t94, ..., t9x, and then goes low.

[0117] Figure 12 The pulse AP4 shown in "Photon Detection" indicates the timing of photon detection. Figure 12 In the example, a photon was detected between time t82 and time t83 in microframe period MF11.

[0118] Figure 12The "count value" indicates the time variation of the count value maintained in the accumulator circuit 251 during microframe periods MF11 and MF12. When the high-level signal P_DECI_CLK is input to the clock input terminal of the trigger circuit 233, the pulse generated by photon detection is maintained in the trigger circuit 233. Therefore, after a photon is detected, the increase in the count value due to photon detection begins from the time t83 when the high-level signal P_DECI_CLK is first input. That is, during the microframe period MF11 in which the photon is detected, the count value increases by 1 whenever the signal P_DECI_CLK goes high at times t83, t84, ..., t8x. Thus, the count at time t8x in microframe period MF11 is six. Furthermore, in the next microframe period MF12, the count value increases by 1 whenever the signal P_DECI_CLK goes high at times t91, t92, t93, t94, ..., t9x. Therefore, the number of counts at time t9x of microframe period MF12 is 14.

[0119] In the first embodiment, the trigger circuit 233 holds the pulse caused by the photon detection during a microframe period following the microframe period after the photon detection. Therefore, in the first embodiment, the count value increases from the microframe period after the photon detection. On the other hand, in this embodiment, when the next pulse of the input signal P_DECI_CLK is received after a photon is detected in a certain microframe period, the trigger circuit 233 holds the pulse caused by the photon detection. Therefore, in this embodiment, the increase in the count value can begin within the microframe period after the photon detection. As a result, in this embodiment, in addition to the same effect as in the first embodiment, the illuminance can be more appropriately reflected in the count value compared to the first embodiment. Third Embodiment

[0120] In this embodiment, a variation of the circuitry for the pixel 200 of the first embodiment will be described. In this embodiment, the description of elements common to those in the first embodiment may be omitted or simplified.

[0121] Figure 13 This is a diagram illustrating the configuration of pixel 200 according to this embodiment. Figure 13 and Figure 7 The difference is that it can generate multiple signals with different weights. Besides... Figure 7 In addition to the components shown, pixel 200 also includes logic circuits 223 and 224. Furthermore, counter circuit 211 includes AND circuits 242 and 243 and accumulation circuits 252 and 253. Additionally, signals P_DECI_CLK1 and P_DECI_CLK2 are input to pixel 200 from weight control unit 116.

[0122] Signals P_DECI_CLK1 and P_DECI_CLK2 are pulse signals similar to signal P_DECI_CLK. However, signals P_DECI_CLK, P_DECI_CLK1, and P_DECI_CLK2 can be signals indicating different weights.

[0123] Each logic circuit in logic circuits 223 and 224 outputs the logical product of the inverted values ​​of the input signals at the first and second input terminals. A signal P_DECI_CLK1 (second reference signal), indicating a weight different from the weight of signal P_DECI_CLK (second weight), is input to the first input terminal of logic circuit 223, and a signal P_RCH_TRG is input to the second input terminal of logic circuit 223. Logic circuit 223 outputs the logical product of the inverted values ​​of signal P_DECI_CLK1 and signal P_RCH_TRG to the first input terminal of AND circuit 242 (second logic circuit). A signal P_DECI_CLK2, indicating a weight different from the weights of signals P_DECI_CLK and P_DECI_CLK1, is input to the first input terminal of logic circuit 224, and a signal P_RCH_TRG is input to the second input terminal of logic circuit 224. Logic circuit 224 outputs the logical product of the inverted values ​​of signal P_DECI_CLK2 and signal P_RCH_TRG to the first input terminal of AND circuit 243. The output signal of flip-flop circuit 233 is input to the second input terminals of AND circuit 242 and AND circuit 243.

[0124] AND circuit 241 outputs the logical product of the output signal of logic circuit 222 and the output signal of flip-flop circuit 233 to accumulator circuit 251. AND circuit 242 outputs the logical product of the output signal of logic circuit 223 and the output signal of flip-flop circuit 233 to accumulator circuit 252 (second accumulator circuit). AND circuit 243 outputs the logical product of the output signal of logic circuit 224 and the output signal of flip-flop circuit 233 to accumulator circuit 253. Each accumulator circuit in accumulator circuits 251, 252, and 253 counts the number of pulses by accumulating the pulses of the input signal. Accumulator circuits 251, 252, and 253 hold a first count value, a second count value, and a third count value, respectively. The first count value, the second count value, and the third count value may be different from each other due to differences in weight. In addition, the signal P_RES is input to accumulator circuits 251, 252, and 253. The signal P_RES goes high at the beginning of a main frame. As a result, the count values ​​held in the accumulator circuits 251, 252, and 253 are reset.

[0125] As described above, in this embodiment, in addition to achieving the same effect as in the first embodiment, three signals weighted based on three types of signals P_DECI_CLK, P_DECI_CLK1, and P_DECI_CLK2 can be acquired in parallel. For example, the three types of signals P_DECI_CLK, P_DECI_CLK1, and P_DECI_CLK2 can be signals indicating a sine-based weight, a cosine-based weight, and a weight that does not change for each subframe period. In this case, correlated images weighted based on a sine function, correlated images weighted based on a cosine function, and regular images can be acquired in parallel, and signals suitable for time-correlated imaging, such as optical flow calculations, can be acquired more efficiently. Fourth embodiment

[0126] In this embodiment, an example configuration of a photoelectric conversion device 100 that acquires signals under various illuminance levels using a circuit configuration and operation method different from those of the first to third embodiments will be described. In this embodiment, the description of elements common to the first to third embodiments may be omitted or simplified. Similar to the third embodiment, this embodiment illustrates a case of weighting based on three types of signals P_DECI_CLK, P_DECI_CLK1, and P_DECI_CLK2; however, the number of types of pulse signals used for weighting is not limited to this.

[0127] Figure 14 This is a diagram illustrating the configuration of pixel 200 according to this embodiment. Figure 14 and Figure 13 The difference lies in that pixel 200 includes a memory that holds the count value for a subframe period and a multiplier that multiplies the count value. Counter circuit 211 includes AND circuits 241, 242, and 243, accumulation circuits 251, 252, 253, and 254, memory 261, and multipliers 271 and 272. Furthermore, with... Figure 13 Unlike other embodiments, in this embodiment, the output holding circuit 230 is not provided.

[0128] The signal TCLK is input from logic circuit 222 to the first input terminal of AND circuit 241, and the output signal of waveform shaping unit 210 is input to the second input terminal of AND circuit 241. AND circuit 241 outputs the logical product of signal TCLK and the output signal (light-receiving signal) of waveform shaping unit 210 to accumulator circuits 251 and 254.

[0129] Accumulator circuits 251 and 254 count pulses by accumulating the pulses of the input signal. Accumulator circuits 251 and 254 hold the resulting count value. The count value output from accumulator circuit 254 is input to memory 261. A signal P_RCH_TRG is input to accumulator circuits 251 and 254 and memory 261. Memory 261 holds the count value output from accumulator circuit 254 at the beginning of a subframe period based on the signal P_RCH_TRG. After the count value is held in memory 261, accumulator circuit 254 resets the held count value based on the signal P_RCH_TRG. That is, accumulator circuit 254 resets the count value at the beginning of each subframe period. Memory 261 holds the count value (light intensity value) obtained in the previous subframe period within each subframe period and updates the held count value at the timing at the beginning of the next subframe period. Since the count value is two or more bits, memory 261 has a storage capacity of two or more bits. On the other hand, the accumulator circuit 251 does not reset when the subframe period is switched, and continues to accumulate the count value from the detection of the photon to the end of the main frame period.

[0130] The output signal of logic circuit 223 is input to the first input terminal of AND circuit 242, and the output signal of waveform shaping unit 210 is input to the second input terminal of AND circuit 242. AND circuit 242 outputs the logical product of the output signal of logic circuit 223 and the output signal of waveform shaping unit 210 to multiplier 271 (conversion unit).

[0131] The output signal of logic circuit 224 is input to the first input terminal of AND circuit 243, and the output signal of waveform shaping unit 210 is input to the second input terminal of AND circuit 243. AND circuit 243 outputs the logical product of the output signal of logic circuit 224 and the output signal of waveform shaping unit 210 to multiplier 272.

[0132] Multiplier 271 multiplies the output signal of AND circuit 242 by the count value stored in memory 261. Multiplier 271 outputs a signal (conversion signal) indicating the value obtained through the arithmetic process of multiplication to accumulator circuit 252 (first accumulator circuit). Multiplier 272 multiplies the output signal of AND circuit 243 by the count value stored in memory 261. Multiplier 272 outputs a signal indicating the value obtained through the arithmetic process of multiplication to accumulator circuit 253.

[0133] Accumulator 252 generates and holds a count value (first count value) by accumulating the signal value input from multiplier 271. Accumulator 253 generates and holds a count value by accumulating the signal value input from multiplier 272. Due to differences in weights, the count values ​​held by accumulators 252 and 253 can be different from each other. Signal P_RES is input to accumulators 251, 252, and 253. Signal P_RES goes high at the beginning of a main frame. As a result, the count values ​​held in accumulators 251, 252, and 253 are reset.

[0134] Figure 15 This is a timing diagram illustrating the driving method of pixel 200 according to this embodiment. Figure 15 The main focus is on the operations within subframe periods SF11 and SF12. Omissions or simplifications will be noted. Figures 8 to 10 and Figure 12 A description of common operations in any graph.

[0135] Figure 15 The term "main frame period" indicates that the main frame period F1 is divided into multiple sub-frame periods SF11, SF12, ... In this embodiment, unlike the first to third embodiments, the sub-frame periods are not divided into multiple micro-frame periods.

[0136] Figure 15 The pulse AP5 shown in "Photon Detection" indicates the timing of photon detection in subframe period SF11. Figure 15 The pulse AP6 shown in "Photon Detection" indicates the timing of photon detection in subframe period SF12.

[0137] Figure 15 The “P_DECI_CLK” indicates multiple pulses of the signal P_DECI_CLK in subframe periods SF11 and SF12. Figure 15 In the diagram, "P_DECI_CLK1" indicates multiple pulses of the signal P_DECI_CLK1 in subframe periods SF11 and SF12. In subframe period SF11, signals P_DECI_CLK and P_DECI_CLK1 go high at times t111, t112, t113, t114, ..., t11x, and then go low. In subframe period SF12, signals P_DECI_CLK and P_DECI_CLK1 go high at times t121, t122, t123, t124, t125, ..., t12y, and then go low.

[0138] Similarly, in this embodiment, the weights in signals P_DECI_CLK and P_DECI_CLK1 can be set using periodic functions such as sine functions, as in the first embodiment. Figure 15 In the example, during subframe period SF11, the number of pulses for signals P_DECI_CLK and P_DECI_CLK1 corresponding to the weights is eight. During subframe period SF12, the number of pulses for signals P_DECI_CLK and P_DECI_CLK1 corresponding to the weights is nine.

[0139] exist Figure 15 In the diagram, "count value of accumulator 254" and "count value of accumulator 251" indicate the time variation of the count values ​​maintained in accumulator circuits 254 and 251, respectively. In accumulator circuits 254 and 251, after a photon is detected, the count value increases due to the photon detection starting from the time the high-level signal P_DECI_CLK is first input. That is, during the subframe period SF11 in which the photon is detected, whenever the signal P_DECI_CLK goes high at times t112, t113, ..., t11x, the count value of accumulator circuits 254 and 251 increases by 1. Therefore, the count value of accumulator circuits 254 and 251 at time t11x in subframe period SF11 is seven.

[0140] As described above, the count value is reset in the accumulator circuit 254 for each subframe period. Therefore, the count value of the accumulator circuit 254 is reset to zero at the beginning of subframe period SF12. Thereafter, in subframe period SF12, whenever the signal P_DECI_CLK goes high at times t123, t124, ..., t12y after photon detection, the count value of the accumulator circuit 254 increases by 1. Therefore, the count value of the accumulator circuit 254 at time t12y in subframe period SF12 is seven.

[0141] On the other hand, in the accumulator circuit 251, the count value is not reset during the switching of subframe periods. Therefore, the count value of the accumulator circuit 254 is maintained at 7 at the beginning of subframe period SF12. Thereafter, in subframe period SF12, whenever the signal P_DECI_CLK goes high at times t123, t124, ..., t12y after photon detection, the count value of the accumulator circuit 251 increases by 1. Therefore, the count value of the accumulator circuit 251 at time t12y in subframe period SF12 is 14.

[0142] Figure 15The "memory value" indicates the time change of the count value held in memory 261. As described above, memory 261 holds the count value output from accumulator circuit 254 at the beginning of a subframe period. Therefore, memory 261 acquires and holds 7 at the beginning of subframe period SF12, where 7 is the count value acquired by accumulator circuit 254 in subframe period SF11. In this way, memory 261 has the function of holding the count value acquired by accumulator circuit 254 in the previous subframe period.

[0143] Figure 15 The "count value of accumulator 252" indicates the time change of the count value held in accumulator 252. In accumulator 252, after a photon is detected, the count value increases due to the photon detection starting from the time of the first input high-level signal P_DECI_CLK1. When the input signal P_DECI_CLK1 is pulsed, multiplier 271 multiplies the input value "1" by the value held in memory 261 and outputs the result to accumulator 252. That is, in subframe period SF12, when signal P_DECI_CLK1 goes high at time t123 after photon detection, multiplier 271 outputs the value obtained by multiplying the input value "1" by 7 held in memory 261 to accumulator 252. As a result, the count value of accumulator 252 increases from zero to seven. Similarly, at times t124, ..., t12y, the count value of accumulator 252 increases by 7. Therefore, the count value of the accumulator circuit 252 at time t12y of subframe period SF12 is 49. Since the change in the count value in the accumulator circuit 253 is the same as the change in the count value in the accumulator circuit 252 except for the input signal P_DECI_CLK2, its description will be omitted.

[0144] As described above, in this embodiment, a memory 261 that stores the count values ​​acquired in the preceding subframe period is arranged in the pixel 200. Next, multipliers 271 and 272, along with accumulation circuits 252 and 253, generate a count value by multiplying the count value of a certain subframe period by the count value of the preceding subframe period. Therefore, the generated count value has a value amplified based on the incident light amount in past subframe periods. Therefore, compared to the case where the count value is generated considering only the incident light amount in one subframe period, this embodiment can handle various light amounts. Therefore, according to this embodiment, a photoelectric conversion device capable of acquiring signals under various illuminance conditions is provided.

[0145] Furthermore, in the photoelectric conversion device of this embodiment, the output signal can be weighted based on a periodic function such as a trigonometric function. Therefore, the photoelectric conversion device of this embodiment is suitable for signal acquisition in time-correlated imaging, such as for calculating optical flow.

[0146] In the description of the operation of pixel 200, specific examples related to weighting are shown in some cases, such as expressions for functions of weights, bit widths of weights, and values ​​of weight weights. However, these examples are not particularly limited and can be changed appropriately. The number of subframe segments can also be set appropriately.

[0147] The conversion processes performed using the count values ​​stored in memory 261 are not limited to multiplication. For example, Figure 14 Each multiplier in multipliers 271 and 272 shown can be replaced by a shift operation circuit. In this case, the shift operation circuit performs a process corresponding to the operation of substantially multiplying the input value by the count value by shifting the input value by bits using a shift amount corresponding to the count value held in memory 261. As a result, the multiplication process can be accelerated. The shift operation circuit may include, for example, a demultiplexer. Fifth Embodiment

[0148] Reference Figure 16 The device according to the fifth embodiment is described. Figure 16 This is a block diagram illustrating a schematic configuration of the device according to this embodiment.

[0149] Figure 16 This is a schematic diagram illustrating a device EQP including a photoelectric conversion device APR. The photoelectric conversion device APR has the functions of the photoelectric conversion device 100 according to the first to fourth embodiments. All or part of the photoelectric conversion device APR is a semiconductor device IC. The photoelectric conversion device APR of this example can be used as, for example, an image sensor, an autofocus (AF) sensor, a light metering sensor, or a rangefinder sensor. The semiconductor device IC has pixel regions PX, each including a pixel circuit PXC comprising a photoelectric conversion unit, arranged in a matrix. The semiconductor device IC may have a peripheral region PR surrounding the pixel regions PX. Circuitry other than the pixel circuits may be arranged in the peripheral region PR.

[0150] The photoelectric conversion device APR can have a structure in which a first semiconductor chip having multiple photoelectric conversion units is stacked and a second semiconductor chip having peripheral circuits is stacked (stacked chip structure). Each peripheral circuit in the second semiconductor chip can be a column circuit corresponding to a pixel column in the first semiconductor chip. Each peripheral circuit in the second semiconductor chip can also be a matrix circuit corresponding to a pixel or pixel block in the first semiconductor chip. The connection between the first and second semiconductor chips can be achieved using through-electrodes (TSVs), inter-chip wiring via direct bonding through a conductor such as copper, connections via microbumps between chips, or connections via wire bonding, etc.

[0151] In addition to semiconductor device ICs, photoelectric conversion devices (APRs) may also include packaged components (PKGs) for mounting the semiconductor device ICs. The packaged component (PKG) may include a substrate on which the semiconductor device IC is fixed, a cover such as glass facing the semiconductor device IC, and connecting members such as bonding wires or bumps for connecting terminals disposed on the substrate and terminals disposed on the semiconductor device IC.

[0152] The device EQP may also include at least one of an optical device OPT, a control device CTRL, a processing device PRCS, a display device DSPL, a storage device MMRY, and a mechanical device MCHN. The optical device OPT corresponds to the photoelectric conversion device APR and is, for example, a lens, shutter, or mirror. The control device CTRL controls the photoelectric conversion device APR and is, for example, a semiconductor device such as an ASIC.

[0153] The processing unit PRCS processes the signal output from the photoelectric conversion device APR and forms an analog front-end (AFE) or digital front-end (DFE). The processing unit PRCS is a semiconductor device such as a central processing unit (CPU) or an application-specific integrated circuit (ASIC). The display device DSPL is an EL display device or liquid crystal display device that displays information (images) obtained from the photoelectric conversion device APR. The storage device MMRY is a magnetic device or semiconductor device that stores information (images) obtained from the photoelectric conversion device APR. The storage device MMRY is a volatile memory such as SRAM or DRAM, or a non-volatile memory such as flash memory or hard disk drive.

[0154] Furthermore, the processing device PRCS can acquire optical flow using signals output from the photoelectric conversion device 100 according to the first to fourth embodiments. For example, the processing device PRCS can generate a weighted correlation image based on a sine function, a weighted correlation image based on a cosine function, and a conventional image, and acquire optical flow from these three images.

[0155] The mechanical device MCHN includes a movable or propulsion unit such as a motor or engine. In the device EQP, the signal output from the photoelectric conversion device APR is displayed on the display device DSPL or transmitted to the outside via a communication device (not shown) included in the device EQP. Therefore, it is preferable that the device EQP also includes a storage device MMRY and a processing device PRCS, separate from the storage circuit unit and arithmetic circuit unit included in the photoelectric conversion device APR. The mechanical device MCHN can be controlled based on the signal output from the photoelectric conversion device APR.

[0156] Figure 16The illustrated device EQP can be an electronic device such as an information terminal with shooting capabilities (e.g., smartphones and wearable devices) or a camera (e.g., interchangeable lens cameras, compact cameras, camcorders, and surveillance cameras). The mechanical device MCHN in the camera can drive components of the optical device OPT for zooming, focusing, and shutter operation. Furthermore, the device EQP can be a transportation device (mobile body) such as a vehicle, ship, drone, or aircraft. The device EQP can be a medical device such as an endoscope or CT scanner. The device EQP can be a measuring device such as a range sensor, an analytical device such as an electron microscope, an office device such as a copier, or an industrial device such as a robot.

[0157] The mechanical device MCHN in the transport unit can be used as a mobile device. The equipment EQP as a transport unit is suitable for transporting the photoelectric conversion device APR, or for assisting driving (manipulation) and / or automating driving (manipulation) through camera functionality. The processing unit PRCS for assisting driving (manipulation) and / or automating driving (manipulation) can perform processing for operating the mechanical device MCHN as a mobile device based on information obtained from the photoelectric conversion device APR.

[0158] According to the first to fourth embodiments, signal acquisition can be satisfactorily achieved. Therefore, the photoelectric conversion device APR according to the first to fourth embodiments can provide high value to designers, manufacturers, sellers, purchasers, and / or their users. Therefore, when the photoelectric conversion device APR is installed on the device EQP, the value of the device EQP can also be increased. Therefore, when manufacturing and selling the device EQP, it is advantageous to determine the installation of the photoelectric conversion device APR of this embodiment on the device EQP to increase the value of the device EQP. Here, increased value corresponds to at least one of the following: adding functionality, improving performance, improving characteristics, improving reliability, increasing manufacturing yield, reducing environmental impact, reducing cost, reducing size, and reducing weight.

[0159] For example, by installing the photoelectric conversion device (APR) in a transport device, excellent performance can be obtained when photographing or measuring the external environment outside the transport device. Therefore, it is advantageous to determine the installation of the photoelectric conversion device (APR) according to this embodiment on the transport device to improve the performance of the transport device itself when manufacturing and selling the transport device. In particular, the photoelectric conversion device (APR) is suitable for transport devices that use information obtained by the photoelectric conversion device (APR) to provide driving support and / or automatic driving. Sixth Embodiment

[0160] Figure 17A and Figure 17B This is a block diagram of a device related to a vehicle-mounted camera according to this embodiment. Figure 17A and Figure 17B An example of applying the above-described photoelectric conversion device to a movable body such as a vehicle is shown. Device 80 includes an imaging device 800 (an example of a photoelectric conversion device) and a signal processing device (processing unit) that processes signals from the imaging device 800. Device 80 includes: an image processing unit 801 that performs image processing on multiple image data acquired by the imaging device 800; and a disparity calculation unit 802 that calculates disparity (phase difference of a disparity image) from the multiple image data acquired by device 80.

[0161] Here, device 80 may include an optical system (not shown) that guides light to imaging device 800. The optical system may include, for example, lenses, shutters, and mirrors. Multiple photoelectric conversion units, substantially conjugate to the pupil of the optical system, may be arranged in pixels included in imaging device 800. For example, the multiple photoelectric conversion units may be arranged corresponding to a microlens. The multiple photoelectric conversion units may receive light beams transmitted through different positions of the pupil of the optical system. Therefore, imaging device 800 outputs multiple image data corresponding to the light beams transmitted through different positions of the pupil of the optical system. The parallax calculation unit 802 can then use the output multiple image data to calculate parallax.

[0162] Device 80 includes: a distance measurement unit 803 that calculates the distance to an object based on the calculated parallax; and a collision determination unit 804 that determines the likelihood of a collision based on the calculated distance. Here, the parallax calculation unit 802 and the distance measurement unit 803 are examples of distance information acquisition units that acquire distance information to an object. That is, the distance information is information related to parallax, defocus, and the distance to the object. The collision determination unit 804 can use any of this distance information to determine the likelihood of a collision. Note that time-of-flight (ToF) technology can be used to acquire the distance information. The distance information acquisition unit can be implemented using specially designed hardware or software modules. Furthermore, it can be implemented using a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), or a combination thereof.

[0163] Device 80 is connected to vehicle information acquisition device 810 and can acquire vehicle information such as vehicle speed, yaw rate, and steering angle. Furthermore, device 80 is connected to control ECU 820, which is a control device that outputs control signals to generate braking force for the vehicle based on the judgment result of collision judgment unit 804. Device 80 is also connected to alarm device 830, which issues an alarm to the driver based on the judgment result of collision judgment unit 804. For example, when the collision judgment unit 804 determines that the probability of a collision is high, control ECU 820 controls the vehicle to avoid a collision or reduce damage by braking, returning the accelerator to its original position, or suppressing engine output. Alarm device 830 alerts the user by emitting alarms such as sounds, displaying alarm information on a screen such as a car navigation system, or vibrating the seat belt or steering wheel. Device 80 serves as a control unit for controlling the operation of the vehicle as described above.

[0164] In this embodiment, the device 80 captures images of the vehicle's surroundings (e.g., in front or behind). Figure 17B The device is shown in the case of capturing images in front of the vehicle (camera range 850). The vehicle information acquisition device 810, which acts as a camera control unit, sends instructions to the device 80 or the camera device 800 to perform camera operation. With this configuration, the accuracy of distance measurement can be further improved.

[0165] While examples of controls for avoiding collisions with other vehicles have been described above, this embodiment is applicable to autonomous driving controls for following other vehicles or for autonomous driving controls that do not leave traffic lanes. Furthermore, the device is not limited to vehicles such as automobiles, and can be applied to mobile bodies (mobile devices) such as ships, aircraft, satellites, industrial robots, and consumer robots. Additionally, the device can be widely applied to devices utilizing object recognition or biometric authentication, such as intelligent transportation systems (ITS) or surveillance systems, and is not limited to mobile bodies. Other embodiments

[0166] This disclosure is not limited to the above embodiments, and various modifications can be made. For example, examples of adding a portion of the configuration of any one of the embodiments to other embodiments or replacing a portion of the configuration of any one of the embodiments with a portion of the configuration of other embodiments are also embodiments of this disclosure.

[0167] The above embodiments can be appropriately modified without departing from the technical concept. Note that the disclosure in this specification includes not only the matters described herein, but also all matters that can be understood from this specification and the accompanying drawings. Furthermore, the disclosure in this specification includes the complement of the concepts described herein. In other words, for example, when the description "A is greater than B" exists in this specification, even if the description "A is not greater than B" is omitted, it can be said that the description "A is not greater than B" is disclosed in this specification. This is because when describing "A is greater than B," the case of "A is not greater than B" is assumed to be considered.

[0168] Embodiments of the present invention can also be implemented by providing software (including computer program products of computer programs) that performs the functions of the above embodiments to a system or device via a network or various storage media, and the computer (central processing unit (CPU) or microprocessor unit (MPU) of the system or device) reads and executes the computer program.

[0169] It should be noted that the above embodiments are merely specific examples for carrying out this disclosure, and the technical scope of this disclosure should not be interpreted in a limiting manner by these embodiments. That is, this disclosure can be implemented in various forms without departing from the technical concept or its main features.

[0170] While this disclosure has been described with reference to embodiments, it should be understood that this disclosure is not limited to the disclosed embodiments. The scope of the appended claims should be given the broadest interpretation to cover all such modifications and equivalent structures and functions.

Claims

1. A photoelectric conversion device, comprising: Avalanche photodiode; An output holding circuit is configured to hold the light-receiving signal based on the output of the avalanche photodiode; A first logic circuit, wherein a light-receiving signal held in the output holding circuit and a first reference signal indicating a first weight are input to the first logic circuit; and A first accumulator circuit is configured to hold a first count value obtained by accumulating the output of the first logic circuit. Specifically, the first accumulation circuit resets the first count value for each first time period. The first time period is further divided into multiple second time periods. Wherein, the first weight changes for each of the second time periods, and The output holding circuit resets the light-receiving signal for each of the second time periods.

2. The photoelectric conversion device according to claim 1, wherein, The output holding circuit holds the received light signal after the avalanche photodiode first detects a photon in the second time period.

3. The photoelectric conversion device according to claim 2, in, The second time period was divided into multiple third time periods, and The output holding circuit holds the light-receiving signal from the third time period following the third time period after the first detection of a photon in the plurality of third time periods.

4. The photoelectric conversion device according to claim 2, in, The second time period was divided into multiple third time periods, and The output holding circuit holds the light-receiving signal from the third time period in which the photon is first detected among the plurality of third time periods.

5. The photoelectric conversion device according to claim 2, wherein, After the output holding circuit first detects a photon in the second time period, it holds the received light signal at the time corresponding to the first reference signal.

6. The photoelectric conversion device according to claim 1, further comprising: The second logic circuit, wherein the light-receiving signal held in the output holding circuit and the second reference signal indicating the second weight are input to the second logic circuit; as well as The second accumulator circuit is configured to hold the second count value obtained by accumulating the output of the second logic circuit.

7. The photoelectric conversion device according to claim 1, wherein, The first weight is determined by a periodic function with different phases for each of the second time periods.

8. The photoelectric conversion device according to claim 7, wherein, The periodic function includes a sine function or a cosine function.

9. The photoelectric conversion device according to claim 8, wherein, The length of the period of the periodic function is equal to the length of the first time period.

10. The photoelectric conversion device according to claim 1, wherein, The first logic circuit is an AND circuit.

11. The photoelectric conversion device according to claim 1, wherein, The first reference signal includes a plurality of pulses, the number of which is associated with the first weight.

12. A photoelectric conversion device, comprising: Avalanche photodiode; A memory configured to hold two or more bits of light quantity values ​​generated by the light-receiving signal based on the output of the avalanche photodiode; A conversion unit, wherein a light intensity value stored in the memory and a first reference signal indicating a first weight are input to the conversion unit, and the conversion unit outputs a conversion signal by calculation based on the light intensity value and the first weight; as well as A first accumulator circuit is configured to hold a first count value obtained by accumulating the conversion signal. The first time period during which the first accumulator circuit accumulates the first count value is divided into multiple second time periods. Wherein, the first weight changes for each of the second time periods, and The light intensity value is generated based on the light received signal input in a second time period.

13. The photoelectric conversion device according to claim 12, wherein, The conversion signal has a value obtained by multiplying the first weight by the light quantity value.

14. The photoelectric conversion device according to claim 12, wherein, The conversion signal has a value obtained by shifting the first weight using a shift amount corresponding to the light intensity value.

15. The photoelectric conversion device according to claim 12, wherein, The memory updates the light intensity value whenever each of the multiple second time periods begins.

16. The photoelectric conversion device according to claim 12, further comprising: A logic circuit, wherein the light-receiving signal and the second reference signal indicating the second weight are input to the logic circuit; as well as A second accumulator circuit is configured to hold a second count value obtained by accumulating the output of the logic circuit.

17. The photoelectric conversion device according to claim 16, wherein, The logic circuit is an AND circuit.

18. The photoelectric conversion device according to claim 12, wherein, The first weight is determined by a periodic function with different phases for each of the second time periods.

19. The photoelectric conversion device according to claim 18, wherein, The periodic function includes a sine function or a cosine function.

20. The photoelectric conversion device according to claim 19, wherein, The length of the period of the periodic function is equal to the length of the first time period.

21. The photoelectric conversion device according to claim 12, wherein, The first reference signal includes a plurality of pulses, the number of which is associated with the first weight.

22. The photoelectric conversion device according to any one of claims 1 to 21, further comprising a processing device configured to acquire optical flow based on the first count value.

23. An apparatus comprising the photoelectric conversion device according to any one of claims 1 to 22, further comprising at least one of the following devices: Optical device, suitable for the photoelectric conversion device, A control device configured to control the photoelectric conversion device. A processing device configured to process the signal output from the photoelectric conversion device. A display device configured to display the information obtained by the photoelectric conversion device. A storage device configured to store information obtained by the photoelectric conversion device, and A mechanical device configured to operate based on information obtained from the photoelectric conversion device.

24. The device according to claim 23, wherein, The processing device acquires optical flow based on the signal output from the photoelectric conversion device.

25. The device according to claim 23, wherein, The processing device acquires information related to the distance from the photoelectric conversion device to the object.

Images