Photoelectric conversion devices and equipment

Abstract

This invention provides a technology for reducing the size of pixel circuits in a photoelectric converter with time-correlated imaging capabilities without compromising temporal resolution. [Solution] The photoelectric conversion device has an avalanche photodiode and includes a light receiving unit that outputs a photon detection signal in response to the incidence of a photon, a first integration unit that integrates a first signal weighted based on first weighting information with respect to the photon detection signal, a second integration unit that integrates a second signal weighted based on second weighting information relating to time correlation weighting for calculating optical flow with respect to the photon detection signal, a control unit that outputs a first control signal that controls the output of the first signal from the light receiving unit to the first integration unit, and a second control signal that controls the output of the second signal from the light receiving unit to the second integration unit. The number of assertions of the second control signal in a unit exposure period is less than the number of assertions of the first control signal in a unit exposure period.

Description

Technical Field

[0001] The present invention relates to a photoelectric conversion device and equipment.

Background Art

[0002] As one of imaging sensors, a time-correlated image sensor is known. Non-Patent Document 1 discloses a structure in which a correlation detection function is incorporated into pixels as a time-correlated image sensor. By providing this structure, it becomes possible to detect a time-varying pattern with a wider bandwidth than the frame rate. Further, Non-Patent Document 1 discloses a structure in which a photodiode that generates a photocurrent and a plurality of capacitors that accumulate this photocurrent are provided as this correlation detection structure.

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the time-correlated image sensor described in Non-Patent Document 1, since the correlation detection function is incorporated into the pixels, the circuit scale of the pixels inevitably increases, and an increase in the size of the photoelectric conversion device cannot be avoided.

[0005] An object of the present invention is to provide a technique for reducing the circuit scale of pixels in a photoelectric conversion device having a time-correlated imaging function.

Means for Solving the Problems

[0006] According to one disclosure of this specification, a photoelectric converter is provided, having an avalanche photodiode, a light receiving unit that outputs a photon detection signal in response to the incidence of a photon, a first integrating unit that integrates a first signal weighted based on first weighting information with respect to the photon detection signal, a second integrating unit that integrates a second signal weighted based on second weighting information relating to time correlation weighting for calculating optical flow with respect to the photon detection signal, a control unit that outputs a first control signal that controls the output of the first signal from the light receiving unit to the first integrating unit, and a second control signal that controls the output of the second signal from the light receiving unit to the second integrating unit, wherein the number of assertions of the second control signal in a unit exposure period is less than the number of assertions of the first control signal in a unit exposure period.

[0007] Furthermore, according to another disclosure of this specification, there is a method for driving a photoelectric converter having an avalanche photodiode and a light receiving unit that outputs a photon detection signal in response to the incidence of a photon, a first integrating unit that integrates a first signal weighted based on first weighting information with respect to the photon detection signal, and a second integrating unit that integrates a second signal weighted based on second weighting information relating to time correlation weighting for calculating optical flow with respect to the photon detection signal, wherein the output of the first signal from the light receiving unit to the first integrating unit is controlled by a first control signal, and the output of the second signal from the light receiving unit to the second integrating unit is controlled by a second control signal having fewer assertions per unit exposure period than the first control signal. [Effects of the Invention]

[0008] According to the present invention, the circuit size of pixels can be reduced in a photoelectric conversion device having a time-correlated imaging function. [Brief explanation of the drawing]

[0009] [Figure 1] This is a block diagram (part 1) showing the schematic configuration of the photoelectric conversion device according to the first embodiment. [Figure 2]This is a block diagram (part 2) showing the schematic configuration of the photoelectric conversion device according to the first embodiment. [Figure 3] This is a block diagram showing the schematic configuration of pixels in a photoelectric converter according to the first embodiment. [Figure 4] This is a perspective view showing an example of the configuration of a photoelectric conversion device according to the first embodiment. [Figure 5] This is a diagram (part 1) illustrating the basic operation of the photoelectric conversion unit in the photoelectric conversion device according to the first embodiment. [Figure 6] This is a diagram (part 2) illustrating the basic operation of the photoelectric conversion unit in the photoelectric conversion device according to the first embodiment. [Figure 7] This is a functional block diagram showing the schematic configuration of the weight control unit and pixels in the photoelectric conversion device according to the first embodiment. [Figure 8] This is a functional block diagram showing the schematic configuration of the weight control unit in the photoelectric conversion device according to the first embodiment. [Figure 9] This is a circuit diagram showing an example of the pixel configuration in a photoelectric converter according to the first embodiment. [Figure 10] This figure shows the relationship between mainframe periods and subframe periods, as well as an example of how the weighting amount changes over time. [Figure 11] This is a timing diagram (part 1) showing the operation of pixels in the photoelectric converter according to the first embodiment. [Figure 12] This is a timing diagram (part 2) showing the operation of pixels in the photoelectric converter according to the first embodiment. [Figure 13] This is a timing diagram (part 3) showing the operation of pixels in the photoelectric converter according to the first embodiment. [Figure 14] This is a functional block diagram showing the schematic configuration of the weight control unit and pixels in the photoelectric conversion device according to the second embodiment. [Figure 15] This is a circuit diagram showing an example of the pixel configuration in a photoelectric converter according to the second embodiment. [Figure 16] This is a block diagram showing the schematic configuration of a photoelectric conversion system according to a third embodiment of the present invention. [Figure 17]It is a diagram showing a configuration example of a photoelectric conversion system and a moving body according to a fourth embodiment of the present invention. [Figure 18] It is a block diagram showing a schematic configuration of a device according to a fifth embodiment of the present invention. [Embodiments for Carrying Out the Invention]

[0010] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. In each of the embodiments described below, as an example of a photoelectric conversion device, a photoelectric conversion device for imaging use will be mainly described. However, each embodiment is not limited to a photoelectric conversion device for imaging use, and can also be applied to other photoelectric conversion devices. For example, other examples of photoelectric conversion devices include a distance measuring device (a device for distance measurement using focus detection or TOF (Time Of Flight)), a photometric device (a device for measuring the amount of incident light), and the like.

[0011] Note that the conductivity type of the transistor described in the embodiments below is an example, and is not limited to only the conductivity type described in the embodiments. With respect to the conductivity type described in the embodiments, the conductivity type can be appropriately changed, and along with this change, the potentials of the gate, source, and drain of the transistor are appropriately changed. For example, for a transistor that operates as a switch, the low level and high level of the potential supplied to the gate may be reversed with respect to the description in the embodiments along with the change in the conductivity type.

[0012] Also, in the following embodiments, the connection between circuit elements may be described. In this case, even if there is another element intervening between the elements of interest, unless otherwise specified, the elements of interest are treated as being connected. For example, assume that element A is connected to one node of a capacitive element C having a plurality of nodes, and element B is connected to the other node. Even in such a case, unless otherwise specified, element A and element B are treated as being connected.

[0013] [First Embodiment] Prior to describing the photoelectric conversion device according to the first embodiment of the present invention, the principles of the time-correlated image sensor and the event-based sensor will be briefly explained.

[0014] The time-correlated image sensor comprises a photodiode and a configuration for acquiring the signal output by the photodiode in multiple parts. The signal for each pixel that generates the image is expressed by the following equation (1).

number

[0015] Here, f(x,y,t) is the brightness of pixel (x,y) at time t. Also, v is the velocity of pixel (x,y) (the time derivative of pixel (x,y)). ∇ is the nabla operator (vector differential operator).

[0016] If the exposure time for acquiring one frame of image is T, then image g n (x,y) is expressed by the following equation (2).

number

[0017] As shown in equation (2), image g n (x,y) is a complex number e with lightness f(x,y,t) -inΔwt The image g is obtained by multiplying the reference signal represented by by and integrating over the duration of one frame. n Let (x,y) satisfy the following equation (3).

number

[0018] The second term on the left side of equation (3) represents the boundary value of the integral. Equation (3) is a system of simultaneous equations because it consists of multiple equations that differ depending on the value of n. Therefore, the boundary value of the integral can be eliminated by solving the system of simultaneous equations using, for example, two images g0(x,y) and g1(x,y). The time-correlated image sensor consists of an intensity image g0(x,y) consisting only of the real part and a correlation image g of a complex number. n The real and imaginary parts of (x,y) can be output (hereinafter, the complex correlation image will also be called the time-correlated signal). Therefore, by substituting the output signal of the time-correlated image sensor into the system of equations in equation (3) and solving it, the velocity v at each pixel (x,y), i.e., the optical flow, can be obtained.

[0019] In the signal processing of a time-correlated image sensor, it is necessary to calculate an integral over the range of one frame period, as shown in equation (2). Therefore, the output timing of the correlated image is limited to one frame period. In a time-correlated image sensor, the period of the reference signal and the period of the shutter opening period are matched. Therefore, the correlated image is output at a frequency corresponding to the period of the shutter opening period.

[0020] Here, we will provide an overview of event-based sensors. An event-based sensor detects changes in brightness within the shooting range and outputs an event signal each time a change in brightness is detected. An event-based sensor includes, for example, multiple pixels arranged in a matrix. That is, an event signal is a signal associated with an event, and an event is a change in the brightness of a pixel. An event signal may include, for example, the time when the event was detected, the position of the pixel where the event was detected, and the change in the pixel value. The time when the event was detected can be measured relative to the time indicated by the internal clock of the event-based sensor (event camera time).

[0021] The time reference for event detection can be reset as needed. A change in pixel value is, for example, a change in brightness. A change in pixel value may be the amount of change itself, or it may be information indicating whether the brightness change is positive or negative.

[0022] An event-based sensor outputs an event signal when a change in brightness occurs, and does not output an event signal when no change in brightness occurs. In other words, an event-based sensor outputs event signals asynchronously. Asynchronous output means that the signal is output independently of the pixel in time.

[0023] The operation of the event-based sensor can be expressed mathematically as follows: Equation (4).

number

[0024] In equation (4), Y(x,y,t) is the image at time t. Time t0 is the measurement start time. Image Y(x,y,t0) is the initial image stored at time t0. Generally, image Y(x,y,t0) can be set to 0. ΔY is the threshold for event occurrence (absolute value of brightness change). p(x,y,s i ) is the event signal that occurred at the i-th pixel (x,y), and p(x,y,s) is the event signal at the time of event detection. i The value of ) is 1 or -1 depending on the sign of the brightness change. δ(ss i ) is the Dirac delta function.

[0025] Event-based sensors can be equipped with a function to output time-correlated signals, such as those of time-correlated image sensors. When time t is the end time of the frame period, the output signal of the time-correlated image sensor can be expressed using the angular velocity ω (ω = 2π / T) as shown in equations (5) to (7) below.

number

[0026] In a time-correlated image sensor, charge based on the current output from the photodiode is stored in a capacitor. The stored charge corresponds to brightness. On the other hand, in an event-based sensor, a signal is output that is a quantized version of the change in current output from the photodiode. Therefore, in an event-based sensor, the output from the photodiode at time s can be divided into a localization term f(x,y,tT) which is constant within the measurement period, and a displacement term δf(x,y,s) which corresponds to the difference from the localization term. Thus, f(x,y,s) is expressed as shown in equation (8) below.

number

[0027] Furthermore, considering the properties of the reference signal, equations (9) and (10) below are satisfied.

number

[0028] Using the relationship between equations (9) and (10), equations (5) through (7) can be rewritten using a localization term and a displacement term. This yields equations (11) through (13) below.

number

[0029] In the event-based sensor, the current output by the photodiode of the time-correlated image sensor is converted into an event signal for the event-based sensor, as shown in equation (14) below.

number

[0030] As a result, equations (11) through (13) can be transformed into equations (15) through (17) below.

number

[0031] As shown in equations (15) to (17), a time-correlated signal can be output using event signals that occurred during the signal acquisition period (period T).

[0032] Next, the schematic configuration of the photoelectric conversion device according to the first embodiment of the present invention will be described with reference to Figure 1. Figure 1 is a block diagram showing the schematic configuration of the photoelectric conversion device according to this embodiment.

[0033] As shown in Figure 1, the photoelectric conversion device 100 according to this embodiment includes a pixel unit 10, a vertical scanning circuit unit 40, a readout circuit unit 50, a horizontal scanning circuit unit 60, an output circuit unit 70, a weight control unit 80, and a control pulse generation unit 90.

[0034] The pixel unit 10 is provided with multiple pixels 12 arranged in multiple rows and multiple columns. Each pixel 12 may include a photoelectric conversion unit including a photoelectric conversion element and a signal processing unit that processes the signal output from the photoelectric conversion unit. The number of pixels 12 constituting the pixel unit 10 is not particularly limited. For example, the pixel unit 10 can be composed of multiple pixels 12 arranged in an array of several thousand rows x several thousand columns, as in a typical digital camera. Alternatively, the pixel unit 10 may be composed of multiple pixels 12 arranged in one row or one column. Alternatively, the pixel unit 10 may be composed of a single pixel 12. The specific configuration and operation of the pixels 12 will be described later.

[0035] Each row of the pixel array in the pixel section 10 has a control line 14 extending in a first direction (horizontal direction in Figure 1). The control line 14 is connected to each pixel 12 arranged in the first direction and forms a common signal line for these pixels 12. The first direction in which the control line 14 extends is sometimes called the row direction or horizontal direction. Each of the control lines 14 may include multiple signal lines for supplying multiple types of control signals to the pixels 12.

[0036] Furthermore, each row of the pixel array in the pixel section 10 has an output line 16 extending in a second direction (vertical direction in Figure 1) that intersects the first direction. The output line 16 is connected to each of the pixels 12 arranged in the second direction, forming a common signal line for these pixels 12. The second direction in which the output line 16 extends is sometimes called the column direction or the vertical direction. Each of the output lines 16 may contain multiple signal lines. For example, the output line 16 may contain multiple signal lines for transferring multi-bit digital signals output from the pixels 12 bit by bit.

[0037] Furthermore, each of the multiple pixels 12 constituting the pixel section 10 is connected to the weight control unit 80. The weight control unit 80 and each of the multiple pixels 12 can be connected by multiple signal lines. In Figure 1, the weight control unit 80 and the pixels 12 are connected via signal lines arranged in each column of the pixel array, but the connection configuration between the weight control unit 80 and the pixels 12 is not limited to the example in Figure 1.

[0038] Each row's control line 14 is connected to the vertical scanning circuit unit 40. The vertical scanning circuit unit 40 is a control circuit that generates control signals to drive the pixels 12 in response to control signals from the control pulse generation unit 90 and supplies the generated control signals to the pixels 12 via the control lines 14. Logic circuits such as shift registers and address decoders may be used in the vertical scanning circuit unit 40. The vertical scanning circuit unit 40 sequentially scans the pixels 12 in the pixel unit 10 row by row and outputs the pixel signal of each pixel 12 to the readout circuit unit 50 via the output line 16.

[0039] The output lines 16 of each row are connected to the readout circuit unit 50. The readout circuit unit 50 has the function of holding the pixel signals output from each row of the pixel array via the output lines 16 in a holding unit provided corresponding to each row. The readout circuit unit 50 may further have the function of performing predetermined arithmetic processing on the pixel signals read out from the pixel unit 10. The arithmetic processing performed by the readout circuit unit 50 may include processing related to time correlation imaging.

[0040] The horizontal scanning circuit unit 60 is a control circuit that receives a control signal output from the control pulse generation unit 90, generates a control signal for reading pixel signals from the holding units of each column of the reading circuit unit 50, and supplies it to the reading circuit unit 50. Logic circuits such as shift registers and address decoders may be used in the horizontal scanning circuit unit 60. The horizontal scanning circuit unit 60 sequentially scans the holding units of each column of the reading circuit unit 50 and sequentially outputs the pixel signals held in each to the output circuit unit 70.

[0041] The output circuit section 70 has an external interface circuit and is a circuit section for outputting the pixel signal output from the readout circuit section 50 to an external signal processing device 110 of the photoelectric converter 100. The external interface circuit provided by the output circuit section 70 is not particularly limited. For example, a SerDes (SERializer / DESerializer) transmission circuit can be applied as an external interface circuit. Examples of SerDes transmission circuits include an LVDS (Low Voltage Differential Signaling) circuit and an SLVS (Scalable Low Voltage Signaling) circuit.

[0042] The weight control unit 80 has the function of controlling the weighting (weighting amount) of the signals generated by the pixels 12. Specifically, the weight control unit 80 generates a first weight signal and a second weight signal, and outputs at least one of the first weight signal and the second weight signal to each of the multiple pixels 12 that make up the pixel unit 10. The weight control unit 80 can also be called a weight signal generation unit that generates the first weight signal and the second weight signal. The specific configuration and operation of the weight control unit 80 will be described later.

[0043] The control pulse generation unit 90 is a control circuit that generates control signals to control the operation and timing of the vertical scanning circuit unit 40, the readout circuit unit 50, the horizontal scanning circuit unit 60, and the weight control unit 80, and supplies them to each functional block. At least a portion of the control signals that control the operation and timing of each functional block may be supplied from outside the photoelectric converter 100.

[0044] The signal processing device 110 is a functional block that performs predetermined signal processing on the signal output from the photoelectric converter 100. The signal processing device 110 can perform time-correlated imaging processing, such as calculating optical flow, using the signal output from the photoelectric converter 100. This processing may be based, for example, on equations (1) to (17) described above. The signal processing device 110 may be located inside the photoelectric converter 100, or it may be located in the equipment on which the photoelectric converter 100 is mounted. When the functions of the signal processing device 110 are located in the photoelectric converter 100, the photoelectric converter 100 can perform predetermined calculation processing on the pixel signal in the preceding stage of the output circuit section 70, for example, in the readout circuit section 50.

[0045] Note that the connection configuration of each functional block of the photoelectric converter 100 is not limited to the configuration example shown in Figure 1, and can also be configured as shown in Figure 2, for example.

[0046] In the configuration example shown in Figure 2, output lines 16 extending in a first direction are provided for each row of the pixel array in the pixel unit 10. Each output line 16 is connected to a pixel 12 arranged in the first direction, forming a common signal line for these pixels 12. In addition, control lines 18 extending in a second direction are provided for each column of the pixel array in the pixel unit 10. Each control line 18 is connected to a pixel 12 arranged in the second direction, forming a common signal line for these pixels 12.

[0047] Each row's control line 18 is connected to the horizontal scanning circuit unit 60. The horizontal scanning circuit unit 60 generates control signals for reading pixel signals from pixels 12 in response to control signals output from the control pulse generation unit 90, and supplies them to the pixels 12 via the control lines 18. Specifically, the horizontal scanning circuit unit 60 sequentially scans a plurality of pixels 12 of the pixel unit 10 in column units, and outputs the pixel signals of the pixels 12 in each row belonging to the selected column to the readout circuit unit 50 via the output line 16. Alternatively, each pixel 12 may be provided with a holding unit that constitutes a row-by-row shift register, and the pixel signals of the pixels 12 in each column may be sequentially transferred to the holding unit of the adjacent pixel 12, thereby transferring the pixel signals of the pixels 12 in each column to the readout circuit unit 50.

[0048] The readout circuit 50 includes a plurality of holding units (not shown) corresponding to each row of the pixel array of the pixel unit 10, and has the function of holding the pixel signals of the pixels 12 of each row, which are output column by column from the pixel unit 10 via the output line 16, in the holding unit of the corresponding row. The readout circuit 50 also receives a control signal output from the control pulse generation unit 90 and sequentially outputs the pixel signals held in the holding unit of each row to the output circuit 70. Other configurations in the example configuration shown in Figure 2 may be the same as those in the example configuration shown in Figure 1.

[0049] Figure 3 is a block diagram illustrating the schematic configuration of a pixel 12. Each pixel 12, as shown in Figure 3, includes a photoelectric conversion unit 20 and a signal processing unit 30. The photoelectric conversion unit 20 has a photoelectric conversion element 22 and outputs a signal corresponding to incident light. The signal processing unit 30 is a signal processing circuit that processes the signal output from the photoelectric conversion unit 20. The signal processing unit 30 may include, for example, a functional block 30A including a quench element 32 and a waveform shaping circuit 34, and a functional block 30B including a processing circuit 36 ​​and a selection circuit 38. In the pixel configuration shown in Figure 3, the control lines 14 of each row may include, for example, a signal line 14A supplied with a control signal PRES from the vertical scanning circuit unit 40, and a signal line 14B supplied with a control signal PSEL from the vertical scanning circuit unit 40.

[0050] The photoelectric conversion element 22 may be an avalanche photodiode (hereinafter referred to as "APD"). The anode of the APD constituting the photoelectric conversion element 22 is connected to a node to which voltage VL is supplied. The cathode of the APD constituting the photoelectric conversion element 22 is connected to one terminal of the quench element 32. The connection node between the photoelectric conversion element 22 and the quench element 32 is the output node of the photoelectric conversion unit 20. The other terminal of the quench element 32 is connected to a node to which a voltage VH higher than voltage VL is supplied. Voltages VL and VH are set so that a reverse bias voltage sufficient for the APD to perform avalanche multiplication operation is applied. In one example, a negative high voltage is applied as voltage VL, and a positive voltage of about the power supply voltage is applied as voltage VH. For example, voltage VL is -30V and voltage VH is 1V.

[0051] The photoelectric conversion element 22 can be composed of an APD as described above. By supplying the APD with a reverse bias voltage sufficient for avalanche multiplication, the carriers generated by the incidence of light on the APD undergo avalanche multiplication, and an avalanche current is generated. There are two operating modes when a reverse bias voltage is supplied to the APD: Geiger mode and linear mode. Geiger mode is an operating mode in which the voltage applied between the anode and cathode is a reverse bias voltage greater than the breakdown voltage of the APD. Linear mode is an operating mode in which the voltage applied between the anode and cathode is a reverse bias voltage near or below the breakdown voltage of the APD. An APD operating in Geiger mode is called a SPAD (Single Photon Avalanche Diode). The APD constituting the photoelectric conversion element 22 may operate in linear mode or in Geiger mode, but a SPAD is more preferable because it has a larger potential difference than a linear mode APD and the effect of improving the signal-to-noise ratio is more pronounced.

[0052] In the circuit configuration shown in Figure 3, the anode of the APD is at a fixed potential and the signal is taken from the cathode side. However, the cathode of the APD may also be at a fixed potential and the signal may be taken from the anode side. In the former case, the signal charge is an electron. In the latter case, the signal charge is a hole. Furthermore, although this embodiment describes the case where one node of the APD is at a fixed potential, the potentials of both nodes may fluctuate.

[0053] The quench element 32 has the function of converting the change in avalanche current generated in the photoelectric conversion element 22 into a voltage signal. Furthermore, the quench element 32 functions as a load circuit (quench circuit) during signal multiplication by avalanche multiplication, reducing the voltage applied to the photoelectric conversion element 22 and suppressing avalanche multiplication. This operation by the quench element 32 to suppress avalanche multiplication is called the quench operation. The quench element 32 also has the function of returning the voltage supplied to the photoelectric conversion element 22 to voltage VH by allowing current to flow to compensate for the voltage drop caused by the quench operation. This operation by the quench element 32 to return the voltage supplied to the photoelectric conversion element 22 to voltage VH is called the recharge operation. The quench element 32 can be composed of a resistor, a MOS transistor, or the like.

[0054] The waveform shaping circuit 34 has an input node to which the output signal from the photoelectric conversion unit 20 is supplied, and an output node. The waveform shaping circuit 34 has the function of converting the analog signal supplied from the photoelectric conversion unit 20 into a pulse signal. The waveform shaping circuit 34 may be composed of logic circuits including NOT gates (inverter circuits), NOR gates, NAND gates, etc. The output node of the waveform shaping circuit 34 is connected to the processing circuit 36.

[0055] The processing circuit 36 ​​has an input node to which the output signal of the waveform shaping circuit 34 is supplied, an input node connected to the control line 14, and an output node. The processing circuit 36 ​​has the function of performing predetermined signal processing on the output signal of the waveform shaping circuit 34 and holding the processed signal or processing result. The processing circuit 36 ​​is not particularly limited, but for example it may be configured to include a counter circuit. In this case the processing circuit 36 ​​counts the pulses superimposed on the signal output from the waveform shaping circuit 34 and holds the count value which is the counting result. The signals supplied from the vertical scanning circuit unit 40 to the processing circuit 36 ​​via the control line 14 may include an enable signal for controlling the pulse counting period (exposure period) and a reset signal for resetting the count value held by the processing circuit 36. Figure 3 shows, as an example, a reset signal (control signal PRES) supplied via the signal line 14A. The output node of the processing circuit 36 ​​is connected to the selection circuit 38.

[0056] The selection circuit 38 has the function of switching the electrical connection state (connected or disconnected) between the processing circuit 36 ​​and the output line 16. The selection circuit 38 switches the connection state between the processing circuit 36 ​​and the output line 16 in accordance with the selection signal supplied from the vertical scanning circuit unit 40 via the control line 14 (in the configuration example of Figure 2, the selection signal supplied from the horizontal scanning circuit unit 60 via the control line 18). Figure 3 shows an example of a selection signal (control signal PSEL) supplied via the signal line 14B. The processing circuit 36 ​​may include a buffer circuit for outputting signals.

[0057] The photoelectric conversion device 100 according to this embodiment may be formed on a single substrate, or it may be configured as a stacked type photoelectric conversion device by stacking multiple substrates. In the latter case, for example, as shown in Figure 4, it can be configured as a stacked type photoelectric conversion device in which a sensor substrate 120 and a circuit board 130 are stacked and electrically connected. At least the photoelectric conversion unit 20, which is a component of the pixel 12, can be arranged on the sensor substrate 120. The signal processing unit 30, which is a component of the pixel 12, can be arranged on the circuit board 130. The photoelectric conversion unit 20 and the signal processing unit 30 are electrically connected via connecting wiring provided for each pixel 12. Furthermore, the circuit board 130 may also include a vertical scanning circuit unit 40, a readout circuit unit 50, a horizontal scanning circuit unit 60, an output circuit unit 70, a weight control unit 80, and a control pulse generation unit 90.

[0058] The photoelectric conversion unit 20 and signal processing unit 30 for each pixel 12 may be provided on the sensor substrate 120 and the circuit board 130 so as to overlap in a plan view. The vertical scanning circuit unit 40, the readout circuit unit 50, the horizontal scanning circuit unit 60, the output circuit unit 70, the weight control unit 80, and the control pulse generation unit 90 may be arranged around the pixel unit 10, which is composed of multiple pixels 12. Here, "plan view" refers to viewing from a direction perpendicular to the surface of the sensor substrate 120.

[0059] By configuring a stacked photoelectric converter 100, the integration density of elements can be increased, leading to improved functionality. In particular, by arranging the photoelectric conversion unit 20 and the signal processing unit 30 on separate substrates, the photoelectric conversion elements 22 can be arranged at high density without sacrificing the light-receiving area of ​​the photoelectric conversion elements 22, thereby improving photon detection efficiency.

[0060] Furthermore, the number of substrates constituting the photoelectric converter 100 is not limited to two; the photoelectric converter 100 may be constructed by stacking three or more substrates. For example, when the photoelectric converter 100 is constructed by stacking three substrates, the photoelectric conversion unit 20, one of the components of the pixel 12, can be placed on the sensor substrate. Alternatively, the functional block 30A, one of the components of the pixel 12, can be placed on the first circuit board, and the functional block 30B, one of the components of the pixel 12, can be placed on the second circuit board. By separating the substrates on which the components are placed according to the characteristics of the elements constituting each functional block, a manufacturing process suitable for each element can be applied, thereby improving the performance of the photoelectric converter.

[0061] Furthermore, while Figure 4 assumes chips that have been diced as sensor substrate 120 and circuit board 130, the sensor substrate 120 and circuit board 130 are not limited to chips. For example, each of the sensor substrate 120 and circuit board 130 may be a wafer. Also, the sensor substrate 120 and circuit board 130 may be stacked in wafer form and then diced, or they may be made into chips and then stacked and bonded.

[0062] Next, the basic operation of the photoelectric conversion unit 20 in the photoelectric conversion device according to this embodiment will be explained using Figures 5 and 6. Figures 5 and 6 are diagrams illustrating the basic operation of the photoelectric conversion element 22, the quench element 32, and the waveform shaping circuit 34 in the photoelectric conversion device according to this embodiment. Figure 5 shows the operation when the quench element 32 is composed of passive elements, and Figure 6 shows the operation when the quench element 32 is composed of active elements. For the sake of simplicity, we will assume here that the waveform shaping circuit 34 is composed of an inverter circuit.

[0063] First, the operation when the quench element 32 is composed of passive elements will be explained. When the quench element 32 is composed of passive elements, for example, the quench element 32 is composed of a resistive element or the quench element 32 is composed of a diode-connected MOS transistor. Figure 5(a) is a circuit diagram of the photoelectric conversion element 22, the quench element 32, and the waveform shaping circuit 34. Figure 5(b) shows the waveform of the signal at the input node (node ​​A) of the waveform shaping circuit 34. Figure 5(c) shows the waveform of the signal at the output node (node ​​B) of the waveform shaping circuit 34.

[0064] At time t0, a reverse bias voltage with a potential difference equivalent to (VH-VL) is applied to the photoelectric element 22. A reverse bias voltage sufficient to cause avalanche multiplication is applied between the anode and cathode of the APD constituting the photoelectric element 22, but when no photons are incident on the photoelectric element 22, there are no carriers that serve as seeds for avalanche multiplication. Therefore, avalanche multiplication does not occur in the photoelectric element 22, and no current flows through the photoelectric element 22.

[0065] At the following time t1, assume that a photon is incident on the photoelectric conversion element 22. When a photon is incident on the photoelectric conversion element 22, electron-hole pairs are generated by photoelectric conversion, and avalanche multiplication occurs using these carriers as a seed, causing an avalanche multiplication current to flow through the photoelectric conversion element 22. This avalanche multiplication current flows through the quench element 32, causing a voltage drop across the quench element 32, and the voltage at node A begins to drop. When the voltage drop at node A becomes large and reaches approximately Vex, avalanche multiplication stops at time t3, and the voltage level at node A stops dropping further. The potential at which avalanche multiplication stops is approximately 0V, that is, the potential at which the voltage across the photoelectric conversion element 22 is approximately Vbd.

[0066] When the avalanche multiplication in the photoelectric conversion element 22 stops, a current flows from the node with voltage VH to node A via the quench element 32 to compensate for the voltage drop, and the voltage at node A gradually increases. Subsequently, at time t5, node A settles back to its original voltage level.

[0067] The waveform shaping circuit 34 binarizes the signal input from node A according to a predetermined threshold and outputs it from node B. Specifically, the waveform shaping circuit 34 outputs a low-level signal from node B when the voltage level at node A exceeds the threshold, and outputs a high-level signal from node B when the voltage level at node A is below the threshold. For example, as shown in Figure 4(b), suppose the voltage at node A was below the threshold during the period from time t2 to time t4. In this case, as shown in Figure 5(c), the signal level at node B is low during the period from time t0 to time t2 and from time t4 to time t5, and high during the period from time t2 to time t4.

[0068] Thus, the analog signal input from node A is waveform-shaped into a digital signal by the waveform shaping circuit 34. The pulse signal output from the waveform shaping circuit 34 in response to the incidence of photons on the photoelectric conversion element 22 is the photon detection signal.

[0069] Next, the operation when the quench element 32 is composed of an active element will be described. When the quench element 32 is composed of an active element, for example, it may be composed of a MOS transistor that operates on an external control signal. Figure 6(a) is a circuit diagram of the photoelectric conversion element 22, the quench element 32, and the waveform shaping circuit 34. Figure 6(b) shows the waveform of the signal at the input node (node ​​C) of the quench element 32. Figure 6(c) shows the waveform of the signal at the input node (node ​​A) of the waveform shaping circuit 34. Figure 6(d) shows the waveform of the signal at the output node (node ​​B) of the waveform shaping circuit 34.

[0070] In the circuit shown in Figure 6(a), the quench element 32 is made up of a P-type MOS transistor. The source of the P-type MOS transistor is connected to the voltage VH node, and the drain of the P-type MOS transistor is connected to the cathode of the photoelectric conversion element 22 and the input node of the waveform shaping circuit 34. Node A is the connection node between the drain of the P-type MOS transistor, the cathode of the photoelectric conversion element 22, and the input node of the waveform shaping circuit 34. A periodic pulse signal (hereinafter referred to as signal PCLKB) is input to the gate (node ​​C) of the P-type MOS transistor, for example, as shown in Figure 6(b). Signal PCLKB includes a periodic falling-edge pulse signal that transitions from a high level to a low level. The P-type MOS transistor turns off when node C is high, disconnecting node A from the voltage VH node, and turns on when node C is low, resetting node A to voltage VH.

[0071] At time t1, when the pulse signal PCLKB is input to node C, the P-type MOS transistor turns on, and node A is reset to voltage VH. When node C returns to high level, the P-type MOS transistor turns off, and node A becomes floating at voltage VH.

[0072] At the following time t2, assume that a photon is incident on the photoelectric conversion element 22. When a photon is incident on the photoelectric conversion element 22, electron-hole pairs are generated by photoelectric conversion, and avalanche multiplication occurs using these carriers as a seed, causing an avalanche multiplication current to flow through the photoelectric conversion element 22. This avalanche multiplication current flows through the quench element 32, causing a voltage drop across the quench element 32, and the voltage at node A begins to drop. When the voltage drop at node A becomes large and reaches approximately Vex, avalanche multiplication stops at time t4, and the voltage level at node A stops dropping further. The potential at which avalanche multiplication stops is approximately 0V, that is, the potential at which the voltage across the photoelectric conversion element 22 is approximately Vbd. Node A remains in a floating state at the lowered potential.

[0073] When the pulse signal PCLKB is input to node C again at time t5, the P-type MOS transistor turns on, and node A is reset to voltage VH again.

[0074] The waveform shaping circuit 34 binarizes the signal input from node A according to a predetermined threshold and outputs it from node B. Specifically, the waveform shaping circuit 34 outputs a low-level signal from node B when the voltage level of node A exceeds the threshold, and outputs a high-level signal from node B when the voltage level of node A is below the threshold. For example, as shown in Figure 6(c), suppose the voltage of node A was below the threshold during the period from time t3 to time t5. In this case, as shown in Figure 6(d), the signal level at node B is low during the period from time t0 to time t3 and from time t5 onwards, and high during the period from time t3 to time t5.

[0075] Thus, the analog signal input from node A is waveform-shaped into a digital signal by the waveform shaping circuit 34. The pulse signal output from the waveform shaping circuit 34 in response to the incidence of photons on the photoelectric conversion element 22 is the photon detection pulse signal.

[0076] In the operation shown in Figure 6, if no photons are incident between time t1 and time t5, node A remains at voltage VH and node B remains at a low level. Furthermore, if a photon is incident again between time t2 and time t5, i.e., when the potential of node A has decreased, the photoelectric conversion element 22 cannot undergo further avalanche multiplication. Therefore, the maximum number of photon incidents that can be detected by the waveform shaping circuit 34 during the period from time t1 to time t5 is one.

[0077] As shown in Figure 6, the operation can distinguish whether the number of photons incident during one period of the PCLKB pulse signal is zero or one or more. On the other hand, if two or more photons are incident during one period of the PCLKB pulse signal, it is not possible to distinguish between them, which may result in signal detection loss.

[0078] When photons are incident one after another without interruption, the operation shown in Figure 5 results in a state where the avalanche multiplication in the photoelectric conversion element 22 does not stop, and the avalanche current continues to flow, a so-called pile-up state. In this pile-up state, no photon detection pulse signal is generated, and only current flows unnecessarily, which is one of the challenges in SPAD operation.

[0079] On the other hand, in the operation shown in Figure 6, it is not possible to distinguish between two or more photons if they are incident during one period of the PCLKB pulse signal, but it has the advantage of not causing the pile-up that occurs in the operation shown in Figure 5. While signal detection loss can certainly occur in the operation shown in Figure 6, SPADs are primarily intended for photon detection in situations with few incident photons, and in such situations, it is rare for multiple photons to be incident during one period of the reset pulse, so there is almost no signal detection loss. For this reason, the operation shown in Figure 6 is often applied, especially when using SPADs as image sensors.

[0080] Next, the schematic configuration of the weight control unit 80 and the pixels 12 in the photoelectric conversion device according to this embodiment will be described with reference to Figures 7 and 8. Figure 7 is a functional block diagram showing the schematic configuration of the weight control unit 80 in the photoelectric conversion device according to this embodiment. Figure 8 is a functional block diagram showing the schematic configuration of the pixels 12 in the photoelectric conversion device according to this embodiment.

[0081] As shown in Figure 7, the weight control unit 80 includes an imaging pulse generation unit 84, a time correlation pulse generation unit 86, and an AND circuit LC1. The weight control unit 80 has the function of processing periodic signals, including periodic pulse signals such as a clock signal CLK, according to the first weight information and the second weight information, and outputting them as the first weight signal and the second weight signal. Here, the first weight information includes information on the weighting amount for the imaging signal, and the second weight information includes information on the weighting amount for the time correlation signal. In this embodiment, the unit exposure period (main frame period) of the photoelectric conversion device is composed of a plurality of sub-exposure periods (sub-frame periods). The first weight information and the second weight information include information on the weighting amount in each of the plurality of sub-frame periods that constitute the main frame period.

[0082] The imaging pulse generation unit 84 generates a first weight signal, for example, by performing a decimation process on the clock signal CLK according to the first weight information. Here, the first weight information may include the same weighting amount for each of a plurality of subframe periods. The first weight signal may include a number of pulse signals within each subframe period corresponding to the weighting amount according to the first weight information.

[0083] The time correlation pulse generation unit 86 generates a multiplied clock signal MPCLK by multiplying the clock signal CLK at a multiplier corresponding to the second weight information, for example. The multiplied clock signal MPCLK does not necessarily have to be generated by the time correlation pulse generation unit 86; it may also be generated by the vertical scanning circuit unit 40 or the control pulse generation unit 90. For multiplying the clock signal CLK, a PLL (Phase Locked Loop) circuit, for example, may be used. In this case, the time correlation pulse generation unit 86 can multiply the clock signal CLK such that, for example, the assertion period of the first weight signal contains a number of pulse signals corresponding to the weighting amount according to the second weight information. The second weight information may be, for example, information that assigns a weighting amount based on a periodic function, such as a sine function or cosine function, to each subframe period.

[0084] The AND gate LC1 generates a second weight signal by performing a logical OR operation between the first weight signal and the multiplier clock signal MPCLK. That is, the AND gate LC1 performs decimation on the multiplier clock signal MPCLK so that the second weight signal is asserted when the first weight signal and the multiplier clock signal MPCLK are asserted. The time correlation pulse generation unit 86 can, for example, multiply the clock signal CLK so that the assertion period of the first weight signal contains a number of pulse signals corresponding to the weighting amount according to the second weight information. The second weight information may be, for example, information that assigns a weighting amount based on a periodic function, such as a sine function or a cosine function, to each subframe period.

[0085] The AND gate LC1 generates the second weight signal by performing a logical OR operation between the first weight signal and the multiplier clock signal MPCLK. In other words, the AND gate LC1 performs decimation on the multiplier clock signal MPCLK so that the second weight signal is asserted when both the first weight signal and the multiplier clock signal MPCLK are asserted. Assertion means that the signal is in an active state, typically when the signal is at a high level. Conversely, negate means that the signal is in an inactive state, typically when the signal is at a low level.

[0086] The weight control unit 80 only needs to have the function of generating a first weight signal corresponding to the first weight information and a second weight signal corresponding to the second weight information, and the configuration of the internal circuit is not limited to the example above. Also, the function of the weight control unit 80 may be provided by each of the multiple pixels 12.

[0087] As shown in Figure 8, the pixel 12 has a light-receiving unit 122, a first integrating unit 124, and a second integrating unit 126. The light-receiving unit 122 corresponds to the photoelectric conversion unit 20 and the functional block 30A in the pixel circuit of Figure 3. The first integrating unit 124 and the second integrating unit 126 correspond to the processing circuit 36 ​​in the pixel circuit of Figure 3.

[0088] The light-receiving unit 122 receives a first weight signal, a second weight signal, and control signals PRC1 and PRC2, and has the function of outputting a predetermined number of pulse signals in accordance with the incidence of light during the exposure period. The control signals PRC1 and PRC2 can be supplied from the vertical scanning circuit unit 40 or the control pulse generation unit 90. Note that the light-receiving unit 122 only needs to have the function of outputting pulse signals in accordance with the incidence of light, and is not necessarily limited to the configuration described in this embodiment.

[0089] The first integrator 124 has the function of counting pulse signals output from the light receiving unit 122 in accordance with the first weight signal and the control signal PRC1. The second integrator 126 has the function of counting pulse signals output from the light receiving unit 122 in accordance with the second weight signal and the control signal PRC2. The count values ​​in the first integrator 124 and the count values ​​in the second integrator 126 are output from the pixel 12 as the first pixel value and the second pixel value, respectively. By repeatedly performing the light receiving operation in the light receiving unit 122 and the counting operation in the first integrator 124 and the second integrator 126 a predetermined number of times, it becomes possible to image the subject. Details of the control signals PRC1 and PRC2 will be described later.

[0090] Next, the specific structure of the pixel 12 in the photoelectric conversion device according to this embodiment will be described with reference to Figure 9. Figure 9 is a circuit diagram showing an example of the configuration of the pixel 12 in the photoelectric conversion device according to this embodiment.

[0091] As shown in Figure 9, the pixel 12 of this embodiment includes a photoelectric conversion element 22, a quench element 32, a waveform shaping circuit 34, a NAND circuit LC2, AND circuits LC3, LC4, LC6, LC7, a flip-flop circuit FF1, and integration circuits 361, 362. Of these, the photoelectric conversion element 22, the quench element 32, the waveform shaping circuit 34, the NAND circuit LC2, and the AND circuits LC3, LC6 correspond to the light receiving unit 122. The flip-flop circuit FF1, the AND circuit LC4, and the integration circuit 361 correspond to the first integration unit 124, and the flip-flop circuit FF1, the AND circuit LC7, and the integration circuit 362 correspond to the second integration unit 126. Note that in Figure 9, for the sake of simplicity, the flip-flop circuit FF1 is not included in the components of the first integration unit 124 and the second integration unit 126.

[0092] The first weight signal is input to one input node of the NAND circuit LC2 and one input node of the AND circuit LC3. The control signal PRC1 is input to the other input terminal of the NAND circuit LC2 and the clock input terminal of the flip-flop circuit FF1. The inverted signal of control signal PRC1 is input to the other input node of the AND circuit LC3. The second weight signal is input to one input node of the AND circuit LC6. The inverted signal of control signal PRC2 is input to the other input node of the AND circuit LC6. In addition, the control signal PRES is input to the reset terminals of the flip-flop circuit FF1, the integrator circuit 361 and the integrator circuit 362. The control signal PRES is a control signal output from the vertical scanning circuit unit 40 or the control pulse generation unit 90.

[0093] In the circuit shown in Figure 9, the quench element 32 is constructed using a P-type MOS transistor. The source of the P-type MOS transistor is connected to the node with voltage VH. The drain of the P-type MOS transistor is connected to the cathode of the photoelectric conversion element 22 and the input node of the waveform shaping circuit 34. The gate of the P-type MOS transistor is connected to the output node of the NAND circuit LC2. The output signal of the NAND circuit LC2, which is input to the gate of the P-type MOS transistor, is the signal PCLKB. The anode of the photoelectric conversion element 22 is connected to the node with voltage VL.

[0094] The flip-flop circuit FF1 is a D-type flip-flop circuit having an input terminal D, an output terminal Q, a reset terminal R, and a clock input terminal. The output node of the waveform shaping circuit 34 is connected to the input terminal D of the flip-flop circuit FF1. The output terminal Q of the flip-flop circuit FF1 and the output node of the AND circuit LC3 are connected to the input node of the AND circuit LC4. The output signal of the AND circuit LC3 is signal TCLK1. Also, the output terminal Q of the flip-flop circuit FF1 and the output node of the AND circuit LC6 are connected to the input node of the AND circuit LC7. The output signal of the AND circuit LC6 is signal TCLK2. The output node of the AND circuit LC4 is connected to the integrator circuit 361. Also, the output node of the AND circuit LC7 is connected to the integrator circuit 362.

[0095] As described above, the unit exposure period (main frame period) in the photoelectric conversion device of this embodiment is composed of multiple sub-exposure periods (sub-frame periods). The control signal PRC1 is a pulse signal that transitions to a high level at the start of each of the multiple sub-frame periods, and after transitioning to a low level, it maintains a low level until the end of the sub-frame period. The control signal PRES is a pulse signal that transitions to a high level at the start of the main frame, and after transitioning to a low level, it maintains a low level until the end of the main frame period. The control signal PRC2 will be described later.

[0096] The NAND circuit LC2 outputs a low-level signal when both the control signal PRC1 and the first weight signal are at a high level, and outputs a high-level signal otherwise. Here, the signal PCLKB, which is the output signal of the NAND circuit LC2, corresponds to the signal PCLKB explained in the operation example in Figure 6. That is, when the signal PCLKB goes low, the P-type MOS transistor constituting the quench element 32 turns on, and the recharge operation of the photoelectric conversion element 22 is performed. In other words, the recharge operation of the photoelectric conversion element 22 is performed during the period when the assertion period of the control signal PRC1 and the assertion period of the first weight signal overlap. In this sense, the control signal PRC1 can be said to be a recharge signal for controlling the recharge operation of the quench element 32 together with the first weight signal. The NAND circuit LC2 is a logic circuit that performs the recharge operation of the photoelectric conversion element 22 during the period when the first weight signal and the control signal PRC1 (the inverted signal of the first control signal) are asserted. The recharge operation of the photoelectric conversion element 22 is performed only once in each subframe period.

[0097] The output signal of the waveform shaping circuit 34 is input to input terminal D of the flip-flop circuit FF1. The control signal PRC1 is input to the clock input terminal of the flip-flop circuit FF1. The control signal PRES is input to the reset terminal R of the flip-flop circuit FF1. The output signal of the flip-flop circuit FF1 becomes low level in accordance with the rising edge of the control signal PRES. Also, the output signal of the flip-flop circuit FF1 becomes the same level as the input signal in accordance with the rising edge of the control signal PRC1 input to the clock input terminal.

[0098] The AND circuit LC3 is a logic circuit that outputs a first weight signal to the first integrator 124 during the assertion period of the inverted signal (first control signal) of the control signal PRC1. The AND circuit LC4 is a logic circuit that outputs the first weight signal from the AND circuit LC3 to the integrator 361 during the assertion period of the output signal of the flip-flop circuit FF1. The AND circuit LC4 generates a pulse signal, which is the result of a logical AND operation between the output signal of the AND circuit LC3 (signal TCLK1) and the output signal of the flip-flop circuit FF1, and outputs it to the integrator 361. The integrator 361 integrates the pulse signals output from the AND circuit LC4 and holds the integrated value (first integrated value). The control signal PRES is also input to the integrator 361. The integrated value of the integrator 361 is initialized to its initial value upon receiving a high-level control signal PRES.

[0099] The AND circuit LC6 is a logic circuit that outputs a second weight signal to the second integrator 126 during the assertion period of the inverted signal (second control signal) of the control signal PRC2. The AND circuit LC7 is a logic circuit that outputs the second weight signal from the AND circuit LC6 to the integrator 362 during the assertion period of the output signal of the flip-flop circuit FF1. The AND circuit LC7 generates a pulse signal, which is the result of a logical AND operation between the output signal of the AND circuit LC6 (signal TCLK2) and the output signal of the flip-flop circuit FF1, and outputs it to the integrator 362. The integrator 362 integrates the pulse signals output from the AND circuit LC7 and holds the integrated value (second integrated value). The control signal PRES is also input to the integrator 362. The integrated value of the integrator 362 is initialized to its initial value upon receiving a high-level control signal PRES.

[0100] In the pixel 12 of this embodiment, the output signal of the waveform shaping circuit 34 becomes high level when a photon is incident on the photoelectric conversion element 22. In addition to this transition of the output signal of the waveform shaping circuit 34 to a high level, the output signal of the flip-flop circuit FF1 becomes high level when the control signal PRC1 also transitions to a high level. The state in which the output signal of the flip-flop circuit FF1 is high level continues until the next subframe period begins. While the output signal of the flip-flop circuit FF1 is high level, each time the pulse signal of the first weight signal transitions from low level to high level (each time the signal TCLK1 becomes high level), the output of the AND circuit LC4 transitions from low level to high level. As a result, the integration circuit 361 integrates the number of times the signal TCLK1 transitioned from low level to high level after the output signal of the flip-flop circuit FF1 transitioned to a high level.

[0101] In other words, the signal of the integration circuit 361 generated when a single photon is incident is weighted by the signal TCLK1. This allows the integration circuit 361 to perform weighted integration for a single incident photon. As mentioned above, the signal TCLK1 is generated based on a weighting such that more integration is performed if the photon is received earlier. Therefore, the value of the integration circuit 361 inside the first integration unit 124 (hereinafter referred to as the first integrated value) is a value that correlates with light visible to the human eye.

[0102] Meanwhile, the AND circuit LC7 generates a pulse signal, which is the result of a logical AND operation between the output signal of the AND circuit LC6 (signal TCLK2) and the output signal of the flip-flop circuit FF1, and outputs it to the integrator circuit 362. As mentioned above, since the signal TCLK2 is generated based on weighting corresponding to the sinusoidal or cosine wave component, the value of the integrator circuit 362 inside the second integrator unit 126 (hereinafter referred to as the second integrated value) becomes the pixel value of the time correlation for calculating the optical flow. Note that the bit width of the integrator circuit 361 and the bit width of the integrator circuit 362 may be the same or different.

[0103] Figure 10 shows the relationship between mainframe periods and subframe periods, and an example of how the weighting amount changes over time. In Figure 10, the horizontal axis represents time, and the vertical axis represents the weighting amount set for each frame period.

[0104] As shown in Figure 10, the main frame period, which is the unit exposure period for generating one frame image, is divided into multiple sub-exposure periods (sub-frame periods). The second weight information input to the weight control unit 80 includes information about the weighting amount set for each sub-frame period. The weighting amount can be set based on a periodic function in which time is a variable and the main frame period is one period. In other words, this periodic function has a different phase for each sub-frame period, and a different weighting amount is set for each of these phases.

[0105] The periodic function used to set the weighting amount is not particularly limited, but could be a sine function, for example. By weighting using a weighting amount based on a sine function, a signal corresponding to equation (17) can be generated. Alternatively, the periodic function used to set the weighting amount may be a cosine function. By weighting using a weighting amount based on a cosine function, a signal corresponding to equation (16) can be generated. Note that one subframe period can be further divided into multiple microframe periods.

[0106] The weight control unit 80 sets a second weighting amount corresponding to each subframe period based on the relationship between the phase and the weighting amount in the periodic function used to set the weighting amount. This allows the integration circuit 362 to generate a signal TCLK2 corresponding to the weighting of each subframe period, enabling time correlation imaging.

[0107] The first weighting amount, set based on the first weighting information, may be the same for each subframe period. The first weight signal, set based on the first weighting amount, is then subjected to decimation according to the photon incidence timing. Decimation according to the photon incidence timing will be described later.

[0108] Next, the operation of the pixel 12 in the photoelectric converter according to this embodiment will be explained in more detail using Figures 11 to 13. Figures 11 to 13 are timing diagrams showing the operation of the pixel 12 in the photoelectric converter according to this embodiment.

[0109] Figure 11 is a timing diagram showing the integration operation of the first integration unit 124 (integration circuit 361). Figure 11 shows the waveforms of the control signal PRC1, the first weight signal, signals PCLKB and TCLK1, the input signal DFF(D) and output signal DFF(Q) of the flip-flop circuit FF1, the timing of photon incidence, and the first integrated value. Figure 12 shows the waveforms of the signal TCLK1, the multiplier clock signal, and the second weight signal, as well as the second integrated value. For the multiplier clock signal MPCLK, the second weight signal, and the second integrated value, the cases where the value of the periodic function (cosine function) used for weighting is 1.000, 0.875, 0.125, and 0.000 are shown.

[0110] First, at time t10, the vertical scanning circuit unit 40 or the control pulse generation unit 90 asserts the control signal PRC1 (controlling it from a low level to a high level). This starts the first subframe period of the mainframe period. At this time, the first weight signal is at a low level, and the signal PCLKB, which is the output signal of the NAND circuit LC2, is at a high level. At time t10, the flip-flop circuit FF1 and the integrator circuit 361 are in a reset state, that is, the input terminal D and output terminal Q of the flip-flop circuit FF1 are at a low level, and the integrated value (first integrated value) of the integrator circuit 361 is 0.

[0111] At the following time t11, the first weight signal is asserted under the control of the weight control unit 80. This causes the signal PCLKB to transition from a high level to a low level, turning on the P-type MOS transistor that constitutes the quench element 32, and performing a recharge operation of the photoelectric conversion element 22. This recharge operation is performed only once at the start of each subframe period.

[0112] At the following time t12, assume that a photon is incident on the photoelectric converter 22 for the first time in this subframe period. Then, an avalanche multiplication current flows using the electron-hole pair generated in the photoelectric converter 22 as a seed, and the potential of the input node of the waveform shaping circuit 34 decreases. At the following time t13, when the potential of the input node of the waveform shaping circuit 34 falls below the judgment threshold, the output node of the waveform shaping circuit 34 transitions from a low level to a high level, and the input terminal D of the flip-flop circuit FF1 also becomes high level. Note that from time t13 until time t14 when the control signal PRC1 is asserted next, the photoelectric converter 22 is not recharged, so the signal level of the input terminal D of the flip-flop circuit FF1 does not change during this period, regardless of whether a photon is incident or not. Furthermore, from time t12 onward, under the control of the weight control unit 80, the first weight signal is asserted a number of times corresponding to the first weight information, but the output terminal Q of the flip-flop circuit FF1 is at a low level, and the integration operation in the integration circuit 361 is not performed.

[0113] If no photons are incident during the subframe period from time t10 to time t14, the input terminal D of the flip-flop circuit FF1 at time t14 will be low level, and the output terminal Q of the flip-flop circuit FF1 at time t14 will also remain low level. Therefore, the state at time t14 will be the same as at time t10, and the same operation as from time t10 to time t14 will be repeated in the next subframe period.

[0114] At the following time t14, the vertical scanning circuit unit 40 or the control pulse generation unit 90 asserts the control signal PRC1. This initiates the next subframe period. At this time, since the control signal PRC1 is also input to the clock input terminal of the flip-flop circuit FF1, the signal level of the output terminal Q of the flip-flop circuit FF1 transitions from a low level to a high level, triggered by the rising edge of the control signal PRC1.

[0115] At the following time t15, the first weight signal is asserted under the control of the weight control unit 80. This causes the signal PCLKB to transition from a high level to a low level, turning on the P-type MOS transistor that constitutes the quench circuit, and performing a recharge operation of the photoelectric conversion element 22. When the photoelectric conversion element 22 is recharged, the input node of the waveform shaping circuit 34 becomes high level, the output node of the waveform shaping circuit 34 transitions from a high level to a low level, and the input terminal D of the flip-flop circuit FF1 also becomes low level.

[0116] At the following time t16, assume that a photon is incident on the photoelectric converter 22 for the first time in this subframe period. Then, an avalanche multiplication current flows using the electron-hole pair generated in the photoelectric converter 22 as a seed, and the potential of the input node of the waveform shaping circuit 34 decreases. At the following time t17, when the potential of the input node of the waveform shaping circuit 34 falls below the judgment threshold, the output node of the waveform shaping circuit 34 transitions from a low level to a high level, and the input terminal D of the flip-flop circuit FF1 also becomes high level. Note that from time t17 until time t24 when the control signal PRC1 next transitions to a high level, the photoelectric converter 22 is not recharged, so the signal level of the input terminal D of the flip-flop circuit FF1 does not change during this period, regardless of whether a photon is incident or not.

[0117] At the following times t18, t19, t20, t21, t22, and t23, the first weight signal is asserted under the control of the weight control unit 80. During the period from time t18 to time t23, the control signal PRC1 is at a low level, so the signal TCLK1, which is the output signal of the AND circuit LC3, has the same waveform as the first weight signal. Also, during the period from time t18 to time t23, the signal level of the output terminal Q of the flip-flop circuit FF1 is at a high level, so the signal TCLK1 is input directly to the integrator circuit 361 as the output signal of the AND circuit LC4. The integrator circuit 361 integrates the number of pulse signals input from the AND circuit LC4. As a result, the integrated value (first integrated value) of the integrator circuit 361 after receiving the sixth rising pulse at time t23 is 6.

[0118] At the following time t24, the vertical scanning circuit unit 40 or the control pulse generation unit 90 asserts the control signal PRC1. This initiates the next subframe period. At this time, the control signal PRC1 is also input to the clock input terminal of the flip-flop circuit FF1, but since the signal level of the output terminal Q of the flip-flop circuit FF1 is the same high level as the signal level of the input terminal D, the signal level of the output terminal Q does not change.

[0119] At the following time t25, the first weight signal is asserted under the control of the weight control unit 80. This causes the signal PCLKB to transition from a high level to a low level, turning on the P-type MOS transistor constituting the quench element 32, and performing a recharge operation of the photoelectric conversion element 22. When the photoelectric conversion element 22 is recharged, the input node of the waveform shaping circuit 34 becomes high level, the output node of the waveform shaping circuit 34 transitions from a high level to a low level, and the input terminal D of the flip-flop circuit FF1 also becomes low level.

[0120] At the following times t26, t27, t28, t30, t31, and t32, the first weight signal is asserted under the control of the weight control unit 80. During the period from time t26 to time t32, the control signal PRC1 is at a low level, so the signal TCLK1, which is the output signal of the AND circuit LC3, has the same waveform as the first weight signal. Also, during the period from time t26 to time t32, the signal level of the output terminal Q of the flip-flop circuit FF1 is at a high level, so the signal TCLK1 is input directly to the integrator circuit 361 as the output signal of the AND circuit LC4. The integrator circuit 361 integrates the number of pulse signals input from the AND circuit LC4. As a result, the integrated value (first integrated value) of the integrator circuit 361 after receiving the sixth rising pulse at time t32 is 12.

[0121] Assume that at time t29, between time t26 and time t32, a photon is incident on the photoelectric converter 22 for the first time during this subframe period. Then, an avalanche multiplication current flows using the electron-hole pairs generated in the photoelectric converter 22 as a seed, and the potential of the input node of the waveform shaping circuit 34 decreases. At the following time t17, when the potential of the input node of the waveform shaping circuit 34 falls below the judgment threshold, the output node of the waveform shaping circuit 34 transitions from a low level to a high level, and the input terminal D of the flip-flop circuit FF1 also becomes high level. However, since the output terminal Q of the flip-flop circuit FF1 is maintained at a high level during this subframe period, the integration circuit 361 integrates all the pulse signals of the signal TCLK1 input during this subframe period, regardless of the timing of the photon incidence.

[0122] If no photons are incident during the subframe period from time t14 to time t24, the input terminal D of the flip-flop circuit FF1 will be low at time t24, and the output terminal Q of the flip-flop circuit FF1 will transition to a low level at time t24. Therefore, the state at time t24 will be the same as the state at time t10, and the same operation as from time t10 to time t14 will occur in the next subframe period.

[0123] At the following time t33, the vertical scanning circuit unit 40 or the control pulse generation unit 90 asserts the control signal PRC1. This initiates the next subframe period. Thereafter, the same operation is repeatedly performed for each subframe period. In other words, the integration circuit 361 integrates the number of pulse signals superimposed on the signal TCLK1 in the next subframe period, in accordance with the number of photons detected in the previous frame period. The first integrated value thus generated is, as mentioned above, a value correlated with light visible to the human eye.

[0124] Next, using the operation during the period from time t18 to time t19 as an example, the integration operation in the second integration unit 126 will be explained using Figure 12. Figure 12 is a timing diagram showing the integration operation of the second integration unit 126 (integration circuit 362) during the period from time t18 to time t19 in Figure 11. As mentioned above, the integrated value (second integrated value) of the integration circuit 362 is a time-correlated value for calculating optical flow.

[0125] As a prerequisite, we will explain how to represent the decimal values ​​of trigonometric functions such as sine and cosine functions in digital circuits in order to calculate the time correlation value based on incident photons. The values ​​of trigonometric functions vary between -1.0 and 1.0. On the other hand, increasing the number of decimal places is important for improving the accuracy of optical flow. Based on this, in order to represent the values ​​of trigonometric functions on a digital circuit, we will use a fixed-point number with an offset of 1 and an arbitrary number of bits as an example. For example, if we take a 1-bit integer and a 3-bit fixed-point number, the binary number 1000 (decimal 8) can be interpreted as 1.0. Similarly, the binary number 0111 (decimal 7) can be interpreted as 0.875, the binary number 0110 (decimal 6) as 0.75, and the binary number 0110 (decimal 5) as 0.625.

[0126] Figure 12 shows the multiplier clock signal MPCLK, the second weight signal, and the second integrated value for each of the following cases where the value of the cosine function is 1.000, 0.875, 0.125, and 0.000. The weight control unit 80 defines the period during which the second weight signal is asserted based on the multiplier clock signal MPCLK and the first weight signal as the period from time t18 to time t180. When the weighting amount according to the value of the cosine function is expressed in 8 levels, the second weight signal can include, for example, 8 pulses when the value of the cosine function is 1.000, 7 pulses when it is 0.875, 1 pulse when it is 0.125, and 0 pulses when it is 0.000. The multiplier clock signal MPCLK can be generated in the time correlation pulse generation unit by appropriately changing the frequency of the clock signal CLK according to the second weight information. In other words, the period of the clock signal CLK can be changed so that the number of pulses that rise during one assertion period of signal TCLK1 is a predetermined number corresponding to the second weight information. Alternatively, the frequency of the clock signal CLK can be controlled so that one assertion period of signal TCLK1 contains the maximum number of pulses corresponding to the second weight information, and then the number of pulses that rise during the assertion period of signal TCLK1 can be controlled by decimating some of the pulses.

[0127] The AND gate LC6 generates the signal TCLK2 in accordance with the second weight signal and the control signal PRC2 asserted between time t18 and time t180. The integrator circuit 362 integrates the number of pulse signals output from the AND gate LC7 in accordance with the output signal of the flip-flop circuit FF1 and the signal TCLK2. For example, if the value of the cosine function is 1.000 (cosθ=1.000), the number of pulse signals input to the integrator circuit 362 between time t18 and time t180 is 8, and the integrated value (second integrated value) of the integrator circuit 362 is 8. Similarly, the integrated values ​​(second integrated values) of the integrator circuit 362 in the cases of cosθ=0.875, cosθ=0.125, and cosθ=0.000 are 7, 1, and 0, respectively.

[0128] Note that the configuration of the weight control unit 80 and the method for generating the first and second weight signals shown in Figure 7 are just examples. The first and second weight signals are pulses that ultimately produce a second integrated value that correlates with the second weighting to the first integrated value. For example, if the frequency of signal TCLK2 becomes fast and power consumption and operating speed become a concern, the second weight signal may be generated such that the number of assertions of signal TCLK2 according to the second weight information is obtained regardless of the assert period of signal TCLK1. The assert period and negate period of the first and second weight signals can also be changed as appropriate.

[0129] Thus, in this embodiment, depending on the detection of a photon in the previous subframe period, it is possible to weight the next subframe period with the number of pulse signals TCLK1 and TCLK2. Here, the weighting offset value and amplitude are described as 1 and the fixed-point number is 3 bits, but these can be changed as appropriate.

[0130] In this embodiment, a method for integrating the number of weighted signals TCLK1 and TCLK2 has been described, but the method is not limited to this method as long as it is an imaging method using an avalanche photodiode that can obtain a time-correlated image. In addition, an integration unit separate from the first integration unit 124 and the second integration unit 126 may be added to further integrate pulse signals based on a third weighting different from the first and second weightings. Also, in Figures 8 and 9, the first and second weighting signals are input to the light receiving unit 122, but the first and second weighting signals may be input to the first integration unit 124 and the second integration unit 126, respectively, to achieve the same integration function. Furthermore, the relationship between the value of the trigonometric function solution and the number of pulses is not limited to the above example, and the bit width of the sine and cosine functions and the number of subframe periods may be arbitrarily changed.

[0131] Next, a method for achieving time-correlated imaging while suppressing an increase in the circuit size of the second integration unit by controlling the drive timing of the control signals PRC1 and PRC2 will be explained using Figure 13. Figure 13 is a timing diagram showing the operation of the pixel 12 in the photoelectric converter according to this embodiment.

[0132] Figure 13 shows the operation during mainframe period MF1, which starts at time t30 and ends at time t34. Figure 13 shows the waveform of control signal PRC1 and the subframe period in the first integration unit, and the waveform of control signal PRC2 and the subframe period in the second integration unit. For the waveform of control signal PRC2 and the subframe period in the second integration unit, two types of operation with different control patterns (control pattern 1 and control pattern 2) are shown. Assuming a frame rate of 30fps, the length of the period from time t30 to time t34 is 33.3ms.

[0133] Here, we assume that the mainframe period MF1 is divided into 2048 subframe periods SF1 to SF2048. In Figure 13, time t31 is the start time of the second subframe period SF2, time t32 is the start time of the fifth subframe period SF5, and time t33 is the start time of the ninth subframe SF9. As mentioned above, the control signal PRC1 is asserted only once during each subframe period. Focusing on subframe period SF1, the control signal PRC1 is asserted once between time t30 and time t31.

[0134] For example, if the maximum value of the first integrated value in one subframe period is 8, then the maximum value of the first integrated value in one mainframe period will be 16384 (= 8 × 2048). In order for the first integrated unit 124 to be able to integrate up to 16384, an integrated circuit 361 with a bit width of 14 bits or more is required.

[0135] On the other hand, if the values ​​of trigonometric functions are represented as 1-bit integers and 3-bit fixed-point numbers as described above, the second integrated value will have eight times more gradations, and the maximum value of the second integrated value in one subframe period can be 64. However, when integrating the time correlation values ​​of one period in one mainframe period, the maximum value of 64 will not be integrated over all subframe periods, and the integrated value per subframe period will at most be the average value of the sine or cosine function.

[0136] For example, if the offset of the sine function and the cosine function are both 1 and the amplitude is 1, the average value will be 1, and the average value of the integrated values ​​over one mainframe period can be considered to be 32. Therefore, the maximum value of the second integrated value over one mainframe period will be 65536 (= 32 × 2048). In order for the second integrated unit 126 to be able to integrate up to 65536, an integrated circuit 362 with a bit width of 16 bits or more is required. Increasing the bit width of integrated circuits 361 and 362 leads to an increase in the circuit size of the pixel 12, and consequently to an increase in the size of the photoelectric converter. Reducing the bit width of integrated circuits 361 and 362 can suppress the increase in circuit size, but it will result in a decrease in resolution.

[0137] Control patterns 1 and 2 shown in Figure 13 are examples of drives designed to reduce the maximum value of the second integrated value during a single mainframe period.

[0138] Control pattern 1 is a drive example in which the subframe period of the second integrator 126 is executed once for every eight executions of the subframe period of the first integrator 124. In this case, the control signal PRC2 is controlled in the same way as the control signal PRC1 during the period from time t30 to time t31, as shown in Figure 13, for example, and controlled at a high level from time t31 to time t33.

[0139] Here, the inverted signal of control signal PRC1 is defined as the first control signal, and the inverted signal of control signal PRC2 is defined as the second control signal. In this case, the first weight signal is output to the first integrator 124 during the high-level assertion period of the first control signal, and the second weight signal is output to the second integrator 126 during the high-level assertion period of the second control signal. In other words, the integration operation in the first integrator 124 is performed during the assertion period of the first control signal, and the integration operation in the second integrator 126 is performed during the assertion period of the second control signal. Therefore, by controlling control signals PRC1 and PRC2 as described above, the number of assertions of the second control signal in the mainframe period MF1 is less than the number of assertions of the first control signal in the mainframe period MF1.

[0140] As a result, during the period from time t30 to time t33, the first integrator 124 executes eight subframe periods SF1 to SF8, while the second integrator 126 executes one subframe period SF1. By repeatedly performing the same operation as during the period from time t30 to time t33 during the subsequent period from time t33 to time 34, the number of subframe periods executed by the second integrator 126 during the mainframe period MF1 can be reduced to one-eighth. As a result, the maximum value of the second integrated value in one mainframe period becomes 8192 (=65536 / 8), taking into account the average values ​​of the sine and cosine functions.

[0141] The reduction in the subframe period in the second integrator 126 is not particularly limited, but can be set according to the number of bits used to represent the fixed-point number of the periodic function, for example. For example, if the fixed-point number is represented by 3 bits as in the example above, the bit width of the integrator circuit 362 can be reduced by the amount corresponding to the number of bits of the fixed-point number by reducing the number of subframe periods in the second integrator 126 to one-eighth.

[0142] Control pattern 2 is a drive example in which the subframe period of the second integrator 126 is executed once for every four executions of the subframe period of the first integrator 124. In this case, the control signal PRC2 is controlled in the same way as the control signal PRC1 during the period from time t30 to time t31, as shown in Figure 13, for example, and controlled at a high level from time t31 to time t32. In this case as well, the number of assertions of the second control signal in the mainframe period MF1 is less than the number of assertions of the first control signal in the mainframe period MF1.

[0143] As a result, during the period from time t30 to time t32, the first integrator 124 executes four subframe periods SF1 to SF4, while the second integrator 126 executes one subframe period SF1. By repeatedly performing the same operation as during the period from time t30 to time t32 during the subsequent period from time t32 to time 34, the number of subframe periods executed by the second integrator 126 during the mainframe period MF1 can be reduced to one-quarter. As a result, the maximum value of the second integrated value in one mainframe period becomes 16384 (=65536 / 4), taking into account the average values ​​of the sine and cosine functions.

[0144] In addition to reducing the subframe period in the second integration unit 126, the control may also be configured to change the second weight signal. For example, if the maximum value of the integrated value in one subframe period of the second integration unit 126 was 64, the second weight signal can be changed so that the maximum value of the integrated value becomes 32. By applying this control to control pattern 2, the period during which imaging is not performed in one mainframe period can be reduced compared to control pattern 1, while reducing the maximum value of the second integrated value in one mainframe period to the same level as in control pattern 1.

[0145] Note that control patterns 1 and 2 are illustrative examples, and the control method of the photoelectric converter according to this embodiment is not limited to these. For example, it is not necessarily required to control the maximum value of the second integrated value to be smaller than the maximum value of the first integrated value; the maximum value of the second integrated value may be controlled to be smaller, or it may be controlled to be larger. Also, the assert period and negate period of the control signals PRC1 and PRC2 shown in Figure 13 are not limited to these, and it is sufficient as long as the number of subframe periods in the first integrated unit 124 and the number of subframe periods in the second integrated unit 126 are different. In other words, the assert timing of the control signals PRC1 and PRC2 may be controlled separately. Furthermore, it is not necessarily required to reduce the subframe period in the second integrated unit 126; it is sufficient to simply make the maximum value of the integrated value during the subframe period in the second integrated unit 126 different from the maximum value of the integrated value during the subframe period in the first integrated unit 124.

[0146] Thus, according to the photoelectric conversion device of this embodiment, time-correlated imaging can be realized. Furthermore, since the photoelectric conversion device of this embodiment uses an APD as the photoelectric conversion element, time-correlated imaging can also be suitably performed in low-luminance shooting scenes. In addition, in the photoelectric conversion device of this embodiment, the output of the signal from the light receiving unit to the first integrating unit is controlled by a first control signal, and the output of the signal from the light receiving unit to the second integrating unit is controlled by a second control signal, and the number of assertions of the second control signal is made less than the number of assertions of the first control signal. Therefore, according to the photoelectric conversion device of this embodiment, time-correlated imaging can be realized while reducing the circuit size of the pixels.

[0147] [Second Embodiment] A photoelectric conversion device according to a second embodiment of the present invention will be described with reference to Figures 14 and 15. Figure 14 is a functional block diagram showing the schematic configuration of the pixel 12 in the photoelectric conversion device according to this embodiment. Figure 15 is a circuit diagram showing an example of the configuration of the pixel 12 in the photoelectric conversion device according to this embodiment. Components similar to those in the photoelectric conversion device according to the first embodiment are denoted by the same reference numerals, and their descriptions are omitted or simplified.

[0148] The photoelectric converter according to this embodiment is the same as the photoelectric converter according to the first embodiment, except for the configuration of the pixels 12. In this embodiment, the differences between the photoelectric converter of this embodiment and the photoelectric converter of the first embodiment will be described, and the similarities with the photoelectric converter of the first embodiment will be omitted as appropriate.

[0149] In the first embodiment of the photoelectric converter, as shown in Figure 7, each of the multiple pixels 12 constituting the pixel section 10 had a light-receiving section 122, a first integrating section 124, and a second integrating section 126. In contrast, in the photoelectric converter according to this embodiment, as shown in Figure 14, the multiple pixels 12 constituting the pixel section 10 include a pixel 12A having a light-receiving section 122 and a first integrating section 124, and a pixel 12B having a light-receiving section 122 and a second integrating section 126. Pixels 12A and 12B can be arranged adjacent to each other.

[0150] In this embodiment, each pixel 12 has only one of the first integrating unit 124 and the second integrating unit 126. Therefore, the area of ​​the pixel circuit can be reduced compared to a pixel 12 having both the first integrating unit 124 and the second integrating unit 126. This makes it possible to reduce the size of the photoelectric conversion device.

[0151] Pixel 12A can be composed of, for example, a photoelectric conversion element 22, a quench element 32, a waveform shaping circuit 34, a flip-flop circuit FF1, a NAND circuit LC2, AND circuits LC3 and LC4, and an integration circuit 361, as shown in Figure 15. Of these, the photoelectric conversion element 22, the quench element 32, the waveform shaping circuit 34, the NAND circuit LC2, and the AND circuit LC3 correspond to the light receiving unit 122, while the flip-flop circuit FF1, the AND circuit LC4, and the integration circuit 361 correspond to the first integration unit 124.

[0152] The first weight signal is input to one input node of the NAND circuit LC2 and one input node of the AND circuit LC3. The control signal PRC1 is input to the other input terminal of the NAND circuit LC2 and the clock input terminal of the flip-flop circuit FF1. The inverted signal of the control signal PRC1 is input to the other input node of the AND circuit LC3. In addition, the control signal PRES is input to the reset terminals of the flip-flop circuit FF1 and the integrator circuit 361.

[0153] The source of the P-type MOS transistor constituting the quench element 32 is connected to the node with voltage VH. The drain of the P-type MOS transistor is connected to the cathode of the photoelectric conversion element 22 and the input node of the waveform shaping circuit 34. The gate of the P-type MOS transistor is connected to the output node of the NAND circuit LC2. The output signal of the NAND circuit LC2 input to the gate of the P-type MOS transistor is the signal PCLKB. The anode of the photoelectric conversion element 22 is connected to the node with voltage VL.

[0154] The output node of the waveform shaping circuit 34 is connected to the input terminal D of the flip-flop circuit FF1. The output terminal Q of the flip-flop circuit FF1 and the output node of the AND circuit LC3 are connected to the input node of the AND circuit LC4. The output node of the AND circuit LC4 is connected to the integration circuit 361.

[0155] Furthermore, pixel 12B can be composed of, for example, a photoelectric conversion element 22, a quench element 32, a waveform shaping circuit 34, a flip-flop circuit FF2, a NAND circuit LC5, AND circuits LC6 and LC7, and an integration circuit 362, as shown in Figure 15. Of these, the photoelectric conversion element 22, the quench element 32, the waveform shaping circuit 34, the NAND circuit LC5, and the AND circuit LC6 correspond to the light receiving unit 122, while the flip-flop circuit FF2, the AND circuit LC7, and the integration circuit 362 correspond to the second integration unit 126.

[0156] The second weight signal is input to one input node of the NAND circuit LC5 and one input node of the AND circuit LC6. The control signal PRC2 is input to the other input terminal of the NAND circuit LC5 and the clock input terminal of the flip-flop circuit FF2. The inverted signal of the control signal PRC2 is input to the other input node of the AND circuit LC6. In addition, the control signal PRES is input to the reset terminals of the flip-flop circuit FF2 and the integrator circuit 362.

[0157] The source of the P-type MOS transistor constituting the quench element 32 is connected to the node with voltage VH. The drain of the P-type MOS transistor is connected to the cathode of the photoelectric conversion element 22 and the input node of the waveform shaping circuit 34. The gate of the P-type MOS transistor is connected to the output node of the NAND circuit LC5. The output signal of the NAND circuit LC5 input to the gate of the P-type MOS transistor is the signal PCLKB. The anode of the photoelectric conversion element 22 is connected to the node with voltage VL.

[0158] The NAND circuit LC5 outputs a low-level signal when both the control signal PRC2 and the second weight signal are at a high level, and outputs a high-level signal otherwise. Here, the signal PCLKB, which is the output signal of the NAND circuit LC5, corresponds to the signal PCLKB described in the operation example in Figure 6. That is, when the signal PCLKB goes low, the P-type MOS transistor constituting the quench element 32 turns on, and the recharge operation of the photoelectric conversion element 22 is performed. In other words, the recharge operation of the photoelectric conversion element 22 is performed during the period when the assertion period of the control signal PRC2 and the assertion period of the second weight signal overlap. In this sense, the control signal PRC2 in this embodiment is also a recharge signal for controlling the recharge operation of the quench element 32 together with the second weight signal.

[0159] The output node of the waveform shaping circuit 34 is connected to the input terminal D of the flip-flop circuit FF2. The output terminal Q of the flip-flop circuit FF2 and the output node of the AND circuit LC6 are connected to the input node of the AND circuit LC7. The output node of the AND circuit LC7 is connected to the integration circuit 362.

[0160] The NAND circuit LC5 outputs a low-level signal when both the control signal PRC2 and the second weight signal are at a high level, and outputs a high-level signal otherwise. Here, the signal PCLKB, which is the output signal of the NAND circuit LC5, corresponds to the signal PCLKB explained in the operation example in Figure 6. That is, when the signal PCLKB goes low, the P-type MOS transistor constituting the quench element 32 of the pixel 12B turns on, and the recharge operation of the photoelectric conversion element 22 of the pixel 12B is performed. The NAND circuit LC5 is a logic circuit that performs the recharge operation of the photoelectric conversion element 22 of the pixel 12B during the period when the second weight signal and the control signal PRC2 (the inverted signal of the second control signal) are asserted. The recharge operation of the photoelectric conversion element 22 is performed only once during each subframe period.

[0161] The output signal of the waveform shaping circuit 34 for pixel 12B is input to input terminal D of the flip-flop circuit FF2. The control signal PRC2 is input to the clock input terminal of the flip-flop circuit FF2. The control signal PRES is input to the reset terminal R of the flip-flop circuit FF2. The output signal of the flip-flop circuit FF2 becomes low level in accordance with the rising edge of the control signal PRES. Also, the output signal of the flip-flop circuit FF2 becomes the same level as the input signal in accordance with the rising edge of the control signal PRC2 input to the clock input terminal.

[0162] The AND circuit LC7 is a logic circuit that outputs a second weight signal from the AND circuit LC6 to the integrator circuit 362 during the assertion period of the output signal of the flip-flop circuit FF2. The AND circuit LC7 generates a pulse signal, which is the result of a logical AND operation between the output signal of the AND circuit LC6 (signal TCLK2) and the output signal of the flip-flop circuit FF2, and outputs it to the integrator circuit 362.

[0163] In this embodiment, the first integrated value of the integration circuit 361 and the second integrated value of the integration circuit 362 are generated by integrating pulse signals based on separate photoelectric conversion elements 22. Specifically, the input to the AND circuit LC4, which outputs a pulse signal to the integration circuit 361, is the output signal of the flip-flop circuit FF1, which holds the output of the light-receiving section 122 of the pixel 12A. In contrast, the input to the AND circuit LC7, which outputs a pulse signal to the integration circuit 362, is the output signal of the flip-flop circuit FF2, which holds the output of the light-receiving section 122 of the pixel 12B.

[0164] Thus, according to the photoelectric conversion device of this embodiment, time-correlated imaging can be realized. Furthermore, since the photoelectric conversion device of this embodiment uses an APD as the photoelectric conversion element, time-correlated imaging can also be suitably performed in low-luminance shooting scenes. In addition, in the photoelectric conversion device of this embodiment, the output of the signal from the light receiving unit to the first integrating unit is controlled by a first control signal, and the output of the signal from the light receiving unit to the second integrating unit is controlled by a second control signal, and the number of assertions of the second control signal is less than the number of assertions of the first control signal. Therefore, according to the photoelectric conversion device of this embodiment, time-correlated imaging can be realized while reducing the size of the pixel circuit. Furthermore, according to the photoelectric conversion device of this embodiment, the area of ​​the pixel circuit can be reduced compared to the first embodiment, and the size of the photoelectric conversion device can be further reduced.

[0165] [Third Embodiment] A photoelectric conversion system according to a third embodiment of the present invention will be described with reference to Figure 16. Figure 16 is a block diagram showing the schematic configuration of the photoelectric conversion system according to this embodiment.

[0166] The photoelectric conversion device 100 described in the first and second embodiments above is applicable to various photoelectric conversion systems. Examples of applicable photoelectric conversion systems include digital still cameras, digital camcorders, surveillance cameras, photocopiers, fax machines, mobile phones, in-vehicle cameras, and observation satellites. Camera modules, which include optical systems such as lenses and imaging devices, are also included in photoelectric conversion systems. Figure 16 shows a block diagram of a digital still camera as an example of these.

[0167] The photoelectric conversion system 200 illustrated in Figure 16 includes an imaging device 201, a lens 202 for forming an optical image of a subject onto the imaging device 201, an aperture 204 for varying the amount of light passing through the lens 202, and a barrier 206 for protecting the lens 202. The lens 202 and aperture 204 are an optical system that focuses light onto the imaging device 201. The imaging device 201 is the photoelectric conversion device 100 described in the first or second embodiment, which converts the optical image formed by the lens 202 into image data.

[0168] The photoelectric conversion system 200 also includes a signal processing unit 208 that processes the output signal output from the imaging device 201. The signal processing unit 208 generates image data from the digital signal output by the imaging device 201. The signal processing unit 208 also performs various corrections and compressions as needed before outputting the image data. The imaging device 201 may include an AD conversion unit that generates the digital signal processed by the signal processing unit 208. The AD conversion unit may be formed on the semiconductor layer (semiconductor substrate) on which the photoelectric conversion unit of the imaging device 201 is formed, or it may be formed on a semiconductor layer different from the semiconductor layer on which the photoelectric conversion unit of the imaging device 201 is formed. Alternatively, the signal processing unit 208 may be formed on the same semiconductor layer as the imaging device 201.

[0169] The photoelectric conversion system 200 further includes a memory unit 210 for temporarily storing image data and an external interface unit (external I / F unit) 212 for communicating with an external computer or the like. Furthermore, the photoelectric conversion system 200 includes a recording medium 214 such as a semiconductor memory for recording or reading imaging data, and a recording medium control interface unit (recording medium control I / F unit) 216 for recording or reading data from the recording medium 214. The recording medium 214 may be built into the photoelectric conversion system 200 or it may be detachable.

[0170] Furthermore, the photoelectric conversion system 200 includes an overall control and calculation unit 218 that controls various calculations and the entire digital still camera, and a timing generation unit 220 that outputs various timing signals to the imaging device 201 and the signal processing unit 208. Here, the timing signals and the like may be input from an external source, and the photoelectric conversion system 200 only needs to include at least an imaging device 201 and a signal processing unit 208 that processes the output signals output from the imaging device 201.

[0171] The imaging device 201 outputs an imaging signal to the signal processing unit 208. The signal processing unit 208 performs predetermined signal processing on the imaging signal output from the imaging device 201 and outputs image data. The signal processing unit 208 generates an image using the imaging signal.

[0172] Thus, according to this embodiment, a photoelectric conversion system can be realized by applying the photoelectric conversion device 100 according to the first and second embodiments.

[0173] [Fourth Embodiment] A photoelectric conversion system and mobile body according to a fourth embodiment of the present invention will be described with reference to Figure 17. Figure 17 is a diagram showing the configuration of the photoelectric conversion system and mobile body according to this embodiment.

[0174] Figure 17(a) shows an example of a photoelectric conversion system related to an in-vehicle camera. The photoelectric conversion system 300 has an imaging device 310. The imaging device 310 is the photoelectric conversion device 100 described in the first or second embodiment above. The photoelectric conversion system 300 has an image processing unit 312 that performs image processing on a plurality of image data acquired by the imaging device 310, and a parallax acquisition unit 314 that calculates parallax (phase difference of parallax images) from a plurality of image data acquired by the photoelectric conversion system 300.

[0175] Here, the photoelectric conversion system 300 may include an optical system (not shown) that guides light to the photoelectric conversion device 100, such as a lens, shutter, or mirror. Furthermore, multiple photoelectric conversion units, each substantially conjugate to the pupil of the optical system, may be arranged in pixels of the photoelectric conversion device 100. For example, multiple photoelectric conversion units substantially conjugate to the pupil may be arranged corresponding to a single microlens. The multiple photoelectric conversion units receive light beams transmitted through different positions in the pupil of the optical system, thereby outputting image data corresponding to the light beams transmitted through different positions in the photoelectric conversion device 100. The parallax acquisition unit 314 may then calculate the parallax using the output image data.

[0176] The photoelectric conversion system 300 also includes a distance acquisition unit 316 that calculates the distance to an object based on the calculated parallax, and a collision determination unit 318 that determines whether or not there is a possibility of collision based on the calculated distance. Here, the parallax acquisition unit 314 and the distance acquisition unit 316 are examples of distance information acquisition means that acquire distance information to an object. That is, distance information is information related to parallax, defocus amount, distance to an object, etc. The collision determination unit 318 may use any of this distance information to determine the possibility of collision. Note that the distance information may be acquired using ToF (Time of Flight) technology. The distance information acquisition means may be implemented by specially designed hardware, or by a software module. It may also be implemented by FPGA (Field Programmable Gate Array), ASIC (Application Specific Integrated Circuit), etc., or by a combination thereof.

[0177] The photoelectric conversion system 300 is connected to the vehicle information acquisition device 320 and can acquire vehicle information such as vehicle speed, yaw rate, and steering angle. The photoelectric conversion system 300 is also connected to a control ECU 330, which is a control device that outputs a control signal to generate braking force on the vehicle based on the judgment result of the collision judgment unit 318. The photoelectric conversion system 300 is also connected to a warning device 340 that issues a warning to the driver based on the judgment result of the collision judgment unit 318. For example, if the collision judgment result of the collision judgment unit 318 indicates a high probability of collision, the control ECU 330 performs vehicle control to avoid a collision or mitigate damage by applying the brakes, releasing the accelerator, or suppressing engine output. The warning device 340 warns the user by sounding an alarm, displaying warning information on a screen such as a car navigation system, or vibrating the seat belt or steering wheel.

[0178] In this embodiment, the photoelectric conversion system 300 images the area around the vehicle, for example, the front or rear. Figure 17(b) shows the photoelectric conversion system when imaging the area in front of the vehicle (imaging range 350). The vehicle information acquisition device 320 sends instructions to the photoelectric conversion system 300 or the imaging device 310. This configuration can further improve the accuracy of distance measurement.

[0179] The above example illustrates control to prevent collisions with other vehicles, but it can also be applied to control systems that automatically follow other vehicles or automatically drive to prevent vehicles from straying from their lanes. Furthermore, the photoelectric conversion system can be applied not only to vehicles such as the vehicle itself, but also to mobile objects (mobile devices) such as ships, aircraft, or industrial robots. In addition, it can be applied not only to mobile objects but also to a wide range of devices that utilize object recognition, such as intelligent transportation systems (ITS).

[0180] [Fifth Embodiment] A fifth embodiment of the present invention will be described with reference to Figure 18. Figure 18 is a block diagram showing the schematic configuration of the device according to this embodiment.

[0181] Figure 18 is a schematic diagram showing an instrument EQP including a photoelectric converter APR. The photoelectric converter APR has the functions of a photoelectric converter 100 of the first or second embodiment. All or part of the photoelectric converter APR is a semiconductor device IC. The photoelectric converter APR in this example can be used, for example, as an image sensor, an AF (Auto Focus) sensor, a photometering sensor, or a distance measuring sensor. The semiconductor device IC has a pixel area PX in which pixel circuits PXC including a photoelectric conversion unit are arranged in a matrix. The semiconductor device IC may have a peripheral area PR around the pixel area PX. Circuits other than pixel circuits can be arranged in the peripheral area PR.

[0182] The photoelectric converter APR may have a stacked structure (chip stacking structure) comprising a first semiconductor chip equipped with multiple photoelectric conversion units and a second semiconductor chip equipped with peripheral circuits. The peripheral circuits on the second semiconductor chip can each be a column circuit corresponding to a pixel row of the first semiconductor chip. Alternatively, the peripheral circuits on the second semiconductor chip can each be a matrix circuit corresponding to a pixel or pixel block of the first semiconductor chip. Connections between the first and second semiconductor chips can be made using through-silicon vias (TSVs), direct bonding of conductors such as copper for inter-chip wiring, microbump connections between chips, or wire bonding.

[0183] The photoelectric converter APR may include a semiconductor device IC as well as a package PKG that houses the semiconductor device IC. The package PKG may include a substrate on which the semiconductor device IC is fixed, a cover made of glass or the like that faces the semiconductor device IC, and connecting members such as bonding wires or bumps that connect terminals provided on the substrate to terminals provided on the semiconductor device IC.

[0184] The EQP device may further comprise at least one of the following: an optical device OPT, a control unit CTRL, a processing unit PRCS, a display unit DSPL, a memory device MMRY, and a mechanical device MCHN. The optical device OPT corresponds to the photoelectric converter APR as a photoelectric converter, and is, for example, a lens, shutter, or mirror. The control unit CTRL controls the photoelectric converter APR and is, for example, a semiconductor device such as an ASIC.

[0185] The processing unit PRCS processes the signal output from the photoelectric converter APR and constitutes either an AFE (analog front end) or a DFE (digital front end). The processing unit PRCS is a semiconductor device such as a CPU (central processing unit) or an ASIC (application-specific integrated circuit). The display device DSPL is an EL display device or liquid crystal display device that displays the information (image) obtained from the photoelectric converter APR. The memory device MMRY is a magnetic device or semiconductor device that stores the information (image) obtained from the photoelectric converter APR. The memory device MMRY is a volatile memory such as SRAM or DRAM, or a non-volatile memory such as flash memory or a hard disk drive. Furthermore, the processing unit PRCS may acquire optical flow using the signal output from the photoelectric converter 100 in each of the embodiments described above. In other words, the processing unit PRCS may generate three images: an image with sinusoidal components, an image with cosine components, and a normal image, and acquire optical flow from these three images.

[0186] The mechanical device MCHN has moving parts or propulsion parts such as motors and engines. The equipment EQP displays the signals output from the photoelectric converter APR on the display device DSPL, or transmits them to the outside using a communication device (not shown) provided by the equipment EQP. For this purpose, it is preferable that the equipment EQP further includes a memory device MMRY and a processing device PRCS, separate from the memory circuit and arithmetic circuit of the photoelectric converter APR. The mechanical device MCHN may be controlled based on the signals output from the photoelectric converter APR.

[0187] The EQP device shown in Figure 18 can be an electronic device such as an information terminal with a shooting function (e.g., a smartphone or wearable device) or a camera (e.g., an interchangeable lens camera, a compact camera, a video camera, or a surveillance camera). In a camera, the MCHN mechanical unit can drive components of the OPT optical unit for zooming, focusing, and shutter operation. Alternatively, the MCHN mechanical unit in a camera can move the APR photoelectric converter for vibration damping.

[0188] Furthermore, the EQP (equipment) can be a transport device (mobile object) such as a vehicle, ship, or aircraft. The mechanical device MCHN (machine controller) in the transport device can be used as a mobile device. The EQP as a transport device is suitable for transporting the photoelectric converter APR (application present transformer) or for assisting and / or automating driving (operation) through its imaging function. The processing device PRCS (processing unit) for assisting and / or automating driving (operation) can perform processing to operate the mechanical device MCHN as a mobile device based on information obtained from the photoelectric converter APR.

[0189] Furthermore, the EQP equipment may also include medical equipment such as endoscopes and CT scanners, measuring instruments such as distance sensors, analytical instruments such as electron microscopes, office equipment such as photocopiers, and industrial equipment such as robots.

[0190] According to the photoelectric converter 100 of the embodiment described above, it is possible to obtain good pixel characteristics. Therefore, the value of the photoelectric converter can be increased. Increasing the value here means at least one of the following: addition of functions, improvement of performance, improvement of characteristics, improvement of reliability, improvement of manufacturing yield, reduction of environmental impact, cost reduction, miniaturization, and weight reduction.

[0191] Therefore, by using the photoelectric converter 100 of the above embodiment in an equipment EQP, the value of the equipment EQP can also be improved. For example, by mounting the photoelectric converter 100 on a transport vehicle, excellent performance can be obtained when taking external images of the transport vehicle or measuring the external environment. Therefore, when manufacturing and selling transport vehicles, deciding to mount the semiconductor device according to this embodiment on the transport vehicle is advantageous in improving the performance of the transport vehicle itself. In particular, the photoelectric converter 100 is suitable for transport vehicles that use information obtained from the semiconductor device to assist in driving and / or perform automatic driving.

[0192] [Modified Embodiment] The present invention is not limited to the embodiments described above and can be modified in various ways.

[0193] For example, an example in which a part of the configuration of one embodiment is added to another embodiment, or in which a part of the configuration of another embodiment is replaced, is also an embodiment of the present invention.

[0194] Furthermore, the disclosures in this specification include not only what is described herein, but also all matters that can be understood from this specification and the drawings attached thereto. The disclosures in this specification also include the complement of the concepts described herein. That is, if this specification contains a statement such as "A is greater than B," then even if the statement "A is not greater than B" is omitted, this specification can be said to disclose that "A is not greater than B." This is because the statement "A is greater than B" presupposes that the case where "A is not greater than B" is being considered.

[0195] Furthermore, the circuit configuration of the pixel 12 is not limited to the above embodiment. For example, a switch such as a transistor may be provided between the photoelectric conversion element 22 and the quench element 32, or between the photoelectric conversion element 22 and the signal processing unit 30, to control the electrical connection state between them. Alternatively, a switch such as a transistor may be provided between the node to which voltage VH is supplied and the quench element 32, and / or between the node to which voltage VL is supplied and the photoelectric conversion element 22, to control the electrical connection state between them.

[0196] Furthermore, the photoelectric conversion systems shown in the third and fourth embodiments above are merely examples of photoelectric conversion systems to which the photoelectric conversion device of the present invention can be applied, and the photoelectric conversion systems to which the photoelectric conversion device of the present invention can be applied are not limited to the configurations shown in Figures 16 and 17.

[0197] The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0198] It should be noted that the above embodiments are merely examples of how the present invention can be implemented, and the technical scope of the present invention should not be interpreted as being limited by them. In other words, the present invention can be implemented in various forms without departing from its technical concept or its main features.

[0199] The above-disclosed embodiments include the following configurations and methods. (Composition 1) A light-receiving unit having an avalanche photodiode and outputting a photon detection signal in response to the incidence of photons, A first integration unit integrates the photon detection signal with a first signal weighted based on first weight information, A second integration unit integrates the aforementioned photon detection signal with a second weighted signal based on second weighting information for time correlation weighting to calculate optical flow, The system includes a control unit that outputs a first control signal for controlling the output of the first signal from the light receiving unit to the first integration unit, and a second control signal for controlling the output of the second signal from the light receiving unit to the second integration unit. The number of assertions of the second control signal during a unit exposure period is less than the number of assertions of the first control signal during the unit exposure period. A photoelectric conversion device characterized by the following features. (Configuration 2) The second weight information is set based on the value of a periodic function in which one period is defined as a unit exposure period that includes multiple sub-exposure periods. The second weight information provided in each of the plurality of sub-exposure periods corresponds to the value of the periodic function in the phase corresponding to each of the plurality of sub-exposure periods. A photoelectric conversion device according to configuration 1, characterized by the features described above. (Composition 3) The above-mentioned first weight information is the same in each of the plurality of sub-exposure periods. A photoelectric conversion device according to configuration 2, characterized by the features described above. (Composition 4) The number of subframe periods in which the integration operation in the second integration unit is performed among the plurality of sub-exposure periods is less than the number of subframe periods in which the integration operation in the first integration unit is performed among the plurality of sub-exposure periods. A photoelectric conversion device according to configuration 2 or 3, characterized by the above. (Composition 5) The integration operation in the first integration unit is performed during the assertion period of the first control signal. The integration operation in the second integration unit is performed during the assertion period of the second control signal. The photoelectric conversion device according to configuration 4, characterized by the features described above. (Composition 6) The system further includes a weight signal generation unit that generates a first weight signal corresponding to the first weight information and a second weight signal corresponding to the first weight information and the second weight information. The weight signal generation unit outputs to the light receiving unit, in each of the plurality of sub-exposure periods, a first weight signal including a plurality of pulse signals corresponding to the first weight information and a second weight signal including a plurality of pulse signals corresponding to the second weight information. The light receiving unit outputs a first signal based on the first weight signal and a second signal based on the second weight signal in response to the incidence of photons. A photoelectric conversion device according to any one of configurations 2 to 5, characterized by the above. (Composition 7) The weight signal generation unit generates the first weight signal by performing a decimation process on a periodic signal that includes a periodic pulse signal during each of the plurality of sub-exposure periods. A photoelectric conversion device according to configuration 6, characterized by the features described above. (Composition 8) The first integrating unit, when a photon is incident during the immediately preceding sub-exposure period, integrates the number of pulse signals superimposed on the first signal input during the next sub-exposure period. A photoelectric conversion device according to configuration 6 or 7, characterized by the above. (Composition 9) The weight signal generation unit outputs the second weight signal according to each of the plurality of pulse signals of the first weight signal. A photoelectric conversion device according to any one of configurations 6 to 8, characterized by the above. (Composition 10) The weight signal generation unit generates the second weight signal, which includes the plurality of pulse signals, during the assertion period of each of the plurality of pulse signals of the first weight signal by multiplying the period signal input to the weight signal generation unit. A photoelectric conversion device according to any one of configurations 7 to 9, characterized by the above. (Composition 11) The bit width of the integration circuit in the second integration unit is set based on the average value of the periodic function. A photoelectric conversion device according to any one of configurations 2 to 10, characterized by the above. (Composition 12) The aforementioned periodic function is either a sine function or a cosine function. A photoelectric conversion device according to any one of configurations 2 to 11, characterized by the above. (Composition 13) The light-receiving unit includes a first light-receiving unit that outputs the first signal in response to the incidence of a photon onto the first avalanche photodiode, and a second light-receiving unit that outputs the second signal in response to the incidence of a photon onto the second avalanche photodiode. A photoelectric conversion device according to any one of configurations 1 to 5, characterized by the above. (Composition 14) The system further includes a weight signal generation unit that generates a first weight signal corresponding to the first weight information and a second weight signal corresponding to the first weight information and the second weight information. The light receiving unit is A waveform shaping circuit connected to the avalanche photodiode, A first logic circuit that performs a recharge operation of the avalanche photodiode during the period when the assertion period of the first weight signal and the assertion period of the third control signal overlap, A second logic circuit that outputs the first weight signal to the first integration unit during the assertion period of the first control signal, The device further includes a third logic circuit that outputs the second weight signal to the second integration unit during the assertion period of the second control signal. A photoelectric conversion device according to any one of configurations 1 to 5, characterized by the above. (Composition 15) The first integration unit includes a flip-flop circuit that receives the output signal of the waveform shaping circuit and whose output signal becomes the level of the output signal of the waveform shaping circuit according to the third control signal, a fourth logic circuit that outputs the first weight signal from the second logic circuit during the assertion period of the output signal of the flip-flop circuit, and a first integration circuit that integrates the first signal output from the fourth logic circuit. The second integration unit includes a fifth logic circuit that outputs the second weight signal from the third logic circuit during the assert period of the output signal of the flip-flop circuit, and a second integration circuit that integrates the second signal output from the fifth logic circuit. A photoelectric conversion device according to configuration 14, characterized by the features described above. (Composition 16) The system further includes a weight signal generation unit that generates a first weight signal corresponding to the first weight information and a second weight signal corresponding to the first weight information and the second weight information. The first light receiving unit is, A first waveform shaping circuit connected to the first avalanche photodiode, A first logic circuit that performs a recharge operation of the first avalanche photodiode during the period when the assertion period of the first weight signal and the assertion period of the third control signal overlap, The system further includes a second logic circuit that outputs the first weight signal to the first integration unit during the assertion period of the first control signal, The second light receiving unit is, A second waveform shaping circuit connected to the second avalanche photodiode, A third logic circuit that performs a recharge operation of the second avalanche photodiode during the period when the assertion period of the second weight signal and the assertion period of the fourth control signal overlap, The present invention further includes a fourth logic circuit that outputs the second weight signal to the second integration unit during the assertion period of the second control signal. A photoelectric conversion device according to configuration 13, characterized by the features described above. (Composition 17) The first integration unit includes a first flip-flop circuit that receives the output signal of the first waveform shaping circuit and whose output signal becomes the level of the output signal of the first waveform shaping circuit according to the third control signal, a fifth logic circuit that outputs the first weight signal from the second logic circuit during the assertion period of the output signal of the first flip-flop circuit, and a first integration circuit that integrates the first signal output from the fifth logic circuit. The second integration unit includes a second flip-flop circuit that receives the output signal of the second waveform shaping circuit and whose output signal becomes the level of the output signal of the second waveform shaping circuit according to the fourth control signal; a sixth logic circuit that outputs the second weight signal from the fourth logic circuit during the assertion period of the output signal of the second flip-flop circuit; and a second integration circuit that integrates the second signal output from the sixth logic circuit. A photoelectric conversion device according to configuration 16, characterized by the features described above. (Composition 18) A photoelectric conversion device according to any one of configurations 1 to 17, A signal processing device that processes the signal output from the aforementioned photoelectric converter and A photoelectric conversion system characterized by having the following features. (Composition 19) It is a mobile object, A photoelectric conversion device according to any one of configurations 1 to 17, Distance information acquisition means that acquires distance information to an object from a parallax image based on a signal from the aforementioned photoelectric converter, Control means for controlling the moving body based on the distance information A mobile body characterized by having the following features. (Composition 20) A photoelectric conversion device according to any one of configurations 1 to 17, Optical device corresponding to the aforementioned photoelectric converter, A control device for controlling the aforementioned photoelectric converter, A processing device that processes the signal output from the aforementioned photoelectric converter, A mechanical device controlled based on information obtained from the aforementioned photoelectric converter, A display device for displaying information obtained by the aforementioned photoelectric converter, and A memory device for storing information obtained by the aforementioned photoelectric converter, and at least one of the following: A device characterized by being equipped with the following features. (Composition 21) The processing unit acquires optical flow based on the signal output from the photoelectric converter. The apparatus according to configuration 20, characterized by the features described above. (Method 1) A method for driving a photoelectric converter, comprising: a light receiving unit having an avalanche photodiode and outputting a photon detection signal in response to the incidence of a photon; a first integration unit that integrates a first signal weighted based on first weighting information with respect to the photon detection signal; and a second integration unit that integrates a second signal weighted based on second weighting information relating to time correlation weighting for calculating optical flow with respect to the photon detection signal, wherein The output of the first signal from the light receiving unit to the first integration unit is controlled by the first control signal. The output of the second signal from the light receiving unit to the second integration unit is controlled by a second control signal having fewer assertions per unit exposure period than the first control signal. A method for driving a photoelectric converter, characterized by the following features. [Explanation of symbols]

[0200] 10...Pixel area 12... pixels 50...Readout circuit section 80...Weight control unit 122...Light receiving section 124... 1st Cost Estimation Department 126... Second Cost Estimation Department

Claims

1. A light-receiving unit having an avalanche photodiode and outputting a photon detection signal in response to the incidence of photons, A first integration unit integrates the photon detection signal with a first signal weighted based on first weight information, A second integration unit integrates a second signal that has been weighted based on second weighting information for time correlation weighting to calculate optical flow for the aforementioned photon detection signal, The system includes a control unit that outputs a first control signal for controlling the output of the first signal from the light receiving unit to the first integration unit, and a second control signal for controlling the output of the second signal from the light receiving unit to the second integration unit. The number of assertions of the second control signal during a unit exposure period is less than the number of assertions of the first control signal during the unit exposure period. A photoelectric conversion device characterized by the following features.

2. The second weight information is set based on the value of a periodic function in which one period is defined as a unit exposure period that includes multiple sub-exposure periods. The second weight information provided in each of the plurality of sub-exposure periods corresponds to the value of the periodic function in the phase corresponding to each of the plurality of sub-exposure periods. The photoelectric conversion device according to claim 1, characterized by the features described above.

3. The above-mentioned first weight information is the same in each of the plurality of sub-exposure periods. The photoelectric conversion device according to claim 2.

4. The number of subframe periods in which the integration operation in the second integration unit is performed among the plurality of sub-exposure periods is less than the number of subframe periods in which the integration operation in the first integration unit is performed among the plurality of sub-exposure periods. The photoelectric conversion device according to claim 2.

5. The integration operation in the first integration unit is performed during the assertion period of the first control signal. The integration operation in the second integration unit is performed during the assertion period of the second control signal. The photoelectric conversion device according to feature 4.

6. The system further includes a weight signal generation unit that generates a first weight signal corresponding to the first weight information and a second weight signal corresponding to the first weight information and the second weight information. The weight signal generation unit outputs to the light receiving unit, in each of the plurality of sub-exposure periods, a first weight signal including a plurality of pulse signals corresponding to the first weight information, and a second weight signal including a plurality of pulse signals corresponding to the second weight information. The light receiving unit outputs a first signal based on the first weighting signal and a second signal based on the second weighting signal in response to the incidence of photons. The photoelectric conversion device according to any one of claims 2 to 5.

7. The weight signal generation unit generates the first weight signal by performing a decimation process on a periodic signal that includes a periodic pulse signal during each of the plurality of sub-exposure periods. The photoelectric conversion device according to claim 6.

8. The first integrating unit, when a photon is incident during the immediately preceding sub-exposure period, integrates the number of pulse signals superimposed on the first signal input during the next sub-exposure period. The photoelectric conversion device according to claim 6.

9. The weight signal generation unit outputs the second weight signal according to each of the plurality of pulse signals of the first weight signal. The photoelectric conversion device according to claim 6.

10. The weight signal generation unit generates the second weight signal, which includes the plurality of pulse signals, during the assertion period of each of the plurality of pulse signals of the first weight signal by multiplying the period signal input to the weight signal generation unit. The photoelectric conversion device according to claim 7.

11. The bit width of the integration circuit in the second integration unit is set based on the average value of the periodic function. The photoelectric conversion device according to any one of claims 2 to 5.

12. The aforementioned periodic function is either a sine function or a cosine function. The photoelectric conversion device according to any one of claims 2 to 5.

13. The light-receiving unit includes a first light-receiving unit that outputs the first signal in response to the incidence of a photon onto the first avalanche photodiode, and a second light-receiving unit that outputs the second signal in response to the incidence of a photon onto the second avalanche photodiode. The photoelectric conversion device according to any one of claims 1 to 5.

14. The system further includes a weight signal generation unit that generates a first weight signal corresponding to the first weight information and a second weight signal corresponding to the first weight information and the second weight information. The light receiving unit is A waveform shaping circuit connected to the avalanche photodiode, A first logic circuit that performs a recharge operation of the avalanche photodiode during the period when the assertion period of the first weight signal and the assertion period of the third control signal overlap, A second logic circuit that outputs the first weight signal to the first integration unit during the assertion period of the first control signal, The device further includes a third logic circuit that outputs the second weight signal to the second integration unit during the assertion period of the second control signal. The photoelectric conversion device according to any one of claims 1 to 5.

15. The first integration unit includes a flip-flop circuit that receives the output signal of the waveform shaping circuit and whose output signal becomes the level of the output signal of the waveform shaping circuit according to the third control signal, a fourth logic circuit that outputs the first weight signal from the second logic circuit during the assertion period of the output signal of the flip-flop circuit, and a first integration circuit that integrates the first signal output from the fourth logic circuit. The second integration unit includes a fifth logic circuit that outputs the second weight signal from the third logic circuit during the assert period of the output signal of the flip-flop circuit, and a second integration circuit that integrates the second signal output from the fifth logic circuit. The photoelectric conversion device according to claim 14.

16. The system further includes a weight signal generation unit that generates a first weight signal corresponding to the first weight information and a second weight signal corresponding to the first weight information and the second weight information. The first light receiving unit is, A first waveform shaping circuit connected to the first avalanche photodiode, A first logic circuit that performs a recharge operation of the first avalanche photodiode during the period when the assertion period of the first weight signal and the assertion period of the third control signal overlap, The system further includes a second logic circuit that outputs the first weight signal to the first integration unit during the assertion period of the first control signal, The second light receiving unit is, The second waveform shaping circuit connected to the second avalanche photodiode, A third logic circuit that performs a recharge operation of the second avalanche photodiode during the period when the assertion period of the second weight signal and the assertion period of the fourth control signal overlap, The system further includes a fourth logic circuit that outputs the second weight signal to the second integration unit during the assertion period of the second control signal. The photoelectric conversion device according to claim 13, characterized in that it is a photoelectric conversion device.

17. The first integration unit includes a first flip-flop circuit that receives the output signal of the first waveform shaping circuit and whose output signal becomes the level of the output signal of the first waveform shaping circuit according to the third control signal, a fifth logic circuit that outputs the first weight signal from the second logic circuit during the assertion period of the output signal of the first flip-flop circuit, and a first integration circuit that integrates the first signal output from the fifth logic circuit. The second integration unit includes a second flip-flop circuit that receives the output signal of the second waveform shaping circuit and whose output signal becomes the level of the output signal of the second waveform shaping circuit according to the fourth control signal; a sixth logic circuit that outputs the second weight signal from the fourth logic circuit during the assertion period of the output signal of the second flip-flop circuit; and a second integration circuit that integrates the second signal output from the sixth logic circuit. The photoelectric conversion device according to claim 16, characterized in that it is a photoelectric conversion device.

18. A photoelectric conversion device according to any one of claims 1 to 5, A signal processing device that processes the signal output from the aforementioned photoelectric converter and A photoelectric conversion system characterized by having the following features.

19. It is a mobile object, A photoelectric conversion device according to any one of claims 1 to 5, Distance information acquisition means that acquires distance information to an object from a parallax image based on a signal from the aforementioned photoelectric converter, Control means for controlling the moving body based on the distance information A mobile body characterized by having the following features.

20. A photoelectric conversion device according to any one of claims 1 to 5, Optical device corresponding to the aforementioned photoelectric converter, A control device for controlling the aforementioned photoelectric converter, A processing device that processes the signal output from the aforementioned photoelectric converter, A mechanical device controlled based on information obtained from the aforementioned photoelectric converter, A display device for displaying information obtained by the aforementioned photoelectric converter, and A memory device for storing information obtained by the aforementioned photoelectric converter, and at least one of the following: A device characterized by being equipped with the following features.

21. The processing unit acquires optical flow based on the signal output from the photoelectric converter. The apparatus according to claim 20, characterized in that it is a device.

22. A method for driving a photoelectric converter, comprising: a light receiving unit having an avalanche photodiode and outputting a photon detection signal in response to the incidence of a photon; a first integration unit that integrates a first signal weighted based on first weighting information with respect to the photon detection signal; and a second integration unit that integrates a second signal weighted based on second weighting information relating to time correlation weighting for calculating optical flow with respect to the photon detection signal, wherein The output of the first signal from the light receiving unit to the first integration unit is controlled by the first control signal. The output of the second signal from the light receiving unit to the second integration unit is controlled by a second control signal having fewer assertions per unit exposure period than the first control signal. A method for driving a photoelectric converter, characterized by the following:

Images