Photoelectric conversion device, apparatus, and movable body
By alternating the operation of avalanche photodiodes in first and second pixels to calculate difference values, the circuit scale is reduced, enabling efficient detection of light changes without the need for extensive memory storage, addressing the complexity issue in existing APD-based devices.
Patent Information
- Application Number
- JP2023078040
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-10
- Publication Date
- 2024-11-21
AI Technical Summary
The existing photoelectric conversion devices using avalanche photodiodes (APDs) require large circuit scales to detect changes in the number of incident photons, necessitating storage of count values from previous exposure periods, which increases circuit complexity.
A photoelectric conversion device with alternating operation of first and second pixels, each equipped with an avalanche photodiode and a counter, where the photodiodes operate at different timings, allowing for the calculation of a difference value between count values during separate exposure periods, reducing circuit scale.
This approach reduces the circuit scale of the pixel circuit while effectively detecting changes in incident light by calculating difference values, thereby simplifying the design and improving efficiency.
Smart Images

Figure 2024162492000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to a photoelectric conversion device, an apparatus, and a mobile object. [Background technology]
[0002] Patent Document 1 discloses an asynchronous solid-state imaging element that operates in response to the detection of an event such as a change in the amount of light. In Patent Document 1, in response to a detection pixel equipped with a general photodiode that does not undergo avalanche breakdown detecting an event, a pixel equipped with an avalanche photodiode (APD) acquires a pixel signal that counts the number of photons incident on the APD within a predetermined exposure period. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2020-096347 A Summary of the Invention [Problem to be solved by the invention]
[0004] In contrast to the configuration shown in Patent Document 1, it is conceivable to use an APD in the detection pixel as well and detect an event from a change in the number of photons incident on the APD. However, in order to detect a change in the number of photons incident on the APD in the detection pixel, it is necessary to store a count value in at least one exposure period before the exposure period of interest in a memory or the like and compare it with the count value in the exposure period of interest, which may increase the circuit scale of the detection pixel.
[0005] An object of the present invention is to provide a technique that is advantageous in reducing the circuit scale of a pixel circuit. [Means for solving the problem]
[0006] In view of the above problems, a photoelectric conversion device according to an embodiment of the present invention is a photoelectric conversion device including a plurality of pixels including a first pixel and a second pixel, each of which includes an avalanche photodiode and a counter for counting the number of times avalanche breakdown occurs in the avalanche photodiode, and a processing circuit, wherein, during a period for detecting a change in the amount of light incident on the photoelectric conversion device, the first pixel and the second pixel operate the avalanche photodiode and the counter counts the number of times, and stop the operation of the avalanche photodiode and store the count value counted during the exposure period as a counter value. a holding period in which the count value is held in a counter, the first pixel and the second pixel alternately start the detection operation at different timings, a second exposure period in the exposure period of the second pixel starts a predetermined time after a start of a first exposure period in the exposure period of the first pixel, and the processing circuit outputs a difference value between a first count value counted in the first exposure period at the first pixel and held in the counter during a first holding period following the first exposure period in the holding period, and a second count value counted in the second pixel during the second exposure period. Effect of the Invention
[0007] According to the present invention, it is possible to provide a technique that is advantageous in reducing the circuit scale of a pixel circuit. [Brief description of the drawings]
[0008] [Figure 1] FIG. 1 is a block diagram showing an example of the configuration of a photoelectric conversion device according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram showing a modification of the photoelectric conversion device in FIG. [Diagram 3] 2 is a block diagram showing an example of the configuration of a pixel of the photoelectric conversion device of FIG. 1. [Figure 4] FIG. 2 is a perspective view showing a configuration example of the photoelectric conversion device in FIG. [Diagram 5] 2A to 2C are diagrams illustrating the operation of the photoelectric conversion device in FIG. 1. [Figure 6]2 is a block diagram showing an example of the configuration of a pixel of the photoelectric conversion device of FIG. 1. [Figure 7] 2 is a timing chart showing an example of driving a pixel of the photoelectric conversion device of FIG. 1. [Figure 8] 2 is a diagram showing an example of the configuration of a counter of a pixel of the photoelectric conversion device of FIG. 1; [Figure 9] 2 is a timing chart showing an example of driving a pixel of the photoelectric conversion device of FIG. 1. [Figure 10] FIG. 2 is a block diagram showing an example of pixel arrangement of the photoelectric conversion device of FIG. 1. [Figure 11] 2 is a timing chart showing an example of driving a pixel of the photoelectric conversion device of FIG. 1. [Figure 12] 2 is a timing chart showing an example of driving a pixel of the photoelectric conversion device of FIG. 1. [Figure 13] 2 is a block diagram showing an example of the configuration of a pixel of the photoelectric conversion device of FIG. 1. [Figure 14] 2 is a block diagram showing an example of the configuration of a pixel of the photoelectric conversion device of FIG. 1. [Figure 15] FIG. 1 is a diagram showing an example of the configuration of a device incorporating a photoelectric conversion device according to an embodiment of the present invention. [Figure 16] FIG. 1 is a diagram showing a configuration example of a moving object in which a photoelectric conversion device according to an embodiment of the present invention is incorporated. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0009] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.
[0010] A photoelectric conversion device according to an embodiment of the present disclosure will be described with reference to FIGS. 1 to 14. FIG. 1 is a block diagram showing a schematic configuration of a photoelectric conversion device 100 according to the present embodiment. As shown in FIG. 1, the photoelectric conversion device 100 may include a pixel section 10, a vertical scanning circuit 40, a readout circuit 50, a horizontal scanning circuit 60, a control pulse generating circuit 80, and an output circuit 90. Hereinafter, the photoelectric conversion device 100 will be described as an asynchronous type so-called imaging device in which an avalanche photodiode (APD) is provided in the pixel 12 arranged in the pixel section 10. However, the photoelectric conversion device according to the present embodiment is not limited to this. Examples of the photoelectric conversion device include a distance measuring device (a device that performs distance measurement using focus detection or Time Of Flight (TOF)), a photometric device (a device that measures the amount of incident light, etc.), and the like, in addition to an imaging device.
[0011] The pixel unit 10 includes a plurality of pixels 12 arranged in an array to form a plurality of rows and a plurality of columns. As described later, each pixel 12 may include a photoelectric conversion unit including an APD, which is a photon detection element, and a processing circuit for processing a signal output from the photoelectric conversion unit. The number of pixels 12 constituting the pixel unit 10 is not particularly limited. For example, the pixel unit 10 may be configured by a plurality of pixels 12 arranged in an array of several thousand rows and several thousand columns, as in a general digital camera. Also, the pixel unit 10 may be configured by a plurality of pixels 12 arranged in one row or one column. Furthermore, the pixel unit 10 may be configured by one pixel 12, for example.
[0012] A control line 14 extending in the row direction (horizontal direction in FIG. 1) is arranged in each row of the pixel array arranged in the pixel unit 10. The control lines 14 are connected to the pixels 12 arranged in the row direction, respectively, and function as a signal line common to the pixels 12 arranged in the row direction. The row direction in which the control lines 14 extend may be referred to as the horizontal direction. The control line 14 arranged corresponding to the pixels 12 in one row may be composed of multiple signal lines to supply multiple types of control signals to the pixels 12.
[0013] In each column of the pixel array arranged in the pixel unit 10, a data line 16 is arranged extending in a column direction (vertical direction in FIG. 1) intersecting with the row direction. The data lines 16 are connected to the pixels 12 arranged in the column direction, respectively, and function as a signal line common to the pixels 12 arranged in the column direction. The column direction in which the data lines 16 extend is sometimes referred to as the vertical direction. The data lines 16 arranged in a row to the pixels 12 may be composed of multiple signal lines to transfer a multi-bit digital signal output from the pixels 12 on a bit-by-bit basis.
[0014] The control lines 14 arranged corresponding to each row of the pixel array arranged in the pixel section 10 are connected to a vertical scanning circuit 40. The vertical scanning circuit 40 receives a control signal output from a control pulse generating circuit 80, generates a control signal for driving the pixels 12, and supplies the control signal to the pixels 12 via the control line 14. The vertical scanning circuit 40 may include logic circuits such as a shift register and an address decoder. The vertical scanning circuit 40 sequentially scans the pixels 12 arranged in the pixel section 10 row by row, and outputs pixel signals of each pixel 12 to a readout circuit 50 via a data line 16.
[0015] The data lines 16 arranged corresponding to each column of the pixel array arranged in the pixel section 10 are connected to a readout circuit 50. The readout circuit 50 includes a plurality of holding circuits (not shown) provided corresponding to each column of the pixel array arranged in the pixel section 10, and has a function of holding pixel signals of the pixels 12 output row by row from the pixel section 10 via the data lines 16 in the holding circuits of the corresponding columns.
[0016] The horizontal scanning circuit 60 receives a control signal output from the control pulse generating circuit 80, generates a control signal for reading out pixel signals from the holding circuits of each column arranged in the readout circuit 50, and supplies the control signal to the readout circuit 50. The horizontal scanning circuit 60 may use logic circuits such as a shift register and an address decoder. The horizontal scanning circuit 60 sequentially scans the holding circuits of each column arranged in the readout circuit 50, and sequentially outputs the pixel signals held in each of them to the output circuit 90.
[0017] The output circuit 90 has an external interface circuit, and is a circuit for outputting the pixel signal output from the readout circuit 50 to the outside of the photoelectric conversion device 100. The external interface circuit included in the output circuit 90 is not particularly limited. The external interface circuit may be configured, for example, by a Serializer / Deserializer (SerDes) transmission circuit. The SerDes transmission circuit may be, for example, a Low Voltage Differential Signaling (LVDS) circuit or a Scalable Low Voltage Signaling (SLVS) circuit.
[0018] The control pulse generating circuit 80 generates control signals for controlling the operations and timing of the vertical scanning circuit 40, readout circuit 50, and horizontal scanning circuit 60, and supplies the control signals to the vertical scanning circuit 40, readout circuit 50, and horizontal scanning circuit 60. All of the control signals for controlling the operations and timing of the operations of the vertical scanning circuit 40, readout circuit 50, and horizontal scanning circuit 60 do not have to be supplied from the control pulse generating circuit 80. Some or all of the control signals for controlling the operations and timing of the operations of the vertical scanning circuit 40, readout circuit 50, and horizontal scanning circuit 60 may be supplied from outside the photoelectric conversion device 100.
[0019] Here, the connection manner of each functional block such as the vertical scanning circuit 40, the readout circuit 50, the horizontal scanning circuit 60, the control pulse generating circuit 80, and the output circuit 90 arranged in the photoelectric conversion device 100 is not limited to the configuration example shown in FIG. 1. For example, each functional block arranged in the photoelectric conversion device 100 may be configured as shown in FIG. 2. In the configuration example shown in FIG. 2, data lines 16 extending in the row direction are arranged in each row of the pixel array arranged in the pixel section 10. The data lines 16 are respectively connected to the pixels 12 arranged in the row direction and function as a common signal line for the pixels 12 arranged in the row direction. In addition, control lines 18 extending in the column direction are arranged in each column of the pixel array arranged in the pixel section 10. The control lines 18 are respectively connected to the pixels 12 arranged in the column direction and function as a common signal line for the pixels 12 arranged in the column direction.
[0020] The control lines 18 arranged corresponding to each column of the pixel array arranged in the pixel section 10 are connected to a horizontal scanning circuit 60. The horizontal scanning circuit 60 receives a control signal output from a control pulse generating circuit 80, generates a control signal for reading out a pixel signal from the pixel 12, and supplies the control signal to the pixel 12 via the control line 18. Specifically, the horizontal scanning circuit 60 sequentially scans the multiple pixels 12 of the pixel section 10 on a column-by-column basis, and pixel signals of the pixels 12 in each row arranged in the selected column are output to the data line 16.
[0021] The data lines 16 arranged corresponding to each row of the pixel array arranged in the pixel section 10 are connected to a readout circuit 50. The readout circuit 50 includes a plurality of holding circuits (not shown) provided corresponding to each row of the pixel array arranged in the pixel section 10, and holds pixel signals of the pixels 12 in each row outputted on a column-by-column basis from the pixel section 10 via the data lines 16 in the holding circuit of the corresponding row.
[0022] The readout circuit 50 receives a control signal output from the control pulse generation circuit 80, and sequentially outputs the pixel signals held in the holding circuits of each row to the output circuit 90. Other configurations in the configuration example shown in FIG. 2 may be similar to the configuration example shown in FIG.
[0023] 3, the pixel 12 arranged in the pixel unit 10 may include a photoelectric conversion unit 20 and a pixel signal processing unit 30. The photoelectric conversion unit 20 may include a photon detection element 22 and a quenching element 24. The pixel signal processing unit 30 may include a waveform shaping circuit 32, a digital processing circuit 34, and a pixel output circuit 36.
[0024] For example, an avalanche photodiode (APD) may be used for the photon detecting element 22. The anode of the APD, which is the photon detecting element 22, is connected to a node to which a potential VL is supplied. The cathode of the APD is connected to one of two terminals of the quench element 24. The connection node between the photon detecting element 22 and the quench element 24 is the output node of the photoelectric conversion unit 20. The other of the two terminals of the quench element 24 to which the cathode of the APD is not connected is connected to a node to which a potential VH higher than the potential VL is supplied. The potential difference (voltage) between the potentials VL and VH is set so that a sufficient reverse bias voltage is applied so that the APD, which is the photon detecting element 22, performs an avalanche multiplication operation. For example, a negative high potential may be applied as the potential VL, and a positive potential of about the power supply potential may be supplied as the potential VH. For example, −30 V may be supplied as the potential VL, and 1 V may be supplied as the potential VH.
[0025] The photon detection element 22 may be constituted by an APD as described above. By supplying a reverse bias voltage sufficient for avalanche multiplication to the APD, avalanche breakdown occurs due to light incident on the APD, and an avalanche multiplication current is generated. There are two operation modes when a reverse bias voltage is supplied to the APD: Geiger mode and linear mode. The Geiger mode is an operation mode in which a voltage applied between the anode and the cathode is a reverse bias voltage greater than the breakdown voltage of the APD. The linear mode is an operation mode in which a voltage applied between the anode and the cathode is a reverse bias voltage close to or less than the breakdown voltage of the APD. An APD operated in Geiger mode is also called a SPAD (Single Photon Avalanche Diode). The APD constituting the photon detection element 22 may be operated in either the linear mode or the Geiger mode.
[0026] The quench element 24 has a function of converting a change in current when avalanche breakdown occurs in the photon detecting element 22 into a voltage signal. The quench element 24 also functions as a load circuit (quench circuit) during current multiplication due to avalanche breakdown, and has a function of reducing the voltage applied to the photon detecting element 22 to suppress the avalanche breakdown. The operation of the quench element 24 to suppress the avalanche breakdown is called a quench operation. The quench element 24 also has a function of returning the potential supplied to the photon detecting element 22 to the potential VH by flowing a current equivalent to the voltage drop caused by the quench operation. The operation of returning the potential supplied to the photon detecting element 22 by the quench element 24 to the potential VH is called a recharge operation. The quench element 24 can be configured by a resistor element, a MOS (MIS) transistor, or the like. In this embodiment, a transistor is used as the quench element 24, as described later.
[0027] The waveform shaping circuit 32 includes an input node to which the output signal of the photoelectric conversion unit 20 is supplied, and an output node. The waveform shaping circuit 32 has a function of converting the analog signal supplied from the photoelectric conversion unit 20 into a pulse signal. As shown in Fig. 3, the waveform shaping circuit 32 can be configured by an inverter circuit or the like. The output node of the waveform shaping circuit 32 is connected to a digital processing circuit 34.
[0028] The digital processing circuit 34 has an input node to which the output signal of the waveform shaping circuit 32 is supplied, an input node connected to the control line 14, and an output node. The digital processing circuit 34 includes a counter, which will be described later. The counter has a function of counting pulses superimposed on the signal output from the waveform shaping circuit 32 and holding a count value that is the result of the counting. The signal supplied from the vertical scanning circuit 40 to the digital processing circuit 34 via the control line 14 may include a timer clock signal for controlling the pulse count period (exposure period), etc. The output node of the digital processing circuit 34 is connected to the data line 16 via the pixel output circuit 36.
[0029] The pixel output circuit 36 has a function of switching the electrical connection state (connected or disconnected) between the digital processing circuit 34 and the data line 16. The pixel output circuit 36 switches the connection state between the digital processing circuit 34 and the data line 16 in response to a control signal supplied from the vertical scanning circuit 40 via the control line 14 (in the case of the configuration example shown in FIG. 2, a control signal supplied from the horizontal scanning circuit 60 via the control line 18). The pixel output circuit 36 may include a buffer circuit for outputting a signal, etc.
[0030] The pixel 12 is typically a unit structure that outputs a pixel signal for forming an image. However, when the purpose is distance measurement using the TOF method, the pixel 12 does not necessarily have to be a unit structure that outputs a pixel signal for forming an image. In other words, the pixel 12 may be a unit structure that outputs a signal for measuring the time when light arrives and the amount of light.
[0031] It is not necessary that one pixel signal processing unit 30 is provided for each pixel 12, but one pixel signal processing unit 30 may be provided for multiple pixels 12. In this case, the single pixel signal processing unit 30 can be used to sequentially execute signal processing for the multiple pixels 12.
[0032] The photoelectric conversion device 100 may be formed on one substrate. Also, for example, the photoelectric conversion device 100 may be configured as a stacked photoelectric conversion device in which a plurality of substrates are stacked. In the latter case, for example, as shown in FIG. 4, the photoelectric conversion device 100 can be configured as a stacked photoelectric conversion device in which a sensor substrate 110 and a circuit substrate 120 are stacked and electrically connected. At least the photon detection element 22, which is one of the components of the pixel 12, may be arranged on the sensor substrate 110. Also, the quench element 24 and the pixel signal processing unit 30, which are one of the components of the pixel 12, may be arranged on the circuit substrate 120. The photon detection element 22 arranged on the sensor substrate 110 and the quench element 24 and the pixel signal processing unit 30 arranged on the circuit substrate 120 may be electrically connected to each other via a connection wiring provided for each pixel 12, for example. Moreover, on the circuit board 120, a vertical scanning circuit 40, a readout circuit 50, a horizontal scanning circuit 60, a control pulse generating circuit 80, an output circuit 90, and the like can further be arranged.
[0033] In the stacked photoelectric conversion device 100, the photon detecting element 22, the quenching element 24, and the pixel signal processing unit 30 arranged in one pixel 12 can be arranged on the sensor substrate 110 and the circuit substrate 120 so as to overlap in an orthogonal projection onto the light incident surface of the sensor substrate 110. The vertical scanning circuit 40, the readout circuit 50, the horizontal scanning circuit 60, the control pulse generating circuit 80, the output circuit 90, and the like can be arranged around the pixel unit 10 composed of a plurality of pixels 12.
[0034] The stacked photoelectric conversion device 100 improves the integration of elements, and as a result, can achieve high functionality. In particular, by arranging the photon detecting elements 22, the quenching elements 24, and the pixel signal processing unit 30 on separate substrates, the photon detecting elements 22 can be arranged at high density without sacrificing the light receiving area of the photon detecting elements 22. This can improve the photon detection efficiency. The number of substrates constituting the photoelectric conversion device 100 is not limited to two. The photoelectric conversion device 100 may be constituted by stacking three or more substrates.
[0035] 4, it is assumed that the sensor substrate 110 and the circuit substrate 120 are chips diced from a wafer, but the sensor substrate 110 and the circuit substrate 120 are not limited to chips. For example, each of the sensor substrate 110 and the circuit substrate 120 may be a wafer. The sensor substrate 110 and the circuit substrate 120 may be stacked in the wafer state and then diced, or may be stacked and bonded after being chipped.
[0036] 5(a) and 5(b) are diagrams for explaining the basic operation of the photoelectric conversion unit 20 and the waveform shaping circuit 32. Fig. 5(a) is a circuit diagram of the photoelectric conversion unit 20 and the waveform shaping circuit 32. Fig. 5(b) shows the waveform of a signal at an input node (node A) of the waveform shaping circuit 32 and the waveform of a signal at an output node (node B) of the waveform shaping circuit 32. Here, for the sake of simplicity of explanation, a case is shown in which the waveform shaping circuit 32 is configured by an inverter circuit.
[0037] At time t0, a reverse bias voltage with a potential difference equivalent to potential VH-potential VL is applied to the photon detecting element 22. A reverse bias voltage sufficient to cause avalanche breakdown when a photon is incident is applied between the anode and cathode of the APD constituting the photon detecting element 22, but there are no carriers to act as the seed of avalanche breakdown when no photons are incident on the photon detecting element 22. Therefore, avalanche breakdown does not occur in the photon detecting element 22, and no current flows through the photon detecting element 22.
[0038] At the next time t1, it is assumed that a photon is incident on the photon detecting element 22. When a photon is incident on the photon detecting element 22, electron-hole pairs are generated by photoelectric conversion, and avalanche breakdown occurs with these carriers as seeds, causing an avalanche multiplication current to flow through the photon detecting element 22. This avalanche multiplication current flows through the quench element 24, causing a voltage drop across the quench element 24, and the voltage at node A begins to drop. The amount of voltage drop at node A increases, and when the avalanche breakdown stops at time t3, the voltage level at node A no longer drops.
[0039] When the avalanche breakdown in the photon detecting element 22 stops, a current that compensates for the voltage drop flows from the node to which the potential VL is supplied to the node A via the photon detecting element 22, and the potential of the node A gradually increases. After that, at time t5, the potential of the node A settles to the original potential level.
[0040] The waveform shaping circuit 32 binarizes the signal input from node A according to a predetermined judgment threshold and outputs it from node B. Specifically, when the potential level of node A exceeds the judgment threshold, the waveform shaping circuit 32 outputs a low-level signal from node B, and when the potential level of node A is equal to or lower than the judgment threshold, the waveform shaping circuit 32 outputs a high-level signal from node B. For example, as shown in FIG. 5(b), assume that the potential level of node A is equal to or lower than the judgment threshold during the period from time t2 to time t4. In this case, the signal level at node B is low during the period from time t0 to time t2 and the period from time t4 to time t5, and is high during the period from time t2 to time t4.
[0041] In this way, the analog signal input from node A is shaped into a digital signal by the waveform shaping circuit 32. The pulse signal output from the waveform shaping circuit 32 in response to a photon being incident on the photon detecting element 22 is a photon detection pulse signal.
[0042] FIG. 6 is a diagram for explaining the configuration of the pixel 12 in this embodiment in more detail. In the explanation of FIG. 6, the explanation of the parts overlapping with FIG. 3 and FIG. 5 will be omitted or simplified as appropriate. In the configuration shown in FIG. 6, as one example, two pixels 12a and 12b are grouped into one group. Furthermore, the readout circuit 50 includes a pixel readout circuit 51, a difference value readout circuit 53, and an exposure control circuit 52. The digital processing circuit 34 includes a counter 342 and a processing circuit 341.
[0043] As described above, the counter 342 counts pulses based on photons incident on the photon detecting element 22 and holds a count value that is the result of the counting. It can also be said that the counter 342 counts the number of times avalanche breakdown occurs in the APD that is the photon detecting element 22. The counter 342 includes, for example, a flip-flop as a bit memory capable of holding a multi-bit digital signal to hold the count value. The count value held in the counter 342 is output to the data line 16 as the count value CNT via the pixel output circuit 36. The count value held in the counter 342 can also be output to the processing circuit 341. In addition, the counter 342 resets the held count value to an initial value at a timing according to a pulse of the reset signal RES input from the exposure control circuit 52.
[0044] The processing circuit 341 calculates a difference between the count value input from the counter 342 arranged in the pixel 12a and the counter 342 arranged in the pixel 12b, and outputs the result to the difference value readout circuit 53 via the pixel output circuit 36. The processing circuit 341 can be a digital circuit configured to be able to perform difference calculation processing based on the count value, which is a digital signal.
[0045] 7 is a timing diagram showing the operation of the pixels 12a and 12b. During a period in which a change in the amount of light incident on the photoelectric conversion device 100 is detected, the pixels 12a and 12b detect a change in the amount of light incident on the photoelectric conversion device 100 by detecting a change in the amount of light incident on the photoelectric conversion device 100. The pixel 12a and the pixel 12b alternately start this detection operation at different times. ...
[0046] For example, in the second frame shown in FIG. 7, the quench element 24 (transistor) disposed in the pixel 12a is turned on to operate the APD, and an exposure period begins. On the other hand, in the pixel 12b, the quench element 24 is turned off to stop the operation of the APD, and a holding period begins. Therefore, the counter 342 disposed in the pixel 12b holds a count value counted in the first frame, which was the exposure period of the pixel 12b. In this manner, the exposure period of the second frame shown in FIG. 7 begins in the exposure period of the pixel 12a after a predetermined time has elapsed from the start of the exposure period of the first frame shown in FIG. 7 in the exposure period of the pixel 12b. The processing circuit 341 outputs a difference value between the count value of the exposure period of the first frame of the pixel 12b, which is counted in the exposure period of the first frame in the pixel 12b and held in the counter 342 in the holding period of the second frame following the first exposure period of the first frame, and the count value of the exposure period of the second frame in the pixel 12a. For example, at the timing when the exposure period of the pixel 12a ends, the processing circuit 341 calculates the difference between the count value of the pixel 12a for the second frame and the count value of the first frame held in the pixel 12b.
[0047] At the timing when the calculation of the difference value in the processing circuit 341 ends, the quench element 24 disposed in the pixel 12a is turned off without resetting the counter 342, and the exposure period ends and the state in which the count value is held in the counter 342 continues during the retention period of the third frame. On the other hand, in the pixel 12b, the count value of the counter 342 is reset in response to the start of the exposure period in which the quench element 24 is turned on, and the count of the number of avalanche breakdowns of the APD, which is the photon detection element 22, begins at the third frame. By repeating such a detection operation, it is possible to continuously obtain the difference value of the count value between the pixel 12a and the pixel 12b, that is, the amount of change in the count value of the incident photons between frames. The change in this count value makes it possible to detect the change in the amount of light incident on the photoelectric conversion device 100. For example, a detection circuit that detects the change in the amount of incident light based on the difference value may be disposed in the difference value readout circuit 53. Furthermore, for example, the difference value may be output from the photoelectric conversion device 100 to the outside, and a change in the amount of incident light may be detected outside the photoelectric conversion device 100.
[0048] 8 shows an example of a logic circuit of the processing circuit 341 for acquiring a difference between the count values of the counter 342 of the pixel 12a and the pixel 12b. As described above, the counter 342 includes a flip-flop circuit. The processing circuit 341 may include an exclusive OR circuit that acquires an exclusive OR of the count value held in the flip-flop circuit of the pixel 12a and the count value held in the flip-flop circuit of the pixel 12b for each digit. This makes it possible to extract a difference between each digit of the count value held in the counter 342. For example, a detection circuit disposed in the difference value readout circuit 53 may determine that an event that changes the amount of incident light has occurred when "1 (the count value has changed)" is output from a digit above a certain digit.
[0049] The length of the exposure period of the pixel 12a and the length of the exposure period of the pixel 12b may be the same as shown in FIG. 7. In this case, in both the event in which the amount of light incident on the photoelectric conversion device 100 increases and the event in which the amount of light incident on the photoelectric conversion device 100 decreases, the difference value can be obtained using the same calculation in the processing circuit 341, and the detection circuit can use the same threshold value to make a judgment. That is, the circuit configuration can be simplified. However, the present invention is not limited to this, and the length of the exposure period of the pixel 12a and the length of the exposure period of the pixel 12b may be different from each other. In this case, for example, the processing circuit 341 may be provided with a correction circuit or the like that corrects at least one of the count values according to the length of the exposure period of the pixel 12a and the pixel 12b.
[0050] Also, as shown in Fig. 7, the exposure period of pixel 12a and the exposure period of pixel 12b do not have to overlap. However, this is not limited thereto, and as shown in Fig. 9, the exposure period of pixel 12a and the exposure period of pixel 12b may partially overlap. When the exposure periods are overlapped, the probability of an event being missed is reduced.
[0051] Although the above description focuses on the pixels 12a and 12b, the pixel unit 10 may further include a pixel 12R sensitive to red, a pixel 12R sensitive to green, and a pixel 12B sensitive to blue, for example, in order to obtain a color image. For example, the four pixels 12a (12b), 12R, 12G, and 12B that detect the change in the amount of incident light may be arranged as one group 420. In this case, for example, as shown in FIG. 10, the groups 420 adjacent to each other in the horizontal direction may be flipped left and right, and the pixels 12a and 12b, which are event pixels that detect the change in the amount of light, may be arranged adjacent to each other. The pixels 12R, 12G, and 12B may also have the same configuration as the pixels 12a and 12b. For example, the APDs in the pixels 12R, 12G, and 12B may function as SPADs. Also, the pixels 12R, 12G, and 12B may not be provided with the processing circuit 341. Also, for example, the processing circuit 341 may be provided in adjacent pixels among the pixels 12R, 12G, and 12B, similarly to the relationship between the pixels 12a and 12b, and may be used for event detection.
[0052] Here, a method for detecting an event from the difference between the count values of the pixels 12a and 12b between frames acquired by the processing circuit 341 will be described. An event means that a change in the amount of light incident on the photon detection elements 22 of the pixels 12a and 12b satisfies a predetermined condition. The predetermined condition may be, for example, that the difference between the count values exceeds a predetermined threshold, or that the amount of change in the count values exceeds a predetermined threshold. As described above, the event may be detected by, for example, a detection circuit disposed in the difference value readout circuit 53, or may be detected outside the photoelectric conversion device 100 by outputting the difference value to the outside of the photoelectric conversion device 100.
[0053] FIG. 11 shows an example of detecting an event in response to a difference in count values exceeding a predetermined threshold value. As shown in FIG. 11, the counters 342 arranged in the pixels 12a and 12b alternately count the photons incident within a period of one frame for each frame. As described above, the difference in count values is obtained for each frame, and an event is detected when the difference value exceeds a predetermined threshold value. For example, the photoelectric conversion device 100 may have a function of transitioning the state of the pixel 12 in response to detection of an event. For example, the photoelectric conversion device 100 may start imaging using each pixel 12 in response to detection of an event.
[0054] In order to detect a change (event) in the amount of incident light using one pixel 12, a memory for storing a count value acquired in a frame prior to the frame of interest and a memory controller are required. For example, in a group 420 consisting of four pixels 12 shown in FIG. 10, one memory needs to be provided for eight pixels 12. That is, the circuit scale of the pixel circuit becomes large. On the other hand, in this embodiment, by using two pixels 12a and 12b, the processing circuit 341 can be configured by adding a small-scale circuit such as an exclusive OR circuit. That is, it is possible to detect an event while suppressing an increase in the circuit scale of the pixel circuit. In addition, the pixels 12a and 12b used for detecting an event can have the same configuration as the pixels 12R, 12G, and 12B. That is, the pixels 12a and 12b for detecting an event do not need to have a different configuration from other pixels, so that the degree of freedom in designing the pixel unit 10 is improved.
[0055] In the above, it has been described that the difference value for detecting the change in the amount of incident light is obtained using two pixels 12a and 12b. However, the detection of the change in the amount of incident light is not limited to being performed using only the group of two pixels 12a and 12b. The detection of the change in the amount of incident light may be performed using three or more pixels 12. FIG. 12 shows an example in which the change in the amount of incident light is detected using four pixels 12a to 12d.
[0056] The pixels 12a to 12d start the above-mentioned detection operation including the exposure period and the holding period in a predetermined order at different timings. In each of the pixels 12a to 12d, the number of photons incident during the exposure period is counted for each frame, and then the count is held. In the case of the configuration shown in FIG. 12, the count values of up to four previous frames can be held. In this case, it is not limited to obtaining a difference value between the pixels 12 whose exposure periods are adjacent to each other, such as a difference value between the count values of the pixels 12a and 12b. For example, it is possible to obtain an irregular difference value in addition to the difference value between adjacent frames, such as a difference value between the count values of the pixels 12a and 12d, or a difference value between the count values of the pixels 12b and 12d. Also, FIG. 12 shows an example in which the exposure periods of the pixels 12a to 12d do not overlap. However, the present invention is not limited thereto, and the exposure period of the pixel 12a may overlap with the exposure period of the pixel 12b, the exposure period of the pixel 12b may overlap with the exposure period of the pixel 12c, and the exposure period of the pixel 12c may overlap with the exposure period of the pixel 12d. The period during which the exposure periods overlap between the pixels 12 can be set longer than that in the case of using the above-mentioned two pixels 12a and 12b. This reduces the probability of missing event information that changes the amount of incident light.
[0057] 13 and 14 show a specific example of switching the operation of the photoelectric conversion device 100 when an event such as a change in the amount of incident light is detected. FIG. 13 is a diagram showing a modified example of the configuration of the photoelectric conversion device 100 shown in FIG. 6. In FIG. 6, the readout circuit 50 is arranged with a pixel readout circuit 51 that reads out signals for generating an image or the like output from a plurality of pixels 12 arranged in the pixel section 10, and a difference value readout circuit 53 that reads out a difference value of the count values of the pixels 12a and 12b used for detecting an event, as separate configurations. On the other hand, in the readout circuit 50 of FIG. 13, a signal for generating an image and a difference value of the count values of the pixels 12a and 12b are read out to the pixel readout circuit 51. Therefore, in the configuration shown in FIG. 13, the pixel readout circuit 51 of the readout circuit 50 can also function as a detection circuit that detects a change in the amount of incident light based on the difference value of the count values of the pixels 12a and 12b.
[0058] The photoelectric conversion device 100 also includes a switching circuit 37 that is connected to the counter 342 and the processing circuit 341, and switches between the output of the counter 342 and the output of the processing circuit 341 and outputs the switched output to the pixel readout circuit 51 of the readout circuit 50. When the function of the detection circuit included in the pixel readout circuit 51 detects a change in the amount of incident light, the pixel readout circuit 51 transmits a selection signal SEL to the switching circuit 37, and switches the signal read out to the pixel readout circuit 51 between the difference value between the pixel 12a and the pixel 12b and the count value of the counter 342 provided in each pixel 12.
[0059] FIG. 14 shows an example of a switching circuit 37 that switches between a difference value output from the processing circuit 341 and a count value output from the counter 342. In the configuration shown in FIG. 14, a multiplexer circuit can be used to switch between the output of the counter 342 and the output of the processing circuit 341 in response to a selection signal SEL. For example, after detecting a change (event) in the amount of incident light, the APD is operated as a SPAD in each of the pixels 12 including the pixels 12a and 12b. That is, in the exposure control according to the exposure control circuit 52, instead of alternate operation of the pixels 12a and 12b, for example, an operation of continuing exposure for a desired time in all the pixels 12 can be performed. Thereby, the count values counted by the counter 342 arranged in the pixels 12 including the pixels 12a and 12b can be switched to an operation of sequentially reading out, and a signal for generating an image can be obtained. In this embodiment, the operation of generating a signal for generating an image can also be performed in the pixels 12a and 12b arranged to detect a change (event) in the amount of incident light. Therefore, the quality of the resulting image can be improved compared to a case where the pixels 12a and 12b are used only for event detection and the pixel values of the pixels 12a and 12b are complemented with the pixel values of other pixels 12 when generating the image.
[0060] Hereinafter, as an application example of the photoelectric conversion device 100, a device EQP including the photoelectric conversion device 100 shown in Fig. 15 will be described. The device EQP may also be called an electronic device. Fig. 15 shows a camera as an example of the device EQP. Here, the concept of a camera includes not only a device whose main purpose is to capture images, but also a device that auxiliary has a capture function (for example, a personal computer or a mobile terminal such as a smartphone).
[0061] The photoelectric conversion device 100 may be a semiconductor chip having a stacked structure in which a pixel unit 10 is provided. As shown in FIG. 15, the photoelectric conversion device 100 is accommodated in a semiconductor package PKG. The package PKG may include a base on which the photoelectric conversion device 100 is fixed, a cover such as glass facing the photoelectric conversion device 100, and a conductive connection member such as a bonding wire or bump that connects a terminal provided on the base to a terminal provided on the photoelectric conversion device 100. The equipment EQP may further include at least one of an optical system OPT, a control device CTRL, a processing device PRCS, a display device DSPL, and a memory device MMRY.
[0062] The optical system OPT forms an image on the photoelectric conversion device 100, and may be, for example, a lens, a shutter, or a mirror. The control device CTRL controls the operation of the photoelectric conversion device 100, and may be, for example, a semiconductor device such as an ASIC. The processing device PRCS functions as a signal processing unit that processes a signal output from the photoelectric conversion device 100, and may be, for example, a semiconductor device such as a CPU or an ASIC. The display device DSPL may be an EL display device or a liquid crystal display device that displays image data obtained by the photoelectric conversion device 100. The memory device MMRY is a magnetic device or a semiconductor device that stores image data obtained by the photoelectric conversion device 100. The memory device MMRY may be a volatile memory such as an SRAM or a DRAM, or a non-volatile memory such as a flash memory or a hard disk drive. The mechanical device MCHN has a moving part or a propulsion part such as a motor or an engine. The mechanical device MCHN in the camera can drive parts of the optical system OPT for zooming, focusing, and shutter operation. In the device EQP, the image data output from the photoelectric conversion device 100 is displayed on the display device DSPL, or transmitted to the outside by a communication device (not shown) included in the device EQP. For this reason, the device EQP may include a memory device MMRY and a processing device PRCS.
[0063] A camera incorporating the photoelectric conversion device 100 can be used as a surveillance camera, or an on-board camera mounted on transportation equipment such as automobiles, railroad cars, ships, aircraft, or industrial robots. In addition, a camera incorporating the photoelectric conversion device 100 can be used not only in transportation equipment but also in a wide range of equipment that uses object recognition, such as an intelligent transport system (ITS).
[0064] Next, a specific application in which the photoelectric conversion device 100 of the present embodiment is mounted on a moving body will be described with reference to Figs. 16(a) and 16(b). Fig. 16(a) shows an example of a photoelectric conversion system related to an on-vehicle camera. The photoelectric conversion system SYS has the above-mentioned photoelectric conversion device 100. The photoelectric conversion system SYS has an image processing unit 801 that performs image processing on a plurality of image data acquired by the photoelectric conversion device 100, and a parallax acquisition unit 802 that calculates parallax (phase difference of parallax images) from the plurality of image data acquired by the photoelectric conversion system SYS. The photoelectric conversion system SYS also has a distance acquisition unit 803 that calculates a distance to an object based on the calculated parallax, and a collision determination unit 804 that determines whether or not there is a possibility of collision based on the calculated distance. Here, the parallax acquisition unit 802 and the distance acquisition unit 803 are examples of distance information acquisition means that acquire distance information to an object. That is, the distance information is information on the parallax, the defocus amount, the distance to the object, and the like. The collision determination unit 804 may determine the possibility of a collision using any of these pieces of distance information. The distance information acquisition means may be realized by dedicated hardware, or may be realized by a software module. In addition, it may be realized by a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or the like, or may be realized by a combination of these.
[0065] The photoelectric conversion system SYS is connected to a vehicle information acquisition device 810, and can acquire vehicle information such as vehicle speed, yaw rate, and steering angle. The photoelectric conversion system SYS is also connected to a control ECU 820, which is a control device that outputs a control signal to generate a braking force for the vehicle based on the judgment result of the collision judgment unit 804. The photoelectric conversion system SYS is also connected to an alarm device 830 that issues an alarm to the driver based on the judgment result of the collision judgment unit 804. For example, when the judgment result of the collision judgment unit 804 indicates that there is a high possibility of a collision, the control ECU 820 performs vehicle control to avoid a collision and reduce damage by applying the brakes, releasing the accelerator, suppressing engine output, etc. The alarm device 830 warns the user by sounding an alarm, displaying alarm information on the screen of a car navigation system, etc., or vibrating the seat belt or steering wheel.
[0066] In this embodiment, the surroundings of the vehicle, for example, the front or rear, are imaged by the photoelectric conversion system SYS. Fig. 16(b) shows the photoelectric conversion system when imaging the area in front of the vehicle (imaging range 850). The vehicle information acquisition device 810 sends instructions to the photoelectric conversion system SYS or the photoelectric conversion device 100. This configuration can further improve the accuracy of distance measurement.
[0067] The above describes an example of control to avoid collision with other vehicles. However, this is not limited to this, and the invention can also be applied to, for example, control to automatically drive a vehicle by following other vehicles, or control to automatically drive a vehicle without straying from its lane.
[0068] The disclosure of this specification includes the following photoelectric conversion device, equipment, and mobile object.
[0069] (Item 1) A photoelectric conversion device comprising: a plurality of pixels including a first pixel and a second pixel, each of which includes an avalanche photodiode and a counter for counting the number of times avalanche breakdown occurs in the avalanche photodiode; and a processing circuit; During a period in which a change in the amount of light incident on the photoelectric conversion device is detected, the first pixel and the second pixel repeat a detection operation including an exposure period during which the avalanche photodiode is operated and the counter counts the number of times, and a retention period during which the operation of the avalanche photodiode is stopped and the count value counted during the exposure period is retained in the counter; the first pixel and the second pixel alternately start the detection operation at different timings; a second exposure period of the exposure period of the second pixel starts after a predetermined time has elapsed since the start of a first exposure period of the exposure period of the first pixel; The processing circuit outputs a difference value between a first count value counted during the first exposure period at the first pixel and held in the counter during a first holding period that follows the first exposure period among the holding periods, and a second count value counted during the second exposure period at the second pixel.
[0070] (Item 2) 2. The photoelectric conversion device according to item 1, further comprising a detection circuit that detects the change based on the difference value.
[0071] (Item 3) a readout circuit for reading out signals output from the plurality of pixels; and a switching circuit connected to the counter and the processing circuit, for switching between an output of the counter and an output of the processing circuit and outputting the output to the readout circuit; 3. The photoelectric conversion device according to item 2, wherein the readout circuit has a function of the detection circuit.
[0072] (Item 4) the counter includes a flip-flop circuit; The photoelectric conversion device described in any one of items 1 to 3, characterized in that the processing circuit includes an exclusive OR circuit that obtains an exclusive OR of the first count value held in the flip-flop circuit of the first pixel and the second count value held in the flip-flop circuit of the second pixel for each digit.
[0073] (Item 5) 5. The photoelectric conversion device according to any one of items 1 to 4, wherein the length of the first exposure period is the same as the length of the second exposure period.
[0074] (Item 6) 6. The photoelectric conversion device according to any one of items 1 to 5, wherein the first exposure period and the second exposure period partially overlap each other.
[0075] (Item 7) 6. The photoelectric conversion device according to any one of items 1 to 5, wherein the first exposure period and the second exposure period do not overlap each other.
[0076] (Item 8) 8. The photoelectric conversion device according to any one of items 1 to 7, wherein the first pixel and the second pixel among the plurality of pixels are arranged adjacent to each other.
[0077] (Item 9) the plurality of pixels further includes a third pixel; During the period in which the change is detected, the third pixel repeats the detection operation, and a third exposure period of the exposure period of the third pixel starts after a predetermined time has elapsed since the start of the second exposure period; The photoelectric conversion device described in any one of items 1 to 8, characterized in that the processing circuit further outputs a difference value between the first count value or the second count value and a third count value counted in the third pixel during the third exposure period.
[0078] (Item 10) the plurality of pixels further includes a fourth pixel; During the period in which the change is detected, the fourth pixel repeats the detection operation, and a fourth exposure period of the exposure period of the fourth pixel is started after a predetermined time has elapsed from the start of the third exposure period; The photoelectric conversion device described in item 9, characterized in that the processing circuit further outputs a difference value between the first count value, the second count value or the third count value and a fourth count value counted at the fourth pixel during the fourth exposure period.
[0079] (Item 11) 11. The photoelectric conversion device according to any one of items 1 to 10, wherein the counter is reset in response to the start of the exposure period.
[0080] (Item 12) A photoelectric conversion device according to any one of items 1 to 11, A processing device that processes a signal output from the photoelectric conversion device; An apparatus comprising:
[0081] (Item 13) A moving object including the photoelectric conversion device according to any one of items 1 to 11, A moving body comprising a control device that controls the movement of the moving body using a signal output from the photoelectric conversion device.
[0082] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]
[0083] 12: pixel, 100: photoelectric conversion device, 341: processing circuit, 342: counter
Claims
1. A photoelectric conversion device comprising: a plurality of pixels including a first pixel and a second pixel, each of which includes an avalanche photodiode and a counter for counting the number of times avalanche breakdown occurs in the avalanche photodiode; and a processing circuit, During a period in which a change in the amount of light incident on the photoelectric conversion device is detected, the first pixel and the second pixel repeat a detection operation including an exposure period during which the avalanche photodiode is operated and the counter counts the number of times, and a retention period during which the operation of the avalanche photodiode is stopped and the count value counted during the exposure period is retained in the counter; the first pixel and the second pixel alternately start the detection operation at different timings; a second exposure period of the exposure period of the second pixel starts after a predetermined time has elapsed since the start of a first exposure period of the exposure period of the first pixel; The processing circuit outputs a difference value between a first count value counted during the first exposure period at the first pixel and held in the counter during a first holding period that follows the first exposure period among the holding periods, and a second count value counted during the second exposure period at the second pixel.
2. The photoelectric conversion device according to claim 1 , further comprising a detection circuit that detects the change based on the difference value.
3. a readout circuit for reading out signals output from the plurality of pixels; and a switching circuit connected to the counter and the processing circuit, for switching between an output of the counter and an output of the processing circuit and outputting the output to the readout circuit, 3. The photoelectric conversion device according to claim 2, wherein the readout circuit has a function of the detection circuit.
4. the counter includes a flip-flop circuit; The photoelectric conversion device according to claim 1, characterized in that the processing circuit includes an exclusive OR circuit that obtains an exclusive OR of the first count value held in the flip-flop circuit of the first pixel and the second count value held in the flip-flop circuit of the second pixel for each digit.
5. 2. The photoelectric conversion device according to claim 1, wherein the length of the first exposure period is the same as the length of the second exposure period.
6. 2 . The photoelectric conversion device according to claim 1 , wherein the first exposure period and the second exposure period partially overlap each other.
7. 2 . The photoelectric conversion device according to claim 1 , wherein the first exposure period and the second exposure period do not overlap each other.
8. The photoelectric conversion device according to claim 1 , wherein the first pixel and the second pixel of the plurality of pixels are arranged adjacent to each other.
9. the plurality of pixels further includes a third pixel, During the period in which the change is detected, the third pixel repeats the detection operation, and a third exposure period of the exposure period of the third pixel is started after a predetermined time has elapsed from the start of the second exposure period; The photoelectric conversion device according to claim 1, characterized in that the processing circuit further outputs a difference value between the first count value or the second count value and a third count value counted in the third pixel during the third exposure period.
10. the plurality of pixels further includes a fourth pixel, During the period in which the change is detected, the fourth pixel repeats the detection operation, and a fourth exposure period of the exposure period of the fourth pixel is started after a predetermined time has elapsed from the start of the third exposure period; The photoelectric conversion device according to claim 9, characterized in that the processing circuit further outputs a difference value between the first count value, the second count value or the third count value and a fourth count value counted at the fourth pixel during the fourth exposure period.
11. 2. The photoelectric conversion device according to claim 1, wherein the counter is reset in response to the start of the exposure period.
12. The photoelectric conversion device according to any one of claims 1 to 11, A processing device that processes a signal output from the photoelectric conversion device; An apparatus comprising:
13. A moving object comprising the photoelectric conversion device according to any one of claims 1 to 11, A moving body comprising a control device that controls the movement of the moving body using a signal output from the photoelectric conversion device.
Citation Information
Patent Citations
Solid-state image sensor, imaging device, and control method of solid-state image sensor
JP2020096347A
