Photoelectric conversion device and photodetection system

Abstract

To provide a photoelectric conversion device and a photo-detection system that have reduced power consumption.SOLUTION: A photoelectric conversion device has a pixel including an avalanche photodiode, a signal processing circuit that includes a counter generating a count value based on photons incident on the avalanche photodiode in a count period and outputs the count value at a predetermined frame rate, a quench transistor that returns the avalanche photodiode after the occurrence of avalanche doubling again to a state in which the avalanche doubling may occur, and a pulse generation unit that outputs, to a gate of the quench transistor, a pulse signal in which the level changes at a predetermined frequency. The pixel makes a transition from a first state to a second state in which the frequency and the frame rate are higher than the first state according to a result of execution of a determination based on the count value and a predetermined threshold.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present invention relates to a photoelectric conversion device and a photodetection system. [Background technology]

[0002] Patent Document 1 discloses a photoelectric conversion device having avalanche photodiodes arranged in a matrix. Avalanche photodiodes amplify signal charges excited by photons by several to several million times by utilizing the avalanche multiplication phenomenon that occurs due to a strong electric field induced in a pn junction of a semiconductor.

[0003] Patent Document 2 discloses an asynchronous solid-state imaging device that operates in response to the detection of an event such as a change in the amount of light. The solid-state imaging device of Patent Document 2 has a detection pixel that detects the event, and a counting pixel that counts the number of photons incident on an avalanche photodiode when an event occurs and outputs a pixel signal. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2020-123847 [Patent Document 2] Japanese Patent Publication No. 2020-096347 Summary of the Invention [Problem to be solved by the invention]

[0005] In an asynchronous photoelectric conversion device such as that described in Patent Document 2, there is a demand for reduction in power consumption.

[0006] An object of the present invention is to provide a photoelectric conversion device and a photodetection system with reduced power consumption. [Means for solving the problem]

[0007] According to one disclosure of the present specification, a pixel includes an avalanche photodiode, a signal processing circuit including a counter that generates a count value based on photons incident on the avalanche photodiode during a count period and outputs the count value at a predetermined frame rate, a quench transistor that returns the avalanche photodiode after avalanche multiplication has occurred to a state where the avalanche multiplication can occur again, and a pulse generating unit that outputs a pulse signal whose level changes at a predetermined frequency to a gate of the quench transistor, and the pixel transitions from a first state to a second state in which the frequency and the frame rate are higher than those of the first state according to a result of a determination based on the count value and a predetermined threshold. The level of the pulse signal changes multiple times within one count period. A photoelectric conversion device characterized by the above features is provided.

[0008] According to another disclosure of the present specification, there is provided a photoelectric conversion device having pixels including an avalanche photodiode, a signal processing circuit including a counter that generates a count value based on photons incident on the avalanche photodiode during a count period and outputs the count value at a predetermined frame rate, a quench transistor that returns the avalanche photodiode after avalanche multiplication has occurred to a state in which the avalanche multiplication can occur again, and a pulse generating unit that outputs a pulse signal whose level changes at a predetermined frequency to the gate of the quench transistor, wherein the pixel transitions from a first state to a second state in which the frequency is higher than in the first state and the number of bits of the count value is greater than in the first state, depending on the result of a determination based on the count value and a predetermined threshold. [Effects of the Invention]

[0009] According to the present invention, it is possible to provide a photoelectric conversion device and a photodetection system with reduced power consumption. [Brief explanation of the drawings]

[0010] [Figure 1]FIG. 1 is a block diagram (part 1) showing a schematic configuration of a photoelectric conversion device according to a first embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram (part 2) showing a schematic configuration of the photoelectric conversion device according to the first embodiment of the present invention. [Figure 3] 1 is a block diagram showing an example of the configuration of a pixel in a photoelectric conversion device according to a first embodiment of the present invention. [Figure 4] 1 is a perspective view showing an example of the configuration of a photoelectric conversion device according to a first embodiment of the present invention. [Figure 5] FIG. 2 is a diagram illustrating the basic operation of a photoelectric conversion unit in the photoelectric conversion device according to the first embodiment of the present invention. [Figure 6] FIG. 2 is a block diagram showing a more specific example of the configuration of a pixel in the photoelectric conversion device according to the first embodiment of the present invention. [Figure 7] 5 is a flowchart illustrating an example of a driving method for the photoelectric conversion device according to the first embodiment of the present invention. [Figure 8] FIG. 3 is a timing chart showing an example of a method for driving pixels in the photoelectric conversion device according to the first embodiment of the present invention. [Figure 9] FIG. 4 is a timing chart showing a modified example of the method for driving pixels in the photoelectric conversion device according to the first embodiment of the present invention. [Figure 10] FIG. 10 is a timing chart showing an example of a method for driving pixels in a photoelectric conversion device according to a second embodiment of the present invention. [Figure 11] 10 is a flowchart showing an example of a driving method for a photoelectric conversion device according to a third embodiment of the present invention. [Figure 12] 10A and 10B are schematic diagrams showing examples of pixel arrangement and driving order in a photoelectric conversion device according to a fourth embodiment of the present invention. [Figure 13] FIG. 10 is a timing chart showing an example of a method for driving pixels in a photoelectric conversion device according to a fourth embodiment of the present invention. [Figure 14] FIG. 10 is a timing chart showing a modified example of the method for driving pixels in the photoelectric conversion device according to the fourth embodiment of the present invention. [Figure 15]10A and 10B are schematic diagrams showing examples of pixel arrangement and a driving method in a photoelectric conversion device according to a fifth embodiment of the present invention. [Figure 16] 10A and 10B are schematic diagrams showing examples of pixel arrangement and a driving method in a photoelectric conversion device according to a sixth embodiment of the present invention. [Figure 17] 13 is a flowchart showing an example of a driving method for a photoelectric conversion device according to a sixth embodiment of the present invention. [Figure 18] 13 is a flowchart showing a modified example of the driving method for the photoelectric conversion device according to the sixth embodiment of the present invention. [Figure 19] 13 is a schematic diagram showing an example of a pixel arrangement and a driving method in a photoelectric conversion device according to a seventh embodiment of the present invention. FIG. [Figure 20] 13 is a flowchart showing an example of a driving method for a photoelectric conversion device according to a seventh embodiment of the present invention. [Figure 21] FIG. 13 is a block diagram showing a schematic configuration of a light detection system according to an eighth embodiment of the present invention. [Figure 22] FIG. 13 is a block diagram showing a schematic configuration of a range image sensor according to a ninth embodiment of the present invention. [Figure 23] FIG. 22 is a schematic diagram showing an example of the configuration of an endoscopic surgery system according to a tenth embodiment of the present invention. [Figure 24] FIG. 22 is a schematic diagram showing an example of the configuration of a moving body according to an eleventh embodiment of the present invention. [Figure 25] FIG. 22 is a block diagram showing a schematic configuration of a light detection system according to an eleventh embodiment of the present invention. [Figure 26] FIG. 23 is a flowchart showing the operation of the light detection system according to the eleventh embodiment of the present invention. [Figure 27] FIG. 26 is a schematic diagram showing a schematic configuration of a light detection system according to a twelfth embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present invention will be described with reference to the drawings. The same or corresponding elements throughout the drawings are designated by common reference numerals, and their description may be omitted or simplified. The embodiments shown below are intended to embody the technical concept of the present invention and are not intended to limit the present invention. The size and positional relationship of components shown in each drawing may be exaggerated for clarity.

[0012] [First embodiment] A photoelectric conversion device according to a first embodiment of the present invention will be described with reference to FIGS. 1 to 9. FIGS. 1 and 2 are block diagrams showing a schematic configuration of the photoelectric conversion device according to this embodiment. FIG. 3 is a block diagram showing an example configuration of a pixel of the photoelectric conversion device according to this embodiment. FIG. 4 is a perspective view showing an example configuration of the photoelectric conversion device according to this embodiment. FIG. 5 is a diagram explaining the basic operation of a photoelectric conversion unit of the photoelectric conversion device according to this embodiment. FIG. 6 is a block diagram showing a more specific example configuration of a pixel in the photoelectric conversion device according to this embodiment. FIG. 7 is a flowchart showing an example of a driving method for the photoelectric conversion device according to this embodiment. FIG. 8 is a timing chart showing an example of a driving method for a pixel in the photoelectric conversion device according to this embodiment. FIG. 9 is a timing chart showing a modified example of a driving method for a pixel in the photoelectric conversion device according to this embodiment.

[0013] 1, the photoelectric conversion device 100 according to this embodiment includes a pixel section 10, a vertical scanning circuit section 40, a readout circuit section 50, a horizontal scanning circuit section 60, a control pulse generating section 80, and an output circuit section 90. In the following description, the photoelectric conversion device is assumed to be an asynchronous imaging device using an avalanche photodiode, but is not limited to this. Examples of photoelectric conversion devices include imaging devices described below, as well as distance measuring devices (devices for measuring distance using focus detection or TOF (Time Of Flight)), photometric devices (devices for measuring the amount of incident light, etc.), etc.

[0014] The pixel unit 10 includes a plurality of pixels 12 arranged in an array of a plurality of rows and a plurality of columns. As described below, each pixel 12 may be composed of a photoelectric conversion unit including a photon detection element and a pixel signal processing unit that processes a signal output from the photoelectric conversion unit. The number of pixels 12 constituting the pixel unit 10 is not particularly limited. For example, the pixel unit 10 may be composed of a plurality of pixels 12 arranged in an array of several thousand rows and several thousand columns, as in a general digital camera. Alternatively, the pixel unit 10 may be composed of a plurality of pixels 12 arranged in a single row or a single column. Alternatively, the pixel unit 10 may be composed of a single pixel 12.

[0015] Control lines 14 are arranged in each row of the pixel array of the pixel unit 10, extending in a first direction (the horizontal direction in FIG. 1 ). The control lines 14 are connected to the pixels 12 aligned in the first direction, respectively, and serve as signal lines common to these pixels 12. The first direction in which the control lines 14 extend may be referred to as the row direction or horizontal direction. Each of the control lines 14 may include multiple signal lines for supplying multiple types of control signals to the pixels 12.

[0016] In addition, data lines 16 are arranged in each column of the pixel array of the pixel unit 10, extending in a second direction (vertical direction in FIG. 1 ) intersecting the first direction. The data lines 16 are connected to the pixels 12 aligned in the second direction, respectively, and serve as signal lines common to these pixels 12. The second direction in which the data lines 16 extend may be referred to as the column direction or vertical direction. Each of the data lines 16 may include multiple signal lines for transferring multi-bit digital signals output from the pixels 12 bit by bit.

[0017] The control lines 14 of each row are connected to a vertical scanning circuit unit 40. The vertical scanning circuit unit 40 is a control unit that receives control signals output from the control pulse generation unit 80, generates control signals for driving the pixels 12, and supplies the control signals to the pixels 12 via the control lines 14. The vertical scanning circuit unit 40 may include logic circuits such as a shift register and an address decoder. The vertical scanning circuit unit 40 sequentially scans the pixels 12 in the pixel unit 10 row by row, and outputs pixel signals of each pixel 12 to the readout circuit unit 50 via the data lines 16.

[0018] The data lines 16 of each column are connected to a readout circuit unit 50. The readout circuit unit 50 includes a plurality of holding units (not shown) provided corresponding to each column of the pixel array of the pixel unit 10, and has a function of holding pixel signals of the pixels 12 of each column, which are output row by row from the pixel unit 10 via the data lines 16, in the holding unit of the corresponding column.

[0019] The horizontal scanning circuit unit 60 is a control unit that receives a control signal output from the control pulse generation unit 80, generates a control signal for reading out pixel signals from the holding units of each column of the readout circuit unit 50, and supplies the control signal to the readout circuit unit 50. The horizontal scanning circuit unit 60 may use logic circuits such as a shift register and an address decoder. The horizontal scanning circuit unit 60 sequentially scans the holding units of each column of the readout circuit unit 50, and sequentially outputs the pixel signals held in each to the output circuit unit 90.

[0020] The output circuit unit 90 has an external interface circuit and is a circuit unit for outputting pixel signals output from the readout circuit unit 50 to the outside of the photoelectric conversion device 100. The external interface circuit included in the output circuit unit 90 is not particularly limited. The external interface circuit may be configured, for example, by a SerDes (SERializer / DESerializer) transmission circuit. The SerDes transmission circuit is, for example, an LVDS (Low Voltage Differential Signaling) circuit or an SLVS (Scalable Low Voltage Signaling) circuit.

[0021] The control pulse generation unit 80 is a control circuit that generates control signals for controlling the operation and timing of the vertical scanning circuit unit 40, the readout circuit unit 50, and the horizontal scanning circuit unit 60, and supplies these to each functional block. Note that at least some of the control signals for controlling the operation and timing of the vertical scanning circuit unit 40, the readout circuit unit 50, and the horizontal scanning circuit unit 60 may be supplied from outside the photoelectric conversion device 100.

[0022] The connection of the functional blocks of the photoelectric conversion device 100 is not limited to the example configuration shown in FIG. 1, and may be configured as shown in FIG. 2, for example.

[0023] In the configuration example of FIG. 2, data lines 16 extending in a first direction are arranged in each row of the pixel array of the pixel unit 10. The data lines 16 are connected to the pixels 12 aligned in the first direction, respectively, and form signal lines common to these pixels 12. Furthermore, control lines 18 extending in a second direction are arranged in each column of the pixel array of the pixel unit 10. The control lines 18 are connected to the pixels 12 aligned in the second direction, respectively, and form signal lines common to these pixels 12.

[0024] The control line 18 of each column is connected to a horizontal scanning circuit unit 60. The horizontal scanning circuit unit 60 receives a control signal output from the control pulse generation unit 80, generates a control signal for reading out pixel signals from the pixels 12, and supplies the control signal to the pixels 12 via the control line 18. Specifically, the horizontal scanning circuit unit 60 sequentially scans the multiple pixels 12 of the pixel unit 10 column by column, and outputs pixel signals of the pixels 12 in each row belonging to the selected column to the data line 16.

[0025] The data lines 16 of each row are connected to a readout circuit unit 50. The readout circuit unit 50 includes a plurality of holding units (not shown) provided corresponding to each row of the pixel array of the pixel unit 10, and has a function of holding pixel signals of the pixels 12 of each row, which are output column by column from the pixel unit 10 via the data lines 16, in the holding unit of the corresponding row.

[0026] The readout circuit unit 50 receives a control signal output from the control pulse generation unit 80, and sequentially outputs the pixel signals held in the holding units of each row to the output circuit unit 90. Other configurations in the configuration example of FIG. 2 may be similar to those in the configuration example of FIG.

[0027] 3, each pixel 12 has a photoelectric conversion unit 20 and a pixel signal processing unit 30 (signal processing circuit). The photoelectric conversion unit 20 has a photon detection element 22 and a quenching element 24. The pixel signal processing unit 30 has a waveform shaping unit 32, a digital processing circuit 34, and a pixel output circuit 36.

[0028] The photon detecting element 22 may be an avalanche photodiode (hereinafter referred to as "APD"). The anode of the APD constituting the photon detecting element 22 is connected to a node to which a voltage VL is supplied. The cathode of the APD constituting the photon detecting element 22 is connected to one terminal of the quench element 24. The connection node between the photon detecting element 22 and the quench element 24 is the output node of the photoelectric conversion unit 20. The other terminal of the quench element 24 is connected to a node to which a voltage VH higher than the voltage VL is supplied. The voltages VL and VH are set so that a reverse bias voltage sufficient for the APD to perform avalanche multiplication operation is applied. For example, a negative high voltage is applied as the voltage VL, and a positive voltage approximately equal to the power supply voltage is applied as the voltage VH. For example, the voltage VL is −30 V, and the voltage VH is 1 V.

[0029] The photon detecting element 22 may be configured with an APD, as described above. When a reverse bias voltage sufficient for avalanche multiplication is applied to the APD, the charge generated by light incident on the APD undergoes avalanche multiplication, generating an avalanche multiplication current. When a reverse bias voltage is applied to the APD, the APD can operate in either Geiger mode or linear mode. In Geiger mode, a reverse bias voltage greater than the breakdown voltage of the APD is applied between the anode and cathode. In linear mode, a reverse bias voltage close to or less than the breakdown voltage of the APD is applied between the anode and cathode. An APD operating in Geiger mode is called a single photon avalanche diode (SPAD). The APD constituting the photon detecting element 22 may operate in either linear mode or Geiger mode. SPADs are particularly preferred because they have a larger potential difference than linear-mode APDs, resulting in a more pronounced breakdown voltage.

[0030] The quench element 24 has a function of converting changes in the avalanche multiplication current generated in the photon detecting element 22 into a voltage signal. The quench element 24 also functions as a load circuit (quench circuit) during signal multiplication by avalanche multiplication, and has a function of reducing the voltage applied to the photon detecting element 22 to suppress avalanche multiplication. The operation of the quench element 24 to suppress avalanche multiplication is called a quench operation. The quench element 24 also has a function of returning the voltage supplied to the photon detecting element 22 to voltage VH by passing a current equivalent to the voltage drop caused by the quench operation. The operation of the quench element 24 to return the voltage supplied to the photon detecting element 22 to voltage VH is called a recharge operation. The quench element 24 can be configured using a resistor element, a MOS transistor, or the like.

[0031] The waveform shaping unit 32 has an input node to which the output signal of the photoelectric conversion unit 20 is supplied, and an output node. The waveform shaping unit 32 has a function of converting the analog signal supplied from the photoelectric conversion unit 20 into a pulse signal. As shown in Fig. 3, the waveform shaping unit 32 may be configured with an inverter circuit or the like. The output node of the waveform shaping unit 32 is connected to the digital processing circuit 34.

[0032] The digital processing circuit 34 has an input node to which the output signal of the waveform shaping unit 32 is supplied, an input node connected to the control line 14, and an output node. The digital processing circuit 34 has a counter, which will be described later. The counter counts pulses superimposed on the signal output from the waveform shaping unit 32 and has the function of holding a count value resulting from the count. The signal supplied from the vertical scanning circuit unit 40 to the digital processing circuit 34 via the control line 14 may include a timer clock signal for controlling the pulse count period (exposure period). The output node of the digital processing circuit 34 is connected to the data line 16 via the pixel output circuit 36.

[0033] The pixel output circuit 36 ​​has a function of switching the electrical connection state (connection or non-connection) between the digital processing circuit 34 and the data lines 16. The pixel output circuit 36 ​​switches the connection state between the digital processing circuit 34 and the data lines 16 in response to a control signal supplied from the vertical scanning circuit unit 40 via the control line 14 (in the configuration example of FIG. 2, a control signal supplied from the horizontal scanning circuit unit 60 via the control line 18). The pixel output circuit 36 ​​may include a buffer circuit for outputting a signal.

[0034] The pixel 12 is typically a unit structure that outputs a pixel signal for forming an image. However, in cases where the purpose is distance measurement using a TOF (Time of Flight) method, the pixel 12 does not necessarily have to be a unit structure that outputs a pixel signal for forming an image. In other words, the pixel 12 can also be a unit structure that outputs a signal for measuring the time and amount of light arrival.

[0035] It is not necessary that one pixel signal processing unit 30 is provided for each pixel 12, but one pixel signal processing unit 30 may be provided for multiple pixels 12. In this case, the single pixel signal processing unit 30 can be used to sequentially perform signal processing for multiple pixels 12.

[0036] The photoelectric conversion device 100 according to this embodiment may be formed on a single substrate, or may be configured as a stacked photoelectric conversion device in which multiple substrates are stacked. In the latter case, for example, as shown in FIG. 4, a stacked photoelectric conversion device can be configured in which a sensor substrate 110 and a circuit substrate 120 are stacked and electrically connected. At least the photon detecting element 22, which is one of the components of the pixel 12, can be arranged on the sensor substrate 110. Furthermore, among the components of the pixel 12, the quenching element 24 and the pixel signal processing unit 30 can be arranged on the circuit substrate 120. The photon detecting element 22, the quenching element 24, and the pixel signal processing unit 30 are electrically connected via connection wiring provided for each pixel 12. Furthermore, the circuit substrate 120 can further include a vertical scanning circuit unit 40, a readout circuit unit 50, a horizontal scanning circuit unit 60, a control pulse generating unit 80, an output circuit unit 90, and the like.

[0037] The photon detecting element 22, quenching element 24, and pixel signal processing unit 30 of each pixel 12 are provided on the sensor substrate 110 and the circuit substrate 120 so as to overlap in a plan view. The vertical scanning circuit unit 40, readout circuit unit 50, horizontal scanning circuit unit 60, control pulse generating unit 80, and output circuit unit 90 can be arranged around the pixel unit 10 composed of multiple pixels 12.

[0038] In this specification, the term "planar view" refers to a view perpendicular to the light incident surface of the sensor substrate 110.

[0039] Constructing a stacked photoelectric conversion device 100 makes it possible to increase the integration density of elements and achieve higher performance. In particular, by arranging the photon detecting elements 22, the quenching elements 24, and the pixel signal processing unit 30 on separate substrates, the photon detecting elements 22 can be arranged at high density without sacrificing the light receiving area of ​​the photon detecting elements 22, thereby improving photon detection efficiency.

[0040] The number of substrates constituting the photoelectric conversion device 100 is not limited to two, and the photoelectric conversion device 100 may be constituted by stacking three or more substrates.

[0041] 4, the sensor substrate 110 and the circuit substrate 120 are assumed to be diced chips, but the sensor substrate 110 and the circuit substrate 120 are not limited to chips. For example, the sensor substrate 110 and the circuit substrate 120 may each be a wafer. The sensor substrate 110 and the circuit substrate 120 may be stacked in the wafer state and then diced, or may be formed into chips and then stacked and bonded.

[0042] Fig. 5 is a diagram illustrating the basic operation of the photoelectric conversion unit 20 and the waveform shaping unit 32. Fig. 5(a) is a circuit diagram of the photoelectric conversion unit 20 and the waveform shaping unit 32, Fig. 5(b) shows the waveform of a signal at an input node (node ​​A) of the waveform shaping unit 32, and Fig. 5(c) shows the waveform of a signal at an output node (node ​​B) of the waveform shaping unit 32. For simplicity of explanation, it is assumed here that the waveform shaping unit 32 is configured with an inverter circuit.

[0043] At time t0, a reverse bias voltage with a potential difference equivalent to (VH-VL) is applied to the photon detecting element 22. A reverse bias voltage sufficient to cause avalanche multiplication is applied between the anode and cathode of the APD that constitutes the photon detecting element 22, but there are no carriers to serve as seeds for avalanche multiplication when no photons are incident on the photon detecting element 22. Therefore, avalanche multiplication does not occur in the photon detecting element 22, and no current flows through the photon detecting element 22.

[0044] At subsequent time t1, a photon is assumed to be incident on the photon detecting element 22. When a photon is incident on the photon detecting element 22, electron-hole pairs are generated by photoelectric conversion, and avalanche multiplication occurs using these carriers as seeds, causing an avalanche multiplication current to flow through the photon detecting element 22. This avalanche multiplication current flows through the quenching element 24, causing a voltage drop across the quenching element 24, and the voltage at node A begins to drop. The amount of voltage drop at node A increases, and when the avalanche multiplication stops at time t3, the voltage level at node A no longer drops.

[0045] When avalanche multiplication in the photon detecting element 22 stops, a current that compensates for the voltage drop flows from the node supplied with voltage VL to node A via the photon detecting element 22, and the voltage at node A gradually increases. After that, at time t5, node A settles to its original voltage level.

[0046] The waveform shaping unit 32 binarizes the signal input from node A in accordance with a predetermined decision threshold and outputs the binarized signal from node B. Specifically, when the voltage level of node A exceeds the decision threshold, the waveform shaping unit 32 outputs a low-level signal from node B, and when the voltage level of node A is equal to or lower than the decision threshold, the waveform shaping unit 32 outputs a high-level signal from node B. For example, as shown in FIG. 5(b), assume that the voltage of node A is equal to or lower than the decision threshold during the period from time t2 to time t4. In this case, as shown in FIG. 5(c), the signal level at node B is low during the period from time t0 to time t2 and the period from time t4 to time t5, and is high during the period from time t2 to time t4.

[0047] In this way, the analog signal input from node A is waveform-shaped into a digital signal by the waveform shaping unit 32. The pulse signal output from the waveform shaping unit 32 in response to a photon incident on the photon detecting element 22 is a photon detection pulse signal.

[0048] FIG. 6 is a diagram illustrating the configuration of the pixel 12 in more detail. In the description of FIG. 6, descriptions of portions that overlap with FIG. 3 or FIG. 5 will be omitted or simplified. The digital processing circuit 34 has a counter 342, a threshold determination unit 344, and an exposure control unit 346. The photoelectric conversion unit 20 has a quench transistor M1 as an example of the quench element 24. The quench transistor M1 is a PMOS transistor. In addition, the pixel 12 has a pulse generation unit 44 that controls the quench transistor M1.

[0049] The drain of the quench transistor M1 is connected to the cathode of the APD that constitutes the photon detecting element 22. The source of the quench transistor M1 is connected to a node to which a voltage VH is supplied. The gate of the quench transistor M1 is connected to the pulse generating unit 44. A pulse signal PL is input from the pulse generating unit 44 to the gate of the quench transistor M1, causing a recharge operation to be performed repeatedly at the frequency of the pulse signal.

[0050] As described above, the counter 342 counts pulses based on photons incident on the photon detecting element 22 and holds a count value that is the result of the counting. To hold the count value, the counter 342 has a bit memory that can hold a multi-bit digital signal. The count value held in the counter 342 is output to the data line 16 as a count value CNT via the pixel output circuit 36. The count value held in the counter 342 is also output to the threshold determination unit 344. The counter 342 also resets the held count value to an initial value at a timing that corresponds to a pulse of the reset signal RES input from the exposure control unit 346.

[0051] The threshold determination unit 344 has a function of determining the event detection result based on the count value input from the counter 342 and a predetermined threshold value. The threshold determination unit 344 may be a digital circuit configured to perform comparison processing based on the count value, which is a digital signal. The threshold determination unit 344 outputs an event detection result signal EDR indicating the determination result to the outside of the pixel signal processing unit 30. This event detection result signal EDR may be used for processing within the photoelectric conversion device 100, may be supplied to other pixels 12, or may be used for signal processing outside the photoelectric conversion device 100. The threshold determination unit 344 also outputs the event detection result signal EDR to the exposure control unit 346 and the pulse generation unit 44.

[0052] The event determined in this process means that the amount of light incident on the photon detecting element 22 of the pixel 12 satisfies a predetermined condition. This predetermined condition may be, for example, that the amount of light exceeds a predetermined threshold, or that the amount of change in the amount of light exceeds a predetermined threshold. The photoelectric conversion device 100 of this embodiment has the function of detecting this event based on the count value and the threshold, and transitioning the state of the pixel 12.

[0053] The pulse generating unit 44 outputs a pulse signal to the gate of the quench transistor M1, thereby switching the quench transistor M1 on and off at the frequency of the pulse signal. This allows the quench transistor M1 to perform a recharge operation, which returns the voltage at the cathode node of the APD that constitutes the photon detecting element 22, at the frequency of the pulse signal. The pulse generating unit 44 may include, for example, a frequency dividing circuit that generates a pulse signal with a variable frequency by dividing the frequency of a clock signal in the photoelectric conversion device 100. The pulse generating unit 44 can also change the frequency of the pulse signal it outputs based on the event detection result signal EDR.

[0054] In the photoelectric conversion device 100 configured as described above, if at least one photon is incident on the photon detecting element 22 during the photon detection waiting period between recharge operations, the count value held in the counter 342 is incremented by one. If no photons are incident on the photon detecting element 22 during the photon detection waiting period, the count value held in the counter 342 is not incremented. In this way, the counter 342 can count the number of periods during which photons are incident and avalanche multiplication occurs, among multiple photon detection waiting periods. Because the number of photon detection waiting periods and the length of each photon detection waiting period vary depending on the frequency of the pulse signal, changing the frequency of the pulse signal changes the frequency of photon counting by the counter 342.

[0055] The exposure control unit 346 has a function of changing the length of the count period (exposure time) of the counter 342 based on the event detection result signal EDR. The exposure control unit 346 can be a digital circuit including a counter that counts the timer clock signal TCLK. The exposure control unit 346 receives the timer clock signal TCLK from the vertical scanning circuit unit 40, the control pulse generation unit 80, or from outside the photoelectric conversion device 100. The exposure control unit 346 counts the number of pulses of the timer clock signal TCLK. When the number of counted pulses reaches a predetermined threshold, the exposure control unit 346 outputs a pulse of a reset signal RES to the counter 342 to reset the count value held in the counter 342. The exposure control unit 346 can change the length of the count period by changing the count threshold for the number of pulses of the timer clock signal TCLK based on the event detection result signal EDR.

[0056] A method for driving the pixel 12 during event detection will be described with mutual reference to Fig. 7 and Fig. 8. Fig. 7 is a flowchart showing the process from an event detection state (first state), which is the initial state, until one pixel 12 detects an event and transitions to an imaging state (second state). Fig. 8 shows the levels of each signal during the process of Fig. 7. Fig. 8 shows the count value CNT, reset signal RES, event detection result signal EDR, The timer clock signal TCLK and the pulse signal PL are shown.

[0057] In step S11, each pixel 12 in the photoelectric conversion device 100 is set to an event detection state. The event detection state is a mode in which pixel signals based on incident light are acquired in a state where power consumption is reduced compared to the normal imaging state. The pixel signals acquired in the event detection state are primarily used to determine whether or not to transition to the imaging state. The period before time t16 in FIG. 8 is the period in which each pixel 12 is in the event detection state. Period T5 in FIG. 8 is the count period from when the counter 342 is reset by the reset signal RES at time t15 to when the counter 342 is next reset at time t16. Each time one count period elapses, the pixel 12 outputs one count value based on the photons incident on the photon detecting element 22 during that count period. For example, after period T5 elapses, the pixel 12 outputs a count value C2 based on the photons detected during period T5. In this manner, the pixel 12 repeatedly outputs a count value each time a count period elapses.

[0058] In step S12, a count value is acquired in the counter 342 of a predetermined pixel 12, and the count value is input to the threshold determination unit 344 of that pixel. The pixels 12 for which count values ​​are acquired in step S12 may be all the pixels 12 in the pixel unit 10, some of the pixels 12, or one pixel 12. The pixels 12 for which count values ​​are acquired may also be selected sequentially from the pixel unit 10.

[0059] In step S13, the threshold determination unit 344 determines whether the count value exceeds a predetermined threshold. If the count value exceeds the predetermined threshold (YES in step S13), it is determined that an event has been detected, and the process proceeds to step S14. If the count value does not exceed the predetermined threshold (NO in step S13), it is determined that an event has not been detected, and the process returns to step S12, and the event detection state continues. Note that FIG. 8 shows the operation timing when it is determined that the count value C2 does not exceed the threshold at time t15, and it is determined that the count value C2 exceeds the threshold at time t16.

[0060] In step S14, the threshold determination unit 344 sets the event detection result signal EDR to high level in response to the determination result that the count value exceeds the predetermined threshold. This process corresponds to time t16 in FIG. 8. The exposure control unit 346 changes the pulse interval of the reset signal RES based on the event detection result signal EDR. This changes the reset cycle of the counter 342 and the count period. The pulse generation unit 44 also changes the frequency of the pulse signal PL based on the event detection result signal EDR. These changes in the count period and frequency change the frame rate of the count value CNT output from the pixel signal processing unit 30. In this way, the pixel 12 transitions from the event detection state to the imaging state. The period after time t16 in FIG. 8 is the period during which each pixel 12 is in the imaging state. Period T6 in FIG. 8 is the count period from when the counter 342 is reset by the reset signal RES at time t16 to when the counter 342 is next reset at time t17. As shown in Figure 8, the length of period T6, which is the count period in the imaging state, is shorter than the length of period T5, which is the count period in the event detection state. Also, as shown in Figure 8, the frequency of pulse signal PL in the imaging state is higher than the frequency of pulse signal PL in the event detection state. Therefore, the frame rate in the imaging state is higher than the frame rate in the event detection state. Operation in the imaging state continues at times t17, t18, etc. after time t16.

[0061] Note that the pixel 12 transitioning to the imaging state in step S14 may be only one pixel 12 determined to have exceeded the threshold, a group of pixels 12 including its surrounding pixels, or all pixels 12 in the pixel unit 10. A configuration in which only one pixel 12 determined to have exceeded the threshold transitions to the imaging state is desirable from the perspective of reducing power consumption. In contrast, a configuration in which all pixels 12 in the pixel unit 10 transition to the imaging state is desirable from the perspective of allowing the entire region of the pixel unit 10 to transition to a state in which imaging can be performed quickly. A configuration in which a group of pixels 12 transition to the imaging state is desirable from the perspective of balancing low power consumption and high processing speed. In this way, the number of pixels 12 transitioning to the imaging state in step S14 can be appropriately selected depending on the required specifications, etc.

[0062] As described above, the photoelectric conversion device 100 of this embodiment includes pixels 12 that transition from the event detection state to the imaging state depending on the result of a determination based on the count value and a predetermined threshold. The effects of this configuration are described below. In asynchronous photoelectric conversion devices 100, reduced power consumption is required. This demand is particularly strong when the photoelectric conversion device 100 is used in an environment with limited power supply, such as when it is battery-powered. In this embodiment, as shown in FIG. 8 , the count period is set longer in the event detection state than in the imaging state, and the frequency of the pulse signal PL is set lower than in the imaging state. Therefore, the frame rate in the event detection state is reduced. This reduces the frequency of counting and the frequency of outputting the count signal, thereby reducing the power consumption of the photoelectric conversion device 100 in the event detection state.

[0063] In the detection pixels for event detection as disclosed in Patent Document 2, no consideration is given to reducing the power consumption of the detection pixels themselves. However, in the photoelectric conversion device 100 of this embodiment, each pixel 12 can operate in an event detection state and an image capturing state, so the power consumption of the pixels used for event detection themselves is reduced.

[0064] As described above, according to this embodiment, a photoelectric conversion device 100 with reduced power consumption is provided.

[0065] Next, a modified example of this embodiment will be described with reference to Fig. 9. In the example of Fig. 8, when the frequency of pulse signal PL is changed at time t16, the pulse width and pulse period of pulse signal PL change at the same ratio. In other words, the duty ratio of pulse signal PL is constant. Such a frequency change can be achieved with a relatively simple frequency divider circuit, which is desirable from the perspective of simplifying the circuit configuration of the pulse generating unit 44. However, the pulse signal PL is not limited to one with a constant duty ratio.

[0066] In the example of FIG. 9, when the frequency of the pulse signal PL is changed, the length of the low-level period of the pulse signal PL is the same between the event detection state before time t26 and the imaging state after time t26. As described above, the low-level period of the pulse signal is the recharge period during which the quench transistor M1 is turned on. This makes the length of the recharge operation period constant, stabilizing the recharge operation. Therefore, the modification of FIG. 9 can improve signal quality. Note that the recharge operation may be performed when the pulse signal PL is at a high level, for example, when the quench transistor M1 is an NMOS. In that case, it is desirable to make the length of the high-level period of the pulse signal PL the same between the event detection state and the imaging state.

[0067] [Second embodiment] A photoelectric conversion device according to a second embodiment of the present invention will be described with reference to Fig. 10. Fig. 10 is a timing chart showing an example of a method for driving pixels in the photoelectric conversion device according to this embodiment.

[0068] The photoelectric conversion device of this embodiment differs from the photoelectric conversion device of the first embodiment in that the number of bits of the count value can be changed when transitioning from the event detection state to the imaging state.

[0069] A method for driving the pixel 12 when an event is detected will be described with reference to Figure 10. The flow of the driving method is the same as that in Figure 7, so a description thereof will be omitted. Figure 10 displays the same signals as in Figure 8, and also lists the number of bits of the count value, such as "m-bit" or "n-bit," in the column for the count value CNT.

[0070] 10 shows the operation timing when it is determined that the count value C1 does not exceed the threshold at time t35 and that the count value C2 exceeds the threshold at time t36. Therefore, the event detection result signal EDR is at a low level before time t36 and at a high level after time t36. That is, the pixel 12 is in an event detection state before time t36 and is in an imaging state after time t36.

[0071] 10, the number of bits of the count values ​​C1 and C2 in the event detection state is m bits, and the number of bits of the count values ​​C3 and C4 in the imaging state is n bits. Here, m is an integer equal to or greater than 1, and n is an integer greater than m. In other words, the number of bits of the count value CNT in the imaging state is greater than the number of bits of the count value CNT in the event detection state. Also, similar to the first embodiment, the frequency of the pulse signal PL in the imaging state is higher than the frequency of the pulse signal PL in the event detection state.

[0072] In this embodiment, in the event detection state, the frequency of the pulse signal PL is set lower than in the imaging state, and the number of bits of the count value CNT is smaller than in the imaging state. In the event detection state, by reducing the number of bits of the count value CNT, the calculation load in the signal processing performed downstream of the counter 342 after the count value CNT is output can be reduced, and power consumption can be reduced. On the other hand, in the imaging state, by increasing the number of bits compared to the event detection state, it is possible to obtain a signal with high accuracy.

[0073] An example of a method for reducing the number of bits is to disable at least the least significant bit in processing after the counter 342 in the event detection state, or to not output at least the least significant bit from the counter 342. In this case, even if data of the least significant bit is missing, the determination in step S13 can be made as long as data of bits near the determination threshold is output. After the transition to the imaging state, the number of bits can be increased by canceling this disablement or stop of output. By using such a method, the bit memory with the number of bits adjusted for the imaging state can also be used in the event detection state.

[0074] 10, the length of period T8, which is the count period in the imaging state, is the same as the length of period T7, which is the count period in the event detection state. However, as in the first embodiment, the length of period T8, which is the count period in the imaging state, may be shorter than the length of period T7, which is the count period in the event detection state. In this case, although the frame rate in the imaging state increases, the shortened count period reduces the accuracy of the acquired signal. By appropriately setting the lengths of periods T7 and T8 in consideration of the balance between frame rate and signal accuracy and the required characteristics, the driving method of the first embodiment and the driving method of the second embodiment can be suitably combined.

[0075] [Third embodiment] A photoelectric conversion device according to a third embodiment of the present invention will be described with reference to Fig. 11. Fig. 11 is a flowchart showing an example of a method for driving pixels in the photoelectric conversion device according to this embodiment.

[0076] The photoelectric conversion device of this embodiment differs from the first embodiment in that the transition to the imaging state is determined based on the amount of change in the count value. As the other circuit configurations and driving methods are applicable to either the first or second embodiment, their explanations will be omitted.

[0077] A method for driving the pixel 12 during event detection will be described with reference to FIG. 11. The operations of steps S11, S12, and S14 are the same as those in FIG. 7, and therefore will not be described again. In step S15, the threshold determination unit 344 determines whether the change over time in the count value exceeds a predetermined threshold. Here, assuming that this determination is performed on the count value output during the nth count period, the change over time can be the difference between the count values ​​obtained by subtracting the count value during the (n-1)th count period from the count value during the nth count period. If the count value exceeds the predetermined threshold (YES in step S15), an event is detected, and the process proceeds to step S14. If the count value does not exceed the predetermined threshold (NO in step S15), an event is not detected, and the process returns to step S12, and the event detection state continues. Note that, to achieve the difference processing described above, the threshold determination unit 344 may include a memory for storing the count value output during the previous count period.

[0078] As described above, the photoelectric conversion device 100 of this embodiment includes pixels 12 that transition from an event detection state to an imaging state depending on the result of a determination based on the change in count value over time and a predetermined threshold. In an asynchronous photoelectric conversion device 100, the change in luminance over time may be more important than the luminance of the object itself. An example of such a situation is when the photoelectric conversion device 100 is monitoring a stationary object and detecting the movement of the object. In this embodiment, the change in count value over time is used as the determination criterion, allowing for more appropriate determination in the above-described situation. Therefore, this embodiment provides a photoelectric conversion device 100 that can perform more appropriate determination.

[0079] In the above explanation, an example of calculating the change over time is shown in which the count value of the nth count period and the count value of the n-1th count period are used, but count values ​​of count periods before the n-2th count period may also be used.

[0080] Note that, when the brightness of an object itself is important, there are cases where a judgment criterion based on the count value itself as shown in Figure 7 is more desirable than a judgment criterion based on the change in the count value over time as shown in Figure 11. By making a judgment by comparing the count value with a threshold as shown in Figure 7, a more appropriate judgment can be made in situations such as when detecting the brightness of the imaging environment as an event.

[0081] [Fourth embodiment] A photoelectric conversion device according to a fourth embodiment of the present invention will be described with reference to Figs. 12 to 14. Fig. 12 is a schematic diagram showing an example of the arrangement and driving order of pixels in the photoelectric conversion device according to this embodiment. Fig. 13 is a timing chart showing an example of a method for driving pixels in the photoelectric conversion device according to this embodiment. Fig. 14 is a timing chart showing a modified example of the method for driving pixels in the photoelectric conversion device according to this embodiment.

[0082] The photoelectric conversion device of this embodiment is a configuration example relating to a method for selecting pixels 12 on which event detection processing is performed. As other circuit configurations and driving methods can be applied to any of the first to fourth embodiments, description thereof will be omitted.

[0083] FIG. 12 schematically illustrates the arrangement of pixels 12 in the pixel unit 10 and the driving sequence in the event detection state. For simplicity's sake, FIG. 12 shows only four rows and four columns of pixels 12; however, this is merely an example, and the number of rows and columns may be greater than this. The (PDEN11) and other signals written in boxes representing the pixels 12 in FIG. 12 indicate control signals for outputting count values ​​from each pixel 12. Two-digit numbers, such as "11," at the end of the control signal name indicate the row number and column number, respectively. The arrows on the boxes representing the pixels 12 in FIG. 12 indicate the scanning sequence of the pixels 12. That is, the driving sequence in the event detection state in this embodiment is as follows: pixel 12 in row 1, column 1 → pixel 12 in row 1, column 2 → pixel 12 in row 1, column 3 → pixel 12 in row 1, column 4 → pixel 12 in row 2, column 1 → .... In this manner, in this embodiment, the driving sequence corresponds to the arrangement of the multiple pixels 12.

[0084] The pixel 12 that outputs the count value can be selected by combining two types of control signals: a control signal output from the vertical scanning circuit unit 40 and a control signal output from the horizontal scanning circuit unit 60. However, for simplicity in this embodiment, the control signals are shown in each diagram as if a single control signal controls the signal output from the pixel 12.

[0085] FIG. 13 shows the levels of the control signals PDEN11, PDEN12, PDEN13, and PDEN14 and the count value CNT that realize the readout order of FIG. 12. Note that each pixel 12 is assumed to be in an event detection state throughout the entire period of FIG. 13. Before time t41, the control signal PDEN11 goes high, enabling the pixel 12 in row 1, column 1. At time t41, the pixel 12 in row 1, column 1 outputs a count value C11. Note that the two-digit number at the end of the count value, such as “11,” indicates the row number and column number, respectively. At this time, event detection is determined based on the count value C11 using a method similar to that described in the above embodiment. From time t41 to time t42, the control signal PDEN12 goes high, enabling the pixel 12 in row 1, column 2. At time t42, the pixel 12 in row 1, column 2 outputs a count value C12. At this time, the event detection is determined based on the count value C11 using the same method as described in the above embodiment. Similarly, thereafter, each time the count period elapses, the count value is output and determined in the corresponding pixel 12 in sequence.

[0086] According to this embodiment, the number of pixels 12 that operate simultaneously is kept small, thereby reducing power consumption, and all of the pixels 12 in the pixel unit 10 are operated in sequence, thereby making it possible to perform determinations that cover the entire pixel unit 10. This provides a photoelectric conversion device 100 that achieves both reduced power consumption and highly accurate event detection.

[0087] Fig. 14 is a modified example of the drive order of Fig. 12 and Fig. 13. As shown in Fig. 14, the drive order in this modified example is as follows: pixel 12 in row 1, column 1 → pixel 12 in row 1, column 3 → pixel 12 in row 1, column 4 → pixel 12 in row 1, column 2 → ... As described above, in this embodiment, the drive order is randomized with respect to the arrangement of the multiple pixels 12. Note that, although Fig. 14 shows an example in which the order within one row is randomized, both the rows and columns may be randomized.

[0088] In imaging environments such as dark places where the number of incident photons is small, the area of ​​the pixel unit 10 where an event can be detected may be extremely small. In such cases, if scanning is performed sequentially along the array of pixels 12 as in the example of Figure 12, the probability of detecting an event may be low. By randomizing the order as in this modified example, it is possible to improve the detection probability when the area of ​​the pixel unit 10 where an event can be detected is small.

[0089] [Fifth embodiment] A photoelectric conversion device according to a fifth embodiment of the present invention will be described with reference to Fig. 15. Fig. 15 is a schematic diagram showing an example of the arrangement and driving method of pixels in the photoelectric conversion device according to this embodiment.

[0090] The photoelectric conversion device of this embodiment is a configuration example relating to a method for selecting pixels 12 on which event detection processing is performed. As other circuit configurations and driving methods can be applied to any of the first to fourth embodiments, description thereof will be omitted.

[0091] 15 is a schematic diagram showing the arrangement of pixels 12 in the pixel unit 10 and whether or not they operate in an event detection state. Hatched boxes in FIG. 15 indicate that in the event detection state, the pixels 12 in question do not output count values ​​and do not determine whether an event has been detected. Pixels 12 in unhatched boxes output count values ​​and determine whether an event has been detected in the event detection state, similar to any of the above-described embodiments.

[0092] In this way, in the event detection state, some of the pixels 12 are thinned out to output count values ​​and determine whether an event has been detected, thereby achieving faster event detection operations or lower power consumption.

[0093] 15, the hatched boxes are distributed in a checkered pattern within the pixel unit 10. By distributing the pixels 12 for which count value output and event detection determination are not performed in this manner, the distribution of the pixels 12 used in event detection processing is made uniform. However, the distribution of the pixels 12 for which count value output and event detection determination are not performed is not limited to that shown in FIG.

[0094] [Sixth embodiment] A photoelectric conversion device according to a sixth embodiment of the present invention will be described with reference to Figs. 16 to 18. Fig. 16 is a schematic diagram showing an example of a pixel arrangement and a driving method in the photoelectric conversion device according to this embodiment. Fig. 17 is a flowchart showing an example of a driving method in the photoelectric conversion device according to this embodiment. Fig. 18 is a flowchart showing a modified example of the driving method in the photoelectric conversion device according to this embodiment.

[0095] The photoelectric conversion device of this embodiment is a configuration example related to the process of transitioning from an event detection state to an image capturing state after an event is detected. As any of the first to fifth embodiments can be applied to other circuit configurations and driving methods, a description thereof will be omitted.

[0096] FIG. 16 schematically shows the arrangement of pixels 12 in the pixel unit 10 and whether or not a transition from an event detection state to an imaging state occurs. The upper part of FIG. 16 shows the arrangement of pixel blocks 10a in the pixel unit 10. The pixel block 10a is a pixel group including multiple pixels 12 within a predetermined range. The lower part of FIG. 16 shows the arrangement of multiple pixels 12 included in one pixel block 10a. The hatched boxes in one pixel block 10a in the lower part of FIG. 16 indicate that the determination result for that pixel 12 satisfies a predetermined condition (count value > threshold) and an event detection determination has been made. In the pixel unit 10 of FIG. 16, the hatched boxes indicate that all pixels 12 included in that pixel block 10a transition to the imaging state.

[0097] A method for driving pixels 12 during event detection will be described with reference to Figure 17. The operations of steps S11, S12, and S13 are the same as those in Figure 7, and therefore will not be described here. In step S16, the pixel block 10a to which the predetermined pixel whose count value has been detected to exceed the threshold belongs, and all of the pixels 12 included in the surrounding pixel blocks 10a, transition from the event detection state to the imaging state. As a result, the pixel block 10a including the pixel 12 that satisfies the predetermined condition and the pixel blocks 10a adjacent to it are driven to transition to the imaging state, as shown in Figure 16.

[0098] As described above, in the photoelectric conversion device 100 of this embodiment, not only the pixel 12 that satisfies a predetermined condition but also the surrounding pixels 12 transition from the event detection state to the imaging state. When a stationary object is monitored using the photoelectric conversion device 100 and the movement of the object is detected, information about the vicinity of the object is important. On the other hand, information about areas far from the object is not very useful. Therefore, when the object is moving, information about the vicinity of the object may be sufficient. In this embodiment, the surrounding pixels of the pixel 12 for which event detection has been determined transition to the imaging state, thereby transitioning the region useful for imaging, as described above, to the imaging state while maintaining the less useful region in the event detection state and maintaining a low power consumption state. Therefore, this embodiment provides a photoelectric conversion device 100 that achieves both reduced power consumption and highly accurate signal acquisition.

[0099] Fig. 18 is a modification of the driving method of Fig. 17. In Fig. 18, steps S17, S18, and S19 are added after the driving method of Fig. 17.

[0100] In step S17, a count value is acquired from any of the surrounding pixels 12 that have transitioned to the imaging state in step S16. Details of the count value acquisition operation are the same as those in step S12.

[0101] In step S18, the threshold determination unit 344 determines whether the count value acquired from the peripheral pixels exceeds a predetermined threshold. If the count value exceeds the predetermined threshold (YES in step S18), it is assumed that an event has been detected, and the process proceeds to step S19. If the count value does not exceed the predetermined threshold (NO in step S18), it is assumed that an event has not been detected, and the process returns to step S17.

[0102] In step S19, all the pixels 12 in the pixel unit 10 transition from the event detection state to the imaging state.

[0103] In imaging environments such as dark places where the number of incident photons is low, even if event detection is performed, there is a possibility that it is a false detection due to noise. If all pixels are transitioned to the imaging state in this state, there is a risk of wasting power if a false detection occurs. Therefore, in this modified example, event detection is performed using information from pixels surrounding the pixel where event detection was performed, so that event detection can be determined with higher accuracy before all pixels are transitioned to the imaging state. This reduces the possibility of power wastage due to false detection.

[0104] [Seventh embodiment] A photoelectric conversion device according to a seventh embodiment of the present invention will be described with reference to Fig. 19 and Fig. 20. Fig. 19 is a schematic diagram showing an example of a pixel arrangement and a driving method in the photoelectric conversion device according to this embodiment. Fig. 20 is a flowchart showing an example of a driving method in the photoelectric conversion device according to this embodiment.

[0105] The photoelectric conversion device of this embodiment is a configuration example related to the process of transitioning from an event detection state to an image capturing state after an event is detected. As any of the first to sixth embodiments can be applied to other circuit configurations and driving methods, a description thereof will be omitted.

[0106] FIG. 19 schematically illustrates the arrangement of pixels 12 in the pixel unit 10 and whether or not a transition from an event detection state to an imaging state occurs. The arrangement of the pixels 12 in the pixel unit 10 and pixel block 10a is the same as in FIG. 16. Each of the hatched boxes in one pixel block 10a in the lower part of FIG. 19 indicates that the determination result for that pixel 12 satisfies a predetermined condition (count value > threshold) and an event detection determination has been made. That is, in the example of FIG. 19, event detection has been determined for five pixels 12. In the pixel unit 10 of FIG. 19, the hatched boxes indicate that all of the pixels 12 included in that pixel block 10a transition to the imaging state.

[0107] A method for driving the pixel 12 when an event is detected will be described with reference to Fig. 20. The operations in steps S11 and S13 are the same as those in Fig. 7, and therefore the description will be omitted or simplified.

[0108] In step S20, count values ​​are acquired from all pixels 12 within a predetermined pixel block 10a. Then, in step S13, it is determined whether each count value exceeds a predetermined threshold. The determination process for each count value is the same as that described in the first embodiment. If the count value output from at least one pixel 12 exceeds the predetermined threshold (YES in step S13), it is determined that an event has been detected, and the process proceeds to step S21. If the count value does not exceed the predetermined threshold (NO in step S13), it is determined that an event has not been detected, and the process returns to step S20, and the event detection state continues.

[0109] In step S21, each pixel in the pixel block 10a for which the count value has been acquired and determined as described above transitions to an imaging state set to different conditions depending on the number of detected pixels 12 exceeding the threshold (five in the example of FIG. 19). In the configuration of the first embodiment, the "different conditions" correspond to changing at least one of the frequency and frame rate after time t16 or t26 depending on the number of detected pixels. In this case, it is preferable to set at least one of the frequency and frame rate after time t16 or t26 higher as the number of detected pixels increases, thereby increasing the frequency of signal output and improving accuracy. In the configuration of the second embodiment, the "different conditions" correspond to changing at least one of the frequency and bit count after time t36 depending on the number of detected pixels. In this case, it is preferable to set the frequency after time t36 higher as the number of detected pixels increases, or to set the bit count after time t36 higher as the number of detected pixels increases, thereby improving the accuracy of the output signal.

[0110] As described above, in the photoelectric conversion device 100 of this embodiment, the state of the pixels 12 transitions to an imaging state set under different conditions depending on the number of detections in the pixel block 10a. Areas with a large number of detections are often areas that require imaging under different conditions, such as places where many people gather. Therefore, it is desirable that the imaging state after the transition be set to different conditions depending on the number of detections. According to this embodiment, a photoelectric conversion device 100 is provided that can acquire signals under more appropriate conditions.

[0111] Furthermore, as described above, it is more desirable that the greater the number of detections, the higher the accuracy of the imaging state after the transition. This example is particularly effective in imaging environments where high accuracy is required for specific parts, such as face recognition.

[0112] [Eighth embodiment] An optical detection system according to an eighth embodiment of the present invention will be described with reference to Fig. 21. Fig. 21 is a block diagram of the optical detection system according to this embodiment. The optical detection system according to this embodiment is an imaging system that acquires an image based on incident light.

[0113] The photoelectric conversion device in the above-described embodiment can be applied to various imaging systems. Examples of imaging systems include digital still cameras, digital camcorders, camera heads, copiers, fax machines, mobile phones, vehicle-mounted cameras, observation satellites, and surveillance cameras. Fig. 21 shows a block diagram of a digital still camera as an example of an imaging system.

[0114] The imaging system 7 shown in Fig. 21 includes a barrier 706, a lens 702, an aperture 704, an imaging device 70, a signal processing unit 708, a timing generating unit 720, an overall control and calculation unit 718, a memory unit 710, a recording medium control I / F unit 716, a recording medium 714, and an external I / F unit 712. The barrier 706 protects the lens, and the lens 702 forms an optical image of a subject on the imaging device 70. The aperture 704 varies the amount of light that passes through the lens 702. The imaging device 70 is configured like the photoelectric conversion device of the above-mentioned embodiment, and converts the optical image formed by the lens 702 into image data. The signal processing unit 708 performs various processes such as correction and data compression on the imaging data output from the imaging device 70.

[0115] The timing generating unit 720 outputs various timing signals to the imaging device 70 and the signal processing unit 708. The overall control / calculation unit 718 controls the entire digital still camera, and the memory unit 710 temporarily stores image data. The recording medium control I / F unit 716 is an interface for recording or reading image data to or from the recording medium 714, and the recording medium 714 is a removable recording medium such as a semiconductor memory for recording or reading image data. The external I / F unit 712 is an interface for communicating with an external computer or the like. Timing signals and the like may be input from outside the imaging system 7, and the imaging system 7 only needs to have at least the imaging device 70 and the signal processing unit 708 for processing image signals output from the imaging device 70.

[0116] In this embodiment, the imaging device 70 and the signal processing unit 708 may be arranged on the same semiconductor substrate, or the imaging device 70 and the signal processing unit 708 may be arranged on different semiconductor substrates.

[0117] Furthermore, each pixel of the image capture device 70 may include a first photoelectric conversion unit and a second photoelectric conversion unit. The signal processing unit 708 processes a pixel signal based on the charge generated in the first photoelectric conversion unit and a pixel signal based on the charge generated in the second photoelectric conversion unit, and can acquire information about the distance from the image capture device 70 to the subject.

[0118] [Ninth embodiment] Fig. 22 is a block diagram of a light detection system according to this embodiment, more specifically, a range image sensor using the photoelectric conversion device described in the above embodiment.

[0119] 22, the range image sensor 401 includes an optical system 402, a photoelectric conversion device 403, an image processing circuit 404, a monitor 405, and a memory 406. The range image sensor 401 receives light (modulated light, pulsed light) emitted from a light source device 411 toward a subject and reflected by the surface of the subject. The range image sensor 401 can obtain a range image according to the distance to the subject, based on the time between light emission and light reception.

[0120] The optical system 402 includes one or more lenses, guides image light (incident light) from a subject to the photoelectric conversion device 403 , and forms an image on the light receiving surface (sensor section) of the photoelectric conversion device 403 .

[0121] The photoelectric conversion devices of the above-described embodiments can be applied as the photoelectric conversion device 403. The photoelectric conversion device 403 supplies the image processing circuit 404 with a distance signal indicating the distance determined from the light reception signal.

[0122] The image processing circuit 404 performs image processing to construct a distance image based on the distance signal supplied from the photoelectric conversion device 403. The distance image (image data) obtained by the image processing can be displayed on a monitor 405 and stored (recorded) in a memory 406.

[0123] The range image sensor 401 configured in this manner can acquire an accurate range image by applying the above-described photoelectric conversion device.

[0124] [Tenth embodiment] The technology according to the present disclosure can be applied to various products. For example, the technology according to the present disclosure may be applied to an endoscopic surgery system, which is an example of an optical detection system.

[0125] Fig. 23 is a schematic diagram of the endoscopic surgery system according to this embodiment. Fig. 23 shows a state in which an operator (doctor) 1131 is performing surgery on a patient 1132 on a patient bed 1133 using an endoscopic surgery system 1103. As shown in the figure, the endoscopic surgery system 1103 includes an endoscope 1100, a surgical tool 1110, an arm 1121, and a cart 1134 on which various devices for endoscopic surgery are mounted.

[0126] The endoscope 1100 includes a lens barrel 1101, a region of a predetermined length from the tip of which is inserted into a body cavity of a patient 1132, and a camera head 1102 connected to the base end of the lens barrel 1101. Although Fig. 23 shows the endoscope 1100 configured as a so-called rigid scope having a rigid lens barrel 1101, the endoscope 1100 may also be configured as a so-called flexible scope having a flexible lens barrel.

[0127] An opening into which an objective lens is fitted is provided at the tip of the lens barrel 1101. A light source device 1203 is connected to the endoscope 1100. Light generated by the light source device 1203 is guided to the tip of the lens barrel 1101 by a light guide extending inside the lens barrel 1101, and is irradiated via the objective lens towards an observation target inside the body cavity of the patient 1132. Note that the endoscope 1100 may be a direct-viewing endoscope, an oblique-viewing endoscope, or a side-viewing endoscope.

[0128] An optical system and a photoelectric conversion device are provided inside the camera head 1102, and light reflected from an observation object (observation light) is focused onto the photoelectric conversion device by the optical system. The observation light is photoelectrically converted by the photoelectric conversion device to generate an electrical signal corresponding to the observation light, i.e., an image signal corresponding to the observation image. The photoelectric conversion device may be any of the photoelectric conversion devices described in the above-described embodiments. The image signal is transmitted to a camera control unit (CCU) 1135 as RAW data.

[0129] The CCU 1135 is configured with a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), etc., and comprehensively controls the operations of the endoscope 1100 and the display device 1136. Furthermore, the CCU 1135 receives an image signal from the camera head 1102, and performs various image processing on the image signal, such as development processing (demosaic processing), to display an image based on the image signal.

[0130] Under the control of the CCU 1135 , the display device 1136 displays an image based on the image signal that has been subjected to image processing by the CCU 1135 .

[0131] The light source device 1203 includes a light source such as an LED (Light Emitting Diode), and supplies the endoscope 1100 with irradiation light when photographing an operation site or the like.

[0132] The input device 1137 is an input interface for the endoscopic surgery system 1103. A user can input various information and instructions to the endoscopic surgery system 1103 via the input device 1137.

[0133] The treatment tool control device 1138 controls the driving of the energy treatment tool 1112 for cauterizing tissue, incising, sealing blood vessels, or the like.

[0134] The light source device 1203 can supply illumination light to the endoscope 1100 when photographing the surgical site, and can be, for example, a white light source such as an LED, a laser light source, or a combination of these. When the white light source is configured by combining RGB laser light sources, the output intensity and output timing of each color (each wavelength) can be controlled with high precision. This makes it possible to adjust the white balance of the captured image in the light source device 1203. In this case, laser light from each of the RGB laser light sources can be irradiated onto the observation object in a time-division manner, and the drive of the image sensor of the camera head 1102 can be controlled in synchronization with the irradiation timing. This makes it possible to capture images corresponding to each RGB in a time-division manner. According to this method, a color image can be obtained without providing a color filter to the image sensor.

[0135] Furthermore, the driving of the light source device 1203 may be controlled so that the intensity of light output from the light source device 1203 is changed at predetermined time intervals. By controlling the driving of the image sensor of the camera head 1102 in synchronization with the timing of the change in the light intensity to acquire images in a time-division manner and combining the images, it is possible to generate an image with a high dynamic range that is free from so-called blocked-up shadows and blown-out highlights.

[0136] Furthermore, the light source device 1203 may be configured to supply light in a predetermined wavelength band corresponding to special light observation. Special light observation can utilize, for example, the wavelength dependence of light absorption in body tissue. Specifically, by irradiating light with a narrower band than the light (i.e., white light) used in normal observation, a specific tissue, such as blood vessels on the surface of the mucosa, can be photographed with high contrast. Alternatively, special light observation may involve fluorescence observation, in which an image is obtained using fluorescence generated by irradiating excitation light. Fluorescence observation can involve irradiating excitation light onto a body tissue and observing the fluorescence from the body tissue, or locally injecting a reagent such as indocyanine green (ICG) into the body tissue and irradiating the body tissue with excitation light corresponding to the fluorescent wavelength of the reagent to obtain a fluorescent image. The light source device 1203 can be configured to supply narrow band light and / or excitation light corresponding to such special light observation.

[0137] [Eleventh embodiment] The light detection system and moving body of this embodiment will be described with reference to Figures 24, 25(a), 25(b), 25(c), and 26. In this embodiment, an in-vehicle camera will be shown as an example of the light detection system.

[0138] FIG. 24 is a schematic diagram of a photodetection system according to this embodiment, illustrating an example of a vehicle system and a photodetection system mounted on the vehicle system. The photodetection system 1301 includes a photoelectric conversion device 1302, an image preprocessing unit 1315, an integrated circuit 1303, and an optical system 1314. The optical system 1314 forms an optical image of a subject on the photoelectric conversion device 1302. The photoelectric conversion device 1302 converts the optical image of the subject formed by the optical system 1314 into an electrical signal. The photoelectric conversion device 1302 is any one of the photoelectric conversion devices according to the above-described embodiments. The image preprocessing unit 1315 performs predetermined signal processing on the signal output from the photoelectric conversion device 1302. The function of the image preprocessing unit 1315 may be incorporated into the photoelectric conversion device 1302. The light detection system 1301 is provided with at least two sets of an optical system 1314 , a photoelectric conversion device 1302 and an image pre-processing unit 1315 , and the output from each set of image pre-processing unit 1315 is input to the integrated circuit 1303 .

[0139] The integrated circuit 1303 is an integrated circuit for use in an imaging system, and includes an image processing unit 1304 including a storage medium 1305, an optical distance measurement unit 1306, a parallax calculation unit 1307, an object recognition unit 1308, and an abnormality detection unit 1309. The image processing unit 1304 performs image processing such as development and defect correction on the output signal of the image pre-processing unit 1315. The storage medium 1305 temporarily stores the captured image and stores the defect positions of the captured pixels. The optical distance measurement unit 1306 focuses or measures the distance to the subject. The parallax calculation unit 1307 calculates distance information (distance information) from multiple image data (parallax images) acquired by multiple photoelectric conversion devices 1302. The object recognition unit 1308 recognizes subjects such as cars, roads, signs, and people. If the abnormality detection unit 1309 detects an abnormality in the photoelectric conversion device 1302, it notifies the main control unit 1313 of the abnormality.

[0140] The integrated circuit 1303 may be realized by dedicated hardware, a software module, or a combination thereof. It may also be realized by an FPGA (Field Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), or the like, or a combination thereof.

[0141] The main control unit 1313 supervises and controls the operations of the light detection system 1301, the vehicle sensor 1310, the control unit 1320, etc. Instead of having the main control unit 1313, the light detection system 1301, the vehicle sensor 1310, and the control unit 1320 may each have a communication interface and send and receive control signals via a communication network according to, for example, the CAN standard.

[0142] The integrated circuit 1303 has a function of receiving a control signal from the main control unit 1313 or transmitting a control signal or a set value to the photoelectric conversion device 1302 by its own control unit.

[0143] The optical detection system 1301 is connected to a vehicle sensor 1310 and can detect the vehicle's driving conditions, such as vehicle speed, yaw rate, and steering angle, as well as the conditions of the environment outside the vehicle and other vehicles and obstacles. The vehicle sensor 1310 also serves as a distance information acquisition means for acquiring distance information to an object. The optical detection system 1301 is also connected to a driving assistance control unit 1311 that performs various driving assistance functions, such as automatic steering, automatic cruising, and collision prevention functions. In particular, the collision determination function determines whether or not a collision with another vehicle or obstacle has occurred based on the detection results of the optical detection system 1301 and the vehicle sensor 1310. This allows for avoidance control when a collision is predicted, and activation of safety devices in the event of a collision.

[0144] The light detection system 1301 is also connected to an alarm device 1312 that issues an alarm to the driver based on the determination result of the collision determination unit. For example, if the collision determination unit determines that there is a high possibility of a collision, the main control unit 1313 performs vehicle control such as applying the brakes, releasing the accelerator, and suppressing engine output, thereby avoiding the collision or mitigating damage. The alarm device 1312 issues an alarm to the user by issuing an alarm such as a sound, displaying alarm information on the display screen of a car navigation system or an instrument panel, applying vibration to the seat belt or steering wheel, etc.

[0145] The light detection system 1301 in this embodiment can capture images of the surroundings of the vehicle, for example, the front or rear. Figures 25(a), 25(b), and 25(c) are schematic diagrams of a moving object in this embodiment, showing a configuration in which the light detection system 1301 captures an image of the area in front of the vehicle.

[0146] The two photoelectric conversion devices 1302 are disposed in front of the vehicle 1300. Specifically, it is preferable that the center line of the vehicle 1300 relative to its forward / backward direction or its outer shape (for example, its width) be regarded as an axis of symmetry, and that the two photoelectric conversion devices 1302 be disposed symmetrically about the axis of symmetry. This makes it possible to effectively obtain distance information between the vehicle 1300 and an object to be photographed and to determine the possibility of a collision. It is also preferable that the photoelectric conversion devices 1302 be disposed in a position that does not obstruct the driver's field of vision when the driver visually checks the situation outside the vehicle 1300 from the driver's seat. It is preferable that the warning device 1312 be disposed in a position that is easily within the driver's field of vision.

[0147] Next, a fault detection operation of the photoelectric conversion device 1302 in the photodetection system 1301 will be described with reference to Fig. 26. Fig. 26 is a flowchart showing the operation of the photodetection system in this embodiment. The fault detection operation of the photoelectric conversion device 1302 can be performed in accordance with steps S1410 to S1480 shown in Fig. 26.

[0148] In step S1410, startup settings are made for the photoelectric conversion device 1302. That is, setting information for the operation of the photoelectric conversion device 1302 is transmitted from outside the photodetection system 1301 (for example, from the main control unit 1313) or from inside the photodetection system 1301, and the photoelectric conversion device 1302 starts imaging operations and fault detection operations.

[0149] Next, in step S1420, the photoelectric conversion device 1302 acquires pixel signals from valid pixels. Furthermore, in step S1430, the photoelectric conversion device 1302 acquires output values ​​from failure detection pixels provided for failure detection. These failure detection pixels include photoelectric conversion elements, just like valid pixels. A predetermined voltage is written to these photoelectric conversion elements. The failure detection pixels output signals corresponding to the voltage written to these photoelectric conversion elements. Note that steps S1420 and S1430 may be executed in reverse order.

[0150] Next, in step S1440, the photodetection system 1301 determines whether the expected output value of the fault detection pixel matches the actual output value from the fault detection pixel. If the result of the determination in step S1440 indicates that the expected output value and the actual output value match, the photodetection system 1301 proceeds to processing in step S1450, determines that the imaging operation is being performed normally, and proceeds to processing in step S1460. In step S1460, the photodetection system 1301 transmits the pixel signals of the scanning row to the storage medium 1305 for temporary storage. Thereafter, the photodetection system 1301 returns to processing in step S1420 and continues the fault detection operation. On the other hand, if the result of the determination in step S1440 indicates that the expected output value and the actual output value do not match, the photodetection system 1301 proceeds to processing in step S1470. In step S1470, the light detection system 1301 determines that there is an abnormality in the imaging operation, and issues an alarm to the main control unit 1313 or the alarm device 1312. The alarm device 1312 displays on the display unit that an abnormality has been detected. Thereafter, in step S1480, the light detection system 1301 stops the photoelectric conversion device 1302, and ends the operation of the light detection system 1301.

[0151] In this embodiment, the flowchart is looped for each line, but the flowchart may be looped for each set of lines, or the fault detection operation may be performed for each frame. The issuance of the alarm in step S1470 may be notified to the outside of the vehicle via a wireless network.

[0152] Furthermore, although the present embodiment has been described as a control for preventing collisions with other vehicles, the present invention is also applicable to control for automatic driving by following other vehicles, control for automatic driving so as not to deviate from a lane, and the like. Furthermore, the light detection system 1301 is not limited to vehicles such as the subject vehicle, but can be applied to moving bodies (moving devices) such as ships, aircraft, and industrial robots. In addition, the present invention is not limited to moving bodies, but can be applied to a wide range of devices that use object recognition, such as intelligent transport systems (ITS).

[0153] The photoelectric conversion device of the present invention may further be configured to be capable of acquiring various types of information such as distance information.

[0154] [Twelfth embodiment] FIG. 27(a) is a diagram showing a specific example of an electronic device according to this embodiment, illustrating glasses 1600 (smart glasses). The glasses 1600 are provided with a photoelectric conversion device 1602 according to any of the above-described embodiments. That is, the glasses 1600 are an example of a light detection system to which the photoelectric conversion device 1602 according to any of the above-described embodiments can be applied. A display device including a light-emitting device such as an OLED or LED may be provided on the rear side of the lens 1601. There may be one or more photoelectric conversion devices 1602. Furthermore, multiple types of photoelectric conversion devices may be combined. The arrangement position of the photoelectric conversion device 1602 is not limited to that shown in FIG. 27(a).

[0155] The glasses 1600 further include a control device 1603. The control device 1603 functions as a power source that supplies power to the photoelectric conversion device 1602 and the display device described above. The control device 1603 also controls the operations of the photoelectric conversion device 1602 and the display device. The lens 1601 is provided with an optical system for focusing light onto the photoelectric conversion device 1602.

[0156] FIG. 27(b) shows glasses 1610 (smart glasses) according to one application example. The glasses 1610 include a control device 1612, which is equipped with a photoelectric conversion device corresponding to the photoelectric conversion device 1602 and a display device. A lens 1611 includes the photoelectric conversion device in the control device 1612 and an optical system for projecting light emitted from the display device, and an image is projected onto the lens 1611. The control device 1612 functions as a power source for supplying power to the photoelectric conversion device and the display device, and controls the operation of the photoelectric conversion device and the display device. The control device 1612 may also include a gaze detection unit for detecting the gaze of the wearer. Infrared light may be used for gaze detection. The infrared light emitter emits infrared light toward the eyeball of a user gazing at a displayed image. An imaging unit having a light receiving element detects the emitted infrared light reflected from the eyeball, thereby obtaining an image of the eyeball. By providing a reduction means for reducing the amount of light from the infrared light emitting section to the display section in a plan view, degradation of image quality is reduced.

[0157] The control device 1612 detects the user's line of sight with respect to the displayed image from the captured image of the eyeball obtained by capturing infrared light. Any known method can be used for gaze detection using the captured image of the eyeball. As an example, a gaze detection method based on the Purkinje image formed by reflection of irradiated light on the cornea can be used.

[0158] More specifically, gaze detection processing is performed based on the pupil-corneal reflex method, which calculates a gaze vector representing the direction (rotation angle) of the eyeball based on the pupil image and Purkinje image included in the captured image of the eyeball, thereby detecting the user's gaze.

[0159] The display device of this embodiment may have a photoelectric conversion device having a light receiving element, and may control the image displayed on the display device based on information on the user's line of sight from the photoelectric conversion device.

[0160] Specifically, the display device determines a first field of view area where the user gazes and a second field of view area other than the first field of view area based on the line-of-sight information. The first field of view area and the second field of view area may be determined by a control device of the display device or by an external control device. In the display area of ​​the display device, the display resolution of the first field of view area may be controlled to be higher than the display resolution of the second field of view area. In other words, the resolution of the second field of view area may be lower than that of the first field of view area.

[0161] The display area may also include a first display area and a second display area different from the first display area. A high-priority area may be determined from the first display area and the second display area based on line-of-sight information. The first field of view area and the second field of view area may be determined by a control device of the display device or by an external control device. The resolution of the high-priority area may be controlled to be higher than the resolution of areas other than the high-priority area. In other words, the resolution of an area with a relatively low priority may be lowered.

[0162] Note that AI (Artificial Intelligence) may be used in determining the first field of view area and the area with high priority. The AI ​​may be a model configured to estimate the angle of gaze and the distance to an object in the line of sight from the image of the eyeball, using as training data an image of the eyeball and the direction in which the eyeball in the image was actually looking. The AI ​​program may be provided in either the display device or the photoelectric conversion device, or in an external device. If the external device has the AI ​​program, it may be transmitted to the display device from a server or the like via communication.

[0163] When display control is performed based on visual recognition detection, this embodiment can be preferably applied to smart glasses that further include a photoelectric conversion device that captures an image of the outside world. The smart glasses can display captured external information in real time.

[0164] [Modified embodiment] The present invention is not limited to the above-described embodiments and can be modified in various ways. For example, an example in which part of the configuration of one embodiment is added to another embodiment, or an example in which part of the configuration of another embodiment is replaced with another embodiment, is also an embodiment of the present invention.

[0165] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0166] It should be noted that the above-described embodiments are merely examples of specific embodiments for carrying out the present invention, and the technical scope of the present invention should not be construed as being limited by these embodiments. In other words, the present invention can be carried out in various forms without departing from its technical concept or main features. [Explanation of symbols]

[0167] 20...Photoelectric conversion unit 22...Photon detection element 24...Quench element 30...Pixel signal processing unit 32...Waveform shaping section 34...Digital processing circuit 36...Pixel output circuit 100...Photoelectric conversion device 342...Counter 344...Threshold value determination unit 346...Exposure control unit

Claims

1. an avalanche photodiode; a signal processing circuit including a counter that generates a count value based on photons incident on the avalanche photodiode during a count period, and that outputs the count value at a predetermined frame rate; a quench transistor that returns the avalanche photodiode to a state in which avalanche multiplication can occur again after avalanche multiplication has occurred; a pulse generating unit that outputs a pulse signal whose level changes at a predetermined frequency to the gate of the quench transistor; and a pixel including the pixel transitions from a first state to a second state in which the frequency and the frame rate are higher than those of the first state, depending on a result of a determination based on the count value and a predetermined threshold value; The level of the pulse signal changes multiple times within one count period. A photoelectric conversion device characterized by:

2. The number of bits of the count value in the second state is greater than the number of bits of the count value in the first state.

2. The photoelectric conversion device according to claim 1.

3. an avalanche photodiode; a signal processing circuit including a counter that generates a count value based on photons incident on the avalanche photodiode during a count period, and that outputs the count value at a predetermined frame rate; a quench transistor that returns the avalanche photodiode to a state in which avalanche multiplication can occur again after avalanche multiplication has occurred; a pulse generating unit that outputs a pulse signal whose level changes at a predetermined frequency to the gate of the quench transistor; and a pixel including The pixel transitions from a first state to a second state in which the frequency is higher than in the first state and the number of bits of the count value is greater than in the first state, depending on a result of a determination based on the count value and a predetermined threshold. A photoelectric conversion device characterized by:

4. The count value in the first state does not include the least significant bit of the count value in the second state.

4. The photoelectric conversion device according to claim 2 or 3.

5. The length of one of the high level period and the low level period of the pulse signal is the same between the first state and the second state.

5. The photoelectric conversion device according to claim 1, wherein the first and second electrodes are electrically connected to each other.

6. The pixel transitions from the first state to the second state when the count value exceeds a predetermined threshold.

6. The photoelectric conversion device according to claim 1, wherein the first and second electrodes are electrically connected to each other.

7. The pixel transitions from the first state to the second state when a change in the count value over time exceeds a predetermined threshold.

6. The photoelectric conversion device according to claim 1, wherein the first and second electrodes are electrically connected to each other.

8. A plurality of the pixels are arranged in a plurality of rows and a plurality of columns.

8. The photoelectric conversion device according to claim 1, wherein the first and second electrodes are electrically connected to each other.

9. In the first state, all of the plurality of pixels perform the determination.

9. The photoelectric conversion device according to claim 8.

10. In the first state, the plurality of pixels sequentially perform the determination in an order according to the arrangement of the plurality of pixels.

10. The photoelectric conversion device according to claim 8 or 9.

11. In the first state, the plurality of pixels perform the determination sequentially in a random order with respect to the arrangement of the plurality of pixels.

10. The photoelectric conversion device according to claim 8 or 9.

12. In the first state, the determination is performed for some of the plurality of pixels, and the determination is not performed for other of the plurality of pixels.

12. The photoelectric conversion device according to claim 8, wherein the first and second electrodes are electrically connected to each other.

13. In response to a result of the determination for one pixel among the plurality of pixels, all of the plurality of pixels transition to the second state.

13. The photoelectric conversion device according to claim 8, wherein the first and second electrodes are electrically connected to each other.

14. In accordance with the result of the determination on one pixel of the plurality of pixels, some pixels of the plurality of pixels including at least the one pixel transition to the second state.

13. The photoelectric conversion device according to claim 8, wherein the first and second electrodes are electrically connected to each other.

15. Two of the pixels are adjacent to each other 15. The photoelectric conversion device according to claim 14.

16. All of the plurality of pixels transition to the second state in accordance with the result of the determination for one pixel among the partial pixels.

16. The photoelectric conversion device according to claim 14 or 15.

17. In accordance with the result of the determination for at least one pixel in a pixel block that is a part of the plurality of pixels and includes the plurality of pixels, all pixels that belong to the pixel block transition to the second state.

13. The photoelectric conversion device according to claim 8, wherein the first and second electrodes are electrically connected to each other.

18. A condition after transition to the second state is set according to the number of pixels in the pixel block for which the result of the determination satisfies a predetermined condition.

18. The photoelectric conversion device according to claim 17.

19. The photoelectric conversion device according to any one of claims 1 to 18, a signal processing device that processes a signal output from the photoelectric conversion device; An optical detection system comprising:

20. The signal processing device generates a distance image representing distance information to an object based on the signal.

20. The optical detection system of claim 19.

21. A mobile object, The photoelectric conversion device according to any one of claims 1 to 18, a distance information acquisition means for acquiring distance information to an object from a parallax image based on a signal output from the photoelectric conversion device; a control means for controlling the moving object based on the distance information; A moving object characterized by having:

Images