Photoelectric converter and photodetection system

Abstract

To provide a photoelectric conversion device and a photo-detection system that have reduced power consumption.SOLUTION: A photoelectric conversion device has a pixel including an avalanche photodiode, and a signal processing circuit that includes a counter generating a count value based on photons incident on the avalanche photodiode in a count period and repeatedly outputs the count value for every count period. The pixel makes a transition from a first state to a second state in which the length of the count period is shorter than the first state according to a result of execution of a determination based on the count value and a predetermined threshold.SELECTED DRAWING: Figure 6

Description

Technical Field

[0006] , ,

[0005] , , ,

[0007] , , ,

[0001] The present invention relates to a photoelectric conversion device and a photodetection system.

Background Art

[0002] Patent Document 1 discloses a photoelectric conversion device having avalanche photodiodes arranged in a matrix. An avalanche photodiode amplifies signal charges excited by photons by several times to about several million times by using the avalanche multiplication phenomenon generated by a strong electric field induced in the pn junction of a semiconductor.

[0003] Patent Document 2 discloses an asynchronous solid-state imaging device that operates in response to the detection of an event such as a change in the amount of light. The solid-state imaging device of Patent Document 2 has a detection pixel that detects an event and a counting pixel that counts the number of photons incident on an avalanche photodiode when an event occurs and outputs a pixel signal.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] >In an asynchronous photoelectric conversion device as described in Patent Document 2, reduction of power consumption is required.

[0006] An object of the present invention is to provide a photoelectric conversion device and a photodetection system with reduced power consumption.

Means for Solving the Problems

[0007] According to one disclosure of this specification, a pixel includes an avalanche photodiode and a signal processing circuit that includes a counter that generates a count value based on photons incident on the avalanche photodiode during a counting period and repeatedly outputs the count value for each counting period, wherein the pixel transitions from a first state to a second state in which the length of the counting period is shorter than that of the first state, depending on the result of a determination based on the count value and a predetermined threshold. Furthermore, the power consumption in the first state is smaller than the power consumption in the second state. A photoelectric conversion device characterized by the above is provided.

[0008] According to another disclosure of this specification, a pixel includes an avalanche photodiode and a signal processing circuit that includes a counter that generates a count value based on photons incident on the avalanche photodiode during a counting period and repeatedly outputs the count value for each counting period, wherein the pixel enters a first state or a second state depending on the result of a determination based on the count value and a predetermined threshold. ra Transition to 2 states Furthermore, the interval from the end time of the first count period in the second state to the start time of the second count period following the first count period is shorter than the interval from the end time of the third count period in the first state to the start time of the fourth count period following the third count period. A photoelectric conversion device characterized by the above is provided. [Effects of the Invention]

[0009] According to the present invention, it is possible to provide a photoelectric conversion device and a photodetection system with reduced power consumption. [Brief explanation of the drawing]

[0010] [Figure 1] This is a block diagram (part 1) showing the schematic configuration of a photoelectric conversion device according to the first embodiment of the present invention. [Figure 2] This is a block diagram (part 2) showing the schematic configuration of a photoelectric conversion device according to the first embodiment of the present invention. [Figure 3] This is a block diagram showing an example of the pixel configuration in a photoelectric conversion device according to the first embodiment of the present invention. [Figure 4] This is a perspective view showing an example configuration of a photoelectric conversion device according to the first embodiment of the present invention. [Figure 5]It is a diagram for explaining the basic operation of the photoelectric conversion unit in the photoelectric conversion device according to the first embodiment of the present invention. [Figure 6] It is a block diagram showing a more specific configuration example of pixels in the photoelectric conversion device according to the first embodiment of the present invention. [Figure 7] It is a flowchart showing an example of a driving method in the photoelectric conversion device according to the first embodiment of the present invention. [Figure 8] It is a timing diagram showing an example of a driving method of pixels in the photoelectric conversion device according to the first embodiment of the present invention. [Figure 9] It is a block diagram showing a configuration example of pixels in the photoelectric conversion device according to the second embodiment of the present invention. [Figure 10] It is a timing diagram showing an example of a driving method of pixels in the photoelectric conversion device according to the second embodiment of the present invention. [Figure 11] It is a block diagram showing a configuration example of pixels in the photoelectric conversion device according to the third embodiment of the present invention. [Figure 12] It is a timing diagram showing an example of a driving method of pixels in the photoelectric conversion device according to the third embodiment of the present invention. [Figure 13] It is a timing diagram showing a modified example of a driving method of pixels in the photoelectric conversion device according to the third embodiment of the present invention. [Figure 14] It is a flowchart showing an example of a driving method in the photoelectric conversion device according to the fourth embodiment of the present invention. [Figure 15] It is a schematic diagram showing an example of the arrangement and driving order of pixels in the photoelectric conversion device according to the fifth embodiment of the present invention. [Figure 16] It is a timing diagram showing an example of a driving method of pixels in the photoelectric conversion device according to the fifth embodiment of the present invention. [Figure 17] It is a timing diagram showing a modified example of a driving method of pixels in the photoelectric conversion device according to the fifth embodiment of the present invention. [Figure 18] It is a schematic diagram showing an example of the arrangement and driving method of pixels in the photoelectric conversion device according to the sixth embodiment of the present invention. [Figure 19]It is a schematic diagram showing an example of pixel arrangement and driving method in a photoelectric conversion device according to the seventh embodiment of the present invention. [Figure 20] It is a flowchart showing an example of a driving method in a photoelectric conversion device according to the seventh embodiment of the present invention. [Figure 21] It is a flowchart showing a modified example of a driving method in a photoelectric conversion device according to the seventh embodiment of the present invention. [Figure 22] It is a schematic diagram showing an example of pixel arrangement and driving method in a photoelectric conversion device according to the eighth embodiment of the present invention. [Figure 23] It is a flowchart showing an example of a driving method in a photoelectric conversion device according to the eighth embodiment of the present invention. [Figure 24] It is a block diagram showing a schematic configuration of a light detection system according to the ninth embodiment of the present invention. [Figure 25] It is a block diagram showing a schematic configuration of a distance image sensor according to the tenth embodiment of the present invention. [Figure 26] It is a schematic diagram showing a configuration example of an endoscopic surgery system according to the eleventh embodiment of the present invention. [Figure 27] It is a schematic diagram showing a configuration example of a moving body according to the twelfth embodiment of the present invention. [Figure 28] It is a block diagram showing a schematic configuration of a light detection system according to the twelfth embodiment of the present invention. [Figure 29] It is a flowchart showing the operation of a light detection system according to the twelfth embodiment of the present invention. [Figure 30] It is a schematic diagram showing a schematic configuration of a light detection system according to the thirteenth embodiment of the present invention.

Embodiments for Carrying out the Invention

[0011] Embodiments of the present invention will be described below with reference to the drawings. Common reference numerals are used for identical or corresponding elements across multiple drawings, and their descriptions may be omitted or simplified. The embodiments shown below are intended to embody the technical concept of the present invention and do not limit it. The sizes and positional relationships of the components shown in each drawing may be exaggerated for clarity.

[0012] [First Embodiment] A photoelectric converter according to a first embodiment of the present invention will be described with reference to Figures 1 to 8. Figures 1 and 2 are block diagrams showing the schematic configuration of the photoelectric converter according to this embodiment. Figure 3 is a block diagram showing an example of the pixel configuration of the photoelectric converter according to this embodiment. Figure 4 is a perspective view showing an example of the configuration of the photoelectric converter according to this embodiment. Figure 5 is a diagram illustrating the basic operation of the photoelectric conversion unit of the photoelectric converter according to this embodiment. Figure 6 is a block diagram showing a more specific example of the pixel configuration in the photoelectric converter according to this embodiment. Figure 7 is a flowchart showing an example of a driving method in the photoelectric converter according to this embodiment. Figure 8 is a timing diagram showing an example of a pixel driving method in the photoelectric converter according to this embodiment.

[0013] As shown in Figure 1, the photoelectric conversion device 100 according to this embodiment includes a pixel unit 10, a vertical scanning circuit unit 40, a readout circuit unit 50, a horizontal scanning circuit unit 60, a control pulse generation unit 80, and an output circuit unit 90. In the following description, the photoelectric conversion device is assumed to be an asynchronous imaging device using an avalanche photodiode, but is not limited to this. Examples of photoelectric conversion devices include, in addition to the imaging device described below, distance measuring devices (devices for distance measurement using focus detection or TOF (Time Of Flight)), photometric devices (devices for measuring the amount of incident light, etc.).

[0014] The pixel unit 10 is provided with multiple pixels 12 arranged in an array such that they form multiple rows and multiple columns. Each pixel 12 may consist of a photoelectric conversion unit including a photon detection element and a pixel signal processing unit that processes the signal output from the photoelectric conversion unit, as will be described later. The number of pixels 12 constituting the pixel unit 10 is not particularly limited. For example, the pixel unit 10 can be composed of multiple pixels 12 arranged in an array of several thousand rows x several thousand columns, as in a typical digital camera. Alternatively, the pixel unit 10 may be composed of multiple pixels 12 arranged in one row or one column. Alternatively, the pixel unit 10 may be composed of a single pixel 12.

[0015] Each row of the pixel array in the pixel section 10 has a control line 14 extending in a first direction (horizontal direction in Figure 1). The control line 14 is connected to each pixel 12 arranged in the first direction and forms a common signal line for these pixels 12. The first direction in which the control line 14 extends may be referred to as the row direction or horizontal direction. Each of the control lines 14 may include multiple signal lines for supplying multiple types of control signals to the pixels 12.

[0016] Furthermore, each row of the pixel array in the pixel section 10 has data lines 16 extending in a second direction (vertical direction in Figure 1) that intersects the first direction. The data lines 16 are connected to each of the pixels 12 arranged in the second direction, forming a common signal line for these pixels 12. The second direction in which the data lines 16 extend may be referred to as the column direction or the vertical direction. Each of the data lines 16 may include multiple signal lines for transferring multi-bit digital signals output from the pixels 12 bit by bit.

[0017] Each row's control line 14 is connected to the vertical scanning circuit unit 40. The vertical scanning circuit unit 40 is a control unit that receives control signals output from the control pulse generation unit 80, generates control signals to drive the pixels 12, and supplies them to the pixels 12 via the control lines 14. Logic circuits such as shift registers and address decoders may be used in the vertical scanning circuit unit 40. The vertical scanning circuit unit 40 sequentially scans the pixels 12 in the pixel unit 10 row by row and outputs the pixel signal of each pixel 12 to the readout circuit unit 50 via the data lines 16.

[0018] Each data line 16 in each column is connected to the readout circuit 50. The readout circuit 50 includes a plurality of holding units (not shown) corresponding to each column of the pixel array of the pixel unit 10, and has the function of holding the pixel signals of the pixels 12 of each column, which are output row by row from the pixel unit 10 via the data lines 16, in the holding unit of the corresponding column.

[0019] The horizontal scanning circuit unit 60 is a control unit that receives a control signal output from the control pulse generation unit 80, generates a control signal for reading pixel signals from the holding units of each column of the reading circuit unit 50, and supplies it to the reading circuit unit 50. Logic circuits such as shift registers and address decoders may be used in the horizontal scanning circuit unit 60. The horizontal scanning circuit unit 60 sequentially scans the holding units of each column of the reading circuit unit 50 and sequentially outputs the pixel signals held in each to the output circuit unit 90.

[0020] The output circuit section 90 has an external interface circuit and is a circuit section for outputting the pixel signal output from the readout circuit section 50 to the outside of the photoelectric converter 100. The external interface circuit provided by the output circuit section 90 is not particularly limited. The external interface circuit may be composed of, for example, a SerDes (SERializer / DESerializer) transmission circuit. The SerDes transmission circuit may be, for example, an LVDS (Low Voltage Differential Signaling) circuit or an SLVS (Scalable Low Voltage Signaling) circuit.

[0021] The control pulse generation unit 80 is a control circuit that generates control signals to control the operation and timing of the vertical scanning circuit unit 40, the readout circuit unit 50, and the horizontal scanning circuit unit 60, and supplies them to each functional block. At least a portion of the control signals that control the operation and timing of the vertical scanning circuit unit 40, the readout circuit unit 50, and the horizontal scanning circuit unit 60 may be supplied from outside the photoelectric converter 100.

[0022] Note that the connection configuration of each functional block of the photoelectric converter 100 is not limited to the configuration example shown in Figure 1, and can also be configured as shown in Figure 2, for example.

[0023] In the configuration example shown in Figure 2, data lines 16 extending in a first direction are provided for each row of the pixel array in the pixel unit 10. The data lines 16 are connected to each pixel 12 arranged in the first direction and form a common signal line for these pixels 12. In addition, control lines 18 extending in a second direction are provided for each column of the pixel array in the pixel unit 10. The control lines 18 are connected to each pixel 12 arranged in the second direction and form a common signal line for these pixels 12.

[0024] Each row's control line 18 is connected to the horizontal scanning circuit unit 60. The horizontal scanning circuit unit 60 receives a control signal output from the control pulse generation unit 80, generates a control signal for reading pixel signals from the pixels 12, and supplies it to the pixels 12 via the control lines 18. Specifically, the horizontal scanning circuit unit 60 sequentially scans multiple pixels 12 of the pixel unit 10 in column units and outputs the pixel signals of the pixels 12 in each row belonging to the selected column to the data lines 16.

[0025] Each row's data line 16 is connected to the readout circuit 50. The readout circuit 50 includes a plurality of holding units (not shown) corresponding to each row of the pixel array of the pixel unit 10, and has the function of holding the pixel signals of the pixels 12 of each row, which are output column by column from the pixel unit 10 via the data line 16, in the holding unit of the corresponding row.

[0026] The readout circuit 50 receives a control signal output from the control pulse generation unit 80 and sequentially outputs the pixel signals held in the holding unit for each row to the output circuit 90. Other configurations in the example configuration in Figure 2 may be the same as those in the example configuration in Figure 1.

[0027] Each pixel 12, as shown in Figure 3, includes a photoelectric conversion unit 20 and a pixel signal processing unit 30 (signal processing circuit). The photoelectric conversion unit 20 includes a photon detection element 22 and a quench element 24. The pixel signal processing unit 30 includes a waveform shaping unit 32, a digital processing circuit 34, and a pixel output circuit 36.

[0028] The photon detection element 22 may be an avalanche photodiode (hereinafter referred to as "APD"). The anode of the APD constituting the photon detection element 22 is connected to a node to which voltage VL is supplied. The cathode of the APD constituting the photon detection element 22 is connected to one terminal of the quench element 24. The connection node between the photon detection element 22 and the quench element 24 is the output node of the photoelectric conversion unit 20. The other terminal of the quench element 24 is connected to a node to which a voltage VH higher than voltage VL is supplied. Voltages VL and VH are set so that a reverse bias voltage sufficient for the APD to perform avalanche multiplication operation is applied. In one example, a negative high voltage is applied as voltage VL, and a positive voltage of about the power supply voltage is applied as voltage VH. For example, voltage VL is -30V and voltage VH is 1V.

[0029] The photon detection element 22 can be composed of an APD as described above. By supplying the APD with a reverse bias voltage sufficient for avalanche multiplication, the charge generated by the incidence of light on the APD undergoes avalanche multiplication, and an avalanche multiplication current is generated. There are two operating modes when a reverse bias voltage is supplied to the APD: Geiger mode and linear mode. Geiger mode is an operating mode in which the voltage applied between the anode and cathode is a reverse bias voltage greater than the breakdown voltage of the APD. Linear mode is an operating mode in which the voltage applied between the anode and cathode is a reverse bias voltage near or below the breakdown voltage of the APD. An APD operating in Geiger mode is called a SPAD (Single Photon Avalanche Diode). The APD constituting the photon detection element 22 may be configured to operate in linear mode or in Geiger mode. In particular, SPADs are preferred because the potential difference is larger than that of a linear-mode APD, and the breakdown voltage effect is more pronounced.

[0030] The quench element 24 has the function of converting the change in avalanche multiplication current generated in the photon detection element 22 into a voltage signal. Furthermore, the quench element 24 functions as a load circuit (quench circuit) during signal multiplication by avalanche multiplication, reducing the voltage applied to the photon detection element 22 and suppressing avalanche multiplication. This operation by the quench element 24 to suppress avalanche multiplication is called the quench operation. The quench element 24 also has the function of returning the voltage supplied to the photon detection element 22 to voltage VH by supplying current to compensate for the voltage drop caused by the quench operation. This operation by the quench element 24 to return the voltage supplied to the photon detection element 22 to voltage VH is called the recharge operation. The quench element 24 can be composed of a resistor, a MOS transistor, or the like.

[0031] The waveform shaping unit 32 has an input node to which the output signal from the photoelectric conversion unit 20 is supplied, and an output node. The waveform shaping unit 32 has the function of converting the analog signal supplied from the photoelectric conversion unit 20 into a pulse signal. As shown in Figure 3, the waveform shaping unit 32 may be composed of an inverter circuit or the like. The output node of the waveform shaping unit 32 is connected to the digital processing circuit 34.

[0032] The digital processing circuit 34 has an input node to which the output signal from the waveform shaping unit 32 is supplied, an input node connected to the control line 14, and an output node. The digital processing circuit 34 also has a counter, which will be described later. The counter has the function of counting pulses superimposed on the signal output from the waveform shaping unit 32 and holding the count value which is the result of the count. The signal supplied from the vertical scanning circuit unit 40 to the digital processing circuit 34 via the control line 14 may include a timer clock signal for controlling the pulse counting period (exposure period). The output node of the digital processing circuit 34 is connected to the data line 16 via the pixel output circuit 36.

[0033] The pixel output circuit 36 ​​has a function to switch the electrical connection state (connected or disconnected) between the digital processing circuit 34 and the data line 16. The pixel output circuit 36 ​​switches the connection state between the digital processing circuit 34 and the data line 16 in accordance with the control signal supplied from the vertical scanning circuit unit 40 via the control line 14 (in the configuration example of Figure 2, the control signal supplied from the horizontal scanning circuit unit 60 via the control line 18). The pixel output circuit 36 ​​may include a buffer circuit for outputting signals.

[0034] Pixel 12 is typically a unit structure that outputs a pixel signal for forming an image. However, in cases where the purpose is distance measurement using the TOF (Time of Flight) method, pixel 12 does not necessarily have to be a unit structure that outputs a pixel signal for forming an image. That is, pixel 12 can also be a unit structure that outputs a signal for measuring the time and amount of light that arrived.

[0035] Furthermore, the pixel signal processing unit 30 does not necessarily need to be provided for each pixel 12; a single pixel signal processing unit 30 may be provided for multiple pixels 12. In this case, a single pixel signal processing unit 30 can be used to sequentially perform signal processing for multiple pixels 12.

[0036] The photoelectric converter 100 according to this embodiment may be formed on a single substrate, or it may be configured as a stacked photoelectric converter by stacking multiple substrates. In the latter case, for example, as shown in Figure 4, it can be configured as a stacked photoelectric converter by stacking a sensor substrate 110 and a circuit board 120 and electrically connecting them. At least one of the components of the pixel 12, the photon detection element 22, can be placed on the sensor substrate 110. The circuit board 120 can also contain the quench element 24 and the pixel signal processing unit 30, which are components of the pixel 12. The photon detection element 22, the quench element 24, and the pixel signal processing unit 30 are electrically connected via connecting wiring provided for each pixel 12. Furthermore, the circuit board 120 can also contain a vertical scanning circuit section 40, a readout circuit section 50, a horizontal scanning circuit section 60, a control pulse generation section 80, an output circuit section 90, and the like.

[0037] The photon detection elements 22, quench elements 24, and pixel signal processing unit 30 of each pixel 12 are provided on the sensor substrate 110 and the circuit board 120 so as to overlap in a plan view. The vertical scanning circuit unit 40, readout circuit unit 50, horizontal scanning circuit unit 60, control pulse generation unit 80, and output circuit unit 90 can be arranged around the pixel unit 10, which is composed of multiple pixels 12.

[0038] In this specification, "planar view" refers to viewing the sensor substrate 110 from a direction perpendicular to the light incident surface.

[0039] By configuring a stacked photoelectric converter 100, the integration density of elements can be increased, leading to improved functionality. In particular, by arranging the photon detection element 22, the quench element 24, and the pixel signal processing unit 30 on separate substrates, the photon detection elements 22 can be arranged at high density without sacrificing the light-receiving area of ​​the photon detection element 22, thereby improving photon detection efficiency.

[0040] Furthermore, the number of substrates constituting the photoelectric converter 100 is not limited to two; the photoelectric converter 100 may be constructed by stacking three or more substrates.

[0041] Furthermore, while Figure 4 assumes chips that have been diced as sensor substrate 110 and circuit board 120, the sensor substrate 110 and circuit board 120 are not limited to chips. For example, each of the sensor substrate 110 and circuit board 120 may be a wafer. Also, the sensor substrate 110 and circuit board 120 may be stacked in wafer form and then diced, or they may be made into chips and then stacked and bonded.

[0042] Figure 5 illustrates the basic operation of the photoelectric conversion unit 20 and the waveform shaping unit 32. Figure 5(a) is a circuit diagram of the photoelectric conversion unit 20 and the waveform shaping unit 32, Figure 5(b) shows the signal waveform at the input node (node ​​A) of the waveform shaping unit 32, and Figure 5(c) shows the signal waveform at the output node (node ​​B) of the waveform shaping unit 32. For the sake of simplicity, it is assumed here that the waveform shaping unit 32 is configured as an inverter circuit.

[0043] At time t0, a reverse bias voltage with a potential difference equivalent to (VH-VL) is applied to the photon detection element 22. A reverse bias voltage sufficient to cause avalanche multiplication is applied between the anode and cathode of the APD constituting the photon detection element 22, but when no photons are incident on the photon detection element 22, there are no carriers that serve as seeds for avalanche multiplication. Therefore, avalanche multiplication does not occur in the photon detection element 22, and no current flows through the photon detection element 22.

[0044] At the following time t1, assume that a photon is incident on the photon detection element 22. When a photon is incident on the photon detection element 22, electron-hole pairs are generated by photoelectric conversion, and avalanche multiplication occurs using these carriers as a seed, causing an avalanche multiplication current to flow through the photon detection element 22. This avalanche multiplication current flows through the quench element 24, causing a voltage drop across the quench element 24, and the voltage at node A begins to drop. When the voltage drop at node A becomes large and avalanche multiplication stops at time t3, the voltage level at node A will no longer drop.

[0045] When the avalanche multiplication in the photon detection element 22 stops, a current flows from the node to which voltage VL is supplied through the photon detection element 22 to node A to compensate for the voltage drop, and the voltage at node A gradually increases. Subsequently, at time t5, node A settles back to its original voltage level.

[0046] The waveform shaping unit 32 binarizes the signal input from node A according to a predetermined threshold and outputs it from node B. Specifically, the waveform shaping unit 32 outputs a low-level signal from node B when the voltage level of node A exceeds the threshold, and outputs a high-level signal from node B when the voltage level of node A is below the threshold. For example, as shown in Figure 5(b), suppose the voltage of node A is below the threshold during the period from time t2 to time t4. In this case, as shown in Figure 5(c), the signal level at node B is low during the period from time t0 to time t2 and from time t4 to time t5, and high during the period from time t2 to time t4.

[0047] Thus, the analog signal input from node A is waveform-shaped into a digital signal by the waveform shaping unit 32. The pulse signal output from the waveform shaping unit 32 in response to the incidence of photons on the photon detection element 22 is the photon detection pulse signal.

[0048] Figure 6 is a diagram illustrating the configuration of pixel 12 in more detail. In the explanation of Figure 6, the explanation of parts that overlap with Figure 3 or Figure 5 will be omitted or simplified. The digital processing circuit 34 includes a counter 342, a threshold determination unit 344, and an exposure control unit 346.

[0049] As described above, the counter 342 counts pulses based on photons incident on the photon detection element 22 and stores the count value, which is the result of the count. To store the count value, the counter 342 has a bit memory capable of holding multiple bits of digital signals. The count value held in the counter 342 is output to the data line 16 as the count value CNT via the pixel output circuit 36. The count value held in the counter 342 is also output to the threshold determination unit 344. In addition, the counter 342 resets the held count value to its initial value at a timing corresponding to the pulse of the reset signal RES input from the exposure control unit 346.

[0050] The threshold determination unit 344 has the function of determining the event detection result based on the count value input from the counter 342 and a predetermined threshold. The threshold determination unit 344 may be a digital circuit configured to perform comparison processing based on the count value, which is a digital signal. The threshold determination unit 344 outputs an event detection result signal EDR indicating the determination result to the outside of the pixel signal processing unit 30. This event detection result signal EDR may be used for processing within the photoelectric converter 100, supplied to other pixels 12, or used for signal processing outside the photoelectric converter 100. The threshold determination unit 344 also outputs the event detection result signal EDR to the exposure control unit 346.

[0051] The event determined by this process means that the amount of light incident on the photon detection element 22 of the pixel 12 satisfies a predetermined condition. This predetermined condition may be, for example, that the amount of light exceeds a predetermined threshold, or that the amount of change in the amount of light exceeds a predetermined threshold. The photoelectric converter 100 of this embodiment has the function of detecting this event based on the count value and the threshold and causing the state of the pixel 12 to change.

[0052] The exposure control unit 346 has the function of changing the length of the count period (exposure time) in the counter 342 based on the event detection result signal EDR. The exposure control unit 346 may be a digital circuit including a counter that counts the timer clock signal TCLK. The exposure control unit 346 receives the timer clock signal TCLK from outside the vertical scanning circuit unit 40, the control pulse generation unit 80, or the photoelectric converter 100. The exposure control unit 346 counts the number of pulses of the timer clock signal TCLK. When the number of counted pulses reaches a predetermined threshold, the exposure control unit 346 outputs a reset signal RES pulse to the counter 342 to reset the count value held in the counter 342. The exposure control unit 346 can change the length of the count period by changing the count threshold of the number of pulses of the timer clock signal TCLK based on the event detection result signal EDR.

[0053] The method for driving pixels 12 during event detection will be explained with reference to Figures 7 and 8. Figure 7 is a flowchart showing the process from the initial event detection state (first state) to the transition to the imaging state (second state) when a certain pixel 12 detects an event. Figure 8 shows the levels of each signal during the process shown in Figure 7. Figure 8 shows the count value CNT, reset signal RES, event detection result signal EDR, and timer clock signal TCLK.

[0054] In step S11, each pixel 12 in the photoelectric converter 100 is set to an event detection state. The event detection state is a mode in which a pixel signal based on incident light is acquired while consuming less power than in the normal imaging state. The pixel signal acquired in the event detection state is mainly used to determine whether or not to transition to the imaging state. In Figure 8, the period before time t12 is the period in which each pixel 12 is in the event detection state. The period T1 in Figure 8 is the counting period from when the counter 342 is reset by the reset signal RES at time t11 until the counter 342 is reset again at time t12. After each counting period has elapsed, the pixel 12 outputs one count value based on the photons incident on the photon detection element 22 during that counting period. For example, after the period T1 has elapsed, the pixel 12 outputs a count value C2 based on the photons detected during the period T1. In this way, the pixel 12 repeatedly outputs a count value after each counting period has elapsed.

[0055] In step S12, a count value is acquired at the counter 342 of a predetermined pixel 12, and the count value is input to the threshold determination unit 344 of that pixel. The pixels 12 from which the count value is acquired in step S12 may be all of the pixels 10 in the pixel unit 10, some of the pixels 12, or just one pixel 12. Alternatively, the pixels 12 from which the count value is acquired may be selected sequentially from the pixel unit 10.

[0056] In step S13, the threshold determination unit 344 determines whether the count value exceeds a predetermined threshold. If the count value exceeds the predetermined threshold (YES in step S13), the process proceeds to step S14, as an event has been detected. If the count value does not exceed the predetermined threshold (NO in step S13), the process returns to step S12, as no event was detected, and the event detection state continues. Figure 8 shows the timing of operations when it is determined that the count value C2 does not exceed the threshold at time t11, and when it is determined that the count value C2 exceeds the threshold at time t12.

[0057] In step S14, the threshold determination unit 344 raises the event detection result signal EDR to a high level according to the determination result that the count value has exceeded a predetermined threshold. This process corresponds to time t12 in Figure 8. The exposure control unit 346 changes the pulse interval of the reset signal RES based on the event detection result signal EDR. This changes the reset period in the counter 342 and changes the count period. In this way, the pixel 12 transitions from the event detection state to the imaging state. The period from time t12 onwards in Figure 8 is the period during which each pixel 12 is in the imaging state. The period T2 in Figure 8 is the count period from when the counter 342 is reset by the reset signal RES at time t12 until the counter 342 is reset again at time t13. As shown in Figure 8, the length of the period T2, which is the count period in the imaging state, is shorter than the length of the period T1, which is the count period in the event detection state. The operation in the imaging state continues at times t13, t14, etc., after time t12.

[0058] In step S14, the pixels 12 that transition to the imaging state may be only one pixel 12 that has been determined to have exceeded the threshold, a group of pixels 12 including the surrounding pixels, or all pixels 12 in the pixel section 10. A configuration in which only one pixel 12 that has been determined to have exceeded the threshold transitions to the imaging state is desirable from the viewpoint of low power consumption. In contrast, a configuration in which all pixels 12 in the pixel section 10 transition to the imaging state is desirable from the viewpoint of enabling the entire area of ​​the pixel section 10 to transition to a state where imaging can be performed at high speed. A configuration in which a group of pixels 12 transitions to the imaging state is desirable from the viewpoint of balancing low power consumption and high processing speed. Thus, the number of pixels 12 that transition to the imaging state in step S14 can be appropriately selected according to the requirements and other specifications.

[0059] As described above, the photoelectric converter 100 of this embodiment includes a pixel 12 that transitions from an event detection state to an imaging state according to the result of a determination based on a count value and a predetermined threshold. The effects of this configuration will now be explained. In asynchronous photoelectric converters 100, there is a need to reduce power consumption. In particular, there is a strong demand for low power consumption when the photoelectric converter 100 is used in an environment where power supply is limited, such as when it is battery-powered. In this embodiment, as shown in Figure 8, the count period is set to be longer in the event detection state compared to the imaging state. As a result, the output frequency of the count signal is reduced, and the power consumption of the photoelectric converter 100 in the event detection state is reduced.

[0060] In the detection pixels for event detection disclosed in Patent Document 2, the reduction of the power consumption of the detection pixels themselves is not considered. However, in the photoelectric converter 100 of this embodiment, since each pixel 12 can operate in both the event detection state and the imaging state, the power consumption of the pixels themselves used for event detection is reduced.

[0061] As described above, this embodiment provides a photoelectric converter 100 with reduced power consumption.

[0062] [Second Embodiment] A photoelectric conversion device according to a second embodiment of the present invention will be described with reference to Figures 9 and 10. Figure 9 is a block diagram showing an example of the pixel configuration of the photoelectric conversion device according to this embodiment. Figure 10 is a timing diagram showing an example of a pixel driving method in the photoelectric conversion device according to this embodiment.

[0063] The photoelectric conversion device of this embodiment differs from the photoelectric conversion device of the first embodiment in that the quench element 24 is a PMOS transistor, and the quench element 24 can be put into a non-operating state.

[0064] As shown in Figure 9, the photoelectric conversion unit 20 has a quench transistor M1 as an example of a quench element 24. The quench transistor M1 is a PMOS transistor. The pixel 12 also has a pixel control unit 42 that controls the quench transistor M1.

[0065] The drain of the quench transistor M1 is connected to the cathode of the APD constituting the photon detection element 22. The source of the quench transistor M1 is connected to a node to which voltage VH is supplied. The gate of the quench transistor M1 is connected to the pixel control unit 42. In this embodiment, the exposure control unit 346 outputs a control signal PEN to the pixel control unit 42. The pixel control unit 42 controls the quench transistor M1 to be on or off by changing the level of the voltage supplied to the gate of the quench transistor M1 based on the control signal PEN. The pixel control unit 42 may include, for example, an inverter circuit.

[0066] The method for driving pixel 12 when an event is detected will be explained with reference to Figure 10. The flow of the driving method is the same as in Figure 7, so the explanation will be omitted. In Figure 10, in addition to the same signals as in Figure 8, the control signal PEN is shown.

[0067] During the period before time t21, the period from time t22 to time t24, and the period after time t25, the control signal PEN is at a high level. During these periods, the quench transistor M1 is controlled to be ON, and the count value CNT is output sequentially. Also, during the period from time t21 to time t22 and the period from time t24 to time t25, the control signal PEN is at a low level. During these periods, the quench transistor M1 is controlled to be OFF, and the output of the count value CNT is stopped. In this way, power consumption is reduced by performing a decimation operation that stops the output of the count value CNT during certain periods.

[0068] Furthermore, Figure 10 shows the timing of operations when it is determined that the count value C1 does not exceed the threshold at time t21, and that the count value C2 exceeds the threshold at time t23. Therefore, the event detection result signal EDR is low level during the period before time t23, and high level during the period after time t23. In other words, pixel 12 is in an event detection state during the period before time t23, and is in an imaging state during the period after time t23.

[0069] As shown in Figure 10, the length of period T4, which is the stop period of the count signal in the imaging state, is shorter than the length of period T3, which is the stop period of the count signal in the event detection state. In other words, the interval of the count period in the imaging state is shorter than the interval of the count period in the event detection state.

[0070] In this embodiment, as shown in Figure 10, the count period interval is set to be longer in the event detection state compared to the imaging state. As a result, the output frequency of the count signal in the event detection state is reduced, similar to the first embodiment, and the power consumption of the photoelectric converter 100 in the event detection state is reduced.

[0071] As described above, this embodiment provides a photoelectric converter 100 with reduced power consumption.

[0072] In the example shown in Figure 10, the length of the count period is the same between the event detection state and the imaging state. However, the length of the count period may be made different between the event detection state and the imaging state by changing the length of the count period based on the event detection result signal EDR, as in the first embodiment.

[0073] [Third Embodiment] A photoelectric conversion device according to a third embodiment of the present invention will be described with reference to Figures 11 to 13. Figure 11 is a block diagram showing an example of the pixel configuration of the photoelectric conversion device according to this embodiment. Figure 12 is a timing diagram showing an example of a pixel driving method in the photoelectric conversion device according to this embodiment. Figure 13 is a timing diagram showing a modified example of the pixel driving method in the photoelectric conversion device according to this embodiment.

[0074] Similar to the second embodiment, the photoelectric converter of this embodiment includes a quench transistor M1 as the quench element 24. However, it differs from the photoelectric converter of the second embodiment in that a pulse signal is input to the gate of the quench transistor M1, and a recharge operation is performed repeatedly at the frequency of the pulse signal.

[0075] As shown in Figure 11, the photoelectric conversion unit 20 has a quench transistor M1 as an example of a quench element 24. The quench transistor M1 is a PMOS transistor. The pixel 12 also has a pulse generation unit 44 that controls the quench transistor M1.

[0076] The gate of the quench transistor M1 is connected to the pulse generation unit 44. In this embodiment, the threshold determination unit 344 outputs an event detection result signal EDR to the pulse generation unit 44. The pulse generation unit 44 switches the quench transistor M1 on and off at the frequency of the pulse signal by outputting a pulse signal to the gate of the quench transistor M1. This allows the quench transistor M1 to perform a recharge operation, which restores the voltage of the cathode node of the APD constituting the photon detection element 22, at the frequency of the pulse signal. The pulse generation unit 44 may include, for example, a frequency divider circuit that generates a pulse signal of a variable frequency by dividing the clock signal in the photoelectric converter 100. Furthermore, the pulse generation unit 44 can change the frequency of the output pulse signal based on the event detection result signal EDR.

[0077] In the photoelectric converter 100 with the above configuration, if at least one photon is incident on the photon detection element 22 during the photon detection waiting period between recharge operations, the count value held in the counter 342 increases by one. If no photons are incident on the photon detection element 22 during the photon detection waiting period, the count value held in the counter 342 does not increase. In this way, the counter 342 can count the number of periods in which photons are incident and avalanche multiplication occurs among multiple photon detection waiting periods. Since the number of photon detection waiting periods and the length of one photon detection waiting period change depending on the frequency of the pulse signal, the frequency of photon counting by the counter 342 changes by changing the frequency of the pulse signal.

[0078] The method for driving pixel 12 during event detection will be explained with reference to Figure 12. The flow of the driving method is the same as in Figure 7, so the explanation will be omitted. In Figure 12, in addition to the same signals as in Figure 8, a pulse signal PL is shown.

[0079] Figure 12 shows the timing of operations when it is determined that the count value C1 does not exceed the threshold at time t31, and that the count value C2 exceeds the threshold at time t32. Therefore, the event detection result signal EDR is at a low level during the period before time t32, and at a high level during the period after time t32. In other words, pixel 12 is in an event detection state during the period before time t32, and is in an imaging state during the period after time t32. As shown in Figure 12, the frequency of the pulse signal PL in the imaging state is higher than the frequency of the pulse signal PL in the event detection state.

[0080] In this embodiment, as shown in Figure 12, the frequency of the pulse signal PL is set lower in the event detection state compared to the imaging state. This reduces the counting frequency in the event detection state and reduces the power consumption of the photoelectric converter 100 in the event detection state.

[0081] As described above, this embodiment provides a photoelectric converter 100 with reduced power consumption.

[0082] Furthermore, after transitioning to the imaging state, the photon counting period can be shortened by increasing the frequency of the pulse signal PL. This enables imaging in situations where the amount of light changes rapidly, such as detecting light emission in dark environments.

[0083] In the example shown in Figure 12, the length of the count period is the same between the event detection state and the imaging state. However, the length of the count period may be different between the event detection state and the imaging state by changing the length of the count period based on the event detection result signal EDR, as in the first embodiment.

[0084] Next, a modified example of this embodiment will be described with reference to Figure 13. In the example of Figure 12, when the frequency of the pulse signal PL is changed at time t32, the pulse width and pulse period of the pulse signal PL change at the same ratio. In other words, the duty cycle of the pulse signal PL is constant. Since such a frequency change can be achieved with a relatively simple frequency divider circuit, it is desirable from the viewpoint of simplifying the circuit configuration of the pulse generation unit 44. However, the pulse signal PL is not limited to having a constant duty cycle.

[0085] In the example shown in Figure 13, when the frequency of the pulse signal PL is changed, the length of the low-level period of the pulse signal PL is the same between the event detection state and the imaging state. As described above, the low-level period of the pulse signal is the recharge period during which the quench transistor M1 is turned on. This ensures that the length of the recharge operation is constant, and the recharge operation is stabilized. Therefore, in the modified example shown in Figure 13, signal quality may be improved. Note that a configuration in which the recharge operation occurs when the pulse signal PL is at a high level is also possible, such as when the quench transistor M1 is an NMOS. In that case, it is desirable to make the length of the high-level period of the pulse signal PL the same between the event detection state and the imaging state.

[0086] [Fourth Embodiment] A photoelectric conversion device according to a fourth embodiment of the present invention will be described with reference to Figure 14. Figure 14 is a flowchart showing an example of a pixel driving method in the photoelectric conversion device according to this embodiment.

[0087] The photoelectric conversion device of this embodiment differs from the first embodiment in that the determination of the transition to the imaging state is based on the amount of change in the count value. Since any of the first to third embodiments can be applied to other circuit configurations and driving methods, a detailed explanation is omitted.

[0088] Referring to Figure 14, the method for driving pixel 12 when an event is detected will be explained. The operations of steps S11, S12, and S14 are the same as in Figure 7, so their explanation will be omitted. In step S15, the threshold determination unit 344 determines whether the time change of the count value exceeds a predetermined threshold. Here, the time change can be the difference in the count value obtained by subtracting the count value of the (n-1)th count period from the count value of the nth count period, assuming that this determination is performed on the count value output in the nth count period. If the count value exceeds the predetermined threshold (YES in step S15), the process proceeds to step S14, assuming that an event has been detected. If the count value does not exceed the predetermined threshold (NO in step S15), the process returns to step S12, assuming that no event was detected, and the event detection state continues. In order to realize the difference processing described above, the threshold determination unit 344 may include a memory that stores the count value output in the previous count period.

[0089] As described above, the photoelectric converter 100 of this embodiment includes a pixel 12 that transitions from an event detection state to an imaging state according to the result of a determination based on the time change of the count value and a predetermined threshold. In an asynchronous photoelectric converter 100, the time change of brightness may be more important than the brightness of the object itself. An example of such a case is when the photoelectric converter 100 is monitoring a stationary object and detecting the movement of the object. In this embodiment, since the time change of the count value is used as the determination criterion, a more appropriate determination can be made in the above-mentioned situation. Therefore, according to this embodiment, a photoelectric converter 100 that can make a more appropriate determination is provided.

[0090] In the above explanation, an example of calculating the change over time is shown using the count value for the nth count period and the count value for the (n-1)th count period. However, the count values ​​for count periods prior to the (n-2)th count may also be used.

[0091] Furthermore, when the brightness of an object itself is important, a judgment criterion based on the count value itself, as shown in Figure 7, may be preferable to a judgment criterion based on the time change of the count value, as shown in Figure 14. By making a judgment by comparing the count value with a threshold, as shown in Figure 7, a more appropriate judgment can be made in situations such as detecting the brightness of the imaging environment as an event.

[0092] [Fifth Embodiment] A photoelectric conversion device according to a fifth embodiment of the present invention will be described with reference to Figures 15 to 17. Figure 15 is a schematic diagram showing an example of the arrangement and driving sequence of pixels in the photoelectric conversion device according to this embodiment. Figure 16 is a timing diagram showing an example of the pixel driving method in the photoelectric conversion device according to this embodiment. Figure 17 is a timing diagram showing a modified example of the pixel driving method in the photoelectric conversion device according to this embodiment.

[0093] The photoelectric conversion device of this embodiment is an example of a configuration for selecting a pixel 12 on which event detection processing is performed. Other circuit configurations and driving methods can be applied from any of the first to fourth embodiments, so their description is omitted.

[0094] Figure 15 schematically shows the arrangement of pixels 12 in the pixel unit 10 and the driving order in the event detection state. Figure 15 only shows a 4x4 pixel arrangement for simplicity; however, this is an example, and in reality, there may be more rows and columns. The (PDEN11) etc. written within the boxes representing pixels 12 in Figure 15 indicate control signals for outputting count values ​​from each pixel 12. The two-digit numbers such as "11" at the end of the control signal names indicate the row number and column number, respectively. The arrows shown on the boxes representing pixels 12 in Figure 15 indicate the scanning order of the pixels 12. That is, the driving order in the event detection state of this embodiment is as follows: 1 row 1 column pixel 12 → 1 row 2 column pixel 12 → 1 row 3 column pixel 12 → 1 row 4 column pixel 12 → 2 row 1 column pixel 12 → ... Thus, in this embodiment, the driving order corresponds to the arrangement of multiple pixels 12.

[0095] The selection of the pixel 12 that outputs the count value can be performed by a combination of two types of control signals: a control signal output from the vertical scanning circuit unit 40 and a control signal output from the horizontal scanning circuit unit 60. However, for simplicity, in this embodiment, the control signals are shown in each figure assuming that the signal output from pixel 12 is controlled by a single control signal.

[0096] Figure 16 shows the levels and count values ​​CNT of the control signals PDEN11, PDEN12, PDEN13, and PDEN14 that realize the readout order of Figure 15. It is assumed that each pixel 12 is in an event detection state throughout the entire period shown in Figure 16. Before time t41, the control signal PDEN11 becomes high, and the 1-row, 1-column pixel 12 is activated. At time t41, the count value C11 is output from the 1-row, 1-column pixel 12. The two-digit number such as "11" at the end of the count value indicates the row number and column number, respectively. At this time, event detection is determined based on the count value C11 using the same method as described in the above embodiment. From time t41 to time t42, the control signal PDEN12 becomes high, and the 1-row, 2-column pixel 12 is activated. At time t42, the count value C12 is output from the 1-row, 2-column pixel 12. At this time, event detection is determined based on the count value C11 using the same method as described in the above embodiment. Thereafter, the count value is output and determination is performed sequentially for the corresponding pixel 12 each time the count period has elapsed.

[0097] According to this embodiment, power consumption is reduced by keeping the number of pixels 12 operating simultaneously low, while the entire pixel unit 10 can be covered by making the entire pixel unit 10 into a determination by operating all of the multiple pixels 12 of the pixel unit 10 in sequence. This provides a photoelectric converter 100 that achieves both reduced power consumption and high-precision event detection.

[0098] Figure 17 shows a modified version of the driving sequence in Figures 15 and 16. As shown in Figure 17, the driving sequence in this modified version is as follows: pixel 12 in row 1, column 1 → pixel 12 in row 1, column 3 → pixel 12 in row 1, column 4 → pixel 12 in row 1, column 2 → ... Thus, in this embodiment, the driving sequence is random for the array of multiple pixels 12. Note that while Figure 17 shows an example where the order within a row is random, both the row and column may be random.

[0099] In shooting environments such as dark places where the number of incident photons is small, the area of ​​the pixel portion 10 in which an event can be detected may be extremely narrow. In such cases, if sequential scanning is performed along the arrangement of pixels 12, as in the example in Figure 15, the probability of detecting an event may be low. By randomizing the order as in this modified example, the detection probability can be improved when the area of ​​the pixel portion 10 in which an event can be detected is narrow.

[0100] [Sixth Embodiment] A photoelectric conversion device according to a sixth embodiment of the present invention will be described with reference to Figure 18. Figure 18 is a schematic diagram showing an example of the arrangement and driving method of pixels in the photoelectric conversion device according to this embodiment.

[0101] The photoelectric conversion device of this embodiment is an example configuration of a method for selecting a pixel 12 on which event detection processing is performed. Other circuit configurations and driving methods can be applied to any of the first to fifth embodiments, so their description is omitted.

[0102] Figure 18 schematically shows the arrangement of pixels 12 in the pixel section 10 and whether or not they operate in the event detection state. In Figure 18, hatched boxes indicate that in the event detection state, the count value is not output and event detection is not determined for that pixel 12. Pixels 12 in boxes that are not hatched perform count value output and event detection is determined in the event detection state in the same manner as in any of the embodiments described above.

[0103] In this way, when an event is detected, some pixels 12 are omitted to output the count value and determine whether an event has been detected, thereby enabling faster or lower power consumption of the event detection operation.

[0104] In Figure 18, the hatched boxes are distributed in a checkerboard pattern within the pixel section 10. By arranging pixels 12 that do not output count values ​​or perform event detection in this way, the distribution of pixels 12 used for event detection processing becomes uniform. However, the distribution of pixels 12 that do not output count values ​​or perform event detection is not limited to that shown in Figure 18.

[0105] [Seventh Embodiment] A photoelectric conversion device according to a seventh embodiment of the present invention will be described with reference to Figures 19 to 21. Figure 19 is a schematic diagram showing an example of pixel arrangement and driving method in the photoelectric conversion device according to this embodiment. Figure 20 is a flowchart showing an example of a driving method in the photoelectric conversion device according to this embodiment. Figure 21 is a flowchart showing a modified example of the driving method in the photoelectric conversion device according to this embodiment.

[0106] This embodiment of the photoelectric conversion device is an example of a configuration relating to the processing of the transition from the event detection state to the imaging state after event detection. Since any of the first to sixth embodiments can be applied to other circuit configurations and driving methods, a detailed explanation is omitted.

[0107] Figure 19 schematically shows the arrangement of pixels 12 in the pixel section 10 and whether or not there is a transition from the event detection state to the imaging state. The upper part of Figure 19 shows the arrangement of pixel blocks 10a within the pixel section 10. A pixel block 10a is a group of pixels containing multiple pixels 12 within a predetermined range. The lower part of Figure 19 shows the arrangement of multiple pixels 12 contained within one pixel block 10a. In the lower part of Figure 19, a hatched box in one pixel block 10a indicates that the determination result for that pixel 12 satisfies a predetermined condition (count value > threshold) and an event detection determination has been made. In the pixel section 10 of Figure 19, a hatched box indicates that all pixels 12 contained in that pixel block 10a transition to the imaging state.

[0108] Referring to Figure 20, the method for driving pixels 12 when an event is detected will be explained. The operations in steps S11, S12, and S13 are the same as in Figure 7, so their explanation will be omitted. In step S16, all pixels 12 included in the pixel block 10a to which the predetermined pixel whose count value has been detected to exceed a threshold belongs, and in the surrounding pixel blocks 10a, transition from the event detection state to the imaging state. As a result, a drive is performed such that the pixel block 10a containing the pixel 12 that satisfies the predetermined conditions, and the adjacent pixel blocks 10a, transition to the imaging state, as shown in Figure 19.

[0109] As described above, in the photoelectric converter 100 of this embodiment, not only the pixels 12 that satisfy predetermined conditions, but also the surrounding pixels 12 transition from the event detection state to the imaging state. When monitoring a stationary object with the photoelectric converter 100 and detecting the movement of the object, information in the vicinity of the object is important. On the other hand, information from a location far from the object is not very useful. Therefore, when the object is moving, information from the vicinity of the object may be sufficient. In this embodiment, by transitioning the pixels surrounding the pixel 12 that has been determined to have detected an event to the imaging state, it is possible to transition the region useful for imaging as described above to the imaging state, while maintaining a low power consumption state in the less useful region by remaining in the event detection state. Therefore, according to this embodiment, a photoelectric converter 100 is provided that achieves both reduced power consumption and high-precision signal acquisition.

[0110] Figure 21 shows a modified version of the driving method in Figure 20. In Figure 21, steps S17, S18, and S19 are added after the driving method in Figure 20.

[0111] In step S17, a count value is obtained from one of the surrounding pixels 12 that transitioned to the imaging state in step S16. The details of this count value acquisition operation are the same as in step S12.

[0112] In step S18, the threshold determination unit 344 determines whether the count value obtained from the surrounding pixels exceeds a predetermined threshold. If the count value exceeds the predetermined threshold (YES in step S18), the event is considered detected and the process proceeds to step S19. If the count value does not exceed the predetermined threshold (NO in step S18), the event is considered not detected and the process returns to step S17.

[0113] In step S19, all pixels 12 within the pixel unit 10 transition from an event detection state to an imaging state.

[0114] In shooting environments such as dark places where the number of incident photons is low, even if an event is detected, it may be a false detection caused by noise. If all pixels are transitioned to the imaging state in this state, there is a risk of wasting power if a false detection occurs. Therefore, in this modified version, event detection is performed using information from pixels surrounding the pixel where an event has been detected, thereby making a more accurate determination of event detection before transitioning all pixels to the imaging state. This reduces the possibility of wasted power due to false detections.

[0115] [Eighth Embodiment] An eighth embodiment of the present invention, a photoelectric conversion device, will be described with reference to Figures 22 and 23. Figure 22 is a schematic diagram showing an example of pixel arrangement and driving method in the photoelectric conversion device according to this embodiment. Figure 23 is a flowchart showing an example of the driving method in the photoelectric conversion device according to this embodiment.

[0116] This embodiment of the photoelectric conversion device is an example of a configuration related to the processing of the transition from the event detection state to the imaging state after event detection. Other circuit configurations and driving methods can be applied to any of the first to seventh embodiments, so their description is omitted.

[0117] Figure 22 schematically shows the arrangement of pixels 12 in the pixel section 10 and whether or not there is a transition from the event detection state to the imaging state. The arrangement of pixels 12 in the pixel section 10 and the pixel block 10a is the same as in Figure 19. In the lower part of Figure 22, each of the multiple hatched boxes in one pixel block 10a indicates that the determination result for that pixel 12 satisfies a predetermined condition (count value > threshold) and an event detection determination has been made. In other words, in the example of Figure 22, an event detection determination has been made for 5 pixels 12. In the pixel section 10 of Figure 22, the hatched boxes indicate that all pixels 12 included in that pixel block 10a transition to the imaging state.

[0118] Referring to Figure 23, the method for driving pixel 12 when an event is detected will be explained. Since the operations of steps S11 and S13 are the same as in Figure 7, the explanation will be omitted or simplified.

[0119] In step S20, count values ​​are obtained from all pixels 12 within a predetermined pixel block 10a. Then, in step S13, it is determined whether each count value exceeds a predetermined threshold. The content of the determination process for each count value is the same as that described in the first embodiment. If the count value output from at least one pixel 12 exceeds the predetermined threshold (YES in step S13), an event is detected, and the process proceeds to step S21. If the count value does not exceed the predetermined threshold (NO in step S13), no event is detected, and the process returns to step S20, and the event detection state continues.

[0120] In step S21, each pixel in the pixel block 10a, where the above-mentioned count value acquisition and determination have been performed, transitions to an imaging state set to different conditions according to the number of pixels 12 that have exceeded the threshold (5 in the example of Figure 22). In the configuration of the first embodiment, "different conditions" corresponds to changing the length of period T2 according to the number of detections. In this case, it is preferable to shorten period T2 and increase the frequency of outputting the count value as the number of detections increases, thereby improving the accuracy of the output signal. In the configuration of the second embodiment, "different conditions" corresponds to changing the length of period T4 according to the number of detections. In this case, it is preferable to shorten period T4 and shorten the period during which no count value is output as the number of detections increases, thereby improving the accuracy of the output signal. In the configuration of the third embodiment, "different conditions" corresponds to changing the frequency after time t32 according to the number of detections. In this case, it is preferable to set the frequency after time t32 higher as the number of detections increases, thereby improving the accuracy of the output signal by increasing the frequency of recharge operations.

[0121] As described above, in the photoelectric converter 100 of this embodiment, the state of the pixel 12 transitions to an imaging state set to different conditions depending on the number of detections in the pixel block 10a. Areas with a large number of detections are often areas where imaging under different conditions than others is required, such as places where many people are gathered. Therefore, it is desirable that the imaging state after the transition is set to different conditions according to the number of detections. According to this embodiment, a photoelectric converter 100 is provided that can acquire signals under more appropriate conditions.

[0122] Furthermore, as mentioned above, it is desirable that the conditions are such that the higher the number of detections, the more accurate the image state becomes after the transition. This example is even more effective in imaging environments where high accuracy in specific parts is required, such as in facial recognition.

[0123] [Ninth Embodiment] A photodetection system according to the ninth embodiment of the present invention will be described with reference to Figure 24. Figure 24 is a block diagram of the photodetection system according to this embodiment. The photodetection system of this embodiment is an imaging system that acquires an image based on incident light.

[0124] The photoelectric conversion device in the above-described embodiment is applicable to various imaging systems. Examples of imaging systems include digital still cameras, digital camcorders, camera heads, photocopiers, fax machines, mobile phones, in-vehicle cameras, observation satellites, and surveillance cameras. Figure 24 shows a block diagram of a digital still camera as an example of an imaging system.

[0125] The imaging system 7 shown in Figure 24 includes a barrier 706, a lens 702, an aperture 704, an imaging device 70, a signal processing unit 708, a timing generation unit 720, an overall control / calculation unit 718, a memory unit 710, a recording medium control I / F unit 716, a recording medium 714, and an external I / F unit 712. The barrier 706 protects the lens, and the lens 702 forms an optical image of the subject on the imaging device 70. The aperture 704 makes the amount of light passing through the lens 702 variable. The imaging device 70 is configured as a photoelectric converter in the above embodiment and converts the optical image formed by the lens 702 into image data. The signal processing unit 708 performs various corrections, data compression, and other processing on the imaging data output from the imaging device 70.

[0126] The timing generation unit 720 outputs various timing signals to the imaging device 70 and the signal processing unit 708. The overall control / calculation unit 718 controls the entire digital still camera, and the memory unit 710 temporarily stores image data. The recording medium control I / F unit 716 is an interface for recording or reading image data to or from the recording medium 714, which is a removable recording medium such as a semiconductor memory for recording or reading image data. The external I / F unit 712 is an interface for communicating with an external computer or the like. Timing signals and the like may be input from outside the imaging system 7, and the imaging system 7 only needs to have at least the imaging device 70 and the signal processing unit 708 that processes the image signals output from the imaging device 70.

[0127] In this embodiment, the imaging device 70 and the signal processing unit 708 may be arranged on the same semiconductor substrate. Alternatively, the imaging device 70 and the signal processing unit 708 may be arranged on separate semiconductor substrates.

[0128] Furthermore, each pixel of the imaging device 70 may include a first photoelectric conversion unit and a second photoelectric conversion unit. The signal processing unit 708 can process the pixel signal based on the charge generated in the first photoelectric conversion unit and the pixel signal based on the charge generated in the second photoelectric conversion unit to obtain distance information from the imaging device 70 to the subject.

[0129] [Tenth Embodiment] Figure 25 is a block diagram of the light detection system according to this embodiment. More specifically, Figure 25 is a block diagram of a distance image sensor using the photoelectric conversion device described in the above embodiment.

[0130] As shown in Figure 25, the distance image sensor 401 comprises an optical system 402, a photoelectric converter 403, an image processing circuit 404, a monitor 405, and a memory 406. The distance image sensor 401 receives light (modulated light, pulsed light) that is emitted from the light source device 411 toward the subject and reflected from the surface of the subject. Based on the time from emission to reception, the distance image sensor 401 can acquire a distance image corresponding to the distance to the subject.

[0131] The optical system 402 includes one or more lenses and guides the image light (incident light) from the subject to the photoelectric converter 403, where it forms an image on the light-receiving surface (sensor part) of the photoelectric converter 403.

[0132] The photoelectric converter 403 can be any of the photoelectric converters described in the above-described embodiments. The photoelectric converter 403 supplies a distance signal indicating the distance obtained from the received light signal to the image processing circuit 404.

[0133] The image processing circuit 404 performs image processing to construct a distance image based on the distance signal supplied from the photoelectric converter 403. The distance image (image data) obtained through image processing can be displayed on the monitor 405 and stored (recorded) in the memory 406.

[0134] The distance image sensor 401 configured in this way can acquire accurate distance images by applying the photoelectric conversion device described above.

[0135] [Embodiment No. 11] The technology described herein can be applied to a variety of products. For example, the technology described herein may be applied to an endoscopic surgical system, which is an example of a photodetection system.

[0136] Figure 26 is a schematic diagram of the endoscopic surgical system in this embodiment. Figure 26 shows a surgeon (physician) 1131 performing surgery on a patient 1132 on a patient bed 1133 using the endoscopic surgical system 1103. As shown in the figure, the endoscopic surgical system 1103 comprises an endoscope 1100, surgical instruments 1110, an arm 1121, and a cart 1134 on which various devices for endoscopic surgery are mounted.

[0137] The endoscope 1100 comprises a barrel 1101, the tip of which is inserted into the body cavity of the patient 1132 for a predetermined length, and a camera head 1102 connected to the proximal end of the barrel 1101. Figure 26 shows the endoscope 1100 configured as a so-called rigid endoscope having a rigid barrel 1101, but the endoscope 1100 may also be configured as a so-called flexible endoscope having a flexible barrel.

[0138] An opening into which an objective lens is fitted is provided at the tip of the endoscope tube 1101. A light source device 1203 is connected to the endoscope 1100. The light generated by the light source device 1203 is guided to the tip of the endoscope tube by a light guide extending inside the endoscope tube 1101, and is irradiated through the objective lens towards the object to be observed inside the body cavity of the patient 1132. The endoscope 1100 may be a straight-viewing endoscope, an oblique-viewing endoscope, or a side-viewing endoscope.

[0139] The camera head 1102 contains an optical system and a photoelectric converter. Reflected light from the object being observed (observation light) is focused by the optical system into the photoelectric converter. The photoelectric converter converts the observation light into electrical signals, generating an electrical signal corresponding to the observation light, i.e., an image signal corresponding to the observed image. The photoelectric converter can be any of the photoelectric converters described in the embodiments described above. The image signal is transmitted as RAW data to the camera control unit (CCU) 1135.

[0140] The CCU1135 consists of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), etc., and comprehensively controls the operation of the endoscope 1100 and the display device 1136. Furthermore, the CCU1135 receives image signals from the camera head 1102 and performs various image processing operations on these image signals, such as development processing (demosaic processing), to display images based on the image signals.

[0141] The display device 1136 displays an image based on an image signal that has been processed by the CCU 1135, under control from the CCU 1135.

[0142] The light source device 1203 is equipped with a light source such as an LED (Light Emitting Diode) and supplies illumination light to the endoscope 1100 when photographing the surgical area, etc.

[0143] The input device 1137 is an input interface for the endoscopic surgical system 1103. The user can input various types of information and instructions to the endoscopic surgical system 1103 via the input device 1137.

[0144] The treatment instrument control device 1138 controls the driving of the energy treatment instrument 1112 for purposes such as tissue cauterization, incision, or blood vessel sealing.

[0145] The light source device 1203 is capable of supplying illumination light to the endoscope 1100 when photographing the surgical area, and may be, for example, an LED, a laser light source, or a white light source consisting of a combination thereof. When a white light source is configured by a combination of RGB laser light sources, the output intensity and output timing of each color (each wavelength) can be controlled with high precision. Therefore, the white balance of the captured image can be adjusted in the light source device 1203. In this case, the laser light from each of the RGB laser light sources may be irradiated onto the observation target in a time-division manner, and the drive of the image sensor of the camera head 1102 may be controlled in synchronization with the irradiation timing. This makes it possible to capture images corresponding to each of the RGB colors in a time-division manner. With this method, a color image can be obtained without providing a color filter on the image sensor.

[0146] Furthermore, the drive of the light source device 1203 may be controlled so that the intensity of the light output from the light source device 1203 is changed at predetermined time intervals. By controlling the drive of the image sensor of the camera head 1102 in synchronization with the timing of the change in light intensity to acquire images in time division and combining these images, it is possible to generate a high dynamic range image without so-called black crushing and white clipping.

[0147] Furthermore, the light source device 1203 may be configured to supply light in a predetermined wavelength band corresponding to special light observation. In special light observation, for example, the wavelength dependence of light absorption in body tissue can be utilized. Specifically, by irradiating with narrowband light compared to the irradiation light used during normal observation (i.e., white light), predetermined tissues such as blood vessels on the surface of mucosa can be imaged with high contrast. Alternatively, in special light observation, fluorescence observation may be performed to obtain an image from fluorescence generated by irradiation with excitation light. In fluorescence observation, excitation light can be irradiated onto body tissue and fluorescence from the body tissue can be observed, or a reagent such as indocyanine green (ICG) can be injected into body tissue and excitation light corresponding to the fluorescence wavelength of the reagent can be irradiated onto the body tissue to obtain a fluorescence image. The light source device 1203 may be configured to supply narrowband light and / or excitation light corresponding to such special light observation.

[0148] [Twelfth Embodiment] The light detection system and mobile body of this embodiment will be described with reference to Figures 27, 28(a), 28(b), 28(c), and 29. In this embodiment, an example of an in-vehicle camera is shown as the light detection system.

[0149] Figure 27 is a schematic diagram of the photodetection system in this embodiment, showing an example of a vehicle system and a photodetection system mounted on the vehicle system. The photodetection system 1301 includes a photoelectric converter 1302, an image preprocessing unit 1315, an integrated circuit 1303, and an optical system 1314. The optical system 1314 forms an optical image of the subject on the photoelectric converter 1302. The photoelectric converter 1302 converts the optical image of the subject formed by the optical system 1314 into an electrical signal. The photoelectric converter 1302 is one of the photoelectric converters in each of the embodiments described above. The image preprocessing unit 1315 performs predetermined signal processing on the signal output from the photoelectric converter 1302. The functions of the image preprocessing unit 1315 may be incorporated into the photoelectric converter 1302. The photodetection system 1301 is provided with at least two sets of optical systems 1314, photoelectric converters 1302, and image preprocessing units 1315, and the output from each set of image preprocessing units 1315 is input to the integrated circuit 1303.

[0150] The integrated circuit 1303 is an integrated circuit for imaging system applications and includes an image processing unit 1304 with a storage medium 1305, an optical distance measuring unit 1306, a parallax calculation unit 1307, an object recognition unit 1308, and an anomaly detection unit 1309. The image processing unit 1304 performs image processing such as development and defect correction on the output signal of the image preprocessing unit 1315. The storage medium 1305 performs primary storage of the captured image and stores the defect positions of the captured pixels. The optical distance measuring unit 1306 focuses on or measures the distance of the subject. The parallax calculation unit 1307 calculates distance information from multiple image data (parallax images) acquired by multiple photoelectric converters 1302. The object recognition unit 1308 recognizes subjects such as cars, roads, signs, and people. When the anomaly detection unit 1309 detects an anomaly in the photoelectric converter 1302, it alerts the main control unit 1313 to the anomaly.

[0151] The integrated circuit 1303 may be implemented by specially designed hardware, by a software module, or by a combination of these. It may also be implemented by an FPGA (Field Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), or a combination of these.

[0152] The main control unit 1313 coordinates and controls the operation of the light detection system 1301, vehicle sensor 1310, control unit 1320, etc. Alternatively, there may be no main control unit 1313, and the light detection system 1301, vehicle sensor 1310, and control unit 1320 may each have their own communication interfaces, and each may send and receive control signals via a communication network, for example using the CAN standard.

[0153] The integrated circuit 1303 has the function of receiving control signals from the main control unit 1313, or transmitting control signals or set values ​​to the photoelectric converter 1302 by its own control unit.

[0154] The light detection system 1301 is connected to the vehicle sensor 1310 and can detect the vehicle's driving conditions, such as vehicle speed, yaw rate, and steering angle, as well as the external environment and the status of other vehicles and obstacles. The vehicle sensor 1310 also serves as a distance information acquisition means for acquiring distance information to objects. Furthermore, the light detection system 1301 is connected to the driver assistance control unit 1311, which performs various driving assistance functions such as automatic steering, automatic cruising, and collision avoidance. In particular, regarding the collision judgment function, the system determines whether a collision with another vehicle or obstacle has occurred and estimates the collision based on the detection results of the light detection system 1301 and the vehicle sensor 1310. This enables avoidance control when a collision is estimated and activation of safety devices in the event of a collision.

[0155] Furthermore, the light detection system 1301 is also connected to a warning device 1312 that issues a warning to the driver based on the judgment result of the collision judgment unit. For example, if the collision judgment unit determines that there is a high probability of collision, the main control unit 1313 performs vehicle control such as applying the brakes, releasing the accelerator, or suppressing engine output to avoid a collision or mitigate damage. The warning device 1312 issues a warning to the user by means of emitting a sound or other warning, displaying warning information on display screens such as the car navigation system and meter panel, and applying vibration to the seat belt and steering wheel.

[0156] The light detection system 1301 in this embodiment can capture images of the area around the vehicle, for example, in front of or behind it. Figures 28(a), 28(b), and 28(c) are schematic diagrams of a moving object in this embodiment, showing a configuration in which the area in front of the vehicle is captured by the light detection system 1301.

[0157] The two photoelectric converters 1302 are positioned in front of the vehicle 1300. Specifically, it is preferable that the center line of the vehicle 1300 with respect to its direction of movement or external shape (e.g., vehicle width) is considered as the axis of symmetry, and that the two photoelectric converters 1302 are positioned symmetrically with respect to the axis of symmetry. This makes it possible to effectively acquire distance information between the vehicle 1300 and the object being photographed and to determine the possibility of collision. Furthermore, it is preferable that the photoelectric converters 1302 are positioned so as not to obstruct the driver's field of view when the driver is viewing the situation outside the vehicle 1300 from the driver's seat. The warning device 1312 is preferably positioned so as to be easily visible to the driver.

[0158] Next, the fault detection operation of the photoelectric converter 1302 in the photodetection system 1301 will be explained using Figure 29. Figure 29 is a flowchart showing the operation of the photodetection system in this embodiment. The fault detection operation of the photoelectric converter 1302 can be performed according to steps S1410 to S1480 shown in Figure 29.

[0159] In step S1410, the startup settings for the photoelectric converter 1302 are performed. Specifically, setting information for the operation of the photoelectric converter 1302 is transmitted from outside the photodetection system 1301 (for example, from the main control unit 1313) or from inside the photodetection system 1301, and the photoelectric converter 1302 starts the imaging operation and fault detection operation.

[0160] Next, in step S1420, the photoelectric converter 1302 acquires a pixel signal from the active pixels. Also, in step S1430, the photoelectric converter 1302 acquires an output value from a fault detection pixel provided for fault detection. This fault detection pixel is equipped with a photoelectric conversion element, just like the active pixels. A predetermined voltage is written to this photoelectric conversion element. The fault detection pixel outputs a signal corresponding to the voltage written to this photoelectric conversion element. Note that steps S1420 and S1430 may be executed in the reverse order.

[0161] Next, in step S1440, the light detection system 1301 determines whether the expected output value of the fault detection pixel matches the actual output value from the fault detection pixel. If the result of the matching determination in step S1440 shows that the expected output value and the actual output value match, the light detection system 1301 proceeds to step S1450, determines that the imaging operation is being performed normally, and proceeds to step S1460. In step S1460, the light detection system 1301 transmits the pixel signals of the scan row to the storage medium 1305 for temporary storage. After that, the light detection system 1301 returns to step S1420 and continues the fault detection operation. On the other hand, if the result of the matching determination in step S1440 shows that the expected output value and the actual output value do not match, the light detection system 1301 proceeds to step S1470. In step S1470, the photodetection system 1301 determines that there is an abnormality in the imaging operation and issues an alarm to the main control unit 1313 or the alarm device 1312. The alarm device 1312 displays that an abnormality has been detected on its display unit. Subsequently, in step S1480, the photodetection system 1301 stops the photoelectric converter 1302 and terminates the operation of the photodetection system 1301.

[0162] In this embodiment, an example is shown where the flowchart is looped every row, but the flowchart may be looped every multiple rows, or the fault detection operation may be performed every frame. The alarm in step S1470 may be notified to an external party via a wireless network.

[0163] Furthermore, although this embodiment describes control to avoid collisions with other vehicles, it can also be applied to control that automatically follows other vehicles or control that automatically drives without deviating from the lane. Moreover, the light detection system 1301 can be applied not only to vehicles such as the vehicle itself, but also to moving objects (mobile devices) such as ships, aircraft, or industrial robots. In addition, it can be applied not only to moving objects, but also to a wide range of devices that utilize object recognition, such as intelligent transportation systems (ITS).

[0164] The photoelectric conversion device of the present invention may further be configured to acquire various types of information, such as distance information.

[0165] [13th Embodiment] Figure 30(a) shows a specific example of an electronic device in this embodiment, and shows eyeglasses 1600 (smart glasses). The eyeglasses 1600 are equipped with the photoelectric converter 1602 described in each of the embodiments described above. That is, the eyeglasses 1600 is an example of a light detection system to which the photoelectric converter 1602 described in each of the embodiments described above can be applied. A display device including a light-emitting device such as an OLED or LED may be provided on the back side of the lens 1601. There may be one or more photoelectric converters 1602. In addition, multiple types of photoelectric converters may be combined. The arrangement position of the photoelectric converter 1602 is not limited to that shown in Figure 30(a).

[0166] The eyeglasses 1600 further include a control device 1603. The control device 1603 functions as a power source that supplies power to the photoelectric converter 1602 and the aforementioned display device. The control device 1603 also controls the operation of the photoelectric converter 1602 and the display device. The lens 1601 is equipped with an optical system for focusing light onto the photoelectric converter 1602.

[0167] Figure 30(b) shows eyeglasses 1610 (smart glasses) relating to one application example. The eyeglasses 1610 have a control device 1612, which is equipped with a photoelectric converter corresponding to a photoelectric converter 1602 and a display device. The lens 1611 is arranged with the photoelectric converter in the control device 1612 and an optical system for projecting light emitted from the display device, and an image is projected onto the lens 1611. The control device 1612 functions as a power supply that provides power to the photoelectric converter and the display device, and also controls the operation of the photoelectric converter and the display device. The control device 1612 may have a gaze detection unit that detects the wearer's gaze. Gaze detection may use infrared light. The infrared light emitter emits infrared light towards the eyeball of the user who is gazing at the displayed image. An imaging unit having a light-receiving element detects the reflected light from the eyeball of the emitted infrared light, thereby obtaining an image of the eyeball. By having a reduction mechanism that reduces the amount of light transmitted from the infrared light-emitting part to the display part in a planar view, the degradation of image quality is reduced.

[0168] The control device 1612 detects the user's gaze toward the displayed image from the captured image of the eyeball obtained by imaging with infrared light. Any known method can be applied to gaze detection using the captured image of the eyeball. As an example, a gaze detection method based on the Purkinje image obtained by the reflection of the irradiated light from the cornea can be used.

[0169] More specifically, gaze detection processing is performed based on the pupil-corneal reflection method. Using the pupil-corneal reflection method, a gaze vector representing the orientation (rotation angle) of the eyeball is calculated based on the pupil image and Purkinje image contained in the captured image of the eyeball, thereby detecting the user's gaze.

[0170] The display device of this embodiment includes a photoelectric converter having a light-receiving element, and may control the display image of the display device based on the user's gaze information from the photoelectric converter.

[0171] Specifically, the display device determines a first field of view area that the user is fixated on, and a second field of view area other than the first field of view area, based on gaze information. The first and second field of view areas may be determined by the control device of the display device, or by an external control device. Within the display area of ​​the display device, the display resolution of the first field of view area may be controlled to be higher than the display resolution of the second field of view area. In other words, the resolution of the second field of view area may be lower than that of the first field of view area.

[0172] Furthermore, the display area may include a first display area and a second display area different from the first display area. Based on line-of-sight information, a higher-priority area may be determined from the first and second display areas. The first and second field-of-sight areas may be determined by the control device of the display device or by an external control device. The resolution of the higher-priority area may be controlled to be higher than the resolution of the areas other than the higher-priority area. In other words, the resolution of areas with relatively lower priority may be lower.

[0173] Artificial Intelligence (AI) may be used in determining the first field of view area and the high-priority area. The AI ​​may be a model configured to estimate the angle of line of sight and the distance to the target object at the end of the line of sight from the image of the eye, using the image of the eye and the direction the eye was actually looking in the image as training data. The AI ​​program may be installed in the display device, the photoelectric converter, or an external device. If the external device has the AI ​​program, it may be transmitted to the display device from a server or the like via communication.

[0174] When display control is performed based on visual detection, this embodiment can be preferably applied to smart glasses further having a photoelectric converter for capturing images of the outside. The smart glasses can display the captured external information in real time.

[0175] [Modified Embodiment] The present invention is not limited to the embodiments described above and can be modified in various ways. For example, an example in which a part of the configuration of one embodiment is added to another embodiment, or in which a part of the configuration of another embodiment is replaced, is also an embodiment of the present invention.

[0176] The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by a process in which one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0177] The embodiments described above are merely examples of how the present invention can be implemented, and the technical scope of the invention should not be interpreted as being limited by them. In other words, the present invention can be implemented in various ways without departing from its technical concept or its main features. [Explanation of Symbols]

[0178] 20... Photoelectric conversion unit 22... Photon detection element 24... Quench element 30...Pixel signal processing unit 32...Waveform shaping section 34…Digital processing circuits 36…Pixel output circuit 100... Photoelectric converter 342... Counter 344...Threshold determination unit 346… Exposure Control Unit

Claims

1. Avalanche photodiode and A signal processing circuit includes a counter that generates a count value based on photons incident on the avalanche photodiode during the counting period, and repeatedly outputs the count value for each counting period. It has pixels that include, The pixel transitions from a first state to a second state in which the length of the count period is shorter than that of the first state, depending on the result of a determination based on the count value and a predetermined threshold. The power consumption in the first state is smaller than the power consumption in the second state. A photoelectric conversion device characterized by the following features.

2. The period during which the counter is reset in the second state is shorter than the period during which the counter is reset in the first state. The photoelectric conversion device according to feature 1.

3. The interval from the end time of the first count period in the second state to the start time of the second count period following the first count period is shorter than the interval from the end time of the third count period in the first state to the start time of the fourth count period following the third count period. The photoelectric conversion device according to claim 1 or 2.

4. Avalanche photodiode and A signal processing circuit includes a counter that generates a count value based on photons incident on the avalanche photodiode during the counting period, and repeatedly outputs the count value for each counting period. It has pixels that include, The pixel transitions from a first state to a second state according to the result of a determination based on the count value and a predetermined threshold. The interval from the end time of the first count period in the second state to the start time of the second count period following the first count period is shorter than the interval from the end time of the third count period in the first state to the start time of the fourth count period following the third count period. A photoelectric conversion device characterized by the following features.

5. The pixel further includes a quench transistor that returns the avalanche photodiode, after avalanche multiplication has occurred, to a state in which avalanche multiplication can occur again. The duration for which the quench transistor is off in the second state is shorter than the duration for which the quench transistor is off in the first state. The photoelectric conversion device according to feature 4.

6. The aforementioned pixel is A quench transistor that returns the avalanche photodiode, after avalanche multiplication has occurred, to a state in which avalanche multiplication can occur again, A pulse generation unit that outputs a pulse signal whose level changes at a predetermined frequency to the gate of the quench transistor, It further possesses, The frequency in the second state is higher than the frequency in the first state. The photoelectric conversion device according to any one of claims 1 to 4.

7. The length of either the high-level period or the low-level period of the pulse signal is the same between the first state and the second state. The photoelectric conversion device according to feature 6.

8. The pixel transitions from the first state to the second state when the count value exceeds a predetermined threshold. The photoelectric conversion device according to any one of claims 1 to 7.

9. The pixel transitions from the first state to the second state when the time change of the count value exceeds a predetermined threshold. The photoelectric conversion device according to any one of claims 1 to 8.

10. Having a plurality of the aforementioned pixels arranged in a plurality of rows and a plurality of columns The photoelectric conversion device according to any one of claims 1 to 9.

11. In the first state, all of the plurality of pixels perform the determination. The photoelectric conversion device according to feature 10.

12. In the first state, the plurality of pixels sequentially perform the determination in an order corresponding to the arrangement of the plurality of pixels. The photoelectric conversion device according to claim 10 or 11.

13. In the first state, the plurality of pixels sequentially perform the determination in a random order with respect to the array of the plurality of pixels. The photoelectric conversion device according to claim 10 or 11.

14. In the first state, some of the plurality of pixels perform the determination, while other parts of the plurality of pixels do not perform the determination. The photoelectric conversion device according to any one of claims 10 to 13.

15. Depending on the result of the determination in one of the plurality of pixels, all of the plurality of pixels transition to the second state. The photoelectric conversion device according to any one of claims 10 to 14.

16. Depending on the result of the determination in one of the plurality of pixels, some of the plurality of pixels, including at least the one pixel, transition to the second state. The photoelectric conversion device according to any one of claims 10 to 14.

17. Two of the aforementioned pixels are adjacent to each other. The photoelectric conversion device according to feature 16.

18. Depending on the result of the determination in one of the aforementioned pixels, all of the plurality of pixels transition to the second state. The photoelectric conversion device according to claim 16 or 17, characterized in that it is a photoelectric conversion device.

19. Depending on the result of the determination in at least one pixel of a pixel block that is a part of the plurality of pixels and contains a plurality of pixels, all pixels belonging to the pixel block transition to the second state. The photoelectric conversion device according to any one of claims 10 to 14.

20. The conditions after transitioning to the second state are set according to the number of pixels in the aforementioned pixel block whose determination results satisfy the predetermined conditions. The photoelectric conversion device according to feature 19.

21. A photoelectric conversion device according to any one of claims 1 to 20, A signal processing device that processes the signal output from the aforementioned photoelectric converter and A light detection system characterized by having the following features.

22. The signal processing device generates a distance image representing distance information to the object based on the signal. The light detection system according to claim 21, characterized in that it is as described above.

23. It is a mobile object, A photoelectric conversion device according to any one of claims 1 to 20, Distance information acquisition means for acquiring distance information to an object from a parallax image based on a signal output from the aforementioned photoelectric converter, Control means for controlling the moving body based on the distance information A mobile body characterized by having the following features.

Images