Photoelectric conversion device, photoelectric conversion system, and mobile

Abstract

To provide a photoelectric conversion device capable of acquiring count information and event information from the same pixel.SOLUTION: A photoelectric conversion device comprises a plurality of pixels. Each of the plurality of pixels includes: an avalanche photodiode; a photon counting circuit for counting the number of photons incident to the avalanche photodiode within an exposure period; a memory connected with the photon counting circuit; and a comparator circuit connected with the photon counting circuit and the memory.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a photoelectric conversion device, a photoelectric conversion system, and a moving object. [Background technology]

[0002] There is a known photoelectric conversion device that has pixels each including a plurality of avalanche photodiodes (APDs). This device makes it possible for each pixel to detect a single photon by utilizing the avalanche multiplication that occurs when a charge is generated when a photon is incident on the APD.

[0003] Patent document 1 describes an imaging element having a detection pixel that detects the presence or absence of an address event and a counting pixel that counts the number of photons incident within a specified exposure period when the address event occurs, and a control method for the imaging element. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2020-96347 A Summary of the Invention [Problem to be solved by the invention]

[0005] In Patent Document 1, the address event detection pixel and the counting pixel are each arranged at a specific address and are arranged separately based on their functions, so it is not possible to obtain both counting information and event information using a signal from the same pixel.

[0006] Therefore, an object of the present invention is to provide a photoelectric conversion device capable of acquiring count information and event information from the same pixel. [Means for solving the problem]

[0007] One aspect of the present invention includes a pixel array having a plurality of pixels, each of which has an avalanche photodiode, a photon counting circuit that counts the number of photons incident on the avalanche photodiode within an exposure period, a memory connected to the photon counting circuit, and a comparison circuit connected to the photon counting circuit and the memory. Effect of the Invention

[0008] According to the present invention, it is possible to provide a photoelectric conversion device capable of acquiring count information and event information from the same pixel. [Brief description of the drawings]

[0009] [Figure 1] 1 is a diagram showing a configuration of a photoelectric conversion device according to a first embodiment. [Diagram 2] FIG. 2 is a diagram showing the arrangement of pixel substrates in the photoelectric conversion device according to the first embodiment. [Diagram 3] 2 is a diagram showing a configuration of a circuit board of the photoelectric conversion device according to the first embodiment. FIG. [Figure 4] 1 is a block diagram of one pixel of a pixel substrate and a circuit substrate according to a first embodiment. [Diagram 5] FIG. 2 is a schematic diagram showing driving of a pixel circuit of the photoelectric conversion device according to the first embodiment. [Figure 6] 4 is a flowchart showing an example of a photoelectric conversion device according to the first embodiment. [Figure 7] 13 is a configuration example of a pixel circuit of a photoelectric conversion device according to a second embodiment. [Figure 8] 10 is a flowchart showing an example of a photoelectric conversion device according to a second embodiment. [Figure 9] FIG. 11 is a functional block diagram of a photoelectric conversion system according to a third embodiment. [Figure 10] FIG. 13 is a functional block diagram of a photoelectric conversion system according to a fourth embodiment. [Figure 11] FIG. 13 is a functional block diagram of a photoelectric conversion system according to a fifth embodiment. [Figure 12] FIG. 13 is a functional block diagram of a photoelectric conversion system according to a sixth embodiment. [Figure 13] FIG. 13 is a functional block diagram of a photoelectric conversion system according to a seventh embodiment. [Figure 14] FIG. 13 is a functional block diagram of a photoelectric conversion system according to an eighth embodiment. [Figure 15] FIG. 13 is a functional block diagram of a photoelectric conversion system according to a ninth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] The following embodiments are intended to embody the technical ideas of the present invention, and are not intended to limit the present invention. The sizes and positional relationships of the components shown in the drawings may be exaggerated to clarify the description. In the following description, the same components may be designated by the same reference numerals, and the description may be omitted.

[0011] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. In the following description, terms indicating specific directions or positions (for example, "upper", "lower", "right", "left" and other terms including these terms) will be used as necessary. The use of these terms is for the purpose of facilitating understanding of the embodiment with reference to the drawings, and the technical scope of the present invention is not limited by the meaning of these terms.

[0012] In this specification, the plan view means a view from a direction perpendicular to the light incident surface of the semiconductor layer. Also, the cross-sectional view means a view of a cross section perpendicular to the light incident surface of the semiconductor layer. Note that, when the light incident surface of the semiconductor layer is a rough surface when viewed microscopically, the plan view is defined based on the light incident surface of the semiconductor layer when viewed macroscopically.

[0013] The semiconductor layer has a first surface and a second surface, which is the surface opposite to the first surface and through which light is incident. In this specification, the depth direction is the direction from the first surface to the second surface of the semiconductor layer on which the avalanche photodiode (hereinafter, APD) is disposed. Hereinafter, the "first surface" may be referred to as the "front surface" and the "second surface" may be referred to as the "rear surface."

[0014] In the following description, the anode of the APD is set to a fixed potential, and a signal is taken out from the cathode side. Therefore, the first conductive type semiconductor region having charges of the same polarity as the signal charge as the majority carrier is an N-type semiconductor region, and the second conductive type semiconductor region having charges of a different polarity than the signal charge as the majority carrier is a P-type semiconductor region. The present invention is also valid when the cathode of the APD is set to a fixed potential, and a signal is taken out from the anode side. In this case, the first conductive type semiconductor region having charges of the same polarity as the signal charge as the majority carrier is a P-type semiconductor region, and the second conductive type semiconductor region having charges of a different polarity than the signal charge as the majority carrier is an N-type semiconductor region. In the following, a case where one node of the APD is set to a fixed potential will be described, but the potentials of both nodes may fluctuate.

[0015] In the following embodiments, the connection between elements of a circuit may be described. In this case, even if another element is interposed between the elements of interest, the elements of interest are treated as being connected to each other unless otherwise specified. For example, assume that element A is connected to one node of a capacitive element C having multiple nodes, and element B is connected to the other node. Even in such a case, element A and element B are treated as being connected to each other unless otherwise specified.

[0016] Metallic members such as wiring and pads described in this specification may be composed of a single metal element or may be a mixture (alloy). For example, wiring described as copper wiring may be composed of a single copper element or may be composed mainly of copper and further contain other components. Also, for example, pads connected to external terminals may be composed of a single aluminum element or may be composed mainly of aluminum and further contain other components. The copper wiring and aluminum pads shown here are examples and can be changed to various metals.

[0017] Moreover, the wiring and pads shown here are one example of metal members used in a photoelectric conversion device, and the present invention can also be applied to other metal members.

[0018] In the following embodiments, an image pickup device will be mainly described as an example of a photoelectric conversion device. However, each embodiment is not limited to an image pickup device, and can be applied to other examples of photoelectric conversion devices. For example, a distance measuring device (a device for measuring distance using focus detection or TOF (Time Of Flight)) or a photometric device (a device for measuring the amount of incident light) may be used.

[0019] (First embodiment) The photoelectric conversion device of the first embodiment and its operation flow will be described with reference to FIGS. 1 to 6. FIG.

[0020] FIG. 1 is a diagram showing the configuration of a photoelectric conversion device 100 in this embodiment. In the following, a case where the photoelectric conversion device 100 is a stacked type photoelectric conversion device will be described as an example. That is, a photoelectric conversion device configured by stacking and electrically connecting two substrates, a pixel substrate 11 and a circuit substrate 21, will be described as an example. However, the photoelectric conversion device is not limited to this. For example, it may be a photoelectric conversion device in which the configuration included in the pixel substrate 11 and the configuration included in the circuit substrate are arranged on a common semiconductor layer, as described below. In the following, a photoelectric conversion device in which the configuration included in the pixel substrate 11 and the configuration included in the circuit substrate are arranged on a common semiconductor layer will also be referred to as a non-stacked photoelectric conversion device.

[0021] The pixel substrate 11 has a first semiconductor layer having a photoelectric conversion section 102 described later, and a first wiring structure. The circuit board 21 has a second semiconductor layer having circuits such as a signal processing circuit 103 described later, and a second wiring structure. The photoelectric conversion device 100 is configured by laminating the second semiconductor layer, the second wiring structure, the first wiring structure, and the first semiconductor layer in this order.

[0022] FIG. 1 illustrates a back-illuminated stacked photoelectric conversion device in which light is incident from the second surface of the first semiconductor layer of the pixel substrate 11, and a circuit board 21 is arranged on the first surface side of the first semiconductor layer, which is the surface opposite to the second surface.

[0023] In the case of a non-stacked photoelectric conversion device, the surface of the semiconductor layer on which the transistors of the signal processing circuit are arranged is called the first surface. In the case of a back-illuminated photoelectric conversion device, the second surface of the semiconductor layer opposite to the first surface is the light incident surface. In the case of a front-illuminated photoelectric conversion device, the first surface of the semiconductor layer is the light incident surface.

[0024] In the following, the pixel substrate 11 and the circuit substrate 21 will be described as diced chips, but are not limited to chips. For example, each substrate may be a wafer. Also, each substrate may be stacked in a wafer state and then diced, or each chip may be stacked and bonded after being made into chips. Also, in this specification, a semiconductor layer in which a semiconductor layer that has been thinned from the substrate state and a wiring structure are stacked is also called a substrate.

[0025] A pixel region 12 is disposed on the pixel substrate 11, and a circuit region 22 that processes signals detected in the pixel region 12 is disposed on the circuit substrate 21.

[0026] 2 is a layout diagram of the pixel substrate 11. A plurality of pixels 101, each having a photoelectric conversion unit 102 including an APD, are arranged two-dimensionally to form a pixel region 12. In other words, some of the APDs are arranged in a predetermined direction (e.g., row direction). Also, some of the APDs are arranged in a direction perpendicular to the predetermined direction (e.g., column direction). In this manner, it is preferable to arrange the pixels 101 in a two-dimensional array.

[0027] The pixel 101 is typically a pixel for forming an image, but when used for TOF (Time of Flight), it does not necessarily have to form an image. That is, the pixel 101 may be a pixel for measuring the time when light arrives and the amount of light.

[0028] 3 is a configuration diagram of the circuit board 21. It has a signal processing unit 103 that processes the charges photoelectrically converted by the photoelectric conversion unit 102 in FIG. 2, a readout circuit 112, a control pulse generation unit 115, a horizontal scanning circuit unit 111, a signal line 113, and a vertical scanning circuit unit 110.

[0029] The photoelectric conversion unit 102 in FIG. 2 and the signal processing unit 103 in FIG. 3 are electrically connected via connection wiring provided for each pixel.

[0030] The vertical scanning circuit section 110 receives a control pulse supplied from a control pulse generating section 115 and supplies the control pulse to each pixel. The vertical scanning circuit section 110 uses logic circuits such as a shift register and an address decoder.

[0031] The control pulse generating unit 115 has a signal generating unit that generates a control signal for the switch 202, which will be described later. The signal generating unit generates a pulse signal that controls the switch, as will be described later. The signal generating unit may generate a control signal in common for all of the pixels in the pixel region, or may generate a control signal in common for a portion of the pixels in the pixel region. When generating a common pulse signal, at least one of the period, the number of pulses, and the pulse width of the signal P_EXP pulse signal that controls the exposure period is generated in common in correspondence with the exposure period. When generating a common control signal for a portion of the pixels, a signal can be generated using both the input signal VR_IN output from the control pulse generating unit 115 and the signal P_EXP that controls the exposure period. The control pulse generating unit 115 preferably has, for example, a frequency dividing circuit. This allows simple control and reduces an increase in the number of elements. A pulse signal is supplied from the control pulse generating unit 115.

[0032] The signal output from the photoelectric conversion unit 102 of each pixel is processed by the signal processing unit 103. The signal processing unit 103 is provided with a photon counting circuit (hereinafter also referred to as a counter) that counts the number of photons incident within a predetermined exposure period, a memory, etc. Digital values ​​are stored in the memory.

[0033] The horizontal scanning circuit unit 111 inputs a control pulse for sequentially selecting each column to the signal processing unit 103 in order to read out the signal from the memory of each pixel in which the digital signal is held.

[0034] For a selected column, a signal is output from the signal processing unit 103 of the pixel selected by the vertical scanning circuit unit 110 to the signal line 113. The signal line 113 is connected to each of the pixels aligned in the column direction, and serves as an output line common to these pixels.

[0035] The signal output to the signal line 113 is output via an output circuit 114 to a recording unit or a signal processing unit outside the photoelectric conversion device 100 .

[0036] 2, a plurality of pixels may be arranged one-dimensionally in the pixel region 12. The function of the signal processing unit 103 does not necessarily need to be provided for each pixel 101, and for example, one signal processing unit 103 may be shared by the photoelectric conversion units 102 of a plurality of pixels 101, and signal processing may be performed sequentially on the outputs from the photoelectric conversion units of the pixels 101.

[0037] Fig. 4 is an example of a block diagram including an equivalent circuit of the pixel 101 in Fig. 2 and Fig. 3. In Fig. 2, a photoelectric conversion unit 102 having an APD 201 is provided on a pixel substrate 11, and other members are provided on a circuit substrate 21. According to the pixel configuration shown in Fig. 4, it is possible to obtain both luminance information (counting information) and event information from one pixel by photon counting operation.

[0038] The APD 201 generates a pair of electric charges according to the incident light by photoelectric conversion. The APD 201 has a first terminal and a second terminal. The first terminal is, for example, a cathode, and the second terminal is, for example, an anode. A voltage VL (first voltage) is supplied to the anode of the APD 201. A voltage VH (second voltage) higher than the voltage VL supplied to the anode is supplied to the cathode of the APD 201. A reverse bias voltage that causes the APD 201 to perform avalanche multiplication is supplied to the anode and cathode. By supplying such a voltage, the electric charges generated by the incident light undergo avalanche multiplication, generating an avalanche current.

[0039] When a reverse bias voltage is supplied, there are two modes: Geiger mode, in which the potential difference between the anode and cathode is greater than the breakdown voltage, and linear mode, in which the potential difference between the anode and cathode is close to or less than the breakdown voltage. An APD that operates in Geiger mode is called a SPAD. For example, the voltage VL (first voltage) is −30 V, and the voltage VH (second voltage) is 1 V. The APD201 may operate in either the linear mode or the Geiger mode.

[0040] The switch 202 is connected to a control line to which a drive voltage VH is supplied and to the APD 201. The switch 202 is connected to one of the anode and cathode nodes of the APD. The switch 202 switches the potential difference between the anode and cathode of the APD between a first potential difference that causes avalanche multiplication and a second potential difference that does not cause avalanche multiplication. Hereinafter, switching from the second potential difference to the first potential difference is also referred to as turning on the switch 202, and switching from the first potential difference to the second potential difference is also referred to as turning off the switch 202. The switch 202 functions as a quenching element. The switch 202 functions as a load circuit (quenching circuit) during signal multiplication by avalanche multiplication, and has a function of suppressing avalanche multiplication by suppressing the voltage supplied to the APD 201 (quenching operation). The switch 202 also has a function of returning the voltage supplied to the APD 201 to the drive voltage VH by passing a current equivalent to the voltage drop caused by the quench operation (recharge operation). In other words, the switch 202 functions as a control circuit for controlling the occurrence of avalanche multiplication in the APD 201.

[0041] The switch 202 can be configured, for example, by a MOS transistor. A control signal is supplied from a signal generating unit to a gate electrode of the switch 202. In this embodiment, the on / off state of the switch 202 is controlled by controlling a voltage applied to the gate electrode of the switch 202.

[0042] The signal processing unit 103 has a waveform shaping unit 210, a photon counting circuit (counter circuit) 211, a memory 213, a comparison circuit 214, a processing circuit 215, and a switch 212. In this embodiment, the signal processing unit 103 is required to have at least the photon counting circuit 211, the memory 213, and the comparison circuit 214.

[0043] The waveform shaping unit 210 shapes the potential change of the cathode of the APD 201 obtained when a photon is detected, and outputs a pulse signal. The node on the input side of the waveform shaping unit 210 is nodeA, and the node on the output side is nodeB. The waveform shaping unit 210 changes the output potential from the node nodeB depending on whether the input potential to the node nodeA is equal to or higher than a predetermined value or lower. For example, in FIG. 5, when the input potential to the node nodeA becomes a high potential equal to or higher than the judgment threshold, the output potential from the node nodeB becomes a low level. When the input potential to the node nodeA becomes a potential lower than the judgment threshold, the output potential from the node nodeB becomes a high level. For example, an inverter circuit is used as the waveform shaping unit 210. In FIG. 4, an example in which one inverter is used as the waveform shaping unit 210 is shown, but a circuit in which multiple inverters are connected in series may be used, or other circuits having a waveform shaping effect may be used.

[0044] FIG. 5 is a diagram showing a schematic relationship between the potential of node nodeA, the potential of node nodeB, and the output signal. In this embodiment, when the control signal is at a high level, the driving voltage VH is not easily supplied to the APD, and when the control signal is at a low level, the driving voltage VH is supplied to the APD. The high level of the control signal is, for example, 1V, and the low level of the control signal VR is, for example, 0V. When the control signal is at a high level, the switch is turned off, and when the control signal is at a low level, the switch is turned on. The resistance value of the switch when the control signal is at a high level is higher than the resistance value of the switch when the control signal is at a low level. When the control signal is at a high level, even if avalanche multiplication occurs in the APD, a recharge operation is not easily performed, so the potential supplied to the APD is a potential equal to or lower than the breakdown voltage of the APD. Therefore, the avalanche multiplication operation in the APD stops.

[0045] As shown in Fig. 5(a), it is preferable to configure the switch 202 with one transistor, and to perform the quenching operation and the recharge operation with one transistor. This makes it possible to reduce the number of circuits compared to the case where the quenching operation and the recharge operation are performed with different circuit elements. In particular, when each pixel has a photon counting circuit and the SPAD signal is read out for each pixel, it is preferable to reduce the circuit area used for the switch in order to place the photon counting circuit. Therefore, the effect of configuring the switch 202 with one transistor becomes significant.

[0046] FIG. 5(b) is a graph showing the time change of the potential of the node nodeA in FIG. 5(a). FIG. 5(c) is a graph showing the time change of the potential of the node nodeB in FIG. 5(a). In the period from time t0 to time t1, a voltage of VH-VL is applied to the APD 201 in FIG. 5(a). When a photon is incident on the APD 201 at time t1, avalanche multiplication occurs in the APD 201. As a result, an avalanche current flows through the switch 202, and the potential of the node nodeA drops. After that, the amount of potential drop becomes larger, and the voltage applied to the APD 201 gradually decreases. Then, at time t2, the avalanche multiplication in the APD 201 stops. As a result, the potential of the node nodeA does not drop below a certain value. Thereafter, during the period from time t2 to time t3, a current flows to node A from the node at potential VH to compensate for the potential drop, and at time t3, node A settles to its original potential.

[0047] In the above process, the potential of node nodeB becomes high level during the period when the potential of node A is lower than a certain threshold (decision threshold). In this way, the waveform of the drop in the potential of node A caused by the incidence of photons is shaped by the waveform shaping unit 210 and output to node node B as a pulse. Then, the photon counting circuit counts the photon, and the count value is incremented by 1 LSB. This pulse signal is input to the readout unit via the photon counting circuit and the signal line.

[0048] It is possible to perform a quench operation and a recharge operation using the switch 202 according to the avalanche multiplication in the APD 201, but depending on the timing of photon detection, the charge generated in the APD may not be determined as an output signal. For example, assume that avalanche multiplication occurs in the APD, the node nodeA becomes low level, and a recharge operation is performed. In general, the decision threshold of the waveform shaping unit 210 is set to a potential higher than the potential difference at which avalanche multiplication occurs in the APD. When a photon is incident on the APD when the potential of the node nodeA is lower than the decision threshold due to the recharge operation and the potential is such that avalanche multiplication is possible in the APD, avalanche multiplication occurs in the APD and the voltage of the node nodeA drops. In other words, since the potential of nodeA drops at a voltage lower than the decision threshold, no potential change occurs across the decision threshold, and the output potential from the node nodeB does not change. Therefore, even though avalanche multiplication occurs, the photon detection is not determined as a signal. Particularly under high illuminance, photons enter the APD continuously in a short period of time, making it difficult for the incident light to be detected as a signal. This can lead to a discrepancy between the actual number of incident photons and the output signal, even under high illuminance.

[0049] On the other hand, by applying a control signal to the switch 202 to switch the switch 202 between the on and off states, it is possible to determine that photons enter the APD continuously in a short period of time as a signal. Therefore, it is preferable that the control signal is a pulse signal with a repetitive cycle. In other words, in FIG. 5, it is preferable that the switch 202 is switched on and off at a predetermined clock frequency.

[0050] The effect of acquiring event information and luminance information in this embodiment can be obtained even if the pulse signal is not a signal with a repetitive cycle. Therefore, the pulse signal does not have to be a signal with a repetitive cycle. A switch other than the switch 202 may be disposed between the switch 202 and the cathode terminal of the APD 201, and the potential difference of the APD 201 may be switched using the switch other than the switch 202. In this case, it is preferable to supply a control signal from the signal generating unit to a switch other than the switch 202. The switch 202 may be configured of a resistor instead of a transistor, and the APD 201 and the potential VH may be connected.

[0051] The photon counting circuit 211 shown in Fig. 4 counts the number of photons incident on the APD 201 within a predetermined exposure period. In Fig. 4, the photon counting circuit 211 counts the pulse signal output from the waveform shaping unit 210 and holds the count value. In addition, when a control pulse is supplied via a drive line, the signal held in the photon counting circuit 211 can be reset.

[0052] The memory 213 is connected to the photon counting circuit 211 and holds the count value output from the photon counting circuit 211. The memory 213 holds a digital value. A digital memory having two or more bits can be used as the memory 213. It is preferable to use a digital memory having several bits to several tens of bits as the memory 213. The number of bits of the memory 213 and the number of bits of the photon counting circuit 211 may be the same, or the number of bits of the memory 213 may be larger than the number of bits of the photon counting circuit 211.

[0053] A switch (first switch) 212 that switches between electrical continuity and non-conduction between the photon counting circuit 211 and the memory 213 is connected between the photon counting circuit 211 and the memory 213. The switch 212 can be configured of, for example, a MOS transistor. The switch 212 can switch between connection and non-connection in response to a detection signal output from a processing circuit 215 described later.

[0054] The comparison circuit 214 is connected to the photon counting circuit 211 and the memory 213. Specifically, the comparison circuit 214 is connected so as to receive the first signal output from the photon counting circuit 211 and the second signal output from the memory 213. The comparison circuit 214 compares the first signal with the second signal and outputs the comparison result. The first signal is a digital signal held in the photon counting circuit 211, and the second signal is a digital signal held in the memory 213.

[0055] The first signal is, for example, a signal counted in a first period. The second signal is, for example, a signal counted in a second period that does not overlap with the first period. The second period is, for example, an n-frame period, and the first period is an n+m (m is an integer equal to or greater than 1) period that is different from the n-frame period. One frame can be defined as a cycle for outputting luminance information. From another perspective, one frame can also be defined as the timing at which the vertical synchronization signal output from the vertical scanning circuit transitions from a High level to a Low level indicating the end timing of the previous frame period and the start timing of the next frame period.

[0056] The processing circuit 215 is connected to the comparison circuit 214. The processing circuit 215 outputs a detection signal according to the comparison result output from the comparison circuit 214 as an event signal. The detection signal is, for example, a binary signal. Alternatively, a signal with three or more values ​​may be output as the detection signal.

[0057] For example, if the difference between the signal held in the photon counting circuit 211 and the signal held in the memory 213 exceeds a predetermined threshold, a signal is output, determining that there is event information.

[0058] First, a case where the comparison circuit 214 and the processing circuit 215 have negative bits will be described. In this case, the predetermined threshold is an absolute value, and when the predetermined threshold is 5, the predetermined threshold is ±5. In this case, when the difference between the first signal, which is the signal held in the photon counting circuit 211, and the second signal, which is the signal held in the memory 213, is within the range of -5 to 5, the processing circuit 215 processes that there is no event information. Also, when the difference between the first signal and the second signal exceeds -5, i.e., is less than -5, or exceeds 5, the processing circuit 215 processes that there is event information.

[0059] For example, when the first signal, which is a signal held in the photon counting circuit 211, is subtracted from the second signal, which is a signal held in the memory 213, and the obtained value exceeds 5, the output of the n+mth frame is smaller than that of the nth frame. In this case, the processing circuit 215 can detect, as event information, that the target object has disappeared between the first period and the second period, and output a positive signal. Also, when the first signal is subtracted from the second signal, and the obtained value exceeds -5, that is, is less than -5, the output of the n+mth frame is larger than that of the nth frame. In this case, the processing circuit 215 can detect, as event information, that the target object has been detected between the first period and the second period, and output a negative signal. Also, when the first signal is subtracted from the second signal, and the obtained value is within the range of -5 to 5, no significant change has occurred between the nth frame and the n+mth frame, and the processing circuit 215 can detect that there has been no event information.

[0060] Next, a comparison process different from the above comparison process will be described. In the above comparison process, a negative bit is required in the comparison circuit 214 and the processing circuit 215. This is because, without a negative bit, the value cannot be reduced below 0 even when a subtraction process is performed, and an accurate value cannot be obtained. According to the comparison process described below, it is possible to detect the presence or absence of event information without providing a negative bit in the comparison circuit 214 and the processing circuit 215. Also, in the comparison process described below, the predetermined threshold does not need to be an absolute value, and can be a predetermined positive number. For example, the predetermined threshold is set to 5.

[0061] When the value obtained by subtracting the first signal from the second signal exceeds the threshold, it corresponds to the output being smaller in the (n+m)th frame than in the nth frame. In this case, the processing circuit 215 outputs negative event information. Here, negative event information is, for example, information that an object with an output larger than the background has disappeared, or an object with an output smaller than the background has appeared, etc.

[0062] Furthermore, when the value obtained by subtracting the second signal from the first signal exceeds a predetermined threshold, this corresponds to the output being greater in the (n+m)th frame than in the nth frame, so the processing circuit 215 outputs positive event information. Here, positive event information is, for example, information such as the occurrence of an object with a greater output than the background, or the disappearance of an object with a smaller output than the background. In this way, it is also possible to output positive event information and negative event information by making the comparison result output from the comparison circuit 214 a positive number and recognizing whether an object has occurred or disappeared through a different route.

[0063] If the threshold is not exceeded, this corresponds to no event occurring, and so no event information is output.

[0064] In this way, by subtracting the first signal from the second signal and the second signal from the first signal, the obtained values ​​are all positive, making it possible to detect the presence or absence of event information without setting negative bits.

[0065] If the predetermined threshold is not exceeded, the processing circuit 215 may output to the readout circuit a message indicating that no event has occurred.

[0066] If the predetermined threshold is not exceeded, this corresponds to a case where event information has not been detected. In this case, it is also possible to control so that writing from the photon counting circuit 211 to the memory 213 is not performed. In this embodiment, when the processing circuit 215 detects that an event has not occurred, the switch 212 disconnects the photon counting circuit 211 from the memory 213. This is because the absence of an event corresponds to the absence of a change in luminance, and the signals of the photon counting circuit 211 and the memory 213 have approximately the same value. By controlling writing in this way, it is possible to reduce the power consumption caused by driving the circuit.

[0067] In the above explanation, a distinction is made between positive and negative events, but a method may be adopted in which only the fact that there is event information is output without distinguishing between positive and negative events.

[0068] In this way, according to the circuit configuration of this embodiment, it is possible to output event information and luminance information from one pixel circuit. In other words, it is possible to output the signal value detected by the APD 201 used to determine the presence or absence of event information as luminance information.

[0069] If the output from the comparison circuit 214 does not exceed a predetermined threshold, the memory 213 may be configured not to output a signal to the outside, thereby making it possible to further reduce power consumption.

[0070] The switch 212 may be controlled using a signal from another path, not only using the signal from the processing circuit 215. For example, after the photon counting circuit 211 and the memory 213 are disconnected by the signal from the processing circuit 215, they may be connected based on a signal from the control pulse generating unit 115 when detecting whether or not the next event has occurred.

[0071] The electrical connection may be switched by disposing a switch such as a transistor between the switch 202 and the APD 201 or between the photoelectric conversion unit 102 and the signal processing unit 103. Similarly, the supply of the voltage VH or the voltage VL to the photoelectric conversion unit 102 may be electrically switched using a switch such as a transistor.

[0072] The operation flow of the photoelectric conversion device of this embodiment will be described with reference to Fig. 6. Fig. 6 shows the operation flow for two frames.

[0073] First, in S101, the exposure period of the first frame starts.

[0074] In S102, the exposure period of the first frame ends, and in S103, the signal held in the photon counting circuit 211 is written to the memory 213.

[0075] In S104, the signal (luminance information) held in the memory 213 is output to the outside. Note that S104 is an optional step, and this process may be omitted.

[0076] In S105, the signal held in the photon counting circuit 211 is reset.

[0077] In S106, the exposure period for the second frame begins.

[0078] In S107, the exposure period of the second frame ends, and in S108, the first signal written in memory 213 in S103 is compared with the second signal held in photon counting circuit 211. If the comparison result in S108 exceeds a predetermined threshold (determination threshold), event information is output from processing circuit 215 in S109, and the process proceeds to S110. Also, if the comparison result in S108 is equal to or less than the determination threshold, the process proceeds to S110.

[0079] In S110, the signal held by the photon counting circuit 211 acquired in the second frame is written to the memory 213.

[0080] In S111, the signal (luminance information) held in the memory 213 is output to the outside. Note that, if the result of the comparison in the comparison circuit 214 is equal to or less than the determination threshold, S111 does not need to be executed.

[0081] In this way, according to this embodiment, it is possible to obtain count information and event information from the same pixel.

[0082] In addition, the photoelectric conversion device may have a function of switching between multiple modes. For example, in a mode in which it is necessary to constantly check the event information, the device may operate according to the above operation flow, and in a mode in which it is not necessary to check the event information and only luminance information is required, the luminance information may continue to be output regardless of the event information. In a mode in which it is not necessary to constantly check the luminance information, the event information may be acquired outside the pixel, and the luminance information may be obtained only when the event information is detected. For example, a function for detecting event information may be provided between the pixel region and the outer edge of the semiconductor layer, and the event information may be detected.

[0083] Second embodiment A photoelectric conversion device according to the second embodiment and its operation flow will be described with reference to Fig. 7 and Fig. 8. This embodiment differs from the first embodiment in that the pixel includes a memory (second memory) 216 connected to a memory 213. Also, Fig. 7 shows a case where the switch 212 in Fig. 4 is not provided, but the switch 212 may be provided. Other than these points and the points described below, the present embodiment is substantially the same as the first embodiment, and therefore the description may be omitted.

[0084] The memory 216 is connected to the memory 213 so as to receive signals from the memory 213. The memory 216 is preferably a multi-bit digital memory.

[0085] The number of bits of the memory 216 is preferably larger than the number of bits of the photon counting circuit 211. Also, the number of bits of the memory 216 is preferably larger than the number of bits of the memory 213. The number of bits of the memory 216 is preferably, for example, three times or more the number of bits of the memory 213. By making the number of bits of the memory 216 larger than the number of bits of the memory 213, it becomes possible to add signals output from the memory 213 in the memory 216 and output the result. For example, the photon counting circuit 211 is 3 bits, the memory 213 is 3 bits, and the memory 216 is 14 bits. Also, the photon counting circuit 211 and the memory 213 can each be 1 bit.

[0086] In this embodiment, one frame is divided into a plurality of subframes, and for each subframe, a comparison process is performed between the signal in the memory 213 and the signal in the photon counting circuit 211. For example, a maximum of three bits can be counted in one subframe, which are sequentially output to the memory 216 and added, and the added signal can be output from the memory 216.

[0087] Since the presence or absence of event information can be expressed with one bit, a large number of bits is not required to detect event information. On the other hand, a certain number of bits is required to detect luminance information. According to this embodiment, the number of bits of the circuit that does not have a problem even if the number of bits is small is reduced, so that it is possible to prevent the circuit scale of the pixel circuit from becoming excessively large.

[0088] The operation flow of the photoelectric conversion device of this embodiment will be described with reference to Fig. 8. Fig. 8 shows the operation flow for one frame.

[0089] First, in S201, the exposure period of the first sub-frame begins.

[0090] In S202, the exposure period of the first subframe ends, and in S203, the signal held in the photon counting circuit 211 is written to the memory 213.

[0091] In S204, the signal held in the memory 213 is written to the memory 216.

[0092] Next, event detection and addition processing of subframe signals are started. In S205, the signals held in the photon counting circuit 211 are reset.

[0093] In S206, the exposure period of the second sub-frame begins.

[0094] In S207, the exposure period of the second frame ends, and in S208, the first signal written in the memory 213 in S203 is compared with the second signal held in the photon counting circuit 211. If the comparison result in S208 exceeds a predetermined threshold (determination threshold), event information is output from the processing circuit 215 in S209, and the process proceeds to S210. Also, if the comparison result in S208 is equal to or less than the determination threshold, the process proceeds to S210.

[0095] In S210, the signal held by the photon counting circuit 211 acquired in the second frame is written to the memory 213.

[0096] In S211, the signal held in memory 213 is written to memory 216, and signal addition processing is performed in memory 216. Then, steps S205 to S211 are repeated i times (i is an integer of 2 or more), and the event detection and subframe addition processing is completed.

[0097] In S212, the signal (luminance information) held in the memory 216 is output.

[0098] In this manner, according to this embodiment as well, it is possible to obtain count information and event information from the same pixel.

[0099] (Third embodiment) The photoelectric conversion system according to this embodiment will be described with reference to Fig. 9. Fig. 9 is a block diagram showing a schematic configuration of the photoelectric conversion system according to this embodiment.

[0100] The photoelectric conversion devices described in the first and second embodiments are applicable to various photoelectric conversion systems. Examples of the applicable photoelectric conversion systems include digital still cameras, digital camcorders, security cameras, copiers, fax machines, mobile phones, car-mounted cameras, and observation satellites. Camera modules equipped with an optical system such as a lens and an imaging device are also included in the photoelectric conversion systems. FIG. 9 illustrates a block diagram of a digital still camera as an example of these.

[0101] 9 includes an image capturing device 1004, a lens 1002 that forms an optical image of a subject on the image capturing device 1004, an aperture 1003 that varies the amount of light passing through the lens 1002, and a barrier 1001 that protects the lens 1002. The lens 1002 and the aperture 1003 form an optical system that focuses light on the image capturing device 1004. The image capturing device 1004 is a photoelectric conversion device according to any one of the above embodiments, and converts the optical image formed by the lens 1002 into an electrical signal.

[0102] The photoelectric conversion system also has a signal processing unit 1007 which is an image generating unit that generates an image by processing an output signal output from the imaging device 1004. The signal processing unit 1007 performs various corrections and compression as necessary to output image data. The signal processing unit 1007 may be formed on a semiconductor substrate on which the imaging device 1004 is provided, or may be formed on a semiconductor substrate separate from the imaging device 1004. Moreover, the imaging device 1004 and the signal processing unit 1007 may be formed on the same semiconductor substrate.

[0103] The photoelectric conversion system further includes a memory unit 1010 for temporarily storing image data, and an external interface unit (external I / F unit) 1013 for communicating with an external computer or the like. The photoelectric conversion system further includes a recording medium 1012 such as a semiconductor memory for recording or reading out imaging data, and a recording medium control interface unit (recording medium control I / F unit) 1011 for recording or reading out data on the recording medium 1012. The recording medium 1012 may be built into the photoelectric conversion system, or may be removable.

[0104] The photoelectric conversion system further includes an overall control / calculation unit 1009 that performs various calculations and controls the entire digital still camera, and a timing generation unit 1008 that outputs various timing signals to the image capture device 1004 and the signal processing unit 1007. Here, the timing signals and the like may be input from outside, and the photoelectric conversion system only needs to include at least the image capture device 1004 and the signal processing unit 1007 that processes the output signal output from the image capture device 1004.

[0105] The imaging device 1004 outputs an imaging signal to a signal processing unit 1007. The signal processing unit 1007 performs predetermined signal processing on the imaging signal output from the imaging device 1004, and outputs image data. The signal processing unit 1007 generates an image using the imaging signal.

[0106] In this way, according to this embodiment, it is possible to realize a photoelectric conversion system to which the photoelectric conversion device (imaging device) according to any one of the above embodiments is applied.

[0107] (Fourth embodiment) The photoelectric conversion system and the moving object of this embodiment will be described with reference to Fig. 10. Fig. 10 is a diagram showing the configuration of the photoelectric conversion system and the moving object of this embodiment.

[0108] FIG. 10(a) shows an example of a photoelectric conversion system related to an in-vehicle camera. The photoelectric conversion system 1300 has an image capture device 1310. The image capture device 1310 is a photoelectric conversion device (image capture device) described in any of the above embodiments. The photoelectric conversion system 1300 has an image processing unit 1312 that performs image processing on a plurality of image data acquired by the image capture device 1310. The photoelectric conversion system 1300 also has a distance acquisition unit 1316 that calculates a distance to an object, and a collision determination unit 1318 that determines whether or not there is a possibility of a collision based on the calculated distance. Here, the distance acquisition unit 1316 may acquire distance information to the object using ToF (Time Of Flight) information, or may acquire distance information using parallax information or the like. That is, the distance information is information related to ToF, parallax, defocus amount, distance to the object, and the like. The collision determination unit 1318 may determine the possibility of a collision using any of these distance information. The distance information acquisition means may be realized by dedicated hardware, a software module, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or a combination of these.

[0109] The photoelectric conversion system 1300 is connected to a vehicle information acquisition device 1320, and can acquire vehicle information such as vehicle speed, yaw rate, and steering angle. The photoelectric conversion system 1300 is also connected to an ECU 1330, which is a control device that outputs a control signal to generate a braking force for the vehicle based on the judgment result of the collision judgment unit 1318. The photoelectric conversion system 1300 is also connected to an alarm device 1340 that issues an alarm to the driver based on the judgment result of the collision judgment unit 1318. For example, when the judgment result of the collision judgment unit 1318 indicates that a collision is highly likely, the ECU 1330 performs vehicle control to avoid a collision and reduce damage by applying the brakes, releasing the accelerator, suppressing engine output, etc. The alarm device 1340 warns the user by sounding an alarm, displaying alarm information on a screen of a car navigation system, etc., or vibrating a seat belt or steering wheel.

[0110] In this embodiment, the surroundings of the vehicle, for example, the front or rear, are imaged by the photoelectric conversion system 1300. Fig. 10(b) shows the photoelectric conversion system when imaging the area in front of the vehicle (imaging range 1350). A vehicle information acquisition device 1320 sends instructions to the photoelectric conversion system 1300 or the imaging device 1310. This configuration can further improve the accuracy of distance measurement.

[0111] Although an example of control to avoid collision with other vehicles has been described above, the present invention can also be applied to control of automatic driving by following other vehicles, control of automatic driving to avoid going out of a lane, and the like. Furthermore, the photoelectric conversion system is not limited to vehicles such as automobiles, but can be applied to moving bodies (moving devices) such as ships, aircraft, and industrial robots. The moving body includes one or both of a driving force generating unit that generates a driving force mainly used for the movement of the moving body, and a rotating body mainly used for the movement of the moving body. The driving force generating unit can be an engine, a motor, and the like. The rotating body can be a tire, a wheel, a screw of a ship, a propeller of an aircraft, and the like. In addition, the present invention can be applied not only to moving bodies, but also to devices that widely use object recognition, such as an intelligent transport system (ITS).

[0112] Fifth embodiment The photoelectric conversion system of this embodiment will be described with reference to Fig. 11. Fig. 11 is a block diagram showing an example of the configuration of a range image sensor which is the photoelectric conversion system of this embodiment.

[0113] 11, the distance image sensor 401 is configured to include an optical system 407, a photoelectric conversion device 408, an image processing circuit 404, a monitor 405, and a memory 406. The distance image sensor 401 can obtain a distance image according to the distance to the subject by receiving light (modulated light or pulsed light) that is projected from a light source device 409 toward the subject and reflected from the surface of the subject.

[0114] The optical system 407 is configured to have one or more lenses, and guides image light (incident light) from a subject to a photoelectric conversion device 408 , forming an image on the light receiving surface (sensor portion) of the photoelectric conversion device 408 .

[0115] As the photoelectric conversion device 408 , the photoelectric conversion device of each of the above-mentioned embodiments is applied, and a distance signal indicating a distance determined from a light reception signal output from the photoelectric conversion device 408 is supplied to the image processing circuit 404 .

[0116] The image processing circuit 404 performs image processing to construct a distance image based on the distance signal supplied from the photoelectric conversion device 408. The distance image (image data) obtained by this image processing is then supplied to a monitor 405 for display, or supplied to a memory 406 for storage (recording).

[0117] In the range image sensor 401 configured in this way, by applying the above-described photoelectric conversion device, it is possible to obtain, for example, a more accurate range image as the pixel characteristics improve.

[0118] Sixth embodiment The photoelectric conversion system of this embodiment will be described with reference to Fig. 12. Fig. 12 is a diagram showing an example of a schematic configuration of an endoscopic surgery system which is the photoelectric conversion system of this embodiment.

[0119] 12 shows a state in which an operator (doctor) 1131 is performing surgery on a patient 1132 on a patient bed 1133 using an endoscopic surgery system 1150. As shown in the figure, the endoscopic surgery system 1150 is composed of an endoscope 1100, a surgical tool 1110, and a cart 1134 on which various devices for endoscopic surgery are mounted.

[0120] The endoscope 1100 is composed of a lens barrel 1101, a region of a predetermined length from the tip of which is inserted into a body cavity of a patient 1132, and a camera head 1102 connected to the base end of the lens barrel 1101. In the illustrated example, the endoscope 1100 is configured as a so-called rigid lens barrel having a rigid lens barrel 1101, but the endoscope 1100 may be configured as a so-called flexible lens barrel having a flexible lens barrel.

[0121] An opening into which an objective lens is fitted is provided at the tip of the lens barrel 1101. A light source device 1203 is connected to the endoscope 1100, and light generated by the light source device 1203 is guided to the tip of the lens barrel 1101 by a light guide extending inside the lens barrel 1101, and is irradiated via the objective lens toward an observation target in a body cavity of a patient 1132. The endoscope 1100 may be a direct-viewing endoscope, an oblique-viewing endoscope, or a side-viewing endoscope.

[0122] An optical system and a photoelectric conversion device are provided inside the camera head 1102, and reflected light (observation light) from an observation target is collected by the optical system onto the photoelectric conversion device. The observation light is photoelectrically converted by the photoelectric conversion device to generate an electrical signal corresponding to the observation light, i.e., an image signal corresponding to an observation image. The photoelectric conversion device may be any of the photoelectric conversion devices (imaging devices) described in the above-described embodiments. The image signal is transmitted to a camera control unit (CCU: Camera Control Unit) 1135 as RAW data.

[0123] The CCU 1135 is configured with a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), etc., and performs overall control of the operations of the endoscope 1100 and the display device 1136. Furthermore, the CCU 1135 receives an image signal from the camera head 1102, and performs various types of image processing on the image signal, such as development processing (demosaic processing), for displaying an image based on the image signal.

[0124] Under the control of the CCU 1135 , the display device 1136 displays an image based on an image signal that has been subjected to image processing by the CCU 1135 .

[0125] The light source device 1203 is composed of a light source such as an LED (Light Emitting Diode), and supplies the endoscope 1100 with irradiation light when photographing an operation site or the like.

[0126] The input device 1137 is an input interface for the endoscopic surgery system 1150. A user can input various information and instructions to the endoscopic surgery system 1150 via the input device 1137.

[0127] The treatment tool control device 1138 controls the driving of the energy treatment tool 1112 for cauterizing tissue, incising, sealing blood vessels, or the like.

[0128] The light source device 1203 that supplies irradiation light to the endoscope 1100 when photographing the surgical site can be composed of, for example, an LED, a laser light source, or a white light source composed of a group of LEDs of multiple colors or a group of lasers of multiple colors. When the white light source is composed of a combination of RGB laser light sources, the output intensity and output timing of each color (each wavelength) can be controlled with high precision, so that the light source device 1203 can adjust the white balance of the captured image. In this case, it is also possible to capture images corresponding to each of the RGB colors in a time-division manner by irradiating the observation target with laser light from each of the RGB laser light sources in a time-division manner and controlling the driving of the image sensor of the camera head 1102 in synchronization with the irradiation timing. According to this method, a color image can be obtained without providing a color filter to the image sensor.

[0129] Furthermore, the light source device 1203 may be controlled to change the intensity of the light it outputs at predetermined time intervals. By controlling the driving of the image sensor of the camera head 1102 in synchronization with the timing of the change in the light intensity to acquire images in a time-division manner and synthesizing the images, it is possible to generate an image with a high dynamic range that is free of so-called blocked-up shadows and blown-out highlights.

[0130] The light source device 1203 may be configured to supply light in a predetermined wavelength band corresponding to the special light observation. In the special light observation, for example, the wavelength dependency of light absorption in body tissue is utilized. Specifically, a predetermined tissue such as blood vessels on the mucous membrane surface is photographed with high contrast by irradiating light in a narrower band than the irradiation light (i.e., white light) in normal observation. Alternatively, the special light observation may be a fluorescent observation in which an image is obtained by fluorescence generated by irradiating excitation light. In the fluorescent observation, it is possible to irradiate excitation light to the body tissue and observe the fluorescence from the body tissue, or to locally inject a reagent such as indocyanine green (ICG) into the body tissue and irradiate the body tissue with excitation light corresponding to the fluorescent wavelength of the reagent to obtain a fluorescent image. The light source device 1203 may be configured to supply narrow band light and / or excitation light corresponding to such special light observation.

[0131] Seventh embodiment The photoelectric conversion system of this embodiment will be described with reference to Figs. 13(a) and (b). Fig. 13(a) describes glasses 1600 (smart glasses) which are the photoelectric conversion system of this embodiment. The glasses 1600 have a photoelectric conversion device 1602. The photoelectric conversion device 1602 is the photoelectric conversion device (imaging device) described in each of the above embodiments. A display device including a light emitting device such as an OLED or LED may be provided on the back side of the lens 1601. The photoelectric conversion device 1602 may be one or more. A combination of multiple types of photoelectric conversion devices may also be used. The arrangement position of the photoelectric conversion device 1602 is not limited to that shown in Fig. 13(a).

[0132] The glasses 1600 further include a control device 1603. The control device 1603 functions as a power source that supplies power to the photoelectric conversion device 1602 and the display device. The control device 1603 also controls the operations of the photoelectric conversion device 1602 and the display device. The lens 1601 is formed with an optical system for focusing light on the photoelectric conversion device 1602.

[0133] FIG. 13(b) illustrates glasses 1610 (smart glasses) according to one application example. The glasses 1610 have a control device 1612, and the control device 1612 is equipped with a photoelectric conversion device corresponding to the photoelectric conversion device 1602 and a display device. The lens 1611 is formed with an optical system for projecting light emitted from the photoelectric conversion device in the control device 1612 and the display device, and an image is projected onto the lens 1611. The control device 1612 functions as a power source for supplying power to the photoelectric conversion device and the display device, and controls the operation of the photoelectric conversion device and the display device. The control device may have a line-of-sight detection unit that detects the line of sight of the wearer. The line of sight may be detected using infrared rays (infrared light). The infrared light emission unit emits infrared light toward the eyeball of a user gazing at a display image. An image of the eyeball is obtained by an image capture unit having a light receiving element detecting the reflected light of the emitted infrared light from the eyeball. By providing a reduction unit that reduces the amount of light from the infrared light emitting unit to the display unit in a plan view, degradation of image quality is reduced.

[0134] The gaze of the user with respect to the displayed image is detected from an image of the eyeball obtained by capturing infrared light. Any known method can be applied to gaze detection using the image of the eyeball. As an example, a gaze detection method based on a Purkinje image formed by reflection of irradiated light on the cornea can be used.

[0135] More specifically, the gaze detection process is performed based on the pupil-corneal reflex method. Using the pupil-corneal reflex method, a gaze vector that indicates the direction (rotation angle) of the eyeball is calculated based on the pupil image and the Purkinje image included in the captured image of the eyeball, thereby detecting the user's gaze.

[0136] The display device of this embodiment may have a photoelectric conversion device having a light receiving element, and may control the display image of the display device based on information about the user's line of sight from the photoelectric conversion device.

[0137] Specifically, the display device determines a first field of view area to which the user gazes and a second field of view area other than the first field of view area based on the line-of-sight information. The first field of view area and the second field of view area may be determined by a control device of the display device, or may be received from an external control device. In the display area of ​​the display device, the display resolution of the first field of view area may be controlled to be higher than the display resolution of the second field of view area. In other words, the resolution of the second field of view area may be lower than that of the first field of view area.

[0138] The display area may have a first display area and a second display area different from the first display area, and a high priority area may be determined from the first display area and the second display area based on line-of-sight information. The first field of view area and the second field of view area may be determined by a control device of the display device, or may be received from an external control device. The resolution of the high priority area may be controlled to be higher than the resolution of areas other than the high priority area. In other words, the resolution of an area with a relatively low priority may be lowered.

[0139] In addition, AI may be used to determine the first field of view area and the area with high priority. The AI ​​may be a model configured to estimate the angle of the line of sight and the distance to an object at the end of the line of sight from the image of the eyeball, using the image of the eyeball and the direction in which the eyeball in the image was actually looking as teacher data. The AI ​​program may be included in the display device, the photoelectric conversion device, or an external device. If included in the external device, it is transmitted to the display device via communication.

[0140] When display control is performed based on visual recognition detection, the present invention is preferably applicable to smart glasses further including a photoelectric conversion device for capturing an image of the outside world. The smart glasses can display captured outside information in real time.

[0141] Eighth embodiment The above-described photoelectric conversion device and photoelectric conversion system may be applied to electronic devices such as so-called smartphones and tablets.

[0142] 14(a) and 14(b) are diagrams showing an example of an electronic device 1500 equipped with a photoelectric conversion device. Fig. 14(a) shows the front side of the electronic device 1500, and Fig. 14(b) shows the rear side of the electronic device 1500.

[0143] 14(a), a display 1510 for displaying an image is disposed in the center of the surface of the electronic device 1500. Then, along the upper side of the surface of the electronic device 1500, front cameras 1521 and 1522 using photoelectric conversion devices, an IR light source 1530 that emits infrared light, and a visible light source 1540 that emits visible light are disposed.

[0144] Also, as shown in FIG. 14(b), along the upper edge of the rear surface of the electronic device 1500, rear cameras 1551 and 1552 using photoelectric conversion devices, an IR light source 1560 that emits infrared light, and a visible light source 1570 that emits visible light are arranged.

[0145] In the electronic device 1500 configured in this manner, by applying the above-mentioned photoelectric conversion device, for example, it is possible to capture a higher quality image. Note that the photoelectric conversion device can also be applied to other electronic devices such as infrared sensors, distance measuring sensors using infrared light sources, security cameras, and personal or biometric authentication cameras. This can improve the accuracy, performance, and the like of these electronic devices.

[0146] Ninth embodiment 15 is a block diagram of an X-ray CT apparatus in this embodiment. The photoelectric conversion devices in the first and second embodiments are applicable to the detectors of the X-ray CT apparatus. The X-ray CT apparatus 30 in this embodiment includes an X-ray generation unit 310, a wedge 311, a collimator 312, and an X-ray detection unit 320. Furthermore, the X-ray CT 30 includes a tabletop 330, a rotating frame 340, a high-voltage generator 350, a data acquisition system (DAS) 351, a signal processing unit 352, a display unit 353, and a control unit 354.

[0147] The X-ray generating unit 310 is composed of, for example, a vacuum tube that generates X-rays. A high voltage and a filament current are supplied to the vacuum tube of the X-ray generating unit 310 from a high voltage generator 350. X-rays are generated by irradiating thermal electrons from the cathode (filament) toward the anode (target).

[0148] The wedge 311 is a filter that adjusts the amount of X-rays irradiated from the X-ray generation unit 310. The wedge 311 attenuates the amount of X-rays so that the X-rays irradiated from the X-ray generation unit 310 to the subject have a predetermined distribution. The collimator 312 is made of a lead plate or the like that narrows down the irradiation range of the X-rays that have passed through the wedge 311. The X-rays generated by the X-ray generation unit 310 are shaped into a cone beam via the collimator 312 and irradiated to the subject on the tabletop 330.

[0149] The X-ray detection unit 320 is configured using the photoelectric conversion device according to the above-mentioned second and third embodiments. The X-ray detection unit 320 detects X-rays that have passed through the subject from the X-ray generation unit 310, and outputs a signal corresponding to the X-ray dose to the DAS 351.

[0150] The rotating frame 340 has an annular shape and is configured to be rotatable. An X-ray generation unit 310 (wedge 311, collimator 312) and an X-ray detection unit 320 are arranged facing each other inside the rotating frame 340. The X-ray generation unit 310 and the X-ray detection unit 320 are rotatable together with the rotating frame 340.

[0151] The high voltage generator 350 includes a boost circuit, and outputs a high voltage to the X-ray generation unit 310. The DAS 351 includes an amplifier circuit and an A / D conversion circuit, and outputs a signal from the X-ray detection unit 320 to the signal processing unit 352 as digital data.

[0152] The signal processing unit 352 includes a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory), and is capable of performing image processing on digital data. The display unit 353 includes a flat display device, and is capable of displaying X-ray images. The control unit 354 includes a CPU, a ROM, a RAM, and the like, and controls the operation of the entire X-ray CT device 30.

[0153] In the above-mentioned embodiment, the description is based on the form of a stacked sensor in which a first substrate and a second substrate are stacked, but the present invention is not limited to this example. In other words, a configuration in which all the components shown in FIG. 3 are arranged on one substrate is also possible. Even in this case, the effect of the embodiment that the transmission delay of the transmission signal VR can be reduced can be obtained.

[0154] In the present specification, expressions such as "A or B", "at least one of A and B", "at least one of A or / and B", "one or more of A or / and B" may be used. In this case, unless otherwise expressly defined, all possible combinations of the listed items can be included. That is, the above expressions are understood to disclose all cases including at least one A, at least one B, and both at least one A and at least one B. This applies equally to combinations of three or more elements.

[0155] The above-described embodiments can be modified as appropriate without departing from the technical concept. The disclosure of this specification includes not only what is described in this specification, but also all matters that can be understood from this specification and the drawings attached hereto. The disclosure of this specification also includes the complement of the concepts described in this specification. In other words, if this specification contains a statement that "A is greater than B," for example, this specification can be said to disclose that "A is not greater than B" even if the statement that "A is not greater than B" is omitted. This is because when it is stated that "A is greater than B," it is assumed that the case in which "A is not greater than B" is taken into consideration.

[0156] The disclosure of this embodiment includes the following configuration.

[0157] (Configuration 1) A plurality of pixels are provided, Each of the plurality of pixels is an avalanche photodiode; a photon counting circuit that counts the number of photons incident on the avalanche photodiode within an exposure period; a memory connected to the photon counting circuit; a comparison circuit connected to the photon counting circuit and the memory; A photoelectric conversion device comprising:

[0158] (Configuration 2) 2. The photoelectric conversion device according to configuration 1, wherein the comparison circuit compares a first signal output from the photon counting circuit with a second signal output from the memory.

[0159] (Configuration 3) the first signal is a signal counted during a first period of time; 3. The photoelectric conversion device according to configuration 2, wherein the second signal is a signal counted during a second period that does not overlap with the first period.

[0160] (Configuration 4) 4. The photoelectric conversion device according to configuration 3, wherein the comparison between the first signal and the second signal in the comparator circuit includes a subtraction process between the first signal and the second signal.

[0161] (Configuration 5) a processing circuit connected to the comparison circuit; 5. The photoelectric conversion device according to any one of configurations 1 to 4, wherein the processing circuit outputs a detection signal according to a result output from the comparison circuit.

[0162] (Configuration 6) 6. The photoelectric conversion device according to any one of configurations 1 to 5, further comprising a first switch between the photon counting circuit and the memory.

[0163] (Configuration 7) 7. The photoelectric conversion device according to configuration 6, wherein the first switch switches between connection and disconnection between the photon counting circuit and the memory in response to the detection signal.

[0164] (Configuration 8) the first switch connects the photon counting circuit and the memory when the output from the comparison circuit exceeds a predetermined threshold; 8. The photoelectric conversion device according to configuration 6 or 7, wherein a writing operation from the photon counting circuit to the memory and an output operation of luminance information from the memory are performed.

[0165] (Configuration 9) 6. The photoelectric conversion device according to configuration 5, wherein the memory does not output a signal when the result output from the comparison circuit is equal to or lower than a predetermined threshold value.

[0166] (Configuration 10) 10. The photoelectric conversion device according to any one of configurations 1 to 9, wherein the pixel has a second memory connected to the memory.

[0167] (Configuration 11) 11. The photoelectric conversion device according to configuration 10, wherein the number of bits of the second memory is greater than the number of bits of the memory.

[0168] (Configuration 12) 12. The photoelectric conversion device according to configuration 10 or 11, wherein the number of bits of the second memory is more than three times the number of bits of the memory.

[0169] (Configuration 13) 13. The photoelectric conversion device according to any one of configurations 9 to 12, wherein the second memory adds signals output from the memories.

[0170] (Configuration 14) 14. The photoelectric conversion device according to configuration 13, wherein the second memory outputs a signal to the outside after the signals output from the second memory are added.

[0171] (Configuration 15) the avalanche photodiode has a first terminal and a second terminal; 15. The photoelectric conversion device according to any one of configurations 1 to 14, further comprising a first switch connected between the first terminal and a power supply voltage, the first switch switching between a charged state and an uncharged state of the avalanche photodiode.

[0172] (Configuration 16) 16. The photoelectric conversion device according to any one of configurations 1 to 15, further comprising a waveform shaping unit connected between the photon counting circuit and the avalanche photodiode.

[0173] (Configuration 17) The photoelectric conversion device according to any one of configurations 1 to 16, a signal processing unit that generates an image using a signal output from the photoelectric conversion device.

[0174] (Configuration 18) A moving object including the photoelectric conversion device according to any one of configurations 1 to 16, A control unit that controls the movement of the moving body using a signal output from the photoelectric conversion device. A moving object characterized by:

Claims

1. A plurality of pixels are provided, Each of the plurality of pixels an avalanche photodiode; a photon counting circuit that counts the number of photons incident on the avalanche photodiode within an exposure period; a memory connected to the photon counting circuit; a comparison circuit connected to the photon counting circuit and the memory; A photoelectric conversion device comprising:

2. 2. The photoelectric conversion device according to claim 1, wherein the comparison circuit compares a first signal output from the photon counting circuit with a second signal output from the memory.

3. the first signal is a signal counted during a first period; 3. The photoelectric conversion device according to claim 2, wherein the second signal is a signal counted during a second period that does not overlap with the first period.

4. 4. The photoelectric conversion device according to claim 3, wherein the comparison between the first signal and the second signal in the comparator circuit includes a subtraction process between the first signal and the second signal.

5. a processing circuit connected to the comparison circuit; 3. The photoelectric conversion device according to claim 2, wherein the processing circuit outputs a detection signal according to a result output from the comparison circuit.

6. 6. The photoelectric conversion device according to claim 5, further comprising a first switch between the photon counting circuit and the memory.

7. 7. The photoelectric conversion device according to claim 6, wherein the first switch switches between connection and disconnection between the photon counting circuit and the memory in response to the detection signal.

8. the first switch connects the photon counting circuit and the memory when the output from the comparison circuit exceeds a predetermined threshold; 7. The photoelectric conversion device according to claim 6, wherein a writing operation from said photon counting circuit to said memory and an output operation of luminance information from said memory are performed.

9. 6. The photoelectric conversion device according to claim 5, wherein the memory does not output a signal when the result output from the comparison circuit is equal to or lower than a predetermined threshold value.

10. 2. The photoelectric conversion device according to claim 1, wherein the pixel has a second memory connected to the memory.

11. 11. The photoelectric conversion device according to claim 10, wherein the number of bits of the second memory is greater than the number of bits of the memory.

12. 12. The photoelectric conversion device according to claim 11, wherein the number of bits of the second memory is more than three times the number of bits of the memory.

13. 12. The photoelectric conversion device according to claim 11, wherein the second memory adds the signals output from the memories.

14. 14. The photoelectric conversion device according to claim 13, wherein the second memory outputs a signal to an outside after the signals output from the second memory are added.

15. the avalanche photodiode has a first terminal and a second terminal; 2. The photoelectric conversion device according to claim 1, further comprising a first switch connected between the first terminal and a power supply voltage, the first switch switching the avalanche photodiode between a charged state and a non-charged state.

16. 16. The photoelectric conversion device according to claim 15, further comprising a waveform shaping section connected between the photon counting circuit and the avalanche photodiode.

17. The photoelectric conversion device according to any one of claims 1 to 16, a signal processing unit that generates an image using a signal output from the photoelectric conversion device.

18. A moving object comprising the photoelectric conversion device according to any one of claims 1 to 16, A control unit that controls the movement of the moving body using a signal output from the photoelectric conversion device. A moving object characterized by:

Images