Photoelectric conversion device, photoelectric conversion system, computing circuit, and mobile body

Abstract

The present application relates to a photoelectric conversion device, a photoelectric conversion system, a calculation circuit, and a mobile body. The photoelectric conversion device includes a circuit provided between an avalanche photodiode and a power supply; a counter configured to count an output signal output from the avalanche photodiode; and a memory, wherein time information indicating a time at which a count value of the counter reaches a threshold value in a predetermined exposure time period shorter than an exposure time is written in the memory, wherein a clock signal is configured to be input to the circuit in the exposure time period.

Description

Technical Field

[0001] This invention relates to a photoelectric conversion device, a photoelectric conversion system, a computing circuit, and a mobile body. Background Technology

[0002] A photodetector using an avalanche photodiode (APD) is known, which can detect extremely weak light at the single-photon level by utilizing the avalanche (electron avalanche) multiplication effect. The APD forms a high electric field region (avalanche multiplication region) by using a first semiconductor region of a first conductivity type having the same polarity as the signal charge and a second semiconductor region of a second conductivity type having a polarity different from the signal charge.

[0003] U.S. Patent Application Publication 2020 / 0244909 describes a photodetector that controls an APD to enter an avalanche multiplication standby state and a recharge state to return to the standby state using a clock signal with a predetermined frequency. More specifically, the clock signal controls the on / off switching of a switch between the APD and a power supply that applies a reverse bias to the APD. For example, when the clock signal is at a first level, the switch is off to put the APD into a standby state. Furthermore, when the clock signal is at a second level, the switch is on to put the APD into a recharge state. The clock signal is also configured to undergo a logical operation with an output signal from the APD. Therefore, when photons are incident on the APD in the standby state, an output signal is output from the APD to a counter at the timing of the clock signal transitioning from the first level to the second level.

[0004] By using the configuration described in U.S. Patent Application Publication 2020 / 0244909, the accumulation phenomenon that prevents photon counting under high light intensity can be prevented. However, with the configuration described in U.S. Patent Application Publication 2020 / 0244909, the count value saturates when the light intensity becomes higher than a predetermined light intensity. Therefore, the dynamic range of the light detection device is not sufficiently expanded. Summary of the Invention

[0005] The present invention relates to a photoelectric conversion device including an avalanche photodiode (APD) whose dynamic range can be further expanded compared to the configuration described in U.S. Patent Application Publication 2020 / 0244909.

[0006] According to one aspect of the present invention, a photoelectric conversion device includes: a photodiode configured to perform avalanche multiplication; a circuit disposed between the photodiode and a power supply and configured to switch between a first state and a second state, wherein in the first state the photodiode is electrically connected to the power supply and in the second state the photodiode is not electrically connected to the power supply; a counter configured to count an output signal output from the photodiode; and a memory wherein time information indicating that the count value of the counter reaches a threshold within a predetermined exposure time period is written into the memory, the predetermined exposure time period being included in and shorter than the exposure time period, wherein a clock signal is configured to be input to the circuit during the exposure time period.

[0007] Further features of the invention will become apparent from the following description of typical embodiments with reference to the accompanying drawings. Attached Figure Description

[0008] Figure 1 This is a diagram showing the configuration of the photoelectric conversion device.

[0009] Figure 2 This is a diagram showing an example of the layout of a sensor substrate.

[0010] Figure 3 This is a diagram showing an example of the layout of a circuit board.

[0011] Figure 4 It is a block diagram of the equivalent circuit including the photoelectric conversion device.

[0012] Figure 5A , Figure 5B and Figure 5C This is a diagram showing the relationship between the operation and output signal of an avalanche photodiode (APD).

[0013] Figure 6A , Figure 6B and Figure 6C These are diagrams illustrating pixel circuit block diagrams and equivalent circuit diagrams according to comparative examples and first exemplary embodiments, respectively.

[0014] Figure 7A and Figure 7B Each graph shows the relationship between light intensity and count value based on the comparative example.

[0015] Figure 8A , Figure 8B and Figure 8C The graphs show the relationship between time and count values ​​according to the comparative example and the first typical embodiment, respectively.

[0016] Figure 9A and Figure 9BThese are graphs showing the relationship between light intensity and count value, and the relationship between light intensity and exposure time, respectively, according to the first typical embodiment.

[0017] Figure 10 This is a block diagram illustrating details of a photoelectric conversion element according to a first exemplary embodiment.

[0018] Figure 11 This is a timing diagram of the photoelectric conversion element according to the first typical embodiment.

[0019] Figure 12 This is a graph showing the relationship between the number of incident photons and the count value according to various comparative examples and the first typical embodiment.

[0020] Figure 13A and Figure 13B This is a block diagram and flowchart illustrating the image reconstruction process according to a first typical embodiment.

[0021] Figure 14 This is a flowchart illustrating the image reconstruction process according to a first typical embodiment.

[0022] Figures 15A to 15E These are diagrams illustrating clock signals according to a second typical embodiment.

[0023] Figure 16 This is a diagram illustrating the clock signal according to a third typical embodiment.

[0024] Figure 17 This is a diagram illustrating the clock signal according to a third typical embodiment.

[0025] Figure 18 This is a block diagram illustrating a photoelectric conversion system according to a fourth exemplary embodiment.

[0026] Figure 19A and Figure 19B This is a block diagram illustrating a photoelectric conversion system according to a fifth exemplary embodiment.

[0027] Figure 20 This is a block diagram illustrating a photoelectric conversion system according to a sixth exemplary embodiment.

[0028] Figure 21 This is a block diagram illustrating a photoelectric conversion system according to a seventh exemplary embodiment.

[0029] Figure 22A and Figure 22B The figures are respectively illustrating specific examples of photoelectric conversion systems according to the eighth exemplary embodiment. Detailed Implementation

[0030] The exemplary embodiments described below are intended to illustrate the technical concept of the invention and are not intended to limit the invention. The size and positional relationships of the components shown in the figures may sometimes be exaggerated for clarity. In the following description, the same reference numerals are assigned to the same components, and repeated descriptions may be omitted. Furthermore, provided there are no technical problems, the configurations described in each exemplary embodiment can be replaced by or combined with the configuration described in another exemplary embodiment.

[0031] In the following description, exemplary embodiments of the invention will be described with reference to the accompanying drawings. Additionally, in the following description, terms indicating specific directions or positions (e.g., "up," "down," "right," "left," and other terms including them) may be used as needed. These terms are used to facilitate understanding of the invention with reference to the accompanying drawings and are not intended to limit the technical scope of the invention by the meaning of these terms.

[0032] In the following description, the anode of the avalanche photodiode (APD) is set to a fixed potential, and the signal is extracted from the cathode side. Therefore, the semiconductor region with the same polarity as the signal charge and being the majority carrier is the N-type semiconductor region, and the semiconductor region with the same polarity as the signal charge and being the majority carrier is the P-type semiconductor region.

[0033] Furthermore, the cathode of the APD can be set to a fixed potential, and the signal can be extracted from the anode side. In this case, the semiconductor region with the same polarity as the signal charge and being the majority carrier of the first conductivity type is a P-type semiconductor region, and the semiconductor region with the same polarity as the signal charge and being the majority carrier of the second conductivity type is an N-type semiconductor region. In the following description, the case where one node of the APD is set to a fixed potential will be explained, but the potentials of the two nodes can be different.

[0034] Figure 1 This diagram illustrates the configuration of a multilayer photoelectric conversion device 100. The photoelectric conversion device 100 is composed of two substrates electrically connected to each other (i.e., a sensor substrate 11 and a circuit substrate 21). The sensor substrate 11 includes a first wiring configuration and a first semiconductor layer, which includes a photoelectric conversion element 102, described below. The circuit substrate 21 includes a second wiring configuration and a second semiconductor layer, which includes circuitry such as a signal processing unit 103, described below. The photoelectric conversion device 100 is formed by stacking the second semiconductor layer, the second wiring configuration, the first wiring configuration, and the first semiconductor layer in this order. The photoelectric conversion device described in various exemplary embodiments is a back-illuminated type photoelectric conversion device. In this back-illuminated type photoelectric conversion device, light is incident from the second surface side, and the circuit substrate is disposed on the first surface side.

[0035] In the following description, sensor substrate 11 and circuit substrate 21 are diced chips. However, they are not limited to chips. For example, each substrate can be a wafer. Furthermore, each substrate can be diced after being stacked in a wafer state, or the chips can be stacked and bonded after being chip-sized from a wafer state.

[0036] The sensor substrate 11 is provided with a pixel area 12, and the circuit substrate 21 is provided with a circuit area 22 for processing the signal detected by the pixel area 12.

[0037] Figure 2 This is a diagram showing an example layout of the sensor substrate 11. Pixels 101 are arranged in a two-dimensional array in the plan view to form a pixel region 12, wherein each pixel 101 includes a photoelectric conversion element 102 containing an APD 201.

[0038] Each pixel 101 is typically a pixel that forms an image, but when pixel 101 is used for time-of-flight (TOF) measurements, pixel 101 does not necessarily need to form an image. More specifically, pixel 101 can be used to measure the time it takes for light to reach pixel 101 and the amount of light.

[0039] Figure 3 This is a block diagram showing the configuration of the circuit board 21. The circuit board 21 includes signal processing units 103, each configured to process signals from... Figure 2 The photoelectric conversion element 102 performs photoelectric conversion of the charge; readout circuit 112; control pulse generation unit 115; horizontal scanning circuit unit 111; signal line 113; and vertical scanning circuit unit 110.

[0040] Figure 2 Each photoelectric conversion element 102 and Figure 3 The corresponding signal processing units 103 are electrically connected to each other via connection wiring provided for each pixel 101.

[0041] The vertical scanning circuit unit 110 receives control pulses supplied from the control pulse generation unit 115 to supply control pulses to each pixel 101. For the vertical scanning circuit unit 110, logic circuits such as shift registers and address decoders are used.

[0042] The signal output from the photoelectric conversion element 102 of pixel 101 is processed by each signal processing unit 103. Each signal processing unit 103 is equipped with a counter and a memory, etc., and digital values ​​are written to and stored in the memory.

[0043] The horizontal scanning circuit unit 111 inputs control pulse signals for sequentially selecting each column to the corresponding signal processing unit 103 so as to read the signals from the memory of each pixel storing digital signals.

[0044] In the column selected by the horizontal scanning circuit unit 111, the signal processing unit 103 of the pixel selected by the vertical scanning circuit unit 110 outputs a signal to the signal line 113.

[0045] The signal output to signal line 113 is output to a recording unit or signal processing unit located outside the photoelectric conversion device 100 via output circuit 114.

[0046] exist Figure 2 In the pixel region 12, the photoelectric conversion elements 102 can be arranged in one dimension. The function of the signal processing unit 103 does not necessarily have to be set in each photoelectric conversion element 102, and for example, one signal processing unit 103 can be shared by multiple photoelectric conversion elements 102, and signal processing can be performed sequentially.

[0047] like Figure 2 and Figure 3 As shown, multiple signal processing units 103 are arranged in the region overlapping with the pixel region 12 in the plan view. Furthermore, in the plan view, the vertical scanning circuit unit 110, the horizontal scanning circuit unit 111, the readout circuit 112, the output circuit 114, and the control pulse generation unit 115 are arranged to overlap with the region between the end of the sensor substrate 11 and the end of the pixel region 12. In other words, the sensor substrate 11 includes the pixel region 12 and non-pixel regions arranged around the pixel region 12. The vertical scanning circuit unit 110, the horizontal scanning circuit unit 111, the readout circuit 112, the output circuit 114, and the control pulse generation unit 115 are arranged in the region overlapping with the non-pixel regions in the plan view.

[0048] Figure 4 It includes Figure 2 and Figure 3 An example of a block diagram of the equivalent circuit is shown. Figure 4 A block diagram of a photoelectric conversion device including a commonly used APD is shown.

[0049] exist Figure 4 In the sensor substrate 11, a photoelectric conversion element 102 including APD 201 is disposed, and other components are disposed in the circuit substrate 21.

[0050] Due to photoelectric conversion, the APD 201 generates charge pairs corresponding to the incident light. A voltage VL (first voltage) is supplied to the anode of the APD 201. Furthermore, a voltage VH (second voltage), higher than the voltage VL supplied to the anode, is supplied to the cathode of the APD 201. A reverse bias voltage is supplied to the anode and cathode to cause the APD 201 to perform avalanche multiplication operation. By supplying such a voltage, the avalanche multiplication of the charges generated by the incident light generates an avalanche current.

[0051] Furthermore, when a reverse bias voltage is supplied to the APD 201, the APD 201 operates in two modes (i.e., Geiger mode and linear mode). In Geiger mode, the APD 201 operates when the voltage difference between the anode and cathode is greater than the breakdown voltage. In linear mode, the APD 201 operates when the voltage difference between the anode and cathode is close to or less than the breakdown voltage.

[0052] An APD operating in Geiger mode is called a single-photon avalanche diode (SPAD). For example, the voltage VL (first voltage) is -30V and the voltage VH (second voltage) is 3V. The APD 201 can be operated in either linear mode or Geiger mode.

[0053] Quenching element 202 is connected to APD 201 and the power supply for voltage VH. Quenching element 202 acts as a load circuit (quenching circuit) in the event of signal multiplication due to avalanche multiplication, and has the function of suppressing the voltage supplied to APD 201 to suppress avalanche multiplication (quenching operation). Furthermore, quenching element 202 has the function of returning the voltage to be supplied to APD 201 to voltage VH by allowing a current flow corresponding to the voltage reduction due to the quenching operation (recharging operation).

[0054] The signal processing unit 103 includes a waveform shaping unit 210, a counter 211, and a selection circuit 212.

[0055] In this invention, the signal processing unit 103 only needs to include any one of the waveform shaping unit 210, the counter 211, and the selection circuit 212.

[0056] The waveform shaping unit 210 shapes the cathode potential change of the APD 201 obtained when a photon is detected and outputs a pulse signal. For example, an inverter circuit is used as the waveform shaping unit 210. Figure 4 The diagram shows an example of a waveform shaping unit 210 using an inverter. However, a circuit configured with multiple inverters connected in series can be used, or another circuit with waveform shaping effects can be used.

[0057] Counter 211 counts the number of pulse signals output from waveform shaping unit 210 and stores the count value. In addition, when a control pulse pRES is supplied to counter 211 via drive line 213, the signal (count value) stored in counter 211 is reset.

[0058] The control pulse pSEL is from Figure 3 The vertical scanning circuit unit 110 shown is via Figure 4 The drive line 214 shown Figure 3 (Not shown) is supplied to selection circuit 212 to switch between an electrically connected state and an electrically disconnected state of counter 211 and signal line 113. Selection circuit 212 includes, for example, a buffer circuit for output signals.

[0059] A switch, such as a transistor, can be provided between the quenching element 202 and the APD 201, or between the photoelectric conversion element 102 and the signal processing unit 103, to switch the electrical connection. Similarly, a switch, such as a transistor, can be provided to electrically switch the supply of voltage VH or voltage VL to the photoelectric conversion element 102.

[0060] In this exemplary embodiment, a configuration using counter 211 is described. However, instead of counter 211, the photoelectric conversion device 100 can use a time-to-digital converter (TDC) and a memory to obtain the pulse detection timing. In this case, the generation timing of the pulse signal output from the waveform shaping unit 210 is converted into a digital signal by the TDC. The control pulse pREF (reference signal) is then transferred from... Figure 3 The vertical scanning circuit unit 110 shown is supplied to the TDC via a drive line to measure the timing of the pulse signal. The TDC obtains the signal as a digital signal when the input timing of the signal output from each pixel 101 via the waveform shaping unit 210 is relative to the time referenced by the control pulse pREF.

[0061] Figure 5A , Figure 5B and Figure 5C This is a diagram that schematically illustrates the relationship between the operation and output signals of the APD 201.

[0062] Figure 5A It shows from Figure 4 The diagram shows the extracted APD 201, quenching element 202, and waveform shaping unit 210. Figure 5A In this context, the input side of the waveform shaping unit 210 is defined as VC, and its output side is defined as VO. Figure 5B Show Figure 5A The voltage of VC in the middle, and Figure 5C Show Figure 5A The signal of VO in the middle.

[0063] From time t0 to time t1, the voltage difference “VH-VL” is applied to Figure 5A In the APD 201, at time t1, when a photon is incident on the APD 201, avalanche multiplication occurs in the APD 201, causing an avalanche multiplication current to flow in the quenching element 202. As a result, the voltage of VC drops. If the voltage drop increases further and the voltage difference applied to the APD 201 becomes smaller, the avalanche multiplication of the APD 201 stops at time t2, and the voltage level of VC does not drop beyond a predetermined value. Then, between time t2 and time t3, current flows from voltage VL to VC to constitute a voltage drop, and at time t3, the potential level of VC statically stabilizes back to its original potential level. At this time, if the output waveform of VC exceeds a threshold, the output waveform is shaped by the waveform shaping unit 210 and output as a signal to VO.

[0064] Furthermore, the layout of signal line 113, readout circuit 112, and output circuit 114 is not limited to... Figure 3 The layout in the diagram. For example, signal line 113 can be arranged to extend in the row direction, and readout circuitry 112 can be arranged where signal line 113 extends.

[0065] Figure 6A , Figure 6B and Figure 6C Each diagram shows both a pixel circuit block diagram and an equivalent circuit diagram. Figure 6A and Figure 6B The configuration based on the comparative example is shown, and Figure 6C The configuration according to the first typical embodiment is shown. Figure 7A and Figure 7B Comparative examples are shown. Figure 8A and Figure 8B Comparative examples are shown, and Figure 8C This typical embodiment is shown.

[0066] Comparative Example 1: Passive Recharge Circuit

[0067] Figure 6A The block diagram and equivalent circuit diagram of the passive recharge pixel circuit, serving as Comparative Example 1, are shown together. The gate of the transistor used as the quenching element 202 connected to the cathode side of APD 201 is at the same node as the output of logic circuit 221 (or (OR) circuit). Logic circuit 221 is configured to accept the signal ENB and the signal STOP. The signal ENB remains low during the exposure time period T and high during other time periods.

[0068] When both the STOP and ENB signals are low, the transistor acting as quenching element 202 is turned on, and APD 201 enters a recharge state. Then, APD 201 transitions to a wait state where avalanche multiplication can occur after a predetermined time period. In other signal modes, quenching element 202 is turned off, and APD 201 does not enter a recharge state, but instead enters a non-wait state where avalanche multiplication is not possible. Waveform shaping unit 210 is configured with an inverter, and counter 211 is equipped with an 11-bit flip-flop circuit. For example, when the count value of counter 211 reaches the maximum count value "2047", the STOP signal transitions from low to high, and the transistor acting as quenching element 202 is turned off. Thus, even when the ENB signal transitions to low, quenching element 202 remains off, and APD 201 remains in a non-wait state.

[0069] Figure 7A The figures on the right each show Figure 6A The voltage of VC and the signal of VO in the middle.

[0070] Under low light intensity, due to the long incident interval of photons, sufficient time can be ensured after the voltage of VC has decreased after the photons are incident, until the voltage of VC returns to a high level through recharging.

[0071] Figure 7A The diagram on the upper right (low light intensity) shows that three pulses can be counted corresponding to three photons. On the other hand, under high light intensity, due to the short incident interval of photons, the voltage of VC remains low and does not return to a high level. Therefore, it takes time for the voltage to rise from the lower to the upper level and exceed the judgment threshold. Figure 7A The diagram on the lower right shows that although approximately 20 photons were incident, only 3 pulses were counted. In other words, some photons were omitted from the count.

[0072] As a result, Figure 6A In the case of the passive rechargeable configuration shown, such as Figure 7A As shown, the count values ​​under low and high light intensity conditions may become equal. Furthermore, if photons are incident more frequently, the voltage of VC remains low, and the count value does not exceed the judgment threshold from the bottom to the top. In this case, the voltage of VO remains high, and no signal is generated. Therefore, in Figure 6A In the passive rechargeable configuration shown, the dynamic range of the photoelectric conversion device 100 is narrowed because the correct count value cannot be obtained under high light intensity.

[0073] Comparative Example 2: Clock Recharge Circuit

[0074] Figure 6B The diagram shows a block diagram and an equivalent circuit diagram of the clock recharge pixel circuit, which serves as Comparative Example 2. Logic circuit 221 (or circuit) is configured to receive signals CLKB and STOP via its input terminals. Signal CLKB is a clock signal comprising Nc pulses during the exposure time period T. Figure 6B In this configuration, when both the STOP and CLKB signals are low, the transistor acting as quenching element 202 is turned on, and APD 201 is recharged. In any other combination of STOP and CLKB signals, quenching element 202 is turned off. For example, when CLKB is high, quenching element 202 is turned off after recharging, thus APD 201 is in a standby state where avalanche multiplication is possible. In other words, a circuit (transistor) is provided between APD 201 and the power supply (voltage VH), and this circuit controls the switching between the first and second states. In the first state, this circuit is electrically connected to APD 201 and the power supply. In the second state, this circuit is not electrically connected to APD 201 and the power supply. In this case, the first state is the recharging state, and the second state is the standby state.

[0075] A logic circuit 222 (an inverted AND circuit with one input terminal) is provided between the cathode of APD 201 and the counter 211, allowing the input voltages of signals CLKB and VC. Logic circuit 222 is a logic circuit whose output from VC is input in an inverted state. When a photon is incident on APD 201, the value of VC transitions from high to low. In this case, when signal CLKB is high, the output of logic circuit 222 becomes high. Then, when signal CLKB transitions from high to low, the output of logic circuit 222 transitions low, thereby generating an output signal.

[0076] Figure 7B The figures on the right each show Figure 6B The signals CLKB, VC voltage, and VO signal are included.

[0077] First, under low light intensity, when a photon is incident while the signal CLKB is high (in the standby state), the voltage of VC drops and transitions to a low level. Since the signal CLKB is high, the output from VO is high, and when the signal CLKB transitions to low, the output from VO transitions from high to low, thus generating a signal.

[0078] On the other hand, under high light intensity conditions, even when photons are frequently incident while the signal CLKB is at a high level (wait state), the value of VO, which transitions to a high level, remains high as long as the signal CLKB is at a high level. Then, when the signal CLKB transitions from a high level to a low level, the output from VO transitions from a high level to a low level, thus generating a signal.

[0079] Thus, the clock recharge circuit solves the problems that occur in passive recharge circuits, such as the count values ​​becoming equal in low and high light intensity states, and the state where no signal is generated. More specifically, the clock recharge circuit has the advantage that the count value in the low light intensity state will not become greater than the count value in the high light intensity state. The signal output from logic circuit 222 is output to the outside via memory 240 from counter 211. Furthermore, counter 211 can be appropriately configured to count on the falling edge of the output pulse of logic circuit 222 or on the rising edge of a previous pulse of logic circuit 222.

[0080] Description of this exemplary embodiment

[0081] Figure 8A , Figure 8B and Figure 8C Each displays the time and count value during the clock recharge drive. Figure 8A , Figure 8B and Figure 8C In each case, "Nsat" is the number corresponding to the maximum count value of counter 211. Details of "Nsat" will be explained below.

[0082] exist Figure 8A Case (ii) illustrates the situation under low light intensity, where the count value does not reach Nsat when the exposure time period T is completed. On the other hand, case (i) illustrates the situation under high light intensity, where the count value reaches Nsat before the exposure time period T is completed. In this case, if the time when the count value reaches Nsat is recorded, the count value at the end of the exposure time period T can be calculated using an extrapolation method. The count value calculated using the extrapolation method is also called the calculated count value. As a result, the dynamic range of the photoelectric conversion device 100 can be expanded.

[0083] Also in Figure 8B In the diagram, case (i) shows the case under high light intensity, and case (ii) shows the case under low light intensity.

[0084] exist Figure 8B In case (i), it is determined whether the count value reaches or exceeds the predetermined count value (Nsat / m) before the completion of the exposure time period T. For example, Figure 8BThe example demonstrates the case where m = 2, and determines whether the count value reaches or exceeds a predetermined count value at half the exposure time T. If the count value reaches or exceeds the predetermined count value, the exposure stops. In this case, the value obtained by multiplying the measured count value by "m" is considered the calculated count value at the end of the exposure time T. For example, m = 2.

[0085] Figure 8C This is a diagram illustrating a driving method according to this exemplary embodiment. More specifically, Figure 8C Showing the adoption Figure 8B The driving method shown in case (i) provides multiple predetermined decision timings (predetermined checkpoints) simultaneously. Figure 8C In case (i), determine the first exposure time period (T / m) 3 Is the count value in the range the predetermined count value (Nsat / m) or greater? Figure 8C In case (ii), determine the second exposure time period (T / m) 2 Is the count value in the range the predetermined count value (Nsat / m) or greater? Figure 8C In case (iii), it is determined whether the count value during the third exposure time period (T / m) is a predetermined count value (Nsat / m) or greater. In this case, the count value is multiplied by "m". 3 “m” 2 The values ​​obtained for “” and “m” (e.g., 8, 4 and 2) are considered as the calculated count values ​​when the exposure time period T is completed.

[0086] In this scenario, at each timing step, it is determined whether the count value is equal to or greater than a predetermined count value. When the count value is equal to or greater than the predetermined count value, exposure stops, and the count value is output. Therefore, when exposure stops, the count value is greater than or equal to Nsat / m but less than or equal to Nsat.

[0087] If judgments are made at multiple checkpoints, the dynamic range of the photoelectric conversion device 100 can be further expanded.

[0088] Figure 9A Show Figure 8B and Figure 8C The relationship between light intensity and count values ​​in the configuration shown. Furthermore, Figure 9B This illustrates the relationship between light intensity and exposure time in these configurations. In the case of areas with the lowest light intensity, since the counting continues until the end of the exposure time period T, therefore... Figure 9B As shown, the exposure time becomes the longest. In the case of the area with the lowest light intensity, since the counting continues until counter 211 saturates, therefore... Figure 9AAs shown, counter 211 counts from 0 to Nsat. Then, in areas with slightly higher light intensity, it is determined that the count value at the third exposure time interval (T / m) is a predetermined count value (Nsat / m) or greater, and the exposure stops. Since the exposure is stopped midway, therefore... Figure 9B As shown, the exposure time period is shortened. Furthermore, as... Figure 9A As shown, in this case, the output count value is above Nsat / m and below Nsat. During the second exposure time period (T / m) 2 Exposure stops during the first exposure period (T / m) and the situation where exposure stops during the first exposure period (T / m) 3 Even when exposure stops, the count value is still above Nsat / m and below Nsat.

[0089] During the first exposure period (T / m) 3 When exposure stops, counter 211 can count up to Nsat. In this case, the count value is calculated by multiplying Nsat by m. 3 (that is, Nsat×m) 3 The value obtained. In other words, the count value that can be used for image formation can be expanded from Nsat to Nsat×m. 3 Therefore, the dynamic range of the photoelectric conversion device 100 can be expanded. Furthermore, it is possible to control whether counting stops at different time points for each pixel. More specifically, the exposure time period can be controlled for each pixel, and the dynamic range can be expanded for each pixel. Below, examples of circuits or driving methods for implementing the above methods will be described.

[0090] Typical Implementation: Pixel Circuit Block Diagram

[0091] Figure 6C This is a block diagram showing pixel circuitry configured to control the exposure time period for each pixel. APD 201, quenching element 202, logic circuit 221, logic circuit 222, and counter 211 are shown. Figure 6B The ones shown are the same, so their descriptions have been omitted.

[0092] Figure 6C The signal CLKB in Figure 6B The similarity between the signal CLKB and the signal in the above context is that a clock signal (Nc pulses) is input during the exposure time period T (hereinafter also referred to as the maximum exposure time period). However, as will be explained below, the difference between the signal CLKB and the signal CLKB is as follows: Figure 6B Compared to China, in Figure 6C The pulse period of the clock signal is shorter. Furthermore, in... Figure 6CIn this circuit, a signal is input from counter 211 to exposure control circuit 230, and a signal is also input from exposure control circuit 230 to memory 223. The number of bits in exposure control circuit 230 is less than the number of bits in the trigger circuit included in counter 211.

[0093] Figure 10 This is a block diagram illustrating details of the pixel circuitry according to this exemplary embodiment. Details regarding the pixel circuitry with... Figure 6C The label or symbol assigned to the component is the same as the label or symbol of the component. Since counter 211 includes, for example, an 11-bit flip-flop circuit, the saturation value of counter 211 is "2047".

[0094] In addition, the signal EN is input to counter 211. The signal EN is used to define the exposure time period T. More specifically, the exposure time period T begins when the signal EN transitions from low to high, and ends when the signal EN transitions from high to low. Then, counter 211 is put into a stop state.

[0095] Counter 211 is configured to input a signal to exposure control circuit 230. Exposure control circuit 230 includes multiple latches 231. Figure 10 From Figure 10 The first to fourth latches 231 are arranged from left to right. These four latches 231 and the multiplexer circuit 232 determine the predetermined count value (threshold) at the predetermined judgment timing.

[0096] The first latch 231 (the leftmost latch) is configured such that when the count value reaches 1 / 8 of the maximum count value of the counter 211 (i.e., m = 8), a signal S is input to the first latch 231. n / 8 As mentioned above, since the maximum count value of counter 211 is "2047", the value obtained by dividing this maximum count value by 8 (i.e., 1 / 8 of the maximum count value) is not an integer. For this reason, for convenience, "2048", which is close to the maximum count value and easy to calculate, is used as a comparable value to the maximum count value, and is used as S. n / 8 S n / 4 and S n / 2 The benchmark.

[0097] In this typical embodiment, the signal output when the count value reaches 2048 / 8 = 256 or greater is defined as signal S. n / 8 .

[0098] Similarly, the second latch 231 (second from the left) is configured such that when the count value reaches 1 / 4 of the maximum count value of the counter 211 (i.e., m = 4), a signal S is input to the second latch 231. n / 4In this typical embodiment, for convenience, the signal output when the count value is 2048 / 4 = 512 or greater is defined as signal S. n / 4 .

[0099] Furthermore, in a similar manner, the third latch 231 (second from the right) is configured such that when the count value reaches half of the maximum count value (2048) of the counter 211 (i.e., m = 2), a signal S is input to the third latch 231. n / 2 In this typical embodiment, for convenience, the signal output when the count value is 2048 / 2 = 1024 or greater is defined as signal S. n / 2 .

[0100] Furthermore, in a similar manner, the fourth latch 231 (the rightmost latch) is configured such that when the count value reaches the maximum count value of the counter 211, a signal S is input to the fourth latch 231. n In this typical embodiment, the signal output when the count value of the 11-bit counter 211 reaches "2047", which is the maximum count value, is defined as signal S. n .

[0101] In signal S n When input is received to the fourth latch 231, the counter 211 reaches its maximum count value, and therefore outputs a signal STOP via logic circuit 234 (or circuit). The signal STOP is input to the counter 211 and logic circuit 221, thus stopping the operation of the counter 211. Consequently, the transistor used as the quenching element 202 is turned off.

[0102] A control signal (not shown) is input to the multiplexer circuit 232, and a signal stored in any of the latches from the first latch 231 to the third latch 231 is selected and stored in the memory 233.

[0103] For example, the signal of the first latch 231 is used to expand the dynamic range. As described above, since the count value available for image formation can be expanded from Nsat to Nsat×m 3 Therefore, the dynamic range can be expanded as "m" increases. For this reason, the signal of the first latch 231 (m=8) is used. On the other hand, if the dynamic range is expanded too much, there may be a situation where the reconstructed image appears unnatural. For example, in the case where there are gradients in the image, the signal stored in the third latch 231 (m=2) is used. In this way, latches from the first latch 231 to the third latch 231 can be appropriately selected according to the application.

[0104] exist Figure 10In this configuration, the first to third memories 233 are set up in a top-to-bottom order. As will be explained below, at each of the timings T0, T1, and T2, the count value is checked to see if it is a threshold or greater. If the count value at timing T0 is a threshold or greater, "1" is recorded in the first memory 233. Similarly, if the count value at timing T1 is a threshold or greater, "1" is recorded in the second memory 233. Similarly, if the count value at timing T2 is a threshold or greater, "1" is recorded in the third memory 233.

[0105] The TC is output from memory 233 as a time code (time information). <0> TC <1> and TC <2> The signal is then stored in memory 223. An 11-bit signal is output from counter 211, and a 3-bit signal is output from memory 233. As a result, memory 223 is a total of 14 bits of memory.

[0106] In TC <0> TC <1> and TC <2> If any of them is "1", this means that the count value is the threshold or greater (Nsat / m or greater), causing the output signal STOP to be sent via logic circuit 234 (or circuit).

[0107] When the read signal WRT is input to memory 223, the 11-bit signal of counter 211 and TC at this time are... <0> TC <1> and TC <2> The 3-bit signal is stored in memory 223.

[0108] Furthermore, when the readout signal READ is input to the selection circuit 212, the signal stored in the memory 223 is read out from the selection circuit 212 and sent to the outside of the photoelectric conversion device 100. For example, the information of all rows is stored together in the memory 223 by the signal WRT, and the information is read out sequentially for each row by the signal READ. In this way, a global shutter system that starts exposure for all rows at the same time can be realized.

[0109] Typical Implementation: Timing Diagram

[0110] Figure 11 This is a timing diagram based on this typical embodiment.

[0111] At time t0, the pulse of signal RES is set to ON. For example... Figure 10 As shown, the signal RES is input to latch 231 and counter 211. This input of signal RES can reset the information stored in latch 231 and counter 211 related to the previous frame. Therefore, the count value COUNT of counter 211 becomes "0". Furthermore, although in Figure 10 Although not shown, the signal RES can be input to memory 223 and memory 233 to reset information related to the previous frame.

[0112] At time t1, signal EN is set to ON, and the exposure time period T begins. Here, the exposure time period T is the maximum exposure time period. With few photons incident, counter 211 is not saturated, allowing it to continue counting during the maximum exposure time period. However, as will be explained below, if counter 211 saturates, or if the count value exceeds a predetermined threshold within the predetermined exposure time period, signal STOP goes high, and the effective exposure time period shortens to less than the maximum exposure time period.

[0113] At time t1, the signal CLKB transitions from high to low. Since the signal STOP is low, APD201 begins to recharge, the voltage of VC gradually increases, and APD201 enters a standby state. When the signal CLKB transitions from high to low at time t1, since no photons have yet been incident, the value of VO changes from high to low.

[0114] At time t2, when the photon is incident, avalanche multiplication begins, and the potential of VC decreases. Since the signal from the input of VC to logic circuit 222 is low and the signal CLKB is high, the value of VO, as the output of logic circuit 222, transitions from low to high. This transition changes the count value COUNT from "0" to "1". In other words, in this typical embodiment, the rising edge of VO is used for counting.

[0115] At time t3, when the signal CLKB transitions from high to low, recharging begins as it was at time t1, and the potential of VC changes. Furthermore, as the signal CLKB transitions from high to low, the value of VO also transitions from high to low. In other words, the waveform of VO, which rises at time t2, falls at time t3.

[0116] Signal T0 has elapsed for T / m since the start of the self-exposure time period T. 3 The time t4 transitions from low to high. Here, m is an arbitrary number, and for example, when using multiplexer circuit 232 to select the receiver S. n / 8 When the signal latch 231 (leftmost latch) is active, m is "8". Therefore, time t4 is the time T / 512 elapsed since the start of the exposure time period T.

[0117] Furthermore, regarding the signal CLKB, since the total number of pulses during the exposure time period T is Nc, and a time t4 of T / 512 has elapsed since the start of the exposure time period T, the total number of pulses is therefore Nc / m. 3 .

[0118] At this point, Nc / m3 Set to a value greater than or equal to the maximum counter value of counter 211, that Nc / m 3 It is until the exposure time period T begins, after which T / m has elapsed. 3 The total number of pulses up to the specified time point. This is because, if Nc / m 3 Setting the value to a value smaller than the maximum counter value of counter 211 limits the expansion of the dynamic range. Additionally, while suppressing the dynamic range to some extent, Nc / m 3 It can be set to a value smaller than the maximum counter value of counter 211. For example, Nc / m 3 It can be set to 3 / 4 or more of the maximum counter value.

[0119] In this typical embodiment, when the timing of the completion of the three exposure time periods included in the exposure time period T (maximum exposure time period), each of which is shorter than the maximum exposure time period, is determined, it is determined whether the count value is a threshold or greater.

[0120] In this scenario, assuming the three exposure time periods are defined in order from the shortest exposure time period as the first exposure time period (first judgment timing), the second exposure time period (second judgment timing), and the third exposure time period (third judgment timing), the first exposure time period is the shortest exposure time period. In this typical embodiment, the first exposure time period, the second exposure time period, and the third exposure time period are respectively related to the exposure time period T / m. 3 Exposure time period T / m 2 This corresponds to the exposure time period T / m. In other words, "m" represents the ratio between the lengths of the first and second exposure time periods, and the ratio between the lengths of the second and third exposure time periods.

[0121] As mentioned above, since m = 8 in this case, the second exposure time period is 8 times longer than the first exposure time period. Furthermore, the third exposure time period is 8 times longer than the second exposure time period. "8 times" is merely an example, and the ratio can be 2 times or greater, or 4 times or greater.

[0122] In these settings, in order to include the time T / m elapsed since the start of the self-exposure period T 3 Total number of pulses Nc / m up to the specified time point 3 Set to the maximum count value of counter 211 or greater; for example, in the case of a maximum count value of "2047", Nc / m 3 Set to "2048". In other words, set Nc / m 3The value is set to the maximum count value of counter 211 or greater. Therefore, the total number of pulses Nc during the exposure time period T is approximately 1 million when m = 8. In the comparative example using normal clock recharge drive, when the number of pulses Nc is set to a number commensurate with the maximum count value of counter 211, the number of pulses Nc during the exposure time period T is "2048". Therefore, compared to the comparative example, the number of pulses Nc according to this typical embodiment is extremely large, and the pulse period of the clock signal is extremely short. For this reason, in Figure 6B and Figure 6C The pulse signal periods are shown to be different from each other.

[0123] At time t4, the count value COUNT is "X1" and is less than the threshold "Nsat / m" (i.e., X1 < Nsat / m). In this case, "Nsat" is, for example, "2048". As mentioned above, "Nsat" is a value commensurate with the maximum count value of counter 211 and is a value that makes calculation easier. In this invention, "a value commensurate with the maximum count value of counter 211" can also be regarded as "the maximum count value of counter 211".

[0124] Furthermore, as mentioned above, "m" is the ratio between the lengths of the exposure time periods, and for example, m = 8. Therefore, "Nsat / m" is, for example, "256". Figure 11 In the example shown, because the count value is less than the threshold, the first latch 231 (the leftmost latch) does not latch the signal S. n / 8 Therefore, the signal VC used as the time code <0> Keep it at a low level and input "0" into the first memory 233 (the topmost memory).

[0125] Between time t4 and time t5, the count value COUNT becomes "Nsat / m" or greater, which is set to the threshold, and the first latch 231 (the leftmost latch) latches the signal S. n / 8 Then, the output of the multiplexer circuit 232 transitions from low to high.

[0126] The self-exposure time period T started and T / m elapsed. 2 At time t5, signal T1 transitions from a low level to a high level. For example, time t5 is T / 64 hours elapsed from time t1, which is the start time of the exposure period T.

[0127] Between time t4 and time t5, since the output of the multiplexer circuit 232 is at a high level, when the input signal T1 is at time t5, the signal TC used as the time code... <1> The signal transitions from low to high. Therefore, a "1" is input to the second memory 233. Furthermore, due to the signal TC... <1> The transition from low to high level, therefore, Figure 10 As shown, the signal STOP is provided to the counter 211 and the logic circuit 221 via the logic circuit 234, and the transistor that serves as the quenching element 202 is turned off.

[0128] Furthermore, regarding the signal CLKB, since the total number of pulses during the exposure time period T is Nc, the total number of pulses is Nc / m after a time t5 of T / 64 elapsed from the start of the exposure time period T. 2 .

[0129] Figure 11 An example is shown where a photon is incident after time t5. In this case, avalanche multiplication occurs due to the photon incident, and the potential of VC drops. However, since the transistor, acting as quenching element 202, remains off, avalanche multiplication does not occur again. Therefore, the value of VO transitions from low to high once and remains high.

[0130] Signal T2 transitions from low to high after a time t6 (T / m) from the start of the exposure period T. For example, time t6 is T / 8 of the time elapsed since the start of the exposure period T. Furthermore, regarding signal CLKB, since the total number of pulses during the exposure period T is Nc, the total number of pulses after T / 8 of the time elapsed since the start of the exposure period T is Nc / m.

[0131] At this time, the output of the multiplexer circuit 232 remains high. When the input signal T2 is at time t6, the signal TC used as the time code... <2> The signal transitions from low to high. Therefore, a "1" is input to the third memory 233.

[0132] Because the photon input at time t5 and thereafter causes the value of VO to be high, the signal EN is controlled to transition from high to low so that the high-level value of VO is not counted. As mentioned above, the signal EN also has the function of defining the start and end of the exposure time period T (maximum exposure time period). Alternatively, signals other than signal EN can be used to define the start and end of the maximum exposure time period.

[0133] At time t7, when the readout signal WRT goes high, the signal is read from memory 223 to selection circuit 212. Furthermore, at time t8, when the readout signal READ goes high, this signal is output from selection circuit 212 to the outside of photoelectric conversion device 100. For example, information from all rows is stored together in memory 223 via signal WRT, and information is read sequentially for each row via signal READ. This enables a global shutter system that allows all rows to begin exposure simultaneously for the same time period.

[0134] In this typical embodiment, the threshold for determining the timing is set to "Nsat / m" for the exposure time period ratio "m". In this case, the threshold for the count value is generalized to "Nsat / n" (where n is 2 or a larger number). In other words, "Nsat / n" is "(maximum count value of counter 211) / n". As used herein, "maximum counter value of counter 211" is a number that includes the maximum count value of counter 211 (e.g., 2147) and a value commensurate with the maximum count value of counter 211 (e.g., 2148).

[0135] In this scenario, if "n" is less than "m" which represents the exposure time ratio, the number of pixels whose count does not exceed the threshold in the first exposure time and whose count reaches the saturation value in the second exposure time can increase, leading to grayscale loss under specific light levels. Conversely, by setting "n" in "Nsat / n" to be greater than or equal to "m" which represents the exposure time ratio, there are advantages such as reducing the number of pixels whose count reaches the saturation value and ensuring grayscale under a wide range of light levels from low to high brightness.

[0136] Figure 12 Showing the use of Figure 6A , Figure 6B and Figure 6C The effects of the photoelectric conversion devices 100 in their respective configurations. In other words, referring to... Figure 7A , Figure 7B and Figure 9A The concept is represented by a specific value. Figure 12 The horizontal axis represents the number of incident photons, and the vertical axis represents the median count. The dashed line (a) contrasts with the reference line. Figure 6A The passive recharging configuration described corresponds to this. The unplotted solid line (b) is the reference line. Figure 6B The clock recharging configuration corresponds to that described. The solid line (c) plotted with dots is consistent with the reference. Figure 6C The configuration for controlling the exposure time period for each pixel is described above.

[0137] Figure 12 The dashed line (a) in the diagram saturates at count 2047, after which the number of photons cannot be counted, and the count value suddenly decreases. This is because, as mentioned above, when the number of incident photons is extremely large, the voltage of VC remains low, and since the count value does not exceed the threshold from the bottom to the top, no signal is generated.

[0138] Since the recharge clock in frame 1 is set to "2048" (Nc = 2048) and the maximum count value of counter 211 is "2047", therefore Figure 12 The unplotted solid line (b) in the graph saturates at count 2047. However, the number of incident photons before saturation is greater than [a certain value]. Figure 12 The dashed line (a) represents the number of incident photons. When the number of incident photons is extremely large, a situation where no signal is generated will not occur. However, since only the number of clock cycles can be counted, the upper limit of the count value is limited by the clock frequency.

[0139] Figure 12 The solid line (c) marked with dots is obtained by combining clock recharging with exposure time control for each pixel, which is one of the typical embodiments of the present invention. The upper limit of the count value is "2047", which is the same as the upper limit in the dashed line (a) and the solid line (b). However, the solid line (c) indicates that the count value reaches the bottom three times. As mentioned above, since "Nsat / m" is set as a threshold, the bottom count is "256" when m=8. This method allows the counting to continue even when the incident light amount is greater than the incident light amount corresponding to a count of 2047, thereby enabling an expanded dynamic range. More specifically, since the shortest exposure time period for outputting timecode is set to T / m 3 And since m = 8, the existence of a dynamic range is due to... Figure 12 The solid line (b) represents the advantage of being 512 times wider.

[0140] Typical Implementation: Computational Processing

[0141] Figure 13A and Figure 13B These are diagrams showing the circuitry used for the computational processing and the computational processing itself.

[0142] like Figure 13A As shown, the signal output from the photoelectric conversion device 100 is input to the computing circuit 300. The computing circuit 300 may be included in the photoelectric conversion device 100. Alternatively, the signal can be input to the computing circuit 300 via wired, wireless, or recording medium.

[0143] Figure 13B The calculation process performed in the calculation circuit 300 is shown.

[0144] In step S301, 14 bits of raw data are read from memory 223.

[0145] In step S302, the photon count information corresponding to 11 bits and the time code information corresponding to 3 bits are separated from the 14-bit raw data.

[0146] In step S303, the count value information and time code information are used to perform logical shift (bit shift).

[0147] Figure 14 This specifically illustrates the logical shifting process. In Figure 14In step S310, the logical shift begins. In step S312, the time code TC is determined. <0> Is it "1"?

[0148] In TC <0> If the value is "1" ("Yes" in step S312), the process proceeds to step S314. In step S314, the photon count value corresponding to the 11th bit is multiplied by m. 3 .

[0149] In step S312, in timecode TC <0> If the value is not "1" ("No" in step S312), proceed to step S316. In step S316, the time code TC is determined. <1> Is it "1"? (In TC) <1> If the value is "1" ("Yes" in step S316), the process proceeds to step S318. In step S318, the photon count value corresponding to the 11th bit is multiplied by m. 2 In step S316, in timecode TC <1> If the value is not "1" ("No" in step S316), the process proceeds to step S320. In step S320, the time code TC is determined. <2> Is it "1"? (In TC) <2> If the value is "1" ("Yes" in step S320), the process proceeds to step S322. In step S322, the photon count value corresponding to the 11th bit is multiplied by m. In step S320, at timecode TC... <2> If the value is not "1" ("No" in step S320), the process proceeds to step S324. In step S324, since counter 211 is not saturated, no logic shift processing is required, and the logic shift calculation ends. Thus, in the logic shift processing, the multiplication rate of the counter 211's count value is changed based on time information.

[0150] Return to reference Figure 13B In step S304, nonlinear correction is performed. For example, referencing... Figure 12 The solid line (c) marked with dots in the image represents the incident photon count of 1 × 10⁻⁶. 6 Up to 1×10 7 The slope of the solid line (c) around the region is not constant. Therefore, in step S304, the region with the non-constant slope is corrected.

[0151] More specifically, assuming the corrected count value is X and the uncorrected count value is Y, the relationship between X and Y is expressed as Y = Nc × (1 - exp(-X / Nc)). )In this formula, as described above, Nc is the number of countable pulses within the exposure time period T. In step S305, demosaic processing (interpolation processing) is performed. In step S306, since the light transmittance and reflectance of the red, green, and blue (RGB) filters are different from each other, appropriate tuning of the gain of each signal is performed. In step S307, high dynamic range (HDR) tone mapping is performed. In step S308, the calculated color image is output. Figure 13B The text describes the case where a color image is output, but a black and white image can also be output. In this case, the processing in steps S305 and S306 can be appropriately omitted.

[0152] Variations

[0153] In the first typical embodiment described above, a description is given for the following situation: it is determined whether the count value has reached a threshold at a predetermined checkpoint, and the threshold is set to be smaller than the maximum count value (saturation value) of counter 211. However, the following configuration can be adopted: the maximum count value of counter 211 is set to the threshold, and counting is performed until the threshold is reached. In this case, time information related to the timing when the count value reaches the threshold can be stored in memory, and the count value can be calculated from the time information using extrapolation.

[0154] Similarly, in this method, the number of clock signal pulses input during the exposure time period T is twice or more than twice the maximum count value of counter 211. For example, assuming that the exposure time period T includes a first exposure time period and a second exposure time period, the number of clock signal pulses during the first exposure time period is set to the maximum count value of counter 211 or greater. Furthermore, assuming that the first exposure time period is shorter than the second exposure time period, the number of clock signal pulses during the first exposure time period can be set to the maximum count value of counter 211 or greater. Additionally, as in the typical embodiment described above, assuming that a number of pulses greater than or equal to the maximum count value of counter 211 is required up to T / m... 3 If m = 2, then the number of pulses in the exposure time period T is 8 times or more than 8 times the maximum count value of counter 211. Furthermore, if m = 8, then the number of pulses in the exposure time period T is 512 times or more than 512 times the maximum count value of counter 211.

[0155] refer to Figures 15A to 15E The following will provide an explanation of the frequency variation of the pulse signal according to the second exemplary embodiment. Figures 15A to 15E Show input to reference Figure 6C The timing diagram of signal CLKB of the logic circuit 221.

[0156] First configuration

[0157] Figure 15A The first configuration shown is that the photoelectric conversion device 100 operates using a constant frequency signal CLKB from the beginning to the end of the exposure time period T. Similar to the first typical embodiment, the shortest exposure time period used to determine the count value is T / m. 3 In the case of exposure time T from the beginning to T / m 3 The number of pulses of signal CLKB within the time period is, for example, Nsat. In this case, the number of countable pulses of signal CLKB within the exposure time period T is Nsat × m. 3 Therefore, the dynamic range can be expanded by inserting Nsat recharges within the shortest exposure time period, using up to the upper limit of the count. Furthermore, the number of pulses in the signal CLKB input during the shortest exposure time period used to determine the count value does not need to be exactly Nsat, and the number of pulses input during the shortest exposure time period can be Nsat or greater. In other words, only a clock signal including the number of pulses corresponding to or greater than the maximum count value of counter 211 is required.

[0158] Second configuration

[0159] Figure 15B This is a timing diagram illustrating the signal CLKB in the second configuration. The difference between the second and first configurations is that the frequency of the signal CLKB varies during the exposure time period T. In the second configuration, the frequency of the signal CLKB varies at each of the timing judgment points T / m used to determine the count value. 3 T / m 2 Switching between various decision points in T / m. In other words, two or more types of frequencies of clock signals are set within the exposure time period T.

[0160] For example, assume the time interval from the start of exposure period T to T / m 3 The frequency of the signal CLKB is frequency f1, from T / m 3 To T / m 2 The frequency of the signal CLKB is frequency f2, from T / m 2 The frequency of the signal CLKB from T / m is frequency f3, and the frequency of the signal CLKB from T / m to T is frequency f4. In this case, the relationship f1>f2>f3>f4 is satisfied. If m=8, then the frequency can be reduced according to the ratio of the exposure time intervals, as f1=f2×8=f3×64=f4×512. Applying these ratios corresponds to setting a minimum frequency when photons are incident at predetermined intervals until each judgment time. The minimum frequency is calculated by dividing the maximum possible number of incident photons Nsat by the interval up to each judgment time (e.g., t / m). 3 t / m 2The frequency is calculated up to the time period (e.g., t / m). If the frequency of the signal CLKB is greater than or equal to the minimum frequency, the dynamic range of each judgment timing can be utilized to the maximum extent up to the upper limit of the count, Nsat. For example, assuming the maximum number of incident photons Nsat, up to T / m... 3 Under the current conditions, the minimum frequency f1 = Nsat / (T / m) 3 ).

[0161] From T / m 3 To T / m 2 During the time period, due to the time point T / m 3 The count value did not exceed the threshold, therefore the maximum incident frequency of the photon and the value at the next time point T / m are... 2 This corresponds to the situation where the count value reaches the count Nsat. In other words, from T / m 3 To T / m 2 During the time period, it is only necessary to satisfy f2 = Nsat / (T / m) 2 By calculating in a similar manner, f3 = Nsat / (T / m), f4 = Nsat / T. Thus, by reducing the frequency of the signal CLKB during the exposure time period T to leave only the required number of pulses, the power consumption due to the signal CLKB can be reduced while ensuring the advantage of expanded dynamic range.

[0162] To change the frequency of the clock signal, a frequency divider circuit can be used. This frequency divider circuit can be located in the vertical scan circuit unit, the pixel circuit unit, or the control pulse generation unit. When the frequency divider circuit is located in the pixel circuit unit, it can provide a clock signal with a different frequency for each pixel.

[0163] Additionally, from the start of the exposure time period T to T / m 3 The frequency of the clock signal during the time period (the first exposure time period) may not be constant. Similarly, from T / m 3 To T / m 2 The frequency of the clock signal during the first exposure time period (the second exposure time period) does not need to be constant. In this case, the average frequency can be used as the frequency. For example, the first frequency, which is the average frequency during the first exposure time period, is greater than the second frequency, which is the average frequency during the second exposure time period.

[0164] Third configuration

[0165] Figure 15C This is a timing diagram showing the signal CLKB in the third configuration.

[0166] The third configuration differs from the second configuration in that, instead of changing the frequency before and after the timing used to determine the count value, the frequency gradually decreases from the beginning of the exposure time period T toward the end of the exposure time period T. By gradually adjusting the frequency instead of changing it before and after the timing determination, the difference in count values ​​generated by frequency switching can be reduced. Furthermore, as a method of changing the frequency, in addition to a frequency divider circuit, the options for circuit configuration can be expanded by using, for example, a frequency modulation circuit.

[0167] Alternatively, this configuration can be expressed as follows: the average frequency of the clock signal varies in a direction that gradually decreases from the beginning of the exposure time period T toward the end of the exposure time period T. Furthermore, this configuration can also be expressed as follows: the average frequency during a predetermined time period before the end of the exposure time period T is less than the average frequency during a predetermined time period after the beginning of the exposure time period T.

[0168] Fourth configuration

[0169] Figure 15D This is a timing diagram showing the signal CLKB in the fourth configuration. Although in Figure 15B In the second configuration shown, the frequency of the signal CLKB changes continuously in the direction that decreases towards the latter half of the exposure time period T, but in the fourth configuration, the frequency is at T / m 2 It increases once between T and T / m. For example, assume the following situation: from the beginning of the exposure time T to T / m 2 During the time period up to the judgment time, photons are not incident very frequently (i.e., under low light intensity conditions), and from t / m 2 During the time period from the timing of the judgment to the end of the exposure time T, many photons are incident (i.e., high light intensity conditions). In this case, if... Figure 15B The driving method shown in the diagram results in photon count loss when the illumination intensity condition is changed to a high illumination intensity condition in the latter half. Figure 15D In the driving method shown, even under such conditions, photon counting loss in the latter half can be reduced. Furthermore, the frequency increase ratio of the signal CLKB or the timing used to increase the frequency can be arbitrarily determined, and any ratio and any timing can only be selected if they satisfy the number of pulses or more required to expand the dynamic range.

[0170] Fifth configuration

[0171] Figure 15E This is a timing diagram showing the signal CLKB in the fifth configuration. The fifth configuration and... Figure 15B The difference in the second configuration shown is that the interval between pulses is equal within a group, but the distance between groups gradually increases. However, in Figure 15B and Figure 15E In this process, the number of pulses is the same in each exposure time period (the time period defined by the timing judgment). According to... Figure 15E In this configuration, when the pulses of signal CLKB are close to each other during each exposure time period, counting can be performed even when the incident timings of photons become close to each other. Therefore, since the photoelectric conversion device 100 can have a sensitivity up to high light intensity, its dynamic range can be expanded.

[0172] Furthermore, the fifth configuration can be represented as follows: the number of clock signal pulses per unit time in the first exposure time period is greater than the number of clock signal pulses per unit time in the second exposure time period. In other words, the pulse distribution of the clock signal in the first exposure time period is denser compared to the pulse distribution of the clock signal in the second exposure time period.

[0173] refer to Figure 16 and Figure 17 A description of a variation of the input method for the input pulse signal according to the third exemplary embodiment will be given. Figure 16 and Figure 17 Each shows the input to the reference. Figure 6C The timing diagram of signal CLKB of the logic circuit 221.

[0174] exist Figure 16 In the middle, the signal CLKB (i.e., CLKB) <0> CLKB <1> , , CLKB <n-1>and CLKB <n>The input corresponds to each vertical scan address 0, 1, ..., n-1 and n. In the global shutter drive method, the frequency control for each vertical scan address can be changed over the exposure time period in multiple rows. In this case, any method for controlling the frequency for each vertical address can be used. For example, the following configuration is conceivable: signals with various frequencies are input from the control pulse generation unit 115 to the vertical scan circuit unit 110, and a frequency is selected for each address in the vertical scan circuit unit 110. Alternatively, a signal with a single frequency can be input to the vertical scan circuit unit 110, and the signal can be divided for each vertical scan address in the middle of the exposure time period. Alternatively, a signal with a constant frequency can be input from the vertical scan circuit unit 110, and the signal can be divided for each pixel in the signal processing unit 103.

[0175] Figure 17 This is an example illustrating another driving method, and is related to... Figure 16 The difference lies in that this driving method is not a global shutter driving method, but a rolling shutter driving method. By controlling the frequency for each address in this way, the photoelectric conversion device 100 can be driven by any driving method.

[0176] Reference Figure 18 The photoelectric conversion system according to the fourth typical embodiment will be described. Figure 18 This is a schematic block diagram illustrating a photoelectric conversion system according to this exemplary embodiment.

[0177] The photoelectric conversion device described in the above exemplary embodiments can be applied to various photoelectric conversion systems. Examples of suitable photoelectric conversion systems include digital still cameras, digital video cameras, surveillance cameras, copiers, fax machines, portable telephones, vehicle-mounted cameras, and observation satellites.

[0178] In addition, a camera module, including a camera device and an optical system such as a lens, is included in the photoelectric conversion system. Figure 18 This is a block diagram illustrating a digital still camera as an example.

[0179] Figure 18 The photoelectric conversion system illustrated as an example includes a camera device 1004, which is an example of a photoelectric conversion device 100, and a lens 1002 for converging an optical image of a subject onto the camera device 1004. The photoelectric conversion system also includes an aperture 1003 for changing the amount of light passing through the lens 1002 and a baffle 1001 for protecting the lens 1002. The lens 1002 and aperture 1003 are included in an optical system for converging light onto the camera device 1004. The camera device 1004 is one of the photoelectric conversion devices 100 according to the above-described typical embodiment, and converts the optical image converged by the lens 1002 into an electrical signal.

[0180] The photoelectric conversion system also includes a signal processing unit 1007, which serves as an image generation unit to generate an image by processing the output signal from the imaging device 1004. The signal processing unit 1007 performs operations such as performing various corrections and compressions on the image data as needed, and then outputs the image data. The signal processing unit 1007 may be formed on the semiconductor layer on which the imaging device 1004 is disposed, or it may be formed on a different semiconductor layer than the semiconductor layer on which the imaging device 1004 is disposed. Furthermore, the imaging device 1004 and the signal processing unit 1007 may be formed on the same semiconductor layer.

[0181] The photoelectric conversion system also includes a memory unit 1010 for temporarily storing image data and an external interface (I / F) unit 1013 for communicating with an external computer or the like. The photoelectric conversion system also includes a recording medium 1012, such as a semiconductor memory, for recording and retrieving image data, and a recording medium control I / F unit 1011 for recording data onto and from the recording medium 1012. The recording medium 1012 can be built into the photoelectric conversion system, or it can be attached to and removed from the photoelectric conversion system.

[0182] Furthermore, the photoelectric conversion system includes an overall control / computing unit 1009 for controlling various calculations and controlling the entire photoelectric conversion system (digital camera), and a timing generation unit 1008 for outputting various timing signals to the imaging device 1004 and the signal processing unit 1007. In this typical embodiment, timing signals can be input from an external source, and the photoelectric conversion system only needs to include at least the imaging device 1004 and the signal processing unit 1007 for processing the output signals output from the imaging device 1004.

[0183] The imaging device 1004 outputs an image signal to the signal processing unit 1007. The signal processing unit 1007 performs predetermined signal processing on the image signal output from the imaging device 1004 and outputs the resulting image data. The signal processing unit 1007 uses the image signal to generate an image.

[0184] In this way, according to this exemplary embodiment, a photoelectric conversion system applying the photoelectric conversion device 100 (camera device) according to any of the above exemplary embodiments can be realized.

[0185] Reference Figure 19A and Figure 19B The photoelectric conversion system and mobile body according to the fifth typical embodiment will be described. Figure 19A and Figure 19B The diagrams show the photoelectric conversion system 2300 and the moving body according to this typical embodiment.

[0186] Figure 19A An example of a photoelectric conversion system 2300 associated with a vehicle-mounted camera is shown. The photoelectric conversion system 2300 includes an imaging device 2310. The imaging device 2310 is any photoelectric conversion device according to the above-described exemplary embodiment. The photoelectric conversion system 2300 includes an image processing unit 2312 for image processing of multiple image data acquired by the imaging device 2310. Furthermore, the photoelectric conversion system 2300 includes a disparity acquisition unit 2314 for calculating disparity (i.e., the phase difference between disparity images) based on the multiple image data acquired by the photoelectric conversion system 2300. Additionally, the photoelectric conversion system 2300 includes a distance acquisition unit 2316 for calculating the distance to an object based on the calculated disparity, and a collision determination unit 2318 for determining the possibility of a collision based on the calculated distance. In this exemplary embodiment, the disparity acquisition unit 2314 and the distance acquisition unit 2316 are each examples of distance information acquisition units for obtaining distance information related to an object. In other words, the distance information is information related to disparity, defocus, and distance to an object, etc. The collision determination unit 2318 can use any distance information to determine the likelihood of a collision. The distance information acquisition unit can be implemented using specially designed hardware components or software modules. Furthermore, the distance information acquisition unit can be implemented using a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), or a combination of these components.

[0187] The photoelectric conversion system 2300 is connected to the vehicle information acquisition device 2320, and the photoelectric conversion system 2300 can acquire vehicle information such as vehicle speed, yaw rate, and steering angle. Furthermore, an electronic control unit (ECU) 2330 for vehicle control is connected to the photoelectric conversion system 2300. The ECU 2330 is a control device (control unit) configured to output a control signal for generating braking force to the vehicle based on the judgment result of the collision judgment unit 2318. Additionally, the photoelectric conversion system 2300 is connected to an alarm device 2340, which is configured to output an alarm to the vehicle driver based on the judgment result of the collision judgment unit 2318. For example, if the collision probability is high according to the judgment made by the collision judgment unit 2318, the ECU 2330 performs vehicle control to avoid a collision or mitigate damage by braking, releasing the accelerator pedal, or suppressing engine power output.

[0188] The alarm device 2340 warns the user by using sound or other means to give an alarm, displaying alarm information on the screen of the car navigation system, or providing vibration to the seat belt or steering wheel.

[0189] In this typical embodiment, the photoelectric conversion system 2300 captures images around the vehicle (e.g., at the front or rear of the vehicle). Figure 19B A photoelectric conversion system 2300 is shown, configured to capture images from the front of a vehicle (with a camera range 2350). A vehicle information acquisition device 2320 sends instructions to the photoelectric conversion system 2300 or the camera device 2310. This configuration improves the accuracy of distance measurements.

[0190] In the above-described typical embodiments, an example of using a photoelectric conversion system to control a vehicle to avoid collisions with another vehicle is illustrated. The photoelectric conversion system can also be applied to controlling a vehicle for autonomous driving to follow another vehicle or for autonomous driving without deviating from its traffic lane. Furthermore, in addition to vehicles such as cars, the photoelectric conversion system can also be applied to mobile bodies (mobile devices) such as ships, aircraft, and industrial robots. Moreover, it is not limited to mobile bodies, and the photoelectric conversion system can be widely applied to devices that use object recognition, such as Intelligent Transportation Systems (ITS).

[0191] Reference Figure 20 To illustrate the photoelectric conversion system according to the sixth typical embodiment. Figure 20 This is a block diagram showing the configuration of the distance image sensor 401 as a photoelectric conversion system.

[0192] like Figure 20 As shown, the distance image sensor 401 includes an optical system 402, a photoelectric conversion device 403, an image processing circuit 404, a monitor 405, and a memory 406. The distance image sensor 401 can obtain a distance image corresponding to the distance to the subject by receiving light (modulated light or pulsed light) emitted from the light source device 411 toward the subject and reflected on the surface of the subject.

[0193] The optical system 402 includes one or more lenses and guides the image light (incident light) from the subject to the photoelectric conversion device 403 so as to focus the image light onto the light-receiving surface (sensor section) of the photoelectric conversion device 403.

[0194] Any photoelectric conversion device according to the above-described typical embodiments can be used as photoelectric conversion device 403, and a distance signal representing the distance obtained from the light-receiving signal output from photoelectric conversion device 403 is supplied to image processing circuit 404.

[0195] The image processing circuit 404 performs image processing to construct a distance image based on the distance signal supplied from the photoelectric conversion device 403. Then, the distance image (image data) obtained through image processing is supplied to the monitor 405 for display, or to the memory 406 for storage (recording).

[0196] As described above, the distance image sensor 401, due to the improved characteristics of the pixels obtained by applying the photoelectric conversion device 403 to the distance image sensor 401, can, for example, obtain a more accurate distance image.

[0197] Reference Figure 21 To illustrate the photoelectric conversion system according to the seventh typical embodiment. Figure 21 This is a block diagram schematically illustrating an example configuration of an endoscopic surgical system 1103 as a photoelectric conversion system according to this exemplary embodiment.

[0198] Figure 21 This image shows a surgeon (doctor) 1131 performing surgery on a patient 1132 lying on a hospital bed 1133 using an endoscopic surgical system 1103. Figure 21 As shown, the endoscope operating system 1103 includes an endoscope 1100, surgical tools 1110, and a trolley 1134 on which various devices for endoscopic surgery are placed.

[0199] Endoscope 1100 includes a tube 1101 and a camera head 1102. A predetermined length region from the leading edge of the tube 1101 is inserted into the body cavity of the patient 1132. The camera head 1102 is connected to the base of the tube 1101. Figure 21 In the example shown, an endoscope 1100 (referred to as a rigid endoscope) with a rigid tube 1101 is shown, but the endoscope 1100 can be configured as a flexible endoscope with a flexible tube.

[0200] An opening for mounting an objective lens is provided at the leading edge of the endoscope tube 1101. A light source device 1203 is connected to the endoscope 1100, and light generated by the light source device 1203 is guided to the leading edge of the endoscope tube 1101 via a light guide extending through the interior of the endoscope tube 1101, and then radiated through the objective lens to the target object in the body cavity of the patient 1132. Alternatively, the endoscope 1100 can be a forward-looking endoscope, a slant-looking endoscope, or a lateral-looking endoscope.

[0201] The camera head 1102 includes an optical system and a photoelectric conversion device, and light reflected from the observed target (observation light) is focused by the optical system onto the photoelectric conversion device. The photoelectric conversion device performs photoelectric conversion on the observation light to generate an electrical signal corresponding to the observation light (i.e., an image signal corresponding to the observed image). Any photoelectric conversion device described above in the exemplary embodiment can be used as a photoelectric conversion device. This image signal is sent as RAW data to the camera control unit (CCU) 1135.

[0202] The CCU 1135 is configured with a central processing unit (CPU) and a graphics processing unit (GPU), and together they control the operation of the endoscope 1100 and the display device 1136. Furthermore, the CCU 1135 receives image signals from the camera head 1102 and performs various image processing operations on the image signals, such as image processing (de-mosaic processing), to display an image based on the image signals.

[0203] Under the control of CCU 1135, display device 1136 displays an image based on an image signal that has undergone image processing by CCU 1135.

[0204] The light source device 1203 is configured with a light source such as a light-emitting diode (LED) and supplies illumination light to the endoscope 1100 for taking images of the surgical site.

[0205] Input device 1137 is the input interface of endoscopic surgery system 1103. Users can input various information and instructions into endoscopic surgery system 1103 via input device 1137.

[0206] The treatment tool control device 1138 controls the drive of the energy treatment tool 1112 used for tissue cauterization, dissection, or vascular closure.

[0207] The light source device 1203 for supplying illumination light to the endoscope 1100 to capture images of the surgical site can be configured as a white light source, such as an LED, a laser light source, or a combination thereof. When the white light source is configured as a combination of RGB laser light sources, the white balance of the captured image can be adjusted using the light source device 1203 because the output intensity and timing of each color (wavelength) can be controlled with high accuracy. In this case, by illuminating the object of observation with laser light in a time-division manner using each RGB laser light source, and by controlling the drive of the image sensor of the camera head 1102 in sync with the illumination timing, images corresponding to red, green, and blue can be captured in a time-division manner. Using this method, color images can be obtained even without a color filter on the image sensor.

[0208] Furthermore, the drive of the light source device 1203 can be controlled so that the intensity of the output light changes at predetermined time intervals. By controlling the drive of the image sensor of the camera head 1102 in time-division multiplexing in sync with the timed changes in light intensity, images can be acquired and synthesized to generate images with high dynamic range, free from underexposure and overexposure.

[0209] Furthermore, the light source device 1203 can be configured to supply light within a predetermined wavelength range suitable for special light observation. In special light observation, for example, wavelength dependence of light absorption by body tissues is utilized. More specifically, by radiating light with a narrow bandwidth (narrower than the illumination light (white light) used for normal observation), images of predetermined tissues such as blood vessels in the superficial portion of the mucous membrane can be captured with high contrast.

[0210] Alternatively, fluorescence observation can be performed under special light. Fluorescence observation utilizes fluorescence generated by radiative excitation light to obtain an image. Fluorescence observation enables the observation of fluorescence obtained by irradiating body tissue with excitation light, or by obtaining a fluorescence image by locally injecting a test reagent such as indocyanine green (ICG) into body tissue and irradiating the body tissue with excitation light having a fluorescence wavelength corresponding to the fluorescence wavelength of the test reagent. The light source device 1203 can be configured to supply narrow-bandgap light and / or excitation light that can be used for special light observation.

[0211] Reference Figure 22A and Figure 22B To illustrate the photoelectric conversion system according to the eighth typical embodiment. Figure 22A This is an example configuration of a pair of glasses 1600 (smart glasses) as a photoelectric conversion system.

[0212] A pair of glasses 1600 includes a photoelectric conversion device 1602. The photoelectric conversion device 1602 is the photoelectric conversion device described in any of the exemplary embodiments described above. Furthermore, a display device including a light-emitting device such as an organic LED (OLED) or LED can be disposed on the back side of the lens 1601. One or more photoelectric conversion devices 1602 can be disposed. Furthermore, multiple types of photoelectric conversion devices 1602 can be used in combination. The arrangement position of the photoelectric conversion device 1602 is not limited to... Figure 22A The layout is shown in the diagram.

[0213] The eyeglasses 1600 also include a control device 1603. The control device 1603 is a power source used to supply power to the photoelectric conversion device 1602 and the aforementioned display device. Furthermore, the control device 1603 controls the operation of the photoelectric conversion device 1602 and the display device. In the lens 1601, an optical system is formed to focus light onto the photoelectric conversion device 1602.

[0214] Figure 22B A pair of glasses 1610 (smart glasses) according to an application example is shown. The glasses 1610 includes a control device 1612, on which a photoelectric conversion device corresponding to a photoelectric conversion device 1602 and a display device are mounted. In a lens 1611, an optical system for guiding light to the photoelectric conversion device in the control device 1612 and an optical system for projecting light from the display device are formed to project an image onto the lens 1611. The control device 1612 is powered by a power source that supplies power to the photoelectric conversion device and the display device, and also controls the operation of the photoelectric conversion device and the display device. The control device 1612 may include a gaze detection unit for detecting the wearer's gaze. Infrared light can be used for gaze detection. An infrared emitting unit emits infrared light toward the eyeballs of a user gazing at the displayed image. An image of the user's eyeballs is acquired using a camera unit including light-receiving elements, which detect reflected light from the infrared light emitted to and reflected from the user's eyeballs. Image quality degradation is reduced by including a reduction unit in the plan view to reduce light leakage from the infrared emitting unit to the display unit.

[0215] This involves detecting the gaze of a user relative to the displayed image from an image of the eye captured by an infrared camera. Any known technique can be used to detect the gaze using the captured image of the eye. As an example, a gaze detection method based on a Purkinje image formed by reflected light emitted from the cornea can be used.

[0216] More specifically, gaze detection processing based on the pupillary-corneal reflection method is performed. Using the pupillary-corneal reflection method, the user's gaze is detected by calculating a gaze vector representing the direction (rotation angle) of the eyeball based on the Purkinje image and the image of the pupil included in the captured image of the eyeball.

[0217] The display device according to this exemplary embodiment may include a photoelectric conversion device comprising a light-receiving element, and may control the display image to be displayed on the display device based on the user's gaze information obtained from the photoelectric conversion device.

[0218] More specifically, the first field of view that the user is currently looking at and the second field of view, excluding the first field of view, are determined by the display device based on gaze information. The first and second field of view can be determined by the control device of the display device, or the display device can receive these two field of view areas determined by an external control device. Within the display area of ​​the display device, the display resolution of the first field of view can be controlled to be higher than that of the second field of view. In other words, the resolution of the second field of view can be controlled to be lower than that of the first field of view.

[0219] Furthermore, the display area may include a first display area and a second display area different from the first display area, and a high-priority area can be determined from the first and second display areas based on gaze information. The first and second display areas may be determined by a control device of the display device, or the display device may receive these display areas determined by an external control device. The resolution of the high-priority area can be controlled to be higher than the resolution of areas different from the high-priority area. In other words, the resolution of relatively low-priority areas can be controlled to be lower.

[0220] Alternatively, artificial intelligence (AI) can be used to determine the primary or high-priority visual field. The AI ​​can be a model configured to estimate the angle of gaze and the distance to a target object located in the gaze direction based on an image of the eye, using the image and the actual direction the eye is looking as supervisory data. The AI ​​program can be included in a display device, a photoelectric conversion device, or an external device. In the case where the external device includes the AI ​​program, the estimation results are transmitted to the display device via communication.

[0221] In the case of display control based on gaze detection, display control can be expected to be applied to smart glasses that further include a photoelectric conversion device for capturing images of external objects. The smart glasses can display the captured external information in real time.

[0222] Without departing from the spirit and scope of the technical concept of the present invention, the above-described typical embodiments may be appropriately modified. Furthermore, examples of adding a portion of the configuration in one typical embodiment to another, and examples of replacing a portion of the configuration in one typical embodiment with a portion of the configuration in another typical embodiment, are also included in the typical embodiments according to the present invention.

[0223] In addition, the disclosure of this exemplary embodiment includes the following configurations and methods.

[0224] Configuration 1

[0225] A photoelectric conversion device includes: a photodiode configured to perform avalanche multiplication; a circuit disposed between the photodiode and a power supply configured to switch between the first state and the second state, wherein in the first state the photodiode is electrically connected to the power supply and in the second state the photodiode is not electrically connected to the power supply; a counter configured to count an output signal output from the photodiode; and a memory wherein time information indicating that the counter's count value reaches a threshold within a predetermined exposure time period is written into the memory, the predetermined exposure time period being included in and shorter than the exposure time period, wherein a clock signal is configured to be input to the circuit during the exposure time period.

[0226] Configuration 2

[0227] According to the photoelectric conversion device described in configuration 1, the predetermined exposure time period includes a first exposure time period and a second exposure time period.

[0228] Configuration 3

[0229] According to the photoelectric conversion device of configuration 2, the first exposure time period is shorter than the second exposure time period, and the number of pulses of the clock signal in the first exposure time period is equal to or greater than the maximum count value of the counter.

[0230] Configuration 4

[0231] According to any one of configurations 1 to 3, the photoelectric conversion device, wherein the threshold is the maximum count value of the counter.

[0232] Configuration 5

[0233] According to any one of configurations 1 to 3, the photoelectric conversion device wherein the threshold is less than the maximum count value of the counter.

[0234] Configuration 6

[0235] According to any one of configurations 1 to 3, the photoelectric conversion device, wherein the threshold is the maximum count value of the counter / n.

[0236] Configuration 7

[0237] According to the photoelectric conversion device described in configuration 6, the predetermined exposure time period includes a first exposure time period and a second exposure time period longer than the first exposure time period, and wherein n is equal to or greater than the value of the length of the second exposure time period / the length of the first exposure time period.

[0238] Configuration 8

[0239] According to the photoelectric conversion device described in configuration 2, the length of the second exposure time period is twice or more than twice the length of the first exposure time period.

[0240] Configuration 9

[0241] According to any one of configurations 2 to 8, the photoelectric conversion device includes a third exposure time period that is longer than the second exposure time period, and the length of the third exposure time period is twice or more than twice the length of the second exposure time period.

[0242] Configuration 10

[0243] According to any one of configurations 2, 8, and 9, the photoelectric conversion device, wherein the predetermined exposure time period includes a third exposure time period longer than the second exposure time period, wherein the length of the third exposure time period is four times or more than four times the length of the second exposure time period, and wherein the length of the second exposure time period is four times or more than four times the length of the first exposure time period.

[0244] Configuration 11

[0245] The photoelectric conversion device according to any one of configurations 1 to 10 further includes a control circuit configured to change the potential of the gate of the transistor included in the circuit when the counter reaches the threshold during the predetermined exposure time period.

[0246] Configuration 12

[0247] The photoelectric conversion device according to any one of configurations 1 to 11 further includes a control circuit configured to stop the counter when the counter reaches the threshold within the predetermined exposure time period.

[0248] Configuration 13

[0249] According to any one of configurations 1 to 12, the photoelectric conversion device wherein the number of bits of the memory is less than the number of bits of the counter.

[0250] Configuration 14

[0251] The photoelectric conversion device according to any one of configurations 1 to 13 further includes a logic circuit between the photodiode and the counter, wherein the clock signal is input to the logic circuit during the exposure time period.

[0252] Configuration 15

[0253] According to any one of configurations 1 to 14, the photoelectric conversion device, wherein the clock signal has two or more different frequencies during the exposure time period.

[0254] Configuration 16

[0255] According to the photoelectric conversion device described in configuration 2, the first frequency, which is the average frequency of the clock signal during the first exposure time period, and the second frequency, which is the average frequency of the clock signal during the second exposure time period, are different from each other.

[0256] Configuration 17

[0257] According to the photoelectric conversion device of configuration 16, the first frequency is higher than the second frequency.

[0258] Configuration 18

[0259] According to the photoelectric conversion device of configuration 2, the number of clock signal pulses per unit time in the first exposure time period is greater than the number of clock signal pulses per unit time in the second exposure time period.

[0260] Configuration 19

[0261] A computing circuit configured to calculate a signal output from a photoelectric conversion device according to any one of configurations 1 to 18, wherein the computing circuit changes the multiplication rate of the counter value based on the time information.

[0262] Configuration 20

[0263] The photoelectric conversion device according to configuration 1 further includes the computing circuit according to configuration 19.

[0264] Configuration 21

[0265] A photoelectric conversion system includes: a photoelectric conversion device according to configuration 1; and a computing circuit according to configuration 19.

[0266] Configuration 22

[0267] A photoelectric conversion system includes: a photoelectric conversion device according to configuration 1; and a signal processing unit configured to generate an image using a signal output from the photoelectric conversion device.

[0268] Configuration 23

[0269] A mobile body includes: a photoelectric conversion device according to configuration 1; and a control unit configured to control the movement of the mobile body using a signal output from the photoelectric conversion device.

[0270] Although the invention has been described with reference to exemplary embodiments, it should be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the appended claims is to be interpreted in the broadest sense to include all such modifications, equivalent structures, and functions.< / n>

Claims

1. A photoelectric conversion device, comprising: A photodiode configured to perform avalanche multiplication; A circuit disposed between the photodiode and the power supply is configured to switch between a first state and a second state, wherein in the first state the photodiode is electrically connected to the power supply, and in the second state the photodiode is not electrically connected to the power supply. A counter configured to count the output signal from the photodiode; as well as A memory is provided in which time information indicating that the counter's count value reaches a threshold within a predetermined exposure time period is written into the memory, the predetermined exposure time period being included in and shorter than the exposure time period. The clock signal is configured to be input to the circuit during the exposure period.

2. The photoelectric conversion device according to claim 1, wherein, The predetermined exposure time period includes a first exposure time period and a second exposure time period.

3. The photoelectric conversion device according to claim 2, wherein, The first exposure time period is shorter than the second exposure time period, and the number of pulses of the clock signal during the first exposure time period is equal to or greater than the maximum count value of the counter.

4. The photoelectric conversion device according to any one of claims 1 to 3, wherein, The threshold is the maximum count value of the counter.

5. The photoelectric conversion device according to any one of claims 1 to 3, wherein, The threshold is less than the maximum count value of the counter.

6. The photoelectric conversion device according to any one of claims 1 to 3, wherein, The threshold is the maximum count value of the counter / n, where n is a number of 2 or greater.

7. The photoelectric conversion device according to claim 6, in, The predetermined exposure time period includes a first exposure time period and a second exposure time period that is longer than the first exposure time period, and Where n is equal to or greater than the length of the second exposure time period / the length of the first exposure time period.

8. The photoelectric conversion device according to claim 2, wherein, The length of the second exposure time period is twice or more than twice the length of the first exposure time period.

9. The photoelectric conversion device according to claim 2 or 8, wherein, The predetermined exposure time period includes a third exposure time period that is longer than the second exposure time period, and the length of the third exposure time period is twice or more than twice the length of the second exposure time period.

10. The photoelectric conversion device according to claim 2 or 8, in, The predetermined exposure time period includes a third exposure time period that is longer than the second exposure time period. Wherein, the length of the third exposure time period is four times or more than four times the length of the second exposure time period, and The length of the second exposure time period is four times or more than four times the length of the first exposure time period.

11. The photoelectric conversion device according to claim 9, in, The predetermined exposure time period includes a third exposure time period that is longer than the second exposure time period. Wherein, the length of the third exposure time period is four times or more than four times the length of the second exposure time period, and The length of the second exposure time period is four times or more than four times the length of the first exposure time period.

12. The photoelectric conversion device according to claim 1, further comprising a control circuit configured to change the potential of the gate of a transistor included in the circuit when the counter reaches the threshold within the predetermined exposure time period.

13. The photoelectric conversion device according to claim 1, further comprising a control circuit configured to stop the counter when the counter reaches the threshold within the predetermined exposure time period.

14. The photoelectric conversion device according to claim 1, wherein, The number of bits in the memory is less than the number of bits in the counter.

15. The photoelectric conversion device according to claim 1, further comprising a logic circuit between the photodiode and the counter. in, The clock signal is input to the logic circuit during the exposure period.

16. The photoelectric conversion device according to claim 1, wherein, The clock signal has two or more different frequencies during the exposure period.

17. The photoelectric conversion device according to claim 2, wherein, The first frequency, which is the average frequency of the clock signal during the first exposure time period, and the second frequency, which is the average frequency of the clock signal during the second exposure time period, are different from each other.

18. The photoelectric conversion device according to claim 17, wherein, The first frequency is higher than the second frequency.

19. The photoelectric conversion device according to claim 2, wherein, The number of clock signal pulses per unit time in the first exposure time period is greater than the number of clock signal pulses per unit time in the second exposure time period.

20. A computing circuit configured to calculate a signal output from the photoelectric conversion device according to claim 1, wherein, The calculation circuit adjusts the multiplication rate of the counter's count value based on the time information.

21. The photoelectric conversion device according to claim 1, further comprising the computing circuit according to claim 20.

22. A photoelectric conversion system, comprising: The photoelectric conversion device according to claim 1; as well as The computing circuit according to claim 20.

23. A photoelectric conversion system, comprising: The photoelectric conversion device according to claim 1; as well as A signal processing unit is configured to generate an image using a signal output from the photoelectric conversion device.

24. A mobile body, comprising: The photoelectric conversion device according to claim 1; as well as A control unit is configured to control the movement of the moving body using signals output from the photoelectric conversion device.

Images