Photoelectric conversion device, photoelectric conversion system, and moving object

Abstract

The invention discloses a photoelectric conversion apparatus, a photoelectric conversion system, and a moving object. The apparatus includes a photodiode performing avalanche multiplication, a charging circuit, a control signal generation circuit for generating a control signal to control the charging circuit, a count pulse generation circuit for generating a pulse signal according to a signal output from the photodiode, and a counter for counting the pulse signal output from the count pulse generation circuit. The control signal generation circuit is configured to output a control signal having a first period and a control signal having a second period longer than the first period based on the clock signal and the mask signal. In a case where a count value of the counter reaches a threshold value, a signal to be output from the control signal generation circuit may be switched from a control signal having a first period to a control signal having a second period.

Description

Technical Field

[0001] Aspects of the present embodiments relate to devices and systems. Background Art

[0002] It is known that there is a light detection device using an avalanche photodiode (APD) that can detect single-photon-level weak light using avalanche (electron avalanche) multiplication. In the APD, a first semiconductor region of a first conductivity type having the same polarity as the polarity of a signal charge and a second semiconductor region of a second conductivity type having a polarity different from the polarity of the signal charge form a high electric field region (avalanche multiplication section).

[0003] Japanese Patent Laid-Open No. 2020-123846 discloses a technology for suppressing power consumption by controlling a MOS transistor used as a passive quenching circuit. Specifically, at the start time of the exposure period, the MOS transistor is turned on to enter the operation period of constant operation of the APD (the inactive period is 0%). Next, when the count value becomes the threshold value N1, the time period in which the MOS transistor is turned off is set, and the time period in which the MOS transistor operates is set to half of the operation period (the inactive period is 50%). Next, when the count value becomes the threshold value N2, the time period in which the MOS transistor operates is set to 1 / 4 of the operation period (the inactive period is 75%). With this configuration, the number of times avalanche multiplication is performed is reduced when the illumination is high. Therefore, the power consumption of the pixel can be reduced.

[0004] By using the configuration described in Japanese Patent Laid-Open No. 2020-123846, power consumption can be reduced. However, in Japanese Patent Laid-Open No. 2020-123846, since a passive quenching circuit is used, the relationship between the illuminance and the count value may be reversed in a high illuminance environment. In addition, when the illuminance is too high, a signal may not be generated. Therefore, when the illuminance is high, an appropriate count value may not be obtained, and thus the dynamic range may be narrow. Summary of the invention

[0005] A photoelectric conversion device, comprising: a photodiode control circuit including a photodiode for avalanche multiplication, a charging circuit for switching between a first state in which the photodiode is connected to a power supply and a second state in which the photodiode is not connected to the power supply, and a control signal generating circuit for generating a control signal to control the charging circuit; a count pulse generating circuit for generating a pulse signal according to an output signal from the photodiode; and a counter for counting the pulse signal output from the count pulse generating circuit. The control signal generating circuit is configured to output a control signal having a first period and a control signal having a second period longer than the first period based on a clock signal and a mask signal. When the count value of the counter reaches a threshold value, the signal to be output from the control signal generating circuit is switched from the control signal having the first period to the control signal having the second period.

[0006] Further features of the present disclosure will become apparent from the following description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 is a diagram showing the configuration of a photoelectric conversion device.

[0008] Figure 2 is a diagram showing an example of arrangement on a sensor substrate.

[0009] Figure 3 is a diagram showing an example of arrangement on a circuit substrate.

[0010] Figure 4 is a block diagram showing an equivalent circuit of a photoelectric conversion element.

[0011] Figure 5 (A), (B) and (C) are diagrams showing the relationship between the operation of the APD and the output signal.

[0012] Fig. 6A 2 is a block diagram and an equivalent circuit diagram of a pixel circuit according to a comparative example.

[0013] Figure 6B 2 is a block diagram and an equivalent circuit diagram of a pixel circuit according to various embodiments.

[0014] Fig. 7A and Figure 7B : is a graph showing the relationship between the illuminance and the count value in each embodiment.

[0015] Fig. 8A , Figure 8B and Figure 8C This is a diagram for explaining problems of a circuit that performs clock charge driving.

[0016] Fig. 9 is a block diagram showing the configuration of a photoelectric conversion apparatus according to the first embodiment.

[0017] Fig.10 is a diagram showing in detail the configuration of the photoelectric conversion device according to the first embodiment.

[0018] Fig.11 is a timing chart related to driving of the photoelectric conversion device according to the first embodiment.

[0019] Fig.12 is a diagram showing the correction concept of the photoelectric conversion device according to the first embodiment.

[0020] Fig.13 is a timing chart related to driving of the photoelectric conversion device according to the second embodiment.

[0021] Fig.14 is a block diagram showing the configuration of a photoelectric conversion apparatus according to a third embodiment.

[0022] Fig.15 is a block diagram showing the configuration of a photoelectric conversion system according to a fourth embodiment.

[0023] Fig.16A and Fig. 16B is a block diagram showing the configuration of a photoelectric conversion system according to a fifth embodiment.

[0024] Fig.17 is a block diagram showing the configuration of a photoelectric conversion system according to a sixth embodiment.

[0025] Fig.18 is a block diagram showing the configuration of a photoelectric conversion system according to a seventh embodiment.

[0026] Fig.19A and Fig.19B is a diagram showing a specific example of the photoelectric conversion system according to the eighth embodiment. DETAILED DESCRIPTION

[0027] The following embodiments are used to embody the technical concept of the present disclosure and do not limit the present disclosure. The size and positional relationship of the components shown in the drawings may be exaggerated to make the description clear. In the following description, the same components are represented by the same reference numerals and the description may be omitted. Unless there are technical problems, the configurations described in the various embodiments can be replaced and combined with one or more configurations described in other embodiments.

[0028] Embodiments of the present disclosure will be described below with reference to the accompanying drawings. In the following description, terms indicating specific directions and specific positions (e.g., "top", "bottom", "right", "left", and other terms including these terms) are used when necessary. These terms are used to easily understand aspects of the embodiments with reference to the accompanying drawings, and the technical scope of aspects of the embodiments is not limited by the meanings of these terms.

[0029] In the following description, connections between circuit elements may be described. In this case, even if other elements are interposed between the object elements, the object elements are regarded as being connected to each other unless otherwise specified herein.

[0030] In the following description, the anode of the avalanche photodiode (APD) is at a fixed potential, and a signal is obtained from the cathode side of the APD. Therefore, the first conductive type semiconductor region in which the charges of the same polarity as the charges of the signal are majority carriers is an N-type semiconductor region, and the second conductive type semiconductor region in which the charges of the polarity different from the polarity of the charges of the signal are majority carriers is a P-type semiconductor region.

[0031] The cathode of the APD may be at a fixed potential, and a signal may be obtained from the anode side of the APD. In this case, the semiconductor region of the first conductivity type in which the charges of the same polarity as the charges of the signal are majority carriers is a P-type semiconductor region, and the semiconductor region of the second conductivity type in which the charges of the polarity different from the polarity of the charges of the signal are majority carriers is an N-type semiconductor region. A case in which one of the nodes of the APD is at a fixed potential will be described, but the potentials of the two nodes may vary.

[0032] Figure 1 1 is a diagram showing the configuration of a stacked photoelectric conversion device 100. The photoelectric conversion device 100 is formed by stacking two substrates (a sensor substrate 11 and a circuit substrate 21) and electrically connecting the sensor substrate 11 and the circuit substrate 21 to each other. The sensor substrate 11 includes a first semiconductor layer having a photoelectric conversion element 102 (described later), and a first wiring structure. The circuit substrate 21 includes a second semiconductor layer having a circuit such as a signal processing unit 103 (described later), and a second wiring structure. The photoelectric conversion device 100 is formed by sequentially stacking the second semiconductor layer, the second wiring structure, the first wiring structure, and the first semiconductor layer. The photoelectric conversion device described in each embodiment is a back-illuminated photoelectric conversion device, light is incident on the back-illuminated photoelectric conversion device from the second surface, and in the back-illuminated photoelectric conversion device, the circuit substrate is arranged on the first surface.

[0033] Although the following describes the case where the sensor substrate 11 and the circuit substrate 21 are diced chips, the sensor substrate 11 and the circuit substrate 21 are not limited to chips. For example, each substrate may be a wafer. Alternatively, each substrate may be stacked on top of each other in a wafer form and cut. Each substrate may be formed into a chip from a wafer form, and the chips may be stacked and bonded to each other.

[0034] The pixel region 12 is disposed on the sensor substrate 11. The circuit region 22 is disposed on the circuit substrate 21, and a signal detected in the pixel region 12 is processed in the circuit region 22.

[0035] Figure 2 is a diagram showing an example of a configuration on the sensor substrate 11. Pixels 101 having photoelectric conversion elements 102 including APDs are arranged in a two-dimensional array in a plan view, and form a pixel region 12.

[0036] The pixel 101 is typically a pixel for forming an image, but may additionally have a time-of-flight (TOF) function.

[0037] Figure 3 1 is a diagram showing the configuration of the circuit substrate 21. The circuit substrate 21 includes a signal processing unit 103, a reading circuit 112, a control pulse generating unit 115, a horizontal scanning circuit 111, a signal line 113, and a vertical scanning circuit 110. The signal processing unit 103 Figure 2 The photoelectric conversion element 102 shown processes the electric charges into which light has been photoelectrically converted.

[0038] Figure 2 The photoelectric conversion element 102 shown is connected to the Figure 3 The signal processing unit 103 is shown electrically connected.

[0039] The vertical scanning circuit 110 receives the control pulse supplied from the control pulse generating unit 115 and supplies the control pulse to each pixel. As the vertical scanning circuit 110, a logic circuit such as a shift register or an address decoder is used.

[0040] A signal output from the photoelectric conversion element 102 of the pixel is processed by a signal processing unit 103. The signal processing unit 103 is provided with a counter, a memory, and the like, and a digital value is written and held in the memory.

[0041] The horizontal scanning circuit 111 transmits a control pulse for sequentially selecting each column to the signal processing unit 103 so as to read a signal from a memory of a pixel holding a digital signal.

[0042] For the selected column, a signal is output from the signal processing unit 103 of the pixel selected by the vertical scanning circuit 110 to the signal line 113 .

[0043] The signal output to the signal line 113 is output to a recorder or a signal processing unit existing outside the photoelectric conversion apparatus 100 via an output circuit 114 .

[0044] exist Figure 2 In the embodiment, the photoelectric conversion elements may be arranged one-dimensionally in the pixel region. Each function of the signal processing unit may not be provided for a corresponding one of all the photoelectric conversion elements. For example, a single signal processing unit may be shared by a plurality of photoelectric conversion elements and may process continuous signals.

[0045] like Figure 2 and Figure 3 As shown, a plurality of signal processing units 103 are arranged in a region overlapping with the pixel region 12 in a plan view. In a plan view, a vertical scanning circuit 110, a horizontal scanning circuit 111, a reading circuit 112, an output circuit 114, and a control pulse generating unit 115 are arranged to overlap with a non-pixel region of the sensor substrate 11 between an end portion of the sensor substrate 11 and an end portion of the pixel region 12. In other words, the sensor substrate 11 includes a pixel region 12 and a non-pixel region arranged around the pixel region 12. The vertical scanning circuit 110, the horizontal scanning circuit 111, the reading circuit 112, the output circuit 114, and the control pulse generating unit 115 are arranged in a region overlapping with the non-pixel region in a plan view.

[0046] Figure 4 It is shown Figure 2 and Figure 3 An example of a block diagram of the equivalent circuit. Figure 4 is a block diagram of a photoelectric conversion device including a general APD.

[0047] exist Figure 4 In the embodiment, the photoelectric conversion element 102 having the APD 201 is arranged on the sensor substrate 11 , and other components are arranged on the circuit substrate 21 .

[0048] APD 201 generates charge pairs according to incident light by photoelectric conversion. Voltage VL (first voltage) is supplied to the anode of APD 201. Voltage VH (second voltage) higher than voltage VL supplied to the anode is supplied to the cathode of APD 201. Reverse bias is supplied to the anode and cathode, so that APD 201 performs avalanche multiplication operation. In the state where such voltage is supplied, the charge generated due to the incident light causes avalanche multiplication, and an avalanche current occurs.

[0049] When a reverse bias is supplied to the APD, the APD has a Geiger mode and a linear mode. In the Geiger mode, the APD operates in a state where the potential difference between the anode and cathode of the APD is higher than the breakdown voltage. In the linear mode, the APD operates in a state where the potential difference between the anode and cathode of the APD is close to the breakdown voltage or is equal to or lower than the breakdown voltage.

[0050] An APD operating in the Geiger mode is called a single photon avalanche diode (SPAD). For example, the voltage VL (first voltage) is -30 V, and the voltage VH (second voltage) is 3 V. The APD 201 may operate in the linear mode, or may operate in the Geiger mode.

[0051] The quenching element 202 is connected to the APD 201 and a power supply that supplies the voltage VH. The quenching element 202 serves as a load circuit (quenching circuit) during signal multiplication caused by avalanche multiplication, and has a function of suppressing the voltage to be supplied to the APD 201 and suppressing avalanche multiplication (quenching operation). The quenching element 202 has a function of returning the voltage to be supplied to the APD 201 to the voltage VH by causing a current corresponding to a voltage drop due to the quenching operation to flow (charging operation).

[0052] The signal processing unit 103 includes a waveform shaping unit 210 , a counter 211 , and a selection circuit 212 .

[0053] In the present specification, the signal processing unit 103 may include any one of the waveform shaping unit 210 , the counter 211 , and the selection circuit 212 .

[0054] The waveform shaping unit 210 shapes the potential change of the cathode of the APD 201 and outputs a pulse signal. When a photon is detected, the potential of the cathode of the APD 201 changes. As the waveform shaping unit 210, for example, a single inverter circuit is used. Figure 4 An example is shown in which a single inverter is used as the waveform shaping unit 210. However, as the waveform shaping unit 210, a circuit in which a plurality of inverters are connected in series to each other may be used, or other circuits having a waveform shaping effect may be used.

[0055] The counter 211 counts the number (number of times) of pulse signals output from the waveform shaping unit 210 and holds the count value. When a control pulse pRES is supplied to the counter 211 through the driving line 213, the signal held in the counter 211 is reset.

[0056] The control pulse pSEL is from Figure 3 The vertical scanning circuit 110 shown in FIG. Figure 4 The drive line 214 ( Figure 3The signal line 113 is not shown in the figure. The signal line 113 is supplied to the selection circuit 212, and the selection circuit 212 switches whether to electrically connect the counter 211 to the signal line 113 or to electrically disconnect the counter 211 from the signal line 113. The selection circuit 212 includes, for example, a buffer circuit that outputs the signal.

[0057] A switch such as a transistor may be arranged between the quenching element 202 and the APD 201 to switch whether the quenching element 202 is electrically connected to the APD 201. A switch such as a transistor may be arranged between the photoelectric conversion element 102 and the signal processing unit 103 to switch whether the photoelectric conversion element 102 is electrically connected to the signal processing unit 103. Similarly, a switch such as a transistor may be used to electrically switch whether the voltage VH or the voltage VL is supplied to the photoelectric conversion element 102.

[0058] Figure 5 (A), (B) and (C) are diagrams schematically showing the relationship between the operation of the APD and the output signal.

[0059] Figure 5 (A) is taken from Figure 4 FIG. 2 is a diagram of an APD 201, a quenching element 202, and a waveform shaping unit 210. The input side of the waveform shaping unit 210 is represented by VC, and the output side of the waveform shaping unit 210 is represented by VO. Figure 5 (B) shows the voltage at VC, and Figure 5 (C) shows the signal from VO.

[0060] During the period from time t0 to time t1, a potential difference VH-VL is applied to Figure 5 APD201 shown in (A). When a photon is incident on the APD 201 at time t1, avalanche multiplication occurs in the APD 201, the avalanche multiplication current flows in the quenching element 202, and the voltage at VC drops. When the amount of voltage drop further increases and the potential difference applied to the APD 201 decreases, the avalanche multiplication in the APD 201 stops at time t2, and the voltage level at VC does not become lower than a specific value. Thereafter, in the time period from time t2 to time t3, a current for compensating the voltage drop from the voltage VL flows at VC, and VC is maintained at the original potential level at time t3. In this case, the portion included in the output waveform and exceeding a specific threshold at VC is shaped by the waveform shaping unit 210 and is output as a signal from VO.

[0061] The configuration of the signal line 113 and the configuration of the read circuit 112 and the output circuit 114 are not limited to Figure 3For example, the signal line 113 may be arranged to extend in the row direction, and the reading circuit 112 may be arranged in a region to which the signal line 113 extends.

[0062] First embodiment

[0063] Passive charging circuit

[0064] Fig. 6A 2 is a block diagram and an equivalent circuit diagram of a pixel circuit for passive charging according to a comparative example. The gate of the transistor of the quenching element 202 connected to the cathode side of the APD 201 and the output of the logic circuit 221 (or (OR) circuit) are the same node. The logic circuit 221 is configured so that the signal ENB and the signal STOP can be input to the logic circuit 221. The signal ENB is at a low (L) level (first level) in the exposure period T and at a high (H) level (second level) in other periods. In this case, the L level is the first level, and the H level is the second level. However, in the case where the conductivity type of the MOS transistor is different, the opposite relationship holds. Therefore, the H level can be the first level, and the L level can be the second level.

[0065] When the signal STOP is at an L level and the signal ENB is at an L level, the transistor as the quenching element 202 is turned on and is in a charging state. After a predetermined period of time, the APD 201 enters a standby state in which avalanche multiplication can be performed. In the case where a signal mode other than the mode in which the signal STOP is at an L level and the signal ENB is at an L level appears, the quenching element 202 is turned off, and the APD 201 enters a non-standby state in which avalanche multiplication cannot be performed. The waveform shaping unit 210 includes an inverter, and the counter 211 is provided with an 11-bit flip-flop. For example, when the count value of the counter 211 becomes 2047, which is the maximum count value, the signal STOP changes from the L level to the H level, and the transistor as the quenching element 202 is turned off. Therefore, even when the signal ENB becomes an L level, the quenching element 202 continues to be turned off, and the non-standby state of the APD 201 is maintained.

[0066] Fig. 7A The right figure shows Fig. 6A The voltage at VC and Fig. 6A The signal from VO is shown. In the case of low illumination, since the time interval of photon incidence on APD 201 is long, sufficient time can be ensured after the voltage at VC becomes low after photon incidence until quenching element 202 is charged and the voltage becomes high. Fig. 7AThe right figure shows a case where three pulses corresponding to three photons can be counted. On the other hand, in the case of high illumination, since the time interval of photons incident on APD 201 is short, the voltage at VC is maintained at a low level and does not return to a high level, and it takes time for the voltage to exceed the judgment threshold from a low level. Fig. 7A The right figure of shows a case where only three pulses are counted despite the fact that about 20 photons are incident on the APD 201. That is, the photon counting fails.

[0067] Therefore, in the Fig. 6A In the passive charging mode shown, Fig. 7A As shown in FIG. 1 , the count value may be the same for low illumination and high illumination. In the case where photons are more frequently incident on the APD 201, the voltage at VC is maintained at a low state, and the voltage does not exceed the judgment threshold from a low level. In this case, the voltage at VO is maintained at a high level, and no signal is generated. That is, in the case of Fig. 6A In the passive charging mode shown, an appropriate count value cannot be obtained under high illumination conditions, and the dynamic range becomes narrow.

[0068] Clock charging circuit

[0069] Figure 6B Logic circuit 221 (OR circuit) is configured so that signal Pctrl and signal STOP are input to the input of logic circuit 221. Signal Pctrl is a clock signal having Nc number of pulse signals in exposure period T.

[0070] exist Figure 6B In the case where the signal STOP is at an L level and the signal Pctrl is at an L level, the transistor 202 is turned on and charged. In the case where a signal pattern other than the pattern in which the signal STOP is at an L level and the signal Pctrl is at an L level occurs, the transistor 202 is turned off. That is, the transistor functions as a charging circuit that switches between a first state in which the APD 201 is connected to a power source and a second state in which the APD 201 is not connected to a power source and charges the APD 201 whenever the APD 201 is connected to a power source. The signal Pctrl is a control signal that controls the charging circuit.

[0071] For example, when the signal Pctrl is at an H level, the transistor is turned off after charging and is therefore in a standby state in which avalanche multiplication can be performed. In other words, the circuit (transistor) is arranged between the APD 201 and the power supply (voltage VH), and controls are performed to switch between a first state in which the APD 201 is connected to the power supply and a second state in which the APD 201 is not connected to the power supply. In this case, the first state is a charging state and the second state is a standby state.

[0072] The logic circuit 222 (AND circuit in which the logic of one input is inverted) is arranged between the cathode of the APD 201 and the counter 211. The logic circuit 222 is configured so that the signal Pctrl and the output from VC can be input to the logic circuit 222. The output from VC is inverted and input to the logic circuit 222, and the voltage at VC changes from the H level to the L level due to the incidence of photons. In this case, when the signal Pctrl is at the H level, the output of the logic circuit 222 becomes the H level. Thereafter, in the case where the signal Pctrl changes from the H level to the L level, the output of the logic circuit 222 becomes the L level, and an output signal is generated.

[0073] Figure 7B The figure on the right shows Figure 6B Shown are the signals Pctrl, the voltage at VC and the signal from VO.

[0074] When the illumination is low and the signal Pctrl is at the H level (standby state), when a photon is incident on the APD 201, the voltage at VC drops and changes to the L level. Since the signal Pctrl is at the H level, the output from VO changes to the H level. Thereafter, when the signal Pctrl changes to the L level, the output from VO changes from the H level to the L level, and a signal is generated.

[0075] On the other hand, when the illuminance is high and the signal Pctrl is at the H level (standby state), even when photons are frequently incident on the APD 201, the voltage at VO that has become the H level is maintained at the H level as long as the signal Pctrl is at the H level. Next, the signal Pctrl changes from the H level to the L level, and thus the output from VO changes from the H level to the L level, and a single signal is generated.

[0076] In the clock charging circuit, the case where the count value of low illumination is the same as the count value of high illumination or the case where no signal is generated in the case of passive charging is eliminated. That is, the clock charging circuit has the advantage that the count value of low illumination does not exceed the count value of high illumination. The clock charging circuit is configured so that the signal output from the logic circuit 222 is output from the counter 211 to the outside via the memory 240. It can be appropriately set whether the pulse output from the logic circuit 222 is counted when the pulse falls or when it rises immediately before the pulse falls. According to this configuration, compared with the passive quenching circuit, an appropriate count value can be obtained when the illumination is high and the dynamic range can be increased.

[0077] Clock charging circuit with inactive period

[0078] Fig. 8A and Figure 8B is a diagram showing a method for generating a clock signal Pctrl to be used for clock charge driving. Fig. 8A The clock signal Pctrl generated by inverting the clock signal PCLK is shown. Fig. 8A The clock signal Pctrl is shown to be used during a time period where the inactive period is 0%. Figure 8B The clock Pctrl generated by inverting the clock signal PCLK by masking every other signal of the clock signal PCLK using a mask circuit is shown. Figure 8B The clock signal Pctrl shown in is used in the time period when the inactive period is 50%. Although not shown, the signal used in the period when the inactive period is 75% is obtained by masking the second to fourth signals of the clock signal PCLK without masking the first signal of the clock signal PCLK.

[0079] Figure 8C is a timing diagram assuming that the inactive period is 50%. Assume that among signals 801 to 804, signals 802 and 804 are masked and the inactive period is set to 50%. However, the time period from the fall of signal 811 generated from signal 801 to the fall of signal 813 generated from signal 803 is the standby period after charging. Therefore, in Figure 8C In the example, avalanche multiplication occurs due to photons that have been incident on the APD 201 during the masking period, the voltage at VC decreases, the voltage at VO increases, and the counter counts 1. That is, without taking measures, the light that has been incident on the APD 201 during the time period in which masking has been performed is counted, and the inactive period cannot be appropriately set.

[0080] Fig. 9 is a block diagram of a photoelectric conversion device configured to solve the above-mentioned problems.

[0081] The photoelectric conversion device includes the APD 201 and a count pulse generating circuit 310 that generates a pulse signal according to a signal output from the APD 201. The counter 311 counts the pulse signal output from the count pulse generating circuit 310.

[0082] The mask signal generating circuit 340 generates a plurality of types of signal patterns for masking the clock signal PCLK. The mask signal selecting circuit 350 selects which of these signals will be output to the next stage.

[0083] The count control circuit 330 generates a signal to control whether a signal to be counted by the counter 311 is output from the count pulse generating circuit 310 .

[0084] Fig.10 It is used to explain in detail Fig. 9 Figure 1. Block diagram of a . Fig.11 is with Fig.10 The timing diagram corresponding to the block diagram.

[0085] Fig.10 The mask signal generating circuit 340 shown generates P1, P2, and P3. P1 is a mask signal corresponding to an inactive period of 0%. P2 is a mask signal corresponding to an inactive period of 50%. P3 is a mask signal corresponding to an inactive period of 75%. The mask signal generating circuit 340 is provided in the signal processing unit 103, and generates a mask signal using a signal input from outside the signal processing unit 103. Specifically, the mask signal generating circuit 340 can generate a mask signal using a frequency dividing circuit.

[0086] The mask signal selection circuit 350 selects which signal to output from P1 to P3. The signal output from the mask signal selection circuit 350 is represented as Ppmsk. For convenience, P1 to P3 can be called mask signals. Ppmsk, which is a signal based on P1 to P3, can be called a mask signal. In addition, as described below, P1 is a mask signal maintained at an L level and does not have the function of masking the PCLK signal. Therefore, P2, P3 and the Ppmsk corresponding to P2 and P3 can be called mask signals.

[0087] At the start time of the exposure period, the signal P1 is output from the mask signal selection circuit 350. Next, in the case where the count value of the counter 311 reaches the threshold value N1, the mask signal selection circuit 350 selects the signal P2 and outputs the signal P2. Then, in the case where the count value of the counter 311 reaches the threshold value N2, the mask signal selection circuit 350 selects the signal P3 and outputs the signal P3.

[0088] The photodiode control circuit 320 includes a PMOS transistor for charging and a control signal generating circuit 322 for outputting Pctrl to be input to the gate of the PMOS transistor. The control signal generating circuit 322 includes a NAND circuit whose logic of one input is inverted. The control signal generating circuit 322 is configured so that PCLK and Ppmsk are input to the control signal generating circuit 322.

[0089] That is, the control signal generating circuit 322 is a control circuit that generates control signals having different cycles based on the clock signal and the mask signal.

[0090] 0% during inactive periods

[0091] refer to Fig.11 , in the time period where the inactive period is 0%, the mask signal P1 at the L level is input as Ppmsk to the control signal generating circuit 322. At time t1, PCLK is at the L level, and thus Pctrl output from the control signal generating circuit 322 is at the L level. Therefore, the PMOS transistor 321 as a charging circuit is in the on state, and a reverse bias is applied to the APD 201, so that avalanche multiplication is possible in the APD 201. Next, when PCLK transitions from the H level to the L level, Pctrl transitions to the H level, and the PMOS transistor 321 turns off and enters the standby state.

[0092] When a photon is incident on the APD 201 in the standby state at time t11, the voltage at VC drops from the H level to the L level, and a signal at the L level is input to the count pulse generating circuit 310. Therefore, the output VO from the count pulse generating circuit 310 changes from the H level to the L level. The counter 311 starts counting when the output VO rises. Although a photon is incident on the APD 201 again after time t11, the PMOS transistor 321 remains off, VC is already at the L level, and therefore avalanche multiplication does not occur and VC is maintained at the L level.

[0093] When PCLK transitions from L level to H level at time t12, Pctrl transitions from H level to L level, the PMOS transistor 321 is turned on, the APD 201 is charged, and VC transitions from L level to H level. Since VC transitions from L level to H level, the output from the count pulse generating circuit 310 transitions from H level to L level.

[0094] The selection circuit 331 is configured to output the output from the logic circuit 334 as Pcmsk when the inactive period is 0%. Pcmsk is a signal to be input to the count pulse generation circuit 310, and controls a signal to be output from the count pulse generation circuit 310 to the counter 311. Therefore, it can be said that Pcmsk is called a count control signal and the count control circuit 330 generates the count control signal.

[0095] In the case where the inactive period is 0%, Ppmsk is maintained at the L level, so at both the rise and fall of PCLK, the output of the D latch 332 and the output of the D latch 333 become the L level, and the output from the logic circuit 334 becomes the L level. As a result, Pcmsk is maintained at the L level, and a signal obtained by inverting the output signal from VC is output from the count pulse generating circuit 310. As described later about time t27, in the case where Ppmsk is at the H level, even when VC transitions from the H level to the L level, the output VO from the count pulse generating circuit 310 is maintained at the L level, so the counter 311 does not count.

[0096] 50% during inactive period

[0097] When the count value of the counter 311 reaches the threshold value N1, the mask signal selection circuit 350 selects the signal P2 and outputs the signal P2. Specifically, at time t2, Ppmsk changes from the signal corresponding to P1 to the signal corresponding to P2. P2 is different from P1 and is a signal that is a combination of an L level and an H level. The time period in which P2 is at the H level is the mask period. Even in the case where PCLK is at the H level during the mask period, Pctrl is set so that the PMOS transistor 321 is not turned on. The inactive period 50% is a time period in which the mask signal P2 is set to mask 50% of the clock signal that forms PCLK. Fig.11 In the embodiment, P2 is set to the H level to mask every other signal included in the clock signal forming PCLK.

[0098] When P2 transitions from H level to L level at time t21, Ppmsk transitions from H level to L level. When PCLK transitions from L level to H level at time t22, Pctrl output from the control signal generating circuit 322 transitions from H level to L level. Thus, the APD 201 is charged.

[0099] Although PCLK changes from the L level to the H level at time t23, P2 and Ppmsk are at the H level, so Pctrl is maintained at the H level, the APD 201 is not charged, and VC is maintained at the L level. That is, P2 and Ppmsk are used as mask signals, and the cycle of Pctrl increases. Specifically, in the case where the cycle of Pctrl in the inactive period 0% is the first cycle, the cycle of Pctrl in the inactive period 50% is the second cycle longer than the first cycle.

[0100] On the other hand, the H level is input to the D latches 332 and 333, and therefore, at the time of PCLK rising and falling, a signal at the H level is output to the logic circuit 334, and the output signal Pcmsk from the selection circuit 331 transitions from the L level to the H level. Since VC is at the L level at time t23, VO transitions from the H level to the L level.

[0101] At time t24, Ppmsk transitions from the H level to the L level, and the L level is input to the D latches 332 and 333. Thereafter, when PCLK rises, a signal at the L level is output from the D latch 332 to the logic circuit 334, and when PCLK falls, a signal at the L level is output from the D latch 333 to the logic circuit 334. Therefore, Pcmsk transitions from the H level to the L level at time t25.

[0102] At time t26 , similarly to the transition at time t23 , Pcmsk transitions from the L level to the H level.

[0103] When a photon is incident on the APD 201 at time t27, VC transitions to the L level, but Pcmsk is at the H level, so VO is maintained at the L level, and the counter 311 does not count. Even when a photon is incident on the APD 201 in the masking period, the count control circuit 330 controls so that a signal is not output from the count pulse generating circuit. Specifically, the count control circuit 330 generates the signal Pcmsk based on the masking signal (P1, P2, and P3) for generating Pctrl of the PMOS transistor 321 to be input to the photodiode control circuit 320. In addition, the count control circuit 330 controls the output signal from the count pulse generating circuit 310 based on the signal Pcmsk. Therefore, it is possible to suppress the counting of light that has been incident on the APD 201 during the time period in which the signal Pctrl has been masked, and the inactive period can be appropriately set.

[0104] Inactive period 75%

[0105] When the count value of the counter 311 reaches the threshold value N2, the mask signal selection circuit 350 selects the signal P3 and outputs the signal P3. Specifically, at time t3, Ppmsk changes from the signal corresponding to P2 to the signal corresponding to P3. The period of the mask signal P3 is longer than the period of the mask signal P2. That is, the time interval of the pulse generating the mask signal P3 is longer than the time interval of the pulse generating the mask signal P2. As a result, the period of Pctrl in the inactive period 75% is longer than the period of Pctrl in the inactive period 50%. Specifically, in the case where the period of Pctrl in the inactive period 50% is the second period, the period of Pctrl in the inactive period 75% is the third period longer than the second period.

[0106] Similar to P2, the period during which P3 is at the H level is a masking period. Even in the case where PCLK is at the H level during the masking period, Pctrl is set so that the PMOS transistor 321 is not turned on. The inactive period 75% is a period during which the masking signal P3 is set to mask 75% of the clock signal forming PCLK. Fig.11 In the embodiment, the signal Pctrl is generated by masking three consecutive signals included in the clock signal forming PCLK and using a single signal included in the clock signal forming PCLK and following the three pulse signals.

[0107] At time t31, similarly to the transitions at time t23 and time t26, Pcmsk transitions from the L level to the H level, and the mask period starts. At time t32, similarly to the transition at time t25, Pcmsk transitions from the H level to the L level, and the mask period ends.

[0108] Correction

[0109] Fig.12 311 is a graph showing the relationship between the number of photons that have been incident on the APD and the number of photons POUT counted by the counter 311. The inclination of the number POUT relative to the number of photons that have been incident on the APD is the inclination a until the count number POUT reaches the threshold value N1. Ideally, when the counter 311 counts all the photons that have been incident on the APD without failing to count photons, the inclination a is 1 and the number of photons PN1 coincides with the threshold value N1.

[0110] When the count number POUT reaches the threshold value N1, the inactive period is switched to the inactive period 50%. Therefore, the inclination of the count number POUT relative to the number of photons that have been incident on the APD becomes a / 2. When the count number POUT reaches the threshold value N2, the inactive period is switched to the inactive period 75%. Therefore, the inclination of the count number POUT relative to the number of photons that have been incident on the APD becomes a / 4.

[0111] When the count value Nx exceeds the threshold value N1, the count number POUT is corrected for the count value in the range where the inclination of the count number POUT relative to the number of photons that have been incident on the APD is different from the inclination a, so that the inclination of the count number POUT relative to the number of photons that have been incident on the APD is consistent with the inclination a. The correction unit for performing the correction may be a signal processing circuit provided in the photoelectric conversion device, or may be a signal processing circuit provided outside the photoelectric conversion device.

[0112] Effect

[0113] With the configuration according to the present embodiment, the dynamic range can be increased and the number of times avalanche multiplication is performed in a case where light with high illuminance is incident on the APD can be reduced, and thus power consumption can be reduced.

[0114] With the configuration according to the present embodiment, the count of light that has been incident on the APD during the period in which the signal Pctrl has been masked can be suppressed, and the inactive period can be appropriately set.

[0115] Second embodiment

[0116] This embodiment is different from the first embodiment described above in that, in the time period when the inactive period is 0%, the selection circuit 331 outputs PCLK as Pcmsk instead of outputting from the logic circuit 334. Other configurations in this embodiment are the same as or similar to those described in the first embodiment.

[0117] In the first embodiment described Fig.11 In the inactive period 0% from time t1 to time t2, Pcmsk is maintained at the L level. Fig.13 In the inactive period 0% from time t1 to time t2, PCLK is Pcmsk. At time t12, Pctrl becomes L level, and the PMOS transistor 321 is turned on and becomes charged. However, as Fig.13As shown, when photons are incident on the APD during charging, VC may not become the H level. In particular, this phenomenon occurs when the illumination is high. In the case where VC does not become the H level, VO does not become the L level, and the photon counting may fail. On the other hand, according to the present embodiment, the count pulse generating circuit 310 is configured so that the signal Pcmsk is input to the count pulse generating circuit 310. Therefore, when Pcmsk changes from the L level to the H level, VO changes from the H level to the L level even when VC is at the L level. Therefore, the present embodiment has the advantage that counting failures can be reduced.

[0118] With the configuration according to the present embodiment, the dynamic range can be increased and the number of times avalanche multiplication is performed in a case where light with high illuminance is incident on the APD can be reduced, and thus power consumption can be reduced.

[0119] Furthermore, with the configuration according to the present embodiment, the count of light that has been incident on the APD during the period in which the signal Pctrl has been masked can be suppressed, and the inactive period can be appropriately set.

[0120] Furthermore, with the configuration according to the present embodiment, counting failures in a case where the illuminance is high can be reduced.

[0121] Third embodiment

[0122] The present embodiment is different from the first embodiment and the second embodiment in that the mask signal generating circuit 340 is not provided in each signal processing unit 103 included in each pixel, but the mask signal generating circuit 340 is provided commonly for a plurality of pixel circuits 410. Other configurations in the present embodiment are the same as or similar to those described in the above embodiments.

[0123] Fig.14 An example is shown in which the mask signal generating circuit 340 is commonly provided for the pixel circuits 410 arranged in each row. In the case where the mask signal generating circuit 340 is arranged for each pixel circuit 410, the circuit size increases, and the size of the space for arranging other circuits such as counters decreases. According to the present embodiment, the increase in the circuit size can be suppressed.

[0124] With the configuration according to the present embodiment, the dynamic range can be increased and the number of times avalanche multiplication is performed in a case where light with high illuminance is incident on the APD can be reduced, and thus power consumption can be reduced.

[0125] Furthermore, with the configuration according to the present embodiment, the count of light that has been incident on the APD during the period in which the signal Pctrl has been masked can be suppressed, and the inactive period can be appropriately set.

[0126] Furthermore, with the configuration according to the present embodiment, an increase in circuit size can be suppressed.

[0127] Modifications

[0128] In the embodiments described herein, the inactive period may not be gradually increased to the exposure period. For example, in the first mode, a drive using only the masking signal P1 and not setting the inactive period may be adopted. In the second mode, a drive using the masking signals P1 to P3 and setting the inactive period may be adopted. Specifically, in the first exposure period, the drive in the first mode may be performed. In the second exposure period after the first exposure period, the drive in the second mode may be performed.

[0129] In the above-described embodiment, three types of mask signals are used, but any types of mask signals may be used as long as the number of types of mask signals is two or more.

[0130] Fourth embodiment

[0131] Will refer to Fig.15 A photoelectric conversion system according to the present embodiment is described. Fig.15 is a block diagram showing a schematic configuration of a photoelectric conversion system according to the present embodiment.

[0132] The above-mentioned photoelectric conversion device in each embodiment can be used for various photoelectric conversion systems. Examples of photoelectric conversion systems that can use the above-mentioned photoelectric conversion device in each embodiment include digital still cameras, digital video cameras, surveillance cameras, copiers, fax machines, mobile phones, vehicle-mounted cameras, and observation satellites.

[0133] Furthermore, examples of the photoelectric conversion system include a camera module having an optical system such as a lens and an image pickup device. Fig.15 A block diagram of a digital still camera is shown as one of examples of the photoelectric conversion system.

[0134] Fig.15 The photoelectric conversion system shown includes an image pickup device 1004 as an example of a photoelectric conversion device, and a lens 1002 that forms an optical image of an object on the image pickup device 1004. The photoelectric conversion system also includes an aperture 1003 that makes the amount of light that has passed through the lens 1002 variable, and a baffle 1001 that protects the lens 1002. The lens 1002 and the aperture 1003 are an optical system that focuses light on the image pickup device 1004. The image pickup device 1004 is a photoelectric conversion device according to any of the above-described embodiments, and converts an optical image formed by the lens 1002 into an electrical signal.

[0135] The photoelectric conversion system further includes a processing unit 1007, which is an image generation unit that generates an image by processing a signal output by the image pickup device 1004. The processing unit 1007 performs various types of correction and compression as required, and performs an operation of outputting image data. The processing unit 1007 may be formed in a semiconductor layer in which the image pickup device 1004 is arranged, or may be formed in a semiconductor layer different from the semiconductor layer in which the image pickup device 1004 is arranged. The image pickup device 1004 and the processing unit 1007 may be formed in the same semiconductor layer.

[0136] The photoelectric conversion system further includes a memory unit 1010 for temporarily storing image data and an external interface unit (external I / F unit) 1013 for communicating with an external computer or the like. The photoelectric conversion system further includes a recording medium 1012 such as a semiconductor memory or the like for recording or reading image data, and a recording medium control interface unit (recording medium control I / F unit) 1011 for recording data in or reading data from the recording medium 1012. The recording medium 1012 may be built in the photoelectric conversion system or may be detachably arranged in the photoelectric conversion system.

[0137] The photoelectric conversion system also includes an overall control / computation unit 1009 that controls various types of calculations and the entire digital still camera, and a timing generation unit 1008 that outputs various types of timing signals to the image pickup device 1004 and the processing unit 1007. The timing signal and the like can be input to the photoelectric conversion system from the outside. It is sufficient that the photoelectric conversion system includes at least the image pickup device 1004 and the processing unit 1007 that processes the signal output from the image pickup device 1004.

[0138] The imaging device 1004 outputs an imaging signal to the processing unit 1007. The processing unit 1007 performs predetermined signal processing on the imaging signal output from the imaging device 1004, and outputs image data. The processing unit 1007 generates an image using the imaging signal.

[0139] According to the present embodiment, a photoelectric conversion system including the photoelectric conversion apparatus (imaging apparatus) according to any of the above-described embodiments can be realized.

[0140] Fifth embodiment

[0141] Will refer to Fig.16A and Fig. 16B A photoelectric conversion system and a moving body according to the present embodiment are described. Fig.16A and Fig. 16B is a diagram showing the configuration of a photoelectric conversion system and a moving body according to the present embodiment.

[0142] Fig.16A An example of a photoelectric conversion system for a vehicle-mounted camera is shown. The photoelectric conversion system 2300 includes a camera 2310. The camera 2310 is a photoelectric conversion device according to any of the above-described embodiments. The photoelectric conversion system 2300 includes an image processing unit 2312 that performs image processing on a plurality of image data acquired by the camera 2310. The photoelectric conversion system 2300 includes a parallax acquisition unit 2314 that calculates parallax (phase difference between parallax images) based on the image data acquired by the camera 2310. The photoelectric conversion system 2300 also includes a distance acquisition unit 2316 that calculates the distance to a target object based on the calculated parallax, and a collision judgment unit 2318 that judges whether there is a possibility of a collision. The parallax acquisition unit 2314 and the distance acquisition unit 2316 are examples of a distance information acquisition unit that obtains distance information related to the distance to the target object. That is, the distance information is information related to parallax, defocus amount, and distance to the target object, etc. The collision judgment unit 2318 can use any of these information items to judge whether there is a possibility of a collision. The distance information acquisition unit can be implemented by specially designed hardware, or can be implemented by a software module. Alternatively, the distance information acquisition unit can be implemented by a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC), etc., or can be implemented by a combination of an FPGA and an ASIC.

[0143] The photoelectric conversion system 2300 is connected to the vehicle information acquisition device 2320, and can acquire vehicle information related to the vehicle speed, yaw rate, and steering angle of the vehicle. The photoelectric conversion system 2300 is connected to the control ECU 2330, which is a control device (control unit) that outputs a control signal to generate a braking force for the vehicle based on the judgment result of the collision judgment unit 2318. The photoelectric conversion system 2300 is connected to the alarm device 2340 that issues an alarm to the driver based on the judgment result of the collision judgment unit 2318. For example, as a result of the judgment of the collision judgment unit 2318, in the case where the possibility of a collision is high, the control ECU 2330 controls the vehicle to avoid the collision or reduce the damage by applying the brake, releasing the accelerator, or suppressing the engine output.

[0144] The alarm device 2340 warns the user by, for example, emitting an alarm sound, displaying an alarm message on a screen of a car navigation system, or vibrating a seat belt or a steering wheel.

[0145] In the present embodiment, the photoelectric conversion system 2300 captures an image of the surroundings of the vehicle, for example, the front side or the rear side of the vehicle. Fig. 16BThe photoelectric conversion system 2300 that captures an image of the front side of the vehicle (image sensing area 2350) is shown. The vehicle information acquisition device 2320 transmits an instruction to the photoelectric conversion system 2300 or the imaging device 2310. With this configuration, the accuracy of focusing can be improved.

[0146] An example of performing control to avoid collision with other vehicles is described above. The photoelectric conversion system can be used for control of an autonomous vehicle by following other vehicles and control of an autonomous vehicle to avoid lane deviation, etc. The photoelectric conversion system is not limited to a system for a vehicle such as an own vehicle, and can be used for a mobile body (mobile device) such as a ship, an airplane, or an industrial robot. The photoelectric conversion system is not limited to a system for a mobile body, and can be used for a device that widely uses object recognition and is an intelligent transportation system (ITS), etc.

[0147] Sixth embodiment

[0148] Will refer to Fig.17 A photoelectric conversion system according to the present embodiment is described. Fig.17 : is a block diagram showing an example of the configuration of a distance image sensor as a photoelectric conversion system according to the present embodiment.

[0149] like Fig.17 As shown, the distance image sensor 401 includes an optical system 402, a photoelectric conversion device 403, an image processing circuit 404, a monitor 405, and a memory 406. The distance image sensor 401 acquires a distance image corresponding to the distance to the object by receiving light (modulated light and pulsed light) emitted from a light source device 411 to the object and reflected from the surface of the object.

[0150] The optical system 402 includes one or more lenses. The optical system 402 guides image light (incident light) from an object to the photoelectric conversion device 403 and forms an image on a light receiving surface (sensor portion) of the photoelectric conversion device 403 .

[0151] The photoelectric conversion device described in any of the embodiments is used as the photoelectric conversion device 403 . A distance signal indicating a distance calculated from a light reception signal output from the photoelectric conversion device 403 is supplied to the image processing circuit 404 .

[0152] The image processing circuit 404 performs image processing to generate a distance image based on the distance signal supplied from the photoelectric conversion device 403. The distance image (image data) obtained by the image processing is supplied to the monitor 405 and displayed on the monitor 405, and is supplied to the memory 406 and stored (recorded) in the memory 406.

[0153] Since the above-described photoelectric conversion device is used in the distance image sensor 401 configured as described above, the distance image sensor 401 can acquire a more accurate distance image as the characteristics of the pixels are improved.

[0154] Seventh embodiment

[0155] Will refer to Fig.18 A photoelectric conversion system according to the present embodiment is described. Fig.18 : is a diagram showing an example of a schematic configuration of an endoscopic surgery system as a photoelectric conversion system according to the present embodiment.

[0156] Fig.18 FIG. 1 shows a state where an operator (doctor) 1131 uses an endoscopic surgery system 1103 to perform surgery on a patient 1132 on a bed 1133. Fig.18 As shown, the endoscopic surgery system 1103 includes an endoscope 1100, a surgical tool 1110, and a cart 1134 on which various devices for endoscopic surgery are installed.

[0157] The endoscope 1100 includes a lens barrel 1101 and a camera head 1102 connected to a base end of the lens barrel 1101. The lens barrel 1101 includes a portion having a predetermined length from a front end of the lens barrel 1101 and is inserted into a body cavity of a patient 1132. Fig.18 In the illustrated example, an endoscope 1100 having a rigid lens barrel 1101 and configured as a so-called rigid scope is illustrated. However, the endoscope 1100 may have a flexible lens barrel and may be configured as a flexible scope.

[0158] The front end of the lens barrel 1101 has an opening, wherein the objective lens is embedded in the opening. The light source device 1203 is connected to the endoscope 1100. The light generated by the light source device 1203 is arranged to be guided to the front end of the lens barrel 1101 by a light guide extending in the lens barrel 1101 and passes through the objective lens, and illuminates the observation object in the body cavity of the patient 1132 with light. The endoscope 1100 can be a forward-looking endoscope, an oblique-looking endoscope, or a side-looking endoscope.

[0159] An optical system and a photoelectric conversion device are arranged in the camera head 1102. Light reflected from the observation object (observation light) is focused onto the photoelectric conversion device through the optical system. The photoelectric conversion device performs photoelectric conversion on the observation light to generate an electrical signal corresponding to the observation light, that is, an image signal corresponding to the observation image. As the photoelectric conversion device, the above-mentioned photoelectric conversion device in any one of the embodiments can be used. The image signal is transmitted to the camera control unit (CCU) 1135 as raw data.

[0160] The CCU 1135 includes a central processing unit (CPU) or a graphics processing unit (GPU), etc., and comprehensively controls the operations of the endoscope 1100 and the display 1136. The CCU 1135 receives an image signal from the camera 1102, and performs various types of image processing such as development processing (demosaic processing) on ​​the image signal to display an image based on the image signal.

[0161] The display 1136 is controlled by the CCU 1135 to display an image based on an image signal image-processed by the CCU 1135 .

[0162] The light source device 1203 includes a light source such as a light emitting diode (LED) and supplies irradiation light for capturing an image of a site to be surgically operated or the like to the endoscope 1100 .

[0163] The input device 1137 is an input interface of the endoscopic surgery system 1103. The user can input various types of information and instructions to the endoscopic surgery system 1103 via the input device 1137.

[0164] The treatment tool control device 1138 controls the driving of the energy treatment tool 1112 to perform cauterization and cutting of tissues, blood vessel sealing, and the like.

[0165] The light source device 1203 that supplies irradiation light for capturing an image of a site to be operated on to the endoscope 1100 may include, for example, an LED, a laser light source, or a white light source formed by combining an LED and a laser light source. In the case where the light source device 1203 includes a white light source formed by combining RGB laser light sources, the intensity of the output light of each color (each wavelength) and the timing of the output light can be accurately controlled, and therefore, the white balance of the captured image can be adjusted in the light source device 1203. In this case, by irradiating the observed object with laser light from the RGB laser light sources in a time-division manner, and controlling the drive of the image sensor element of the camera 1102 in synchronization with the timing of the irradiation, an image corresponding to RGB can be formed in a time-division manner. According to this method, a color image can be obtained without providing a color filter in the image sensor element.

[0166] The driving of the light source device 1203 may be controlled so that the intensity of the light to be output is changed at predetermined time intervals. The driving of the image sensor element of the camera 1102 may be controlled in synchronization with the timing of changing the light intensity so that the image sensor element acquires images in a time-division manner, and the camera 1102 may generate an image having a high dynamic range and having no black underexposed picture images and overexposed white portions by synthesizing the acquired images.

[0167] The light source device 1203 may be configured to be able to supply light in a predetermined wavelength range for observation using special light. For example, in observation using special light, the wavelength dependence of light absorption in body tissue is used. Specifically, a high-contrast image of a predetermined tissue such as a blood vessel on a mucosal surface layer is captured by irradiating the predetermined tissue with light having a narrower range than the irradiation light (i.e., white light) used in normal observation.

[0168] Alternatively, in observation using special light, fluorescence observation may be performed to acquire an image using fluorescence generated by irradiation with excitation light. In fluorescence observation, body tissue is irradiated with excitation light and fluorescence from the body tissue is observed, or a fluorescence image is obtained by locally injecting an agent such as indocyanine green (ICG) into the body tissue and irradiating the body tissue with excitation light corresponding to the fluorescence wavelength of the agent. The light source device 1203 may be configured to be capable of supplying light in a narrow band and / or excitation light for such observation using special light.

[0169] Eighth embodiment

[0170] Will refer to Fig.19A and Fig.19B A photoelectric conversion system according to the present embodiment is described. Fig.19A is a diagram showing a configuration example of glasses 1600 (smart glasses) as a photoelectric conversion system.

[0171] The glasses 1600 include a photoelectric conversion device 1602. The photoelectric conversion device 1602 is the photoelectric conversion device described in the first to third embodiments. A display including a light emitting device such as an OLED or an LED is arranged on the rear side of the lens 1601. The glasses 1600 may include one photoelectric conversion device 1602 or a plurality of photoelectric conversion devices 1602. The glasses 1600 may include a plurality of types of photoelectric conversion devices. The configuration position of the photoelectric conversion device 1602 is not limited to Fig.19A Position shown.

[0172] The glasses 1600 further include a control device 1603. The control device 1603 serves as a power source for supplying power to the photoelectric conversion device 1602 and the above-mentioned display. The control device 1603 controls the operation of the photoelectric conversion device 1602 and the display. An optical system for focusing light onto the photoelectric conversion device 1602 is formed on the lens 1601.

[0173] Fig.19BGlasses 1610 (smart glasses) according to an application example are shown. Glasses 1610 include a control device 1612. In the control device 1612, a photoelectric conversion device corresponding to the photoelectric conversion device 1602 and a display are included. An optical system for projecting light emitted from the display and the photoelectric conversion device in the control device 1612 is formed in a lens 1611, and an image is projected on the lens 1611. The control device 1612 serves as a power supply for supplying power to the photoelectric conversion device and the display, and controls the operation of the photoelectric conversion device and the display. The control device may include a sight line detector for detecting the sight line of a person wearing the glasses 1610. Infrared radiation can be used to detect the sight line. An infrared light emitter emits infrared light to the eyeball of a user who is looking at a displayed image. A camera unit having a light receiving element detects light reflected from the eyeball in the emitted infrared light, and obtains a captured image of the eyeball. Glasses 1610 include a reducing unit for reducing light emitted from the infrared light emitter toward the display unit in a plan view. Therefore, glasses 1610 reduce the degradation of image quality.

[0174] The sight line detector detects the user's sight line directed to the display image from the captured eyeball image obtained by imaging using infrared light. Any known method can be applied to detect the sight line using the captured eyeball image. As an example, a method for detecting the sight line based on the Purkinje image caused by the reflection of the irradiated light by the cornea can be used.

[0175] Specifically, the line of sight detection based on the pupil cornea reflection method is performed. The line of sight of the user is detected by using the pupil cornea reflection method to calculate the line of sight vector indicating the orientation (rotation angle) of each eyeball based on the pupil image and Purkinje image included in the captured eyeball image.

[0176] The display according to the present embodiment may include a photoelectric conversion device having a light emitting element, and control an image displayed on the display based on information received from the photoelectric conversion device and indicating a line of sight of a user.

[0177] Specifically, the display determines a first field of view area that the user is looking at and a second field of view area other than the first field of view area based on the line of sight information. The first field of view area and the second field of view area may be determined by a control device for the display, or the display may receive the first field of view area and the second field of view area determined by an external control device. In the color gamut of the display, the display resolution of the first field of view area may be controlled to be higher than the display resolution of the second field of view area. That is, the resolution of the second field of view area may be set to be lower than the resolution of the first field of view area.

[0178] The color gamut includes a first color gamut and a second color gamut different from the first color gamut. A color gamut with a higher priority in the first color gamut and the second color gamut can be determined based on line of sight information. The first field of view area and the second field of view area can be determined by a control device for the display, or the display can receive the first field of view area and the second field of view area determined by an external control device. The resolution of the area with a higher priority can be controlled to be higher than the resolution of the area other than the area with a higher priority. That is, the resolution of the area with a relatively low priority can be reduced.

[0179] AI can be used to determine the first field of view area and the area with higher priority. AI can be a model configured to use the image of the eyeball and the direction the eyeball is actually looking at as teacher data to estimate the angle of sight and the distance to the object to which the sight is directed based on the image of the eyeball. The AI ​​program can be included in a display, a photoelectric conversion device, or an external device. In the case where the AI ​​program is included in an external device, the AI ​​program is transmitted to the display via communication.

[0180] In the case of display and control based on visual recognition and detection, smart glasses that also include a photoelectric conversion device that captures images outside the smart glasses can be applied to display and control. The smart glasses can display information related to the captured images outside the smart glasses in real time.

[0181] The above-mentioned embodiments may be appropriately modified without departing from the technical concept of the present disclosure. In addition, examples in which the configuration described in any one embodiment is added to one or more other embodiments and examples in which the configuration described in any one embodiment is replaced with the configuration described in one or more other embodiments are included in the embodiments of the present disclosure.

[0182] The disclosure of this embodiment includes the following configurations and methods.

[0183] Configuration 1

[0184] A device comprising:

[0185] Photodiodes for avalanche multiplication;

[0186] a photodiode control circuit comprising a charging circuit for switching between a first state in which the photodiode is connected to a power source and a second state in which the photodiode is not connected to the power source, and a control signal generating circuit for generating a control signal to control the charging circuit;

[0187] a counting pulse generating circuit, configured to generate a pulse signal according to a signal output from the photodiode;

[0188] a counter for counting the pulse signal output from the counting pulse generating circuit,

[0189] wherein the control signal generating circuit is configured to output a control signal having a first period and a control signal having a second period longer than the first period based on a clock signal and a mask signal, and

[0190] In a case where the count value of the counter reaches a threshold value, the signal to be output from the control signal generating circuit is switched from the control signal having the first period to the control signal having the second period.

[0191] Configuration 2

[0192] The device according to configuration 1, further comprising a count control circuit for generating a count control signal to be input to the count pulse generating circuit,

[0193] The counting control circuit generates the counting control signal based on the clock signal and the mask signal.

[0194] Configuration 3

[0195] The device according to configuration 2, wherein:

[0196] The count pulse generating circuit outputs a signal corresponding to the signal output from the photodiode to the counter when the count control signal is at a first level, and

[0197] The count pulse generating circuit does not output a signal corresponding to the signal output from the photodiode to the counter when the count control signal is at a second level different from the first level.

[0198] Configuration 4

[0199] The device according to configuration 2 or 3, wherein:

[0200] The counting control circuit includes a selection circuit, and

[0201] The selection circuit is configured to be able to select an output of the count control signal and an output of the clock signal.

[0202] Configuration 5

[0203] The device according to Configuration 4, wherein the selection circuit outputs the clock signal before the count value of the counter reaches the threshold value.

[0204] Configuration 6

[0205] A device according to any one of configurations 1 to 5, wherein the control signal generating circuit is configured to be able to output a control signal having the second period and a control signal having a third period longer than the second period based on the clock signal and the mask signal, and when the count value of the counter reaches another threshold value for indicating a value greater than the threshold value, the signal to be output from the control signal generating circuit is switched from the control signal having the second period to the control signal having the third period.

[0206] Configuration 7

[0207] The device according to any one of configurations 1 to 6 further includes: a correction unit for correcting the count value based on the first period and the second period when the count value of the counter reaches the threshold.

[0208] Configuration 8

[0209] The device according to any one of configurations 1 to 7 further includes: a mask signal generating circuit for generating the mask signal.

[0210] Configuration 9

[0211] The device according to configuration 8, further comprising: a plurality of pixels each including the photodiode, the photodiode control circuit, the count pulse generation circuit, and the counter,

[0212] The mask signal generating circuit is arranged in each pixel among the plurality of pixels.

[0213] Configuration 10

[0214] The device according to configuration 8, further comprising: a plurality of pixels each including the photodiode, the photodiode control circuit, the count pulse generation circuit, and the counter,

[0215] Wherein, the mask signal generating circuit is commonly provided for the plurality of pixels.

[0216] Configuration 11

[0217] A system comprising:

[0218] An apparatus according to any one of configurations 1 to 10; and

[0219] A processing unit is used to generate an image using the signal output from the device.

[0220] Configuration 12

[0221] A mobile object, comprising:

[0222] An apparatus according to any one of configurations 1 to 10; and

[0223] A control unit is configured to control the movement of the moving body using the signal output from the device.

[0224] According to aspects of the embodiments, a photoelectric conversion device having a wide dynamic range and having a low power consumption function even in a high illuminance environment can be provided.

[0225] While the present disclosure has been described with reference to exemplary embodiments, it is to be understood that the present disclosure is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest description so as to encompass all such modifications and equivalent structures and functions.

Claims

1. A photoelectric conversion device, comprising: Photodiodes for avalanche multiplication; a photodiode control circuit comprising a charging circuit for switching between a first state in which the photodiode is connected to a power source and a second state in which the photodiode is not connected to the power source, and a control signal generating circuit for generating a control signal to control the charging circuit; a counting pulse generating circuit, configured to generate a pulse signal according to a signal output from the photodiode; a count control circuit for generating a count control signal to be input to the count pulse generating circuit; as well as a counter for counting the pulse signal output from the counting pulse generating circuit, wherein the control signal generating circuit is configured to output a control signal having a first period and a control signal having a second period longer than the first period based on a clock signal and a mask signal, In a case where the count value of the counter reaches a threshold value, the signal to be output from the control signal generating circuit is switched from the control signal having the first period to the control signal having the second period, and The count control circuit generates the count control signal based on the clock signal and the mask signal.

2. The photoelectric conversion device according to claim 1, wherein The count pulse generating circuit outputs a signal corresponding to the signal output from the photodiode to the counter when the count control signal is at a first level, and The count pulse generating circuit does not output a signal corresponding to the signal output from the photodiode to the counter when the count control signal is at a second level different from the first level.

3. The photoelectric conversion device according to claim 1, wherein The counting control circuit includes a selection circuit, and The selection circuit is configured to be able to select an output of the count control signal and an output of the clock signal.

4. The photoelectric conversion device according to claim 3, wherein The selection circuit outputs the clock signal before the count value of the counter reaches the threshold value.

5. The photoelectric conversion device according to claim 1, wherein The control signal generating circuit is configured to be capable of outputting a control signal having the second period and a control signal having a third period longer than the second period based on the clock signal and the mask signal, and When the count value of the counter reaches another threshold indicating a value greater than the threshold, the signal to be output from the control signal generating circuit is switched from the control signal having the second period to the control signal having the third period.

6. The photoelectric conversion device according to claim 1, further comprising: A correction unit is used to correct the count value based on the first cycle and the second cycle when the count value of the counter reaches the threshold.

7. The photoelectric conversion device according to claim 1, further comprising: The mask signal generating circuit is used to generate the mask signal.

8. The photoelectric conversion device according to claim 7, further comprising: a plurality of pixels, each of which includes the photodiode, the photodiode control circuit, the count pulse generating circuit and the counter, The mask signal generating circuit is arranged in each pixel among the plurality of pixels.

9. The photoelectric conversion device according to claim 7, further comprising: a plurality of pixels, each of which includes the photodiode, the photodiode control circuit, the count pulse generating circuit and the counter, Wherein, the mask signal generating circuit is commonly provided for the plurality of pixels.

10. A photoelectric conversion system comprising: The photoelectric conversion device according to any one of claims 1 to 9; as well as A processing unit is configured to generate an image using the signal output from the photoelectric conversion device.

11. The photoelectric conversion system according to claim 10, wherein: In the photoelectric conversion device, The count pulse generating circuit outputs a signal corresponding to the signal output from the photodiode to the counter when the count control signal is at a first level, and The count pulse generating circuit does not output a signal corresponding to the signal output from the photodiode to the counter when the count control signal is at a second level different from the first level.

12. The photoelectric conversion system according to claim 10, wherein: In the photoelectric conversion device, The counting control circuit includes a selection circuit, and The selection circuit is configured to be able to select an output of the count control signal and an output of the clock signal.

13. The photoelectric conversion system according to claim 10, wherein: In the photoelectric conversion device, The control signal generating circuit is configured to be capable of outputting a control signal having the second period and a control signal having a third period longer than the second period based on the clock signal and the mask signal, and When the count value of the counter reaches another threshold indicating a value greater than the threshold, the signal to be output from the control signal generating circuit is switched from the control signal having the second period to the control signal having the third period.

14. The photoelectric conversion system according to claim 10, further comprising: A correction unit is used to correct the count value based on the first cycle and the second cycle when the count value of the counter reaches the threshold.

15. A mobile object, comprising: The photoelectric conversion device according to any one of claims 1 to 9; as well as A control unit for controlling the movement of the moving body using the signal output from the photoelectric conversion device.

16. The moving object according to claim 15, wherein: In the photoelectric conversion device, The count pulse generating circuit outputs a signal corresponding to the signal output from the photodiode to the counter when the count control signal is at a first level, and The count pulse generating circuit does not output a signal corresponding to the signal output from the photodiode to the counter when the count control signal is at a second level different from the first level.

17. The moving object according to claim 15, wherein: In the photoelectric conversion device, The counting control circuit includes a selection circuit, and The selection circuit is configured to be able to select an output of the count control signal and an output of the clock signal.

18. The moving object according to claim 15, wherein: In the photoelectric conversion device, The control signal generating circuit is configured to be capable of outputting a control signal having the second period and a control signal having a third period longer than the second period based on the clock signal and the mask signal, and When the count value of the counter reaches another threshold indicating a value greater than the threshold, the signal to be output from the control signal generating circuit is switched from the control signal having the second period to the control signal having the third period.

19. The mobile body according to claim 15, further comprising: A correction unit is used to correct the count value based on the first cycle and the second cycle when the count value of the counter reaches the threshold.

Images