Photoelectric conversion device, photoelectric conversion device control method and program

Abstract

To provide a photoelectric conversion device which has an APD and reduces the shift of a black level even when it is driven for a long time.SOLUTION: A photoelectric conversion device includes: photoelectric conversion means which has multiple pixels including an effective pixel that is constituted by an avalanche photodiode and receives light and an optical black pixel that blocks the light; decision means which decides a correction value for correcting a signal value of the effective pixel on the basis of a signal obtained from the optical black pixel; and output means which outputs an image by using the correction value. The decision means decides the correction value on the basis of a difference between the first signal value showing the signal value of the optical black pixel in a state where a light shielding part is inserted to an optical path of an image formation optical system and the second signal value showing the signal value of the effective pixel, and a third signal value showing the signal value of the optical black pixel in a state where the light shield part is removed from the optical path of the image formation optical system.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present invention relates to a photoelectric conversion device. [Background technology]

[0002] In recent years, a photoelectric conversion device has been proposed that digitally counts the number of photons arriving at an avalanche photodiode (APD) and outputs the counted value from a pixel as a photoelectrically converted digital signal.

[0003] The signal of a pixel having an APD includes not only a signal based on photoelectric conversion in response to incident light but also a signal resulting from dark electrons generated at the trap level of the pixel. Non-Patent Document 1 reports a phenomenon in which dark current increases when a photoelectric conversion device having an APD is driven for a long period of time.

[0004] When the dark current increases due to the long-term operation of the photoelectric conversion device, the black level of the signal gradually shifts, resulting in a deterioration in image quality when an image is formed using the output from the photoelectric conversion device. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 2022-106649 [Non-patent literature]

[0006] [Non-Patent Document 1] C. Zhang, “SPAD requirements from consumer electronics to automotive,” Int. SPAD Sensor Workshop (2022) Summary of the Invention [Problem to be solved by the invention]

[0007] In imaging devices using a general solid-state imaging element, a method is known in which optical black pixels are provided outside the effective pixel area and the black level of the effective pixels is corrected using the values ​​of the optical black pixels (OB pixels).

[0008] However, in a photoelectric conversion device having an APD, the more light incident on the APD, the larger the avalanche current that flows, which may cause new trap levels to be generated in the pixel.Furthermore, the large avalanche current may cause a local temperature rise, which may increase the possibility that dark electrons generated in the trap levels will flow into the avalanche multiplication region.

[0009] Therefore, when a photoelectric conversion device is driven for a long time, the deviation in the black level differs between the effective pixel area where a lot of light is incident and the OB pixels where no light is incident. As a result, when using conventional black level correction using OB pixels, it is difficult to sufficiently correct the deviation in the black level of the signal.

[0010] Therefore, an object of the present invention is to reduce the deviation of the black level in a photoelectric conversion device having an APD even when the device is driven for a long period of time. [Means for solving the problem]

[0011] a photoelectric conversion means having a plurality of pixels, each of which is an avalanche photodiode and includes effective pixels for receiving light and optical black pixels for blocking light; a determination means for determining a correction value for correcting the signal value of the effective pixel based on a signal acquired from the optical black pixel; and a correction value Correcting the signal value of the effective pixel based on With means, death The determining means determines whether the light blocking portion is inserted into the optical path of the imaging optical system. in a first signal value indicating a signal value of the optical black pixel; When the light blocking unit is inserted into the optical path of the imaging optical system, a second signal value indicating the signal value of the effective pixel; 、 and a third signal value indicating the signal value of the optical black pixel in a state where the light blocking portion is removed from the optical path of the imaging optical system.、 a correction coefficient for correcting a difference between a first image acquisition condition in a state where the light-shielding unit is inserted into the optical path of the imaging optical system and a second image acquisition condition in a state where the light-shielding unit is removed from the optical path of the imaging optical system; The correction value is determined based on the above. [Effects of the Invention]

[0012] According to the present invention, in a photoelectric conversion device having an APD, deviation in black level can be reduced even when the device is driven for a long period of time. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 1 shows an example of the configuration of a photoelectric conversion element. [Figure 2] FIG. 1 shows an example of the configuration of a sensor substrate 11. [Figure 3] FIG. 2 shows an example of the configuration of a circuit board 21. [Figure 4] Timing chart of the vertical scanning circuit 110 [Figure 5] FIG. 1 is a diagram showing an example of an equivalent circuit of a pixel 101 and a signal processing circuit 104 corresponding to the pixel 101. [Figure 6] FIG. 1 is a block diagram showing an example of the functional configuration of a photoelectric conversion device according to a first embodiment. [Figure 7] 1 is a flowchart illustrating processing executed by the photoelectric conversion device according to the first embodiment. [Figure 8] 10 is a flowchart illustrating processing executed by a photoelectric conversion device according to a second embodiment. [Figure 9] Setting screen of the photoelectric conversion device according to the second embodiment [Figure 10] FIG. 10 is a diagram for explaining signal processing in the photoelectric conversion device according to the second embodiment. [Figure 11] 10 is a flowchart illustrating processing executed by a photoelectric conversion device according to a third embodiment. [Figure 12] Block diagram showing an example of the functional configuration of a photoelectric conversion system DETAILED DESCRIPTION OF THE INVENTION

[0014] The following describes in detail embodiments of the present invention. Note that the embodiments described below are examples for realizing the present invention, and should be modified or adjusted as appropriate depending on the configuration of the device to which the present invention is applied and various conditions, and the present invention is not limited to the following embodiments. Furthermore, parts having the same functions in all figures are designated by the same numerals, and repeated explanations thereof will be omitted.

[0015] <Embodiment 1> FIG. 1 is a diagram showing an example of the configuration of a photoelectric conversion element of this embodiment. The following description will be given taking as an example a photoelectric conversion device having a so-called stacked structure in which a photoelectric conversion element 100 is configured by stacking and electrically connecting two substrates, a sensor substrate 11 and a circuit substrate 21. However, a so-called non-stacked structure in which the components included in the sensor substrate and the components included in the circuit substrate are arranged on a common semiconductor layer may also be used. The sensor substrate 11 includes a pixel region 12. The circuit substrate 21 includes a circuit region 22 that processes signals detected in the pixel region 12.

[0016] <Sensor board> FIG. 2 is a diagram illustrating an example of the configuration of a sensor substrate 11. The pixel region 12 of the sensor substrate 11 includes a plurality of pixels 101 arranged two-dimensionally across multiple rows and columns. The pixel region 12 includes an effective pixel region 13, into which light can enter through the imaging optical system of the photoelectric conversion device, and an optical black pixel (hereinafter, referred to as OB pixel) region 14, in which light entering the photoelectric conversion unit is blocked by a light-shielding member disposed on the light-incident surface of the pixel. The pixel 101 includes a photoelectric conversion unit 102 including an avalanche photodiode (hereinafter, referred to as APD). The number of rows and columns of the pixel array constituting the pixel region 12 is not particularly limited. While FIG. 2 illustrates a case in which the OB pixel is located above the effective pixel region, it may also be located below, on the left or right, or in multiple regions, such as above and below.

[0017] Although not shown in Figure 2, it is preferable to arrange the effective pixel area and the OB pixels at a distance of at least a certain distance. The reason for this is explained below. A phenomenon known as avalanche emission is known to occur in photoelectric conversion devices that include APDs. When avalanche emission occurs, the generated secondary electrons enter adjacent pixels, increasing the count number of the adjacent pixel values ​​and resulting in miscounting. Therefore, if the effective pixels and the OB pixels are close to each other, avalanche emission that occurs when light enters the effective pixels may increase the count number of the OB pixels. Therefore, it is preferable to arrange the effective pixel area and the OB pixels at a distance of at least a certain distance to prevent miscounting in the OB pixels.

[0018] <Circuit board> Fig. 3 is a diagram showing an example of the configuration of the circuit board 21. The circuit board 21 has a signal processing circuit 103 that processes charges photoelectrically converted by the photoelectric conversion units 102 of the pixels 101 including the OB pixels of Fig. 2, a readout circuit 112, a control pulse generation unit 115, a horizontal scanning circuit 111, signal lines 113, and a vertical scanning circuit 110.

[0019] The vertical scanning circuit 110 receives control pulses from a control pulse generation unit 115 and supplies the control pulses to each pixel. The vertical scanning circuit 110 uses logic circuits such as a shift register and an address decoder. Signals output from the photoelectric conversion units 102 of the pixels are processed by a signal processing circuit 103. The signal processing circuit 103 is provided with a counter, memory, etc., and digital values ​​are stored in the memory. The horizontal scanning circuit 111 inputs control pulses to the signal processing circuit 103 to sequentially select each column in order to read the signals from the memory of each pixel that stores the digital signals. For the selected column, a signal is output to a signal line 113 from the signal processing circuit 103 of the pixel selected by the vertical scanning circuit unit 110. The signal output to the signal line 113 is output to the outside of the photoelectric conversion device 100 via an output circuit 114.

[0020] <Connection between sensor board and circuit board> 2 and 3, a plurality of signal processing circuits 103 are arranged in a region overlapping the pixel region 12 in a planar view. A vertical scanning circuit unit 110, a horizontal scanning circuit unit 111, a readout circuit 112, an output circuit 114, and a control pulse generation unit 115 are arranged so as to overlap between an end of the sensor substrate 11 and an end of the pixel region 12 in a planar view. In other words, the sensor substrate 11 has the pixel region 12 and a non-pixel region arranged around the pixel region 12. The vertical scanning circuit unit 110, the horizontal scanning circuit unit 111, the readout circuit 112, the output circuit 114, and the control pulse generation unit 115 are arranged in a region overlapping the non-pixel region in a planar view.

[0021] The arrangement of the signal lines 113, the readout circuits 112, and the output circuits 114 is not limited to that shown in Fig. 3. For example, the signal lines 113 may be arranged extending in the row direction, and the readout circuits 112 may be arranged at the ends of the signal lines 113. Furthermore, the function of the signal processing unit does not necessarily need to be provided for each photoelectric conversion unit, and one signal processing unit may be shared by multiple photoelectric conversion units to perform signal processing sequentially.

[0022] <Pixel equivalent circuit> FIG. 4 is a diagram showing an equivalent circuit of the pixel 101 and the signal processing circuit 103 corresponding to the pixel 101 in FIGS. 2 and 3. The APD 201 generates charge pairs in response to incident light through photoelectric conversion. One of the two nodes of the APD 201 is connected to a power supply line that supplies a drive voltage VL (first voltage). The other of the two nodes of the APD 201 is connected to a power supply line that supplies a drive voltage VH (second voltage) higher than the voltage VL. In FIG. 4, one node of the APD 201 is an anode, and the other node of the APD is a cathode. A reverse bias voltage is supplied to the anode and cathode of the APD 201 so that the APD 201 performs avalanche multiplication. With this voltage supplied, charges generated by incident light undergo avalanche multiplication, generating an avalanche current.

[0023] When a reverse bias voltage is supplied, there are two modes: Geiger mode, in which the voltage difference between the anode and cathode is greater than the breakdown voltage, and linear mode, in which the voltage difference between the anode and cathode is close to or less than the breakdown voltage. An APD operating in Geiger mode is called a SPAD. In the case of a SPAD, for example, the voltage VL (first voltage) is -30V and the voltage VH (second voltage) is 1V.

[0024] The quench element 202 is connected to a power supply line to which a drive voltage VH is supplied and to one of the anode and cathode nodes of the APD 201. The quench element 202 functions as a load circuit (quench circuit) during signal multiplication by avalanche multiplication, suppressing the voltage supplied to the APD 201 and suppressing avalanche multiplication (quench operation). The switch 202 also functions to return the voltage supplied to the APD 201 to the drive voltage VH by passing a current equivalent to the voltage drop caused by the quench operation (recharge operation).

[0025] The signal processing circuit 103 has a waveform shaping section 210, a counter circuit 211, and a selection circuit 212. In Fig. 4, the signal processing circuit 103 has the waveform shaping section 210, the counter circuit 211, and the selection circuit 212, but in this specification, it is sufficient for the signal processing circuit 103 to have at least one of the waveform shaping section 210, the counter circuit 211, and the selection circuit 212.

[0026] The waveform shaping unit 210 shapes the voltage change at the cathode of the APD 201 obtained when a photon is detected, and outputs a pulse signal. For example, an inverter circuit is used as the waveform shaping unit 210. While Fig. 4 shows an example in which one inverter is used as the waveform shaping unit 210, a circuit in which multiple inverters are connected in series, or another circuit with a waveform shaping effect, may also be used.

[0027] The counter circuit 211 counts the pulse signal output from the waveform shaping unit 210 and holds the count value. When a control pulse RES is supplied via a drive line 213, the signal held in the counter circuit 211 is reset.

[0028] A control pulse SEL is supplied to the selection circuit 212 from the vertical scanning circuit unit 110 in Fig. 3 via a drive line 214 (not shown in Fig. 3) in Fig. 4, and switches between electrical connection and disconnection between the counter circuit 211 and the signal line 113. The selection circuit 212 includes, for example, a buffer circuit for outputting a signal. An output signal OUT shown in Fig. 4 is an output signal from a pixel.

[0029] The electrical connection may be switched by disposing a switch such as a transistor between the quench element 202 and the APD 201 or between the photoelectric conversion element 102 and the signal processing circuit 103. Similarly, the supply of the voltage VH or the voltage VL to the photoelectric conversion element 102 may be electrically switched using a switch such as a transistor.

[0030] <Circuit drive> FIG. 5 is a diagram illustrating the relationship between APD operation and output signals. The input side of the waveform shaping unit 210 is designated node A, and the output side is designated node B. Between time t0 and time t1, a potential difference of VH-VL is applied to the APD 201. When a photon is incident on the APD 201 at time t1, avalanche multiplication occurs in the APD 201, an avalanche multiplication current flows through the quench element 202, and the voltage at node A drops. As the voltage drop increases further and the potential difference applied to the APD 201 decreases, avalanche multiplication in the APD 201 stops, as shown at time t2, and the voltage level at node A no longer drops below a certain value. Subsequently, between time t2 and time t3, a current flows through node A from voltage VL to compensate for the voltage drop, and at time t3, node A stabilizes to its original potential level. At this time, the part of the output waveform at node A that exceeds a certain threshold is waveform-shaped by the waveform shaping section 210 and is output as a signal at node B.

[0031] <System block diagram> FIG. 6 is a system block diagram of the photoelectric conversion device 300 shown in the first embodiment. Some of the functional blocks shown in FIG. 6 are implemented by causing a computer (e.g., a control unit 302, which is a CPU) included in the photoelectric conversion device 300 to execute a computer program stored in a memory (storage unit 304) serving as a storage medium. However, some or all of these may be implemented by hardware. Examples of hardware that can be used include a field programmable gate array (FPGA), a dedicated circuit (ASIC), and a processor (reconfigurable processor, DSP). Furthermore, the functional blocks shown in FIG. 6 do not have to be built into the same housing, and may be configured as separate devices connected to each other via signal paths.

[0032] The photoelectric conversion device 300 includes an imaging unit 301 (photoelectric conversion element 100, imaging optical system 3010, and shading mechanism 3011 for blocking light entering the photoelectric conversion element 100), a control unit 302, a signal processing unit 303 for processing signals acquired by the photoelectric conversion element, a memory unit 305, and a detection unit 304.

[0033] The imaging unit 301 outputs an image signal and is composed of a photoelectric conversion element 100, an imaging optical system 3010, and a light-blocking mechanism 3011 for blocking light from entering the photoelectric conversion element 100. The photoelectric conversion element 100 is composed of an avalanche photodiode and has multiple pixels, including effective pixels that receive light and optical black pixels that are blocked from light. The photoelectric conversion element 100 acquires signal values ​​from each of the OB pixels and effective pixels. The light-blocking mechanism 3011 is composed of a light-blocking section made of a material such as a light-absorbing dye and a drive section that controls the open / close state of the light-blocking section. The drive section is composed of, for example, a galvanometer, and the open / close state of the light-blocking mechanism can be controlled by controlling the voltage applied to the galvanometer. The light-blocking mechanism 3011 may also be composed of an aperture in the imaging optical system and an aperture opening / closing mechanism. Light entering the photoelectric conversion element can be blocked by completely closing the aperture. Furthermore, instead of changing the position or shape of the light-shielding portion or aperture, a liquid crystal filter whose light transmittance can be controlled by the applied voltage may be used, and the light incident on the photoelectric conversion portion may be blocked by changing the applied voltage.

[0034] The control unit 302 controls the image capture. Specifically, it accepts a user's instruction and determines whether to start or end image capture. The control unit 302 also detects the image capture conditions of the photoelectric conversion device. The image capture conditions are a set of parameters that indicate the time of image capture, parameters, the temperature of the photoelectric conversion element, and other conditions.

[0035] The signal processing unit 303 acquires the signal output by the photoelectric conversion element 100 and outputs an image based on a correction value determined based on the signal value of the OB pixel. The signal processing unit 303 includes a signal acquisition unit 3030 that determines a black level correction value from the signal acquired by the photoelectric conversion device, and a storage unit 305 that stores the black level correction value determined by the signal acquisition unit 3030. The signal processing unit 303 also includes an image output unit 3031 that corrects the signal value of each pixel acquired by the photoelectric conversion element 100 using the black level correction value stored in the storage unit 305 and outputs an image.

[0036] The photoelectric conversion device 300 further includes a detection unit 304 that detects a change in the image acquisition conditions (change from a first condition to a second condition) in the imaging device 300. The image acquisition conditions detect internal changes due to changes in parameters set in the photoelectric conversion device 300, and external changes caused by the duration of continuous use or the external environment. The detection unit 304 detects changes based on the temperature of a sensor acquired by a thermometer attached to the photoelectric conversion element 100, or detects changes in the digital gain set in the photoelectric conversion device 300, for example.

[0037] In the photoelectric conversion device of this embodiment, three types of signals are used to correct the black level as follows. Specifically, a difference is calculated between a first signal value calculated from the signal of the OB pixel region when the light blocking mechanism is inserted into the optical path of the imaging optical system and a second signal value calculated from the signal of the effective pixel region. A third signal value is then calculated from the signal of the OB pixel region when the light blocking mechanism is removed from the optical path of the imaging optical system. A fourth signal value, which is the difference between the first and second signal values, and the third signal value are then used to calculate a black level correction value. This allows a photoelectric conversion device with an APD to sufficiently reduce black level deviations even when driven for long periods of time.

[0038] <Flowchart> FIG. 7 is a diagram illustrating a flowchart of signal processing for black level correction. The processing shown in the flowchart of FIG. 7 (and FIGS. 8 and 11, which will be described later) is executed by a control unit 801 (CPU) in FIG. 12, which is a computer, in accordance with a computer program stored in the storage device 104. In the following description, each process (step) is denoted by adding an S to the beginning, thereby eliminating the need to refer to the process (step). Black level correction in this embodiment is made up of two stages: a preparation stage (S400 to S404) in which a light blocking mechanism is inserted into the optical path of the imaging optical system to acquire data for correction, and a correction stage (S405 to S409) in which correction processing is performed while a captured image is being acquired. A signal processing method for black level correction, which is a method for controlling a photoelectric conversion device, is described below.

[0039] First, the preparation stage will be described. In S400, the control unit 302 determines whether or not the correction value needs to be updated. If some time has passed since the first or previous image capture, or if the image acquisition conditions have changed, the process proceeds to S401. As will be described later, if a previous correction value can be used, the process may skip the preparation stage and proceed to S405. In S401, the image capture unit 301 inserts a light-blocking mechanism into the optical path of the imaging optical system. In S402, the signal processing unit 303, while the light-blocking mechanism remains inserted in the optical path of the imaging optical system, acquires a first signal value from the signal of the OB pixel region and stores it in the storage unit 305. The first signal value may be the average or median of multiple pixels in the OB pixel region. At this time, pixels that deviate from the expected signal value by more than a certain threshold may be excluded as defective pixels. Alternatively, the first signal value may be calculated by averaging multiple frames.

[0040] Next, in S403, the signal processing unit 303 acquires a second signal value from the signal of the effective pixel region while the light blocking mechanism remains inserted in the optical path of the imaging optical system, and stores the second signal value in the storage unit 305. The second signal value may be the average or median value of multiple pixels in the effective pixel region, and, as with the first signal value, may exclude defective pixels or may be calculated as an average over multiple frames. Thereafter, in S404, the signal processing unit 303 determines a fourth signal value that indicates the difference between the second signal value and the first signal value. This completes the preparation stage.

[0041] Next, the correction stage will be described. In S405, the control unit 302 determines whether or not to start imaging. If the update of the correction value is complete and imaging is ready to start, the process proceeds to S406. If the user issues an instruction to stop imaging, the process ends. If imaging is to start, the imaging unit 301 removes the light blocking mechanism from the optical path of the imaging optical system and starts acquiring a captured image. In S407, the signal processing unit 303 obtains a third signal value from the signal of the OB pixel region while acquiring the captured image. Then, in S408, the signal processing unit 303 determines a black level correction value C from the fourth signal value and the third signal value. Specifically, the correction value C is calculated by multiplying the fourth signal value L by the fourth signal value L. OFF and a third signal value L OBis used to calculate as follows: Correction value C=L OB +L OFF ×α (Number 1)

[0042] Here, α is a correction coefficient for correcting the difference between the first image acquisition condition in the preparation stage where the light blocking mechanism is inserted and the second image acquisition condition in the correction stage where the light blocking mechanism is removed. As can be seen from the formula, the correction value C is the fourth signal value L OFF The fifth signal value corrected by α and the third signal value L OB The signal value L of each pixel in the effective pixel area can be calculated by the following formula. B(X,Y) By subtracting C from the black level, the signal value L of the effective pixel with corrected black level is obtained. B_COR(X,Y) That is, the signal processing unit 303 (image output unit 3031) uses the correction value to correct the signal values ​​of the effective pixels, and outputs the image. L B_COR(X,Y) =L B(X,Y) -C (number 2)

[0043] In S409, the control unit 302 determines whether or not to continue capturing images. If capturing images is to be continued, the process returns to S407. Note that if the correction value needs to be updated or if there is an instruction from the user, the process may return to S401. Furthermore, capturing images is terminated when there is an instruction to end capturing images or when a predetermined time for capturing images has been determined and the time has expired.

[0044] As described above, in the photoelectric conversion device of this embodiment, the black level correction value C is calculated using the fourth signal value and the third signal value. This makes it possible to sufficiently reduce the deviation in the black level even when driven for a long period of time. The effects of the present invention will be explained below in comparison with conventional black level correction methods.

[0045] <Effects> For simplicity, the following description will be given taking as an example a case where α is 1, i.e., there is no difference between the first image acquisition condition in the preparation stage when the light-blocking mechanism is inserted and the second image acquisition condition in the correction stage when the light-blocking mechanism is removed. A case where α is not 1, i.e., there is a difference between the first image acquisition condition in the preparation stage when the light-blocking mechanism is inserted and the second image acquisition condition in the correction stage when the light-blocking mechanism is removed, will be described later.

[0046] When the signal value of the OB pixel in the correction stage is L3 and the signal value of the effective pixel is L5, L5 and L3 are the black level L of the OB pixel. OB , the black level of the effective pixels L EFF , the number of photons incident on the effective pixel L PH can be calculated using the following: L3=L OB , L5=L EFF +L PH (Number 3)

[0047] In the conventional black level correction method, the black level is corrected by subtracting the signal value L3 of the OB pixel from the signal value L5 of the effective pixel. 99 becomes: L 99 =L5-L3=L EFF +L PH -L OFF (Number 4)

[0048] As can be seen from this equation, L 99 is the number of photons incident on the effective pixel, L PH In other words, to properly correct the black level, L EFF and L OB However, as mentioned above, in a photoelectric conversion element using an APD, when the photoelectric conversion device is driven for a long time, the black level differs between the effective pixel area where a lot of light is incident and the OB pixels where no light is incident. Therefore, when using the conventional black level correction method, the L EFF and L OB The difference between the black levels of the signals becomes large, making it difficult to sufficiently correct the deviation in the black levels of the signals.

[0049] On the other hand, in the photoelectric conversion element of the present invention, in addition to the signal value L3 of the OB pixel in the correction stage, the signal value L1 of the OB pixel in the preparation stage and the signal value L2 of the effective pixel are acquired, and the black level is corrected using the difference L4 between L2 and L1. When α is 1, L1 is L OB and L2 is L EFF Therefore, the signal value L6 of the effective pixel after correction is as follows: L6=L5-(L3+L2-L1)=L5-L2=L EFF +L PH -L EFF =L PH (Number 5)

[0050] That is, by performing the correction according to this embodiment, even if the black level differs between the effective pixel area where a lot of light is incident and the OB pixel where no light is incident, the number of photons L incident on the effective pixel can be PH can be calculated correctly.

[0051] <Correction coefficient> Next, we will explain the correction coefficient α for correcting the difference between the first image acquisition condition in the preparation stage when the light-shielding mechanism is inserted and the second image acquisition condition in the correction stage when the light-shielding mechanism is removed. The image acquisition conditions include, for example, the time required to integrate the number of photons incident on the photoelectric conversion unit (integration time), the digital gain multiplied by the integrated number of photons, and the temperature of the photoelectric conversion element. The value of the dark current changes depending on the integration time, digital gain, and temperature of the photoelectric conversion element. Using α, the difference in the dark current between the first image acquisition condition in the preparation stage and the second image acquisition condition in the correction stage is corrected. The detection unit 304 detects changes in the image acquisition conditions of the photoelectric conversion device including the photoelectric conversion element. The detection unit 304 may also detect changes in the image acquisition conditions when image processing parameters such as digital gain are controlled by the control unit 302.

[0052] The black level of the OB pixels and effective pixels in the preparation stage is L OB and L EFF When this is done, the difference in black level between the effective pixels and the OB pixels in the preparation stage, LOFF(準備) becomes: L OFF(準備) =L EFF -L OB (Number 6)

[0053] Here, the dark current is proportional to the integration time and digital gain, so the difference L between the effective pixels and the OB pixels OFF The difference in black level at the correction stage is L OFF(補正) becomes: L OFF(補正) =L OFF(準備) ×(T O ×D O )÷(T P ×D P ) (Number 7)

[0054] where T O and D O is the integration time and digital gain in the preparation stage, T P and D P are the integration time and digital gain in the preparation stage. Therefore, the correction coefficient α can be calculated as follows: α=(T O ×D O )÷(T P ×D P ) (Number 8)

[0055] The above explains how to correct the difference between the integration time and the digital gain, but it is preferable to calculate the correction coefficient by taking into account the difference in temperature of the photoelectric conversion element between the preparation stage and the correction stage. This is because, although it depends on the characteristics of the APD used in the photoelectric conversion element, the higher the temperature, the larger the dark current, that is, the dark current is temperature dependent.

[0056] As described above, by using the correction coefficient α, it is possible to reduce the deviation in the black level while taking into consideration the difference in dark current between the first image acquisition condition in the preparation stage and the second image acquisition condition in the correction stage. That is, by calculating the black level correction value using the fifth signal value obtained by correcting the fourth signal value by α and the third signal value, it is possible to reduce the deviation in the black level while taking into consideration the difference in dark current between the first image acquisition condition and the second image acquisition condition.

[0057] Generally, the optimal values ​​of the integration time and digital gain used in the correction stage vary depending on the subject being photographed in the correction stage, and therefore are not necessarily the same as the integration time and digital gain used in the preparation stage. By using the correction coefficient α, it is possible to reduce the deviation in the black level while using the integration time and digital gain that are optimal for the subject being photographed in the correction stage, which is preferable.

[0058] <Variation 1> In a photoelectric conversion device in which the pixels of the photoelectric conversion element have on-chip color filters and can acquire color information, signal values ​​of OB pixels and effective pixels may be calculated for each color channel in steps S402, S403, and S406. As described above, the more light incident, the greater the potential for increased dark current. When photographing a typical subject, more light is incident on green pixels than on red and blue pixels, so the green pixels may experience a greater increase in dark current than the red and blue pixels. By calculating signal values ​​for each color channel, it is possible to reduce discrepancies in black levels due to differences in the increase in dark current between color channels. Specifically, first signal values ​​for red, blue, and green pixels are calculated from OB pixels when light is blocked. Similarly, second signal values ​​for red, blue, and green pixels are calculated from effective pixels when light is blocked. Furthermore, third signal values ​​for red, blue, and green pixels are calculated from OB pixels when light is received. A correction value for each color can be calculated from these signal values.

[0059] <Embodiment 2> As mentioned above, in a photoelectric conversion device with an APD, the more light incident on the APD, the larger the avalanche current that flows, which can lead to the generation of new trap levels in the pixel. The trap levels newly generated when the photoelectric conversion device is operated for a long period of time do not necessarily have the same characteristics as the trap levels that were causing the dark current before the photoelectric conversion device was operated, so the temperature dependence of the dark current may differ. As a result, when the photoelectric conversion device is operated for a long period of time, the temperature dependence of the dark current may change depending on the operating time.

[0060] Therefore, in the photoelectric conversion device of embodiment 2, a plurality of fourth signal values ​​at different temperatures are calculated in the preparation stage, and the deviation of the black level is corrected taking into consideration the change in the temperature dependency of the dark current caused by driving the photoelectric conversion device. Note that the functional configuration and hardware configuration of the photoelectric conversion device are basically the same as those of embodiment 1, and differences will be explained below.

[0061] In this embodiment, the control unit 302 controls the temperature of the photoelectric conversion device. The temperature control includes, for example, acquiring a measurement (temperature) from a thermometer and controlling the activation and cooling level of a temperature control means (cooling device). The temperature control means may be a Peltier element, a heater, a fan, or the like.

[0062] FIG. 8 is a flowchart illustrating signal processing for black level correction according to the second embodiment. First, in S500, the control unit 302 determines whether the correction value needs to be updated. If some time has passed since the first or previous image capture, or if the image acquisition conditions have changed, the process proceeds to S501. As will be described later, if a previous correction value can be used, the process skips the preparation step and proceeds to S511. In S501, the image capture unit 301 inserts a light blocking mechanism into the optical path of the imaging optical system. Next, in S502, the control unit 302 controls the temperature of the photoelectric conversion element to a first temperature TL. After the temperature of the photoelectric conversion element reaches TL, the signal processing unit 303 acquires a first signal value at temperature TL from the signal of the OB pixel region and a second signal value at temperature TL from the signal of the effective pixel in S502 to S504, similar to S402 to S404 of FIG. 7. The signal processing unit 303 then determines a fourth signal value at temperature TL from the difference between the first and second signal values.

[0063] Next, in S506, the control unit 302 controls the temperature of the photoelectric conversion element to a second temperature TH higher than the first temperature TL. After the temperature of the photoelectric conversion element reaches TH, in S507-S509, similar to steps S402-S404 of FIG. 7, the signal processing unit 303 acquires a first signal value at temperature TH from the signal of the OB pixel region and a second signal value at temperature TH from the signal of the effective pixel. Then, the signal processing unit 303 determines a fourth signal value at temperature TH from the difference between the first and second signal values ​​at temperature TH. Finally, in S510, the signal processing unit 303 determines a temperature dependence coefficient β of the dark current from the second signal value when the temperature is TL calculated in S504 and the second signal value when the temperature is TH calculated in S508. It is generally known that the temperature dependence of the dark current can be approximated by the Arrhenius equation below, where T is the temperature and k is the Boltzmann constant.

[0064]

number

[0065] <Calculation of β> Therefore, the second signal value when the temperature is TL is L EFF_TL , the second signal value when the temperature is TH is L EFF_TH Then, the temperature dependency coefficient β of the dark current can be calculated as follows:

[0066]

number

[0067] In this way, in the photoelectric conversion device of embodiment 2, the second signal value is calculated at a plurality of temperatures in the preparation stage, thereby calculating the temperature dependency coefficient β of the dark current after the photoelectric conversion device is driven. This makes it possible to correct the deviation of the black level by taking into consideration the change in the temperature dependency of the dark current caused by driving the photoelectric conversion device.

[0068] The correction stage is the same as S405 to S409 in FIG. 7. That is, in S405, the control unit 302 determines whether or not to start imaging. If imaging is to be started, the process proceeds to S512. If imaging is not to be performed, the process ends. In S512, the imaging unit 301 removes the light blocking mechanism and starts imaging. In S513, the signal processing unit 303 calculates a third signal value from the signal of the OB pixel region. At this time, the control unit 302 acquires the temperature of the photoelectric conversion element. Then, in S514, the signal processing unit 303 acquires a correction value from the temperature of the photoelectric conversion element, the fourth signal value, the third signal value, and the temperature dependency coefficient β of the dark current. The signal processing unit 303 uses the correction value to correct the signal values ​​of the effective pixels and output an image.

[0069] <Variation: Setting implementation conditions> In FIG. 8, steps S501 to S504 for calculating the signal value at temperature TL and steps S507 to S509 for calculating the signal value at temperature TH are performed consecutively. However, the steps for temperature TL and the steps for temperature TH may be performed separately. In this case, the control unit 302 controls the acquisition of the first signal value of the OB pixel and the second signal value of the effective pixel to be performed under execution conditions set by the user. In this case, step S506 may be changed to a process in which the control unit 302 determines whether the execution conditions set by the user are satisfied. Furthermore, the control unit 302 acquires each signal value at each temperature when a predetermined temperature set by the user is reached. Furthermore, the condition set by the user may be temperature rather than time. Specifying temperature as a condition is preferable because it allows the correction signal to be acquired at the same temperature each time. In this case, two different values, for example, 20 degrees and 40 degrees, may be set.

[0070] Although it depends on the type of temperature control means, it generally takes several minutes to change the temperature. Therefore, if steps S504 and S505 are performed consecutively, normal image capture will not be possible for several minutes. On the other hand, if the light blocking mechanism is removed in step S504 and the steps from step S505 are performed after the temperature reaches TH, normal image capture will be possible in the several minutes between steps S504 and S505, thereby shortening the time during which image capture is not possible, which is preferable.

[0071] The control unit 302 may acquire the first signal value and the second signal value using a time set by the user as a trigger. While FIG. 8 illustrates an example in which the photoelectric conversion device includes a temperature control unit and controls the temperature of the photoelectric conversion element to TL and TH, the photoelectric conversion device does not necessarily need to include a temperature control unit. For example, when a photoelectric conversion device is used for outdoor monitoring, the photoelectric conversion device is generally used continuously 24 hours a day, so the temperature of the photoelectric conversion device may differ between daytime and nighttime. Therefore, by utilizing the temperature difference between daytime and nighttime, a signal when the temperature is assumed to be low at night and a signal when the temperature is relatively high during the daytime may be acquired.

[0072] A specific example will be described. FIG. 9 shows a setting screen for acquiring the temperature dependence of dark current when the photoelectric conversion device is used for monitoring. The photoelectric conversion device shown in FIG. 9 is configured to acquire first and second signal values ​​for correction at a predetermined time every day. That is, in the photoelectric conversion device shown in FIG. 9, the control unit 302 controls the acquisition of first signal values ​​of OB pixels and second signal values ​​of effective pixels to be performed under execution conditions (predetermined times) set by the user. FIG. 9 shows an example in which data for correction is set to be acquired at 4:00 a.m. during the night when the temperature is relatively low and at 2:00 p.m. during the day when the temperature is relatively high. The photoelectric conversion device automatically engages the light blocking mechanism at the times set in FIG. 9 and performs steps S501 to S504 at night and steps S507 to S509 during the day. In this case, step S506 is skipped.

[0073] 10A and 10B show a flow for updating the value of the temperature dependence coefficient β of the dark current. As shown in FIG. 10A, the photoelectric conversion device first calculates the second signal value L when the temperature is TL0. EFF_TL0 and the second signal value L when the temperature is TH0. EFF_TH0 Therefore, the initial value β0 of the temperature dependence coefficient of the current can be calculated as follows:

[0074]

number

[0075] As shown in FIG. 10(b), when S501 to S504 are performed at night, the second signal value L EFF_TL1 At this time, the temperature dependence coefficient of the dark current is L EFF_TL0 L EFF_TL1 and become β1.

[0076]

number

[0077] Next, as shown in FIG. 10(c), when steps S507 to S509 are performed in the daytime, the second signal value L EFF_TH1 Similarly, the temperature dependence coefficient of the dark current is obtained as L EFF_TH0 L EFF_TH1 and become β2.

[0078]

number

[0079] In this way, by periodically updating the temperature dependency coefficient of the dark current as the operating time of the photoelectric conversion device increases, it is possible to correct the black level deviation taking into account changes in the temperature dependency of the dark current regardless of the operating time of the photoelectric conversion device.

[0080] 9 shows an example in which data for correction can be acquired at a fixed time every day, but it does not have to be every day, and may be once a week, once a month, once a year, etc. Furthermore, the time for acquiring the signal for correction may be set to three or more types instead of two types (daytime and nighttime), or the user may be able to acquire the signal for correction at any time of their choice.

[0081] When the photoelectric conversion element does not have a temperature control means and acquires a signal for correction using changes in the ambient temperature, especially when the user sets the condition to be time rather than temperature, the temperature of the photoelectric conversion element when the signal is acquired is not necessarily the same each time. Therefore, if the temperature of the photoelectric conversion element when the signal is acquired is equal to or higher than the threshold value, the second signal value (L _EFF_TH1 ) and updates the second signal value (L _EFF_TL1) is preferably updated. In particular, when the user acquires a signal for correction at a desired time, it is preferable to automatically change the second signal value to be updated based on the threshold value, since the temperature of the photoelectric conversion element cannot be determined. In other words, it is preferable to store data for correction separately as temperature-specific data depending on the temperature of the photoelectric conversion element when executed under conditions set by the user. The control unit 302 determines whether the temperature of the photoelectric conversion element at the time set by the user is equal to or higher than the threshold value, and if the temperature is equal to or higher than the threshold value, the signal processing unit 303 acquires the second signal value at a high temperature. If the temperature is lower than the threshold value, the signal processing unit 303 acquires the second signal value at a low temperature.

[0082] <Embodiment 3: Correcting differences in black levels for each pixel> As mentioned above, in a photoelectric conversion device having an APD, the more light incident on the APD, the larger the avalanche current that flows, which can cause new trap levels to occur in the pixels. Therefore, when the photoelectric conversion device is used in a surveillance camera that monitors a specific location, the rate of increase in dark current in pixels that receive a large amount of incident light can be greater than the rate of increase in dark current in pixels that receive a small amount of incident light. In other words, the black level can vary from pixel to pixel.

[0083] Therefore, in the photoelectric conversion device shown in embodiment 3, a second signal value is acquired for each pixel, thereby performing correction to reduce the difference in black level for each pixel. Note that the functional configuration and hardware configuration of the photoelectric conversion device are basically the same as those of embodiment 1, and differences will be explained below.

[0084] FIG. 11 is a flowchart illustrating signal processing for black level correction according to the third embodiment. First, in S600, the control unit 302 determines whether the correction value needs to be updated. If the correction value needs to be updated, the process proceeds to S601. If not, the process proceeds to S605. In S601, the imaging unit 301 inserts a light-blocking mechanism into the optical path of the imaging optical system. Next, in S602, the signal processing unit 303, while the light-blocking mechanism remains inserted in the optical path of the imaging optical system, acquires a first signal value from the signal of the OB pixel region using the signal acquisition unit and stores the first signal value in the storage unit 305. Next, in S603, while the light-blocking mechanism remains inserted in the optical path of the imaging optical system, the processing unit 303, while the light-blocking mechanism remains inserted in the optical path of the imaging optical system, acquires a second signal value from the signal of the effective pixel region using the signal acquisition unit and stores the second signal value in the storage unit 305. At this time, the signal processing unit 303 acquires not only the average value of multiple pixels in the effective pixel region as the second signal value, but also a sixth signal value for each pixel in the effective pixel region, and stores these values ​​in the storage unit 305 as well. Thereafter, the signal processing unit 303 determines a fourth signal value by subtracting the first signal value from the second signal value in 6404. This is the preparation stage.

[0085] Next, the correction stage will be described. In S605, the control unit 302 determines whether or not to start imaging. If imaging is to be started, the process proceeds to S606, and if imaging is not to be started, the process ends. In S606, the imaging unit 301 removes the light blocking mechanism from the optical path of the imaging optical system and starts acquiring a captured image. In S607, the signal processing unit 303 obtains a third signal value from the signal of the OB pixel region while acquiring the captured image. Then, in S608, the signal processing unit 303 determines a correction value from the sixth signal value, the fourth signal value, and the third signal value. Specifically, the correction value C is calculated by multiplying the sixth signal value L by the sixth signal value L. D(X,Y) , the fourth signal value L OFF and a third signal value L OB is used to calculate as follows: L B_COR =L B(X,Y) -C1-C2(X,Y) (Number 14) C1=L OB +L OFF ×α1 (number 15) C2(X,Y)=L D(X,Y) ×α2 (Number 16)

[0086] Here, α1 and α2 are correction coefficients for correcting the difference between the image acquisition conditions in the preparation stage when the light blocking mechanism is inserted and the image acquisition conditions in the correction stage when the light blocking mechanism is removed. As described above, by using not only the third signal value and the fourth signal value but also the sixth signal value for each pixel in the effective pixel area, it is possible to correct the deviation in the black level while taking into account the change in dark current for each pixel. In S609, the control unit 302 determines whether or not to continue capturing images. If capturing images is to be continued, the process returns to S607.

[0087] Although the sixth signal value is calculated for each pixel, it may be calculated for each area obtained by cropping the image area to a predetermined size. In this case, the image processing load can be reduced compared to calculating for each pixel.

[0088] <Embodiment 4: Photoelectric Conversion System> 12 shows a system block diagram of a photoelectric conversion system using the photoelectric conversion device according to Embodiments 1 to 3. The photoelectric conversion system 800 includes a photoelectric conversion device 300 having a photoelectric conversion element 100, a control unit 801, a storage unit 802, and a communication unit 803.

[0089] The photoelectric conversion element 100 captures an optical image formed by the imaging optical system 301. The signal read from the photoelectric conversion element 100 is subjected to image generation, first correction processing, second correction processing, black level correction, gamma curve adjustment, noise reduction, data compression, and other processing in the signal processing unit 303, and a final image is generated. If the photoelectric conversion element 100 has an on-chip RGB color filter, it is further preferable to perform processing such as white balance correction and color conversion.

[0090] The control unit 801 has a built-in CPU as a computer, and functions as a control means that controls the operation of each part of the entire photoelectric conversion device 800 based on a computer program stored in a memory as a storage medium. In addition, the control unit 802 controls the length of the exposure period of each frame of the photoelectric conversion element 100 via a control pulse generation unit of the photoelectric conversion element 100, and controls the insertion and removal of the light blocking mechanism 302.

[0091] The recording unit 803 includes a recording medium such as a memory card, a hard disk, etc. The communication unit 804 has a wireless or wired interface, and outputs the generated image to the outside of the photoelectric conversion device 800 and receives signals from the outside.

[0092] <Other embodiments> Note that a computer program that realizes part or all of the control in this embodiment and the functions of the above-described embodiment may be supplied to the photoelectric conversion device via a network or various storage media. Then, a computer (or a CPU, MPU, etc.) in the photoelectric conversion device may read and execute the program. In this case, the program and the storage medium storing the program constitute the present invention. [Explanation of symbols]

[0093] 100 Photoelectric conversion element 11 Sensor board 12 pixel area 13 Effective pixel area 14 ОB pixel area 21 Circuit Board 22 Circuit area

Claims

1. a photoelectric conversion means having a plurality of pixels, each of which is constituted by an avalanche photodiode, and each of which is composed of effective pixels for receiving light and optical black pixels for blocking light; a determination unit that determines a correction value for correcting the signal value of the effective pixel based on the signal obtained from the optical black pixel; a correction unit that corrects the signal value of the effective pixel based on the correction value, The determining means a difference between a first signal value indicating a signal value of the optical black pixel in a state where a light-shielding portion is inserted into the optical path of the imaging optical system and a second signal value indicating a signal value of the effective pixel in a state where the light-shielding portion is inserted into the optical path of the imaging optical system; a third signal value indicating a signal value of the optical black pixel in a state where the light-shielding portion is removed from the optical path of the imaging optical system; and a correction coefficient for correcting a difference between a first image acquisition condition in a state where the light-shielding unit is inserted into the optical path of the imaging optical system and a second image acquisition condition in a state where the light-shielding unit is removed from the optical path of the imaging optical system; The photoelectric conversion device according to claim 1, wherein the correction value is determined based on the above.

2. further comprising a detection means for detecting an image acquisition condition of the photoelectric conversion device; When the detection means detects that the first image acquisition condition in a state where the light-shielding unit is inserted into the optical path of the imaging optical system is different from the second image acquisition condition in a state where the light-shielding unit is removed from the optical path of the imaging optical system, The photoelectric conversion device according to claim 1, characterized in that the determination means determines the correction value based on a fifth signal value obtained by correcting a fourth signal value indicating the difference between the first signal value and the second signal value using the correction coefficient, and the third signal value.

3. 3. The photoelectric conversion device according to claim 2, wherein the first image acquisition condition and the second image acquisition condition indicate at least one of an integration time for integrating the number of photons incident on a pixel of the photoelectric conversion means and a digital gain by which the integrated number of photons is multiplied.

4. 3. The photoelectric conversion device according to claim 2, wherein the first image acquisition condition and the second image acquisition condition indicate temperatures of the photoelectric conversion means.

5. the photoelectric conversion means has an on-chip color filter; 2. The photoelectric conversion device according to claim 1, wherein the determining means obtains the first signal value, the second signal value, and the third signal value for each color channel.

6. 3. The photoelectric conversion device according to claim 2, wherein the determining means determines the correction value using the correction coefficient calculated using the second signal value at a first temperature and the second signal value at a second temperature higher than the first temperature.

7. A photoelectric conversion device as described in Claim 6, characterized in that the shading portion is removed and a normal captured image is output between the acquisition of the second signal value at the first temperature and the acquisition of the second signal value at the second temperature.

8. 7. The photoelectric conversion device according to claim 6, further comprising a control means for controlling so that the first signal value and the second signal value are acquired under execution conditions set by a user.

9. If the execution condition is time, 9. The photoelectric conversion device according to claim 8, wherein the control means controls the acquisition of the first signal value and the second signal value using a time set by a user as a trigger.

10. the control means determines whether the temperature of the photoelectric conversion means at a time set by a user is equal to or higher than a threshold value; The photoelectric conversion device according to claim 9, characterized in that the determination means acquires the second signal value at a high temperature when the temperature is equal to or higher than the threshold, and acquires the second signal value at a low temperature when the temperature is lower than the threshold.

11. 2. The photoelectric conversion device according to claim 1, wherein the determining means determines the correction value by further using a signal value of each of the effective pixels in a state where the light-shielding portion is inserted into the optical path of the imaging optical system.

12. A method for controlling a photoelectric conversion device having a photoelectric conversion means having a plurality of pixels, each of which is made up of an avalanche photodiode, and each of which is made up of effective pixels that receive light and optical black pixels that are shielded from light, comprising: a determination step of determining a correction value for correcting the signal value of the effective pixel based on the signal acquired from the optical black pixel; a correction step of correcting the signal value of the effective pixel based on the correction value, The correction value is a difference between a first signal value indicating a signal value of the optical black pixel in a state where a light-shielding portion is inserted into the optical path of the imaging optical system and a second signal value indicating a signal value of the effective pixel in a state where the light-shielding portion is inserted into the optical path of the imaging optical system; a third signal value indicating a signal value of the optical black pixel in a state where the light-shielding portion is removed from the optical path of the imaging optical system; and a correction coefficient for correcting the difference between a first image acquisition condition in a state where the shading portion is inserted into the optical path of the imaging optical system and a second image acquisition condition in a state where the shading portion is removed from the optical path of the imaging optical system.

13. A computer program for controlling each means of the photoelectric conversion device according to any one of claims 1 to 11 by a computer.

Images