Dark current pattern estimation method

Abstract

Dark current pattern estimation method The dark current pattern estimation method of the present invention is a method for estimating a dark current pattern of a two-dimensional sensor for a shutterless uncooled camera, the two-dimensional sensor including a matrix of pixels (x, y), the dark current pattern being divided into a dark current rate and a dark current offset, the method including a step of setting a new temperature, a step of waiting for the new temperature to stabilize, and a step of estimating a dark current pattern of at least two exposure times (t exp ), determining a dark current offset and a dark current rate for each pixel from each averaged image, interpolating the dark current offsets obtained for each pixel, interpolating the dark current rates obtained for each pixel, and determining a dark current pattern with respect to temperature, exposure time, and pixel location.

Description

[Technical Field]

[0001] The present invention relates to a method for estimating a dark current pattern, and more particularly to a method for estimating a dark current pattern of each pixel of a two-dimensional sensor of a shutterless uncooled camera using images previously obtained in a laboratory. [Background technology]

[0002] Dark current is a relatively small amount of electrical current that flows through a photosensitive device, such as a photomultiplier tube, photodiode, or charge-coupled device, even when no photons are incident on the device. Dark current consists of charge generated within the detector when no external radiation is incident on the detector. Physically, dark current results from the random generation of electrons and holes within the depletion region of the device.

[0003] Some 2D image sensors, such as those based on InGaAs (indium gallium arsenide) or microbolometers, have high dark current rates that vary from pixel to pixel, meaning that all acquired images must be corrected by subtracting the appropriate dark current pattern.

[0004] The dark current pattern is highly dependent on the temperature of the sensor and the selected exposure time. In some applications, these variables cannot be fixed, so a typical solution is to capture a dark reference image (with the shutter closed) immediately before the image of interest (with the shutter open). If the temperature is constant, the same dark reference image can be used for multiple images of interest.

[0005] However, there are applications where capturing a suitable dark reference frame is not possible: one example is video capture without temperature control (the shutter is always open), another example is single image capture without temperature control and a mechanical shutter.

[0006] There are methods in the literature that attempt to estimate the dark current, but these methods are considered to be not very robust or versatile, as they attempt to fit laboratory measured data to equations that attempt to model the dependence of the dark current on temperature. Summary of the Invention [Problem to be solved by the invention]

[0007] The present invention has been made in consideration of the above-mentioned problems in the conventional art, and an object of the present invention is to provide a method for estimating a dark current pattern in a two-dimensional sensor for a shutterless uncooled camera. [Means for solving the problem]

[0008] The method of the present invention solves the challenges of shutterless, uncooled imaging by estimating the dark current pattern for each image of interest based on information previously obtained in the laboratory.

[0009] To do this, each pixel of the 2D sensor is first characterized independently as a function of exposure time and temperature, and then a quasi-empirical model based on regression and interpolation is defined, keeping in mind efficient implementation in embedded devices.

[0010] The proposed method of the present invention is useful in applications using sensors with high dark current rates (e.g., InGaAs (indium gallium arsenide)), in situations where temperature cannot be controlled (e.g., due to severe power constraints), and when no mechanical shutter is available.

[0011] To explain this method in detail, the first step is to determine the temperature and exposure time (t exp ) in order to be able to estimate the dark current pattern for any temperature and exposure time combination without further in situ measurements.

[0012] The dark current pattern can be divided into two terms: the dark current rate (ADU / sec) and the dark current offset (ADU), where ADU (Analog to Digital Units) stands for "Analog to Digital Conversion Unit." The offset is t exp = 0, and the dark current rate is the exposure time (t exp ) is modeled. Because the dark current pattern is non-uniform, each pixel must be assigned a different dark current rate and dark current offset. Furthermore, this dark current pattern has a strong (exponential) dependence on temperature.

[0013] In most modern sensors, the dark current pattern varies over a wide range of t exp As a result, the exposure time (t exp ) can be linearly fitted to the dark current pattern as a function of

[0014]

number

[0015] where T is the FPA temperature (K) and (x, y) are the pixel coordinates.

[0016] By performing this fit for a set of FPA temperatures, the temperature dependence of the dark current pattern and dark current offset can be investigated.

[0017] Ideally, the first step of characterizing each pixel involves the following substeps: setting a new FPA temperature; the substep of waiting for the new FPA temperature to stabilize; Various exposure times (t exp ) to obtain the averaged image, repeating the above substeps to obtain samples over the entire temperature range of interest.

[0018] To minimize the effect of temporal noise, it is essential to acquire averaged images rather than single images (i.e., multiple N avg (Images are acquired and then combined into a single image by averaging each pixel coordinate.) The sources of temporal noise are readout noise and dark current noise. Dark current noise increases dramatically with temperature.

[0019] Unfortunately, the above sub-steps are time-consuming in practice. Instead, the first step of this method involves the following simplified sub-steps: Initiating a slow temperature ramp so that when two consecutive averaged images are taken, the images will have approximately the same temperature (the slower the temperature ramp, the more accurate this method is), and setting at least two exposure times (t exp ) alternately and successively acquiring averaged images (note that averaging more samples will result in greater accuracy if the temperature is approximately constant between successive samples).

[0020] Preferably, at least two criteria t exp The criteria for selecting the value are as follows:

[0021] That exp Ensure that the value is within the linear zone of the sensor under investigation (e.g., a zone with a deviation from linearity of <1%) (for many sensors, t exp (Note that =0 is outside the linear zone).

[0022] A relatively short t is used to minimize temperature fluctuations during image acquisition corresponding to the same temperature sample. exp Select a value (the actual exposure time limit depends on the sensor under investigation and the number of image samples selected to be averaged).

[0023] Two T's expThe values ​​should not be too close to each other, as a poor fit will result if the SNR (signal-to-noise ratio) of the averaged image is low for the chosen settings (in most cases a factor of 2 difference in values ​​is sufficient, but the method has been tested with a factor of 10 difference for better accuracy).

[0024] The second step in our method is to define a pseudo-empirical model based on regression and interpolation. Rather than attempting to fit the dark current rate and dark current offset versus temperature, we propose to save a subset of samples (e.g., take one sample out of each "step") and obtain the desired values ​​by spline interpolation (a form of interpolation in which the interpolant is a special type of piecewise polynomial called a spline). That is, instead of fitting one high-order polynomial to all values ​​at once, spline interpolation fits multiple lower-order polynomials to a small subset of values.

[0025] The second step includes the following substeps: interpolating the dark current offsets obtained for each pixel; interpolating the dark current rates obtained for each pixel; A step of obtaining a dark current pattern according to the following equation (1):

[0026]

number

[0027] where T is the FPA temperature (K) and (x, y) are the pixel coordinates.

[0028] Spline interpolation can be performed using linear, quadratic and cubic interpolation (first, second and third order) with natural boundary conditions.

[0029] The error from linear interpolation is significantly larger than that from other methods. Cubic methods are significantly more complex to implement and compute, but do not significantly reduce the error. As a result, quadratic methods are the preferred choice for embedded systems, as they offer a good compromise. For conventional computers, cubic methods may be preferable. [Effects of the Invention]

[0030] Our method for estimating dark current patterns is believed to be more robust and general than other current state-of-the-art methods because it does not require any assumptions about the temperature dependence of the dark current or manual tuning, which means that our method can be more easily applied to other sensor models and technologies. [Brief explanation of the drawings]

[0031] In order to supplement the description and to enable a better understanding of the features of the present invention, a set of drawings are presented as an integral part of the description, in accordance with a preferred example of a practical embodiment of the invention, showing, together with illustrative and non-limiting features, the following:

[0032] [Figure 1A] 10 shows exemplary images before and after subtraction of the dark current pattern estimated by the proposed method. [Figure 1B] 10 shows exemplary images before and after subtraction of the dark current pattern estimated by the proposed method. [Figure 1C] 10 shows exemplary images before and after subtraction of the dark current pattern estimated by the proposed method. [Figure 1D] 10 shows exemplary images before and after subtraction of the dark current pattern estimated by the proposed method. [Figure 2] 1 is a graph showing the dark current measured at a single pixel during an ascending and descending temperature sweep. [Figure 3] 10 is a graph showing the median dark current rate. [Figure 4]10 is a graph showing the median value of the dark current offset. [Figure 5A] 10 is a graph showing examples of pixels with different types of exponential fits to the dark current rate; [Figure 5B] 10 is a graph showing examples of pixels with different types of exponential fits to the dark current rate; [Figure 5C] 10 is a graph showing examples of pixels with different types of exponential fits to the dark current rate; [Figure 6A] 10 is a graph showing measured and estimated dark current rate and dark current offset for various step sizes using cubic interpolation. [Figure 6B] 10 is a graph showing measured and estimated dark current rate and dark current offset for various step sizes using cubic interpolation. [Figure 6C] 10 is a graph showing measured and estimated dark current rate and dark current offset for various step sizes using cubic interpolation. [Figure 6D] 10 is a graph showing measured and estimated dark current rate and dark current offset for various step sizes using cubic interpolation. [Figure 6E] 10 is a graph showing measured and estimated dark current rate and dark current offset for various step sizes using cubic interpolation. [Figure 6F] 10 is a graph showing measured and estimated dark current rate and dark current offset for various step sizes using cubic interpolation. [Figure 7] This image shows the estimation error at 45.88°C, step=20. The error image has been scaled so that black corresponds to -20 ADU and white corresponds to 20 ADU. [Figure 8A] 10 is a graph showing measured and estimated dark current rate and dark current offset for several interpolation types using step=20. [Figure 8B]10 is a graph showing measured and estimated dark current rate and dark current offset for several interpolation types using step=20. [Figure 8C] 10 is a graph showing measured and estimated dark current rate and dark current offset for several interpolation types using step=20. [Figure 8D] 10 is a graph showing measured and estimated dark current rate and dark current offset for several interpolation types using step=20. [Figure 8E] 10 is a graph showing measured and estimated dark current rate and dark current offset for several interpolation types using step=20. [Figure 8F] 10 is a graph showing measured and estimated dark current rate and dark current offset for several interpolation types using step=20. [Figure 9] 1 is a table showing the settings for image acquisition in the first step of the method. [Figure 10] 1 is a table showing fitted constants of the Arrhenius equation. [Figure 11] 10 is a table showing estimation error versus step size. [Figure 12] 10 is a table showing estimation errors versus interpolation type. DETAILED DESCRIPTION OF THE INVENTION

[0033] A preferred embodiment of the dark current pattern estimation method will now be described with reference to Figures 1 to 12. This method uses a two-dimensional sensor, which includes a matrix of pixels (x, y) for a shutterless uncooled camera.

[0034] The first step of this method is to measure each pixel of the two-dimensional sensor with the temperature and exposure time (t exp ) independently as a function of

[0035] The dark current pattern can be divided into two terms: the dark current rate (ADU / sec) and the dark current offset (ADU). The offset is t exp = 0, and the dark current rate is the exposure time (t exp ) is modeled as a linear dependence on the dark current rate and offset. Each pixel must be assigned a different dark current rate and offset. Furthermore, this dark current pattern has a strong (exponential) dependence on temperature.

[0036] To better demonstrate the method of the present invention, a specific two-dimensional sensor was tested.

[0037] The dark current pattern is exp As a result, the exposure time (t exp A linear fit of the dark current pattern as a function of

[0038]

number

[0039] where T is the FPA temperature (K) and (x, y) are the pixel coordinates.

[0040] Ideally, the first step of characterizing each pixel involves the following substeps: setting a new FPA temperature; waiting for the new FPA temperature to stabilize; Various exposure times (t exp ) to obtain an averaged image; Repeat the above substeps if necessary.

[0041] Unfortunately, the above sub-steps are time-consuming in practice. Instead, the first step of this method involves the following simplified sub-steps: initiating a slow temperature ramp; At least two exposure times (t exp ) alternately and successively acquiring averaged images.

[0042] In the test, an ambient temperature ramp was set to change from 0°C to 50°C over approximately 2.5 hours. For each investigated gain, 200 sets of averaged images were acquired, each set corresponding to one temperature sample.

[0043] Figure 9 shows the settings determined for the test based on previous measurements of the linearity of this sensor. exp Gain 2 was actually skipped because acquiring 200 images at 200ms (milliseconds) would take at least 40 seconds, during which the temperature could change significantly.

[0044] Considering that all gain settings were acquired alternately during the same test, excluding gain 2 resulted in more samples with gains 0 and 1, which were more interesting for the application scenario under investigation.

[0045] In order for the dark current pattern to be predictable for a particular temperature and exposure time, there should be no hysteresis in either of these two parameters.

[0046] In this sense, specific tests were conducted to measure the dark current pattern as a function of exposure time (t exp ) (as expected) and was found to be reproducible after increasing and decreasing temperatures. The results of the temperature sweep are shown in Figure 2.

[0047] The fact that no hysteresis was measured gives the green light to develop methods to predict dark current patterns.

[0048] Meanwhile, to better understand the overall behavior of the sensor, the median dark current rate (median of all pixels in the 2D array) was investigated as a function of temperature. It was found that the rate (ADU / s) follows the Arrhenius equation, with the offset being attributable to the offset of the readout electronics.

[0049]

number

[0050] where T is the FPA temperature (K), A is the pre-exponential factor (ADU / sec), and E a is the activation energy (eV), k B is the Boltzmann constant (8.617333×10 -5 eV / K), C is the dark current offset (ADU) of the dark current rate.

[0051] This function was fitted to a temperature sweep at a gain of 0 and a gain of 1. The best fit was found at a gain of 0, where R 2 The obtained parameters are shown in Figure 10, and the dark current rate plot is shown in Figure 3.

[0052] The most common InGaAs sensor composition for short-wave infrared (SWIR) imaging is In0.53Ga0.47As, which has a fundamental gap of 0.73 eV, which is in perfect agreement with the measurements.

[0053] On the other hand, we found that the median dark current offset has a nearly linear dependence on temperature, decaying slowly with increasing temperature (see Figure 4). It is unclear whether this decay is the result of some kind of saturation effect, and a function that better models the temperature dependence of the offset is also unclear.

[0054] Detailed pixel-by-pixel analysis leads to the conclusion that many pixels do not obey the Arrhenius equation, but instead appear to obey a similar equation formed by the sum of up to three exponential terms.

[0055]

number

[0056] where T is the temperature in Kelvin and A through G are constants that define the shape of the curve. All constants (including the dark current rate, of course) depend on the pixel coordinates (x,y).

[0057] In equation (3), the first two terms can be said to represent the diffusion dark current and the depletion dark current, respectively. It is unclear where the third term comes from, or whether it should actually have the same exponential form.

[0058] We have verified that not all pixels require three exponential terms for a good fit. Most pixels require only one exponential, many require two, and only a few pixels are modeled with three exponentials. See Figure 5, which shows examples of pixels with different types of exponential fits to the dark current rate (Figure 5A for one, Figure 5B for two, and Figure 5C for three).

[0059] A similar study was performed on the offset, and it was found that most pixels exhibited behavior similar to the plot in Figure 4. The offset was expected to vary linearly with temperature, but after measurements it is not clear whether the best model consists of a sum of exponentials, a polynomial, or a combination thereof. However, for temperatures up to 60°C, a linear fit seems sufficient.

[0060] Characterization results show that the total number of parameters required to fit the dark current pattern is a minimum of seven (a bi-exponential fit for the dark current rate and a linear fit for the dark current offset). For a 640x512 pixel, single-precision floating-point (32-bit) sensor, the fitted parameters would require 8.75 MBytes of memory.

[0061] On the other hand, fitting a complex function such as that defining the dark current rate requires a lot of manual tuning, such as defining the appropriate starting point and step size so that all pixels converge to the optimal solution. Furthermore, an algorithm that converges for a particular data set may not converge for another data set acquired with the same sensor due to slight differences in such data.

[0062] On the other hand, trying to fit the dark current offset without knowing the actual equation that describes its temperature dependence is not a good idea, and an equation that works for one sensor may not be applicable to another.

[0063] The problems described in the above paragraph suggest that a different, more robust method for estimating dark current patterns would be beneficial. Rather than attempting to fit the dark current rate and dark current offset versus temperature, we propose saving a subset of samples (e.g., one sample per "substep") and using spline interpolation to obtain the desired value.

[0064] Therefore, the method of the present invention includes a second step consisting of defining a pseudo-empirical model based on previously acquired images, which includes a substep of determining the desired values ​​(dark current rate and dark current offset for each pixel) using spline interpolation.

[0065] Cubic interpolation proved surprisingly accurate for a small number of samples across the entire temperature range, as shown in Figure 11. The data in this table was taken from 200 temperature samples. In Figure 11, disk usage is calculated as follows:

[0066]

number

[0067] In these tests, n_params=2 (dark current rate and dark current offset), n_pixels=640x512, n_bytes=4 (single precision floating point), n_samples=200 (one for each measurement temperature).

[0068] The "Error" columns show the absolute median difference between the measured and estimated dark patterns. The term "Training" refers to the training images, i.e., the images used during the dark current fit (in these tests, 6 ms at 51.18°C). On the other hand, "Test" refers to the images with exposure times (t exp ) (for these tests, 2 ms at 45.88°C). All results in Figure 11 were obtained with a gain of 0.

[0069] The magnitude of the error depends primarily on the distance between the target temperature and the nearest multi-temperature sample, as shown in Figure 6 (Figure 6A shows the median dark current rate at step = 10; Figure 6B shows the median dark current offset at step = 10; Figure 6C shows the median dark current rate at step = 20; Figure 6D shows the median dark current offset at step = 20; Figure 6E shows the median dark current rate at step = 40; Figure 6F shows the median dark current offset at step = 40).

[0070] For example, the significantly higher error in the training images at step = 40 shown in Figures 6E and 6F can be explained by the fact that 51.18°C is significantly different from the surrounding samples (40.78°C and 60.48°C). Of course, using a smaller step size reduces the probability of encountering such a situation, but this comes at a cost in computational and memory costs.

[0071] In the present investigation, setting step=20 seems to be a good compromise for embedded systems. Indeed, with step=20, 45.88°C is far from the surrounding samples (40.96°C and 51.16°C), but this does not result in significant errors in the estimated image, as shown in Figure 11 and Figure 7. For a normal computer, step=10 might be a better choice.

[0072] In any case, the estimated error is always less than the read noise itself (approximately 16.8 ADU measured in other tests with this sensor), and if we also take into account the dark noise, the estimated error becomes even less significant.

[0073] Estimation of the dark current rate and dark current offset was tested with three different methods: linear, quadratic, and cubic interpolation (first, second, and third order) with natural boundary conditions.

[0074] The goal is to find the simplest method that gives good results. The estimated error measured in the test setup (2 ms at 45.8 °C, step = 20) is shown in Figure 12. The resulting plots are shown in Figure 8 (Figure 8A: median dark current rate by linear interpolation; Figure 8B: median dark current offset by linear interpolation; Figure 8C: median dark current rate by quadratic interpolation; Figure 8D: median dark current offset by quadratic interpolation; Figure 8E: median dark current rate by cubic interpolation; Figure 8F: median dark current offset by cubic interpolation).

[0075] It can be seen that the error due to linear interpolation is significantly larger than that due to the other methods. This behavior was expected due to the exponential shape of the curve and the fact that the selected FPA temperature is midway between the two reference temperature samples. On the other hand, the cubic method is more complex to implement and calculate, but does not result in significantly smaller errors.

[0076] The results obtained suggest that the quadratic method may be a good compromise for embedded systems. For conventional computers, the cubic method may be preferable.

[0077] Some hotter pixels may reach saturation at high temperatures, and if this situation is ignored during the dark current pattern fit, the dark current rate and dark current offset will not be estimated correctly for these temperatures.

[0078] For example, for very hot pixels, the rate may be reduced or zero, and the offset may be close to the pixel's full well or the upper limit of the ADC (whichever comes first). Even if the dark current rate and dark current offset measurements are incorrect for hot pixels, in most cases the estimated pattern will be correct and the hot pixels will show the expected high values.

[0079] Since hot pixels are typically removed in subsequent processing steps, the estimation error in these pixels is insignificant. The values ​​of pixels close to saturation can be extrapolated to emulate the absence of saturation effects. However, due to noise at high temperatures and the difficulty of fitting the pixels to a known equation, this method was deemed not robust and was discarded.

[0080] This invention describes a method for estimating the dark current pattern of uncooled InGaAs sensors based on preliminary laboratory measurements. The results show that this method is highly accurate (the error is much smaller than the readout noise) over a wide range of exposure times and temperatures.

[0081] As an example of the above, Fig. 1 shows example images before (Fig. 1A and Fig. 1C) and after (Fig. 1B and Fig. 1D) subtraction of the dark current pattern estimated by the proposed method of the present invention. These images were taken from orbit with an InGaAs sensor on a small satellite, and no active temperature control or mechanical shutter was used.

Claims

1. A method for estimating a dark current pattern of a two-dimensional sensor for a shutterless uncooled camera, comprising: the two-dimensional sensor comprises a matrix of pixels (x, y); The dark current pattern is divided into a dark current offset in analog to digital units (ADU) and a dark current rate (ADU / sec); The method comprises: Focal Plane Array (FPA) temperature and exposure time (t ｅｘｐ characterizing each pixel (x,y) independently as a function of setting a new FPA temperature (T); waiting for the new FPA temperature (T) to stabilize; At least two exposure times (t ｅｘｐ ) by acquiring an averaged image with a camera; determining the dark current offset and the dark current rate for each pixel from each of the averaged images by linear adjustment; repeating all of the substeps until a predetermined number of averaged images have been acquired; defining a dark current pattern for the two-dimensional sensor; a sub-step of spline interpolating the dark current offset (offset(T, x, y)) obtained for each pixel; spline interpolating the dark current rate (rate(T, x, y)) obtained for each pixel; a sub-step of determining the dark current pattern for each pixel using the following equation (1): [Equation 1] (where T is the FPA temperature in Kelvin and (x,y) are pixel coordinates)

2. 2. The method of claim 1, wherein the substeps of setting a new FPA temperature (T) and waiting for the temperature to stabilize follow an increasing or decreasing temperature ramp.

3. The at least two exposure times (t ｅｘｐ 2. The method of claim 1, wherein the linearity is selected within a linear zone with a deviation from linearity of less than 1%.

4. The exposure time (t ｅｘｐ 2. The method of claim 1, wherein the value of (i) is short enough to minimize temperature fluctuations during the substep of acquiring the averaged images corresponding to each temperature with a camera.

5. 2. The method of claim 1, wherein the spline interpolation is selected from linear, quadratic, and cubic interpolation.

Images