Non-uniformity correction method

Abstract

The present invention provides a method that enables the calculation of non-uniformity correction coefficients for an imaging device, a non-uniformity correction method using the obtained coefficients, and an imaging device implementing this method. The method in question includes the steps of obtaining an average output value function for raw output values of at least F homogeneous images, each with different light intensity values; obtaining a desired output value function by expanding the obtained function to a range between zero and a predetermined desired value; calculating correction coefficients for each pixel element by fitting a sigmoid function to this function.

Description

[0001] NON-UNIFORMITY CORRECTION METHOD

[0002] Relevant Technical Field

[0003] The present invention relates to the technical field of image data processing, and in particular to a non-uniformity correction (NUC) method developed for imaging devices.

[0004] State of the Art

[0005] Non-uniformity Correction (NUC) is an important application used in imaging systems, especially infrared and thermal cameras. This application is a process that reduces the response differences exhibited by the pixel elements in an imaging system and improves the image quality. Response differences can occur due to reasons such as differences in the sensor's manufacturing process, environmental effects, effects of electro-optic components and I or aging of sensor components. The aim of the NUC process is to ensure that each pixel in the sensor responds equally to the same light intensity, thus obtaining more homogeneous and less noisy images. There are various NUC methods in the state of the art. These methods are divided into two main categories: calibration-based and scene-based (non-calibration).

[0006] In calibration-based methods, the camera is calibrated using a homogeneous photon source. In this process, images are taken at different light intensity (or temperature) levels and the response of each pixel to homogeneous conditions is recorded, and then, based on this data, correction coefficients are calculated for each pixel. For example, in the method using the linear response model, these coefficients include pixel-based gain and offset values and are saved in the internal memory of the imaging device to be used in the NUC process. In scenebased (non-calibration) methods, pixel values in the live image are processed according to various criteria (continuity of the image, etc.) to ensure uniformity between pixels. However, it is known that scene-based methods are negatively affected by very bright and very dark areas in the live image.

[0007] The NUC methods in the state of the art are mostly derived from a calibration-based linear response model for the pixel elements contained in the imaging device. This approach is based on the calculation and correction of the non-uniformity in the sensor outputs using a linear model. However, since sensor behaviors are not always linear in reality, these methods are inadequate and do not fully reflect the real sensor behaviors. This deficiency becomes more apparent especially in low I high light conditions and I or scenes with high dynamic range. Therefore, alternative methods that can better model the non-linear behaviors in the sensor outputs are needed. Object of the invention

[0008] The object of the present invention is to develop a calibration-based NUC method that enables the behavior of the pixel elements contained in the display device to be modeled more realistically by using a non-linear model.

[0009] Another object of the present invention is to develop a NUC method with higher accuracy compared to the known applications in the state of the art.

[0010] Definition of the figures

[0011] Graphical representations in accordance with the exemplary applications of the method according to the present invention are shown in the attached figures and from these figures;

[0012] Figure 1 is an exemplary graphical representation of the luminous flux - pixel element raw output value distribution for image data taken during the calibration phase.

[0013] Figure 2 is an exemplary graphical representation of the average raw output value curve versus light flux, obtained by using the data shown in Figure 1.

[0014] Detailed description of the invention

[0015] In order to provide a solution to the technical problems mentioned above, a computer-based method for calculating NUC parameters; a non-uniformity correction (NUC) method using the calculated parameters and an imaging system to which the method is applied are developed with the present invention. With the method in question, the behavior of the pixel elements contained in the imaging device according to the variable light flux is modeled using the sigmoid function. Thus, unlike linear modeling, a modeling closer to the real behavior of the pixel elements is provided.

[0016] The method according to the subject of the invention comprises the following process steps which are applied following the process of taking at least F homogeneous images having different light intensity values (f) with an imaging device: taking at least F images in total for a total of F different light intensities (f=fk) with the value (k) ranging from 1 to F, and therefore taking the pixel-based raw output values [DLj] for each image; calculating the average of the raw output values [DLave] of the pixels of the relevant image for each light intensity value (f) and obtaining an average output value function [DL ave (f)] based on the light intensity (f); determining the maximum value [DLmax] and the minimum value [DLmin] for the average output values [DLave] based on the calculated light intensity; obtaining a desired output value function [DLdes(f)] by expanding the average output value function [DLave(f)] based on the obtained light intensity such that its maximum value is pulled to a desired value [DRdes] and its minimum value is pulled to zero; calculating a first correction coefficient [a] and a second correction coefficient [p] for each pixel element by fitting a sigmoid function to the obtained desired output value function [DLdes(f)]; recording the first correction coefficient [a] and the second correction coefficient [ ] data for each pixel element in the imaging device.

[0017] The method according to the present invention is a computer-based method which is applied after taking images of homogeneous scenes with different light intensities (f) in a laboratory environment with an imaging device and which uses the acquired image data. Figure 1 shows an example graphical representation of the data obtained with the images taken. In the graph shown in Figure 1 , the horizontal axis shows the light intensity value (f), and the vertical axis shows the raw digital output value (DL) of the pixel elements. In the exemplary representation in Figure 1 , the light intensity values range from small to large, between fi and fF, and the measurement values for a total of F homogeneous images are included. Therefore, F also shows the number of calibration measurements taken in the laboratory environment. As seen in Figure 1 , pixel elements give different responses for each light intensity value (fk). First of all, for each light intensity measurement value, the average of the raw output values (DL) of all pixel elements belonging to the image I images recorded under the relevant light intensity is taken. The applied process is shown in the equation below. f = fl'f2fF

[0018] With Equation 1 , the average output value function [DLave(f)] is obtained based on the light intensity (f). Figure 2 shows the average output [DLave(f)] for each light intensity (f). When the average output values [DLave] are combined, an S-shaped curve is expected to form as shown in Figure 2. This curve has a maximum [DLmax] and a minimum [DLmin] value, as shown in Figure 2. Ideally, the minimum value [DLmin] is expected to be zero, and the maximum value [DL max ] is expected to be the maximum possible value according to the imaging device specifications, and this value is defined as the maximum desired output value for the imaging device [DRdes]. For example, in a camera that can output 16 bits, the output data of each pixel is 16 bits. Since the maximum value that 16 bits of data can take is 65,535, the maximum output value desired in such a camera [DRdes] is 65,535. However, in practice, the output value given by pixel elements can be, for example, a minimum of 20.000 and a maximum of 43.000. In this direction, the average output value function [DLave(f)] obtained is modified so that the maximum value [DLmax] is the desired output value [DRdes] for the imaging device in question and the minimum value [DLmm] is zero, and a desired output value function [DLdes(f)] is obtained. Thus, the average output value function [DLave(f)] evolves into the desired output value function [DLdes(f)], which is the ideal situation where the entire dynamic range of the imaging device is used. The process is shown in Equation 2 below.

[0019] DLdese [0,DRdes]

[0020] A first correction coefficient [a] and a second correction coefficient [p] are calculated for each pixel by fitting the sigmoid function [DLdes(f)] to the desired output value function obtained with Equation 2. The process is shown in the equation below.

[0021] The equation on the right side of Equation 3 is a sigmoid equation. The “i” in the equation is the pixel number and the equation is valid for all pixel elements with the same “f’ value. Thus, the desired output value function [DLdes(f)] is defined as a sigmoid equation. The first correction coefficient [a] and the second correction coefficient [ ] in the equation are the pixel-based correction coefficients to be used for the NUC process; by using these coefficients, the raw output values given by each pixel converge to the desired response [DLdes(f)]. After these coefficients are obtained, they are recorded on the imaging device. Later, during the usage process, the camera applies the NUC process to the raw image data using these coefficients.

[0022] In a preferred embodiment of the invention, the expressions in Equation 3 are arranged in order to turn the NUC coefficients into a linear equation and as a result, Equation 4 is obtained.

[0023] In Equation 4, the raw output value [DLj(f)] of the pixel elements for the received image data is known and the desired output value [DLdes(f)] can be calculated as shown in Equation 2. Using this data, the adapted desired output value [DLdes’(f)] can also be calculated.

[0024] As mentioned before, in the first stage, image data is obtained with the imaging device for F different light intensity conditions. Since at least one image is recorded for each light intensity condition, a total of at least F images are obtained. Therefore, there are at least F measurements and two unknowns [a, p] for each pixel element. As long as the F value is equal to or greater than 2, it is theoretically possible to estimate the two unknowns. In this direction, in a preferred application of the invention, the least squares method is applied to make estimates for the unknown coefficients. In this method, an error function is defined and the solution with the smallest sum of the squares of the errors is determined. The applied process steps are as follows: minE iei (fk) (5)

[0025] The error function [ei(f)] defined in Equation 6 is defined as the difference between the output value that should be for the relevant pixel and the output value of the model, and this error value varies for each measurement. When the least squares method is applied to estimate the unknown coefficients, the following result is obtained as the solution that satisfies the condition given in Equation 5.

[0026] The vectors and matrices used in Equation 7 are defined by the following equations. In addition, the i index in the relevant variables indicates the pixel element number and takes values starting from 1 to the total number of pixels.

[0027] As mentioned before, in order to obtain a solution, the F value must be at least 2, and increasing this number reduces the error. Therefore, at least two homogeneous image data with different light intensity values are used in the method in question. The wLSvector in the above equations shows the solution obtained specifically for each pixel with the least squares method. As a result, as many wLSvectors as the number of pixel elements, namely the first coefficients [aLS] and the second coefficients [f>LS], are obtained and recorded in the imaging device for each pixel. Since this recorded data contains two floating point number values for each pixel element, it can be considered as two images in total. Therefore, for the correction method in question, two image-sized data is recorded in the raw memory of the imaging device. All the above-mentioned processing steps are performed by a processing unit, which may be an internal processing unit of the imaging device or an external processing unit.

[0028] The present invention also provides a NUC method in which the first coefficient [aLS] and the second coefficient [fLS] specified above are used. In the said method, when the imaging device takes an image; the raw data [£>L obtained for the relevant pixel is corrected according to the relation given below by Equation 11 [DLNUC i]. The operations specified herein are preferably performed for all pixel elements by an internal processing unit contained in the imaging device. The equation related to the operation performed is given below: The roundQ function in Equation 11 represents the operation of rounding the floating-point number [0, DRdes], which is its argument, to the nearest integer within the closed value range.

[0029] The present invention also provides an imaging device comprising at least one memory unit in which said first coefficient [aLS] and second coefficient [fLS] data are previously recorded for each pixel element it contains and at least one processing unit, which is arranged to apply non- uniformity correction to the images it receives, using said coefficient data in the said memory unit.

Claims

CLAIMS1. It is a computer-based method that enables the determination of non-uniformity correction parameters for an imaging device, comprising the following process steps which are applied following the acquisition of at least F homogeneous images with F different light intensity values (f) with the imaging device: calculating the average [DLave] of the raw output values [DLj] of the pixels of the relevant image for each light intensity value (f) and obtaining an average output value function [DLave(f)] based on the light intensity (f); determining the maximum value [DLmax] and the minimum value [DLmm] for the average output values [DLave] based on the calculated light intensity; obtaining a desired output value function [DLdes(f)] by expanding the average output value function [DLave(f)] based on the obtained light intensity such that its maximum value is pulled to a predetermined desired value [DRdes] and its minimum value is pulled to zero; calculating a first correction coefficient [a] and a second correction coefficient [P] for each pixel element by fitting a sigmoid function to the obtained desired output value function [DLdes(f)]; recording the first correction coefficient [a] and the second correction coefficient [ ] data for each pixel element into the imaging device2. A method in accordance with claim 1 , wherein F is at least two.

3. A method in accordance with any Claim 1 or Claim 2, comprising the least squares method processing step applied to estimate the mentioned first correction coefficient [a] and second correction coefficient [p],4. A non-uniformity correction method using the first correction coefficient [a] and the second correction coefficient [p] obtained by a method in accordance with any one of claims 1 to 3.

5. An imaging device comprising a memory in which said first coefficient [a] and second coefficient [p] data obtained for each pixel element therein by a method in accordance with any one of Claims 1 to 3 are stored and at least one processing unit configured to apply non-uniformity correction using said coefficient data [a, p].

Images