METHOD AND SYSTEM FOR CORRECTING FIXED NOISE IN AN IMAGE

Abstract

A method for correcting fixed noise in at least one image, the method comprising the following steps: - acquisition (E1) of at least N substantially distinct and regular images by at least one rotating image sensor such that the combination of the N images forms a panorama of a surrounding scene, the fixed noise contained in each of the N acquired images being substantially identical, - determination (E2) of at least one fixed noise correction parameter by a processing module (2) from the at least N acquired images, the at least one correction parameter minimizing a functional, - correction (E3) of the fixed noise contained in each of the N acquired images by the processing module (2) from the at least one fixed noise correction parameter determined. Figure for the abstract: Fig. 2

Description

Title of the invention: METHOD AND SYSTEM FOR CORRECTING FIXED NOISE IN AN IMAGE technical field

[0001] The present invention relates to a method for correcting fixed noise in at least one image. It also relates to a system for correcting fixed noise in at least one image. Prior art

[0002] Although factory-calibrated, infrared cameras are known to exhibit slight deviations in sensor responses. These deviations change slowly over time, from one image capture to the next. However, this change is relatively slow compared to the camera's refresh rate. In the image, these deviations cause a spatially non-uniform response for the same light signal, which appears visually as fixed noise from one image to the next, hence the name "Fixed Pattern Noise" (FPN). This noise is typically modeled as having a fixed component and a multiplicative component with respect to the input signal (offsets and gains). State-of-the-art documents on methods for correcting image non-uniformity are known, such as document US8503821.This document proposes to determine the movement of the scene relative to the sensor between two shots of the same scene and to use the different pixel responses for the same location in the scene to determine said correction.

[0003] Document US9900526 proposes using a visual shutter to calibrate a scene. This visual shutter corresponds to an object that appears almost uniform and covers a large part of the camera.

[0004] US patent US9208542 proposes a method for reducing pixel-level noise in thermal images. This method counts the number of times a pixel is smaller or larger than its neighboring pixels and, accordingly, corrects the pixel in question.

[0005] The methods in question are therefore based either on comparing different views of the same object with different sets of pixels, or on the presence of a physical or virtual shutter, or on assumptions about the regularity of real images. Methods based on the regularity of real images have the problem that the regularity assumption is not well verified everywhere in the image. Some pixels therefore cannot be corrected correctly from a single image. Methods based on the physical shutter have the disadvantage of not exactly correcting the noise observed in practice. Methods based on the virtual shutter involve a This requires a certain complex logic to implement. Furthermore, methods comparing two views of the same object or scene after movement, or using multiple sensors, are quite sensitive to the quality of the matching between the two scenes and the similarity between the images. In addition, the matching algorithm can be computationally expensive.

[0006] The object of the present invention is to resolve at least one of the aforementioned drawbacks. Description of the invention

[0007] This objective is achieved with a fixed noise correction method in at least one image. The method comprises the following steps:

[0008] - acquisition of at least N substantially distinct and regular images by at least a rotating image sensor such that the combination of N images forms a panorama of a surrounding scene, the fixed noise included in each of the N acquired images being substantially identical,

[0009] - determination of at least one fixed noise correction parameter by a module of processing from at least N acquired images, with at least one correction parameter minimizing a functional,

[0010] - correction of the fixed noise included in each of the N images acquired by the module processing from at least one determined fixed noise correction parameter.

[0011] Many environmental parameters can affect sensor non-uniformity, such as small temperature variations. This non-uniformity therefore varies over time. However, it can be considered to remain virtually constant with respect to several consecutive images taken at the sensor's maximum frame rate. It is therefore necessary to update the correction frequently.

[0012] The present invention relies on several images acquired from different scenes taken with the same image sensor in a short period of time. This is achieved with a rotating image sensor that can reconstruct a panorama using images taken during the rotation of the image sensor: each image corresponds to a different piece or sector of the panoramic scene. An acquisition frequency F of the image sensor is higher than a fixed noise change frequency between the acquisition of two images. The acquisition frequency F is sufficient to ensure that any changes in the fixed noise parameters are very small. The invention thus avoids the drawbacks of traditional methods based on regularity, because each fixed noise correction is estimated not from one, but from several acquired images.

[0013] It is also assumed that the acquired images are substantially regular over a majority of the image, and that in each pixel one or more locally regular images can be found among the acquired images. By "Regular" means that the intensity gradient in an image does not change abruptly across the majority of the image. An image is also said to be "locally regular" if the pixel values ​​in a small neighborhood can be described with a simple mathematical formula, such as the equation of a plane, with few errors. Natural images are generally considered to be, with a high probability, regular across a majority of pixels. This corresponds, for example, to flat areas of the image, or areas with slight gradients. High-contrast areas, such as the edges of objects, or highly textured areas, such as a stack of branches, are not considered regular.

[0014] The non-uniformity correction parameters can be re-estimated frequently enough to compensate for variations in sensor non-uniformity. "Variations in non-uniformity" are defined as variations in fixed noise. Given the high image acquisition frequency, it is assumed here that the value of fixed noise in each image is substantially identical. "Substantially identical" means that the fixed noise has a negligible change between the first and last of the N acquired images.

[0015] At least one fixed noise correction parameter determined by the processing module according to the invention minimizes a functional of the form [00161 E(O, G) = O(L fj + htG.O)

[0017] In this functional, f. corresponds to the image i, and corresponds to the image i corrected, that is to say

[0018] 7 = + O J i 7 i

[0019] where G, O and / . are image-sized matrices, and * is the Hadamard product. G and O correspond respectively to a gain and offset matrix.

[0020] The values ​​of matrices O and G are the parameters that are estimated.

[0021] A variable number of parameters can be determined from the N acquired images. When only O is estimated, the invention provides more robust and faster results. When several parameters are estimated, the signal is better represented and the correction is more effective in practice. The values ​​of the matrices O and G are the parameters that are estimated in this case.

[0022] The step of determining at least one fixed noise correction parameter by the processing module may include the following step:

[0023] - determination of a set of at least two correction parameters for the set of the N images acquired by the processing module, the correction parameter set includes a gain value and an offset value.

[0024] In order to minimize the functional, a set of two parameters is directly determined from the N acquired images. The set of N images is taken into account to determine the parameter set. In this implementation, the functional can be defined by:

[0025] Q(a, b) = TV(b)

[0026] and

[0027] h(ao) = || Y || ; + || G-1 || |

[0028] The step of determining at least one fixed noise correction parameter by the processing module can be carried out via a direct implementation.

[0029] Direct implementation may include the following steps:

[0030] - first filtering of the N images,

[0031] - second filtering along the N images.

[0032] Advantageously, the first filtering of the N images may include the use of a high-pass filter.

[0033] A judicious choice of 0 and h allows for a direct implementation, where a series of filtering operations directly obtains the minimum of the functional. At least one correction parameter can be determined in this way. In one embodiment, the judicious choice of £ and h corresponds to

[0034] -a + è[| J

[0035] Which gives I0036! Q{ / f Î;) = 1^( / / )+(0-1)^+0^.

[0037] where ï is the image-sized matrix containing 1 in each cell,

[0038] where F is a high-pass filter on the acquired image, such that the subtraction of the median of neighboring pixels have the value of each pixel,

[0039] and

[0040] h(G, O) = 0 if G = ï otherwise / z (G, O) = + -

[0041] h^G, O) = + oo forces the gain to be one in each pixel without penalize the offset value. We then obtain that the functional is minimized by:

[0042] q _ -mediani=i ^ ctG = l

[0043] where the median operator is understood to be applied independently at each position in the matrix.

[0044] A direct implementation of the minimization of the functional then consists first of filtering the N images according to the high-pass filter F. The high-pass filter corresponds to a spatial filter. Then the results of these filterings are filtered by the median operator along the N images.

[0045] Other similar functionals have direct implementations. By direct implementation, we mean that the result of minimizing the functional is obtained directly without using a functional minimization algorithm.

[0046] Using p(a, b) = ||F(fl) -a4-b|| has the effect of replacing the median with the average operator in the direct implementation. A weighted average is obtained using Qi operators incorporating a different weighting for each image. In the case of the direct implementation using the median operator, the advantage of using the 1-in-6 norm can be seen. Indeed, applying the median to the result of the high-pass filters has the effect of not taking into account scenes where, for the pixel in question, there is high contrast in the scene, and therefore an aberrant response from the high-pass filter. The majority of scenes will have a non-erroneous value at this pixel, and that is therefore what the median returns.

[0047] The step of determining at least one fixed noise correction parameter by a processing module may also include the following step:

[0048] - temporal smoothing of at least one correction parameter common to each of the N images acquired.

[0049] This time smoothing step allows the results of the determined correction parameter(s) to be refined.

[0050] According to yet another aspect of the invention, a system configured for fixed noise correction of at least one image is proposed, the system comprising:

[0051] - at least one rotating image sensor arranged to acquire at least N images visibly distinct and regular, so that the combination of the distinct images forms a panorama of a surrounding scene, the fixed noise included in each of the N acquired images being substantially identical,

[0052] - a processing module arranged and / or programmed to determine at least one fixed noise correction parameter from the N images acquired by the image sensor, at least one correction parameter minimizing a functional, and to correct the fixed noise in each of the N images acquired from said estimation of at least one determined correction parameter.

[0053] According to yet another aspect of the invention, a computer program product is proposed comprising instructions which, when the program is executed by a computer, lead the latter to implement the steps of the process according to the invention. Description of the figures and methods of implementation

[0054] Other advantages and features of the invention will become apparent upon reading the detailed description of implementations and embodiments, which are by no means limiting, and the following accompanying drawings:

[0055] [Fig.1] is a schematic cross-sectional profile view of a system according to the invention in one embodiment.

[0056] [Fig.2] is a flowchart of the process of the invention according to one embodiment.

[0057] These embodiments are in no way limiting; in particular, one may Consider variants of the invention comprising only a selection of features described or illustrated hereafter, isolated from other described or illustrated features (even if this selection is isolated within a sentence including these other features), if this selection of features is sufficient to confer a technical advantage or to differentiate the invention from the prior art. This selection includes at least one preferably functional feature without structural details, and / or with only a portion of the structural details if this portion alone is sufficient to confer a technical advantage or to differentiate the invention from the prior art.

[0058] We will first describe, with reference to [Fig. 1], a system configured for fixed noise correction of at least one image. The system comprises a rotating image sensor 1 arranged to acquire N substantially distinct and regular images, such that the combination of the distinct images forms a panorama of a surrounding scene, the fixed noise contained in each of the N acquired images being substantially identical. In another embodiment, the system may comprise several image sensors, whether rotating or not. The image sensor 1 is rotated to acquire several images of the environment by following a panorama. The acquired images are distinct from one another, that is, each acquired image captures a sector of the panorama. In other embodiments, there may also be images of the same scene or images with strong overlap with other images among the acquired images.

[0059] An image is defined as a set of pixel values ​​measured during a capture by an image sensor 1. A scene corresponds to a physical reality in front of the image sensor 1, which is captured by said image sensor 1, thus creating an image of it. When the image sensor 1 rotates, it observes another scene. In a preferred embodiment, the rotating image sensor 1 corresponds to a rotating infrared image sensor 1. The image sensor is composed of a set of sensors, each corresponding to a pixel. Indeed, depending on the model, the rotating infrared image sensor 1 acquires multiple images to reconstruct a panorama. A panorama therefore corresponds to a set of scenes, or images, placed end to end to produce a 360-degree observation, preferably of at least 90 degrees. The rotating image sensor 1 rotates at a frequency typically from 2 Hz to 0.5 Hz.The image sensor 1 can rotate clockwise and counterclockwise. Preferably, the rotating image sensor 1 acquires images at a minimum acquisition frequency F. F can take values ​​such as 7.5 Hz, 45 Hz, or 64 Hz. The invention may be applied to others. systems moving or rotating the system at a sufficiently high frequency so that several images are acquired according to the embodiment presented.

[0060] The system also includes a processing module (not shown in the figure) arranged and / or programmed to determine at least one fixed noise correction parameter from the N images acquired by the image sensor, the at least one correction parameter minimizing a functional, and to correct the fixed noise in each of the N images acquired from said estimation of the at least one determined correction parameter.

[0061] We will describe, with reference to [Fig.2], a flowchart of the process of the invention according to one embodiment.

[0062] The method for correcting fixed noise in at least one image comprises three steps (E1 to E3). Step E1 corresponds to an acquisition by the image sensor 1 of N substantially distinct and regular images, such that the combination of the distinct images forms a panorama of a surrounding scene, the fixed noise contained in each of the N acquired images being substantially identical. The image acquisition is processed in the same way as shown [Fig. 1].

[0063] Step E2 corresponds to the determination of at least one fixed noise correction parameter by a processing module 2 from the N acquired images, the at least one correction parameter minimizing a functional.

[0064] According to a first embodiment, the processing module 2 is configured to directly determine at least one fixed noise correction parameter by minimizing a functional. The processing module 2 is then configured to estimate a set of parameters for all N acquired images, the parameter set comprising a gain value and an offset value.

[0065] The functional returns a number, often positive definite, which is related to the amount of fixed noise present in the group of N acquired images considered. The objective is to decrease this value to eliminate the fixed noise. The functional varies according to the defined correction parameters (offset, gain, etc.). The value of the functional corresponds to the value of a function related to the amount of noise present in the image to which these corrections are applied. The set of correction parameters determined by the processing module 2 minimizes this functional.

[0066] In practice, the functional often consists of a data-attached term and a regularization term. The choice of these two terms reflects regularity assumptions made about the scenes expected to be observed in practice. Here, a "total variation" term is used as the regularization term, and a quadratic penalty, based on the norm of the estimated gain (minus one) and offset corrections, is used as the data-attached term. The regularization term, based on the total variation, reflects the assumption that the image after correction = G*f + O' should yield an image whose variation The total gain will be low. The data attachment term reflects that the image intensities must be close, and therefore the gain must be close to 1 in each pixel. and the offset close to 0. Thus, correction parameters are applied to the image, here an offset parameter (of order 0) and a gain parameter (of order 1). The parameters that minimize this quantity are sought.

[0067] Here, several images are available (the image index i ranging from 1 to N), and a single set of gain and offset parameters is determined for all the images (the fixed noise is assumed to be constant across this set of acquired images). The total variations are summed for the different images because the discontinuities of all the images must be reduced by the correct fixed noise correction. Other aggregation methods could also be used, such as the minimum or the median.

[0068] We then obtain the functional: [0 ° 69] G) = L^TV(G*f + O) + || O||^+ ||

[0070] Since the fixed noise varies slightly over time, the correction parameters are updated for a new group of images. Each time, the correction parameters are estimated from several images. A "new group" is defined as a set of N images acquired during a rotation by the image sensor. Once the correction has been estimated for N images, it is assumed that, for the following N images, the fixed noise will have changed very little. Therefore, predetermined correction parameters can be allocated and adjusted.

[0071] In this embodiment, or others, in order to minimize the functional and obtain the correction parameters, several techniques can be used, for example:

[0072] - an iterative optimization method,

[0073] - a method with machine learning, where a function is learned for directly produce a good approximation of the solution for minimizing the functional. This function will take the N images as input and either return the correction parameters or directly the N corrected images (the parameter estimation being considered implicit).

[0074] According to a second embodiment, the processing module 2 is configured to determine at least one fixed noise correction parameter from the N images acquired via a direct implementation.

[0075] This embodiment consists of taking each of the acquired images individually and applying a treatment equivalent to a high-pass filter to each of the N acquired images. More precisely, this high-pass filter consists of taking, for each pixel of the acquired image, the median value of its eight direct neighboring pixels and subtracting it from the value of the pixel in question. The result of the high-pass filter on Each image contains several components, such as the natural difference in scene content at each pixel compared to its neighbors, and noise composed of a time-dependent component (fixed noise) and a time-independent component (photon and electronic noise). In other embodiments, a number of alternative filters are possible. For example, for the first determination step, one can use the subtraction of the average of the neighbors or the subtraction of any other low-pass filter (which is equivalent to applying a high-pass filter). Machine learning can also be used to independently predict the correction at each pixel for each image.

[0076] According to the described embodiment, step E2 also includes a filtering step of the N processed images to extract at least one correction parameter common to each of the N acquired images. Filtering significantly reduces the contribution of the observed scenes, which implies that the scenes must be varied. This is the case here since, according to the invention, a rotating image sensor is used, with each acquired image covering a different viewing angle and with little overlap with the previous image. The result of the processing step with a high-pass filter on each acquired image is therefore filtered to produce a single image, i.e., a single estimate at each pixel. More precisely, a median at each pixel of the set of images is calculated, retrieved, and used as the correction parameter.This second filtering significantly reduces the contribution of time-independent noise and observed scenes to each pixel, while preserving fixed noise. The common correction parameter therefore corresponds to an estimate of the fixed noise present in the acquired images. In other embodiments, filters other than the one presented here are possible, such as averaging after removing elements beyond the extreme quantiles or filtering the most frequently occurring value.

[0077] This second filtering step is equivalent to seeking a consensus between the independent estimates of the correction parameters of each image.

[0078] If the number of images used is small and the frequency of variation of the non-uniformity is sufficiently slow, a temporal smoothing of at least one common correction parameter is used in order to refine the results of at least one common correction parameter.

[0079] Step E3 of the process corresponds to the correction of the fixed noise included in each of the N images acquired by the processing module from at least one determined fixed noise correction parameter.

[0080] Regardless of the embodiment, the correction is applied in the same way. This involves applying the determined correction parameter(s) to the N images, that is, multiplying each image by the gain and adding the determined offset. correction can be applied following a factory-preset image sensor calibration, for example.

[0081] Typically, at least one of the means of the device according to the invention described above, preferably each of the means of the device according to the invention described above, is a technical means.

[0082] Typically, each of the means of the device according to the invention described above may include at least one computer, a central processing unit or computing unit, an analog electronic circuit (preferably dedicated), a digital electronic circuit (preferably dedicated), and / or a microprocessor (preferably dedicated), and / or software means.

[0083] Of course, the invention is not limited to the examples just described and many modifications can be made to these examples without departing from the scope of the invention.

[0084] Of course, the different embodiments, features, forms and variants of the invention can be combined with each other in various ways, provided that they are not incompatible or mutually exclusive.

Claims

Claims

1. Method for correcting fixed noise in at least one image, characterized in that the method comprises the following steps: - acquisition (El) of at least N substantially distinct and regular images by at least one rotating image sensor (1) so that the combination of the N images forms a panorama of a surrounding scene, the fixed noise included in each of the N acquired images being substantially identical, - determination (E2) of at least one fixed noise correction parameter by a processing module (2) from the at least N acquired images, the at least one correction parameter minimizing a functional, - correction (E3) of the fixed noise included in each of the N acquired images by the processing module (2) from the at least one determined fixed noise correction parameter.

2. Method according to claim 1, characterized in that the step of determining at least one fixed noise correction parameter by the processing module (2) comprises the following step: - determining a set of at least two correction parameters for all of the N images acquired by the processing module (2), the set of correction parameters comprising a gain value and an offset value.

3. Method according to claim 1, characterized in that the step of determining at least one fixed noise correction parameter by the processing module (2) is carried out via a direct implementation.

4. Method according to claim 3, characterized in that the direct implementation comprises the following steps: - first filtering of the N images, - second filtering along the N images.

5. Method according to claim 4, characterized in that the first filtering of the N images comprises the use of a high-pass filter.

6. Method according to any one of claims 1 to 5, characterized in that the step of determining at least one fixed noise correction parameter by a processing module (2) also comprises the following step: - temporal smoothing of the at least one correction parameter common to each of the N acquired images.

7. System configured for the fixed noise correction of at least one image, the system comprising: - at least one rotating image sensor (1) arranged to acquire at least N substantially distinct and regular images so that the combination of the distinct images forms a panorama of a surrounding scene, the fixed noise included in each of the N acquired images being substantially identical, - a processing module (2) arranged and / or programmed to determine at least one fixed noise correction parameter from the N images acquired by the image sensor, the at least one correction parameter minimizing a functional, and to correct the fixed noise in each of the N acquired images from said estimation of the at least one determined correction parameter.

8. A computer program product comprising instructions which, when the program is executed by a computer, cause the latter to implement the steps of the method according to any one of claims 1 to 6.

Images