Method and device for correcting raw thermal images

Abstract

Method and Device for Correcting Raw Thermal Images This description relates to a device for processing a raw infrared image (RI), comprising: - a memory (206) configured to store at least one gain (G) and a family of at least one bias image (BI); and - a processor (210) configured to calculate a corrected image (CI) from the raw image, the at least one gain, and a bias image (B) associated with the raw image, wherein the bias image associated with the raw image is generated from the at least one gain (G), the family of at least one bias image (BI), and an image quality metric associated with the corrected image (CI). Figure for the abstract: Fig. 2

Description

Title of the invention: Method and device for correcting raw thermal images. Technical field

[0001] This description relates generally to infrared image processing devices, infrared image sensors, and infrared image processing methods. Previous technique

[0002] Infrared imaging devices, such as bolometers, include an array of detectors sensitive to infrared radiation forming a pixel array.

[0003] Spatial non-uniformity between the pixels of the pixel array, which causes a shift in their values ​​to be corrected in the image, varies over time as a function of the temperature of the focal plane array (TFPA), the acquisition parameters, and the thermal environment. This problem is generally addressed by using an internal mechanical shutter in the imaging device, and involves periodically capturing an image while the shutter is closed in order to obtain a reference image of a relatively uniform scene that can then be used for calibration, for example, by subtraction. However, the use of a shutter has several drawbacks, such as added weight and cost, and the fragility of this component.Furthermore, for some applications, the use of a shutter is unacceptable due to the time lost while the shutter is closed and calibration is performed. During this calibration time, no image of the scene can be captured.

[0004] Image processing techniques for correcting the offset have been proposed as an alternative to using a shutter. There are image processing methods based on a calibration phase during which the noise of the image sensor is measured. There are also methods based on assumptions made about the properties of the acquired image and using, for example, statistical analysis to correct the acquired image.

[0005] However, the known methods are expensive, complex, not very robust to changes in the thermal environment of the image sensor and sometimes have a significant computation time.

[0006] There is therefore a need in the technology for an improved method and device for correcting an infrared image. Summary of the invention

[0007] According to one aspect, a device for processing a raw infrared image is provided, comprising: - a memory configured to store at least one gain and a family of at least one bias image; and - a processor configured to calculate a corrected image from the raw image, at least one gain and a bias image associated with the raw image, wherein the bias image associated with the raw image is generated from at least one gain, the family of at least one bias image, and an image quality metric associated with the corrected image.

[0008] According to one embodiment, the device further comprises an infrared camera configured to acquire at least two raw reference images and the raw image and in which the processor is configured to calculate at least one gain from the at least two raw reference images.

[0009] According to one embodiment, the processor is further configured to calculate one from the family of at least one bias image from at least one of the at least two reference raw images.

[0010] According to one embodiment, the processor is further configured to calculate the image quality metric.

[0011] According to one embodiment, the processor is further configured to calculate an estimate of the bias image associated with the raw image by finding a linear combination of at least one bias image that minimizes the image quality metric, according to the following equation:

[0012] B = ^ âiBi such that B eargmin F(G IB-B)

[0013] where n denotes the number of reference bias images in the at least one bias image, denotes a coefficient of the linear combination and Bi corresponds to a bias image of the at least one bias image.

[0014] According to another aspect, a method for processing a raw infrared image is provided, the method comprising: - the recording of at least one gain and a family of at least one skewed image in a memory; and - the calculation of a corrected image from the raw image, the gain and a bias image associated with the raw image, in which the bias image associated with the raw image is generated from at least one gain, the family of at least one bias image, and an image quality metric associated with the corrected image.

[0015] According to one embodiment, the method further comprises the acquisition, by an infrared camera, of at least two raw reference images and of the raw image and the calculation of at least one gain from the at least two raw reference images.

[0016] According to one embodiment, the method further comprises calculating one of at least one bias image from at least one of at least two reference raw images.

[0017] According to one embodiment, at least two raw reference images are captured in a thermally and spatially inhomogeneous environment.

[0018] According to one embodiment, the method further includes a calculation of the image quality metric.

[0019] According to one embodiment, an estimate of the bias image associated with the raw image is calculated by a linear combination of at least one bias image that minimizes the image quality metric, according to the following equation:

[0020] B = 2” =1 &i B; such that B € argmin^^^FiG IB-B)

[0021] where n denotes the number of reference bias images in the at least one bias image, â- denotes a coefficient of the linear combination and Bi corresponds to a bias image of the at least one bias image.

[0022] According to one embodiment, the calculation of the image quality metric is performed according to a Euclidean standard satisfying the following equation:

[0023] yvepf2 Fl(v) = || v|L= JŸ v(xy)2

[0024] where v denotes an image comprising M rows and N columns and it denotes the domain {0, 1, ...,Ml}x{0, 1, ...,TV-1}-

[0025] According to one embodiment, the calculation of the image quality metric is performed according to a semi-norm defined by a norm 2 of the gradient satisfying the following equation: [00261 VvëP Q , F2(v)MVv|| 22 :=Jy |IVv(xy)lll

[0027] where 1 denotes an image comprising M rows and N columns, it denotes the domain {0,1, M-1} x{0, 1, TV-1} and 7: (7XV, 7yV) represents a linear gradient operator for finite differences.

[0028] According to one embodiment, the calculation of the image quality metric is performed according to a total variation defined by the following equation:

[0029] Vv€Pn,

[0030] where v denotes an image comprising M rows and N columns, it denotes the domain {0,1, Ml}x{0, 1, ..., TV-1} and 7: (7xv,VyV) represents a linear gradient operator for finite differences.

[0031] According to another aspect, an uncooled infrared camera comprising the above device is provided. Brief description of the drawings

[0032] These features and advantages, as well as others, will be described in detail in the following description of particular embodiments, given by way of non-limiting example, in relation to the accompanying figures, among which:

[0033] Fig. 1 schematically illustrates an imaging device according to an example of an embodiment of the present description;

[0034] [Fig.2] schematically illustrates, in more detail, an image processing block of the imaging device of [Fig.1] according to an example of an embodiment of the present description;

[0035] [Fig.3] is a flowchart illustrating steps of an image processing method by the imaging device of [Fig.1] according to an embodiment of the present description;

[0036] [Fig.4A] represents an example of a responsive image according to an embodiment of the present description;

[0037] [Fig.4B] represents an example of a shift image according to an embodiment of the present description;

[0038] [Fig.5] represents, in the form of a flowchart, images involved in the process of [Fig.3] according to an embodiment of the present description;

[0039] [Fig.6] represents an example of a raw image corrected by different models and for different temperatures of the focal plane of a camera;

[0040] Fig. 7 represents another example of a raw image corrected by different models and for different temperatures of the focal plane of a camera. Description of the implementation methods

[0041] The same elements have been designated by the same reference numerals in the different figures. In particular, the structural and / or functional elements common to the different embodiments may have the same reference numerals and may have identical structural, dimensional and material properties.

[0042] For the sake of clarity, only the steps and elements useful for understanding the described embodiments have been shown and are detailed. In particular, methods for acquiring infrared images are assumed to be known to a person skilled in the art.

[0043] Unless otherwise specified, when referring to two elements connected together, this means directly connected without intermediate elements other than conductors, and when referring to two elements connected (in English "coupled") together, this means that these two elements can be connected or linked through one or more other elements.

[0044] In the following description, when referring to absolute positional qualifiers, such as the terms "front", "back", "top", "bottom", "left", "right", etc., or relative, such as the terms "above", "below", "superior", "inferior", etc., or to orienting qualifiers, such as the terms "horizontal", "vertical", etc., it refers, unless otherwise specified, to the orientation of the figures.

[0045] Unless otherwise specified, the expressions "approximately", "about", "Approximately" and "on the order of" mean within 10%, preferably within 5%.

[0046] Although some of the embodiments of this description are described in relation to a microbolometer-type pixel array, those skilled in the art will note that the methods described herein also apply to other types of infrared imaging devices, including cooled devices.

[0047] Fig. 1 schematically illustrates a 100 infrared imaging device according to an example embodiment.

[0048] The 100 infrared imaging device is for example a camera, a photo camera, a mobile phone, an electronic tablet, etc.

[0049] The infrared imaging device 100 comprises a matrix 102 of pixels sensitive to infrared light. For example, in some embodiments, the pixel matrix is ​​sensitive to long-wavelength infrared light, such as light having a wavelength between 7 and 13 μm.

[0050] To simplify the illustration, a 102-pixel matrix of only 144 pixels, arranged in 12 rows and 12 columns, is shown in [Fig. 1]. In other embodiments, the 102-pixel matrix can comprise any number of rows and columns of pixels. Typically, the matrix comprises, for example, 640 by 480, or 1024 by 768 pixels.

[0051] Each column of pixels in the matrix 102 is associated with a corresponding reference structure 106. Although not functionally an imaging element, this structure will be called here a "reference pixel" by structural analogy with the imaging (or active) pixels 104. In addition, an output block 108 ("OUTPUT") is coupled to each column of the pixel matrix 102 and to each reference pixel 106, and provides a raw image IB.

[0052] A control circuit 110 provides, for example, control signals to the pixel matrix, to the reference pixels 106, and to the output block 108. The raw image IB is, for example, provided to an image processing block 112, which applies biases and gains to the pixels of the image to produce a corrected image IC.

[0053] The imaging device 100 is, for example, configured to acquire the raw image IB in the infrared. The temperature of the focal plane of the imaging device 100 is, for example, at room temperature and varies with the ambient temperature. The imaging device 100 is, for example, not cooled or maintained at a constant temperature. The imaging device 100 includes, for example, a temperature probe in its focal plane to measure the temperature of the focal plane. The imaging device 100, for example, does not include a shutter. In this case, the image processing performed by the image processing unit 112 corrects the raw infrared image IB sufficiently to avoid the need for a shutter. In other embodiments, the imaging device 100 includes a shutter, and in this case, the image processing performed by the image processing unit 112 reduces, at least partially, the need for the shutter.

[0054] Fig. 2 schematically illustrates, in more detail, the image processing block 112 of the imaging device 100 of Fig. 1 according to an example embodiment.

[0055] The functions of the image processing block 112 are, for example, implemented by software, and the image processing block 112 includes a data processing device 202 ("PROCESSING DEVICE") comprising one or more processors 210 (CPU, from the English "Central Processing Unit") controlled by instructions stored in an instruction memory 204 ("INSTR MEM"). In other embodiments, the functions of the image processing block 112 may be implemented at least partially by dedicated hardware. In this case, the data processing device 202 includes, for example, an ASIC (Application Specified Integrated Circuit) or an FPGA (Field Programmable Gate Array), and the instruction memory 204 may be omitted.

[0056] The processing device 202 receives the raw input image IB, and generates the corrected image IC, which is for example provided to a display (not shown) of the imaging device 100. The data processing device 202 is also connected to a data memory 206 (“MEM”) storing at least one image of a gain G and a set of one or more reference bias images Bl, B2, ..., Bn forming a family BF of n bias images described in more detail below.

[0057] According to a variant of the embodiment illustrated in Figures 1 and 3, the image processing unit 112 is not connected to the imaging device 100. The raw image IB is, for example, acquired by the imaging device 100 and sent to the image processing unit 112, for example via a wired or wireless connection. The image processing unit 112 is, for example, a computer performing image processing to generate the corrected image IC.

[0058] Figure 3 is a flowchart illustrating steps in an image processing method 300 using the imaging device 100 of Figure 1 according to an embodiment of the present description. This method is implemented, for example, by the image processing block 112.

[0059] It is assumed that a raw image IB was captured by the pixel matrix 102 of [Fig.1].

[0060] The raw image IB is written, for example, in the form:

[0061] [Math.l] V(x,y)ۮ , IB(x,y) = R(x,y)-S(x,y) + O(x,y) # (1)

[0062] where x and y denote the coordinates of a pixel of an image comprising M columns and N rows, Q denotes the domain Q={0,l,...,Ml}x{0,l,...,Nl], p denotes a responsive image, S denotes a scene image, represented in irradiance, and O denotes an intensity offset image.

[0063] Fig. 4A represents an example of a 400 responsive image according to an embodiment of the present description.

[0064] The responsiveness of a pixel corresponds to the sensitivity of the voltage response of that pixel to an received thermal flux. It depends, for example, on the optics and the size of the microbolometer that makes up that pixel.

[0065] The responsiveness image 400 is, for example, the responsiveness image p of the imaging device 100 of [Fig. 1], for example, an uncooled infrared camera without a shutter. The responsiveness image 400 is, for example, generated from two images. The first image corresponds, for example, to a first uniform scene, for example, a black body at a first temperature, for example, between 0°C and 20°C. The second image corresponds, for example, to a second uniform scene, for example, a black body at a second temperature different from the first temperature, and for example, between 30°C and 50°C.

[0066] The 400 responsive image exhibits slow spatial variations.

[0067] Fig. 4B represents an example of a 450 offset image according to an embodiment of the present description.

[0068] The 450 shift image is the O shift image of the imaging device 100 of [Fig. 1], for example, an uncooled infrared camera without a shutter. The 450 shift image is, for example, generated from two images. The first image corresponds, for example, to a first uniform scene, for example, a black body at a first temperature, for example, between 0°C and 20°C. The second image corresponds, for example, to a second uniform scene, for example, a black body at a second temperature different from the first temperature, for example, between 30°C and 50°C.

[0069] The offset is an additive voltage resulting from all parasitic heat fluxes, i.e., those not originating from the scene, present during acquisition. Due to the electronics involved, the offset in a pixel (x, y) depends strongly on its columnar positioning x; in other words, an offset image has a column effect that makes it highly irregular, meaning that pixels of the same The columns of the image with a 450 offset have similar values. The image with a 450 offset is spatially irregular.

[0070] With reference to Figure 3, the scene image S corresponds to a noise-free image and the responsivity image P and the shift image O correspond to the noise added by the imaging device 100 to the scene image S.

[0071] By noting y ) _ v(xy ) 'am°yenne of an image v and in posing q _ Pæ) the corrected image IC is written:

[0072] [Math.2] V(x,y)GD , IC(x, y) = (IB(x, y)-O(x, y)j - G(x, y)# (2)

[0073] By setting B = GOp(GO) as a bias image of the raw image IB, equation (2) can be rewritten in the form:

[0074] [Math.3] V(x,y)eO , IC(x,y) = G(x,y) -IB(x,y)-B(x,y) = (i(P).S(x, y)+p(GO) # (3)

[0075] The corrected image IC has an affine dependence on the scene image S. The image processing method 300 aims for example to generate the corrected image IC from the raw image IB.

[0076] Furthermore, it is assumed that at least one gain image G and the family of bias images BF are available.

[0077] Each of the at least one gain G image is, for example, generated based on at least two raw IBC reference images, for example, acquired during a calibration phase. The raw IBC reference images are, for example, image pairs consisting of a first image of a first blackbody at a first temperature and a second image of a second blackbody at a second temperature, both images having been acquired in a thermally spatially homogeneous and time-stabilized environment. The image pairs are, for example, acquired for focal plane temperatures of the imaging device that are different from each other. The raw IBC reference images are, for example, obtained during an initial phase preceding the acquisition of the raw IB image.

[0078] According to another embodiment, the raw IBC reference images are captured in a thermally spatially inhomogeneous environment.

[0079] The BF bias image family is, for example, generated using at least one of the raw IBC reference images captured during the calibration phase. Each of the B1, B2, ..., Bn reference bias images is, for example, calculated by multiplying a raw IBC reference image pixel by pixel by at least one image with gain G and then subtracting its spatial mean value. The BF bias image family can, for example, be enriched at any time by adding new images of Reference bias. Recording metadata associated with the raw IBC reference images, such as focal plane temperature, is not necessary for calculating the reference bias images Bl, B2, Bn. The model thus takes into account variations in corrections due to various measurement parameters such as focal plane temperature or external causes such as parasitic heat fluxes.

[0080] According to one embodiment, new raw reference IBC images are captured subsequently, after the calibration phase, to recalculate and update the at least one gain G image and the bias BF image family, for example, with raw reference IBC images acquired over a range of focal plane temperatures of the imaging device 100 that differs from the range of focal plane temperatures of the imaging device 100, for example, including the focal plane temperature of the imaging device 100 at the time of acquisition of the raw IB image to be more adapted to the current thermal environment and thus more accurate. The at least one gain G image and the bias BF image family are updated, for example, to compensate for fluctuations over time, for example, caused by wear of the imaging device 100.

[0081] At least one image of gain G is assumed to be independent of the temperature of the focal plane during the acquisition of the raw image IB.

[0082] In the example in [Fig. 3], it is assumed that the at least one gain image G comprises a single gain image G. In other embodiments, the at least one gain image G comprises several gain images, and each of the reference bias images B1, B2, ..., Bn is, for example, calculated from a gain image selected from among the several gain images. Those skilled in the art will be able to adapt the following calculations accordingly. For example, since the biases of the calibration family have been calculated with their respective gains, there are at most as many gains as biases, and it is necessary to add a gain selection step for the current raw image IB to be corrected, for example, deterministically based on the current electronic parameters of the sensor.

[0083] During a step 320 (“GENERATE BIAS IMAGE”), the estimated bias image B for the raw image IB is generated by the image processing block 112.

[0084] The image of the estimated bias B is an element of the vector space Vect(BF) spanned by the family of bias images BF. The image of the estimated bias B can therefore be written according to the following equation:

[0085] [Math.4] B = (4)

[0086] where â2 denotes a coefficient of a decomposition of B in BF, with € Pn and B; corresponds to a bias image of the family of bias images BF.

[0087] It is further assumed that the image of the estimated bias B is generated in such a way as to obtain the corrected image IC based on an image quality metric F, and for example, to maximize the image quality in the sense of the metric F. The metric F is associated with the corrected image, and for example, the metric F is correlated with the spatial regularity of the corrected image (IC). By convention, the image quality metric F is assumed, for example, to decrease with image quality; in other words, the quality of an image v will be greater the smaller F(v) is. We therefore seek to minimize F(IC), that is, the image of the corrected image IC by the metric F.

[0088] The image of the estimated bias B satisfies, for example, the following equation:

[0089] [Math.5] B eargmin^ IB-B).# (5) BeVect(BF) / \ z

[0090] According to a first embodiment, a metric Fl is used as the metric F. The metric Fl is a Euclidean norm that satisfies the equation:

[0091] [Math.6] , Fl(v) = ]| “v* tj”

[0092] where v is an image of domain G and ||. || is the L2 norm.

[0093] In this embodiment, equation (5) has a unique corresponding solution to the orthogonal projection of the pixel-by-pixel product of the gain image G and the raw image IB, denoted G'IBB, onto Vect(BF). This unique solution also satisfies the following equation:

[0094] [Math.7] B = Bt for all a ∈ ar gmin^ || G • IB - Bj || # ( 7 )

[0095] where = J corresponds to the coefficients of a decomposition of B in BF and q = @^correspond to coefficients of a combination linear of the reference bias images Bl, B2, ..., Bn.

[0096] The calculation of the coefficients satisfying this last equation is for example carried out explicitly from the pixel-by-pixel product of the gain image G and the raw image IB, denoted G " IB, and the reference bias images Bl, B2, ..., Bn. This calculation involves the inversion, in the least squares sense, of a Gram matrix of the family of bias images BF.

[0097] When the BF family is linearly independent, it forms a basis of Vect(BF) and the coordinates âj of B in the basis BF are defined in such a way unique. Otherwise, the image of the estimated bias B admits, for example, several decompositions according to the images of the bias family BF.

[0098] According to one embodiment, the Gram matrix is, for example, arbitrarily ill-conditioned or non-invertible if the bias family BF is related. The bias family BF is, for example, orthonormalized, for example by performing a singular value decomposition, or by using the Gram-Schmidt algorithm, to construct an orthonormal basis BF'={B1',B2', .. .,Bm'}, with m less than or equal to n, of Vect(BF). Since the families BF and BF' both generate the same vector space (i.e., Vect(BF)=Vect(BF')), the estimated bias image B also corresponds to the orthogonal projection of the term-by-term product of the gain image G and the raw image IB onto Vect(BF'), which can be calculated explicitly and simply using the following equation:

[0099] [Math. 8] 3 = ^0 G-IB, BJ B>(8)

[0100] where < ' ' ' > denotes the canonical dot product, the terms < g . > , for 1 < i < m, correspond to the coordinates of B in the orthonormal basis BF' and whose evaluation poses no numerical difficulty.

[0101] This first Fl metric has the advantage of being simple and inexpensive to implement.

[0102] According to a second embodiment, an F2 metric is used as metric F. The metric F2 is a semi-norm that satisfies the equation:

[0103] [Math.9] VveP°, F2(v)MVv|| 22 :=Jÿ“^^

[0104] where v is an image of domain G and V : V «-» ( VXV, V yV) represents a linear gradient operator at finite differences and || || 2 corresponds to the Euclidean norm in p2.

[0105] By noting VBF = {VBp VB2, and by linearity of V, the image of the bias The estimated B is a solution of equation (5) if and only if its finite difference gradient 7B is equal to the orthogonal projection of the signal V ( G • IB) onto Vect( VBF), that is, if and only if:

[0106] [Math. 10] VB = argmin X7B&Vect ^ BF) || V ( G • IB ) - VB || 2 2 # ( 10)

[0107] where VB is an element of Vect(VBF).

[0108] This orthogonal projection is uniquely defined and can be expressed, for example, in terms of decomposition in the VBF generating family of Vect(VBF). Thus, we have:

[0109] [Math. 11] = for all Seargmïn^lIvCG-IB)-2^¾ #(H)

[0110] By linearity of the operator V and assumption of zero spatial mean on the biases Bp the solution B — ai gminBeVect^BF^F2 (G • IB-B) is defined by coefficients a solutions of equation (11). The coefficients from (11), substituted into (4), give the solution of (5) when F=F2.

[0111] The calculation of the coefficients a involves, for example, the inversion, in the least squares sense, of the Gram matrix of VBF.

[0112] According to one embodiment, VBF is orthonormalized, for example by performing a singular value decomposition or by using a Gram-Schmidt orthonormalization procedure, to construct an orthonormal basis VBF'=(VB1', VB2',... , VBp'), with p less than or equal to n, of Vect(VBF). Once the coordinates of y B in the basis VBF' have been calculated, the coefficients a of a decomposition of g B in VBF, and therefore of a decomposition of B in BF, can be recovered, for example by a change of basis.

[0113] Using the F2 metric, the search for the B bias image is optimized so that two neighboring pixels of the corrected image have relatively close values ​​and a relatively low gradient.

[0114] One advantage of using the F2 metric is that it allows the image of the estimated bias B to be calculated explicitly using inexpensive calculations and it leads to relatively sharp corrected images.

[0115] According to a third embodiment, an F3 metric is used as the F metric. The F3 metric is a total variation satisfying the equation:

[0116] [Math. 12] Vv€P" , ^3(^)=117^1^2:=2,^^^^^(^-7)112 #( 12 )

[0117] where v is an image of domain G.

[0118] Proximal algorithms are for example used to perform a search for minimizers of non-differentiable convex functions and to solve equation (5) for the metric F3.

[0119] According to one embodiment, BF is orthonormalized, for example by performing a singular value decomposition, or by using the Gram-Schmidt algorithm, to construct the basis BF'={Br,B2',. ..,Bm'}. Such an orthonormal basis makes it easy to establish a numerically simple expression for evaluating the projection onto the subspace Vect(BF). In this context, the implementation of proximal schemes to solve equation (5) will be within the competence of a person skilled in the art.

[0120] According to another embodiment, equation (5) is reformulated in terms of finding decomposition coefficients of the image of the estimated bias B in BF, similarly to the embodiment based on the metric F2. The orthonormalization of the family 7gp = {VBj VB2 VBn} and the use Preconditioning techniques for proximal algorithms are used and allow for the establishment of more efficient numerical schemes, without being more complex to implement, than by performing an orthonormalization of BF.

[0121] During step 320 of Figure 3, the estimated bias image B of the raw image IB is generated by the image processing block 112, using the gain image G and the bias image family BF, by solving equations (4) and (5) for the image quality metric F.

[0122] Although three examples of Fl, F2, F3 metrics have been detailed, other image quality measurement metrics could be used.

[0123] During a step 330 (“GENERATE MODIFIED IMAGE”) following step 320, the corrected image IC is generated by the image processing block 112, for example by solving equation (2) using the raw image IB, the gain image G and the bias image B generated in step 320.

[0124] Fig. 5 represents, in the form of a flowchart, images involved in process 300 of Fig. 3 according to an embodiment of the present description.

[0125] During the process 300 described in relation to [Fig. 3], at least two raw reference images 510 (“IBC”) are captured during the calibration phase, for example by a camera. At least two of the raw reference images 510 are used to generate a gain image 520, represented by the gain image G. At least one of the raw reference images 510 is used to generate at least one reference bias image 530, represented by the set of reference bias images B1, B2, ..., Bn forming the bias family BF.

[0126] The raw image 540 (“IB”) is captured, for example by the camera.

[0127] According to one embodiment, the raw image 540 is corrected according to a process of Image processing differs from method 300 in Figure 3 to obtain a new reference bias image Bn+1. The new reference bias image Bn+1 is, for example, added to the bias family BF. In one embodiment, the new reference bias image Bn+1 replaces one or more elements of the bias family BF. The raw image 540 is, for example, an image acquired with a shutter or a blackbody image. The offset image O is, for example, calculated from the raw image 540 using an algorithm known to a person skilled in the art, and not detailed here, and the new reference bias image Bn+1 is calculated using the following equation:

[0128] [Math. 13] B = G - (13)

[0129] A bias image 550, represented by the bias image B, is generated, for example according to step 320 of process 300, based on the gain image 520 and at least two reference bias images 530.

[0130] The corrected image 560 (“IC”) is generated, for example according to step 330 of process 300, based on the bias image 550 and the gain image 520.

[0131] According to one embodiment, the BF bias family is transformed, for example during a dimensionality reduction step, for example by an orthonormalization process, for example by performing a singular value decomposition, for example by using the Gram-Schmidt orthonormalization procedure.

[0132] One advantage of being able to acquire new raw IBC reference images in a non-homogeneous or non-homogeneous and uncontrolled or uncontrolled environment is that the image processing device 100 does not need to be returned to the vendor or manufacturer to update the BF bias family over time. For example, the raw IBC reference images vary in terms of thermal environment.

[0133] Fig. 6 represents an example of a raw image corrected by different models and for different temperatures of the focal plane of a camera.

[0134] An image of a scene is acquired, for example by the imaging device 100 of [Fig.1], for seven focal plane temperatures (“TFPA”): -39.0°C, -24.0°C, -1.0°C, 27.0°C, 38.0°C, 66.0°C and 88.0°C.

[0135] The raw images acquired from the scene are then corrected using a polynomial model of the intensity shift as a function of the temperature of the focal plane (“(a)”), by the method 300 of [Fig.3] using the Fl metric (“(b)”), the F2 metric (“(c)”) and the F3 metric (“(d)”).

[0136] For each of these image processing methods, the same raw IBC reference images are acquired. The raw IBC reference images are obtained, for example, in a stabilized oven, in a thermally spatially homogeneous and time-stabilized environment, and are composed of pairs of a first blackbody at 70°C and a second blackbody at 10°C acquired in 15 different thermal environments corresponding to 15 focal plane temperature measurements distributed in the calibration range [-23°C, 62°C].

[0137] When the temperature of the focal plane of the image to be corrected is outside the calibration range, i.e., for -39.0°C, -24.0°C, 66.0°C and 88.0°C, the images Images corrected using the F2 and F3 metrics offer a level of correction where contours and scene texture are more clearly defined than those corrected using the polynomial model. Images corrected using the Fl metric allow for fairly good contour definition, but texture rendering, such as that of the sky, is degraded by line and column effects.

[0138] In conclusion, in focal plane temperature interpolation at -1.0°C, 27.0°C, and 38.0°C, method 300 provides a correction quality at least similar to the polynomial model. In extrapolation at 39.0°C, -24.0°C, 66.0°C, and 88.0°C, method 300 offers a corrected image of better quality than the polynomial model.

[0139] Fig. 7 represents another example of a raw image corrected by different models and for different temperatures of the focal plane of a camera.

[0140] The image of a scene in [Fig.7] is for example acquired by the imaging device 100 of [Fig.1], for seven focal plane temperatures (“TFPA”): -39.0°C, -24.0°C, -1.0°C, 27.0°C, 38.0°C, 66.0°C and 88.0°C.

[0141] The raw images acquired from the scene are then corrected by the polynomial model (“(a)”) mentioned in relation to [Fig.6] and by the 300 process of [Fig.3] using the Fl metric (“(b)”), the F2 metric (“(c)”) and the F3 metric (“(d)”).

[0142] For each of these image processing methods, the same raw IBC reference images are acquired. The raw IBC reference images are, for example, obtained in a spatially and thermally inhomogeneous environment. Calibration image pairs are acquired in 12 different thermal environments. It should be noted that, due to spatial thermal inhomogeneity, the measured focal plane temperature, in the range [-2 l'C, 43°C], is not representative of the complete thermal environment.

[0143] The IC-corrected images corresponding to focal plane temperatures of -1.0°C, 27.0°C, and 38.0°C are therefore within the calibration range. The images corrected using the F2 and F3 metrics are comparable to, or of better quality than, the images corrected using the polynomial model, for which row and column effects are visible, particularly at -1.0°C.

[0144] When the temperature of the focal plane of the image to be corrected is outside the calibration range, i.e. for -39.0°C, -24.0°C, 66.0°C and 88.0°C, the images corrected using the F metric as defined by Fl, F2 and F3 offer a correction where it is possible to better distinguish the contours and texture of the scene than for the images corrected using the polynomial model.

[0145] In conclusion, for raw IBC reference images acquired in a spatially and thermally inhomogeneous environment, the 300 method offers a The corrected image is of better quality than the polynomial model. The 300 process is therefore more robust to changes in the thermal environment than the polynomial model.

[0146] One advantage of method 300 is that even if the desired estimated bias image B is highly irregular, this is not the case for the corrected image IC that one wishes to calculate. A metric favoring the regularity of the corrected image IC is therefore very unlikely to lead to the estimation of an aberrant estimated bias image B, especially since the number of elements n in the bias image family BF will in practice be significantly smaller than the size, MxN, of the corrected image IC. Disregarding the metadata associated with the elements of the bias image family BF confers a very high degree of robustness to method 300, which allows the BF elements to be used while considering only the quality of the corrected image IC.

[0147] According to one embodiment, the imaging device 100 does not include a shutter. The bias image family BF is generated from the raw reference images IBC, acquired without shutters, during an initial calibration phase or subsequently, and allows the calculation of the estimated bias image B corresponding to the raw image IB. The image processing method 300 allows for continuous and latency-free acquisition of raw images, without interruption to acquire new reference images with a shutter.

[0148] According to another embodiment, the imaging device 100 includes a shutter. The raw IBC reference images are, for example, acquired with the shutter. The number of raw IBC reference images used for the image processing method 300 can be relatively small and does not require periodic acquisition of new raw reference images.

[0149] One advantage of the described embodiments is that a small number of raw reference images is sufficient, thus requiring little memory. Solving the equations presented in Method 300 requires few numerical resources. Method 300 has low algorithmic complexity and is, for example, implemented in an embedded imaging device. According to one embodiment, the generation of the gain image G and the bias image family BF is performed during an initial phase, using an off-board electronic device, and the calculation of the estimated bias image B of the raw image is performed by the embedded image processing device 100.

[0150] The responsiveness image p and the offset image O appearing in equation (1) are highly dependent on the thermal environment, in particular the temperature of the focal plane during the acquisition of the raw image IB. Method 300 includes the generation of reference bias images Bl, B2, ..., Bn to solve equations (3) and (4), which do not require the estimation of the responsiveness image or the image of offset. The reference bias images Bl, B2, ..., Bn, and therefore the 300 process, are less dependent on the thermal environment during the acquisition of the raw reference images Bl, B2, ..., Bn and the raw image IB. In particular, the 300 process makes it possible to extrapolate and generate a good quality IC-corrected image for a focal plane temperature outside the range used during the acquisition of the reference bias images B1, B2, ..., Bn.

[0151] Various embodiments and variations have been described. Those skilled in the art will understand that certain features of these various embodiments and variations could be combined, and other variations will become apparent to those skilled in the art. In particular, the bias image family BF can be represented by a manifold or a vector space, and several bias image families BF are, for example, used to correct the raw image IB, each bias image family BF being used to correct an area of ​​the raw image IB.

[0152] Finally, the practical implementation of the embodiments and variants described is within the reach of a person skilled in the art, based on the functional indications given above.

Claims

Demands

1. A device for processing a raw infrared image (IB), comprising: - a memory (206) configured to store at least one gain (G) and a family of at least one bias image (BF); and - a processor (210) configured to calculate a corrected image (IC) from the raw image, the at least one gain and a bias image (B) associated with the raw image, wherein the bias image associated with the raw image is generated from the at least one gain (G), the family of at least one bias image (BF), and an image quality metric (F, Fl, F2, F3) associated with the corrected image (IC).

2. Device according to claim 1, further comprising an infrared camera configured to acquire at least two raw reference images (IBC) and the raw image (IB) and in which the processor is configured to calculate at least one gain (G) from the at least two raw reference images (IBC).

3. Device according to claim 2, wherein the processor is further configured to calculate one (Bl, B2, ..., Bn) from the family of at least one bias image (BF) from at least one of at least two reference raw images (IBC).

4. Device according to any one of claims 1 to 3, wherein the processor is further configured to calculate the image quality metric (F1, F2, F3).

5. A device according to any one of claims 1 to 4, wherein the processor is further configured to calculate an estimate (^ of the bias image (B) associated with the raw image (IB) by finding a linear combination of the at least one bias image (BF) that minimizes the image quality metric (F), according to the following equation: B~yn B; such that B eargmin^,. IB-B) —j=ii 1 x B£vect{BF) where n denotes the number of reference bias images in the at least one bias image (BF), OLi denotes a coefficient of the linear combination and B; corresponds to a bias image of the at least one bias image (BF).

6. Device according to claim 4, wherein the calculation of the image quality metric (Fl) is performed according to a Euclidean standard satisfying the following equation: Vv€Pn , Fl(v)=\\v\\2 = ^x^Qv(xfy)2 where v denotes an image comprising M rows and N columns and ü denotes the domain {0, 1, ..., M - 1} X {0, 1, ..., N - 1}-

7. Device according to claim 4, wherein the calculation of the image quality metric (F2) is carried out according to a semi-norm defined by a norm 2 of the gradient satisfying the following equation: Vvepn , F2(v)=h7vh2,7:=^ ||7v(xy)ll2 where v denotes an image comprising M rows and N columns, denotes the domain {0, 1, M-1} x{0, 1, and V : VH (SV, VyV) represents a linear gradient operator with finite differences.

8. Device according to claim 4, wherein the calculation of the image quality metric (F3) is carried out according to a total variation defined by the following equation: Vv£P° , F3(v) = ||7v||1?:=y ||Vv(xy)||2 where v denotes an image comprising M rows and N columns, £? denotes the domain {0, 1, ...,Ml}x{0, 1, and V : vh ( V¥V, VyV) represents a linear gradient operator with finite differences.

9. Method for processing a raw image (IB) in the infrared, the method comprising: - recording at least one gain (G) and a family of at least one bias image (BF) in a memory (206); and - calculating a corrected image (IC) from the raw image, the gain and a bias image (B) associated with the raw image, wherein the bias image associated with the raw image is generated from the at least one gain (G), the family of at least one bias image (BF), and an image quality metric (F, Fl, F2, F3) associated with the corrected image (IC).

10. A method according to claim 9, further comprising acquiring, by an infrared camera, at least two raw reference images (IBC) and the raw image (IB) and calculating at least one gain (G) from the at least two raw reference images (IBC).

11. Method according to claim 10, further comprising calculating one (Bl, B2, ..., Bn) from at least one bias image (BF) from at least one of at least two reference raw images (IBC).

12. A method according to claim 10 or 11, wherein at least two raw reference images (IBC) are captured in a thermally spatially inhomogeneous environment.

13. A method according to any one of claims 9 to 12, further comprising a calculation of the image quality metric (F1, F2, F3).

14. A method according to any one of claims 9 to 13, wherein an estimate (Jj) of the bias image (B) associated with the raw image (IB) is calculated by a linear combination of the at least one bias image (BF) that minimizes the image quality metric (F), according to the following equation: B = 2"=A Bi such that B e argmin^^^F(G IB-B) where n denotes the number of reference bias images in the at least one bias image (BF), denotes a coefficient of the linear combination and B; corresponds to a bias image of the at least one bias image (BF).

15. Uncooled infrared camera comprising the device of claim 1.

Images