Method for correcting vignetting in an image sequence using calibration tables

Abstract

Method for correcting a vignetting defect in an image sequence from calibration tables One aspect of the invention relates to a computer-implemented method for correcting a vignetting defect in an image sequence acquired by a camera for a focal length F', the image sequence comprising a plurality of image sub-sequences, the method being carried out from a set of calibration tables corresponding to the vignetting defect and each associated with a specific focal length, at least one specific focal length associated with a calibration table being different from F', the method comprising a plurality of iterations. Figure to be published with the abstract: Figure 2

Description

Title of the invention: Method for correcting a vignetting defect in a sequence of images from calibration tables TECHNICAL FIELD OF THE INVENTION

[0001] The technical field of the invention is that of vignetting correction in image sequences.

[0002] The present invention relates to a method for correcting a vignetting defect in a sequence of images. TECHNOLOGICAL BACKGROUND OF THE INVENTION

[0003] Infrared cameras or cameras detecting visible light included in sights or aircraft for example comprise a system of glass lenses and / or mirrors, the position of the lenses and mirrors being variable to adjust the focal length of the system and magnify the observed objects. In particular, the movement of the lenses makes it possible to continuously vary the magnification and in particular to dynamically adjust the focusing plane.

[0004] However, the movement of mirrors or lenses included in cameras can contribute to time-varying vignetting defects, these defects affecting the quality of images produced in real time. In particular, vignetting defects comprise a predominantly radial component and correspond to a darkening of the peripheries of an image, with external light being attenuated at the edges of the image. Vignetting corresponds to a progressive loss of illumination at the periphery of an image, moving from the center of this image towards its periphery and therefore reduces the quality of the acquired images.

[0005] [Fig.l] is an example of an optical system 100 comprising a cooled infrared optical zoom 101, a detector 102 and a cold screen 103. In this case, the cold screen 103 is the exit pupil of the optical system. The optical zoom 101 comprises in particular a front lens 1011 which is the entrance pupil of the optical system and for which it is sought to obtain a conjugation with the cold screen 103. The conjugation between the two pupils is not perfect and, for reasons of size of the optical zoom 101, it is sometimes necessary to limit the useful diameter, in other words the opening, of the front lens to the passage zone Z of the rays on the optical axis. Therefore, the further the object moves away from the axis, the fewer light rays the image receives: part of the V rays arriving on the front lens and not included in the Z zone is cut and does not reach the detector 102, which is the cause of vignetting.

[0006] There are methods for correcting vignetting of acquired image sequences, from calibration tables corresponding to the shape of the vignetting, the calibration tables also being called calibration tables. The calibration tables are each associated with a vignetting defect for parameters such as a given focal length, a given integration time and a given temperature. However, the calibration tables do not include parameters relating to the scene filmed by the camera and do not make it possible to compensate for the absence of rays causing the vignetting.

[0007] It is possible to correct vignetting by means of a correlation between the high frequencies of an image acquired in real time and the high frequencies of the calibration tables. The estimation of the coefficients from the high frequencies is for example described in patents EP2127359B1 and FR3118558. However, the use of the high frequencies of the images to determine and correct vignetting can introduce defects greater than the original radial defects.

[0008] It is possible to estimate the radial component of vignetting defects by statistical processing on concentric circles and then applying it radially over the entire image as described in patent US9730958B2. However, the residual obtained after correction by a radial or elliptical estimation of the defect remains substantial.

[0009] There is thus a need to correct the vignetting of images acquired in real time in an optimal manner. Summary of the invention

[0010] The invention provides a solution to the problems mentioned above, by considering the vignetting defect as an additive radial and low / medium frequency spatial noise. In particular, the invention highlights this defect in sequences of images to be corrected by removing the high frequencies with a radial filter. Calibration tables characterizing the vignetting defect and produced in the factory or by modeling obtained for conditions close to the images to be corrected are projected onto the filtered images to be corrected, to quantify the presence of the defect images in the images to be processed. Finally, the calibration tables are subtracted from the images to be corrected according to their presence in the images to be corrected.

[0011] One aspect of the invention relates to a computer-implemented method for correcting a vignetting defect in a sequence of images acquired by a camera for a focal length F', the image sequence being temporally ordered and comprising a plurality of temporally ordered image sub-sequences, the method comprising a set of iterations, each iteration using a sub-sequence of images from the plurality of image sub-sequences and a calibration table, each iteration comprising: - Reducing the dimensions of a first image of the sequence of images to obtain a first reduced image, the first image being corrected for vignetting defect from a correction of the vignetting defect determined at the previous iteration or from an initial correction; - Reduce the dimensions of the calibration table to obtain a reduced calibration table; - Apply a high-pass radial filter to the first reduced image and to the reduced calibration table to obtain a first filtered reduced image and a filtered reduced calibration table; - Determine a correlation coefficient between the first filtered reduced image and the filtered reduced calibration table; - Determine a correction of the vignetting defect from the calibration table, the correlation coefficient associated with the calibration table and the correction of the vignetting defect obtained in the previous iteration; - Correct each other image in the image subsequence from the determined defect correction to obtain each other image corrected for vignetting.

[0012] Thanks to the invention, it is possible to express and quantify the optical vignetting defect included in an image in a linear combination of calibration tables, the linear combination being updated at each iteration, and thus to correct at least one other image of the sub-sequence of images by subtracting the vignetting defect. In particular, the correlation coefficient determined at each step depends on a previously corrected image and therefore on the correlation coefficients determined previously, which allows the method to converge.

[0013] In particular, when the optical defect to be corrected comprises a strong radial component at low or medium frequency, the step of reducing the dimensions of the first image makes it possible to adapt the filtering step to the characteristic dimensions of the vignetting defect and to eliminate the high-frequency patterns which do not correspond to the vignetting defect to be corrected. Furthermore, the radial filter generally highlights all the high-frequency patterns and all the details of the image included in concentric circles, by eliminating longitudinal or vertical patterns. However, in the context of the invention, the application of the radial filter following the reduction of dimensions of the first image and of the calibration table highlights the medium frequencies of the reduced first image and of the reduced calibration table.Thus, the radial filter allows to remove all the patterns not oriented in the direction of the ray and not corresponding to the vignetting defect. In addition, the correlation of each calibration table with each first image of a sub-sequence of images allows to quantify the presence of the calibration table in . the first image in order to subtract the calibration table proportionally to its presence. Finally, the correction is updated by taking into account, for each sub-sequence, the correction of the previous sub-sequence in order to obtain high correction precision and allow convergence of the method.

[0014] In addition to the characteristics which have just been mentioned in the preceding paragraph, the method according to one aspect of the invention may have one or more complementary characteristics among the following, considered individually or according to all technically possible combinations.

[0015] According to one embodiment, the first image of a sub-sequence of images used at a given iteration corresponds to the last image of a sub-sequence of images used at the previous iteration.

[0016] According to one embodiment, the method is carried out for a set of K calibration tables, K being a non-zero natural integer, and for each given iteration of the set of iterations, the calibration table used is distinct from the calibration table used during the previous iteration. Advantageously, when a number K of calibration tables is used, the method according to the invention can be implemented on an FPGA card and the linear combination of the tables making it possible to determine the correction is carried out in 2*K first images of the sub-sequences processed in parallel.

[0017] According to one embodiment, the set of K calibration tables is chosen according to a predefined rule from among M calibration tables, M being a non-zero natural integer with K less than M, the predefined rule being such that, for each calibration table from among the K calibration tables: • the focal length associated with the calibration table corresponds to one of the K focal lengths closest to F'; or • The difference between the focal length associated with the calibration table and F' is less than an average focal length between at least two successive calibration tables among the K calibration tables chosen.

[0018] According to one embodiment, K is between 2 and 10. In particular, the number of calibration tables K can be adapted according to the compromise between the responsiveness of the method and the desired precision. The higher the number of calibration tables, the more precise the correction, but the higher the convergence time of the method. Thus, K between 2 and 10 is a compromise between the precision of the correction and the convergence.

[0019] According to one embodiment, the vignetting defect is a spatial additive noise, having a radial and low-frequency component. "Low frequency" means a period frequency of between 10 and 100 pixels.

[0020] According to one embodiment, the reduction of dimensions of the first image and of the calibration table is made by an averaging filter.

[0021] According to one embodiment, the radial filter is a gradient filter or a Laplacian filter oriented in the direction of the center of each image of each sub-sequence of images. Advantageously, the radial filter makes it possible to remove parasitic information which is not in the direction of the radius of the radial vignetting defect to gain in robustness.

[0022] According to one embodiment, the correlation coefficient between the first filtered reduced image and the filtered reduced calibration table is determined from the ratio: • of a canonical scalar product between the first filtered reduced image and the filtered reduced calibration table; and • a standard of the filtered reduced calibration table. Advantageously, the scalar product between the two images makes it possible to project the filtered reduced calibration table onto the first filtered reduced image in order to obtain the quantity of table in the scene.

[0023] Another aspect of the invention relates to a computer program product comprising instructions which, when the program is executed by a computer, cause the latter to implement the method according to any one of the preceding embodiments.

[0024] Another aspect of the invention relates to a computer-readable recording medium comprising instructions which, when executed by a computer, cause the computer to implement the method according to any of the preceding embodiments.

[0025] The invention and its various applications will be better understood upon reading the following description and examining the accompanying figures. BRIEF DESCRIPTION OF THE FIGURES

[0026] The figures are presented for information purposes only and in no way limit the invention.

[0027] [Fig.l] is a schematic representation of a cooled infrared optical zoom in which a portion of the cut incident optical rays causes vignetting.

[0028] [Fig.2] is a block diagram representing steps of a method for correcting a vignetting defect according to the invention.

[0029] [Fig.3] is a representation of the orientation of a radial gradient filter used in the correction method according to the invention.

[0030] [Fig.4] is a block diagram showing steps of a first embodiment of a method for determining a calibration table.

[0031] [Fig.5] is a block diagram showing steps of a second embodiment of the method for determining a calibration table.

[0032] [Fig.6] is a representation of a device configured to implement the method according to the invention. DETAILED DESCRIPTION

[0033] The invention relates to a method for correcting a vignetting defect in a sequence of images.

[0034] The image sequence comprises a plurality of images and is ordered temporally, that is to say that the plurality of images is arranged by increasing acquisition time in the image sequence.

[0035] Thus, if a first image is acquired at a time t, a second image is acquired at a time t+1, and a third image is acquired at a time t+2, the first image, the second image and the third image will be arranged in this order in a sequence of images comprising them.

[0036] The image sequence is acquired by an infrared camera or a visible camera.

[0037] The sequence of images is obtained from a camera located for example in a aircraft or in an observation sight located in a ground vehicle.

[0038] Thus, the camera makes it possible to acquire sequences of images which can be displayed in real time to a user of the aircraft or the land vehicle, for example.

[0039] The image sequence comprises a plurality of image sub-sequences. The image sequence preferably comprises at least two sub-sequences. A sub-sequence of images comprises at least two images for example and preferably comprises three images. The image sub-sequence may in particular comprise four, five or even ten images.

[0040] In particular, the sequence of images being acquired for successive instants, the sub-sequences of images of the sequence of images follow one another temporally.

[0041] In particular, the images of each sub-sequence of images follow one another in time.

[0042] "Infrared camera" means a camera configured to measure and record heat waves and infrared radiation emitted by a body or object, in order to generate sequences of so-called infrared images.

[0043] "Visible camera" means a camera configured to measure and record visible light, in the visible spectrum which extends from 300 to 750 nanometers in wavelength in order to generate sequences of images in the visible spectrum.

[0044] In particular, the acquisition camera comprises a variable focal length lens. In the following, we will confuse "focal length" and "focal length".

[0045] We note F' the focal length of the camera lens during the acquisition of the image sequence.

[0046] In particular, the image sequences acquired by the camera may comprise a vignetting defect which reduces the quality of the images and which is corrected using the method according to the invention.

[0047] According to the invention, the vignetting defect is modeled according to a plurality of hypotheses described below.

[0048] According to a first hypothesis, the vignetting defect is a constant image or "offset" from the English, modelable as a spatial additive noise and fixed in time. "Fixed noise in time" means a noise whose characteristic time of evolution is large compared to the convergence time of the method according to the invention, the characteristic time of evolution being a few seconds while the convergence time of the method according to the invention corresponds to a few images. In particular, when the camera acquires sequences of images at a rate between 25Hz and 50Hz, "convergence time of a few images" means a time of approximately a few hundred milliseconds. In particular, the characteristic time of evolution of the vignetting defect is approximately ten times greater than the convergence time of the method according to the invention.

[0049] According to a second hypothesis, the vignetting defect comprises a signal of medium to low spatial frequency. Medium to low frequency is understood to mean a frequency whose period is between 10 and 100 pixels.

[0050] According to a third hypothesis, the vignetting defect has a dominant radial spatial component relative to a horizontal component or a vertical component. A defect with a dominant radial spatial component relative to a horizontal component or a vertical component is a defect whose average gradient is greatest in the direction of the rays passing through the center of the image. This visually results in concentric patterns on the image comprising the vignetting defect. In practice, the concentricity is not exact and the position of the center is difficult to characterize, hence the elimination of radial corrections based on a strict model.The term "radial dominant component image" means an image whose average of the absolute values ​​of the image filtered by a radial filter is at least five times greater than the average of the absolute values ​​of said image filtered by a filter oriented orthogonally to the rays which pass through the center of the image.

[0051] The correction method according to the first aspect of the invention is carried out from a set of K calibration tables selected from a set of M calibration tables, with K and M two unaffected natural integers and K less than M.

[0052] For example, M is between 1 and 100.

[0053] For example, K is between 1 and 10. For example, K may be equal to 1. In particular, K is preferably equal to 2, 3 or 4.

[0054] Each calibration table among the set of M calibration tables can by example correspond to an image acquired or modeled for a focal length F'm of the camera lens, with m between 1 and M. The calibration tables include coefficients characterizing the optical vignetting defect as a function of parameters such as the focal length of the camera lens, the camera temperature and the integration time for example. In particular, the calibration tables have the same size as the pixel matrix of the camera detector and therefore the same size as the image sequences acquired by the camera. The calibration tables are for example produced in the factory.

[0055] In particular, the method according to the invention can be carried out from any set of calibration tables each corresponding to a vignetting defect verifying the hypotheses cited previously and obtained for different focal lengths.

[0056] Examples of calibration tables are detailed later in the description.

[0057] In particular, the correction method according to the invention is carried out for a sequence of images acquired at a focal distance F' and the calibration tables of the set of K calibration tables making it possible to carry out the method are selected from among the M calibration tables, the K calibration tables being selected according to a predetermined condition from among the M calibration tables.

[0058] According to one embodiment, the predetermined condition is that the focal length F'k of each calibration table of the set of K calibration tables, with k a natural integer between 1 and K, is one of the K focal lengths closest to the focal length F' among the M focal lengths of the set of M calibration tables. In other words, the calibration tables associated with the K focal lengths closest to the focal length F' among the M focal lengths are selected.

[0059] In particular, the K focal lengths closest to F' among the M focal lengths of the set of M defect images are determined by calculating a norm of the difference F'm-F', the norm being for example a Euclidean norm, and by selecting the K minimum norms.

[0060] According to another embodiment, the predetermined condition is that the difference between each focal length F'k and the focal length F' is less than a predefined threshold corresponding to a ratio of an average focal length between at least two successive calibration tables among the K chosen calibration tables and the focal length F'. The predefined threshold is for example less than or equal to 20%. Thus, a predetermined number of calibration tables is not selected but all the calibration tables corresponding to a focal length F'k meeting the predetermined condition that the difference between the focal length F'k and the focal length F' is less than a predefined threshold.

[0061] In particular, at least one focal length F'k associated with a calibration table of the set of K calibration tables is different from the focal length F'.

[0062] Thus, the K selected calibration tables vary according to the acquisition focal length F' of the image sequence.

[0063] [Fig.2] shows a schematic representation of the correction method 200 of a vignetting defect according to the invention in a sequence of images as described previously.

[0064] The image sequence is divided into a plurality of image sub-sequences each comprising a plurality of successive images in the image sequence. The plurality of image sub-sequences is therefore temporally ordered in the image sequence.

[0065] For example, the image sequence comprises P sub-sequences of images, P being a non-zero natural integer, between 1 and 100.

[0066] Each sub-sequence of images preferably comprises the same number of images, for example equal to two or preferably three.

[0067] For example, each image sequence is split such that a given image subsequence comprises the last image of the image subsequence preceding the given image subsequence in the image sequence and the first image of the image subsequence following the given image subsequence in the image sequence.

[0068] For example, considering a sequence of images comprising seven images SEQ = {IM1, IM2, IM3, IM4, IM5, IM6, IM7], the sequence of images is divided into a first sub-sequence of images SEQi = {IM1, IM2, IM3], a second sub-sequence of images SEQ2 = {IM3, IM4, IM5], a third sub-sequence of images SEQ3 = {IM5, IM6, IM7}.

[0069] Thus, the correction method comprises steps 201, 202, 203, 204, 205 and 206 carried out iteratively for each sub-sequence SEQ; of images of a sequence of images acquired by an infrared camera or a visible camera as described previously. In particular, the sub-sequence of images is acquired for a focal length F' of the camera and includes a vignetting defect on each of its images.

[0070] In particular, the method comprises a set of iterations, the set comprising a number of iterations preferably equal to the number of sub-sequences SEQ;. At each iteration a single sequence is processed.

[0071] Thus, consider a given iteration i, with i between 1 and P, the given iteration using an i-th sub-sequence SEQ; of images of the sequence of images to be corrected and on which the method 200 is applied. At each iteration i, a calibration table, denoted Tk, among the set of K calibration tables is used.

[0072] In one embodiment, K is equal to 1 and the set of K calibration tables comprises a single calibration table, denoted Th

[0073] In another embodiment, K is greater than or equal to 2 and each iteration among the set of iterations uses a separate calibration table. Preferably, K is less than the total number of iterations in the iteration set and each calibration table is used at least once.

[0074] Thus, consider the set of calibration tables with K equal to 2 for example. The set of calibration tables thus includes the calibration table Ti and the calibration table T2. In this example, if the calibration table Ti is used at iteration i, then the calibration table T2 is used at iteration i+1, and the table Ti is used again at iteration i+2.

[0075] Thus, for the given iteration i, the method comprises a step 201 of reducing the dimensions of a first image IMu of the sequence SEQ; by a reduction factor N.

[0076] In particular, the first image IMij is corrected for the vignetting defect using a vignetting defect correction, noted Corri, i)k, determined at the previous iteration, i.e. iteration i-1.

[0077] If the given iteration i is such that i is equal to 1, then there is no previous iteration and the first image IMij is corrected for the vignetting defect from an initial correction, an example of which is given later in the description.

[0078] The reduction factor N is a non-zero natural integer between 1 and 64.

[0079] The term "dimension" of an image means its size expressed in number of pixels.

[0080] In particular, each image of the sequence of images to be corrected comprises L rows of pixels and C columns of pixels, its dimension being equal to L*C pixels, L and C are two unaffected natural integers. Thus, it is the same for each image of each sub-sequence of images.

[0081] The first image therefore has a dimension of L*C pixels before the dimension reduction step 201. After the dimension reduction step 201, the number of columns of the first image is divided by N, and the number of rows of the first image is divided by N. Thus, the reduction of the first image IMu makes it possible to obtain a first reduced image IMRij of dimension A = LL.

[0082] The method comprises a step 202 of reducing the dimensions of the calibration table Tk used during the current iteration i, by the reduction factor N.

[0083] The calibration table Tk having the same dimension as the first image IMij, the reduction of the calibration table Tk makes it possible to obtain a reduced calibration table TR k of dimension L2L. N2

[0084] The dimension reduction steps 201 and 202 are carried out using an averaging filter, which, on each image to which it is applied, carries out the averaging of each sub-image of N*N successive pixels on said image. The image resulting from the reduction thus comprises N2 times fewer pixels than the image before reduction. The reduction in the dimension of the first image advantageously makes it possible to adapt the filtering to the characteristic dimensions of the vignetting defect, and to remove high-frequency patterns which, by hypothesis, do not belong to the vignetting defect to be corrected. In fact, the vignetting defect comes from the absence of light rays but also from reflections in the lenses of the optical system, which create medium and low-frequency patterns.

[0085] The reduction factor N is adapted to the resolution of the camera sensor and to the nature of the vignetting defect which can be characterized in the factory. This factor can possibly be dynamically adapted to the position of the focal length or to other parameters which influence the characteristic dimensions of the vignetting defect.

[0086] Step 203 of method 200 is a step of applying a high-pass radial filter to the first reduced image IMu and the reduced calibration table TRk.

[0087] The high-pass radial filter only filters low and medium frequencies. In particular, the high-pass radial filter only allows low and medium frequencies with periods less than 3 pixels to pass on images whose dimensions are reduced by a factor of N. Thus, following steps 201 and 202, the high-pass radial filter is equivalent to a band-pass filter which only allows frequencies with periods close to 3*N pixels to pass (in the first reduced image IMu and the reduced calibration table TRk).

[0088] For example, the radial filter is a Laplacian of Gaussian (LoG) filter, a Gradient filter oriented in the direction of the center of the first reduced image or any filter based on the differences oriented in the direction of the center of the first reduced image.

[0089] The radial filter has a dimension (or a size) noted F*F', with F and F' two unnullified natural integers.

[0090] An example of a radial filter with F = 3 and F' = 3 is as follows: [0091 ] / q- f ) ' + “ ) 0 + ^) my(0 + 7T) +

[0092] For each image to be filtered and for a given pixel of the image to be filtered, an angle 0 is associated which corresponds to the angle formed between a first straight line passing through the central pixel of the image and through the position of said given pixel and a second horizontal straight line (in the direction of the lines of the image to be filtered) and passing through the central pixel, the vertex of the angle being the central pixel. In particular, the angle 0 is measured between the second straight line and the first straight line, and is positive in the trigonometric direction. Thus, several pixels can be associated with the same angle 0.

[0093] In particular, the radial filter comprises concentric circles, each circle corresponding to a set of pixels distant from the same number of pixels relative to the central pixel.

[0094] In addition, the radial filter calculates for example the difference between two diagonal neighboring pixels associated with the same angle 0.

[0095] Advantageously, the application of the radial filter to the first reduced image makes it possible to highlight the radial component of the vignetting defect. The radial filter makes it possible to remove all the patterns not oriented in the direction of the radial component (or radius) to keep only the patterns of interest and thus highlights all the high-frequency patterns and all the details of the image in the direction of the radii of the concentric circles. Indeed, the radial component of the vignetting defect only presents patterns in the direction of the radii of the concentric circles. Thus, a radial filter is advantageous compared to a non-oriented filter.

[0096] [Fig. 3] is an example of the local orientation of the radial filter, which in this case is a gradient filter. Thus, [Fig. 3] represents the orientation of the gradient for each pixel position of the first image. The lines shown represent the direction of the gradient.

[0097] Referring again to [Fig.2], at the end of step 203, a first filtered reduced image IMRFij and a filtered reduced calibration table Tm are obtained.

[0098] Step 204 is a step of determining a correlation coefficient noted cik, from the first filtered reduced image IMRFij and from the filtered reduced calibration table TRFk.

[0099] The correlation coefficient is determined from a canonical scalar product, whose operator is noted <.l.>.

[0100] In the rest of the writing, we note v^ the line vector for example comprising all the pixel values ​​of the first filtered reduced image IMRFij, and vT kRF the line vector comprising all the pixel values ​​of the filtered reduced calibration table TRFk.

[0101] Thus, at the given iteration, the correlation coefficient between the first filtered reduced image IMRFu and the filtered reduced calibration table Tj^ is equal to the following term:

[0102] _ (vlRFKr_kRF^ ( vT_kRFÎvT_kRF 1

[0103] In particular, the term { vT kRp|vT kRF ) corresponds to the squared norm of the filtered reduced calibration table Tj^.

[0104] The term ( V1rf]vt kRF ) makes it possible to compare the first filtered reduced image and the filtered reduced calibration table Tj^, the filtered reduced calibration table T^ being projected onto the first filtered reduced image.

[0105] The scalar product makes it possible to measure the presence of the TRFk calibration table in the first IMijet image and therefore to measure the presence of the associated vignetting defect to the TRFk calibration table in the first IMu image

[0106] In particular, the correlation coefficient cik can take any real value. The coefficient cik can thus be positive or negative for example. It is also possible, depending on the normalization of the calibration tables, to obtain a correlation coefficient cik whose amplitude is greater than 1.

[0107] The correlation coefficient cik thus makes it possible to quantify the similarities between the first filtered reduced image IMRFu and the filtered reduced calibration table TRFk and to determine to what extent the vignetting defect of the calibration table considered is present in the first image.

[0108] Step 205 is a step of determining a correction Corrik of the vignetting defect from the calibration table TRFk, from the correlation coefficient cik obtained in step 204 from the filtered reduced calibration table TRFk and from the correction Corrt 4 )k of the vignetting defect determined in the previous iteration, i.e. iteration i-1

[0109] If the given iteration i is such that i is equal to 1, then there is no previous iteration then and the correction Corr(i4)k corresponds to the initial correction Corr0.

[0110] In particular, the correction Corrik of the vignetting defect is for example equal to a sum of the correction Corr(i4)kob held at the previous iteration and a product of the given calibration table multiplied by the correlation coefficient cik associated with it.

[0111] Thus, a Corrik correction of the vignetting defect corresponds to an image of the same size as each image of a sub-sequence of images acquired by the camera.

[0112] Thus, Corr^_ CoiT(.i)t +Cjt.,Tk =

[0113] Thus, the correction is updated at each iteration i, for each sub-sequence of images SEQ; and depends on the correlation coefficients determined at the iterations preceding iteration i, in order to allow the convergence of the method 200.

[0114] Thanks to the iterative aspect of the method, it converges: the calculation of the correlation coefficient for a calibration table at iteration i is calculated from the first image corrected by the correction previously determined at iteration (i-1) to take into account the contributions of the calibration tables used at the previous iterations.

[0115] Step 206 is a step of correcting each other image of the sub-sequence SEQi of images given from the Corrik correction.

[0116] For example, if the sequence of images SEQ; is noted: {IMu, IM2 i, IM3J} then steps 201 to 205 are carried out on the image IMi and the correction step 206 is carried out on the second and third images IM2j and IM3i in order to obtain respective second and third corrected images IM'2J and IM'3.

[0117] In particular, the second corrected image IM'2J is for example obtained by sub traction of the Corrik correction to the second image IM2 j of the subsequence SEQ; of images.

[0118] Thus, IM'2 i = IM2; - 0¾

[0119] The image IM'2j thus corresponds to the image IM2 iafter correction of the defect of vi gnettage according to method 200.

[0120] Thus, IM'3i = IM3 i - 00¾.

[0121] In particular, if the image sub-sequences comprise for example a plurality of images greater than third images, the correction is preferably applied to all the images of the image sub-sequences with the exception of the first image.

[0122] At the following iteration, i.e. at iteration i+1, the subsequence SEQ(i+i) comprises the last image of the subsequence SEQ; as the first image

[0123] Thus, in this embodiment, SEQ^is denoted: { IMi_(i+i), IM2 (i+i), IM3_(i+i)} with IMi_(i+i)= IM'3i.

[0124] With reference to the initial correction Corr0 used in the method 200 according to the invention, the initial correction is a matrix of the same size as the camera detector and can be obtained by a 1-point calibration method of the camera. The 1-point calibration of a cooled infrared camera is carried out by adjusting a gain table G and an offset table O, the adjustment being carried out from a defocused image or an image of a diaphragm (from the English "shutter") of the camera, used to obtain an image assumed to be uniform on the sensor. The gain G and the offset O are matrices of the same size as the camera detector.

[0125] In particular, the response of a pixel of the camera detector to a flux can be characterized by a linear function characterized by a direction coefficient which represents a gain and an ordinate at the origin which represents an offset. Each pixel has its own gain and its own offset. Consequently, upon receipt of a uniform luminous flux, the gray levels generated by the pixels differ, which results in the appearance of non-uniformities on the image generated by the camera.

[0126] In order to eliminate pixel non-uniformities, it is necessary that the pixels have the same gain and the same offset. A correction is made by applying the gain correction table G and the offset correction table denoted O.

[0127] The gain table G is multiplied by the flux received by each of the pixels, and is expressed in the form of a table of factors applied to each of the pixels which is determined in the factory before the camera is put into service for example. After its application, all the pixels have the same response to a given flux variation and their characteristic functions therefore have the same direction coefficient. The gain table can be determined by carrying out the partial derivative of order 1 of two images taken at two different scene temperatures. In particular, the partial derivative is carried out by relative to the luminance parameter which depends on the temperature.

[0128] The offset table O is added to the flux received by each of the pixels, and is therefore expressed in the form of a table of coefficients added to each of the pixels. The offset table O makes it possible to correct defects both intrinsic to the detector and due to parasitic cold fluxes.

[0129] The shift table O is obtained from the image of a scene acquired in front of a diaphragm, noted Idiaphragm, multiplied by the gain table G. In particular, the shift table O is obtained from the following formula:

[0130] O = Idiaphragm*G-Mean(Idiaphragm*G). Thus, the initial correction is equal to the table O.

[0131] Concerning the calibration table(s) used in the correction method according to the invention, several embodiments of these tables are described below.

[0132] According to one embodiment, a calibration table may result from the difference between two images acquired from the same uniform background respectively with different focal distances of the camera. Thus, in order to construct M defect images, it is possible to acquire two series of M images each of the same uniform background, the two series being acquired for two different focal lengths, the M defect images being obtained by differences of the images of each series two by two.

[0133] According to one embodiment, the calibration tables may be those cited in patent US8645104B2, figures 1 and 2 of said US patent making it possible to generate correction tables called "tables for correction" which correspond to the calibration tables.

[0134] According to one embodiment, for each focal length F';, the calibration table T; associated with the focal length F'; can be determined according to a method for determining the calibration table, the determination method having a first and a second distinct embodiment.

[0135] [Fig.4] presents a block diagram of the steps of the first embodiment of the method 300 for determining the calibration table T; associated with the focal length F';. In particular, each calibration table T; is associated with an internal calibration temperature T; of the camera and a calibration integration time T; of the camera.

[0136] "Internal temperature of the camera" means the temperature acquired near the optics of the camera lens and in particular near the front lens of the lens for example. In the following, any internal temperature value of the camera is included for example in the interval [-20; +70] °C.

[0137] The method 300 comprises a step 301 of acquisition by the camera of an image Isj of a thermal flux emitted by a source, for the focal length F'i of the camera, the internal calibration temperature of the camera and the calibration integration time. of the camera which are stored by a user in a memory external to the camera for example.

[0138] The source emits for example a thermal flux (or thermal radiation) similar to the thermal fluxes emitted by a black body, that is to say an isotropic thermal flux, so the radiation depends only on the temperature and not on the directions in space for example. Advantageously, the use of a source emitting isotropic radiation makes it possible to have a homogeneous scene and to observe only the optical defects of the camera.

[0139] In practice, the source may be a heating plate covered entirely with black paint, and having a uniform, constant temperature that can be adjusted by a user. Thus, the source can be likened to a black body.

[0140] The thermal flux emitted by the source covers the entire pupil of the camera lens.

[0141] The method 300 comprises a step 302 of calibrating the image Isj according to the one-point calibration method of the camera. The one-point calibration is carried out from the gain table G and the offset table O described previously. After the calibration step, an image of the calibrated source Is2j is obtained from the image of the source IS i, such that:

[0142] IS2_i= G*IS iO

[0143] The image IS2jobtenue corresponds to the image of the source Isjcorrected for the defects of the detector and includes residual optical defects as mentioned previously, that is to say residual optical defects corresponding to parasitic thermal fluxes of the camera lens relative to the focal length F';.

[0144] The method 300 further comprises a step 303 of obtaining a third image IS3_i resulting from the subtraction of an average image of the image IS2J, denoted Average(I S2J), from the calibrated image IS2j. In particular, Average(IS2_i) corresponds to an image of the same size as IS2j and each coefficient of which corresponds to the average of all the pixel values ​​of IS2j.

[0145] Thus, IS3i = IS2 i - Average(lS2j)-

[0146] According to a first embodiment, the calibration table T; associated with the focal length F'; corresponds to IS3 and can therefore be noted: T, = Issj.

[0147] Thus, the calibration table T can be obtained at the end of step 303.

[0148] According to a second embodiment, the method 300 comprises an optional step 304 of spatial filtering of the third image IS3j to obtain a filtered image F(I S3_i)-

[0149] The filtering may for example correspond to a spatial filtering of the image IS3j in the case where intrinsic defects of the detector remain even after the calibration step 301, for example because of one or more defective pixels of the detector. this second embodiment, the calibration table T; can be equal to F(IS3j).

[0150] According to a third embodiment, the calibration table T; is obtained at the end of optional steps not shown in the method 300, described below.

[0151] The method 300 may comprise a first optional step of carrying out steps 301, 302 and 303 at several different times t and obtaining a plurality of images (IS3_it)t>o

[0152] The method 300 may comprise a second optional step of averaging the images IS3i_t to obtain an average image Average(IS3_it)t>o- In particular, the averaging step 306 corresponds to an averaging of each pixel value of each image IS3jt having the same position in the image. In other words, each coefficient at a given position of the image Average(IS3_it)t>o corresponds to an average of the values ​​of the pixels of the same position of each image IS3_it.

[0153] The method 300 may comprise a third optional step of obtaining the calibration table T; by subtracting the image Average(Is3 it) from the image IS3jt. Thus, Ti= W Average(IS3jt).

[0154] In particular, the third embodiment advantageously makes it possible to eliminate temporal noise present in the image IS3j, the temporal noise degrading the quality of the image by giving it a granular appearance.

[0155] According to an embodiment not shown, the optional steps are applied to the image F(IS3_i) following step 304.

[0156] The method 300 for obtaining the calibration table T; associated with the focal length F'; can be carried out for a plurality of M focal lengths rp j , which makes it possible to obtain M first calibration tables qA each associated with a focal length different F';, the index i being a natural integer. In particular, each calibration table among the first M calibration tables is obtained for a different focal length but with the same internal calibration temperature of the camera and the same calibration integration time of the camera.

[0157] In particular, each calibration table may be compressed before being saved in a memory for use. The compression method must be adapted to the spatial morphology of the residual optical defects represented by each calibration table, in order to avoid any degradation of the quality of the calibration table.

[0158] [Fig.5] presents a block diagram of the steps of the second embodiment of the method 400 for determining the calibration table T; associated with the focal length F'i.

[0159] The method 400 comprises a step 401 of acquisition by the camera of a first image IlSj of a thermal flux emitted by the source defined in the method of determination mination of the calibration table. In particular, the acquisition step 401 is carried out for the focal length F'; of the camera, at an internal calibration temperature T of the camera and with the calibration integration time T of the camera which are stored by a user in a memory external to the camera for example or in a memory of the camera.

[0160] The method 400 further comprises a step 402 identical to step 302, the step 402 being applied to the image Ilsj and making it possible to obtain the calibrated image IlS2_i. The calibrated image IlS2j corresponds to the image Ilsj corrected for the defects of the detector and comprising residual optical defects as described previously. The residual optical defects correspond to parasitic thermal fluxes of the camera lens for the focal length F';.

[0161] The method 400 further comprises a step 403 of obtaining an image IlS3j resulting from the subtraction of an average image of the image IlS2j, denoted Average(IlS2 _i) from the calibrated image IlS2j- In particular, Average(Ils2j) corresponds to an image of the same size as IlS2j and each coefficient of which corresponds to the average of all the pixel values ​​of IlS2_i. Thus, IlS3_i J1S2j - Average(IlS2j).

[0162] The method 400 further comprises a step 404 of varying the internal temperature of the camera, to obtain an internal temperature T2calibration of the camera different from T1 calibration- The variation of the internal temperature of the camera can be carried out by placing the camera in a temperature-adjustable oven for example.

[0163] The method 400 comprises a step 405 of acquisition by the camera of a second image I2Sj of the thermal flux source defined in the method 200, for the focal length F'i of the camera, the internal calibration temperature T2 of the camera and the calibration integration time T of the camera.

[0164] The method 400 further comprises a step 406 identical to step 302, step 406 being applied to the image I2SJ and making it possible to obtain the calibrated image I2S2_i. The calibrated image I2S2_i corresponds to the image I2s_i corrected for the defects of the detector and comprising residual optical defects as mentioned previously.

[0165] The method 400 further comprises a step 407 of obtaining an image rS3j resulting from the subtraction of an average image of the image I2S2_i, denoted Average(I2S2 j) from the calibrated image I2S2_i- In particular, Average(I2S2j) corresponds to an image of the same size as I2S2_i and each coefficient of which corresponds to the average of all the pixel values ​​of I2S2j. Thus, I2S3j = 12^ - Average(l2S2j)-

[0166] The method 400 comprises a step 408 of obtaining the calibration table T; from the image IlS3j, the calibration temperature Tletaionnage, the image I2S3j and the calibration temperature T2etaionnage.

[0167] In particular, the calibration table T; makes it possible to estimate the variation of the defects residual optical defects as a function of the internal temperature variation. An interpolation is carried out between the variation of the residual optical defects and the internal temperature variation, the interpolation order being chosen according to predetermined criteria. Indeed, the higher the interpolation order, the more the quality of the images to be corrected for residual optical defects is improved because the relationship between the variation of the residual optical defects and the internal temperature variation is more precise. However, the higher the interpolation order, the more complex the method 400 is to implement because a plurality of blackbody image acquisitions are required.

[0168] In particular, the variation of residual optical defects can depend linearly and indirectly on the variation of internal temperature. Indeed, the variation of residual optical defects depends linearly on the variation of integrated luminance which depends on temperature.

[0169] Thus, the calibration table T is obtained according to the following formula:

[0170] y = L(Thla <onbage>«T2éwtaage)

[0171] with L(Tletaionnage) the integrated luminance generated by the temperature Tletaionnage and L(T2 calibration) the integrated luminance generated by the temperature T2 calibration.

[0172] Thus, the integrated luminance is linked to the internal temperature T that the camera integrates (captures). The gray level captured by the camera depends on a multitude of sources. For the purposes of interpolation, a simulation of the variation of gray levels captured by the detector as a function of the variation in internal temperature is carried out. The determination of the luminance as a function of the temperature (denoted L(T1) and L(T2)) is for example obtained via an infrared camera simulation method.

[0173] In particular, the integrated luminance takes into account a fixed temperature T of a scene integrated by the camera and passing through the optics, the internal luminance of the camera (i.e. the photons leaving the camera and reaching the detector) and the dark current of the camera detector.

[0174] When the infrared camera is placed in front of a scene of fixed temperature T, the internal temperature of the camera is varied in order to observe the evolution of the variation in gray level as a function of the variation in the internal luminance of the camera.

[0175] The integrated luminances are obtained from a resolution of the Planck equations in a context applied to the camera, that is to say by taking into account the variation in gray level as a function of the variation in the internal luminance of the camera.

[0176] Using luminance values ​​as a function of temperature is advantageous because the luminance of a pixel in an image is modeled linearly as a function of its gray level, for any temperature.

[0177] The method 400 for obtaining the calibration table T; associated with the focal length F'i can be carried out for a plurality of M focal lengths / p A , which makes it possible to obtain M "Ei <m second calibration tables rp) each associated with a focal length different F';. In particular, each calibration table among the first M calibration tables is obtained for a different focal length but with the same camera calibration integration time.

[0178] Another aspect of the invention relates to a device configured to implement the method according to the invention.

[0179] [Fig.6] represents the device 500 according to one or more embodiments of the invention.

[0180] In these embodiments, the device comprises a computer 500, comprising a memory 501 for storing defect images and / or image sequences for example.

[0181] The computer 500 further comprises a circuit 502. This circuit may be, for example, a processor capable of interpreting instructions in the form of a computer program, an electronic card whose steps of the method of the invention are described in the silicon, or even a programmable electronic chip such as an FPGA chip (for "Field-Programmable Gate Array" in English). In particular, steps 201 to 205 may be programmed to be implemented on the FPGA chip.

[0182] The computer 500 comprises an input interface 503 for receiving sequences of images acquired by the camera, and an output interface 504. Finally, the computer may comprise, to allow easy interaction with a user, a screen 505 displaying the sequences of images acquired by the camera, the calibration tables and the corrected images for example.

[0183] The computer 500 may also include a keyboard 506. Of course, the keyboard is optional, particularly in the context of a computer in the form of a touchscreen tablet, for example.

[0184] Of course, the present invention is not limited to the embodiments described above as examples; it extends to other variants.< / m> < / onbage>

Claims

Claims

1. A computer-implemented method (200) for correcting a vignetting defect in a sequence of images acquired by a camera for a focal length F', the sequence of images being temporally ordered and comprising a plurality of temporally ordered sub-sequences of images, the method comprising a set of iterations, each iteration using a sub-sequence of images from the plurality of sub-sequences of images and a calibration table, each iteration comprising: - Reducing (201) dimensions of a first image of the sequence of images to obtain a first reduced image, the first image being corrected for the vignetting defect from a correction of the vignetting defect determined at the previous iteration or from an initial correction; - Reducing (202) dimensions of the calibration table to obtain a reduced calibration table;- Applying (203) a high-pass radial filter to the first reduced image and to the reduced calibration table to obtain a first filtered reduced image and a filtered reduced calibration table; - Determining (204) a correlation coefficient between the first filtered reduced image and the filtered reduced calibration table; - Determining (205) a correction of the vignetting defect from the calibration table, the correlation coefficient associated with the calibration table and the correction of the vignetting defect obtained in the previous iteration; - Correcting (206) each other image of the sub-sequence of images from the determined defect correction to obtain each other image corrected for the vignetting defect.;

2. Method (200) according to the preceding claim according to which the first image of a sub-sequence of images used at a given iteration corresponds to the last image of a sub-sequence of images used at the previous iteration.

3. Method (200) according to one of the preceding claims according to which method (200) is performed for a set of K calibration tables, K being a non-zero natural integer, and for each given iteration of the set of iterations, the calibration table used is distinct from the calibration table used during the previous iteration.

4. Method (200) according to the preceding claim according to which the set of K calibration tables is chosen according to a predefined rule from among M calibration tables, M being a non-zero natural integer with K less than M, the predefined rule being such that, for each calibration table from among the K calibration tables: - the focal length associated with the calibration table corresponds to one of the K focal lengths closest to F'; or - The difference between the focal length associated with the calibration table and F' is less than an average focal length between at least two successive calibration tables from among the K chosen calibration tables.

5. Method (200) according to one of claims 3 to 4 according to which K is between 2 and 10.

6. Method (200) according to one of the preceding claims according to which the vignetting defect is a spatial additive noise, having a radial and low frequency component.

7. Method (200) according to one of the preceding claims according to which the reduction of dimensions of the first image and of the calibration table is carried out by an averaging filter.

8. Method (200) according to one of the preceding claims according to which the radial filter is a gradient filter or a Laplacian filter oriented in the direction of the center of each image of the sub-sequence of images.

9. Method (200) according to one of the preceding claims, according to which the correlation coefficient between the first filtered reduced image and the filtered reduced calibration table is determined from the ratio: • of a canonical scalar product between the first filtered reduced image and the filtered reduced calibration table; and • of a norm of the filtered reduced calibration table.

10. Computer program product comprising instructions which, when the program is executed by a computer, cause the latter to implement the method according to one of the preceding claims.

11. A computer-readable recording medium comprising instructions which, when executed by a computer, cause the computer to carry out the method according to one of claims 1 to 9.

Images