Method for correcting vignetting in an image sequence using calibration tables

Abstract

Method for correcting vignetting in an image sequence using calibration tables. One aspect of the invention relates to a computer-implemented method for correcting vignetting in an image sequence acquired by a camera at a focal length F', the image sequence comprising a plurality of image sub-sequences. The method is performed using a set of calibration tables corresponding to the vignetting defect and each associated with a specific focal length, at least one specific focal length associated with a calibration table being different from F'. The method comprises a plurality of iterations. Figure to be published with the abbreviation: Figure 2

Description

Title of the invention: Method for correcting vignetting defects in an image sequence from calibration tables. TECHNICAL FIELD OF THE INVENTION

[0001] The technical field of the invention is that of vignetting correction in image sequences.

[0002] The present invention relates to a method for correcting a vignetting defect in a sequence of images. TECHNOLOGICAL BACKGROUND OF THE INVENTION

[0003] Infrared or visible light-detecting cameras included in sights or aircraft, for example, comprise a system of glass lenses and / or mirrors, the position of the lenses and mirrors being variable to adjust the focal length of the system and magnify the observed objects. In particular, the movement of the lenses allows for continuous variation of the magnification and, notably, for dynamic adjustment of the focal plane.

[0004] However, the movement of mirrors or lenses within cameras can contribute to time-varying vignetting defects, which affect the quality of images produced in real time. In particular, vignetting defects have a predominantly radial component and correspond to a darkening of the peripheries of an image, as the ambient light is attenuated at the edges of the image. Vignetting corresponds to a progressive loss of illumination at the periphery of an image, moving from the center of the image to its periphery, and therefore reduces the quality of the acquired images.

[0005] Figure 1 is an example of an optical system 100 comprising a cooled infrared optical zoom 101, a detector 102, and a cold screen 103. In this case, the cold screen 103 is the exit pupil of the optical system. The optical zoom 101 includes, in particular, a front lens 1011, which is the entrance pupil of the optical system, and for which conjugation with the cold screen 103 is sought. The conjugation between the two pupils is not perfect, and, for reasons of size constraints of the optical zoom 101, it is sometimes necessary to limit the effective diameter, in other words the aperture, of the front lens to the area Z through which the rays pass on the optical axis. Therefore, the further the object is from the axis, the less light rays the image receives: part of the V rays arriving on the front lens and not included in the Z zone is cut off and does not reach detector 102, which is the origin of vignetting.

[0006] There are methods for correcting vignetting in acquired image sequences, at Using calibration tables that correspond to the vignetting pattern, these tables are also called calibration tables. Each calibration table is associated with a vignetting defect for parameters such as a given focal length, integration time, and temperature. However, calibration tables do not include parameters related to the scene filmed by the camera and cannot compensate for the absence of rays that cause the vignetting.

[0007] Vignetting can be corrected by correlating the high frequencies of a real-time acquired image with the high frequencies of calibration tables. Estimating the coefficients from the high frequencies is described, for example, in patents EP2127359B1 and FR3118558. However, using the high frequencies of the images to determine and correct vignetting can introduce defects greater than the original radial defects.

[0008] It is possible to estimate the radial component of vignetting defects by statistical processing on concentric circles and then applying it radially over the entire image as described in US patent 9730958B2. However, the residual obtained after correction by a radial or elliptical estimation of the defect remains significant.

[0009] There is therefore a need to correct the vignetting of images acquired in real time in an optimal way. Summary of the invention

[0010] The invention provides a solution to the problems mentioned above by considering vignetting as an additive radial spatial noise at low / medium frequencies. In particular, the invention highlights this defect in image sequences to be corrected by filtering out high frequencies with a radial filter. Calibration tables characterizing the vignetting defect, generated either in the factory or through modeling obtained for conditions similar to those of the images to be corrected, are projected onto the filtered images to be corrected in order to quantify the presence of the defect images in the processed images. Finally, the calibration tables are subtracted from the images to be corrected according to their presence in the corrected images.

[0011] One aspect of the invention relates to a computer-implemented method for correcting vignetting in a sequence of images acquired by a camera for a focal length F', the image sequence being time-ordered and comprising a plurality of time-ordered image subsequences, the method comprising a set of iterations, each iteration using a subsequence of images from the plurality of image subsequences and a calibration table, each iteration comprising: - Reduce the dimensions of a first image in the image sequence to obtain a first reduced image, the first image being corrected for vignetting defects from a vignetting defect correction determined in the previous iteration or from an initial correction; - Reduce the dimensions of the calibration table to obtain a reduced calibration table; - Apply a radial high-pass filter to the first reduced image and to the reduced calibration table to obtain a filtered first reduced image and a filtered reduced calibration table; - Determine a correlation coefficient between the first reduced filtered image and the reduced filtered calibration table; - Determine a correction for the vignetting defect from the calibration table, the correlation coefficient associated with the calibration table, and the vignetting defect correction obtained in the previous iteration; - Correct each other image in the subsequence of images from the determined defect correction to obtain each other image corrected for the vignetting defect.

[0012] Thanks to the invention, it is possible to express and quantify the vignetting optical defect in an image as a linear combination of calibration tables, the linear combination being updated at each iteration, and thus to correct at least one other image in the subsequence of images by subtracting the vignetting defect. In particular, the correlation coefficient determined at each step depends on a previously corrected image and therefore on the previously determined correlation coefficients, which allows the method to converge.

[0013] In particular, when the optical defect to be corrected includes a strong radial component at low or medium frequencies, the step of reducing the dimensions of the first image makes it possible to adapt the filtering step to the characteristic dimensions of the vignetting defect and to remove high-frequency patterns that do not correspond to the vignetting defect to be corrected. Furthermore, the radial filter generally highlights all high-frequency patterns and all image details contained within concentric circles, eliminating longitudinal or vertical patterns. However, in the context of the invention, the application of the radial filter following the reduction of the dimensions of the first image and the calibration table highlights the mid-frequencies of the first reduced image and the reduced calibration table.Thus, the radial filter allows the removal of all patterns not oriented in the direction of the ray and not corresponding to the vignetting defect. Furthermore, the correlation of each calibration table with the first image of a subsequence of images allows the presence of the calibration table to be quantified. The first image is subtracted from the calibration table proportionally to its presence. Finally, the correction is updated by taking into account, for each sub-sequence, the correction of the previous sub-sequence in order to obtain high correction accuracy and allow convergence of the method.

[0014] In addition to the characteristics just mentioned in the preceding paragraph, the method according to an aspect of the invention may have one or more complementary characteristics from among the following, considered individually or according to all technically possible combinations.

[0015] According to one embodiment, the first image of a subsequence of images used at a given iteration corresponds to the last image of a subsequence of images used at the previous iteration.

[0016] According to one embodiment, the method is performed for a set of K calibration tables, where K is a non-zero natural number, and for each given iteration of the set of iterations, the calibration table used is distinct from the calibration table used in the previous iteration. Advantageously, when K calibration tables are used, the method according to the invention can be implemented on an FPGA board, and the linear combination of the tables for determining the correction is performed on the first 2*K images of the subsequences processed in parallel.

[0017] According to one embodiment, the set of K calibration tables is chosen according to a predefined rule from among M calibration tables, M being a non-zero natural number with K less than M, the predefined rule being such that, for each calibration table among the K calibration tables: • the focal length associated with the calibration table corresponds to one of the K focal lengths closest to F'; or • The difference between the focal length associated with the calibration table and F' is less than an average focal length between at least two successive calibration tables among the K calibration tables chosen.

[0018] According to one embodiment, K is between 2 and 10. In particular, the number of calibration tables K can be adapted according to the trade-off between the responsiveness of the method and the desired accuracy. The higher the number of calibration tables, the more accurate the correction, but the longer the convergence time of the method. Thus, K, being between 2 and 10, represents a compromise between the accuracy of the correction and convergence.

[0019] According to one embodiment, the vignetting defect is an additive spatial noise, having a radial and low-frequency component. "Low frequency" is understood to mean a frequency with a period between 10 and 100 pixels.

[0020] According to one embodiment, the reduction in dimensions of the first image and of The calibration table is created using an averaging filter.

[0021] According to one embodiment, the radial filter is a gradient filter or a Laplacian filter oriented in the direction of the center of each image in each subsequence of images. Advantageously, the radial filter makes it possible to remove spurious information that is not in the direction of the radius of the radial vignetting defect, thus increasing robustness.

[0022] According to one embodiment, the correlation coefficient between the first filtered reduced image and the filtered reduced calibration table is determined from the ratio of: • a canonical dot product between the first filtered reduced image and the filtered reduced calibration table; and • of a standard of the reduced filtered calibration table. Advantageously, the dot product between the two images allows the reduced filtered calibration table to be projected onto the first reduced filtered image in order to obtain the amount of table in the scene.

[0023] Another aspect of the invention relates to a computer program product comprising instructions which, when the program is executed by a computer, lead the computer to implement the method according to any one of the preceding embodiments.

[0024] Another aspect of the invention relates to a computer-readable recording medium comprising instructions which, when executed by a computer, lead the computer to implement the method according to any one of the preceding embodiments.

[0025] The invention and its various applications will be better understood by reading the following description and examining the accompanying figures. BRIEF DESCRIPTION OF THE FIGURES

[0026] The figures are presented for illustrative purposes only and are in no way limiting of the invention.

[0027] Fig. 1 is a schematic representation of a cooled infrared optical zoom in which part of the incident optical rays being cut off is the cause of the vignetting.

[0028] Fig. 2 is a synoptic diagram representing steps of a method for correcting a vignetting defect according to the invention.

[0029] Fig. 3 is a representation of the orientation of a radial gradient filter used in the correction method according to the invention.

[0030] Fig. 4 is a synoptic diagram representing steps of a first embodiment of a method for determining a calibration table.

[0031] Fig. 5 is a synoptic diagram representing steps of a second embodiment of the method for determining a calibration table.

[0032] Fig. 6 is a representation of a device configured to implement the method according to the invention. DETAILED DESCRIPTION

[0033] The invention relates to a method for correcting a vignetting defect in a sequence of images.

[0034] The image sequence comprises a plurality of images and is ordered temporally, that is to say that the plurality of images is arranged by increasing acquisition instant in the image sequence.

[0035] Thus, if a first image is acquired at a time t, a second image is acquired at a time t+1, and a third image is acquired at a time t+2, the first image, the second image and the third image will be arranged in that order in a sequence of images comprising them.

[0036] The image sequence is acquired by an infrared camera or a visible camera.

[0037] The image sequence is obtained from a camera located, for example, in a aircraft or in an observation sight located in a land vehicle.

[0038] Thus, the camera makes it possible to acquire image sequences which can be displayed in real time to a user of the aircraft or land vehicle, for example.

[0039] The image sequence comprises a plurality of image subsequences. The image sequence preferably comprises at least two subsequences. A subsequence of images comprises at least two images, for example, and preferably three images. The image subsequence may, in particular, comprise four, five, or even ten images.

[0040] In particular, since the image sequence is acquired for successive instants, the image sub-sequences of the image sequence follow each other temporally.

[0041] In particular, the images of each sub-sequence of images follow each other temporally.

[0042] An "infrared camera" is understood to be a camera configured to measure and record heat waves and infrared radiation emitted by a body or object, in order to generate sequences of images called infrared images.

[0043] The term "visible camera" means a camera configured to measure and record visible light, in the visible spectrum which extends from 300 to 750 nanometers in wavelength in order to generate image sequences in the visible spectrum.

[0044] In particular, the acquisition camera includes a lens with a variable focal length. In the following, "focal length" and "focal length" will be used interchangeably.

[0045] We denote F' the focal length of the camera lens during the acquisition of the image sequence.

[0046] In particular, the image sequences acquired by the camera may include a vignetting defect which reduces image quality and which is corrected according to the method according to the invention.

[0047] According to the invention, the vignetting defect is modeled according to a plurality of hypotheses described below.

[0048] According to a first hypothesis, the vignetting defect is a constant image or "offset," which can be modeled as additive spatial noise that is fixed in time. "Fixed noise" is understood to mean noise whose characteristic evolution time is much greater than the convergence time of the method according to the invention, the characteristic evolution time being a few seconds, while the convergence time of the method according to the invention corresponds to a few images. In particular, when the camera acquires image sequences at a rate between 25 Hz and 50 Hz, the "convergence time of a few images" is understood to be a time of approximately a few hundred milliseconds. Specifically, the characteristic evolution time of the vignetting defect is approximately ten times greater than the convergence time of the method according to the invention.

[0049] According to a second hypothesis, the vignetting defect comprises a low-frequency spatial mean signal. A low-frequency mean a frequency with a period between 10 and 100 pixels.

[0050] According to a third hypothesis, the vignetting defect has a dominant radial spatial component compared to a horizontal or vertical component. A defect with a dominant radial spatial component compared to a horizontal or vertical component is a defect whose average gradient is greatest in the direction of the rays passing through the center of the image. This is visually manifested by concentric patterns on the image containing the vignetting defect. In practice, the concentricity is not exact and the position of the center is difficult to characterize, hence the elimination of radial corrections based on a strict model.An image with a dominant radial component is defined as an image in which the average absolute values ​​of the image filtered by a radial filter are at least five times greater than the average absolute values ​​of the same image filtered by a filter oriented orthogonally to the rays passing through the center of the image.

[0051] The correction method according to the first aspect of the invention is carried out from a set of K calibration tables selected from a set of M calibration tables, with K and M two non-nullified natural numbers and K less than M.

[0052] For example, M is between 1 and 100.

[0053] For example, K is between 1 and 10. For example, K can be equal to 1. In particular, K is preferably equal to 2, 3 or 4.

[0054] Each calibration table among the set of M calibration tables can by For example, this corresponds to an image acquired or modeled for a focal length F'm of the camera lens, with m ranging from 1 to M. Calibration tables include coefficients characterizing the optical vignetting defect as a function of parameters such as the focal length of the camera lens, the camera temperature, and the integration time. Specifically, calibration tables have the same size as the pixel array of the camera detector and therefore the same size as the image sequences acquired by the camera. Calibration tables are typically generated at the factory.

[0055] In particular, the method according to the invention can be carried out from any set of calibration tables, each corresponding to a vignetting defect verifying the assumptions mentioned above and obtained for different focal lengths.

[0056] Examples of calibration tables are detailed later in the description.

[0057] In particular, the correction method according to the invention is carried out for a sequence of images acquired at a focal length F' and the calibration tables from the set of K calibration tables enabling the method to be carried out are selected from among the M calibration tables, the K calibration tables being selected according to a predetermined condition from among the M calibration tables.

[0058] According to one embodiment, the predetermined condition is that the focal length F'k of each calibration table in the set of K calibration tables, with k a natural number between 1 and K, is one of the K focal lengths closest to the focal length F' among the M focal lengths in the set of M calibration tables. In other words, the calibration tables associated with the K focal lengths closest to the focal length F' among the M focal lengths are selected.

[0059] In particular, the K focal lengths closest to F' among the M focal lengths of the set of M defect images are determined by calculating a norm of the difference F'm-F', the norm being for example a Euclidean norm, and by selecting the K minimum norms.

[0060] According to another embodiment, the predetermined condition is that the difference between each focal length F'k and the focal length F' is less than a predefined threshold corresponding to a ratio of the average focal length between at least two successive calibration tables among the K selected calibration tables and the focal length F'. The predefined threshold is, for example, less than or equal to 20%. Thus, instead of selecting a predetermined number of calibration tables, all calibration tables corresponding to a focal length F'k are selected, provided they satisfy the predetermined condition that the difference between the focal length F'k and the focal length F' is less than a predefined threshold.

[0061] In particular, at least one focal length F'ka associated with a calibration table of the set of K calibration tables is different from the focal length F'.

[0062] Thus, the K selected calibration tables vary according to the acquisition focal length F' of the image sequence.

[0063] Figure 2 shows a schematic representation of the 200 correction method of a vignetting defect according to the invention in an image sequence as described above.

[0064] The image sequence is divided into a plurality of image subsequences, each comprising a plurality of successive images within the image sequence. The plurality of image subsequences is therefore ordered temporally within the image sequence.

[0065] For example, the image sequence comprises P image subsequences, P being a non-zero natural number, between 1 and 100.

[0066] Each subsequence of images preferably comprises the same number of images, for example equal to two or preferably to three.

[0067] For example, each image sequence is cut so that a given image subsequence comprises the last image of the image subsequence preceding the given image subsequence in the image sequence and the first image of the image subsequence following the given image subsequence in the image sequence.

[0068] For example, considering a sequence of images comprising seven images SEQ = {IM1, IM2, IM3, IM4, IM5, IM6, IM7], the sequence of images is cut into a first sub-sequence of images SEQi = {IM1, IM2, IM3], a second sub-sequence of images SEQ2 = {IM3, IM4, IM5], a third sub-sequence of images SEQ3 = {IM5, IM6, IM7}.

[0069] Thus, the correction method comprises steps 201, 202, 203, 204, 205, and 206 performed iteratively for each subsequence SEQ of images from a sequence of images acquired by an infrared camera or a visible camera as described above. In particular, the subsequence of images is acquired for a focal length F' of the camera and has a vignetting defect on each of its images.

[0070] In particular, the method comprises a set of iterations, the set comprising a number of iterations preferably equal to the number of sub-sequences SEQ;. At each iteration a unique sequence is processed.

[0071] Thus, consider a given iteration i, with i between 1 and P, the given iteration using an i-th subsequence SEQ of images from the sequence of images to be corrected and on which method 200 is applied. At each iteration i, a calibration table, denoted Tk, from the set of K calibration tables is used.

[0072] In one embodiment, K is equal to 1 and the set of K calibration tables comprises a single calibration table, denoted Th

[0073] In another embodiment, K is greater than or equal to 2 and each iteration in the set of iterations uses a separate calibration table. Preferably, K is less than the total number of iterations in the iteration set and each calibration table is used at least once.

[0074] Thus, consider the set of calibration tables with K equal to 2, for example. The set of calibration tables thus includes the calibration table Ti and the calibration table T2. In this example, if the calibration table Ti is used in iteration i, then the calibration table T2 is used in iteration i+1, and the table Ti is used again in iteration i+2.

[0075] Thus, for the given iteration i, the method includes a step 201 of reducing the dimensions of a first image IMu of the SEQ sequence; by a reduction factor N.

[0076] In particular, the first image IMijest corrected for the vignetting defect by means of a vignetting defect correction, denoted Corri, i)k, determined at the previous iteration, i.e. iteration i-1.

[0077] If the given iteration i is such that i is equal to 1, then there is no previous iteration and the first image IMij is corrected for the vignetting defect from an initial correction, an example of which is given later in the description.

[0078] The reduction factor N is a non-zero natural number between 1 and 64.

[0079] The "dimension" of an image means its size expressed in number of pixels.

[0080] In particular, each image in the image sequence to be corrected comprises L rows of pixels and C columns of pixels, its dimension being equal to L*C pixels, where L and C are two non-null natural numbers. Thus, the same applies to each image in each subsequence of images.

[0081] The first image therefore has a dimension of L*C pixels before the dimension reduction step 201. After the dimension reduction step 201, the number of columns of the first image is divided by N, and the number of rows of the first image is divided by N. Thus, the reduction of the first image IMupermet gives a first reduced image IMRijde dimension A = LL.

[0082] The method includes a step 202 of reducing the dimensions of the calibration table Tk used during the current iteration i, by the reduction factor N.

[0083] Since the calibration table Tk has the same dimensions as the first image IMij, reducing the calibration table Tk allows us to obtain a reduced calibration table TR k of dimension L2L. N2

[0084] The dimension reduction steps 201 and 202 are performed using an averaging filter, which, on each image to which it is applied, averages each successive sub-image of N*N pixels on said image. The image resulting from the reduction thus contains N2 times fewer pixels than the image before reduction. Reducing the dimension of the first image advantageously allows for adapting the Filtering is performed to determine the characteristic dimensions of the vignetting defect, and to remove high-frequency patterns that, by hypothesis, do not belong to the vignetting defect being corrected. Indeed, the vignetting defect arises from the absence of light rays but also from reflections within the lenses of the optical system, which create medium and low-frequency patterns.

[0085] The reduction factor N is adapted to the resolution of the camera sensor and to the nature of the vignetting defect, which can be characterized at the factory. This factor can optionally be dynamically adapted to the focal position or to other parameters that influence the characteristic dimensions of the vignetting defect.

[0086] Step 203 of method 200 is a step of applying a radial high-pass filter to the first reduced image IMu and the reduced calibration table TRk.

[0087] The radial high-pass filter only filters low and mid frequencies. In particular, the radial high-pass filter only allows low and mid frequencies with periods less than 3 pixels to pass through in images whose dimensions are reduced by a factor of N. Thus, following steps 201 and 202, the radial high-pass filter is equivalent to a band-pass filter which only allows frequencies with periods close to 3*N pixels to pass through (in the first reduced image IMu and the reduced calibration table TRk).

[0088] For example, the radial filter is a Laplacian Gaussian (LoG) filter, a Gradient filter oriented in the direction of the center of the first reduced image or any filter based on differences oriented in the direction of the center of the first reduced image.

[0089] The radial filter has a dimension (or size) denoted F*F', with F and F' two non-nuisable natural numbers.

[0090] An example of a radial filter with F = 3 and F' = 3 is as follows:

[0091] / q-f) ' + “ ) 0 + ^) my(0 + 7T) +

[0092] For each image to be filtered and for a given pixel of the image to be filtered, an angle θ is associated which corresponds to the angle formed between a first line passing through the central pixel of the image and the position of said given pixel, and a second horizontal line (in the direction of the lines of the image to be filtered) and passing through the central pixel, the vertex of the angle being the central pixel. In particular, the angle θ is measured between the second line and the first line, and is positive in the trigonometric direction. Thus, several pixels can be associated with the same angle θ.

[0093] In particular, the radial filter comprises concentric circles, each circle corresponding to a set of pixels separated by the same number of pixels from the central pixel.

[0094] In addition, the radial filter calculates, for example, the difference between two diagonal neighboring pixels associated with the same angle 0.

[0095] Advantageously, applying the radial filter to the first reduced image highlights the radial component of the vignetting defect. The radial filter removes all patterns not oriented in the direction of the radial component (or radius), retaining only the patterns of interest and thus highlighting all the high-frequency patterns and all the image details in the direction of the radii of the concentric circles. Indeed, the radial component of the vignetting defect only exhibits patterns in the direction of the radii of the concentric circles. Thus, a radial filter is advantageous compared to a non-oriented filter.

[0096] Figure 3 is an example of the local orientation of the radial filter, which in this case is a gradient filter. Thus, Figure 3 represents the gradient orientation for each pixel position in the first image. The lines shown represent the gradient direction.

[0097] With reference again to [Fig.2], at the end of step 203, a first reduced filtered image IMRFij and a reduced filtered calibration table Tm is obtained.

[0098] Step 204 is a step of determining a correlation coefficient denoted cik, from the first reduced filtered image IMRFij and from the reduced filtered calibration table TRFk.

[0099] The correlation coefficient is determined from a canonical scalar product, whose operator is denoted <.l.>.

[0100] In the following text, we denote v^ the line vector for example comprising all the pixel values ​​of the first reduced filtered image IMRFij, and vT kRF the line vector comprising all the pixel values ​​of the reduced filtered calibration table TRFk.

[0101] Thus, at the given iteration, the correlation coefficient between the first reduced filtered IMRFu image and the reduced filtered calibration table Tj^ is equal to the following term:

[0102] _ (vlRFKr_kRF^ ( vT_kRFÎvT_kRF 1

[0103] In particular, the term { vT kRp|vT kRF ) corresponds to the squared norm of the reduced filtered calibration table Tj^.

[0104] The term (V1rf]vt kRF ) allows comparison of the first reduced filtered image and the reduced filtered calibration table Tj^, the reduced filtered calibration table T^ being projected onto the first reduced filtered image.

[0105] The dot product makes it possible to measure the presence of the TRFk calibration table in the first IMijet image, and therefore to measure the presence of the associated vignetting defect at the TRFk calibration table in the first IMu image

[0106] In particular, the correlation coefficient cik can take any real value. The coefficient cik can thus be positive or negative, for example. It is also possible, depending on the normalization of the calibration tables, to obtain a correlation coefficient cik with an amplitude greater than 1.

[0107] The correlation coefficient of correlation cik thus makes it possible to quantify the similarities between the first reduced filtered IMRFu image and the reduced filtered calibration table TRFk and to determine to what extent the vignetting defect of the calibration table considered is present in the first image.

[0108] Step 205 is a step of determining a Corrik correction of the vignetting defect from the TRFk calibration table, of the correlation coefficient cik obtained in step 204 from the filtered reduced calibration table TRFket of the Corrt 4 )k correction of the vignetting defect determined in the previous iteration, i.e. iteration i-1

[0109] If the given iteration i is such that i is equal to 1, then there is no previous iteration and the correction Corr(i4)k corresponds to the initial correction Corr0.

[0110] In particular, the Corrik correction of the vignetting defect is for example equal to a sum of the Corr(i4)kob correction held at the previous iteration and a product of the given calibration table multiplied by the correlation coefficient cik associated with it.

[0111] Thus, a Corrik correction of the vignetting defect corresponds to an image of the same size as each image of a subsequence of images acquired by the camera.

[0112] Thus, Corr^_ CoiT(.i)t +Cjt., Tk =

[0113] Thus, the correction is updated at each iteration i, for each subsequence of SEQ images; and depends on the correlation coefficients determined at the iterations preceding iteration i, in order to allow the convergence of the method 200.

[0114] Thanks to the iterative aspect of the method, it converges: the calculation of the correlation coefficient for a calibration table at iteration i is calculated from the first image corrected by the correction previously determined at iteration (i-1) to take into account the contributions of the calibration tables used in the previous iterations.

[0115] Step 206 is a correction step for each other image in the given SEQi image subsequence from the Corrik correction.

[0116] For example, if the image sequence SEQ; is noted: { IMu, IM2 i, IM3J} then steps 201 to 205 are carried out on image IMi and the correction step 206 is carried out on the second and third images IM2j and IM3i in order to obtain second and third corrected images respectively IM'2J and IM'3.

[0117] In particular, the second corrected image IM'2J is obtained, for example, by sub traction of the Corrik correction to the second image IM2 j of the SEQ subsequence; of images.

[0118] Thus, IM'2 i = IM2; - 0¾

[0119] Image IM'2j thus corresponds to image IM2 after correction of the defect of vi gnettage according to method 200.

[0120] Thus, IM'3i = IM3 i - 00¾.

[0121] In particular, if the image subsequences include, for example, a plurality of images greater than three images, the correction is preferably applied to all images in the image subsequences except for the first image.

[0122] At the next iteration, that is to say at iteration i+1, the subsequence SEQ(i+i) includes the last image of the subsequence SEQ; as first image

[0123] Thus, in this embodiment, SEQ^ is noted: { IMi_(i+i), IM2 (i+i), IM3_(i+i)} with IMi_(i+i)= IM'3i.

[0124] With reference to the initial correction Corr0 used in method 200 according to the invention, the initial correction is a matrix of the same size as the camera detector and can be obtained by a one-point camera calibration method. The one-point calibration of a cooled infrared camera is performed by fitting a gain table G and an offset table O, the fitting being carried out using a defocused image or an image of a camera shutter, used to obtain an assumed uniform image on the sensor. The gain G and the offset O are matrices of the same size as the camera detector.

[0125] In particular, the response of a pixel of the camera detector to a light flux can be characterized by a linear function characterized by a slope representing a gain and a y-intercept representing a shift. Each pixel has its own gain and its own shift. Consequently, upon receiving a uniform light flux, the gray levels generated by the pixels differ, resulting in the appearance of non-uniformities in the image generated by the camera.

[0126] In order to eliminate pixel non-uniformities, it is necessary that the pixels have the same gain and the same offset. A correction is performed by applying the gain correction table G and the offset correction table denoted O.

[0127] The gain table G is multiplied by the flux received by each pixel and is expressed as a table of factors applied to each pixel, which is determined at the factory before the camera is put into operation, for example. After its application, all pixels have the same response to a given flux variation, and their characteristic functions therefore have the same slope. The gain table can be determined by performing the first-order partial derivative of two images taken at two different scene temperatures. In particular, the partial derivative is performed by related to the luminance parameter which depends on temperature.

[0128] The O offset table is added to the flux received by each pixel, and is therefore expressed as a table of coefficients added to each pixel. The O offset table makes it possible to correct defects both intrinsic to the detector and due to parasitic cold fluxes.

[0129] The shift table O is obtained from the image of a scene acquired in front of a diaphragm, denoted Idiaphragm, multiplied by the gain table G. In particular, the shift table O is obtained from the following formula:

[0130] O = Diaphragm*G-Average(Diaphragm*G). Thus, the initial correction is equal to table O.

[0131] Regarding the calibration table(s) used in the correction method according to the invention, several embodiments of these tables are described below.

[0132] According to one embodiment, a calibration table can result from the difference between two images acquired from the same uniform background, respectively with different focal lengths of the camera. Thus, in order to construct M defect images, it is possible to acquire two series of M images, each from the same uniform background, the two series being acquired at two different focal lengths, the M defect images being obtained by subtracting the images from each series two by two.

[0133] According to one embodiment, the calibration tables may be those cited in US patent 8645104B2, figures 1 and 2 of said US patent allowing the generation of correction tables called "tables for correction" which correspond to the calibration tables.

[0134] According to one embodiment, for each focal length F';, the calibration table T; associated with the focal length F'; can be determined according to a method for determining the calibration table, the method of determination having a first and a second distinct embodiment.

[0135] Figure 4 presents a block diagram of the steps of the first embodiment of method 300 for determining the calibration table T associated with the focal length F'. In particular, each calibration table T is associated with an internal calibration temperature of the camera and a calibration integration time of the camera.

[0136] The term "internal camera temperature" refers to the temperature acquired near the optics of the camera lens, and in particular near the front lens of the lens, for example. In the following, any value of the internal camera temperature is, for example, within the range [-20; +70] °C.

[0137] Method 300 includes a step 301 of acquiring by the camera an image Isj of a thermal flux emitted by a source, for the focal length F'i of the camera, the internal calibration temperature Tcalibration of the camera and the calibration integration time from the camera which are stored by a user in a memory external to the camera for example.

[0138] The source emits, for example, a heat flux (or thermal radiation) similar to the heat flux emitted by a black body, that is, an isotropic heat flux, meaning the radiation depends only on the temperature and not on directions in space, for example. Advantageously, using a source emitting isotropic radiation makes it possible to have a homogeneous scene and to observe only the optical defects of the camera.

[0139] In practice, the source can be a heating plate entirely covered with black paint, and having a uniform, constant temperature that can be adjusted by a user. Thus, the source can be considered a black body.

[0140] The thermal flux emitted by the source covers the entire pupil of the camera lens.

[0141] Method 300 includes a step 302 of calibrating the image Isj according to the one-point calibration method of the camera. The one-point calibration is performed using the gain table G and the offset table O described previously. After the calibration step, an image of the calibrated source Is2j is obtained from the image of the source IS i, such that:

[0142] IS2_i= G*IS iO

[0143] The IS2jobtenue image corresponds to the image of the source Isj corrected for detector defects and includes residual optical defects as mentioned above, i.e. residual optical defects corresponding to parasitic thermal fluxes of the camera lens relative to the focal length F';.

[0144] Method 300 further includes a step 303 of obtaining a third image IS3_i resulting from the subtraction of an average image of the image IS2J, denoted Average(IS2J), from the calibrated image IS2j. In particular, Average(IS2_i) corresponds to an image of the same size as IS2j and in which each coefficient corresponds to the average of all the pixel values ​​of IS2j.

[0145] Thus, IS3i = IS2 i - Average(lS2j)-

[0146] According to a first embodiment, the calibration table T; associated with the focal length F'; corresponds to IS3 jet can therefore be noted: T, = Issj.

[0147] Thus, the calibration table T; can be obtained at the end of step 303.

[0148] According to a second embodiment, method 300 includes an optional step 304 of spatial filtering of the third image IS3j to obtain a filtered image F(I S3_i)-

[0149] The filtering can, for example, correspond to a spatial filtering of the IS3j image in the case where intrinsic defects of the detector remain even after the calibration step 301, for example because of one or more defective pixels of the detector. In this second embodiment, the calibration table T; can be equal to F(IS3j).

[0150] According to a third embodiment, the calibration table T; is obtained at the end of optional steps not shown in method 300, described below.

[0151] Method 300 may include an optional first step of carrying out steps 301, 302 and 303 at several different times t and obtaining a plurality of images (IS3_it)t>o

[0152] Method 300 may include an optional second step of averaging the IS3i_t images to obtain an average image Average(IS3_it)t>o. In particular, the averaging step 306 corresponds to averaging each pixel value of each IS3jt image having the same position in the image. In other words, each coefficient at a given position in the image Average(IS3_it)t>o corresponds to an average of the pixel values ​​at the same position in each IS3_it image.

[0153] Method 300 may include an optional third step of obtaining the calibration table T; by subtracting the Average image(Is3 it) from the IS3jt image. Thus, Ti = W Average(IS3jt).

[0154] In particular, the third embodiment advantageously makes it possible to remove temporal noise present in the IS3j image, temporal noise degrading the quality of the image by giving it a grainy appearance.

[0155] According to an embodiment not shown, the optional steps are applied to the image F(IS3_i) following step 304.

[0156] The method 300 for obtaining the calibration table T associated with the focal length F' can be carried out for a plurality of M focal lengths rp j, which makes it possible to obtain M first calibration tables qA each associated with a focal length different F';, the index i being a natural number. In particular, each calibration table among the first M calibration tables is obtained for a different focal length but with the same internal calibration temperature of the camera and the same calibration integration time of the camera.

[0157] In particular, each calibration table can be compressed before being saved in memory for use. The compression method must be adapted to the spatial morphology of the residual optical defects represented by each calibration table, in order to avoid any degradation of the calibration table's quality.

[0158] Fig. 5 presents a synoptic diagram of the steps of the second embodiment, the method 400 for determining the calibration table T associated with the focal length F'i.

[0159] Method 400 includes a step 401 of acquiring by the camera a first image IlSj of a thermal flux emitted by the source defined in the method of determination calibration of the calibration table. In particular, the acquisition step 401 is carried out for the focal length F'; of the camera, at an internal calibration temperature of the camera and with the calibration integration time of the camera which are stored by a user in an external memory to the camera for example or in a memory of the camera.

[0160] Method 400 further includes a step 402 identical to step 302, step 402 being applied to the image Ilsj and allowing the calibrated image IlS2_i to be obtained. The calibrated image IlS2j corresponds to the image Ilsj corrected for detector defects and including residual optical defects as described above. The residual optical defects correspond to parasitic heat fluxes from the camera lens for the focal length F'.

[0161] Method 400 further includes a step 403 of obtaining an image IlS3j resulting from the subtraction of an average image of the image IlS2j, denoted Average(IlS2_i) from the calibrated image IlS2j- In particular, Average(IlS2j) corresponds to an image of the same size as IlS2j and of which each coefficient corresponds to the average of all the pixel values ​​of IlS2_i. Thus, IlS3_i J1S2j - Average(IlS2j).

[0162] Method 400 further includes a step 404 of varying the internal temperature of the camera, to obtain an internal temperature T2 calibration of the camera different from T1 calibration. The variation of the internal temperature of the camera can be achieved by placing the camera in a temperature-adjustable oven, for example.

[0163] Method 400 includes a step 405 of acquisition by the camera of a second image I2Sj of the heat flux source defined in method 200, for the focal length F'i of the camera, the internal calibration temperature T2etaionnage of the camera and the calibration integration time tetaionnage of the camera.

[0164] Method 400 further includes a step 406 identical to step 302, step 406 being applied to the image I2SJ and allowing the calibrated image I2S2_i to be obtained. The calibrated image I2S2_i corresponds to the image I2s_i Corrected for detector defects and including residual optical defects as mentioned above.

[0165] Method 400 further includes a step 407 of obtaining an image rS3j resulting from the subtraction of an average image of the image I2S2_i, denoted Average(I2S2 j), from the calibrated image I2S2_i. In particular, Average(I2S2j) corresponds to an image of the same size as I2S2_i and in which each coefficient corresponds to the average of all the pixel values ​​of I2S2j. Thus, I2S3j = 12^ - Average(I2S2j)-

[0166] Method 400 includes a step 408 of obtaining the calibration table T; from the image IlS3j, the calibration temperature Tlétaionnage, the image I2S3j and the calibration temperature T2étaionnage.

[0167] In particular, the calibration table T; allows the variation of defects to be estimated Residual optical defects are analyzed as a function of internal temperature variation. An interpolation is performed between the variation of these residual optical defects and the variation in internal temperature, with the interpolation order chosen according to predetermined criteria. Indeed, the higher the interpolation order, the better the image quality for correcting residual optical defects, because the relationship between the variation of residual optical defects and the variation in internal temperature becomes more precise. However, the higher the interpolation order, the more complex the 400 method becomes to implement, as it requires multiple blackbody image acquisitions.

[0168] In particular, the variation of residual optical defects can depend linearly and indirectly on the variation of internal temperature. Indeed, the variation of residual optical defects depends linearly on the variation of integrated luminance, which is temperature-dependent.

[0169] Thus, the calibration table T; is obtained according to the following formula:

[0170] y = L(Thla <onbage>«T2éwtaage)

[0171] with L(Tlétaionnage) the integrated luminance generated by the temperature Tlétaionnage and L(T2 harmonisation) the integrated luminance generated by the temperature T2étaionnage.

[0172] Thus, the integrated luminance is related to the internal temperature T that the camera integrates (captures). The gray level captured by the camera depends on a multitude of sources. For interpolation purposes, a simulation of the variation of gray levels captured by the detector as a function of the variation of the internal temperature is performed. The determination of the luminance as a function of temperature (denoted L(T1) and L(T2)) is obtained, for example, via a simulation method of the infrared camera.

[0173] In particular, the integrated luminance takes into account a fixed temperature T of a scene integrated by the camera and passing through the optics, the internal luminance of the camera (i.e. the photons leaving the camera and reaching the detector) and the dark current of the camera detector.

[0174] When the infrared camera is placed in front of a scene with a fixed temperature T, the internal temperature of the camera is varied in order to observe the evolution of the variation of the grey level as a function of the variation of the internal luminance of the camera.

[0175] The integrated luminances are obtained from a resolution of Planck's equations in a context applied to the camera, i.e. taking into account the variation of grey level as a function of the variation of the internal luminance of the camera.

[0176] The use of luminance values ​​as a function of temperature is advantageous because the luminance of a pixel of an image is modeled linearly as a function of its grey level, for any temperature.

[0177] Method 400 of obtaining the calibration table T associated with the focal length F'i can can be performed for a plurality of focal lengths M / p A, which allows obtaining M "Ei <m second calibration tables (rp) each associated with a focal length different F';. In particular, each calibration table among the first M calibration tables is obtained for a different focal length but with the same calibration integration time of the camera.

[0178] Another aspect of the invention relates to a device configured to implement the method according to the invention.

[0179] Figure 6 represents device 500 according to one or more embodiments of the invention.

[0180] In these embodiments, the device includes a computer 500, comprising a memory 501 for storing defect images and / or image sequences for example.

[0181] The computer 500 further comprises a circuit 502. This circuit may be, for example, a processor capable of interpreting instructions in the form of a computer program, an electronic board whose steps of the method of the invention are described in silicon, or a programmable electronic chip such as an FPGA (Field-Programmable Gate Array) chip. In particular, steps 201 to 205 may be programmed to be implemented on the FPGA chip.

[0182] The computer 500 includes an input interface 503 for receiving image sequences acquired by the camera, and an output interface 504. Finally, the computer may include, to allow easy interaction with a user, a screen 505 displaying the image sequences acquired by the camera, calibration tables and corrected images for example.

[0183] The computer 500 may also include a keyboard 506. Of course, the keyboard is optional, particularly in the context of a computer in the form of a touch tablet, for example.

[0184] Of course, the present invention is not limited to the embodiments described above by way of example; it extends to other variants.< / m> < / onbage>

Claims

Claims

1. A computer-implemented method (200) for correcting a vignetting defect in a sequence of images acquired by a camera for a focal length F', the sequence of images being temporally ordered and comprising a plurality of temporally ordered sub-sequences of images, the method comprising a set of iterations, each iteration using a sub-sequence of images from the plurality of sub-sequences of images and a calibration table, each iteration comprising: - Reducing (201) dimensions of a first image of the sequence of images to obtain a first reduced image, the first image being corrected for the vignetting defect from a correction of the vignetting defect determined at the previous iteration or from an initial correction; - Reducing (202) dimensions of the calibration table to obtain a reduced calibration table;- Applying (203) a high-pass radial filter to the first reduced image and to the reduced calibration table to obtain a first filtered reduced image and a filtered reduced calibration table; - Determining (204) a correlation coefficient between the first filtered reduced image and the filtered reduced calibration table; - Determining (205) a correction of the vignetting defect from the calibration table, the correlation coefficient associated with the calibration table and the correction of the vignetting defect obtained in the previous iteration; - Correcting (206) each other image of the sub-sequence of images from the determined defect correction to obtain each other image corrected for the vignetting defect.;

2. Method (200) according to the preceding claim according to which the first image of a sub-sequence of images used at a given iteration corresponds to the last image of a sub-sequence of images used at the previous iteration.

3. Method (200) according to one of the preceding claims according to which method (200) is performed for a set of K calibration tables, K being a non-zero natural integer, and for each given iteration of the set of iterations, the calibration table used is distinct from the calibration table used during the previous iteration.

4. Method (200) according to the preceding claim according to which the set of K calibration tables is chosen according to a predefined rule from among M calibration tables, M being a non-zero natural integer with K less than M, the predefined rule being such that, for each calibration table from among the K calibration tables: - the focal length associated with the calibration table corresponds to one of the K focal lengths closest to F'; or - The difference between the focal length associated with the calibration table and F' is less than an average focal length between at least two successive calibration tables from among the K chosen calibration tables.

5. Method (200) according to one of claims 3 to 4 according to which K is between 2 and 10.

6. Method (200) according to one of the preceding claims according to which the vignetting defect is a spatial additive noise, having a radial and low frequency component.

7. Method (200) according to one of the preceding claims according to which the reduction of dimensions of the first image and of the calibration table is carried out by an averaging filter.

8. Method (200) according to one of the preceding claims according to which the radial filter is a gradient filter or a Laplacian filter oriented in the direction of the center of each image of the sub-sequence of images.

9. Method (200) according to one of the preceding claims, according to which the correlation coefficient between the first filtered reduced image and the filtered reduced calibration table is determined from the ratio: • of a canonical scalar product between the first filtered reduced image and the filtered reduced calibration table; and • of a norm of the filtered reduced calibration table.

10. Computer program product comprising instructions which, when the program is executed by a computer, cause the latter to implement the method according to one of the preceding claims.

11. A computer-readable recording medium comprising instructions which, when executed by a computer, cause the computer to carry out the method according to one of claims 1 to 9.

Images