Image processing method

Abstract

One aspect of the invention relates to an image processing method (100) applied to images successively acquired by an image sensor. The method (100) according to the invention provides a solution to the problems mentioned above by enabling the correction of defective pixels based solely on previously processed and healthy pixels. Furthermore, the method according to the invention detects and corrects defective pixels recursively, that is, continuously by correcting pixels as the scan progresses, which ensures optimal correction of defective pixels even when they form a cluster, since the method (100) according to the invention allows the propagation of the value of healthy pixels within these clusters of defective pixels. Figure to be published with the abstract: Figure 1

Description

Title of the invention: Image processing method TECHNICAL FIELD OF THE INVENTION

[0001] The technical field of the invention is that of image processing.

[0002] The present invention relates in particular to an image processing method and an image processing device. TECHNOLOGICAL BACKGROUND OF THE INVENTION

[0003] Optronic systems such as cameras, video cameras, binoculars, telescopes, sights, gyrostabilized balls (GSBs), and homing devices, for example, equipping an airborne observation system, are known to exist. These optronic systems include image acquisition devices comprising at least one image sensor capable of acquiring images in various frequency ranges, such as a frequency range corresponding to frequencies perceptible to the human eye or a frequency range in the infrared, for example in the mid-infrared, i.e., for wavelengths between 3000 and 5000 nanometers, or the far-infrared, i.e., for wavelengths between 8000 and 14000 nanometers.

[0004] An image sensor provides images in the form of a pixel matrix. An image sensor consists of a grid of active elements, called photosites, capable of converting an incident beam of light into an electrical signal. Each pixel of an image corresponds to a photosite on the image sensor. It is common for some photosites of the image sensor to have a defect, rendering them unable to provide a valid pixel value. Thus, the response of some pixels in the matrix does not follow a predictable law based on the received light intensity. These pixels are therefore called defective pixels since it is not possible to directly estimate the value of the input flux for these defective pixels.

[0005] In order to maintain a consistent image for all pixels, these defective pixels must be detected and corrected by image processing algorithms. Various methods for managing defective pixels are currently known.

[0006] So-called "factory calibration" methods are based, for example, on a uniform scene or an arbitrary scene in order to detect and correct defective pixels in an image. Factory detection of defective pixels consists of detecting pixels that do not follow a linear function of luminance by imaging a blackbody at several temperatures. The blackbody ensures an equal input flux for all pixels. Pixels whose response deviates sufficiently from a predefined linear model are considered defective and are recorded in a table of defective pixels. Detection on a uniform scene is performed on an onboard mechanical cover which This method allows the incoming light to be interrupted during processing using a shutter. This detection on a uniform scene can also be performed by blurring the image through defocusing or by averaging several images of a moving scene. When the incoming light is uniform, pixels whose intensity deviates (beyond a predefined threshold) from their neighbors, relative to the sensor's non-uniformity noise, are considered defective. These so-called "factory calibration" or shutter-based methods only correct the pixels detected during factory calibration or during shutter calibration. Therefore, this method is not suitable for certain sensors (such as cooled infrared image sensors based on micro-carbon technology) where defective pixels can randomly switch between a healthy and a defective state.

[0007] There are also methods in which defective pixel detection can be performed on a real scene (also called dynamic detection) based on the difference, for each pixel in the array, between the actual value of the current pixel and a predicted value of the current pixel. Existing methods predict the value of the current pixel using the values ​​of neighboring pixels in time and / or space. The values ​​of neighboring pixels in time can be used when several successive images from the same sensor array of a photodetector are processed. Thus, the neighboring pixels in time are the pixels having the same position as the pixel in question but for the previous image(s). Neighboring pixels in space can also be used, particularly when the method uses only a single image.Neighboring pixels in space are the pixels surrounding the pixel in question within the image containing that pixel. The predicted pixel values ​​can also be reused for pixel value correction, thus pooling computational resources for both detection and correction. Pixel value prediction is often performed by averaging the values ​​of neighboring pixels. Scene detection methods that operate on a single image are susceptible to false detections of defective pixels because current methods rely on isotropic filters and do not account for pixel temporality. Furthermore, high-contrast elements in the scene are likely to be detected as defective pixels and removed from the processed image.

[0008] Other methods for treating defective pixels use more complex techniques, particularly those based on dense motion estimates, such as the method disclosed in patent FR3028376B1, or using iterative neighborhood search methods, which are much more computationally expensive and therefore poorly suited for use in embedded systems. Furthermore, these techniques are poorly suited for implementation on a programmable logic circuit, commonly called an FPGA for "Field-programmable gate array". Finally, these techniques are sensitive to noise and defective pixels, which can generate numerous estimation errors.

[0009] Finally, methods specific to devices with color sensors have also been developed. These methods for managing defective pixels rely on the specific characteristics of color filters, for example, Bayer patterns. These methods are therefore not applicable to monochrome cameras.

[0010] There is therefore a need to provide an image processing method limiting, at least in part, the problems associated with the use of the aforementioned prior art methods. Summary of the invention

[0011] The method according to the invention offers a solution to the problems mentioned above by enabling the correction of defective pixels based solely on previously processed and healthy pixels. Furthermore, the method according to the invention detects and corrects defective pixels recursively, that is, continuously by correcting pixels as the scan progresses, which ensures optimal correction of defective pixels even when they form a cluster, since the method according to the invention allows the propagation of the value of healthy pixels within these clusters of defective pixels.

[0012] One aspect of the invention relates to an image processing method applied to images taken successively by an image sensor, each image among the successively taken images comprising a pixel matrix, each pixel of the pixel matrix corresponding to an active element of the image sensor capable of converting an incident beam of light into an electrical signal, each pixel being associated with a value corresponding to a measure of the light intensity received by that pixel, and each pixel being in one of a state among an unprocessed state, a healthy state and a defective state, the method iteratively processing the successively taken images and comprising, for each current image, an initial step of initializing the state of each pixel of the pixel matrix of the current image to the unprocessed state and then subsequent steps performed recursively for each current pixel of the current image, the subsequent steps being: • Calculation of a prediction value for the current pixel, the calculation of the prediction value being performed using a recursive averaging filter, the recursive averaging filter using the value of at least one healthy pixel to calculate the prediction value of the current pixel, the at least one healthy pixel used being at least one pixel chosen from: • At least one pixel vertically adjacent to the current pixel, • At least one pixel horizontally adjacent to the current pixel, • At least one pixel diagonally adjacent to the current pixel, • At least one healthy pixel whose value has previously been replaced by the prediction value, • Determining the state of the current pixel using hysteresis thresholding of a confidence counter, where the confidence counter value is determined by comparing the value of the current pixel with the predicted value of the current pixel, and • when the current pixel is defective, replace the value of the current pixel with the predicted value of the current pixel and change the state of the current pixel to the healthy state.

[0013] Thanks to the invention, it is possible to detect and correct defective pixels in order to obtain a processed image free of defective pixels, thus improving the visual quality of the restored image. Furthermore, the method according to the invention is resource-efficient, particularly due to its recursive nature, which allows it to be adapted for use in embedded systems. Thus, the method according to the invention reduces hardware resources and power consumption and facilitates real-time detection of defective pixels on the sensor. This correction of defective pixels, which is locally adaptive to the textures of the scene and performed recursively—that is, continuously by correcting pixels as the image is scanned—makes this method particularly well-suited for implementation, for example, on an FPGA-type computer.

[0014] In addition to the characteristics mentioned in the preceding paragraph, the method according to one aspect of the invention may have one or more additional characteristics from among the following, considered individually or in all technically possible combinations: • Determining the value of the confidence counter includes: • an increment of the confidence counter when a difference between the value of the current pixel and the predicted value of the current pixel exceeds a first predetermined threshold, and • a decrement of the confidence counter when a difference between the value of the current pixel and the predicted value of the current pixel is less than or equal to the first predetermined threshold, and in which the state of the current pixel is determined as: • healthy when the confidence counter is less than or equal to a second predetermined threshold, or • defective when the confidence counter is above a second predetermined threshold; the incrementing or decrementing of the confidence counter of the current pixel of the current image is performed only when, for the current image, an overall movement of a scene on the current image is greater than a third predetermined threshold; the choice of said at least one healthy pixel whose value is used to calculate the predicted value of the current pixel is made taking into account local gradients of the current image, the local gradients of the current image being determined by an oriented high-frequency filter; said at least one healthy pixel chosen whose value is used to predict the value of the current pixel is: • At least one horizontally adjacent pixel to the current pixel when: W^current^ \FV{Pixdcurrent^ + $ With: * \FH(Pixel J' a standard of the horizontal high-frequency filter, evaluating the amplitude of the horizontal variation of intensities and obtained by using the value of the horizontal neighboring pixel in order to predict the value of the current pixel, * \FV(Pixel a standard of the vertical high-frequency filter, evaluating the amplitude of the vertical variation of intensities and obtained by using the value of the vertical neighboring pixel in order to predict the value of the current pixel, and • s, a threshold of texture homogeneity, The at least one vertically adjacent pixel of the current pixel when: \FV(PlXel current, fluent / • At least one horizontal neighboring pixel of the current pixel and one vertical neighboring pixel of the current pixel when: \FH(Pix&lcurrent^ " \FH(_PiX€lcurrent^ $ • The initialization steps, calculation of a predicted value for the current pixel, determination of the current pixel's state, replacement of the current pixel's value with the predicted value, and modification of the current pixel's state to a healthy state are implemented by a programmable logic circuit in which: • The vertical neighbor pixel of the current pixel is the upper neighbor pixel of the current pixel. • The horizontal neighboring pixel of the current pixel is the left neighboring pixel of the current pixel. • A pixel diagonally adjacent to the current pixel is the upper left or upper right neighboring pixel of the current pixel.

[0015] Another aspect of the invention relates to an image processing device comprising a pixel processing module configured to process pixels from at least one image from an image sensor, by implementing the image processing method according to the invention.

[0016] An additional aspect of the invention relates to an optronic system comprising an image acquisition device equipped with an image sensor, an image processing device according to the invention and an image display and / or image storage device.

[0017] A complementary aspect of the invention relates to an aircraft comprising an optronic system according to the invention.

[0018] Another aspect of the invention relates to a computer program product, characterized in that it includes instructions to implement, by a device, the process according to the invention, when said program is executed by a processor of said device.

[0019] An additional aspect of the invention relates to a computer-readable recording medium comprising instructions which, when executed by a computer, cause the computer to implement the process according to the invention.

[0020] The invention and its various applications will be better understood by reading the following description and examining the accompanying figures. BRIEF DESCRIPTION OF THE FIGURES

[0021] The figures are presented for illustrative purposes only and are in no way limiting of the invention. • Figure [1] is a block diagram illustrating the steps of an example of the image processing method according to the invention • Figure 2 is a schematic representation of an example system optronics comprising an image processing device capable of implementing the invention, • Fig. 3 schematically illustrates an example of the hardware architecture of a device capable of implementing the invention. • Fig. 4 is a schematic representation of an image comprising a pixel matrix that can be processed using the method according to the invention. DETAILED DESCRIPTION

[0022] Unless otherwise specified, the same element appearing on different figures has a unique reference.

[0023] Fig. 1 is a synoptic diagram illustrating the steps of an example of the image processing method 100 according to the invention.

[0024] The method 100 can be implemented in the broader context of an optronic system comprising an image acquisition device and an image processing device. For example, the present invention can be applied to a camera, a video camera, binoculars, a telescope, a viewfinder, a gyrostabilized ball (GSB), a homing device, or any device using a matrix detector. In this broader context, the invention offers an efficient solution enabling, for example, the optronic system to apply post-processing to images in order to mitigate degradation caused by defective pixels in an image. In one example, the method 100 is applied to images in a frequency range located in the mid- or far-infrared.

[0025] Figure 2 schematically represents an example of an optronic system 200 comprising an image processing device 220 capable of implementing the method 100 according to the invention. Furthermore, the optronic system 200 includes an image acquisition device 210 comprising an image sensor 211. In addition, the optronic system 200 optionally includes a display device 230. A communication bus 240 enables the image acquisition device 210, the image processing device 220, and the optional display device 230 to communicate. For example, the communication bus 240 allows the image acquisition device 210 to provide raw, i.e. unprocessed images to the image processing device 220. In addition, the communication bus 240 allows the image processing device 220 to provide output data to the display device 230, such as images whose defective pixels have been processed.In one embodiment, the image processing device 220 includes an image processing module 221, which is, for example, a software module.

[0026] In one embodiment, the optronic system 200 may also include a communication device (not shown) and / or a storage device (not shown). The communication device enables the communication of output data, for example, processed images, from the image processing device 220 to a remote device (not shown). The storage device enables the storage of output data from the image processing device 220.

[0027] Figure 3 schematically illustrates an example of the hardware architecture of a device capable of implementing the image processing method according to the invention. In the example in Figure 3, the hardware architecture 300 is that of the image processing device 220. According to the hardware architecture example shown in Figure 3, the image processing device then comprises, connected by a communication bus 360: a processor or CPU (Central Processing Unit) 310; a random access memory (RAM) 320; a read-only memory (ROM) 330; a storage unit such as a hard disk drive (HDD) and / or a storage media reader, such as an SD card reader (Secure Digital) 340; and optionally at least one communication interface 350 enabling, for example, the image processing device to communicate with a remote device.

[0028] The processor 310 is capable of executing instructions loaded into RAM 320 from ROM 330, external memory (not shown), a storage medium (such as an SD card), or a communication network. When the image processing device is powered on, the processor 310 is capable of reading instructions from RAM 320 and executing them. These instructions form a computer program causing the processor 310 to implement all or part of the algorithms and steps described in connection with the method 100 according to the invention.

[0029] All or part of the algorithms and steps described in relation to the method 100 according to the invention can be implemented in hardware form by a dedicated device, machine or component, for example a programmable logic circuit.

[0030] Method 100 is an image processing method consisting of detecting and then correcting, i.e., processing, defective pixels in successively received raw images. These images to be processed are acquired successively by an image sensor. The images comprise a pixel matrix. Each pixel in the image corresponds to an active element of the image sensor capable of converting an incident beam of light into an electrical signal. Furthermore, each pixel in the image has a position in the pixel matrix. The position of each pixel can be expressed by a row index and a column index to which the pixel belongs. For an image comprising a pixel matrix with m rows and n columns, the position of any pixel in the image can be expressed in the form (x; y) with x between [0; ml] and y between [0; nl]. For example, the images to be processed with Method 100 can have a minimum resolution of 4 by 4 pixels.Furthermore, each pixel in the image is associated with a value corresponding to a measurement of the light intensity received by the active element of the image sensor corresponding to that pixel. Finally, each pixel in the image is also associated with a state value representing a state of that pixel. This state is determined using process 100. A pixel can have one of the following three states. The first state is the "unprocessed" state, meaning that this pixel has not yet been processed by process 100, and thus its state has not yet been predicted. The second state is the healthy state, corresponding to a pixel whose operation is considered normal, or more precisely, the operation of the active element of the image sensor corresponding to the pixel is considered normal, meaning that the response of this... The pixel follows a predictable law based on the light intensity received by that pixel. The third state is the defective state corresponding to a pixel whose operation is considered abnormal, or more precisely, the operation of the active element of the image sensor corresponding to the pixel is considered abnormal; that is, the response of that pixel does not follow a predictable law based on the light intensity received by that pixel.

[0031] Process 100 is performed iteratively for each image to be processed. Thus, one iteration of Process 100, i.e., comprising the four steps 110 to 140 of Process 100, is performed for each image to be processed. For example, when a new image is taken by the image sensor, the four steps of Process 100 are performed to process the raw image, called the current image. In addition, steps 120 to 140 are performed recursively for each pixel of the pixel matrix of the current image, the pixel of interest for these steps 120 to 140 being called the current pixel. For example, steps 120 to 140 are performed by considering the pixels of the pixel matrix continuously and in a predefined scanning direction. For example, steps 120 to 140 are performed by successively considering the pixels of the pixel matrix from left to right and from top to bottom.

[0032] A step 110 of the process 100 includes initializing the state of each pixel of the current image to the "unprocessed" state.

[0033] A step 120 of the process 100 includes calculating a prediction value for the current pixel. The calculation of the prediction value of the current pixel is performed using a recursive averaging filter. The recursive averaging filter uses, in particular, the value of at least one healthy pixel to calculate the prediction value of the current pixel. The number of healthy pixels used in calculating the prediction value of the current pixel depends on the size of the recursive averaging filter. This size can be variable. The at least one healthy pixel whose value is used can vary depending on different parameters. The selection of the at least one healthy pixel is made from a set of confidence pixels comprising: • At least one pixel vertically adjacent to the current pixel, • At least one pixel horizontally adjacent to the current pixel, • At least one pixel diagonally adjacent to the current pixel, • At least one healthy pixel whose value has previously been replaced by the predictive value.

[0034] The terms "vertical neighbor," "horizontal neighbor," and "diagonal neighbor" refer to neighbors in the space of the current pixel. The term "neighbor in space" means, in this application, "neighbor of rank R" with R < m-2 and R < n-2 for a pixel matrix with m rows and n columns. Thus, when R=l, the pixels neighboring the current pixel, called neighboring pixels of rank 1, are the 8 pixels positioned next to the current pixel in the pixel matrix, i.e., the pixels located to the left, diagonally above and to the left, above, diagonally above and to the right, to the right, diagonally below and to the right, below, diagonally below, and to the left of the current pixel. When R=2, the pixels adjacent to the current pixel are the pixels adjacent to rank 1 as well as the pixels adjacent to the pixels adjacent to rank 1. For clarity only, an example with R=1 is detailed later in the application.

[0035] Each pixel in the pixel matrix has one or two vertical neighboring pixels positioned in the rows before and after each pixel and in the same column as each pixel. For example, for a pixel matrix with m rows and n columns, a pixel with position (x, y) where x is between [0, ml] and y is between [0, nl] has as vertical neighboring pixels the pixels with positions (x-1, y) and (x+1, y). Thus, the pixels in the first and last rows of the pixel matrix have only one vertical neighboring pixel.

[0036] Each pixel in the pixel matrix has one or two horizontal neighboring pixels positioned in the same row as each pixel and in the preceding and following columns. For example, for a pixel matrix with m rows and n columns, a pixel with position (x, y) where x is between [0, ml] and y is between [0, nl] has horizontal neighboring pixels (x, yl) and (x, y+l). Thus, the pixels in the first and last columns of the pixel matrix have only one horizontal neighboring pixel.

[0037] Each pixel in the pixel matrix has between one and four diagonally adjacent pixels positioned in the rows and columns preceding and following each pixel. For example, for a pixel matrix with m rows and n columns, a pixel with position (x, y) where x is between [0, ml] and y is between [0, nl] has as its diagonally adjacent pixels the pixels with positions (x-1, yl), (x-1, y+l), (x+1, yl), and (x+1, y+l). Thus, the pixels located at the four corners of the pixel matrix have only one diagonally adjacent pixel, and the pixels in the first and last column or the first and last row of the pixel matrix have only two diagonally adjacent pixels.

[0038] In an example, compatible with the preceding examples, the set of confidence pixels can be determined from an arbitrary number of orientations, including for example continuous orientations with sine and cosine weights of the orientation defined by the local gradient of the current image.

[0039] In an example consistent with the preceding examples, the selection of the at least one healthy pixel whose value is used to predict the value of the current pixel is performed taking into account local gradients of the current image. A gradient of an image is a directional change in the intensity (in the case of a monochrome image) or the color (in the case of a color image) of an image. It is

[0040]

[0041] Note that the intensity or color of each pixel in the image is stored in the value of that pixel. The term "local" here means that the gradient is determined locally around the current pixel, i.e., for a sub-region of the image containing the current pixel. The goal is therefore to give each pixel information about the direction of the intensity variation in its neighborhood. The sub-region of the image considered for calculating the local gradients can, for example, be 3x3 pixels in size. The local gradients of the current image are determined by a directed high-frequency filter. For example, it is possible to use known operators such as a compass gradient operator or a Laplacian operator. By taking into account the local gradients of the image, this example of method 100 allows for the reduction of potential blurring effects that may appear in processed images, particularly for images containing high-contrast objects. In an example consistent with the previous examples, the choice of at least one healthy pixel whose value is used to predict the value of the current pixel is: • At least one horizontal neighboring pixel of the current pixel when: \FH(PixelcmJ > \FV(Pixel current^ • With: * \FH(Pixel a standard of the horizontal high-frequency filter, evaluating the amplitude of the horizontal variation of intensities and obtained by using the value of the horizontal neighboring pixel in order to predict the value of the current pixel, * \FV(Pixel J' is a standard of the vertical high-frequency filter, evaluating the amplitude of the vertical variation of intensities and obtained by using the value of the vertical neighboring pixel in order to predict the value of the current pixel, and • s, a threshold of texture homogeneity, • At least one vertical neighboring pixel of the current pixel when: \FV(Pixelcouran^ > \FH[PixelcmtratJ^ + s • At least one horizontal neighboring pixel of the current pixel and one vertical neighboring pixel of the current pixel when: \FH(Pix€lcourimf$ - \FH(PixelaMrant$ < s1 In an example consistent with the previous examples, the prediction value of the current pixel is calculated by non-isotropically averaging the values ​​of at least two pixels from the set of confidence pixels. For example, when the local gradient of the image around the pixel of interest is vertical, the prediction value of the pixel is calculated by averaging the values ​​of the healthy pixels. nearest vertical neighbors. In a second example, when the local gradient of the image around the pixel of interest is horizontal, the pixel's prediction value is calculated by averaging the values ​​of the nearest horizontally adjacent healthy pixels. In a third example, when the local gradient of the image around the pixel of interest is diagonal, the pixel's prediction value is calculated by averaging the values ​​of the nearest diagonally adjacent healthy pixels. When the local gradient of the image does not allow for the determination of an orientation trend, it is possible to use the average of all the values ​​of the adjacent healthy pixels, i.e., the horizontally, vertically, and diagonally adjacent healthy pixels.

[0042] It is worth noting that the set of confidence pixels are confidence pixels that are recursively healthy because these pixels have already been processed using method 100. The recursion of the averaging filter therefore ensures that no defective pixel is used in calculating the prediction value of a current pixel. Thus, within a cluster of defective pixels, the recursive averaging filter allows the propagation of the value of the healthy pixels throughout the image. Furthermore, the selection of at least one pixel from this set of at least one healthy pixel, whose value has previously been replaced by the prediction value, can be carried out taking into account the local gradient of the image so that this propagation respects the local gradient of the image.

[0043] It is also worth noting that the set of confidence pixels includes at least one healthy pixel whose value has previously been replaced by the predicted value. This at least one healthy pixel may belong to the pixel matrix of the current image or to the pixel matrix of a previously processed image. The at least one healthy pixel belonging to the pixel matrix of a previously processed image, for example, a time neighbor of the current pixel, may be used if no spatial neighbor exists for a current pixel or in addition to one or more spatial neighbors. Certain pixels of the image, by virtue of their position in the image, may, for example, be handled in a specific way.

[0044] A first example of specific handling concerns the first pixel processed in an image (corresponding to the top-left pixel of the image when pixels are processed from top to bottom and left to right). For this first processed pixel, it is possible to use: • the average of the pixel values ​​of the first row of the previously processed image, or • the value of at least one pixel of the previously processed image, for example the value of at least one of the pixels of the first N lines of the previously processed image, with N for example equal to 1, 2 or 5.

[0045] A second example of specific handling concerns the pixels in the first row of an image (corresponding to the pixels at the top of the image when pixels are processed from top to bottom and left to right). For each of these pixels, it is possible to use, in addition to or instead of the value of the previously processed pixels in the first row of the image: • the average of the pixel values ​​of the first row of the previously processed image, or • the value of at least one pixel of the previously processed image, for example the value of at least one of the pixels of the first N lines of the previously processed image, with N for example equal to 1, 2 or 5.

[0046] A third example of specific handling concerns the pixels in the first column of an image (corresponding to the pixels on the left side of the image when pixels are processed from top to bottom and left to right). For each of these pixels, it is possible to use: • the average of the values ​​of the first M pixels of the N rows preceding the current pixel, with, for example, M between 1 and 32 and N between 1 and 32, or • the value of the first M pixels of N lines preceding the current pixel, with for example M between 1 and 32 and N between 1 and 32.

[0047] An example of an embodiment, compatible with the preceding examples, of predicting the value of the current pixel is now detailed with reference to [Fig. 4]. In this example, for a 400-pixel matrix of an image to be processed, the current pixel is pixel 2, for "positioned in position 2". Furthermore, in this example, the pixels of the 400-pixel matrix are processed from top to bottom and from left to right. In this example, depending on the local gradients of the image, the predicted value of a pixel can be calculated with a horizontal, vertical, or isotropic prediction.

[0048] In an example of horizontal prediction, i.e. by primarily using at least one horizontal neighbor pixel of the current pixel 2, a left pixel denoted pxA can be defined according to the following rule: • if pixel 3x is healthy and pixel 2 does not belong to the first column of the 400 pixel matrix, • then the predicted value of pxA is equal to the value of pixel 3x, • otherwise the prediction value of pxA is equal to the value of pixel pRec.

[0049] With: • pRec, a healthy pixel whose value has been replaced by the previously determined prediction value and equal to the average between the value of pixel 3xy and the value of pixel 3xly.

[0050] In this example of horizontal prediction, a right pixel denoted pxB can be defined according to the following rule: • if pixel 2 belongs to the last column of the 400 pixel matrix or if the following conditions are met: • Pixel Ix is defective, i.e., pixel Ix is not healthy, and • Pixel 2 belongs to the second-to-last column of the matrix Pixel 400 or pixel Ox is defective. • then the predicted value of pxB is equal to the value of pixel pxA • Otherwise • if pixel Ix is healthy • then the predicted value of pxB is equal to the value of pixel Ix, • otherwise the prediction value of pxB is equal to the value of pixel Ox.

[0051] In this example of horizontal prediction, the prediction value of the current pixel 2 can be equal to the average of the values ​​of pixels pxA and pxB.

[0052] In an example of vertical prediction, i.e. by primarily using at least one vertical neighbor pixel of the current pixel 2, a top pixel denoted pyA can be defined according to the following rule: • if pixel pyA belongs to the first row of the 400 pixel matrix • then the predicted value of pyA is equal to the average of the pixel values ​​in the first row of the pixel matrix of the previous image, • otherwise the prediction value of pyA is equal to the value of pixel 3y, which is healthy since it has been previously processed.

[0053] In this example of vertical prediction, a low pixel denoted pyB can be defined according to the following rule: • if pixel 2 belongs to the last row of the 400 pixel matrix or if the following conditions are met: • the pixel is defective and • Pixel 2 belongs to the second-to-last column of the 400-pixel matrix, or pixel Oy is defective. • then the predicted value of pyB is equal to the value of pixel pyA, • Otherwise • if the pixel ly is healthy • then the prediction value of pyB is equal to the value of pixel ly, • otherwise the prediction value of pyB is equal to the value of pixel 0y-

[0054] In this example of vertical prediction, the prediction value of the current pixel can be equal to the average of the values ​​of pixels pyA and pyB.

[0055] In an example of isotropic prediction, the prediction value of the current pixel may be equal to the average of the results of the horizontal and vertical predictions.

[0056] In an example consistent with the preceding examples, the predicted value of the current pixel calculated in step 120 can be stored in RAM. Thus, this predicted value of the current pixel will remain accessible during the subsequent processing of other pixels in the pixel matrix. However, depending on the size of the RAM, a predetermined maximum number of previously predicted values ​​will be accessible during the execution of steps 120 to 140 for a current pixel.

[0057] A step 130 of the process 100 comprises determining the state of the current pixel. In other words, this step consists of defining the state, whether healthy or defective, of the current pixel, which is in the unprocessed state at the beginning of this step 130. The determination of the current pixel's state is performed using hysteresis thresholding of a confidence counter. Thus, a confidence counter dedicated to a pixel having the same position on the pixel matrix for the different images to be processed is used. In other words, a current pixel and its neighboring pixels in time share the same confidence counter. For example, for a pixel positioned at (x;y), a single confidence counter is used for all the images to be processed by the process 100. Thus, for each processed image, the confidence counter can be incremented and decremented.The confidence counter of a current pixel can be incremented when the difference between the current pixel's value and its predicted value exceeds a predetermined prediction threshold. The confidence counter can be decremented when the difference between the current pixel's value and its predicted value is less than or equal to the predetermined prediction threshold. For example, the predetermined prediction threshold might be between 0 and 1 times the image's dynamic range. Based on the confidence counter's value for the current pixel, its state can then be determined. For instance, when the current counter's value is less than or equal to a predetermined confidence threshold, the pixel can be considered to be in a healthy state.When the current counter value exceeds the predetermined confidence threshold, the pixel can be considered defective. The predetermined confidence threshold is, for example, between 0 and 1 times the image dynamic range. Using this confidence counter makes defective pixel detection robust to scene hotspots. Indeed, a pixel is considered defective when its value remains far from its predicted value for several successive or near-successive images. Furthermore, since the predicted value of the current pixel is calculated from the... value of healthy pixels, preferably neighbors in space or time of the current pixel, this prediction value can be considered as a confidence value.

[0058] In an example consistent with the preceding examples, the increment or decrement of the confidence counter of the current pixel in the current image is performed only when the overall movement of the scene exceeds a third predetermined threshold. This third predetermined threshold is, for example, between 0 and 100 pixels. This threshold can also be converted into an angle by taking into account the size of a single pixel on the measurement surface, commonly called the "ifov" for "instantaneous field of view." Indeed, the images to be processed may be taken when the image sensor is moving, i.e., the motion sensor has changed position between the different image captures. Furthermore, whether the image sensor is moving or not, elements of the real environment being imaged may also be in motion.Thus, in this example, the incrementing or decrementing of the current pixel's confidence counter is contingent upon the detection of a sufficiently large global scene motion. This detection can be performed using a motion estimator based on comparing the spatial and temporal gradients of the images. Furthermore, this motion estimator can utilize image processing, inertial measurements, or line-of-sight servoing to determine the global scene motion. This example of method 100 prevents pixels located on scene hot spots from being flagged as defective. It is important to note that this example of method 100 assumes that the scene hot and cold spots are not located on the same pixels—that is, pixels with the same position in different images—over time.

[0059] In an example, consistent with the preceding examples, the determination of the state 130 of the current pixel can be carried out in a manner similar to that disclosed in patent FR3028376B1.

[0060] In an example, compatible with the previous examples, the confidence counter of the current pixel is stored in DDR SDRAM type memory, for "Double Data Rate Synchronous Dynamic Random Access Memory".

[0061] A step 140 of the process 100 includes replacing the current pixel value with the predicted current pixel value calculated in step 120. Step 140 is performed only when the current pixel has been determined, in step 130, to be defective. Furthermore, this step 140 includes changing the state of the current pixel, which is therefore, at the end of this step, in the healthy state.

[0062] In an example, compatible with the preceding examples, steps 110, 120, 130 and 140 of process 100 are implemented by a programmable logic circuit. In this example, information about a pixel, such as its value and predicted value, can be stored in RAM. Furthermore, since the pixel matrix of each image to be processed is scanned from left to right and top to bottom when process 100 is implemented by a programmable logic circuit, the choice of the neighboring pixel is dictated by this scanning direction. Thus, in this example: • The vertical neighbor pixel of the current pixel is the upper neighbor pixel (i.e., located above, therefore on the previous line) of the current pixel. • The horizontal neighboring pixel of the current pixel is the left neighboring pixel (that is, the one located in the previous column) of the current pixel, and • The pixel diagonally adjacent to the current pixel is the upper left or upper right neighboring pixel of the current pixel.

[0063] Indeed, in this example, the pixels located below the current pixel or to the right on the same line as the current pixel are not yet processed and stored in memory.

Claims

1. Demands Image processing method (100) applied to images taken successively by an image sensor (211), each image among the successively taken images comprising a pixel matrix, each pixel of the pixel matrix corresponding to an active element of the image sensor capable of converting an incident beam of light into an electrical signal, each pixel being associated with a value corresponding to a measure of the light intensity received by that pixel, and each pixel being in one of a state among an unprocessed state, a healthy state and a defective state, the method iteratively processing the successively taken images and comprising, for each current image, an initial initialization step (110) of the state of each pixel of the pixel matrix of the current image to the unprocessed state and then subsequent steps performed recursively for each current pixel of the current image, the subsequent steps being: - Calculation (120) of a prediction value for the current pixel, the calculation of the prediction value being performed using a recursive averaging filter, the recursive averaging filter using the value of at least one healthy pixel in order to calculate the prediction value of the current pixel, the at least one healthy pixel used being at least one pixel chosen from: • At least one vertical neighbor pixel of the current pixel, • At least one pixel horizontally adjacent to the current pixel, • At least one pixel diagonally adjacent to the current pixel, • At least one healthy pixel whose value has been previously replaced by the predicted value, - Determination of the state (130) of the current pixel using hysteresis thresholding of a confidence counter, a value of the confidence counter being determined by comparing the value of the current pixel and the predicted value of the current pixel, and - when the current pixel is defective, replace (140) the value of the current pixel with the predicted value of the current pixel and change the state of the current pixel to the healthy state.

2. A method (100) of image processing according to claim 1 wherein the determination (130) of the value of the confidence counter comprises: - an increment of the confidence counter when a difference between the value of the current pixel and the prediction value of the current pixel is greater than a first predetermined threshold, and - a decrement of the confidence counter when a difference between the value of the current pixel and the prediction value of the current pixel is less than or equal to the first predetermined threshold, and wherein the state of the current pixel is determined as: - healthy when the confidence counter is less than or equal to a second predetermined threshold, or - defective when the confidence counter is greater than a second predetermined threshold.

3. A method (100) of image processing according to claim 2 wherein the incrementing or decrementing of the confidence counter of the current pixel of the current image is performed only when, for the current image, an overall movement of a scene on the current image is greater than a third predetermined threshold.

4. A method (100) of image processing according to any one of the preceding claims wherein the selection of said at least one healthy pixel whose value is used to calculate the predicted value of the current pixel is carried out taking into account local gradients of the current image, the local gradients of the current image being determined by an oriented high-frequency filter.

5. A method (100) of image processing according to any one of the preceding claims, wherein said at least one selected healthy pixel whose value is used to predict the value of the current pixel is: - At least one horizontally adjacent pixel to the current pixel when: \FH(Pix e l„„„J > iFVÏCurrent Pixel) $ With: \FH(Pixel Y) is a standard for the horizontal high-frequency filter, evaluating the amplitude of the horizontal variation in intensities and obtained by using the value of the horizontal neighboring pixel to predict the value of the current pixel; \FV(Pixel) is a standard for the vertical high-frequency filter, evaluating the amplitude of the vertical variation in intensities and obtained by using the value of the vertical neighboring pixel to predict the value of the current pixel; and \ is a texture homogeneity threshold, where at least one vertical neighboring pixel of the current pixel is present when: \FV(Pixel cma™^ > \FH(Pixel couranî ) + s The at least one horizontal neighboring pixel of the current pixel and the vertical neighboring pixel of the current pixel when:

6. \FH(Pixel courant ) \FH(Pixel courant ) < s A method (100) for image processing according to any one of the preceding claims, wherein the initialization steps (110), calculation (120) of a prediction value for the current pixel, determination (130) of the state of the current pixel, and replacement (140) of the current pixel value with the prediction value of the current pixel and modification of the current pixel state to the healthy state are implemented by a programmable logic circuit and wherein: The vertical neighbor pixel of the current pixel is the upper neighbor pixel of the current pixel. The horizontal neighbor pixel of the current pixel is the left neighbor pixel of the current pixel. A pixel diagonally adjacent to the current pixel is the upper left or upper right neighboring pixel of the current pixel.

7. Image processing device (220) comprising a pixel processing module (221) configured to process pixels from at least one image from an image sensor (211), by implementing the image processing method (100) according to any one of the preceding claims.

8. Optronic system (200) comprising an image acquisition device (210) equipped with an image sensor (211), an image processing device (220) according to the preceding claim and an image display device (230) and / or image storage device.

9. Aircraft comprising an optronic system (200) according to the preceding claim.

10. Product computer program, characterized in that it includes instructions to implement, by a device, the method according to any one of claims 1 to 6, when said program is executed by a processor of said device.