An infrared image correction method

Abstract

The present application belongs to the field of image processing, and particularly relates to an infrared image correction method. The correction method establishes a blind pixel detection model according to characteristic hyperplane division of a nonlinear equation, and obtains a blind pixel detection method capable of accurately dividing blind pixel and normal pixel through training of the model, thereby avoiding false detection and missed detection of blind pixels and improving the reliability of image correction. Meanwhile, the blind pixel compensation process is optimized to avoid the influence of other blind pixels existing in the neighborhood of the blind pixel on blind pixel compensation, and the accuracy of blind pixel compensation is enhanced. On the basis of the above, through adaptive non-uniform correction, a shear wave change improved threshold method and a non-sharpening mask method are combined to respectively perform noise suppression and image enhancement on the infrared image, and a weak small target is detected to verify the correction effect of the infrared image. The present application optimizes the infrared image correction process aiming at each step in the correction, especially the blind pixel detection and compensation steps, so that more accurate correction results are obtained.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of image processing, and particularly relates to an infrared image correction method. BACKGROUND

[0002] The infrared focal plane array is affected by non-uniformity, resulting in the generation of blind pixel of the infrared image. The number of blind pixels in the infrared focal plane array gradually increases over time, reducing the image quality. The blind pixel is divided into dead pixel and hot pixel, that is, a dark spot or a bright spot is displayed on the image.

[0003] In the prior art, the blind pixel is detected by comparing the pixel with the mean value of the nearby background, and then the detected blind pixel is compensated to realize image correction. For example, the Chinese patent application publication No. CN104330164A discloses a blind pixel detection method, which sequentially performs windowing on each pixel as the center. If the difference between the pixel and the mean value of the gray scale in the window is too large, the pixel is determined as a blind pixel.

[0004] Although the above blind pixel detection method can realize blind pixel detection, the detection result is greatly affected by the quality of the infrared image. If there are many blind pixels in the infrared image, the mean value of the gray scale in the window will be affected, so that the identification of the blind pixel is prone to false detection, resulting in reduced reliability of the final correction result. SUMMARY

[0005] The present application aims to provide an infrared image correction method to solve the problem of low reliability of the correction result of the infrared image containing blind pixels in the prior art.

[0006] In order to achieve the above-mentioned purpose, the present application provides a technical scheme of an infrared image correction method, which comprises the following steps:

[0007] 1) obtaining an infrared image, and detecting the obtained infrared image for blind pixels, and the detection process is as follows:

[0008] According to the characteristic hyperplane, a non-linear equation is divided to establish a blind pixel detection model:

[0009] h(x)=θ T F≥0

[0010] Wherein F is the blind pixel feature of the infrared image, and Wherein X represents the gray value of the pixel, L represents the blind pixel marker point, and l 11 =x 11 ,...,l mn =x mn ; for any pixel x ij , the blind pixel feature Wherein, The infrared image is represented by a matrix as wherein i, j represent pixel x ij The coordinate position on the infrared image, m*n represents the size of the infrared image; let Y be a blind pixel identifier, each pixel of the infrared image corresponds to a unique identifier 0 or 1, represented as

[0011] The sequence infrared image data D = {(x ij ,y ij )} with known blind pixel position, x ij ∈[0, 255], i ∈[1, m], j ∈[1, n], y ij ∈{0, 1} is input into the blind pixel detection model for training;

[0012] The loss function during training is The objective function is wherein C is a penalty parameter, C > 0;

[0013] θ is obtained by minimizing the objective function, and the training of the blind pixel detection model is realized;

[0014] The obtained infrared image is input into the trained blind pixel detection model, and a blind pixel detection image is output, so as to realize blind pixel detection of the infrared image;

[0015] 2) Blind pixel compensation is performed on the blind pixel detection image of step 1), and a blind pixel compensation image is obtained;

[0016] 3) The blind pixel compensation image of step 2) is subjected to two-point correction, and a two-point correction image is obtained;

[0017] 4) The two-point correction image of step 3) is input into an adaptive correction model, and a non-uniformity adaptive correction image is obtained;

[0018] 5) The non-uniformity adaptive correction image of step 4) is subjected to noise suppression and then image enhancement, and a corrected infrared image is obtained;

[0019] 6) The corrected infrared image of step 5) is subjected to target detection to verify the correction effect.

[0020] The correction method divides the blind pixels and normal pixels by constructing a characteristic hyperplane to divide a non-linear equation, trains a blind pixel detection model accordingly, detects the blind pixels of the infrared image, can therefore fully extract the blind pixel features, reduce misjudgment and omission, and obtain more accurate blind pixel detection results, thereby indirectly increasing the reliability of image correction when subsequently performing targeted supplementary correction on the blind pixels, and finally performing target detection on the corrected infrared image to verify the correction effect.

[0021] Further, to avoid the interference of other blind pixels existing near the blind pixel on the blind pixel compensation, the blind pixel detection image is compensated for blind pixels, and the calculation formula for the blind pixel compensation is:

[0022] P(i, j) = k1 x P(i, j-a) + k2 x P(i, j+b) + m1 x P(i-c, j) + m2 x P(i-c, j)

[0023] where k1+k2+m1+m2 = 1,

[0024] The specific steps are as follows:

[0025] According to the blind pixel detection result, the position coordinates of the blind pixel point in the blind pixel detection image are determined, denoted as P(i, j);

[0026] For the blind pixel point P(i, j), the first non-blind pixel in the four directions above and below and left and right of the point is selected, and the sizes of a, b, c, and d are determined;

[0027] If a-b > 5 or c-d > 5, the first non-blind pixel in the four directions of the diagonal line of the blind pixel point is reselected, and the sizes of a, b, c, and d are determined again;

[0028] If the values of a, b, c, and d obtained on the diagonal line satisfy a-b≤5 and c-d≤5, the values of a, b, c, and d are used to compensate for the blind pixel of P(i, j); if the values of a, b, c, and d on the diagonal line still do not satisfy the above relationship, the difference between a-b and c-d in the two groups of a, b, c, and d is compared, and the group of a, b, c, and d with smaller difference is used to compensate for the blind pixel of P(i, j).

[0029] Further, the infrared image correction method according to claim 1, characterized in that the two-point correction adopts a two-point correction non-uniformity real-time correction algorithm.

[0030] The algorithm first obtains gain and offset coefficients through two-point correction, then filters the two correction coefficients to remove residual noise, to achieve the function of real-time correction, and at the same time can improve the signal-to-noise ratio.

[0031] Further, the adaptive correction model adopts an adaptive correction model based on a UNet structure residual encoding-decoding network.

[0032] The UNet structure-based residual encoding-decoding network can improve the feature extraction capability of the network through multi-scale sampling, add residual mapping to generate a residual image to reduce the non-linear mapping range. The model can effectively suppress noise while removing "ghosting".

[0033] Further, the adaptive correction model uses mean square error as loss function, and the mean square error is as follows:

[0034]

[0035] wherein, W and H represent the width and height of the infrared image respectively; y represents the image to be corrected; f represents the correction process of the adaptive correction model; and x represents the output of the real infrared image without non-uniformity noise.

[0036] Further, the real infrared image without non-uniformity noise is subjected to noise suppression by using a shear wave change improved threshold method to obtain the structural feature of the real infrared image, and the method is as follows:

[0037] The real infrared image is decomposed into at least one low-frequency sub-band image and a plurality of high-frequency sub-band images of different scales and directions; the structural feature of the high-frequency sub-band image is to obtain the maximum value of the high-frequency sub-band coefficient of the high-frequency sub-band image in the same scale space and different directions; and the mathematical expression of the structural feature is as follows:

[0038]

[0039] wherein, (i,j) is the pixel coordinate on the input real infrared image, represents the improved shear coefficient of the pixel (i,j) in the lth scale and the hth direction, feat l (i,j) represents the structural feature of the input real infrared image in the lth scale;

[0040] wherein,

[0041]

[0042] σ1 and σ2 are the standard deviations of the high-frequency sub-band coefficients in each scale of the input real infrared image; and k is a coefficient without actual meaning.

[0043] Further, in order to prevent over-enhancement of edges and amplification of noise, a non-sharpening mask method improved by a feature attribute map is used to perform image enhancement on the structural feature of the real infrared image to obtain a corrected infrared image.

[0044] Further, in order to improve the accuracy of segmentation and detection of weak and small targets in the infrared image, the target detection uses a neural network model method of partial sum of nuclear norm of infrared slice tensor to detect the weak and small targets in the infrared image, and the method is as follows:

[0045] a) Local priority feature extraction; the input corrected infrared image is used to obtain a prior weight map related to target and background information by calculation;

[0046] b) Slice tensor construction: The original infrared noise-suppressed image and the prior weight map are transformed into the original slice tensor and the prior weight slice tensor respectively by sliding a window from the upper left to the lower right.

[0047] c) Target-background separation: The input slice tensor is decomposed into low-rank slice tensor and sparse slice tensor by using the alternating direction method of multipliers;

[0048] d) Image reconstruction and target detection: The target image and background image are reconstructed from the low-rank slice tensor and the sparse slice tensor. The reconstruction process is the reverse of the construction process. At the same time, one-dimensional median filtering is used to determine the values ​​of the overlapping positions of multiple slices. After reconstruction, adaptive thresholding is used to detect weak targets.

[0049] Infrared weak target detection is performed on the corrected infrared image to verify the effect of infrared image correction. The higher the detection rate of weak targets, the better the correction effect of the infrared image. Attached Figure Description

[0050] Figure 1 This is a structural block diagram of the infrared image correction method in an embodiment of the present invention;

[0051] Figure 2 This is a flowchart illustrating the process of detecting weak targets in an embodiment of the method of the present invention. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0053] Infrared image correction method embodiment

[0054] This embodiment provides an infrared image correction method, referring to... Figure 1 Specifically, it includes the following steps:

[0055] 1) Acquire infrared image sequences from an infrared camera and perform blind pixel processing; refer to Figure 2 Blind pixel processing consists of two parts: blind pixel detection and blind pixel compensation. Blind pixel detection is performed on a single frame of infrared image using a blind pixel detection model to identify blind pixel pixels and obtain a blind pixel detection image.

[0056] The specific process of blind pixel detection using the blind pixel detection model is as follows:

[0057] ① Establish a blind pixel detection model, representing a single-frame infrared image using a matrix. Where i and j represent pixels x ijIn the infrared image, the coordinates are m*n, representing the size of the infrared image; let Y be the blind pixel identifier, where each pixel in the infrared image corresponds to a unique identifier of 0 or 1, represented as...

[0058] ② When Y = 1, the blind pixel detection model equation is defined as h(x) = θ T F≥0, where θ∈R m ×n, if a pixel satisfies this equation, then the pixel is a blind pixel;

[0059] F is the blind pixel feature of the infrared image, and Where X represents the grayscale value of a pixel, L represents a blind pixel marker, and l 11 =x 11 ,...,l mn =x mn ;

[0060] For any pixel x ij Blind element features in,

[0061] ③ The sequence of infrared image data D={(x ij ,y ij )},x ij ∈[0,255],i∈[1,m],j∈[1,n],y ij Input ∈{0,1} into the blind pixel detection model for training, and the loss function during training is: The objective function is Where C is the penalty parameter, and C>0;

[0062] The blind pixel detection model is obtained by minimizing the objective function to obtain θ.

[0063] 2) Perform blind pixel compensation on the blind pixel detection image in step 1) to obtain the blind pixel compensated image;

[0064] The formula for calculating blind cell compensation is as follows:

[0065] P(i,j)=k1×P(i,ja)+k2×P(i,j+b)+m1×P(ic,j)+m2×P(ic,j)

[0066] Where k1+k2+m1+m2=1,

[0067] The specific steps for blind cell compensation are as follows:

[0068] Based on the blind cell detection results in step 1), determine the position coordinates of the blind cell points, denoted as P(i, j);

[0069] Based on the results of blind pixel detection in step 1), determine the position coordinates of the blind pixel points in the blind pixel detection image, denoted as P(i, j);

[0070] For a blind element point P(i,j), select the first non-blind element point in the four directions of up, down, left, and right, and determine the magnitudes of a, b, c, and d respectively;

[0071] If ab > 5 or cd > 5, then reselect the first non-blind element point in the four directions diagonally opposite the blind element point, and re-determine the values ​​of a, b, c, and d.

[0072] If the values ​​of a, b, c, d obtained on the diagonal satisfy ab≤5 and cd≤5, then the values ​​of a, b, c, d in this set are used to perform blind element compensation on P(i,j); if the values ​​of a, b, c, d on the diagonal still do not satisfy the above relationship, then the difference between ab and cd in the two sets of a, b, c, d is compared, and the set of a, b, c, d with the smaller difference is used to perform blind element compensation on P(i,j).

[0073] 3) Perform two-point correction on the blind pixel compensation image of the sequence frame in step 2) to obtain a two-point corrected image;

[0074] in,

[0075] Two-point correction employs a non-uniform real-time correction algorithm based on two-point correction. The response value after correction by this algorithm can be expressed by the following formula:

[0076] Y i，j =K i，j X i，j (Φ)+B i.j

[0077] Where Φ is the infrared radial flux, X i，j (Φ) is the response value of pixel (i, j); K i，j To correct the gain, B i，j Here is the offset coefficient; the formulas for calculating the gain coefficient and offset coefficient are as follows:

[0078]

[0079]

[0080] Where X i，j (Φ L (X) is a raw infrared image of a standard blackbody at low temperature acquired using an infrared focal plane detector. i，j (Φ HThis refers to the use of an infrared focal plane detector to acquire the raw infrared image of a standard blackbody at high temperatures. The high and low temperatures here are related to the actual ambient temperature at which the detector acquires the image; typically, the low temperature range is -80 to 230℃, and the high temperature range is 230 to 1600℃. Y(Φ) L ) and Y(Φ H ) are the average response values ​​of a standard blackbody at high and low temperatures, respectively.

[0081] 4) Input the two-point correction images from step 3) into the adaptive correction model to obtain the non-uniform adaptive correction image;

[0082] The adaptive correction model adopts an adaptive correction model based on a residual encoder-decoder network with a UNet structure. This model has the following advantages:

[0083] Using a left-right symmetrical encoder-decoder network, the non-uniform residual image at multiple scales is obtained by learning the residual mapping. The non-uniform residual image is then subtracted from the non-uniform infrared image using global skip connections to obtain the final correction result. This model can effectively suppress noise while removing "ghosting".

[0084] To achieve better correction results, batch normalization is added after the convolutional layer;

[0085] The PReLU activation function is used to address the problem of partial neuron necrosis.

[0086] The UNet structure can effectively combine shallow and deep image features to obtain comprehensive image features and increase the accuracy of image correction.

[0087] This adaptive correction model uses mean squared error as the loss function, and the mean squared error is as follows:

[0088]

[0089] In the formula, W and H represent the width and height of the infrared image, respectively; y represents the image to be corrected; f represents the correction process of the adaptive correction model; and x represents the output true infrared image after correction, free from non-uniform noise.

[0090] 5) First, perform noise suppression on the non-uniformity adaptive correction image from step 4), and then perform image enhancement to obtain an infrared noise-suppressed image;

[0091] First, a shear wave transformation-based thresholding method is used to suppress noise in real infrared images free of non-uniform noise, thereby obtaining the structural features of the real infrared image, as detailed below:

[0092] A real infrared image is decomposed into at least one low-frequency subband image and multiple high-frequency subband images of different scales and orientations. The structural feature of the high-frequency subband image is to obtain the maximum value of the high-frequency subband coefficients in different orientations within the same scale space. The mathematical expression of the structural feature is:

[0093]

[0094] Where (i,j) are the pixel coordinates on the input real infrared image. The improved shearing coefficients for pixel (i,j) at the l-th scale and h-th direction are represented by feat. l (i,j) represents the structural features of the input real infrared image at the l-th scale at coordinate (i,j);

[0095] in,

[0096]

[0097] σ1 and σ2 are the standard deviations of the high-frequency subband coefficients at each scale of the input real infrared image.

[0098] After obtaining the high-frequency subband image structural features of the real infrared image, an improved non-sharpening mask method based on feature attribute maps is used to enhance the structural features, resulting in a corrected infrared image.

[0099] The correlation between shear wave coefficients at two adjacent scales is used to measure the properties of image structure. The specific calculation is described mathematically as follows:

[0100]

[0101] Where scale_num is the number of shear wave decomposition layers. This is the inner product operator. This operation is performed in two steps: First, select two image patches centered at coordinates (i,j) from the two-scale feature maps, feature... l and featured l+1 Then Attri_feat l (i,j) is calculated as the inner product between the two vectors stretched from these two image patches.

[0102] 6) Perform weak target detection on the corrected infrared image from step 5) to verify the effect of the correction.

[0103] Reference Figure 2 The target detection employs a neural network model method based on the partial sum of the nuclear norm of infrared slice tensors, as detailed below:

[0104] a) Local priority feature extraction: Input an infrared noise-suppressed image and calculate a prior weight map related to target and background information;

[0105] b) Slice tensor construction: The original infrared noise-suppressed image and the prior weight map are transformed into the original slice tensor and the prior weight slice tensor respectively by sliding a window from the upper left to the lower right.

[0106] c) Target-background separation: The input slice tensor is decomposed into low-rank slice tensor and sparse slice tensor by using the alternating direction method of multipliers;

[0107] d) Image reconstruction and target detection: The target image and background image are reconstructed from the low-rank slice tensor and the sparse slice tensor; at the same time, one-dimensional median filtering is used to determine the values ​​of the overlapping positions of multiple slices; after reconstruction, adaptive thresholding is used to detect the target.

[0108] This invention establishes a blind pixel detection model for infrared images based on nonlinear equations derived from feature hyperplanes. Training the model yields a blind pixel detection method capable of accurately distinguishing between blind pixel and normal pixels, thus avoiding false detections and missed detections of blind pixels and improving the reliability of infrared image correction. Simultaneously, the blind pixel compensation process is optimized to prevent other blind pixels in the neighborhood from affecting compensation, enhancing its accuracy. Building upon this, adaptive non-uniform correction, combined with a threshold method improved by shear wave transformation and a non-sharpening mask method improved by feature attribute maps, are used to suppress noise and enhance the infrared image. Furthermore, weak infrared targets are detected to verify the correction effect, ultimately outputting a more accurate infrared image correction result.

[0109] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. An infrared image correction method, characterized in that, Includes the following steps: 1) Acquire infrared images and perform blind pixel detection on the acquired infrared images. The detection process is as follows: Establish a blind pixel detection model: h(x)=θ T F≥0 Where F is the blind pixel feature of the infrared image, and Where X represents the grayscale value of a pixel, L represents a blind pixel marker, and l 11 =x 11 ,...,l mn =x mn For any pixel x ij Blind element features in, Infrared images are represented by matrices. Where i and j represent pixels x ij In the infrared image, the coordinates are m*n, representing the size of the infrared image; let Y be the blind pixel identifier, where each pixel in the infrared image corresponds to a unique identifier of 0 or 1, represented as... The sequence of infrared image data D = {(x ij ,y ij )},x ij ∈[0,255],i∈[1,m],j∈[1,n],y ij Input ∈{0,1} into the blind pixel detection model for training; The loss function during training is The objective function is Where C is the penalty parameter, C > 0; The blind pixel detection model is trained by minimizing the objective function to obtain θ. The acquired infrared image is input into the trained blind pixel detection model, and the blind pixel detection image is output to realize blind pixel detection of infrared images. 2) Perform blind pixel compensation on the blind pixel detection image described in step 1) to obtain a blind pixel compensated image; 3) Perform two-point correction on the blind pixel compensation image obtained in step 2) to obtain a two-point corrected image; 4) Input the two-point correction image obtained in step 3) into the adaptive correction model to obtain the non-uniform adaptive correction image; 5) The non-uniform adaptive correction image described in step 4) is first subjected to noise suppression, and then image enhancement is performed to obtain the corrected infrared image; 6) Perform target detection on the corrected infrared image described in step 5) to verify the correction effect.

2. The infrared image correction method according to claim 1, characterized in that, Blind pixel compensation is performed on the blind pixel detection image. The formula for calculating blind pixel compensation is as follows: P(i,j)=k1×P(i,ja)+k2×P(i,j+b)+m1×P(ic,j)+m2×P(i+d,j) Where k1+k2+m1+m2=1, The specific steps are as follows: Based on the results of blind pixel detection, the position coordinates of the blind pixel points in the blind pixel detection image are determined and denoted as P(i,j); For a blind element point P(i,j), select the first non-blind element point in the four directions of up, down, left, and right, and determine the magnitudes of a, b, c, and d respectively; If ab > 5 or cd > 5, then reselect the first non-blind element point in the four directions diagonally opposite the blind element point, and re-determine the values ​​of a, b, c, and d. If the values ​​of a, b, c, d obtained on the diagonal satisfy ab≤5 and cd≤5, then the values ​​of a, b, c, d in this set are used to perform blind element compensation on P(i,j); if the values ​​of a, b, c, d on the diagonal still do not satisfy the above relationship, then the difference between ab and cd in the two sets of a, b, c, d is compared, and the set of a, b, c, d with the smaller difference is used to perform blind element compensation on P(i,j).

3. The infrared image correction method according to claim 1, characterized in that, The two-point correction employs a non-uniform real-time correction algorithm.

4. The infrared image correction method according to claim 1, characterized in that, The adaptive correction model adopts an adaptive correction model based on a residual codec network with a UNet structure.

5. The infrared image correction method according to claim 4, characterized in that, The adaptive correction model uses mean squared error as the loss function, and the mean squared error is as follows: In the formula, W and H represent the width and height of the infrared image, respectively; y represents the image to be corrected; f represents the correction process of the adaptive correction model; and x represents the output true infrared image after correction, free from non-uniform noise.

6. The infrared image correction method according to claim 1, characterized in that, A shear wave transformation-based thresholding method is used to suppress noise in real infrared images free of non-uniform noise, yielding the structural features of the real infrared images, as detailed below: The real infrared image is decomposed into at least one low-frequency subband image and multiple high-frequency subband images of different scales and orientations; the structural feature of the high-frequency subband image is to obtain the maximum value of the high-frequency subband coefficients in different orientations within the same scale space; the mathematical expression of the structural feature is: Where (i,j) are the pixel coordinates on the input real infrared image. The improved shearing coefficients for pixel (i,j) at the l-th scale and h-th direction are represented by feat. l (i,j) represents the structural features of the input real infrared image at the l-th scale; in, σ1 and σ2 are the standard deviations of the high-frequency subband coefficients at each scale of the input real infrared image; k is a coefficient and has no practical significance.

7. The infrared image correction method according to claim 6, characterized in that, An improved non-sharpening masking method based on feature attribute maps is used to enhance the structural features of real infrared images, resulting in corrected infrared images.

8. The infrared image correction method according to claim 1, characterized in that, The target detection employs a neural network model method based on the partial sum of the nuclear norm of infrared slice tensors to detect weak infrared targets, as detailed below: a) Local priority feature extraction: Input the corrected infrared image and calculate the prior weight map related to target and background information; b) Slice tensor construction: The original infrared noise-suppressed image and the prior weight map are transformed into the original slice tensor and the prior weight slice tensor respectively by sliding a window from the upper left to the lower right. c) Target-background separation: The input slice tensor is decomposed into low-rank slice tensor and sparse slice tensor by using the alternating direction method of multipliers; d) Image reconstruction and target detection: The target image and background image are reconstructed from low-rank slice tensors and sparse slice tensors; at the same time, one-dimensional median filtering is used to determine the values ​​of multiple overlapping slice positions; after reconstruction, adaptive thresholding is used to detect the target.

Images