A method for detecting and correcting abnormal pixels in images of a scanning infrared camera in orbit

Abstract

The application discloses a method for detecting and correcting abnormal pixels of an image of a wobble scan type infrared camera in orbit, comprising the following steps: dividing abnormal pixels into flash pixels and over-weak or over-strong pixels, performing gradient analysis on pixel response values in a wobble scan direction of the infrared image, calculating the variance of the pixel response value gradient in the wobble scan direction, detecting abnormal values according to the Euclidean distance of adjacent gradient variances, and obtaining the position of the flash pixels in the infrared image; performing over-weak or over-strong pixel detection on the infrared image, taking a total of five pixels of a target pixel and its upper and lower adjacent pixels as a group, detecting abnormal values according to the 3σ criterion, and obtaining the position of the over-weak or over-strong pixels; and correcting the response values of the abnormal pixels, and obtaining an abnormal pixel table. The method is suitable for wobble scan imaging of a linear array detector, the algorithm design is simple, abnormal pixels in the infrared image can be effectively detected, and the image quality is enhanced.

Description

Technical Field

[0001] This invention relates to the field of abnormal pixel detection technology, and in particular to a method for detecting and correcting abnormal pixels in on-orbit images of a oscillating infrared camera. Background Technology

[0002] Infrared thermal imagers offer advantages such as all-weather observation, wide swath, and high resolution, and infrared remote sensing images are now being used in an increasing number of fields. However, due to uneven material distribution and manufacturing processes in infrared detectors, anomalous pixels are unavoidable. The presence of anomalous pixels in the detector can blur the edges and details of the imaged object, or even obscure the object, affecting the image quality and quantitative applications. Therefore, effective anomalous pixel detection is of great significance.

[0003] Existing techniques for blind pixel processing include: Bart Dierickx et al., who, based on the different properties of individually existing blind pixels and the one-dimensional spatial distribution of point targets, used median filtering based on linear extrapolation to directly compensate for blind pixels in the image without blind pixel detection. However, this resulted in blurred image edges. Goma et al. improved upon Bart D. et al.'s algorithm by first detecting blind pixels and then compensating, but they still couldn't overcome the limitation of not being able to detect consecutive blind pixels. Dudas, Ghosh et al. used a sequence of images from changing scenes to statistically distinguish the characteristics of each pixel, and then used a Bayesian algorithm to distinguish between blind pixels and normal pixels. While they achieved good results, their methods were complex and computationally expensive.

[0004] Existing solutions for flash element processing: The literature "Liu Gaorui, Sun Shengli, Lin Changqing, et al. Analysis and suppression method of flash element noise in infrared linear detectors [J]. Journal of Infrared and Millimeter Waves, 2018, 37(4):421-426+432." analyzes the characteristics of flash element phenomenon in infrared scanned images and designs a flash element detection and compensation method based on the maximum value projection of multiple time-series frames. Considering that flash elements still retain a certain information acquisition capability, a one-dimensional median filtering method is used to suppress the background at the flash element position separately. Conventional target detection is then performed on the image after flash element compensation. This method uses 150 frames of images for detection, which requires a large amount of image data for flash element detection, which is not conducive to the need for rapid detection.

[0005] The literature “Wang Xiaolong, Li Dongbing, Zhang Xingsheng, et al. Research on blind pixel detection technology of linear infrared detector [J]. Infrared, 2020(21-28+35).” refers to the flare pixel as a “temporal blind pixel”. By reading multiple frames of data from the linear infrared scan image, the noise threshold is calculated using the long-term flare discrimination method to determine whether the pixel is a flare pixel. This method is based on blackbody image testing and does not perform real-time infrared image flare pixel detection.

[0006] The paper “Yang Dezhen, Yu Songlin, Feng Jinjun. Dynamic real-time repair algorithm for defect pixels based on spatiotemporal statistical features [J]. Infrared and Laser Engineering, 2022(463-474)” proposes a dynamic real-time repair algorithm for defect pixels based on spatiotemporal statistical features for infrared point target detection, in order to detect defect pixels in the image. However, this method can only detect single defect pixels, and local distortion will occur when repairing large-area images. Summary of the Invention

[0007] This invention provides a method for detecting and correcting abnormal pixels in on-orbit images of a oscillating infrared camera, which can achieve accurate and rapid detection and correction of abnormal pixels.

[0008] A method for detecting and correcting abnormal pixels in on-orbit images from a oscillating infrared camera includes the following steps:

[0009] (1) Abnormal pixels in the on-orbit image of the oscillating infrared camera are divided into flash pixels and pixels that are too weak or too strong.

[0010] (2) Calculate the gradient of the pixel response value f of the on-orbit image and store it in the gradient array GARD[M×N]; where M is the number of rows of the image and N is the number of columns of the image;

[0011] (3) Calculate the variance of the gradient values ​​in the gradient array and store it in the variance array VAR[M×1];

[0012] (4) Calculate the distance between adjacent values ​​for each variance in the variance array, take the smaller value, sort them in descending order, and store them in the array DVA[M×1].

[0013] (5) Take the first t median values ​​in the array DVA[M×1] as the flash element detection threshold THF;

[0014] (6) Select the values ​​greater than THF in the array DVA[M×1], find the row of the corresponding image, denote the row as the flash element, and store it in the array #I[a×1];

[0015] (7) Perform weak or strong pixel detection on the infrared image, select a window in the image, and store it in array f. win [5×1] in;

[0016] (8) Calculate the median and standard deviation of the window, and further calculate the range of the pixel response value error 3σ;

[0017] (9) Use the range of 3σ to determine whether the response value of the central pixel is abnormal. If it is abnormal, record it as 1, otherwise record it as 0.

[0018] (10) Perform anomaly detection on N columns of pixels in each row of the image, and accumulate the anomaly count and store it in array CONT[M×1];

[0019] (11) Find the maximum value in the array CONT[M×1] and use the T% of the maximum value as the detection threshold THB for weak or strong pixels;

[0020] (12) Compare the abnormal cell array CONT[M×1] with the threshold THB. If CONT>THB, mark it as a weak or strong cell and store the cell position in array #II[b×1]; where b is the number of weak or strong cells.

[0021] (13) Merge arrays #I[a×1] and #II[b×1] to obtain the abnormal pixel table and complete the detection of abnormal pixels;

[0022] (14) Correct the detected abnormal pixels.

[0023] Furthermore, in step (2), the gradient calculation formula for the pixel response value f is:

[0024]

[0025] Where i is the row number and j is the column number.

[0026] In step (3), the variance of the gradient values ​​in the gradient array is calculated using the following formula:

[0027]

[0028]

[0029] In the formula, N represents the number of image columns.

[0030] The specific process of step (4) is as follows:

[0031] Take the i-th variance VAR[i] in the variance array, and subtract it from the (i-1)-th variance VAR[i-1] and the (i+1)-th variance VAR[i+1] to get two difference values. Take the smaller difference value as the result, denoted as DVA[i], and store it in the variance array VAR[M×1].

[0032] In step (7), the process of selecting a window for the image is as follows:

[0033] The detected pixel is called the central pixel, denoted as f(i,j). The upper 2-neighborhood pixels f(i - 2,j) and f(i - 1,j) and the lower 2-neighborhood pixels f(i + 1,j) and f(i + 2,j) of the central pixel are selected to jointly form the window f win (i,j)[5×1], where i is the row number and j is the column number.

[0034] In step (8), the range of the pixel response value error 3σ is calculated. The process is as follows:

[0035] For the window f win [5×1], the median and standard deviation are calculated, denoted as DN_med and DN_σ respectively;

[0036] Let RNAGE[1] = DN_med - 3×DN_σ and RNAGE[2] = DN_med + 3×DN_σ. Then the range of 3σ is: RNAGE[1] ≤ 3σ ≤ RNAGE[2].

[0037] In step (9), the range of 3σ is used to determine whether the central pixel response value is abnormal. Specifically:

[0038] If the central pixel f(i,j) is not within the range of 3σ, specifically f(i,j) < RANGE[1] or f(i,j) > RANGE[2], then the response value of this central pixel is abnormal, and this pixel is marked as 1; otherwise, it is marked as 0.

[0039] In step (14), the detected pixel response values that are too weak or too strong are corrected. The specific process is as follows:

[0040] Calculate the weighted average of the response values of the upper and lower 4-neighboring pixels of this pixel as the correction value of this pixel;

[0041] The calculation formula for the pixel correction value is

[0042] where u(i,j) is the pixel correction value, and λ op [5×1] is the correlation operator, and λ op [5×1] = [1 / 6, 1 / 3, 0, 1 / 3, 1 / 6], and f win [5×1] is the window.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] This invention addresses the accurate and rapid detection and correction of abnormal pixels in infrared images generated by a sweeping infrared camera. The average overlap rate for detecting excessively weak or strong pixels is 97.9%, and the average overlap rate for detecting scintillation pixels is 83.1%. This method improves the flexibility of abnormal pixel detection. Attached Figure Description

[0045] Figure 1 This is a flowchart of a method for detecting and correcting abnormal pixels in on-orbit images of a oscillating infrared camera, as proposed in an embodiment of this application.

[0046] Figure 2 This is a partial illustration of remote sensing image data;

[0047] Figure 3 This is a schematic diagram of the infrared focal plane of the thermal infrared imager of the SDGSAT-1 satellite, which is based on the method described in this application.

[0048] Figure 4 The abnormal pixels were classified, and a basic visual presentation and related descriptions were provided.

[0049] Figure 5 We analyzed the response values ​​of excessively weak or excessively strong pixels in the anomalous pixels and selected a portion of the remote sensing image for magnification.

[0050] Figure 6 The specific method for calculating the gradient variance distance during flash element detection;

[0051] Figure 7 This is a schematic diagram of the flash memory detection data stream;

[0052] Figure 8 We analyzed the flash response values ​​in anomalous pixels and selected a portion of the remote sensing image for magnification.

[0053] Figure 9 A schematic diagram illustrating the specific implementation of a sliding window for detecting pixels that are too weak or too strong.

[0054] Figure 10 This is a diagram illustrating outlier statistics.

[0055] Figure 11 A schematic diagram of the data stream for detecting pixels that are too weak or too strong.

[0056] Figure 12 A schematic diagram of the data stream for pixel response value correction;

[0057] Figure 13 Flash element overlap analysis under different threshold conditions, specifically the median value of the 4th, 6th, 8th and 10th gradient variance distances is selected as the detection threshold;

[0058] Figure 14 Analysis of the overlap of weak or strong pixel locations under different threshold conditions, with the specific thresholds being 70%, 75%, 80%, and 85% of the maximum outlier as the detection thresholds;

[0059] Figure 15 A schematic diagram of pixel grayscale value analysis after pixel correction for abnormal pixels. Detailed Implementation

[0060] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be noted that the embodiments described below are intended to facilitate the understanding of the present invention and do not constitute any limitation thereof.

[0061] Due to uneven material distribution and manufacturing processes, anomalous pixels inevitably appear in infrared detectors. This invention proposes a method for detecting and correcting anomalous pixels in on-orbit images from a oscillating infrared camera, aiming to solve the problem of accurate and rapid detection and correction of anomalous pixels. Anomalous pixels are categorized into flickering pixels and pixels that are too weak or too strong, such as... Figure 4 As shown, the specific values ​​of the flash pixel response and the overly weak or overly strong pixel response are as follows: Figure 5 and Figure 8 As shown.

[0062] like Figure 1 As shown, a method for detecting and correcting abnormal pixels in on-orbit images from a oscillating infrared camera includes the following steps:

[0063] Step 1: Perform scintillation detection on the infrared image, calculate the gradient of its pixel response value f, and store it in the array GARD[M×N]; where M is the number of image rows and N is the number of image columns;

[0064] Step 2: Calculate the variance of the gradient values ​​GARD[M×N] of the image data and store them in the array VAR[M×1].

[0065] Step 3: Calculate the distance between adjacent values ​​for each variance in the variance array, take the smaller value, sort them in descending order, and store them in the array DVA[M×1].

[0066] Step 4: Take the median of the first t values ​​in the array DVA[M×1] as the flash element detection threshold THF;

[0067] Step 5: Select the values ​​greater than THF from the array DVA[M×1], find the row of the corresponding image, denote the row as a flash element, and store it in the array #I[t×1].

[0068] Step 6: Correct the detected flash element response value;

[0069] Step 7: Perform further detection of weak or strong pixels in the infrared image, select a window in the image, and store it in array f. win [5×1] in;

[0070] Step 8: Calculate the median and standard deviation for the window, and store the results in DN_med and DN_σ;

[0071] Step 9: Calculate the 3σ range and store it in the array RANGE[2×1];

[0072] Step 10: Determine whether the response value of the center pixel is abnormal. If it is abnormal, record it as 1; otherwise, record it as 0.

[0073] Step 11: Perform anomaly detection on N columns of pixels in each row of the image, accumulate the anomaly count and store it in array CONT[M×1];

[0074] Step 12: Find the maximum value in the array CONT[M×1], and use T% of the maximum value as the detection threshold THB for weak or strong pixels;

[0075] Step 13: Compare the abnormal cell array CONT[M×1] with the threshold THB. If CONT>THB, mark it as a weak or strong cell and store the cell position in array #II[b×1]; where b is the number of weak or strong cells.

[0076] Step 14: Correct the detected weak or strong pixel response values;

[0077] Step 15: Merge arrays #I[a×1] and #II[b×1] to obtain the abnormal cell table, and store it in array POSI[(a+b)×1]; where a is the number of flash cells and b is the number of cells that are too weak or too strong.

[0078] The above implementation method is illustrated below through examples.

[0079] This embodiment combines Figure 1 The flowchart shown illustrates the technical solution in detail. The infrared focal plane detector of the SDGSAT-1 thermal imager payload involved in the embodiment is as follows: Figure 2 As shown, it is composed of four 512×3 detector imaging modules stitched together. The remote sensing images selected in the embodiment are, for example, as shown below. Figure 3 The image shown is a 10000×10240 image created by stitching together a 10000×2048 image.

[0080] First, calculate the gradient of each row of data in the example image to obtain the gradient value GARD[10000×].

[0081]

[10240] ; Calculate the variance of the gradient values, grouping them into rows to obtain the variance array VAR[10000×1]; Calculate the distance between adjacent values ​​in the variance array, taking the smaller of the left and right distances to obtain the distance array DVA[10000×1]. The method for obtaining the distance array is as follows: Figure 6 As shown; the median of the first t values ​​in the distance array DVA[10000×1] is taken as the flash element detection threshold THF, where t = 4, 6, 8, or 10; the entire distance array DVA is compared with the threshold THF, and those distances in the distance array DVA that are greater than the flash element detection threshold THF are marked to obtain the flash element position #I[a×1]. The data stream is as follows: Figure 7 As shown, the detection results under different threshold selection conditions are as follows: Figure 13 As shown; and the pixel position is corrected using pixel response values.

[0082] Subsequently, the image data after flash response value correction is further subjected to weak or strong pixel detection. A window is selected for the image data. Here, to avoid edge conflicts, weak or strong pixel detection starts from the third row and ends at row 10238. The first column of rows 1, 2, 3, 4, and 5 is selected to form a sliding window, resulting in an array f. win [5×1], such as Figure 9 As shown; calculate the median and standard deviation of the window array, DN_med and DN_σ, respectively, and calculate the 3σ range RNAGE[2×1] using the median and standard deviation. The specific calculation method is as follows:

[0083] RNAGE[1]=DN_med-3×DN_σ, RNAGE[2]=DN_med+3×DN_σ

[0084] Determine if the response value of the center pixel (i.e., the pixel in row 3, column 1) is within the 3σ range. If it is, record it as 0; otherwise, mark it as an outlier and record it as 1. Starting from the pixel in row 3, column 1, loop until the pixel in row 3, column 10000, and repeat this operation until the pixel in row 10238, column 10000 is reached. Statistically record the outliers for each row as CONT[10240×1]. Figure 10 As shown; the maximum value in the outlier array CONT is taken, and T% of the maximum value is used as the detection threshold THB for weak or strong pixels; here T can be 70, 75, 80, or 85; the outlier array CONT is compared with the threshold THB, and those outliers in CONT that are greater than the detection threshold THB are marked to obtain the location of weak or strong pixels.

[0085] #I[b×1], data stream as follows Figure 11 As shown, the detection results under different threshold selection conditions are as follows: Figure 14As shown; and pixel response value correction is performed on the locations of pixels that are too weak or too strong, the data stream is as follows Figure 12 As shown.

[0086] Finally, the pixel location array #I[a×1] and the overly weak or overly strong pixel location array #II[b×1] are merged to obtain the abnormal pixel table POSI[(a+b)×1]; at the same time, the finally corrected image is output, as shown. Figure 15 As shown in the figure, (a) is the pixel gray value analysis after pixel correction for pixels that are too weak or too strong; (b) is the pixel gray value analysis after pixel correction for pixels that are too strong.

[0087] The embodiments described above provide a detailed explanation of the technical solutions and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting and correcting abnormal pixels in on-orbit images from a oscillating infrared camera, characterized in that, Includes the following steps: (1) Abnormal pixels in the on-orbit image of the oscillating infrared camera are divided into flash pixels and pixels that are too weak or too strong. (2) Calculate the gradient of the pixel response value f of the on-orbit image and store it in the gradient array GARD[M×N]; Where M is the number of rows in the image and N is the number of columns in the image; (3) Calculate the variance of the gradient values ​​in the gradient array and store it in the variance array VAR[M×1]; (4) Calculate the distance between adjacent values ​​for each variance in the variance array, take the smaller value, sort them in descending order, and store them in the array DVA[M×1]. (5) Take the first t median values ​​in the array DVA[M×1] as the flash element detection threshold THF; (6) Select the values ​​greater than THF in the array DVA[M×1], find the row of the corresponding image, denote the row as the flash element, and store it in the array #I[a×1]; (7) Perform weak or strong pixel detection on the infrared image, select a window in the image, and store it in array f. win [5×1] in; (8) Calculate the median and standard deviation of the window, and further calculate the range of the pixel response value error 3σ; (9) Use the range of 3σ to determine whether the response value of the central pixel is abnormal. If it is abnormal, record it as 1, otherwise record it as 0. (10) Perform anomaly detection on N columns of pixels in each row of the image, and accumulate the anomaly count and store it in array CONT[M×1]; (11) Find the maximum value in the array CONT[M×1] and use the T% of the maximum value as the detection threshold THB for weak or strong pixels; (12) Compare the abnormal cell array CONT[M×1] with the threshold THB. If CONT>THB, mark it as a weak or strong cell and store the cell position in array #II[b×1]; where b is the number of weak or strong cells. (13) Merge arrays #I[a×1] and #II[b×1] to obtain the abnormal pixel table and complete the detection of abnormal pixels; (14) Correct the detected abnormal pixels.

2. The method for detecting and correcting abnormal pixels in on-orbit images of a oscillating infrared camera according to claim 1, characterized in that, In step (2), the gradient calculation formula for the pixel response value f is: Where i is the row number and j is the column number.

3. The method for detecting and correcting abnormal pixels in on-orbit images of a sweeping infrared camera according to claim 2, characterized in that, In step (3), the variance of the gradient values ​​in the gradient array is calculated using the following formula: In the formula, N represents the number of image columns.

4. The method for detecting and correcting abnormal pixels in on-orbit images of a sweeping infrared camera according to claim 1, characterized in that, The specific process of step (4) is as follows: Take the i-th variance VAR[i] in the variance array, and subtract it from the (i-1)-th variance VAR[i-1] and the (i+1)-th variance VAR[i+1] to get two difference values. Take the smaller difference value as the result, denoted as DVA[i], and store it in the variance array VAR[M×1].

5. The method for detecting and correcting abnormal pixels in on-orbit images of a oscillating infrared camera according to claim 1, characterized in that, In step (7), the process of selecting a window for the image is as follows: The pixel being detected is called the center pixel, denoted as f(i,j). Two neighborhoods above the center pixel, f(i-2,j) and f(i-1,j), and two neighborhoods below the center pixel, f(i+1,j) and f(i+2,j), are selected to form a window f. win (i,j)[5×1], where i is the row number and j is the column number.

6. The method for detecting and correcting abnormal pixels in on-orbit images of a oscillating infrared camera according to claim 1, characterized in that, In step (8), the range of pixel response value error 3σ is calculated. The process is as follows: For window f win [5×1] Calculate the median and standard deviation, denoted as DN_med and DN_σ, respectively; Let RNAGE[1] = DN_med - 3 × DN_σ, RNAGE[2] = DN_med + 3 × DN_σ, then the range of 3σ is: RNAGE[1] ≤ 3σ ≤ RNAGE[2].

7. The method for detecting and correcting abnormal pixels in on-orbit images of a oscillating infrared camera according to claim 6, characterized in that, In step (9), the range of 3σ is used to determine whether the response value of the central pixel is abnormal, specifically: If the central pixel f(i, j) is not within the 3σ range, specifically f(i, j) < RANGE[1] or f(i, j) > RANGE[2], then the response value of this central pixel is abnormal, and this pixel is marked as 1, otherwise it is marked as 0.

8. The method for detecting and correcting abnormal pixels in on-orbit images of a oscillating infrared camera according to claim 1, characterized in that, In step (14), the detected abnormal pixels are corrected. The specific process is as follows: Calculate the weighted average of the response values of the upper, lower, left, and right 4 neighboring pixels of this pixel as the correction value of this pixel; The formula for calculating the pixel correction value is as follows: Where u(i,j) is the pixel correction value, λ op [5×1] is the relevance operator, λ op [5×1]=[1 / 6,1 / 3,0,1 / 3,1 / 6], f win [5×1] is the window.

Images