Remote sensing image non-uniformity correction method based on neighborhood statistics and total variation model

Abstract

The present application relates to a remote sensing image non-uniformity correction method based on neighborhood statistics and total variation model, and belongs to the technical field of image processing. The present application is aimed at the abnormal pixel response problem existing in the imaging process of a remote sensing camera and the non-linear response problem caused by factors such as manufacturing process, and sequentially performs abnormal pixel and non-uniformity correction processing on the imaging data of the remote sensing camera. The abnormal pixel correction is completed by using the neighborhood information of the abnormal pixels in the remote sensing image, the non-uniformity correction is based on the total variation model to solve the optimal correction result of the remote sensing image, and the texture details of the image are effectively preserved in combination with the adaptive wavelet transform. The remote sensing camera non-uniformity correction algorithm proposed by the present application can effectively correct the abnormal pixels and non-linear response in the remote sensing image.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a remote sensing image non-uniformity correction method based on neighborhood statistics and a total variation model. Background Art

[0002] Remote sensing cameras can perform tasks such as earth observation and target detection, and are of great significance in geographic surveying and mapping, land planning, resource surveys, earthquake mitigation, meteorological observation, environmental monitoring, agriculture, forestry, and animal husbandry. However, due to factors such as manufacturing materials and process technology, as well as changes in operating environment and temperature, remote sensing cameras inevitably suffer from anomalous pixels and non-uniformity. This significantly reduces the signal-to-noise ratio of the system's output image, seriously impacting image quality and subsequent applications. Therefore, correcting for these anomalous pixels and non-uniform responses is a critical processing task.

[0003] At present, the existing remote sensing image non-uniformity correction technologies can be divided into two categories according to their correction principles:

[0004] The first category involves non-uniformity correction techniques based on radiation source calibration. These can be categorized into single-point, two-point, and multi-point corrections, depending on the amount of uniform radiation imaging data collected. By acquiring multiple uniform radiation images at varying brightnesses, single-point correction for a single uniform radiation image, two-point correction for two uniform radiation images, multi-point correction, and curve fitting correction methods are developed, resulting in more accurate non-uniformity correction coefficients. However, focal plane non-uniformity exhibits temporal drift. Changes in detector operating conditions, such as changes in focal plane temperature or readout circuit voltage, alter the response coefficients of each detector pixel, rendering the non-uniformity correction coefficients calculated using calibration techniques inapplicable. Consequently, images using the original correction coefficients exhibit a different, fixed form of non-uniformity noise, further degrading image quality. Therefore, if calibration techniques are used for non-uniformity correction, the focal plane must be periodically recalibrated and the correction coefficients recalculated and updated to compensate for temporal non-uniformity.

[0005] The second category involves non-uniformity correction techniques based on actual ground object scenes. This technique aims to utilize the data characteristics of the scene itself for non-uniformity correction and has broader application prospects than calibration algorithms. However, due to the selection of a large, uniform ground object scene that fits the camera's width as the calibration field, and the uniform radiation within the field, this technique can only effectively correct the non-uniformity of detection pixels for pixels with good linearity. For detectors with poor linearity, this technique can only effectively correct the camera's non-uniformity within a certain radiance range and cannot achieve non-uniformity correction across a wide dynamic range. Furthermore, relying on a large, uniform ground field as the calibration field makes it difficult to achieve high-frequency on-orbit calibration of the camera. Summary of the Invention

[0006] In view of this, the present invention proposes a remote sensing image non-uniformity correction method based on neighborhood statistics and total variation model, which can deal with the abnormal pixel response problem and nonlinear response problem caused by factors such as manufacturing process in the remote sensing camera imaging process.

[0007] The technical solution adopted in the present invention is:

[0008] A remote sensing image non-uniformity correction method based on neighborhood statistics and total variation model includes the following steps:

[0009] Step 1: Input the remote sensing image data to be processed, set the detection threshold of abnormal pixels according to the calculation result of the grayscale mean of the entire image, and set the judgment condition of strip-shaped abnormal pixels;

[0010] Step 2: Detect the strip-shaped abnormal pixel column according to the image mean, column mean, and strip-shaped abnormal pixel judgment conditions. If the two adjacent columns of the abnormal pixel column are both strip-shaped abnormal pixels, keep the column data unprocessed. If there is a normal column between the two adjacent columns, compensate for it.

[0011] Step 3: Detect the compensated image. If there are still strip-shaped abnormal pixels, return to step 2. If there are no strip-shaped abnormal pixels and the image has not been transposed, transpose the image and return to step 2. If there are no strip-shaped abnormal pixels and the image has been transposed, proceed to step 4.

[0012] Step 4: Input the image data after strip-shaped abnormal pixel processing, detect point-block abnormal pixels according to the abnormal pixel detection threshold, and compensate for the abnormal pixels according to the conditions of the pixels around the abnormal pixels;

[0013] Step 5: Detect the image after abnormal pixel compensation. If there are still point-block abnormal pixels, repeat step 4 until all point-block abnormal pixels are compensated.

[0014] Step 6: Construct a total variation model for the compensated image, and use the split Bregman method to iteratively solve the model. After obtaining the optimal solution of the total variation model, use adaptive wavelet transform to retain the texture details of the remote sensing image and obtain the final remote sensing image non-uniformity correction result.

[0015] Furthermore, the specific method of step 2 is:

[0016] Step 2.1: Calculate the ratio of abnormal pixels in each column, i.e., the column ratio. Determine whether the column ratio is greater than the strip-shaped abnormal pixel determination threshold, and whether the column mean is greater than twice the image mean. If any of these conditions are met, the column is determined to be an abnormal pixel strip column, and the strip column positions of all abnormal pixels are recorded.

[0017] Step 2.2: Extend the left and right boundaries of the image data to determine whether the left and right columns of the abnormal pixel strip column to be compensated are also abnormal pixel strip columns. If neither is the case, replace the strip column to be compensated with the average of the left and right columns; if one column is an abnormal pixel strip column, replace the value of the column to be compensated with the value of the normal column; if both the left and right columns are abnormal pixel strip columns, compensate the other columns first.

[0018] Furthermore, the specific method of step 4 is:

[0019] Take the abnormal pixel as the center and select eight neighboring pixels. Group the two horizontal pixels into one group, the two vertical pixels into one group, the two left oblique pixels into one group, and the two right oblique pixels into one group, for a total of four groups. If both pixels in a group are normal pixels, the group is called a normal group.

[0020] According to the abnormal pixel conditions in the eight neighboring pixels, corresponding compensation methods are adopted respectively;

[0021] Method 1: If there are at least two normal groups in the eight-neighborhood pixels, calculate the absolute value of the grayscale difference between the two pixels in each normal group, select the normal group with the smallest absolute value, and replace the grayscale value of the central pixel with the mean of the grayscale values ​​of the two pixels in the group;

[0022] Method 2: If there are at least three abnormal pixels in the eight-neighborhood pixels, and these three abnormal pixels are in three different groups, the median value of all normal pixels in the eight-neighborhood pixels is taken to replace the grayscale value of the central pixel;

[0023] Method 3: If all eight neighborhood pixels are abnormal pixels, return to step 2 to compensate for the abnormal pixels in the neighborhood, and then compensate for the central pixel.

[0024] Furthermore, the specific method of step 6 is:

[0025] Step 6.1: Build a highly flexible and scalable energy functional model based on the imaging system distortion model. Experimentally adjust the remote sensing image model and the regularization coefficients to the optimal value, and finally construct the total variation model.

[0026] Step 6.2: Use the split-Bregman method to iteratively solve the constructed total variation model. In the solution process, two auxiliary variables and two iteration parameters are introduced to decompose the problem into several sub-problems and solve them one by one. Among them, the auxiliary variables are used to solve the total variation model, and the iteration parameters are used to accelerate the iteration process.

[0027] Step 6.3: After obtaining the optimal solution of the total variation model, select the wavelet basis and the number of wavelet decomposition layers, perform wavelet decomposition on the spatial domain of the remote sensing image, and obtain the wavelet decomposition coefficients;

[0028] Step 6.4: Based on the noise variance and signal standard deviation of the image, an approximate solution for the Bayes Shrink threshold is obtained. The approximate solution is adjusted using the constants set in the Bayesian optimization process to obtain an adaptive threshold for filtering out the high-frequency wavelet coefficients. The processed wavelet coefficients are reconstructed to complete the adaptive wavelet transform processing of the image.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] The present invention can be applied to improving the imaging quality of remote sensing cameras. While effectively retaining the detailed information of ground objects in the image, it can remove abnormal pixels and non-uniformity in the image, providing high-quality remote sensing data support for subsequent remote sensing image processing and applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a schematic diagram of the overall process of remote sensing image non-uniformity correction of the present invention;

[0032] Figure 2 It is a remote sensing image with non-uniform response;

[0033] Figure 3 This is the result after the non-uniformity correction is completed. DETAILED DESCRIPTION

[0034] The technical solution of the present invention is further described in detail below with reference to the accompanying drawings.

[0035] A remote sensing image non-uniformity correction method based on neighborhood statistics and total variation model is proposed. The method first uses the neighborhood information of abnormal pixels in the remote sensing image to complete the correction of abnormal pixels. Then, based on the total variation model, the non-uniformity optimization correction result of the remote sensing image after abnormal pixel correction is solved. The adaptive wavelet transform is combined to effectively retain the texture details of the image. Finally, the corrected image is output.

[0036] like Figure 1 As shown, the specific process of this method is:

[0037] Step 1: Input the remote sensing image data to be processed, set the detection threshold of abnormal pixels according to the calculation result of the grayscale mean of the entire image, and set the judgment condition of strip-shaped abnormal pixels;

[0038] Step 2: Detect strip-shaped abnormal pixels according to the image mean, column mean, and strip-shaped abnormal pixel determination threshold, calculate the proportion of abnormal pixels in each column, determine whether the column proportion is greater than the strip-shaped abnormal pixel determination threshold, and determine whether the column mean is greater than twice the image mean. If any of the above conditions are met, the column is determined to be an abnormal pixel strip column, record the strip column position, and remove duplicate values; perform left and right boundary expansion on the image data (directly copy the boundary column for expansion), determine whether the left and right columns of the abnormal pixel strip column to be compensated are also abnormal pixel strip columns, if not, replace the strip column to be compensated with the mean of the two columns; if one column is an abnormal pixel strip column, replace the value of the column to be compensated with the value of the normal column; if both columns are abnormal pixel strip columns, compensate the other columns first;

[0039] Step 3: Detect the compensated image. If there are still strip-shaped abnormal pixels, repeat step 2 until all strip-shaped abnormal pixels are compensated. If there are no strip-shaped abnormal pixels and the image has not been transposed in step 3, transpose the image and return to step 2. If there are no strip-shaped abnormal pixels and the image has been transposed once in step 3, proceed to step 4.

[0040] Step 4: Input the image data after strip-shaped abnormal pixel processing. Take the abnormal pixel as the center and select eight neighboring pixels. Group the two horizontal pixels into one group, the two vertical pixels into one group, the two left oblique pixels into one group, and the two right oblique pixels into one group, for a total of four groups. If both pixels in a group are normal pixels, the group is called a normal group.

[0041] According to the abnormal pixel conditions in the eight neighboring pixels, corresponding compensation methods are adopted respectively;

[0042] Method 1: If there are at least two normal groups in the eight-neighborhood pixels, calculate the absolute value of the grayscale difference between the two pixels in each normal group, select the normal group with the smallest absolute value, and replace the grayscale value of the central pixel with the mean of the grayscale values ​​of the two pixels in the group;

[0043] Method 2: If there are at least three abnormal pixels in the eight-neighborhood pixels, and these three abnormal pixels are in three different groups, the median value of all normal pixels in the eight-neighborhood pixels is taken to replace the grayscale value of the central pixel;

[0044] Method 3: If all eight neighboring pixels are abnormal pixels, return to step 2 to compensate for the abnormal pixels in the neighborhood, and then compensate for the central pixel;

[0045] Step 5: Detect the compensated image. If there are still abnormal point-block pixels, repeat step 4 until all abnormal point-block pixels are compensated.

[0046] Step 6: Solve the total variation model for the compensated image, and use the split Bregman method (see the reference “The Split Bregman Method for L1-Regularized Problems”) to solve the optimal correction result. Then, perform an adaptive wavelet transform on the optimal solution of the total variation model to obtain the final non-uniformity correction result.

[0047] The specific method of step 6 is:

[0048] Step 6.1: Based on the imaging system distortion model, a highly flexible and scalable energy functional model is established (see reference: Zhao J, Zhou Q, Chen Y, et al. Single image stripe nonuniformity correction with gradient-constrained optimization model for infrared focal plane arrays[J]. Optics Communications, 2013, 296: 47-52.). The remote sensing image model and the set regularization coefficient are adjusted to the optimal value through experiments (see reference: Yang J, Zhao X, Ma T, et al. Remote sensing images destriping using unidirectional hybrid total variation and nonconvex low-rank regularization[J]. Journal of Computational and Applied Mathematics, 2020, 363: 124-144.). The final total variation model is obtained.

[0049] Step 6.2: Use the split Bregman method to iteratively solve the constructed total variation model. In the solution process, two auxiliary variables and two iteration parameters are introduced to decompose the problem into several sub-problems and solve them one by one. The auxiliary variables are used to solve the total variation model, and the iteration parameters are used to accelerate the iteration process (see the reference "The Split Bregman Method for L1-Regularized Problems");

[0050] Step 6.3: After obtaining the optimal solution of the total variation model, select the wavelet basis and the number of wavelet decomposition layers, perform wavelet decomposition on the spatial domain of the remote sensing image, and obtain the wavelet decomposition coefficients;

[0051] Step 6.4: Based on the noise variance and signal standard deviation of the image, obtain an approximate solution for the Bayesian Shrink threshold (see the reference "Adaptive Wavelet Thresholding for Image Denoising and Compression"). Adjust the approximate solution using the constants set in the Bayesian optimization process to obtain an adaptive threshold for filtering out the high-frequency wavelet coefficients. Reconstruct the processed wavelet coefficients to complete the adaptive wavelet transform processing of the image.

[0052] Here's a more concrete example:

[0053] Figure 2 This is an example of a remote sensing image with non-uniform response. As can be seen in the figure, there are several abnormal pixel strips and blocks, as well as nonlinear responses. Abnormal pixels include dead pixels that have no response or a small response to light signals, and overheated pixels that have an excessive response to light signals. The nonlinear response is mainly manifested in the figure as inconsistent grayscale values ​​in uniform areas. A remote sensing image non-uniformity correction method based on neighborhood statistics and a total variation model can effectively correct abnormal pixels and nonlinear responses in remote sensing images. The specific steps are as follows:

[0054] Step 1: Input the remote sensing image data to be processed and set the pixel grayscale threshold DN according to the calculation result of the grayscale mean of the entire image. min =0~15、DN max =200~255, and set the threshold value for determining strip-shaped abnormal pixels to 0.5;

[0055] Step 2: Detect strip-shaped abnormal pixels according to the image mean, column mean, and strip-shaped abnormal pixel judgment conditions, calculate the proportion of abnormal pixels in each column, determine whether the column proportion is greater than the strip-shaped abnormal pixel judgment threshold, and determine whether the column mean is greater than twice the image mean. If any of the above conditions are met, the column is determined to be an abnormal pixel strip column, record the strip column position, and remove duplicate strip column positions; perform left and right boundary extension on the image data, determine whether the left and right columns of the abnormal pixel strip column to be compensated are also abnormal pixel strip columns, if not, replace the strip column to be compensated with the mean of the two columns; if one column is an abnormal pixel strip column, replace the value of the column to be compensated with the value of the normal column; if both columns are abnormal pixel strip columns, compensate the other columns first;

[0056] Step 3: Detect the compensated image. If there are still strip-shaped abnormal pixels, repeat step 2 until all strip-shaped abnormal pixels are compensated. If there are no strip-shaped abnormal pixels and the image has not been transposed in step 3, transpose the image and return to step 2. If there are no strip-shaped abnormal pixels and the image has been transposed once in step 3, proceed to step 4.

[0057] Step 4: Input the image data after strip-shaped abnormal pixel processing, take the abnormal pixel as the center and select eight neighboring pixels. Group the two horizontal pixels in the eight neighboring pixels into one group, the two vertical pixels into one group, the two left oblique pixels into one group, and the two right oblique pixels into one group, for a total of four groups. According to the abnormal pixel situation in the eight neighboring pixels, adopt the corresponding compensation method.

[0058] Step 5: Detect the compensated image. If there are still abnormal point-block pixels, repeat step 4 until all abnormal point-block pixels are compensated.

[0059] Step 6: Solve the total variation model for the compensated image and solve the optimal correction result by Split Bregman splitting method; perform adaptive wavelet transform on the optimal solution of the total variation model to obtain the final non-uniformity correction result, such as Figure 3 shown.

[0060] In summary, the present invention addresses the issues of abnormal pixel responses and nonlinear responses caused by factors such as the manufacturing process in remote sensing camera imaging. It sequentially performs abnormal pixel and nonuniformity correction on remote sensing camera imaging data. Abnormal pixel correction utilizes information about the neighborhood of abnormal pixels in the remote sensing image to correct abnormal pixels. Nonuniformity correction optimizes the correction results of the remote sensing image based on a total variation model, combined with an adaptive wavelet transform to effectively preserve image texture details. Experiments have shown that the proposed remote sensing camera nonuniformity correction algorithm can effectively preserve ground feature details in remote sensing camera images while removing abnormal pixels and nonlinear responses, thereby effectively correcting remote sensing images.

[0061] The specific embodiments described above further illustrate the objectives, technical solutions, and beneficial effects of the present invention. It should be understood that the above description is merely a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A remote sensing image non-uniformity correction method based on neighborhood statistics and total variation model, characterized in that: The following steps are involved: Step 1: Input the remote sensing image data to be processed, set the detection threshold of abnormal pixels according to the calculation result of the grayscale mean of the entire image, and set the judgment condition of strip-shaped abnormal pixels; Step 2: Detect the strip-shaped abnormal pixel column according to the image mean, column mean, and strip-shaped abnormal pixel judgment conditions. If the two adjacent columns of the abnormal pixel column are both strip-shaped abnormal pixels, keep the column data unprocessed. If there is a normal column between the two adjacent columns, compensate for it. Step 3: Detect the compensated image. If there are still strip-shaped abnormal pixels, return to step 2. If there are no strip-shaped abnormal pixels and the image has not been transposed, transpose the image and return to step 2. If there are no strip-shaped abnormal pixels and the image has been transposed, proceed to step 4. Step 4: Input the image data after strip-shaped abnormal pixel processing, detect point-block abnormal pixels according to the abnormal pixel detection threshold, and compensate for the abnormal pixels according to the conditions of the pixels around the abnormal pixels; Step 5: Detect the image after abnormal pixel compensation. If there are still point-block abnormal pixels, repeat step 4 until all point-block abnormal pixels are compensated. Step 6: Construct a total variation model for the compensated image, and use the split Bregman method to iteratively solve the model. After obtaining the optimal solution of the total variation model, use adaptive wavelet transform to retain the texture details of the remote sensing image and obtain the final remote sensing image non-uniformity correction result.

2. The remote sensing image non-uniformity correction method based on neighborhood statistics and total variation model according to claim 1, characterized in that: The specific method of step 2 is: Step 2.1: Calculate the ratio of abnormal pixels in each column, i.e., the column ratio. Determine whether the column ratio is greater than the strip-shaped abnormal pixel determination threshold, and whether the column mean is greater than twice the image mean. If any of these conditions are met, the column is determined to be an abnormal pixel strip column, and the strip column positions of all abnormal pixels are recorded. Step 2.2: Extend the left and right boundaries of the image data to determine whether the left and right columns of the abnormal pixel strip column to be compensated are also abnormal pixel strip columns. If neither is the case, replace the strip column to be compensated with the average of the left and right columns; if one column is an abnormal pixel strip column, replace the value of the column to be compensated with the value of the normal column; if both the left and right columns are abnormal pixel strip columns, compensate the other columns first.

3. The remote sensing image non-uniformity correction method based on neighborhood statistics and total variation model according to claim 2, characterized in that: The specific method of step 4 is: Take the abnormal pixel as the center and select eight neighboring pixels. Group the two horizontal pixels into one group, the two vertical pixels into one group, the two left oblique pixels into one group, and the two right oblique pixels into one group, for a total of four groups. If both pixels in a group are normal pixels, the group is called a normal group. According to the abnormal pixel conditions in the eight neighboring pixels, corresponding compensation methods are adopted respectively; Method 1: If there are at least two normal groups in the eight-neighborhood pixels, calculate the absolute value of the grayscale difference between the two pixels in each normal group, select the normal group with the smallest absolute value, and replace the grayscale value of the central pixel with the mean of the grayscale values ​​of the two pixels in the group; Method 2: If there are at least three abnormal pixels in the eight-neighborhood pixels, and these three abnormal pixels are in three different groups, the median value of all normal pixels in the eight-neighborhood pixels is taken to replace the grayscale value of the central pixel; Method 3: If all eight neighborhood pixels are abnormal pixels, return to step 2 to compensate for the abnormal pixels in the neighborhood, and then compensate for the central pixel.

4. The remote sensing image non-uniformity correction method based on neighborhood statistics and total variation model according to claim 1, characterized in that: The specific method of step 6 is: Step 6.1: Build a highly flexible and scalable energy functional model based on the imaging system distortion model. Experimentally adjust the remote sensing image model and the regularization coefficients to the optimal value, and finally construct the total variation model. Step 6.2: Use the split-Bregman method to iteratively solve the constructed total variation model. In the solution process, two auxiliary variables and two iteration parameters are introduced to decompose the problem into several sub-problems and solve them one by one. Among them, the auxiliary variables are used to solve the total variation model, and the iteration parameters are used to accelerate the iteration process. Step 6.3: After obtaining the optimal solution of the total variation model, select the wavelet basis and the number of wavelet decomposition layers, perform wavelet decomposition on the spatial domain of the remote sensing image, and obtain the wavelet decomposition coefficients; Step 6.4: Based on the noise variance and signal standard deviation of the image, an approximate solution for the Bayes Shrink threshold is obtained. The approximate solution is adjusted using the constants set in the Bayesian optimization process to obtain an adaptive threshold for filtering out the high-frequency wavelet coefficients. The processed wavelet coefficients are reconstructed to complete the adaptive wavelet transform processing of the image.

Images