Similar pixel screening method for removing cloud coverage algorithm

Abstract

The invention relates to a similar pixel screening method for a cloud coverage removal algorithm, which can effectively solve the problem that in the prior art, a similar pixel in an under-cloud rapid change area cannot be searched by using single-scene reference data, can effectively judge the change condition of an under-cloud ground feature and predict under-cloud real information, so that the under-cloud real information is adapted to the sudden change condition of the ground feature, and the accuracy of the under-cloud ground feature detection is improved. According to the technical scheme, the method comprises the following steps: 1) inputting remote sensing image data for preprocessing; 2) performing unsupervised classification on the multiband image; 3) preliminarily determining the positions of similar pixels; 4) further confirming the positions of similar pixels through spectral difference by using the reference image; the method is simple, easy to operate, wide in application range, high in precision, stable and reliable, cloud coverage in the remote sensing image can be effectively removed, the quality and usability of the image are improved, the application value of the remote sensing image in the fields of environment monitoring, land utilization planning, agricultural prediction and the like is improved, and the method is an innovation of a similar pixel screening method for a cloud coverage removal algorithm.

Description

1. Technical Field

[0001] The invention relates to the field of remote sensing technology, in particular to a similar pixel screening method for removing cloud cover algorithm. 2. Background Technology

[0002] Remote sensing images are images of the earth's surface acquired through platforms such as remote sensing satellites or drones. The advantage of dynamic monitoring of remote sensing technology has made it quickly become an important technical means in the field of environmental monitoring. However, due to cloud cover, atmospheric interference and other reasons, there are inevitably a large number of clouds and shadows in remote sensing images. These cloud pollutions will affect the extraction and analysis of ground object information and reduce the quality and availability of remote sensing images. There are two types of cloud pollution: thin clouds and thick clouds. Thin cloud pollution can be eliminated based on some image enhancement algorithms, but thick clouds almost completely block the propagation of light, lose information under the clouds, and limit the application of remote sensing images in environmental monitoring, land use planning, agricultural forecasting and other fields. The area under the cloud can only be filled based on pixels that are similar in time and space.

[0003] However, existing algorithms all use spectral thresholds to select pixel positions in the reference image with smaller spectral differences from the target cloud pixel as similar pixels, and weight them according to the size of the difference. Then, using the information of these similar pixel positions, weighted predictions are made in the cloud image to fill the cloud coverage area. For example, the NSPI and improved MNSPI algorithms that weight the time changes of two periods, as well as the WLR algorithm that uses similar pixels to establish linear equations. However, when the local objects mutate, the similar pixels in this type of algorithm will fail both in theory and in practical applications.

[0004] Existing methods can be divided into three categories: the first is based on spectral differences. The basic assumption of this method is that similar pixels have similar change trends over a period of time. Similar pixels are screened by spectral differences at the same position on the reference image. This type of method is difficult to capture fast time change information. The second is based on time change characteristics, and a large amount of data is fitted to find pixels with the same change characteristics as the target pixel. This type of method requires a large amount of long-term series data for reference, and for areas covered by clouds all year round, this type of method cannot be widely promoted and used. The third type is based on the spectrum-time method, which uses cloud images on the basis of the second type. Using multiple images that are not completely covered by clouds over a period of time, the same change trend is found in the time dimension, and the spectral differences are used in space to find similar pixels in the same position of a certain scene cloud image in multiple images, and finally the final position is determined by combining the two. The implementation of this method also requires a large amount of data. At the same time, the diverse mutations of multi-scene image objects may affect the effectiveness of finding similar pixels.

[0005] In summary, the current algorithm cannot fully obtain the temporal change information using the single-view reference image, which affects the performance of the remote sensing cloud removal algorithm. The method of combining the single-view reference image with the target image for classification is one of the effective methods to solve this problem. The temporal change characteristics of the ground objects under the cloud are obtained by unsupervised classification of the clean part of the combined image, and then the classification results are used as training samples, and the reference clean image is supervised to obtain the temporal change characteristics. It can be realized that the temporal change characteristics are extracted according to the single-view reference image, and similar pixels are screened. Therefore, it is of great practical significance to develop an effective method based on change information to remove cloud cover in remote sensing images. III. Summary of the invention

[0006] In view of the above situation, in order to solve the defects of the prior art, the purpose of the present invention is to provide a similar pixel screening method for removing cloud cover algorithm, which can effectively solve the problem that the prior art cannot use single-scene benchmark data to find similar pixels in the rapidly changing area under the cloud, and can effectively identify the changes of the ground objects under the clouds and predict the real information under the clouds, so as to adapt to the situation of sudden changes in the ground objects.

[0007] The technical solution provided by the present invention is that the similar pixel screening method for removing cloud cover algorithm comprises the following steps:

[0008] 1) Preprocessing of input remote sensing image data:

[0009] The input data is a reference cloud-free clear image, a target cloud image and its cloud mask. The two images are merged into a multi-band image and the cloud area is removed by masking to obtain a multi-band image for backup.

[0010] 2) Unsupervised classification of multi-band images:

[0011] Through the unsupervised classification of the multi-band images merged in step 1), we can simply judge the changes of the ground objects in a short period of time, and set the same type of pixels with the same changes into one category. Based on this, we can preliminarily introduce the time variation to participate in the selection of similar pixels, and use the following formula for K-means unsupervised classification:

[0012]

[0013] Where: Euclidean distance ∥x j -u i ∥ 2 is the distance metric, k is the number of categories, x j is the jth element, S i is the i-th category, u i For S i Category center of

[0014] 3) Preliminary determination of the location of similar pixels

[0015] The unsupervised classification results obtained in step 2) are used to perform supervised classification on the reference clean image to determine the change of the area under the cloud and its type, so as to represent the similar change process between pixels, preliminarily determine the position of similar pixels, and use the Bayesian formula to confirm the category of the pixels. The formula is as follows:

[0016]

[0017] Where: It means that when the pixel value is X, the probability that X is of class m. is the probability that the pixel value is X and is of class m, P (X) is the probability that the pixel value is X, represents the probability of a certain class in the sample, It represents the probability that the pixel value in class m is X, and n is the total number of classification categories;

[0018] 4) Use reference images to further confirm the location of similar pixels through spectral differences

[0019] In the reference image, the threshold method needs to be used to find the pixel position with a small difference from the target pixel, so that the pixel position with similar spectrum and similar change can be found by superimposing the pixel with similar change found in step 3). The screening of spectral difference must meet two conditions at the same time: ① The difference between the similar pixel and the target pixel must be smaller than the overall difference of the image, and the calculation formula is shown in formula 3; ② The difference between the similar pixel and the target pixel must be smaller than the difference between the target pixel and its surroundings, and the calculation formula is shown in formula 4;

[0020]

[0021] Where: f t is the target pixel, f i is the ith similar pixel, μ is the mean of the 5*5 area around the similar pixel point, L(x i ,y i ,t1,b) is the value of the b-th band of the ith similar pixel found on the reference image, L(x,y,t1,b) is the value of the b-th band of the target pixel on the reference image, σ(b) is the standard deviation of the entire input image in the b-band, and m is the land cover type of the image. Subsequently, similar pixels are used to establish a linear equation and input the target unknown pixel value of the reference image to calculate the cloud information.

[0022] The method of the invention is simple, easy to operate, has a wide range of applications, high accuracy, and is stable and reliable. It can make full use of the spectral characteristics and change characteristics of the clean area of ​​the image, effectively screen similar pixels with the same other characteristics as the target pixel, and effectively predict the cloud information under the rapidly changing area. It can effectively remove cloud cover in remote sensing images, improve the quality and availability of images, and improve the application value of remote sensing images in environmental monitoring, land use planning, agricultural prediction and other fields. It is an innovation in the similar pixel screening method for cloud cover removal algorithm. IV. Description of the drawings

[0023] Figure 1 The figure shows the experimental results of cloud removal simulated by the improved algorithm of the present invention, where (a) and (b) are the reference image and the baseline image under the cloud during the target period, respectively; (c) is the simulated cloud coverage area; (d) and (f) are the cloud removal results obtained by the WLR and MNSPI algorithms, respectively; (e) and (g) are the cloud removal results obtained by the improved WLR and improved MNSPI algorithms, respectively.

[0024] Figure 2 Schematic diagram of images from different periods. V. Specific implementation methods

[0025] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples.

[0026] Example 1

[0027] The present invention, when implemented, specifically comprises the following steps:

[0028] 1) Preprocessing of input remote sensing image data:

[0029] Using high spatial and spectral resolution images, as well as high temporal resolution images, image data is acquired. The input data is a reference cloud-free clear image, a target cloud image, and its cloud mask. The reference cloud-free clear image and the target cloud image are merged into a multi-band image, and the cloud area is removed by masking to obtain a multi-band image for backup.

[0030] 2) Unsupervised classification of multi-band images:

[0031] Through the unsupervised classification of the multi-band images merged in step 1), we can simply judge the changes of the ground objects in a short period of time, and set the same type of pixels with the same changes into one category. Based on this, we can preliminarily introduce the time variation to participate in the selection of similar pixels, and use the following formula for K-means unsupervised classification:

[0032]

[0033] Where: Euclidean distance ∥x j -ui ∥ 2 is the distance metric, k is the number of categories, x j is the jth element, S i is the i-th category, u i For S i Category center of

[0034] 3) Preliminary determination of the location of similar pixels

[0035] The unsupervised classification results obtained in step 2) are used to perform supervised classification on the reference clean image to determine the change of the area under the cloud and its type, so as to represent the similar change process between pixels, preliminarily determine the position of similar pixels, and use the Bayesian formula to confirm the category of the pixels. The formula is as follows:

[0036]

[0037] Where: It means that when the pixel value is X, the probability that X is of class m. is the probability that the pixel value is X and is of class m, P (X) is the probability that the pixel value is X, represents the probability of a certain class in the sample, It represents the probability that the pixel value in class m is X, and n is the total number of classification categories;

[0038] Use unsupervised classification (Formula 1) and supervised classification (Formula 2) to predict the direction of change of the ground objects under the clouds, and find the pixel positions with the same change in the clean area;

[0039] 4) Use reference images to further confirm the location of similar pixels through spectral differences

[0040] In the reference image, the threshold method needs to be used to find the pixel position with a small difference from the target pixel, so that the pixel position with similar spectrum and similar change can be found by superimposing the pixel with similar change found in step 3). The screening of spectral difference must meet two conditions at the same time: ① The difference between the similar pixel and the target pixel must be smaller than the overall difference of the image, and the calculation formula is shown in formula 3; ② The difference between the similar pixel and the target pixel must be smaller than the difference between the target pixel and its surroundings, and the calculation formula is shown in formula 4;

[0041]

[0042] Where: f t is the target pixel, f i is the ith similar pixel, μ is the mean of the 5*5 area around the similar pixel point, L(x i ,y i,t1,b) is the value of the bth band of the i-th similar pixel found on the reference image, L(x,y,t1,b) is the value of the bth band of the target pixel on the reference image, σ(b) is the standard deviation of the entire input image in the b band, and m is the land cover type of the image. Subsequently, similar pixels are used to establish a linear equation and input the target unknown pixel value of the reference image to calculate the cloud information;

[0043] In the clear image, similar pixel positions with smaller spectral differences from the target pixel are screened out by using equations (3) and (4) using the spectral threshold method, and the final similar pixel positions are determined by combining the pixel positions with the same changes in step 2);

[0044] 5) The similar pixels are used to measure the difference between them and the target pixels using the following formula, and the weighted weight is assigned according to the similarity Di;

[0045] D i =|f i -f t +c|*[(x i -x t ) 2 +(y i -y t ) 2 ] Formula (5)

[0046]

[0047] Where: f t is the target pixel, f i is the i-th similar pixel, c is a constant, to prevent D i is 0, |f i -f t +c| represents the spectral difference, [(x i -x t )2+(y i -y t ) 2 ] represents the distance between the target pixel and the similar pixel, and then the coefficient of the linear equation is calculated using the following formula:

[0048]

[0049] Where: f i is the i-th similar pixel in the reference image, p i is the i-th similar pixel of the main image, W i is the weight of the i-th pixel calculated by formula (6), a is the slope value of the predicted area under the cloud, used in formula (9), b is the intercept value of the predicted area under the cloud, used in formula (9), Represent the average values ​​of all similar pixels in the search window of the main image and the reference image respectively. Finally, the target pixel value of the reference image is substituted into the following formula to calculate the final image value P t :

[0050] P t =a*f t +b Formula (9)

[0051] After actual tests and applications, the addition of the similar pixel screening method of the present invention has been significantly improved compared with the original algorithm, and the area of ​​land cover change can be identified. After repeated tests, the same or similar results were obtained, indicating that the screening method of the present invention is stable and reliable, and has strong practical application value. The following is the relevant test data that the cloud removal effect of the screening method of the present invention is significantly improved compared with the existing WLR algorithm and MNSPI algorithm:

[0052] Taking farmland landscape as an example, the similar pixel screening method for removing cloud cover algorithm of the present invention is used for experiment:

[0053] 1) Preprocessing of input remote sensing image data

[0054] First, through the website https: / / sentinels.copernicus.eu / web / sentinel / Acquire Sentinel-2 images (such as Figure 1 As shown), the data time is September 14, 2022 and September 24, 2022, which are used as reference images and benchmark images respectively. The location is part of Chenzhuang Township, Linying County, Luohe City, Henan Province. The reference clean image is the September 14 image set as cloud cover simulation in the September 24 forecast. The input data refers to the cloud-free clear image, the target cloud image and its cloud mask. The two images are merged into a multi-band image and the cloud area is removed by masking to obtain a multi-band image for standby use;

[0055] 2) Classify multi-band images:

[0056] Through the unsupervised classification of the multi-band images merged in step 1), we can simply judge the changes of the ground objects in a short period of time, and set the same type of pixels with the same changes into one category. Based on this, we can preliminarily introduce the time variation to participate in the selection of similar pixels, and use K-means for unsupervised classification;

[0057] 3) Preliminary determination of the location of similar pixels

[0058] Using the unsupervised classification results obtained in step 2), supervised classification is performed on the reference clean image to determine the change of the area under the cloud and its type, so as to represent the similar change process between pixels and preliminarily determine the position of similar pixels;

[0059] 4) Use reference images to further confirm the location of similar pixels through spectral differences

[0060] In the reference image, the threshold method needs to be used to find the pixel position with a smaller difference from the target pixel, so that the pixel position with similar spectrum and similar change can be found by superimposing the pixel with similar change found in step 3).

[0061] The screening method of the present invention was experimentally compared with the existing WLR and MNSPI algorithms. The similar pixel filling algorithms: WLR and MNSPI were selected as comparisons. The WLR and MNSPI algorithms were directly used to remove clouds to obtain a set of results. Then, the screening method of the present invention (hereinafter referred to as improved WLR and improved MNSPI) was added to calculate a set of results. The two sets of results were compared to show the effectiveness of the strategy.

[0062] The prediction results were evaluated using RMSE (root mean square error) and SSIM (structural similarity index).

[0063]

[0064] Among them, F i Represents the value of the i-th pixel in the image after filling by the cloud removal algorithm, R i represents the value of the i-th pixel in the real image, N represents the total number of calculated pixels, the smaller the RMSE, the closer the image is, and the SSIM calculation formula is as follows:

[0065]

[0066]

[0067] Among them, C1 and C2 are constants to avoid the situation where the calculation cannot be performed when the two images are the same. They are determined based on the grayscale level L of the image and a constant K that is much smaller than 1. μ is the mean value of the image, and σ F is the variance, σ FR Represents the covariance calculation formula. The larger the SSIM is, the closer the visual effects of the two images are.

[0068] Table 1 Comparison of the results of the existing algorithm and the screening method of the present invention

[0069]

[0070] From the two accuracy indicators in the above table, it can be seen that the algorithm with the similar pixel optimization strategy is significantly improved compared with the original algorithm, which shows that the present invention has achieved better results. From the image results, it can be seen that when similar pixels are not screened by the present invention, obvious cloud boundary areas are formed in both the WLR algorithm and the MNSPI algorithm, indicating that the two algorithms are difficult to accurately capture such changes when facing such rapid changes in the surface. However, after combining the method proposed by the present invention, the resulting image is very close to the reference image.

[0071] After repeated experiments, the same or similar results were obtained, indicating that the method is stable, reliable and has strong practical application value.

[0072] In summary, the similar pixel search optimization strategy of the present invention accurately and effectively screens similar pixels that are closer to the target pixel. The predicted results calculated by accessing the commonly used MNSPI and WLR algorithms are improved to a certain extent compared with the original algorithm. The present invention is an optimization method for the similar pixel screening part in the existing pixel filling-based algorithm, which can more accurately find similar pixels to generate clean and cloudless remote sensing images, effectively remove cloud cover in remote sensing images, improve image quality and availability, and improve the application value of remote sensing images in environmental monitoring, land use planning, agricultural forecasting and other fields. It can be applied to all algorithms that use similar pixels to achieve cloud removal, forming a series of cloud removal algorithms that can be used in rapidly changing areas. It is an innovation in the similar pixel screening method for cloud cover removal algorithms, and has good economic and social benefits.

[0073] It should be pointed out that the above is only a preferred embodiment of the present invention and does not limit the present invention in any form. Any technician familiar with the profession can make changes or modify the technical contents disclosed above into equivalent embodiments with equivalent changes without departing from the scope of the technical solution of the present invention, which all fall within the protection scope of the present invention.

Claims

1. A similar pixel screening method for cloud cover removal algorithm, It is characterized in that The following steps are involved: 1) Preprocessing of input remote sensing image data: The input data is a reference cloud-free clear image, a target cloud image and its cloud mask. The two images are merged into a multi-band image and the cloud area is removed by masking to obtain a multi-band image for backup. 2) Unsupervised classification of multi-band images: Through the unsupervised classification of the multi-band images merged in step 1), we can simply judge the changes of the ground objects in a short period of time, and set the same type of pixels with the same changes into one category. Based on this, we can preliminarily introduce the time variation to participate in the selection of similar pixels, and use the following formula for K-means unsupervised classification: Where: Euclidean distance ∥x j -u i ∥ 2 is the distance metric, k is the number of categories, x j is the jth element, S i is the i-th category, u i For S i Category center of 3) Preliminary determination of the location of similar pixels The unsupervised classification results obtained in step 2) are used to perform supervised classification on the reference clean image to determine the change of the area under the cloud and its type, so as to represent the similar change process between pixels, preliminarily determine the position of similar pixels, and use the Bayesian formula to confirm the category of the pixels. The formula is as follows: Where: It means that when the pixel value is X, the probability that X is of class m. is the probability that the pixel value is X and is of class m, P (X) is the probability that the pixel value is X, represents the probability of a certain class in the sample, It represents the probability that the pixel value in class m is X, and n is the total number of classification categories; 4) Use reference images to further confirm the location of similar pixels through spectral differences In the reference image, the threshold method needs to be used to find the pixel position with a small difference from the target pixel, so that the pixel position with similar spectrum and similar change can be found by superimposing the pixel with similar change found in step 3). The screening of spectral difference must meet two conditions at the same time: ① The difference between the similar pixel and the target pixel must be smaller than the overall difference of the image. The calculation formula is shown in formula 3; ② The difference between the similar pixel and the target pixel is smaller than the difference between the target pixel and its surroundings. The calculation formula is shown in formula 4; Where: f t is the target pixel, f i is the ith similar pixel, μ is the mean of the 5*5 area around the similar pixel point, L(x i ,y i ,t 1 , b) is the value of the bth band of the ith similar pixel found on the reference image, L(x,y,t 1 ,b) is the value of the b-th band of the target pixel on the reference image, σ(b) is the standard deviation of the entire input image in the b-band, and m is the land cover type of the image. Subsequently, similar pixels are used to establish a linear equation and input the target unknown pixel value of the reference image to calculate the cloud information.

2. The similar pixel screening method for removing cloud cover algorithm according to claim 1, It is characterized in that The specific steps include: 1) Preprocessing of input remote sensing image data: Using high spatial and spectral resolution images, as well as high temporal resolution images, image data is acquired. The input data is a reference cloud-free clear image, a target cloud image, and its cloud mask. The reference cloud-free clear image and the target cloud image are merged into a multi-band image, and the cloud area is removed by masking to obtain a multi-band image for backup. 2) Unsupervised classification of multi-band images: Through the unsupervised classification of the multi-band images merged in step 1), we can simply judge the changes of the ground objects in a short period of time, and set the same type of pixels with the same changes into one category. Based on this, we can preliminarily introduce the time variation to participate in the selection of similar pixels, and use the following formula for K-means unsupervised classification: Where: Euclidean distance ∥x j -u i ∥ 2 is the distance metric, k is the number of categories, x j is the jth element, S i is the i-th category, u i For S i Category center of 3) Preliminary determination of the location of similar pixels The unsupervised classification results obtained in step 2) are used to perform supervised classification on the reference clean image to determine the change of the area under the cloud and its type, so as to represent the similar change process between pixels, preliminarily determine the position of similar pixels, and use the Bayesian formula to confirm the category of the pixels. The formula is as follows: Where: It means that when the pixel value is X, the probability that X is of class m. is the probability that the pixel value is X and is of class m, P (X) is the probability that the pixel value is X, represents the probability of a certain class in the sample, It represents the probability that the pixel value in class m is X, and n is the total number of classification categories; 4) Use reference images to further confirm the location of similar pixels through spectral differences In the reference image, the threshold method needs to be used to find the pixel position with a small difference from the target pixel, so that the pixel position with similar spectrum and similar change can be found by superimposing the pixel with similar change found in step 3). The screening of spectral difference must meet two conditions at the same time: ① The difference between the similar pixel and the target pixel must be smaller than the overall difference of the image. The calculation formula is shown in formula 3; ② The difference between the similar pixel and the target pixel is smaller than the difference between the target pixel and its surroundings. The calculation formula is shown in formula 4; Where: f t is the target pixel, f i is the ith similar pixel, μ is the mean of the 5*5 area around the similar pixel point, L(x i ,y i ,t 1 , b) is the value of the bth band of the ith similar pixel found on the reference image, L(x,y,t 1 , b) is the value of the b-th band of the target pixel on the reference image, σ(b) is the standard deviation of the entire input image in the b-band, and m is the land cover type of the image. Subsequently, similar pixels are used to establish a linear equation and input the unknown pixel value of the target in the reference image to calculate the cloud information; 5) The similar pixels are used to measure the difference between them and the target pixels using the following formula, and the weighted weight is assigned according to the similarity Di; D i = |f i - f t + c| * [(x i - x t ) 2 + (y i - y t ) 2 Equation (5) Where: c is a constant to prevent D i is 0, |f i -f t +c| represents the spectral difference, [(x i -x t )2+(y i -y t ) 2 ] represents the distance between the target pixel and the similar pixel, and then the coefficient of the linear equation is calculated using the following formula: Where: f i is the i-th similar pixel in the reference image, p i is the i-th similar pixel of the main image, W i is the weight of the i-th pixel calculated by formula (6), a is the slope value of the predicted area under the cloud, used in formula (9), b is the intercept value of the predicted area under the cloud, used in formula (9), Represent the average values ​​of all similar pixels in the search window of the main image and the reference image respectively. Finally, the target pixel value of the reference image is substituted into the following formula to calculate the final image value P t : P t =a*f t +b Formula (9).

3. Application of the similar pixel screening method for cloud cover removal algorithm described in claim 1 in an algorithm for cloud removal using similar pixels.

Images