Timing sequence remote sensing image cloud and mist removal method based on multi-index fusion

Abstract

The invention discloses a time sequence remote sensing image cloud and mist removal method based on multi-index fusion, which comprises the following steps: 1, collecting a time sequence RGB remote sensing image of a target area, carrying out geometric registration and radiation normalization, and taking the image with the highest transmissivity as a reference base map; 2, constructing a single-image multi-dimensional quality evaluation system comprising seven quality indexes; 3, identifying a cloud pollution type by adopting a layered pollution detection strategy, generating a pollution mask and calculating a pixel-level quality score; 4, calculating a similarity matrix among the images to eliminate polluted images; and 5, carrying out statistics on pixel-level quality scores of pixel positions in all time sequences, generating pixel-level confidence coefficients according to the quality scores and time sequence ranking, carrying out spatial smoothing by using guided filtering, implementing adaptive Top-K weighted fusion, carrying out global hue matching and edge feathering processing, and outputting cloudless images, pixel-level confidence coefficients and pollution type distribution diagrams. According to the method, thick cloud, thin cloud and cloud shadow in the time sequence remote sensing image can be accurately identified and removed.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing image processing technology, specifically a method for removing clouds and fog from time-series remote sensing images based on multi-index fusion. Background Technology

[0002] Remote sensing technology, as an important means of Earth observation, has been widely applied in many fields such as resource surveys, environmental monitoring, urban planning, and disaster assessment. High-quality remote sensing imagery is the foundation and prerequisite for these applications. However, remote sensing imagery is often affected by atmospheric conditions during acquisition, with cloud, thin cloud, and cloud shadow pollution being one of the most significant limiting factors. Statistics show that over 60% of optical remote sensing images globally suffer from varying degrees of cloud pollution, which seriously affects image quality and the accuracy of subsequent applications. This is especially true in cloudy and rainy areas, where acquiring cloud-free imagery is even more difficult, resulting in a large amount of remote sensing data that cannot be used directly, causing serious data waste and application bottlenecks.

[0003] Cloud pollution affects remote sensing images in several ways: First, thick clouds completely obscure surface information, causing the complete loss of ground features in the covered areas; second, although thin clouds do not completely obscure the surface, they can increase image brightness, decrease color saturation, and distort the spectral characteristics of ground features; third, cloud shadows significantly reduce surface brightness and cause color temperature shifts, making it difficult to identify ground features. Moreover, these three types of pollution often coexist in a single image, posing a significant challenge to image interpretation and automated processing. Therefore, how to effectively remove clouds, thin clouds, and cloud shadows from remote sensing images and restore clear surface information has always been a key and difficult problem in the field of remote sensing image processing.

[0004] Traditional methods for removing clouds from single-view images mainly include dehazing algorithms based on dark channel priors, cloud detection and inpainting based on morphology, and cloud region identification based on texture analysis. These methods can detect cloud-contaminated areas to some extent, but they are significantly insufficient in terms of information recovery from cloud areas. Since the information of ground features obscured by clouds in a single-view image is completely lost, interpolation, inpainting, or filtering methods can only infer from surrounding pixels, making it difficult to accurately restore surface features. This is especially true for large areas covered by clouds, where the inpainting effect is often unsatisfactory. In addition, traditional methods have limited ability to process thin clouds and cloud shadows. The semi-transparent nature of thin clouds makes them difficult to segment accurately, and the spectral characteristics of cloud shadows are similar to those of normal dark areas (such as water bodies and shadowed buildings), which can easily lead to misjudgments.

[0005] In recent years, with the shortening of the revisit cycle of remote sensing satellites and the improvement of data acquisition capabilities, the availability of time-series remote sensing images has increased significantly. Using multiple images of the same area at different time series for cloud and fog removal has become a new research approach. The basic principle of time-series image fusion is that clouds, thin clouds, and cloud shadows are temporal anomalies, while surface features are relatively stable in the short term. By comprehensively analyzing multiple time-series images, the true surface information can be extracted from the "clean" time series and replaced with cloud-polluted areas in the "polluted" time series. This method fundamentally solves the problem of missing information and, theoretically, can achieve true cloud and fog removal.

[0006] However, existing temporal declouding methods still have many problems and limitations: 1) In terms of pollution type identification, most methods mainly target thick clouds, using simple brightness thresholds or whiteness indices for cloud detection, which is insufficient for identifying thin clouds and cloud shadows. Specifically, the semi-transparent nature of thin clouds makes their spectral features lie between those of clouds and the ground surface, making it difficult to accurately segment them using fixed thresholds; the low brightness features of cloud shadows are easily confused with normal dark areas such as water bodies, road asphalt, and building shadows, resulting in a high false positive rate for methods based solely on brightness; 2) In terms of quality assessment, existing methods have relatively simple evaluation indicators, mostly considering only brightness or simple spectral indicators, lacking a comprehensive consideration of spectral features, temporal stability, and spatial consistency. 3) In terms of fusion strategies, many methods use fixed weights or simple statistical methods such as temporal median and average, failing to adaptively select the optimal temporal sequence based on the actual degree of pixel contamination and quality differences. This easily leads to a "meaning" effect, causing the fusion results to lose detailed information. 4) In terms of versatility, some advanced methods rely on special bands such as near-infrared and short-wave infrared, or require a large number of labeled samples and GPU computing power when using deep learning models. This limits the application of these methods on ordinary RGB remote sensing images (such as aerial images, some satellite images, and UAV images), making it difficult to meet the needs of different data sources and application scenarios.

[0007] In addition, existing time-series declouding methods face some technical challenges when dealing with complex scenarios, such as: how to select the "relatively optimal" time series for fusion when a certain region of all time series is cloud-contaminated; how to preserve ground feature details while removing cloud effects when thin clouds cover the area; how to accurately distinguish and process cloud shadows and normal shadows when they coexist; and how to avoid introducing time series inconsistencies while declouding when there are seasonal or ground feature changes in different time series. These problems are common in practical applications, but there is currently a lack of systematic solutions.

[0008] In summary, although temporal image fusion provides an effective approach for cloud and fog removal in remote sensing images, existing methods still have significant shortcomings in terms of the comprehensiveness of pollution type identification, the multidimensionality of quality assessment, the adaptability of fusion strategies, and the generality of algorithms. In particular, for remote sensing images containing only RGB bands, there is a lack of a high-precision, robust, and easily deployable temporal cloud removal method that can comprehensively identify and remove thick clouds, thin clouds, and cloud shadows without deep learning. Therefore, developing a general cloud and fog removal algorithm based on multi-index fusion, employing a hierarchical detection strategy, and achieving adaptive temporal fusion has significant theoretical and practical value for improving the usability of remote sensing images and expanding the application scope of remote sensing technology. Summary of the Invention

[0009] The purpose of this invention is to provide a method for removing clouds and fog from time-series remote sensing images based on multi-index fusion, which can accurately identify and remove thick clouds, thin clouds, and cloud shadows in time-series remote sensing images by relying solely on the RGB band.

[0010] This invention is achieved through the following technical solution: A method for removing clouds and fog from time-series remote sensing images based on multi-index fusion includes the following steps: Step 1: Collect time-series RGB remote sensing images of the target area and perform geometric registration and radiometric normalization. Then, select the time-series RGB remote sensing image with the highest transmittance as the reference base map based on the dark channel prior method. Step 2: First, construct a multi-dimensional quality assessment system for a single image, including 7 quality indicators: whiteness index, thin cloud index, blue light shift index, brightness anomaly index, temporal stability index, clear sky line index, and color consistency index; then normalize each indicator, and store the normalized indicators as a four-bit array. Step 3: Using a layered contamination detection strategy, identify three types of contamination, including thick clouds, thin clouds, and cloud shadows, generate a contamination mask, and calculate pixel-level quality scores. Step 4: Calculate the similarity matrix between images to identify clean image groups and remove severely contaminated images; Step 5: First, calculate the pixel-level quality score of each pixel location across all time series, then calculate the temporal ranking of the image at that pixel location. Next, generate pixel-level confidence based on the quality score and temporal ranking, use guided filtering for spatial smoothing, implement adaptive Top-K weighted fusion, and perform global tone matching and edge feathering to output cloudless image, pixel-level confidence map, and pollution type distribution map.

[0011] Furthermore, the specific process of step 1 is as follows: Step 1.1: Collect time-series RGB remote sensing images of the target area, including the number of images. Furthermore, the time span of the temporal RGB remote sensing images covers different seasons and weather conditions; Step 1.2: Perform geometric registration on all time-series RGB remote sensing images to achieve pixel-level alignment, with the registration error controlled within 1 pixel; Step 1.3: Perform radiometric normalization on all temporal RGB remote sensing images, unifying the luminance value DN value range to [0,1] or [0,255], thereby eliminating radiometric differences between different time series; Step 1.4: Calculate the coordinates of each temporal RGB remote sensing image based on the dark channel prior method. Transmittance at , is represented as: In the formula: For coordinates A local window centered on the user; Atmospheric light value, The retention factor is set to 0.95. , representing the RGB three channels; Indicates in Intra-image in coordinates Color channels Pixel values; Step 1.5: Select the time-series RGB remote sensing image with the highest transmittance as the reference base map.

[0012] Furthermore, the specific process of step 2 is as follows: Step 2.1: Construct the whiteness index The RGB values ​​of the thick cloud are all high and close, therefore Value close to 1, low Corresponding features and shadows, Represented as: In the formula: This represents the minimum value of the three RGB channels; This represents the average value of the three RGB channels; ; Step 2.2: Constructing the Thin Cloud Index Thin clouds increase the brightness of the image but reduce color differences. The value increases, Represented as: In the formula: , and These represent the values ​​of the three RGB channels respectively; Step 2.3: Construct the blue light shift index When the blue light is strongly scattered in the cloud shadow area, it appears in a cool tone. The value is significantly higher than the normal range. Represented as: Step 2.4: Construct the Clear Sky Line Index Based on the principle of clear sky lines, the more cloud and fog pixels deviate from the clear sky lines, the... The larger the value, Represented as: Step 2.5: Construct a brightness anomaly index Clouds increase the brightness of an image, while shadows decrease its brightness. For adaptive detection of brightness anomalies, it is represented as: In the formula: This represents the median of the time-series brightness. The current time-series brightness is represented as: Step 2.6: Construct the color consistency index If clouds and fog cause the RGB channel values ​​to converge, then... The smaller the value, the greater the color difference of the ground features. The larger the value, Represented as: In the formula: Indicates the image in the color channel Pixel values; The average pixel value of an image is represented as: Step 2.7: Construct the temporal stability index Clouds, thin clouds, and shadows are all time-series anomalies. A higher value indicates that the pixel deviates from the normal timing pattern. Represented as: In the formula: This represents the median of the absolute time series deviation. The current timing deviation is represented as: In the formula: Indicates the current image in the color channel Pixel values ​​on; Indicates the image in the color channel Pixel median; Step 2.8: Normalize each indicator constructed in Steps 2.1 to 2.7 and scale it uniformly to the [0,1] interval; Step 2.9: Store each normalized indicator as... A four-dimensional array, wherein: It is a time series number; Image size.

[0013] Furthermore, the specific process of step 3 is as follows: Step 3.1: Based on the differences in spectral characteristics of different pollution types, a stratified detection strategy is adopted, wherein: 3.1.1) The detection strategy for thick clouds is expressed as follows: Where: threshold ; This represents the pixel-level quality score of thick clouds; Indicates a thick cloud contamination mask; 3.1.2) The detection strategy for thin clouds is expressed as follows: Where: threshold ; This represents the pixel-level quality score of thin clouds; Indicates a thin cloud pollution mask; 3.1.3) The cloud shadow detection strategy is expressed as follows: In the formula: This represents the pixel-level quality score of cloud shadows; surface Showing cloud shadows contaminating the mask; 3.1.4) Cloud contamination mask is represented as: Step 3.2: Calculate pixel position Basic quality score Then multiply by a penalty correction factor to obtain the pixel position. Pixel-level quality score , is represented as: Then, Normalize to the [0,1] interval.

[0014] Furthermore, the specific process of step 4 is as follows: Step 4.1: Calculate image pairs Weighted pixel difference , is represented as: In the formula: Image size; Image In coordinates Color channels Pixel values; Image In coordinates Color channels Pixel values; The weight is represented as: In the formula: and Representing images and images Pixel-level quality score; Next, calculate the image. and images Similarity matrix , is represented as: Step 4.2: Identify clean image groups and remove severely contaminated images. The specific process is as follows: First, calculate the average similarity of each image. , is represented as: like Then the first Images marked as severely polluted will be removed during subsequent fusion, including: and These are the mean and standard deviation, respectively.

[0015] Furthermore, the specific process of step 5 is as follows: Step 5.1: First, count the position of each pixel. Pixel-level quality scores across all time series , is represented as: In the formula: It is a time series number; Next, calculate the first... The ranking of the image at this pixel position ,Will Sort by size from smallest to largest; the worst value is 1, and the best value is... Then the time series confidence level Represented as: In the formula: Indicates pixel position In the Pixel-level quality scores for each time series ; Step 5.2: Smooth the confidence plot using guided filtering, as shown below: In the formula: For guided filtering; It is a grayscale image; The filter radius is... ; For regularization parameters, ; Step 5.3: Perform weighted adaptive fusion for each pixel position. Choose the one with the highest confidence level. A single image is represented as: In the formula: Represents the position of each pixel Sort all time series confidence scores; Then, the normalized weights are expressed as: In the formula: Indicates the first The image at pixel location Time series confidence at the point Therefore, the fusion result is expressed as: In the formula: Indicates the image's pixel position First Color channels in time sequence Pixel values ​​on; Step 5.4: Perform global tone matching. Using the reference base map as a benchmark, perform histogram matching on the fusion results using the cumulative distribution function. The tone matching result of the image will then be obtained. , is represented as: In the formula: , representing the RGB three channels; Indicates channel The fusion result; and Both are cumulative distribution functions. Used to calculate fusion results Histogram of distribution; Used to map a square diagram to a reference base map; Step 5.5: Perform edge feathering processing. In regions where the absolute value of the temporal confidence gradient is greater than a threshold, use... The Gaussian filter smoothing of the splicing marks is represented as follows: In the formula: Represents the absolute value of the time series confidence gradient; Step 5.6: Output cloudless image Pixel-level confidence map And pollution type distribution map, including pixel-level confidence map. Represented as: In the formula: This represents the maximum value of the time series confidence score.

[0016] The present invention has the following beneficial technical effects: First, this invention constructs a multi-dimensional evaluation system comprising seven quality indicators from three dimensions: spectral characteristics, temporal characteristics, and spatial characteristics. It overcomes the limitations of existing methods that rely on a single evaluation indicator by employing a hierarchical detection strategy to apply differentiated weight combinations to different pollution types. This facilitates the accurate identification and removal of three types of pollution: thick clouds, thin clouds, and cloud shadows. Specifically: 1) For thick clouds, a whiteness index and a HOT index are designed, focusing on whiteness and color consistency in thick cloud detection; 2) For thin clouds, a thin cloud index based on the ratio of brightness to color difference is designed, focusing on the ratio of brightness to color in thin cloud detection, effectively identifying semi-transparent thin clouds; 3) For cloud shadows, a blue light shift index is designed, focusing on blue light shift in shadow detection. Utilizing the strong blue light scattering characteristic of shadow areas, it accurately distinguishes cloud shadows from normal dark areas (such as water bodies and roads); 4) A temporal stability index is introduced, enabling the algorithm to automatically identify temporal anomalies. Even in the absence of significant spectral characteristic differences, cloud pollution can be detected through temporal comparison, facilitating subsequent removal.

[0017] Secondly, in order to overcome the problem that traditional fixed threshold methods may refuse to fuse or produce holes when a certain pixel is contaminated in all time sequences, this invention adopts a confidence calculation method based on time sequence ranking. This method comprehensively considers the absolute quality and relative ranking of pixels, and can automatically select the relatively optimal time sequence for fusion, thereby improving the integrity of the output image. Moreover, the weighted fusion strategy avoids the mean effect caused by simple averaging. By selecting only the K images with the highest confidence, it not only improves the fusion quality but also preserves the image details.

[0018] Third, this invention relies only on the RGB bands, and all quality indicators can be directly calculated from RGB images without the need for special bands such as near-infrared and short-wave infrared. This makes the invention widely applicable to various remote sensing data sources, including high-resolution satellite images (such as Gaofen-2 and WorldView), medium-resolution satellite images (such as the true-color band of Sentinel-2), aerial images, and UAV images. In addition, compared with methods that rely on deep learning, this invention uses traditional image processing techniques, which do not require a large number of labeled samples and GPU computing power, nor do they require model training and parameter tuning. This makes it easy to deploy and apply directly, significantly reducing the barrier to entry and computational costs.

[0019] Fourth, this invention not only outputs cloudless images, but also simultaneously outputs pixel-level confidence maps and contamination type distribution maps. The confidence map visually displays the reliability of each pixel in the fusion result, allowing users to assess the quality of the results. For low-confidence areas (such as areas contaminated by clouds in all time series), targeted processing or manual inspection can be performed. The contamination type distribution map identifies the spatial distribution of thick clouds, thin clouds, and cloud shadows in the original image, facilitating users to analyze cloud contamination characteristics and verify the cloud removal effect. This traceable and evaluable design enables transparent and reliable processing of time-series remote sensing images. Attached Figure Description

[0020] Figure 1 This is a flowchart of the present invention; Figure 2 This is a schematic diagram of the quality index distribution box curve of the present invention; Figure 3 These are multiple remote sensing images of cloud-covered areas in region L15-1558E-1203N, as per the present invention. Figure 4 This is a diagram showing the calculation results of remote sensing imagery in the region numbered L15-1558E-1203N, based on the present invention. Figure 5 This is the result of cloud and fog removal from remote sensing images of the region numbered L15-1558E-1203N, based on the present invention. Detailed Implementation

[0021] The present invention will be further described in detail below with reference to specific embodiments. These descriptions are for explanation purposes only and are not intended to limit the scope of the invention.

[0022] like Figure 1 As shown, a method for removing clouds and fog from time-series remote sensing images based on multi-index fusion includes the following steps: Step 1, Data Accuracy and Preprocessing: Collect time-series RGB remote sensing images of the target area and perform geometric registration and radiometric normalization. Then, based on the dark channel prior method, select the time-series RGB remote sensing image with the highest transmittance as the reference base map. The specific process is as follows: Step 1.1: Collect time-series RGB remote sensing images of the target area, including the number of images. Furthermore, the time span of the temporal RGB remote sensing images covers different seasons and weather conditions; Step 1.2: Perform geometric registration on all time-series RGB remote sensing images to achieve pixel-level alignment, with the registration error controlled within 1 pixel; Step 1.3: Perform radiometric normalization on all temporal RGB remote sensing images, unifying the luminance value DN value range to [0,1] or [0,255], thereby eliminating radiometric differences between different time series; Step 1.4: Calculate the coordinates of each temporal RGB remote sensing image based on the dark channel prior method. Transmittance at , is represented as: In the formula: For coordinates A local window centered on the user; Atmospheric light value, The retention factor is set to 0.95. , representing the RGB three channels; Indicates in Intra-image in coordinates Color channels Pixel values; Step 1.5: Select the time-series RGB remote sensing image with the highest average transmittance as the reference base map; Step 2, Multidimensional Data Quality Assessment: First, construct a multidimensional quality assessment system for a single image, such as... Figure 2 As shown, there are seven quality indicators: whiteness index, thin cloud index, blue light shift index, brightness anomaly index, temporal stability index, clear sky line index, and color consistency index. Each quality indicator is then normalized, and the normalized indicators are stored as a four-bit array. The specific process is as follows: Step 2.1: Construct the whiteness index The RGB values ​​of the thick cloud are all high and close, therefore Value close to 1, low Corresponding features and shadows, Represented as: In the formula: This represents the minimum value of the three RGB channels; This represents the average value of the three RGB channels; ; Step 2.2: Constructing the Thin Cloud Index Thin clouds increase the brightness of the image but reduce color differences. The value increases, Represented as: In the formula: , and These represent the values ​​of the three RGB channels respectively; Step 2.3: Construct the blue light shift index When the blue light is strongly scattered in the cloud shadow area, it appears in a cool tone. The value is significantly higher than the normal range. Represented as: Step 2.4: Construct the Clear Sky Line Index Based on the principle of clear sky lines, the more cloud and fog pixels deviate from the clear sky lines, the... The larger the value, Represented as: Step 2.5: Construct a brightness anomaly index Clouds increase the brightness of an image, while shadows decrease its brightness. For adaptive detection of brightness anomalies, it is represented as: In the formula: This represents the median of the time-series brightness. The current time-series brightness is represented as: Step 2.6: Construct the color consistency index If clouds and fog cause the RGB channel values ​​to converge, then... The smaller the value, the greater the color difference of the ground features. The larger the value, Represented as: In the formula: Indicates the image in the color channel Pixel values; The average pixel value of an image is represented as: Step 2.7: Construct the temporal stability index Clouds, thin clouds, and shadows are all time-series anomalies. A higher value indicates that the pixel deviates from the normal timing pattern. Represented as: In the formula: This represents the median of the absolute time series deviation. The current timing deviation is represented as: In the formula: Indicates the current image in the color channel Pixel values ​​on; Indicates the image in the color channel Pixel median; Step 2.8: Normalize each indicator constructed in Steps 2.1 to 2.7 and scale it uniformly to the [0,1] interval; Step 2.9: Store each normalized indicator as... A four-dimensional array, wherein: It is a time series number; Image size; Step 3, Data Layered Contamination Detection: A layered contamination detection strategy is adopted to identify three types of contamination, including thick clouds, thin clouds, and cloud shadows, generate contamination masks, and calculate pixel-level quality scores. The specific process is as follows: Step 3.1: Based on the differences in spectral characteristics of different pollution types, a stratified detection strategy is adopted, wherein: 3.1.1) The detection strategy for thick clouds is expressed as follows: Where: threshold ; This represents the pixel-level quality score of thick clouds; Indicates a thick cloud contamination mask; 3.1.2) The detection strategy for thin clouds is expressed as follows: Where: threshold ; This represents the pixel-level quality score of thin clouds; Indicates a thin cloud pollution mask; 3.1.3) The cloud shadow detection strategy is expressed as follows: In the formula: This represents the pixel-level quality score of cloud shadows; surface Showing cloud shadows contaminating the mask; 3.1.4) Cloud contamination mask is represented as: Step 3.2: Calculate pixel position Basic quality score Then multiply by a penalty correction factor to obtain the pixel position. Pixel-level quality score , is represented as: Then, Normalize to the [0,1] interval; Step 4: Image Similarity Analysis: Calculate the similarity matrix between images to identify clean image groups and remove severely contaminated images. The specific process is as follows: Step 4.1: Calculate image pairs Weighted pixel difference , is represented as: In the formula: Image size; Image In coordinates Color channels Pixel values; Image In coordinates Color channels Pixel values; The weight is represented as: In the formula: and Representing images and images Pixel-level quality score; Next, calculate the image. and images Similarity matrix , is represented as: Step 4.2: Identify clean image groups and remove severely contaminated images. The specific process is as follows: First, calculate the average similarity of each image. , is represented as: like Then the first Images marked as severely polluted will be removed during subsequent fusion, including: and These are the mean and standard deviation, respectively. Step 5: Cloud and fog removal results based on pixel quality: First, calculate the pixel-level quality score of each pixel location across all time series. Then, calculate the temporal ranking of the image at that pixel location. Next, generate pixel-level confidence scores based on the quality scores and temporal rankings. Guided filtering is used for spatial smoothing, adaptive Top-K weighted fusion is implemented, and global tone matching and edge feathering are performed. The output includes a cloud-free image, a pixel-level confidence map, and a pollution type distribution map. The specific process is as follows: Step 5.1: First, count the position of each pixel. Pixel-level quality scores across all time series , is represented as: In the formula: It is a time series number; Next, calculate the first... The ranking of the image at this pixel position ,Will Sort by size from smallest to largest; the worst value is 1, and the best value is... Then the time series confidence level Represented as: In the formula: Indicates pixel position In the Pixel-level quality scores for each time series ; Step 5.2: Smooth the confidence plot using guided filtering, as shown below: In the formula: For guided filtering; It is a grayscale image; The filter radius is... ; For regularization parameters, ; Step 5.3: Perform weighted adaptive fusion for each pixel position. Choose the one with the highest confidence level. A single image is represented as: In the formula: Represents the position of each pixel Sort all time series confidence scores; Then, the normalized weights are expressed as: In the formula: Indicates the first The image at pixel location Time series confidence at the point Therefore, the fusion result is expressed as: In the formula: Indicates the image's pixel position First Color channels in time sequence Pixel values ​​on; Step 5.4: Perform global tone matching. Using the reference base map as a benchmark, perform histogram matching on the fusion results using the cumulative distribution function. The tone matching result of the image will then be obtained. , is represented as: In the formula: , representing the RGB three channels; Indicates channel The fusion result; and Both are cumulative distribution functions. Used to calculate fusion results Histogram of distribution; Used to map a square diagram to a reference base map; Step 5.5: Perform edge feathering processing. In regions where the absolute value of the temporal confidence gradient is greater than a threshold, use... The Gaussian filter smoothing of the splicing marks is represented as follows: In the formula: Represents the absolute value of the time series confidence gradient; Step 5.6: Output cloudless image Pixel-level confidence map And pollution type distribution map, including pixel-level confidence map. Represented as: In the formula: This represents the maximum value of the time series confidence score.

[0023] Figure 2 The quality index information of some remote sensing images in the L15-1558E-1203N region is shown. It can be seen that there are differences between the various indicators, indicating that there is significant complementarity between the images. Figure 3Multiple remote sensing images of the L15-1558E-1203N region are presented. Each image contains cloud and fog information, and there is information complementarity between the images. The complementarity between the images lays the foundation for the multi-index fusion time-series remote sensing image cloud and fog removal method proposed in this embodiment. Based on the three RGB bands, the three types of cloud pollution can be accurately identified and removed by existing image processing technology.

[0024] Figure 4 The study demonstrates seven index parameters calculated based on the first scene of remote sensing imagery in the L15-1558E-1203N region. Seven matrices are generated for each image to evaluate the reliability of pixels. This approach does not require a large number of labeled samples or GPU computing power, which helps to reduce the barrier to entry and computational costs.

[0025] Figure 5 This demonstrates how cloudless image information can be obtained by sorting the index parameters of remote sensing images in the L15-1558E-1203N region and using pixel iterative replacement.

Claims

1. A method for removing clouds and fog from time-series remote sensing images based on multi-index fusion, characterized in that, Includes the following steps: Step 1: Collect time-series RGB remote sensing images of the target area and perform geometric registration and radiometric normalization. Then, select the time-series RGB remote sensing image with the highest transmittance as the reference base map based on the dark channel prior method. Step 2: First, construct a multi-dimensional quality assessment system for a single image, including 7 quality indicators: whiteness index, thin cloud index, blue light shift index, brightness anomaly index, temporal stability index, clear sky line index, and color consistency index; then, normalize each quality indicator, and store the normalized indicators as a four-bit array. Step 3: Using a layered contamination detection strategy, identify three types of contamination, including thick clouds, thin clouds, and cloud shadows, generate a contamination mask, and calculate pixel-level quality scores. Step 4: Calculate the similarity matrix between images to identify clean image groups and remove severely contaminated images; Step 5: First, calculate the pixel-level quality score of each pixel location across all time series, then calculate the temporal ranking of the image at that pixel location. Next, generate pixel-level confidence based on the quality score and temporal ranking, use guided filtering for spatial smoothing, implement adaptive Top-K weighted fusion, and perform global tone matching and edge feathering to output cloudless image, pixel-level confidence map, and pollution type distribution map.

2. The method for removing clouds and fog from time-series remote sensing images based on multi-index fusion according to claim 1, characterized in that, The specific process of step 1 is as follows: Step 1.1: Collect time-series RGB remote sensing images of the target area, including the number of images. Furthermore, the time span of the temporal RGB remote sensing images covers different seasons and weather conditions; Step 1.2: Perform geometric registration on all time-series RGB remote sensing images to achieve pixel-level alignment, with the registration error controlled within 1 pixel; Step 1.3: Perform radiometric normalization on all temporal RGB remote sensing images, unifying the luminance value DN value range to [0,1] or [0,255], thereby eliminating radiometric differences between different time series; Step 1.4: Calculate the coordinates of each temporal RGB remote sensing image based on the dark channel prior method. Transmittance at , represented as: In the formula: For coordinates A local window centered on the user; Atmospheric light value, The retention factor is set to 0.

95. , representing the RGB three channels; Indicates in Intra-image in coordinates Color channels Pixel values; Step 1.5: Select the time-series RGB remote sensing image with the highest transmittance as the reference base map.

3. The method for removing clouds and fog from time-series remote sensing images based on multi-index fusion according to claim 1, characterized in that, The specific process of step 2 is as follows: Step 2.1: Construct the whiteness index The RGB values ​​of the thick cloud are all high and close, therefore Value close to 1, low Corresponding features and shadows, Represented as: In the formula: This represents the minimum value of the three RGB channels; This represents the average value of the three RGB channels; ; Step 2.2: Constructing the Thin Cloud Index Thin clouds increase the brightness of the image but reduce color differences. The value increases, Represented as: In the formula: , and These represent the values ​​of the three RGB channels respectively; Step 2.3: Construct the blue light shift index When the blue light is strongly scattered in the cloud shadow area, it appears in a cool tone. The value is significantly higher than the normal range. Represented as: Step 2.4: Construct the Clear Sky Line Index Based on the principle of clear sky lines, the more cloud and fog pixels deviate from the clear sky lines, the... The larger the value, Represented as: Step 2.5: Construct a brightness anomaly index Clouds increase the brightness of an image, while shadows decrease its brightness. For adaptive detection of brightness anomalies, it is represented as: In the formula: This represents the median of the time-series brightness. The current time-series brightness is represented as: Step 2.6: Construct the color consistency index If clouds and fog cause the RGB channel values ​​to converge, then... The smaller the value, the greater the color difference of the ground features. The larger the value, Represented as: In the formula: Indicates the image in the color channel Pixel values; The average pixel value of an image is represented as: Step 2.7: Construct the temporal stability index Clouds, thin clouds, and shadows are all time-series anomalies. A higher value indicates that the pixel deviates from the normal timing pattern. Represented as: In the formula: This represents the median of the absolute time series deviation. The current timing deviation is represented as: In the formula: Indicates the current image in the color channel Pixel values ​​on; Indicates the image in the color channel Median pixel value; Step 2.8: Normalize each indicator constructed in Steps 2.1 to 2.7 and scale it uniformly to the [0,1] interval; Step 2.9: Store each normalized indicator as... A four-dimensional array, wherein: It is a time series number; Image size.

4. The method for removing clouds and fog from time-series remote sensing images based on multi-index fusion according to claim 3, characterized in that, The specific process of step 3 is as follows: Step 3.1: Based on the differences in spectral characteristics among different types of pollution, a stratified detection strategy is adopted. in: 3.1.1) The detection strategy for thick clouds is expressed as follows: Where: threshold ; This represents the pixel-level quality score of thick clouds; Indicates a thick cloud pollution mask; 3.1.2) The detection strategy for thin clouds is expressed as follows: Where: threshold ; This represents the pixel-level quality score of thin clouds; Indicates a thin cloud pollution mask; 3.1.3) The cloud shadow detection strategy is expressed as follows: In the formula: This represents the pixel-level quality score of cloud shadows; Indicates cloud Shadow pollution mask; 3.1.4) Cloud contamination mask is represented as: Step 3.2: Calculate pixel position Basic quality score Then multiply by a penalty correction factor to obtain the pixel position. pixel-level quality score , represented as: Then, Normalize to the [0,1] interval.

5. The method for removing clouds and fog from time-series remote sensing images based on multi-index fusion according to claim 4, characterized in that, The specific process of step 4 is as follows: Step 4.1: Calculate image pairs Weighted pixel difference , represented as: In the formula: Image size; Image In coordinates Color channels Pixel values; Image In coordinates Color channels Pixel values; The weight is represented as: In the formula: and Representing images and images Pixel-level quality score; Next, calculate the image. and images Similarity matrix , represented as: Step 4.2: Identify clean image groups and remove severely contaminated images. The specific process is as follows: First, calculate the average similarity of each image. , represented as: like Then the first Images marked as severely polluted will be removed during subsequent fusion, including: and These are the mean and standard deviation, respectively.

6. The method for removing clouds and fog from time-series remote sensing images based on multi-index fusion according to claim 1, characterized in that, The specific process of step 5 is as follows: Step 5.1: First, count the position of each pixel. Pixel-level quality scores across all time series , represented as: In the formula: It is a time series number; Next, calculate the first... The ranking of the image at this pixel position ,Will Sort by size from smallest to largest; the worst value is 1, and the best value is... Then the time series confidence level Represented as: In the formula: Indicates pixel position In the Pixel-level quality scores for each time series ; Step 5.2: Smooth the confidence plot using guided filtering, as shown below: In the formula: For guided filtering; It is a grayscale image; The filter radius is... ; For regularization parameters, ; Step 5.3: Perform weighted adaptive fusion for each pixel position. Choose the one with the highest confidence level. A single image is represented as: In the formula: Represents the position of each pixel Sort all time series confidence scores; Then, the normalized weights are expressed as: In the formula: Indicates the first The image at pixel location Time series confidence at the point Therefore, the fusion result is expressed as: In the formula: Indicates the image at pixel position First Color channels in time sequence Pixel values ​​on; Step 5.4: Perform global tone matching. Using the reference base map as a benchmark, perform histogram matching on the fusion results using the cumulative distribution function. The tone matching result of the image will then be obtained. , represented as: In the formula: , representing the RGB three channels; Indicates channel The fusion result; and Both are cumulative distribution functions. Used to calculate fusion results Histogram of distribution; Used to map a square diagram to a reference base map; Step 5.5: Perform edge feathering processing. In regions where the absolute value of the temporal confidence gradient is greater than a threshold, use... The Gaussian filter smoothing of the splicing trace is represented as follows: In the formula: Represents the absolute value of the time series confidence gradient; Step 5.6: Output cloudless image Pixel-level confidence map And pollution type distribution map, including pixel-level confidence map. Represented as: In the formula: This represents the maximum value of the time series confidence level.

Images