A cloud removal method combining physical model guidance and Mamba

Abstract

The present invention relates to a cloud removal method combining physical model guidance and Mamba, and the method comprises the following steps: constructing an optical remote sensing cloud removal data set; dividing cloud distribution saliency map regions based on physical model guidance; constructing a cloud removal model based on Mamba and training model parameters; obtaining optical remote sensing image data with clouds to be processed; and verifying cloud removal performance. Compared with the prior art, the saliency map calculation is used to assist in analyzing the degree of interference of cloud cover remote sensing images by clouds and fog, and to guide the cloud removal model to make targeted parameter adjustments to enhance the model's sensitivity to information recovery in specific areas. At the same time, a cloud removal model is constructed based on a Mamba state space model with efficient global feature extraction capabilities to achieve recovery of missing information in thin cloud and thick cloud areas. In the case of coexistence of thin clouds and thick clouds, it is ensured that the cloud removal results are not over-corrected in the thin cloud area, and the thick cloud area is fully filled and optimized.

Description

Technical Field

[0001] The invention relates to the technical field of optical remote sensing image processing, and in particular to a cloud removal method combining physical model guidance and Mamba. Background Art

[0002] Optical remote sensing images have rich spectral information, strong interpretability and good visual effects, and have been widely used in environmental monitoring, urban planning, agricultural management and other fields. However, clouds in the atmosphere will prevent remote sensing satellite sensors from receiving electromagnetic waves reflected by surface objects, resulting in missing optical remote sensing image information, causing data failure and unable to meet the needs of subsequent applications. Therefore, in the process of optical remote sensing image data preprocessing, it is very important to remove clouds from the acquired optical remote sensing images, that is, to restore the missing information.

[0003] With the development of deep learning technology, cloud removal methods based on deep learning have become one of the important means to accurately restore surface information in the field of remote sensing image declouding. Existing cloud removal methods are often limited to the use of convolution operations, which makes it difficult to capture global information, which causes trouble for the recovery of missing information in cloud-covered areas. Although the Transformer structure is good at extracting global information, its computational complexity is high, resulting in insufficient cloud removal efficiency. Recently, methods based on the Mamba structure can efficiently achieve global information acquisition while maintaining linear complexity. Incorporating the Mamba structure into the cloud removal method can significantly improve the model's missing information capability and improve the quality of cloud removal.

[0004] At the same time, the state of clouds varies greatly, especially in complex situations where thin clouds and thick clouds coexist. The effect of cloud removal is often difficult to guarantee, and the reconstruction results may show problems such as over-correction of thin cloud areas and insufficient filling of thick cloud areas. In order to address these challenges, prior knowledge combined with physical models can be introduced to constrain the learning process of deep learning networks. This method not only provides theoretical support for deep learning, but also can better cope with different degrees of cloud interference, thereby improving the reliability of cloud removal effects. Summary of the invention

[0005] The purpose of the present invention is to provide a cloud removal method combining physical model guidance and Mamba to solve the problem that existing optical remote sensing image cloud removal methods are difficult to achieve efficient cloud removal in a complex environment where thin clouds and thick clouds coexist.

[0006] In order to achieve the above object, the technical solution of the present invention is as follows:

[0007] A cloud removal method combining physical model guidance and Mamba comprises the following steps:

[0008] 11) Constructing the dataset: Collect pairs of cloud-covered optical remote sensing images and cloud-free optical remote sensing images, and perform matching and radiation correction operations to eliminate the overall spectral differences between different images caused by the collection time; perform 50% overlap cropping on the collected large-size optical remote sensing satellite images and crop them into a fixed size of 256×256 to ensure the implementation effect and efficiency of the cloud removal method; select data from the three bands of red, green, and blue from the multispectral images for band synthesis. These four bands are common bands of existing optical remote sensing satellites to ensure that the cloud removal method is compatible with different optical remote sensing satellites; through manual screening, control the data ratio of completely cloud-free images, thin cloud images, thick cloud images, and images with coexistence of thin clouds and thick clouds in the cloud optical images. The ratio of the four types of images in the dataset is 1:2:2:5; The final dataset is divided into a training set and a validation set, with a data volume of 10,000 pairs and 2,000 pairs of 256×256-sized slices of cloud-covered and cloud-free images, respectively, to complete the dataset construction;

[0009] 12) Physical model-guided cloud distribution saliency map estimation: Different areas in a single image are affected differently by clouds and fog. Thin cloud-covered areas appear concentrated in large areas and dispersed in small areas on remote sensing images, and have no specific shape. Their hue is slightly higher than that of the surrounding cloudless objects. In most cases, thick cloud-covered areas have a smaller coverage area than thin clouds, and usually gather in a certain area in the shape of tufts, with a bright white hue. Therefore, based on the principle of visual saliency, saliency maps of remote sensing image intensity, color, and directional features are calculated to estimate the different degrees of interference of remote sensing images with clouds and fog. Based on the physical model of atmospheric radiation transmission, the cloud distribution is divided into thin cloud areas and thick cloud areas. This knowledge is used to assist the cloud removal model based on Mamba to achieve cloud removal.

[0010] 13) Construction of cloud removal model based on Mamba and training of model parameters: Mamba visual state space basic module is a basic deep learning network structure with global information extraction capability. It converts two-dimensional images into one-dimensional input sequences by scanning in four directions and uses a discretized iterative formula for modeling. Combining the advantages of Mamba visual state space basic module and the characteristics of different cloud distribution area divisions in images, a cloud removal model based on Mamba is constructed. The cloud optical remote sensing images in the training set are used as input, and the paired cloud-free optical remote sensing images are used as references to perform supervised iterative training on the model. The weights with the best cloud removal performance on the validation set are saved as parameters of the cloud removal model.

[0011] 14) Verification of cloud removal performance: Obtain the cloud-containing optical satellite image data to be processed, perform radiation correction, band synthesis and overlapping cropping, and then input the data into the trained model to finally obtain the cloud removal result.

[0012] The cloud distribution saliency map region division based on physical model guidance includes the following steps:

[0013] 21) Construct a Gaussian pyramid to obtain a saliency map of intensity, color and direction features;

[0014] 211) Calculate the intensity saliency map. A color image is composed of three components: red, green, and blue. The intensity map is realized by calculating the average value of the three components. Use the Gaussian pyramid to process the intensity map, obtain the pyramid features of the intensity map, and synthesize the intensity saliency map. The formula is as follows:

[0015] ,

[0016] in, Represents the intensity pyramid image, each level of the pyramid image is represented by To distinguish, Indicates that the intensity pyramid image has 9 levels. and are the c-th and s-th level intensity pyramid images respectively, , represents the superposition operation, which adds the multiple intensity feature maps selected by the c and k parameters pixel by pixel to obtain the average value; the above formula superimposes 6 intensity feature maps, Θ represents Zoom to After the size is , the two images are subtracted pixel by pixel, and then the absolute value is taken to generate an intensity feature map. Six intensity feature maps are superimposed and squared and normalized. Forming the final intensity saliency map ;

[0017] 212) Calculate the color saliency map. Before calculating the color feature map, calculate the maximum value in the intensity map. When a pixel in the intensity image is less than 1 / 10 of the maximum value, set the R, G, and B values ​​of the corresponding point in the color image to 0 and recreate the color component. The calculation formula is as follows:

[0018] ,

[0019] R, G, B represent the standardized color component matrix. Use Gaussian pyramid to process RR, GG, BB, Y respectively to form four image pyramids; , , , Represent the images in each pyramid respectively. Each image pyramid has 9 levels. The color feature map is calculated as follows:

[0020] ,

[0021] The above formula can generate 12 color feature maps, which are superimposed to form a color saliency map. , the calculation formula is as follows:

[0022] ;

[0023] 213) Calculate the directional saliency map and take the Gabor kernels in four directions, namely , use the Gabor kernel in one direction to process the intensity image and generate a Gabor pyramid in one direction. Four directions can generate four Gabor pyramids. represents the Gabor pyramid feature, It means that each image pyramid has 9 levels, and the calculation formula of the directional feature map is as follows:

[0024] ,

[0025] and Respectively indicate the direction The c-th and s-th level pyramid images in the pyramid are operated by the operator , generating a directional feature map , the directional saliency map is formed by superimposing 24 directional feature maps, and the calculation formula is as follows:

[0026] ;

[0027] 22) According to the atmospheric radiation transfer model, the transmittance represents the degree of surface information retention of the current pixel in the image. The lower the transmittance, the thicker the cloud layer and the more difficult it is to recover the surface information. The transmittance is estimated using the calculated saliency map of intensity, color and direction features. The thin cloud area and thick cloud area are divided by the transmittance weight, guiding the cloud removal network to perceive the difficulty of recovering surface information in different areas of the image;

[0028] 221) The atmospheric radiation transfer model is as follows:

[0029] ,

[0030] in, is a pixel position in the image, Represents cloud images, Represents a cloudless image. Represents the global atmospheric light value, which is set to the maximum pixel value in the current image slice. represents transmittance;

[0031] 222) Based on the atmospheric radiation transfer model, the relative transmittance of each pixel in the image is calculated using the intensity, color and directional saliency maps , the calculation formula is as follows:

[0032] ,

[0033] in, Represents the relative transmittance map of the current image, with a value range of [0,1]. Represents a correction term to filter outliers in the saliency map.

[0034] The cloud removal model construction and model parameter training based on Mamba include the following steps:

[0035] 31) By probabilistically mixing the relative transmittance map with the image to be de-clouded, different areas of the image to be de-clouded are divided into thin cloud areas and thick cloud areas according to the degree of cloud and fog interference;

[0036] 32) The Mamba cloud removal model consists of the Mamba thin cloud removal module and the Mamba thick cloud removal module. The two modules output the cloud removal results for thin cloud removal and thick cloud removal respectively. Finally, the two cloud removal results are fully mixed through the adaptive regional aggregation module.

[0037] 321) Construct a Mamba thin cloud removal module, which consists of four Mamba residual modules. The image after probability mixing is used as the module input. The input image passes through the four Mamba residual modules in sequence. The Mamba residual modules are used to obtain the global information of the image and perform information repair on the thin cloud coverage area to obtain the cloud removal result of removing thin clouds.

[0038] 3211) The Mamba residual module adopts a residual structure design. The entire structure is divided into two stages, consisting of a main branch and a residual branch. The interaction between the branches is realized by pixel-by-pixel summation. In the first stage, the main branch consists of a layer normalization operation and a visual state space module, and the residual branch consists of a 1×1 convolution operation. The result of the interaction between the two branches is summed pixel by pixel as the input of the second stage. In the second stage, the main branch consists of a layer normalization, a depth-separable convolution and a channel attention. The residual branch still uses a 1×1 convolution operation, and the results of the two branches are summed pixel by pixel to obtain the output.

[0039] 3212) The visual state space module in the Mamba residual module consists of a main branch and a residual branch, wherein the main branch consists of a 1×1 convolution operation, a depth-wise separable convolution operation, a SiLU activation operation, a two-dimensional selective scanning module and a layer normalization operation, and the residual branch consists of a 1×1 convolution operation and a SiLU activation operation, and the results of the two branches are aggregated by a pixel-by-pixel multiplication operation;

[0040] 3213) The core operation in the visual state space module is the two-dimensional selection scanning module. First, the two-dimensional feature map is divided and scanned along four different directions: from the upper left to the lower right, from the lower right to the upper left, from the upper right to the lower left, and from the lower left to the upper right. Each pixel can integrate information from all other pixels in different directions. Then each sequence is re-merged into a single feature map, and all images are integrated into a new output feature map. Before the sequence is re-merged, each sequence is calculated for the visual state space separately. The calculation formula is as follows:

[0041] ,

[0042] Among them, A, B, C, and D represent the four random initialization parameters in the current state space. represents the exponential function, represents the scaling operation of the state transition parameters, represents the hidden state variable at time t, t-1 represents the previous moment, Represents the feature map of the current moment in the state space, The output result of the feature map representing the current moment in the state space;

[0043] 322) Construct a Mamba thick cloud removal module, which consists of eight Mamba residual modules. The image after probability mixing is mixed with the cloud removal result of thin cloud removal and then used as the module input. The input image passes through eight Mamba residual modules in sequence. The Mamba residual modules are used to further restore the thick cloud coverage area and obtain the cloud removal result of thick cloud removal.

[0044] 323) Construct an adaptive regional aggregation module, and integrate the thin cloud removal results and the thick cloud removal results through the adaptive regional aggregation module; the module takes the thin cloud removal results and the thick cloud removal results as input, and the adaptive weight calculation branch first splices the two inputs in the channel dimension, performs spatial pooling on the splicing results, takes the average of the results obtained by the two pooling operations, and performs 1×1 convolution operation and sigmoid operation on the average to obtain the adaptive aggregation weight; the calculated weight is divided into two parts from the channel dimension, and the thin cloud removal results and the thick cloud removal results are respectively multiplied pixel by pixel to achieve weighted operation, and the two weighted results are finally aggregated by pixel by pixel summation operation; the aggregation result is restored to the same red, green and blue channels as the cloud image input through 1×1 convolution operation to obtain the final cloud removal image;

[0045] 33) Use the L1 loss function to supervise model training, calculate the loss between the cloud-removed image and the reference cloud-free image, calculate the gradient vector through the loss value, and update the model parameters through back propagation;

[0046] 34) Set the number of training rounds and finally save the model parameters with the best cloud removal effect in the verification set. Beneficial Effects

[0047] The present invention relates to a cloud removal method combined with physical model guidance and Mamba. Aiming at the problem that it is difficult to recover the effective missing information in the scene where thin clouds and thick clouds coexist in the existing remote sensing image processing cloud removal process, the present invention proposes a cloud removal method combined with physical model guidance and Mamba. Through physical model guidance, the thin cloud area and thick cloud area in the optical remote sensing image are probabilistically divided, and a cloud removal model is built using the basic results of the Mamba state space, which significantly improves the cloud removal performance. For complex cloud scenes, it is ensured that the cloud removal result is not over-corrected in the thin cloud area, and the thick cloud area is fully filled and optimized, and finally a cloud removal result with color fidelity, clear details and high contrast is obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 is a method sequence diagram of the present invention;

[0049] Figure 2 This is the overall flow chart involved in the present invention;

[0050] Figure 3 This is a diagram of a regional cloud removal structure involved in the present invention;

[0051] Figure 4 This is a module structure diagram of the Mamba cloud removal model involved in the present invention;

[0052] Figure 5 Resulting image for cloud removal. DETAILED DESCRIPTION

[0053] In order to have a further understanding and recognition of the structural features and the effects achieved by the present invention, a preferred embodiment and accompanying drawings are used for detailed description as follows:

[0054] like Figure 1 As shown, a cloud removal method combining physical model guidance and Mamba according to the present invention comprises the following steps:

[0055] The first step is to build an optical remote sensing image dataset. The specific steps are as follows:

[0056] (1) Considering the diversity of land surface types, we selected paired optical images of four common land feature types, including rivers, mountains, cities, and farmlands, and performed registration and radiometric correction operations on the cloud-covered optical satellite images and the corresponding cloud-free optical images;

[0057] (2) Only the red, green and blue bands are selected for band synthesis to synthesize color images; the cloud optical remote sensing satellite images and cloud-free optical remote sensing satellite images are cropped with 50% overlap and cropped to a fixed size of 256×256;

[0058] (3) The cropped slice images were manually selected to control the data ratio of completely cloud-free images, thin cloud images, thick cloud images, and images with both thin and thick clouds in the cloud optical images. The ratio of the four types of images in the dataset was 1:2:2:5 to form the dataset. The dataset was divided into a training set and a validation set, with data volumes of 10,000 and 2,000, respectively. Each set contained optical remote sensing images with clouds and optical remote sensing images without clouds, with a size of 256 × 256.

[0059] The second step is to divide the cloud distribution saliency map into regions based on the physical model. Different regions in the cloud optical remote sensing image are affected differently by clouds and fog. The thin cloud coverage area and the thick cloud coverage area have different performances, so the regional division needs to be combined with prior knowledge. Figure 2 As shown in the figure, firstly, the saliency map of remote sensing image intensity, color and direction characteristics is calculated based on the principle of visual saliency, and the different degrees of interference of remote sensing images by clouds and fog are estimated; and the probability division of different regions is realized based on the physical model. The specific steps are as follows:

[0060] (1) Visual saliency analysis: Figure 2 As shown, visual saliency analysis consists of three features: intensity, color, and direction;

[0061] (1-1) Calculation of intensity saliency map: A color image is composed of three components: red, green, and blue. The intensity map is realized by calculating the average value of the three components. The intensity map is processed using a Gaussian pyramid to obtain the pyramid features of the intensity map and synthesize the intensity saliency map. The formula is as follows:

[0062] ,

[0063] in represents the intensity pyramid image, Used to represent different levels of pyramid images, It means that the intensity map obtained after Gaussian pyramid processing has 9 levels. , and are the c-th level and s-th level intensity pyramid images respectively. Θ means: Zoom to After the size is adjusted, the corresponding pixels of the two images are subtracted, and then the absolute value is taken to generate the intensity feature map. The superposition operation means adding the multiple intensity feature maps selected by the c and k parameters pixel by pixel to obtain the average value. The above formula forms the final intensity saliency map by superimposing 6 intensity feature maps and performing N() square normalization operation. ;

[0064] (1-2) Color saliency map calculation: Calculate the color saliency map. Before calculating the color feature map, calculate the maximum value in the intensity map. When a pixel in the intensity image is less than 1 / 10 of the maximum value, set the R, G, and B values ​​of the corresponding point in the color image to 0 and recreate the color component. The calculation formula is as follows:

[0065] ,

[0066] R, G, B represent the standardized color component matrix. Use the Gaussian pyramid to process RR, GG, BB, and Y respectively to form four image pyramids: , , , , It means that the images in each pyramid are of 9 levels, and the color feature map is calculated as follows:

[0067] ,

[0068] The above formula can generate 12 color feature maps, which are superimposed to form a color saliency map. , the calculation formula is as follows:

[0069] ;

[0070] (1-3) Directional saliency map calculation: To calculate the directional saliency map, take the Gabor kernels in four directions, namely , use the Gabor kernel in one direction to process the intensity image and generate a Gabor pyramid in one direction. Four directions can generate four Gabor pyramids. , It means that the Gabor pyramid is divided into 9 levels of features in each method. Similarly, the calculation formula of the directional feature map is as follows:

[0071] ,

[0072] and Respectively indicate the direction The c-th and s-th level pyramid images in the pyramid are operated by the operator , generating a directional feature map , the directional saliency map is formed by superimposing 24 directional feature maps, and the calculation formula is as follows:

[0073] ;

[0074] (2) Probabilistic regional mixing based on physical models uses the calculated intensity, color and direction feature saliency maps to estimate the transmittance, divides thin cloud areas and thick cloud areas by transmittance weights, and guides the cloud removal network to perceive the difficulty of restoring surface information in different areas of the image. Based on the atmospheric radiation transfer model, the relative transmittance of each pixel in the image is calculated based on the calculated intensity, color and direction saliency maps. , the calculation formula is as follows:

[0075] ,

[0076] in, Represents the relative transmittance map of the current image, with a value range of [0,1]. Represents a correction term to filter outliers in the saliency map.

[0077] In the third step, the specific steps of constructing the cloud removal model and training the model parameters based on Mamba are as follows:

[0078] (1) Introducing regional guidance knowledge, such as Figure 3 As shown, the thin cloud area is processed first, and the relative transmittance map is probabilistically mixed with the input image to be declouded to achieve the difference in the degree of interference of clouds and fog in different areas of the image. The mixed result is input into the Mamba thin cloud removal module to obtain the cloud removal result for thin clouds;

[0079] (2) Obtain the cloud removal results for thin clouds and construct the Mamba thin cloud removal module, such as Figure 3 As shown in the figure, the Mamba thin cloud removal module is composed of four Mamba residual modules. The image after probability mixing is used as the module input. The input image passes through the four Mamba residual modules in sequence. The Mamba residual modules are used to obtain the global information of the image and perform information repair on the thin cloud coverage area to obtain the cloud removal result of removing thin clouds.

[0080] (2-1) If Figure 4 As shown in the figure, the Mamba residual module adopts a residual structure design. The whole structure is divided into two stages, consisting of a main branch and a residual branch. The interaction between the branches is realized by pixel-by-pixel summation. In the first stage, the main branch consists of a layer normalization operation and a visual state space module, and the residual branch consists of a 1×1 convolution operation. The result of the interaction between the two branches is summed pixel by pixel as the input of the second stage. In the second stage, the main branch consists of a layer normalization, a depth-separable convolution and a channel attention. The residual branch still uses a 1×1 convolution operation. The results of the two branches are summed pixel by pixel as the output of the Mamba residual module.

[0081] (2-2) If Figure 4 As shown in the figure, the visual state space module in the Mamba residual module is also composed of a main branch and a residual branch, where the main branch consists of a 1×1 convolution operation, a depth-separable convolution operation, a SiLU activation operation, a two-dimensional selective scanning module and a layer normalization operation, and the residual branch consists of a 1×1 convolution operation and a SiLU activation operation. The results of the two branches are aggregated by pixel-by-pixel multiplication;

[0082] (2-3) If Figure 4 As shown in the figure, the core operation in the visual state space module is a two-dimensional selection scanning module. By scanning in four different directions: from top left to bottom right, from bottom right to top left, from top right to bottom left, and from bottom left to top right, each pixel can integrate information from all other pixels in different directions, and then re-merge each sequence into a single feature map, and integrate all images into a new output feature map; before re-merging the sequences, each sequence is calculated for the visual state space separately, and the calculation formula is as follows:

[0083] ,

[0084] Among them, A, B, C, and D represent the four random initialization parameters in the current state space. represents the exponential function, represents the scaling operation of the state transition parameters, represents the hidden state variable at time t, t-1 represents the previous moment, Represents the feature map of the current moment in the state space, The output result of the feature map representing the current moment in the state space;

[0085] (3) Obtain cloud removal results for thick clouds;

[0086] (3-1) If Figure 3 As shown, compared with removing clouds from thin cloud areas, by probabilistically mixing the relative transmittance map with the input image to be declouded, the mixture before repairing the thick cloud area will be added with the result after thin cloud restoration, which can obtain more information about the cloud-free area;

[0087] (3-2) Constructing a Mamba thick cloud removal module, which consists of eight Mamba residual modules. The thick cloud area probability mixture combined with the cloud removal result of thin cloud removal is used as the module input. The input image passes through the eight Mamba residual modules in sequence. The thick cloud area is further restored by the Mamba residual modules to obtain the cloud removal result of thick cloud removal.

[0088] (4) Mix the two cloud removal results and construct an adaptive regional aggregation module to integrate the thin cloud removal results and the thick cloud removal results; Figure 4As shown in the figure, the adaptive weight calculation branch first concatenates the two inputs in the channel dimension, performs spatial pooling on the concatenation results, takes the average of the results obtained by the two pooling operations, and performs 1×1 convolution and sigmoid operations on the average to obtain the adaptive aggregation weight; the calculated weight is divided into two parts from the channel dimension, and the thin cloud removal results and the thick cloud removal results are respectively multiplied pixel by pixel to achieve weighted operation, and the two weighted results are finally aggregated by pixel by pixel summation operation to achieve the aggregation of the two modes; the aggregation result is restored to the same red, green and blue channels as the input through a 1×1 convolution operation to obtain the final cloud removal image;

[0089] (5) Supervised training and weight preservation: Use the L1 loss function to supervise model training, calculate the loss between the cloud-removed image and the reference cloud-free image, and use the loss value to calculate the gradient vector and back-propagate to update the model parameters; iterate the training and save the model parameters with the best cloud removal effect in the validation set.

[0090] The fourth step is to acquire and process cloud optical remote sensing image data.

[0091] The fifth step is to import the trained model parameters into the cloud removal model based on Mamba, input the pre-processed optical remote sensing image data to be de-clouded, and obtain the cloud removal results after model processing, such as Figure 5 As shown in Figure 3, our method is able to obtain cloud removal results with preserved colors, sharp details and high contrast.

[0092] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions only describe the principles of the present invention. The present invention may be subject to various changes and improvements without departing from the spirit and scope of the present invention. These changes and improvements fall within the scope of the present invention. The scope of protection claimed by the present invention is defined by the attached claims and their equivalents.

Claims

1. A cloud removal method combining physical model guidance and Mamba, characterized in that: The following steps are involved: 11) Construct an optical remote sensing cloud removal dataset: collect pairs of cloud-covered optical remote sensing images and cloud-free optical remote sensing images, and perform matching and radiation correction operations to eliminate spectral differences between different images; perform 50% overlap cropping on the collected large-size optical remote sensing satellite images and crop them into a fixed size of 256×256; select three bands of red, green, and blue from multispectral images for data synthesis; through manual visual screening, control the data ratio of completely cloud-free images, thin cloud images, thick cloud images, and images with coexistence of thin clouds and thick clouds in the cloud optical images, and the ratio of the four types of images in the dataset is 1:2:2:5; the final dataset is divided into a training set and a validation set, with a data volume of 10,000 and 2,000 pairs of 256×256-sized slices of cloud-covered and cloud-free images, respectively, to complete the dataset construction; 12) Regional division of cloud distribution saliency map based on physical model guidance: Based on the principle of visual saliency, saliency maps of remote sensing image intensity, color and direction characteristics are calculated to evaluate the degree of interference of remote sensing images by clouds and fog. Regional division is performed according to the optical thickness of clouds based on the physical model of atmospheric radiation transmission. This knowledge is used to assist the cloud removal model based on Mamba to achieve cloud removal. 13) Construction of cloud removal model based on Mamba and training of model parameters: The Mamba visual state space basic module uses four-directional scanning to convert two-dimensional images into one-dimensional input sequences, and uses the discretized state space iterative formula for modeling; Combining the advantages of the Mamba visual state space basic module and the characteristics of the division of different cloud distribution areas in the image, a cloud removal model based on Mamba is constructed. The cloud optical remote sensing images in the training set are used as input, and the paired cloud-free optical remote sensing images are used as references to perform supervised iterative training on the model. The weights with the best cloud removal performance on the validation set are saved as parameters of the cloud removal model; 14) Verification of cloud removal performance: Obtain the cloud-containing optical satellite image data to be processed, perform radiation correction, band synthesis and cropping, and then input the trained Mamba-based cloud removal model to finally obtain the cloud removal result.

2. The cloud removal method combining physical model guidance and Mamba according to claim 1, characterized in that: The cloud distribution saliency map region division based on physical model guidance includes the following steps: 21) Use saliency calculation to assist in analyzing the degree of cloud cover remote sensing images affected by fog and clouds, and construct Gaussian pyramids to obtain saliency maps of intensity, color and direction characteristics, providing data input for subsequent calculation of regional separation weight maps in combination with physical models; 22) Calculation of the intensity, color and orientation feature significance of the input image; 221) First, the intensity saliency map is calculated. A color image is composed of three components: red, green, and blue. The intensity map is realized by calculating the average value of the three components. The intensity map is processed using a Gaussian pyramid to obtain the pyramid features of the intensity map and synthesize the intensity saliency map. The formula is as follows: , in, Represents the intensity pyramid image, each level of the pyramid image is represented by To distinguish, Indicates that the intensity pyramid image has 9 levels. and are the c-th and s-th level intensity pyramid images respectively, , represents the superposition operation, Θ represents the Zoom to After the size is , the two images are subtracted pixel by pixel, and then the absolute value is taken to generate an intensity feature map. Six intensity feature maps are superimposed and squared and normalized. Forming the final intensity saliency map ; 222) Secondly, calculate the color saliency map. Before calculating the color feature map, calculate the maximum value in the intensity map. When a pixel in the intensity image is less than 1 / 10 of the maximum value, set the R, G, and B values ​​of the corresponding point in the color image to 0 and recreate the color component. The calculation formula is as follows: , R, G, B represent the standardized color component matrix. Use Gaussian pyramid to process RR, GG, BB, and Y respectively to form four image pyramids. , , , Represent the images in each pyramid respectively, It means that each image pyramid consists of 9 levels of images, and the color feature map is calculated as follows: , The above formula can generate 12 color feature maps, which are superimposed to form a color saliency map. , the calculation formula is as follows: ; 223) Finally, calculate the directional saliency map and take the Gabor kernels in four directions, namely , each time a Gabor kernel in one direction is used to process the intensity image, a Gabor pyramid in the corresponding direction is generated. Four directions can generate four Gabor pyramids. represents the Gabor pyramid feature, , indicating that each The directional image pyramid has 9 levels, and the calculation formula of the directional feature map is as follows: , in, and Respectively indicate the direction The c-th and s-th level pyramid images in the pyramid are operated by the operator , generating a directional feature map , the directional saliency map is formed by superimposing 24 directional feature maps, and the calculation formula is as follows: ; 23) Saliency analysis combined with physical models to achieve regional separation; From the atmospheric radiation transfer model, it can be seen that the transmittance represents the degree of surface information retention of the current pixel in the image. The lower the transmittance, the thicker the cloud layer and the more difficult it is to recover the surface information. The transmittance is calculated using the saliency map of intensity, color and direction characteristics, and the thin cloud area and thick cloud area are divided by the transmittance weight, guiding the cloud removal network to perceive the difficulty of recovering surface information in different areas of the image; 231) The atmospheric radiation transfer model is as follows: , in, is a pixel position in the image, Represents cloud images, Represents a cloudless image. Represents the global atmospheric light value, which is set to the maximum pixel value in the current image slice. represents transmittance; 232) Based on the atmospheric radiation transfer model, the relative transmittance of each pixel of the image is calculated according to the intensity, color and direction saliency map calculated in step 22) , the calculation formula is as follows: , in, Represents the relative transmittance map of the current image, with a value range of [0,1]. Represents a correction term used to filter outliers.

3. The cloud removal method combining physical model guidance and Mamba according to claim 1, characterized in that: The cloud removal model construction and model parameter training based on Mamba include the following steps: 31) Introducing regional guidance knowledge, that is, by probabilistically mixing the relative transmittance map with the image to be declouded, different areas of the image to be declouded are divided into thin cloud areas and thick cloud areas according to the degree of cloud and fog interference; 32) The cloud removal model based on Mamba is composed of Mamba thin cloud removal module and Mamba thick cloud removal module. The two modules output the cloud removal results for thin clouds and thick clouds respectively. Finally, the two cloud removal results are mixed through the adaptive regional aggregation module. 321) Construct a Mamba thin cloud removal module, which consists of four Mamba residual modules. The image after probability mixing is used as the module input. The input image passes through the four Mamba residual modules in sequence. The Mamba residual modules are used to obtain the global information of the image and perform information repair on the thin cloud coverage area to obtain the cloud removal result of removing thin clouds. 3211) The Mamba residual module adopts a residual structure design. The entire structure is divided into two stages, consisting of a main branch and a residual branch. The interaction between the branches is realized by pixel-by-pixel summation. In the first stage, the main branch consists of a layer normalization operation and a visual state space module, and the residual branch consists of a 1×1 convolution operation. The result of the interaction between the two branches is summed pixel by pixel as the input of the second stage; in the second stage, the main branch consists of a layer normalization, a depth-separable convolution and a channel attention, and the residual branch still uses a 1×1 convolution operation. The results of the two branches are summed pixel by pixel as the output of the Mamba residual module. 3212) The visual state space module in the Mamba residual module also consists of a main branch and a residual branch, where the main branch consists of a 1×1 convolution operation, a depth-separable convolution operation, a SiLU activation operation, a two-dimensional selective scanning module and a layer normalization operation, and the residual branch consists of a 1×1 convolution operation and a SiLU activation operation. The results of the two branches are aggregated through a pixel-by-pixel multiplication operation; 3213) The core operation in the visual state space module is a two-dimensional selection scanning module. By scanning along four different directions: from top left to bottom right, from bottom right to top left, from top right to bottom left, and from bottom left to top right, each pixel can integrate information from all other pixels in different directions, and then re-merge the four sequences one by one into a single feature map, and integrate all images into a new output feature map; before re-merging the sequences, each sequence is calculated for the visual state space separately, and the calculation formula is as follows: , Among them, A, B, C, and D represent the four random initialization parameters in the current state space. represents the exponential function, represents the scaling operation of the state transition parameters, represents the hidden state variable at time t, t-1 represents the previous moment, Represents the feature map of the current moment in the state space, The output result of the feature map representing the current moment in the state space; 322) constructing a Mamba thick cloud removal module, the Mamba thick cloud removal module is composed of eight Mamba residual modules, and the Mamba residual module is constructed and implemented by step 3211) to step 3213); the image after probability mixing is mixed with the cloud removal result of removing thin clouds and used as the module input, the input image passes through the eight Mamba residual modules in turn, and the thick cloud coverage area is restored by the Mamba residual modules to obtain the cloud removal result of removing thick clouds; 323) Construct an adaptive regional aggregation module to integrate the thin cloud removal results and the thick cloud removal results through the adaptive regional aggregation module, which takes the thin cloud removal results and the thick cloud removal results as input; 3231) The adaptive weight calculation branch first concatenates the two inputs in the channel dimension, performs maximum pooling and average pooling on the concatenated results, takes the average of the results obtained by the two pooling operations, and performs a 1×1 convolution operation and a sigmoid operation on the average to obtain the adaptive aggregation weight; 3232) The calculated weights are evenly divided from the channel dimension, and the thin cloud removal results and the thick cloud removal results are multiplied pixel by pixel to achieve weighted operation. The two weighted results are aggregated by pixel summation operation. The aggregated results are restored to the same red, green and blue channels as the input through 1×1 convolution operation to obtain the final cloud removal image; 33) Supervised training and weight preservation: Use the L1 loss function to supervise model training, calculate the loss between the cloud-removed image and the reference cloud-free image, calculate the gradient vector through the loss value, and update the model parameters through back propagation; finally save the model parameters with the best cloud removal effect in the verification set.

Images