Remote sensing image cloud removal method based on microwave-guided diffusion model

Abstract

The invention provides a remote sensing image cloud removal method based on a microwave-guided diffusion model. The method comprises the following steps: S1, establishing a cloud body mask; S2, analyzing and extracting multiband microwave features; S3, training and generating a visible light image by using the microwave-guided diffusion model; according to the method, principal component analysis is carried out on multiband microwave data to extract significant features, the significant features are input into a diffusion model, meanwhile, ControlNet is introduced, a conditional branch network is added into the diffusion model through the ControlNet, additional conditional information (such as a cloud mask, multiband microwave principal component features or edge features and the like) is embedded into a specific stage of the diffusion process, and the diffusion efficiency is improved. The method achieves the precise control of a generation result, further improves the performance of a diffusion model in a remote sensing image cloud removal task, achieves the high-quality image generation and the precise recovery of earth surface information, and better retains the texture and detail information of ground features.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image processing, and in particular to a remote sensing image cloud removal method based on a microwave guided diffusion model. Background Art

[0002] Optical remote sensing images are widely used in land planning, natural resource monitoring, disaster response and other fields due to their advantages of strong timeliness, wide coverage and obvious visual features. However, cloud cover is the main challenge in the application of optical remote sensing images, which makes it difficult to extract ground information and significantly reduces the utilization value of images. Synthetic aperture radar (SAR), as a microwave active earth observation system, can obtain high-resolution radar images in low-visibility weather, is almost unaffected by clouds and meteorological conditions, and realizes all-weather observation, making up for the problem of missing data of optical images under complex meteorological conditions.

[0003] Although the existing microwave-assisted cloud removal methods based on deep learning have made some progress, they still have many shortcomings, such as insufficient SAR feature extraction, difficulty in cross-modal feature fusion, poor model interpretability, limited multi-source data fusion capabilities, and insufficient utilization of time series or multi-angle observation data. These problems make it difficult to meet the needs of actual deployment and large-scale applications. Summary of the invention

[0004] The purpose of the present invention is to provide a remote sensing image declouding method based on a microwave-guided diffusion model. By performing principal component analysis on multi-band microwave data to extract significant features and inputting them into the diffusion model, ControlNet is introduced. A conditional branch network is added to the diffusion model using ControlNet, and additional conditional information (such as cloud masks, multi-band microwave principal component features or edge features, etc.) is embedded into a specific stage of the diffusion process to achieve precise control of the generated results, thereby further improving the performance of the diffusion model in the remote sensing image declouding task, achieving high-quality image generation and accurate restoration of surface information, and better retaining the texture and detail information of the objects.

[0005] To achieve the above object, the technical solution adopted by the present invention is: A remote sensing image cloud removal method based on a microwave guided diffusion model comprises the following steps: S1. Establish cloud mask: Use the maximum likelihood classification method to establish classification training samples of clouds and shadows, and obtain the cloud mask map; S2. Analyze and extract multi-band microwave features: S21, expanding the pixel values ​​of the two microwave images taken at the same position into a matrix form, and performing standardization processing on each column of data so that the mean is 0 and the variance is 1; S22, calculating the covariance matrix of the standardized matrix, and performing eigenvalue decomposition on it to obtain two corresponding eigenvectors; S23, select the eigenvector corresponding to the maximum eigenvalue, project the original data to the direction of the first principal component PC1, and reshape the projection result into an image form to replace the two microwave images in step S21; wherein the eigenvectors are as follows starting from the original data: , , ,... ; S3. Generate visible light images using microwave-guided diffusion model training: S31, forward diffusion: from the original data The process of transforming complex real data distribution into a standard Gaussian distribution by gradually adding noise to the data. Specifically, forward diffusion starts with the original data (such as an image) and introduces a small amount of random noise at each step. The amplitude of this noise increases with the increase of time steps, gradually destroying the structure and content of the original data. Forward diffusion from the original data Initially, noise is introduced at each time step t : , in, is the original data, is the result of forward diffusion after time t, is the noise attenuation coefficient, which determines the ratio of data to noise in each step. is the standard Gaussian distribution noise; In this process, each moment t is only related to moment t-1, which can be expressed as a Markov process: , As t increases, is getting closer and closer to pure noise; when , is pure Gaussian noise; S32, Reverse Diffusion: From Random Noise To begin, follow the diffusion steps The order of removing noise step by step; at each time step , the diffusion model will be based on the current data Predicting noise components , and use the estimated result of the noise to update the data , thereby approximating the target data distribution; wherein the diffusion model is constructed by a U-Net neural network, and the network architecture of U-Net includes a ControlNet network and a Stable Diffusion network; from arrive In the process, each step is modeled by the U-Net neural network, which learns the noise distribution in the forward diffusion process and learns how to recover the result closer to the original data from the noise data at any time point. The main calculation formula is: , in, is the predicted mean, is the predicted variance, is the noise attenuation coefficient in the forward diffusion process, is the cumulative noise attenuation coefficient, is the noise predicted by the neural network; The learning method of the U-Net network includes training the ControlNet network and training the Stable Diffusion network; the ControlNet network copies the encoder of the U-Net network into its own network and trains it together with the U-Net network through zero convolution without interference from the generation process; the specific steps are: S321, additional condition input: The microwave image is input into the ControlNet network as an additional condition, and the weight is initialized to zero through zero convolution to ensure that the initial impact of the condition input is controllable and does not interfere with the basic network; S322, conditional guided encoding: The ControlNet network combines the microwave image, the noise image and the time step encoding and passes them to the encoder module; this part consists of multiple Encoder Blocks, each of which includes the following operations: Convolution operation: extract input features and compress spatial resolution; Normalization: use normalization to standardize feature distribution; Activation function: use GELU function to increase nonlinearity; Downsampling: reduce the size of the feature map through convolution, and change the input map from 64×64 to 32×32, 16×16, 8×8 in sequence; When training ControlNet, the weights of the U-Net network of Stable Diffusion are frozen to maintain the generation ability of the original model; S323, intermediate feature processing: At each scale, the output of the ControlNet network is fused into the encoder features of the main U-Net through the SkipConnection module, and then a multi-scale feature map that incorporates the conditional input information is output; S324, conditional decoding: The decoder upsamples the low-resolution features through deconvolution; each decoding block increases the feature resolution from 8×8 to 16×16, then to 32×32, and finally restores it to the original resolution of 64×64; at the same time, the Skip Connection module is used to retain high-resolution local detail information; After these four steps, the arrive prediction, and gradually restore it to the original image; S325. Loss function: At a given time step t, the text prompt and microwave images , through learning network , used to predict the value added to the noisy image The noise on the , the loss function is expressed as follows: , is the overall learning objective of the entire diffusion model, which can be directly used to fine-tune the diffusion model.

[0006] S33, style transfer: Image style transfer based on optimal transmission transfers the color style of the target domain visible light image space to the source domain, where the image space is composed of the numerical information of the image plane and the color channel; the specific steps are: Assume that the color distribution of the source domain and target domain images are and ,in, represents Gaussian distribution; optimal transmission is determined and A closed-form mapping between , in is the transmission matrix, and Represent the source domain and target domain samples respectively; determine the optimal mapping through optimal transmission To minimize the distance between the source domain and the target domain distribution, that is: , in, represents the cost function; the optimal transmission matrix is: .

[0007] S4. Replace the cloud image with the microwave generated image: Use the ControlNet network to generate a visible light image from the microwave band, and use the mask image established in step S1 to replace the cloud image with the generated part, thereby completing the de-clouding work.

[0008] Compared with the prior art, the present invention has the following advantages or beneficial effects: 1. This invention proposes for the first time a method combining microwaves and diffusion models to achieve cloud removal. By utilizing the characteristics of microwaves penetrating clouds, the interference of clouds on surface observations is eliminated. At the same time, combined with the generation capability of the diffusion model, the microwave characteristics of the surface can be restored almost realistically.

[0009] 2. In order to further extract the microwave characteristics of the surface, principal component analysis (PCA) is used to perform spatial dimension reduction on multi-band data, which effectively improves the generation quality and efficiency of the diffusion model. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0011] Figure 1 This is a flowchart of the remote sensing image cloud removal method based on the microwave guided diffusion model; Figure 2 This is a schematic diagram of the diffusion model training process; Figure 3 This is a diagram of ControlNet network training; Figure 4 Generate result graphs for C, Ka, and PC1; Figure 5 The figure shows the comparison of the results when microwave guidance is used and when not. DETAILED DESCRIPTION

[0012] The technical solutions in the embodiments of the present invention will be described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0013] like Figure 1 As shown, the present invention provides a remote sensing image declouding method based on a microwave guided diffusion model. When there are clouds and fog in the remote sensing satellite image, the corresponding microwave band is input into the latent space as a condition, and the diffusion model is used to learn the denoising method from each time point to the next time point in the reverse diffusion process, thereby achieving the use of microwave data to guide the model to obtain a remote sensing image close to the real one from a noise map, and replace the part of the original image with clouds and fog, thereby completing the declouding work. In addition, in order to better extract the data features of microwaves, the present invention uses principal component analysis to aggregate the features of the microwave C band and Ka band into the first principal component, and uses the first principal component to guide the removal of clouds in the visible light band. The specific steps are as follows: S1. Create cloud mask: The maximum likelihood classification method is used to establish classification training samples for clouds and shadows, and obtain the cloud mask map.

[0014] S2. Analyze and extract multi-band microwave features: S21. Expand the pixel values ​​of the two microwave images taken at the same position into a matrix form, and standardize each column of data so that the mean is 0 and the variance is 1.

[0015] S22. Calculate the covariance matrix of the standardized matrix, and perform eigenvalue decomposition on it to obtain two corresponding eigenvectors.

[0016] S23, select the eigenvector corresponding to the maximum eigenvalue, project the original data to the direction of the first principal component PC1, and reshape the projection result into an image form to replace the two microwave images in step S21; wherein the eigenvectors are as follows starting from the original data: , , ,... .

[0017] S3, using microwaves to guide diffusion model training (such as Figure 2 ) to generate a visible light image: S31, forward diffusion: from the original data The forward diffusion is a process that transforms the complex real data distribution into a standard Gaussian distribution by gradually adding noise to the data; specifically, forward diffusion starts with the original data (such as an image) and introduces a small amount of random noise in each step. The amplitude of this noise increases with the increase of time steps, gradually destroying the structure and content of the original data.

[0018] Forward diffusion from the original data Initially, noise is introduced at each time step t : , in, is the original data, is the result of forward diffusion after time t, is the noise attenuation coefficient, which determines the ratio of data to noise in each step. is the standard Gaussian distributed noise.

[0019] In this process, each moment t is only related to moment t-1, which can be expressed as a Markov process: , As t increases, is getting closer and closer to pure noise; when , is pure Gaussian noise.

[0020] S32, Reverse Diffusion: From Random Noise To begin, follow the diffusion steps The order of removing noise step by step; at each time step , the diffusion model will be based on the current data Predicting noise components , and use the estimated result of the noise to update the data , thereby approximating the target data distribution; wherein the diffusion model is constructed by a U-Net neural network, and the network architecture of U-Net includes a ControlNet network and a Stable Diffusion network.

[0021] from arrive In the process, each step is modeled by the U-Net neural network, which learns the noise distribution in the forward diffusion process and learns how to recover the result closer to the original data from the noise data at any time point. The main calculation formula is: , in, is the predicted mean, is the predicted variance, is the noise attenuation coefficient in the forward diffusion process, is the cumulative noise attenuation coefficient, is the noise predicted by the neural network.

[0022] The learning method of the U-Net network includes training the ControlNet network and training the Stable Diffusion network; Figure 3 As shown in the figure, the ControlNet network copies the encoder of the U-Net network into its own network and trains it together with the U-Net network through zero convolution without interfering with the generation process; the specific steps are:

[0023] S321, additional condition input: The microwave image is input into the ControlNet network as an additional condition, and the weight is initialized to zero through zero convolution to ensure that the initial impact of the conditional input is controllable and does not interfere with the basic network.

[0024] S322, conditional guidance coding: The ControlNet network combines microwave images, noise images, and time step encodings and passes them to the encoder module; this part consists of multiple Encoder Blocks, each of which includes the following operations: Convolution operation: extract input features and compress spatial resolution; Normalization: use normalization to standardize feature distribution; Activation function: use GELU function to increase nonlinearity; Downsampling: reduce the size of feature maps through convolution, and change the input map from 64×64 to 32×32, 16×16, and 8×8 in sequence; When training ControlNet, the weights of the U-Net network of Stable Diffusion are frozen to maintain the generation ability of the original model.

[0025] S323, intermediate feature processing: At each scale, the output of the ControlNet network is fused into the encoder features of the main U-Net through the Skip Connection module, and then a multi-scale feature map that incorporates the conditional input information is output.

[0026] S324, conditional decoding: The decoder upsamples low-resolution features through deconvolution; each decoding block increases the feature resolution from 8×8 to 16×16, then to 32×32, and finally restores it to the original resolution of 64×64; at the same time, the SkipConnection module is used to retain high-resolution local detail information.

[0027] After these four steps, the arrive prediction and gradually restore it to the original image.

[0028] S325, loss function: At a given time step t, the text prompt and microwave images , through learning network , used to predict the value added to the noisy image The noise on the , the loss function is expressed as follows: , is the overall learning objective of the entire diffusion model, which can be directly used to fine-tune the diffusion model.

[0029] S33, style transfer: Image style transfer based on optimal transmission transfers the color style of the target domain visible light image space to the source domain, where the image space is composed of the numerical information of the image plane and the color channel; the specific steps are: Assume that the color distribution of the source domain and target domain images are and ,in, represents Gaussian distribution; optimal transmission is determined and A closed-form mapping between ,Right now: , in is the transmission matrix, and Represent the source domain and target domain samples respectively; determine the optimal mapping through optimal transmission To minimize the distance between the source domain and the target domain distribution, that is: , in, represents the cost function; the optimal transmission matrix is: , The specific steps for optimal transmission of image space are as follows: (1) Statistical analysis of the color space distribution histograms of the source domain and target domain images to obtain the color distribution parameters of the source domain and target domain , , , ; (2) Calculate the optimal transmission matrix ; (3) For source domain images Convert the source domain image to the target domain color style .

[0030] S4. Replace the cloud map with the microwave generated map:

[0031] Using the ControlNet network, the microwave band is generated into a visible light image, and the mask image established in step S1 is used to replace the cloud image with the generated part, thereby completing the cloud removal work.

[0032] The remote sensing image cloud removal method provided in the above embodiment is used to carry out the following experiment: 1. Effectiveness of main component analysis and extraction: Table 1 Comparison of C, Ka, and PC1 band indicators index C Ka PCA (C / Ka) SSIM 0.515494667 0.506486111 0.557158667 PSNR 17.56272222 16.46173333 18.38193333

[0033] The above three groups of comparative experiments are all based on the results of running after a training step of 10,000 steps to ensure that the indicator results are comparable.

[0034] From the data in the table, it can be seen that the PCA (C / Ka) method performs better than the data fusion using C-band or Ka-band alone in terms of indicators. Specifically, the SSIM value of the PCA method is 0.5572, which is significantly higher than 0.5155 of the C-band and 0.5065 of the Ka-band, indicating that the structural similarity of the image generated after PCA fusion is higher and can more accurately reflect the structural characteristics of the original image. In addition, the PSNR value of PCA is 18.3819, which is also higher than 17.5627 of the C-band and 16.4617 of the Ka-band, indicating that the image fused by the PCA method has a higher signal-to-noise ratio, a lower noise level, and better image quality. These results show that the PCA fusion method significantly improves the clarity and detail fidelity of the generated image by effectively integrating the feature information of the C-band and Ka-band, and is a more superior data fusion method.

[0035] Combination Figure 4 It can be seen that the advantages of principal component generation in denoising and detail enhancement are particularly significant: the outline of the building is clear, the texture of the vegetation area is rich, and the boundary of the object is accurate, while the C-band and Ka-band are weak in detail preservation, with obvious noise or blurred areas. This effect shows that the principal component generation method not only improves the image quality by fusing the information of the C-band and Ka-band, but also retains richer spatial and spectral features, and its comprehensive performance is significantly better than the generation result of a single band.

[0036] 2. Comparison of the effect of microwave diffusion generation with and without introduction: Table 2 Whether to use microwave guidance to generate images index no yes SSIM 0.34224225 0.6345115 PSNR 15.511325 20.730675

[0037] The above two groups of experiments are the results of running after a training step of 40,000 steps to ensure that the indicator results are comparable.

[0038] As can be seen from the table, after using microwave guidance, SSIM increased from 0.3422 to 0.6345, and PSNR increased from 15.5113 to 20.7307, indicating that microwave guidance has significantly improved the quality of the generated results. The improvement of SSIM reflects the enhancement of the structural similarity of the generated image, indicating that microwave guidance can better capture the detailed information and texture features of the image; while the improvement of PSNR indicates that the noise level of the generated image is reduced and the signal quality is significantly improved. This shows that the introduction of microwave guidance can not only enhance the ability to restore the image content, but also effectively improve the clarity and authenticity of the generated results, further verifying the important role of microwave guidance in multi-source data fusion.

[0039] from Figure 5It can be seen that in the case of cloud occlusion, the non-microwave guidance and microwave guidance methods show significant differences in eliminating cloud interference. Although the images generated by the non-microwave guidance method can partially restore the ground information, there are still obvious afterimages or information loss in the cloud area, especially in complex terrain and building areas, and the detail restoration is not accurate enough. The microwave guidance method shows a better cloud removal ability, and the generated image has richer detail information in the cloud area and clearer ground features, especially in areas such as woodlands, buildings and roads. The restoration effect is significantly better than that of non-microwave guidance. This improvement in effect shows that microwave guidance can effectively utilize the ability of microwave data to penetrate clouds, make up for the problem of information loss of optical images under cloud occlusion, and significantly improve the image restoration quality and environmental perception ability. At the same time, after introducing the style transfer method, the visual consistency and detail performance of the generated image are further enhanced. Style transfer can effectively combine the structural characteristics of microwave images and the color information of optical images to achieve more natural texture generation and ground restoration in cloud occluded areas. Especially in complex scenes (such as dense vegetation, urban buildings and crisscrossing roads), the style transfer method can more accurately capture the spatial relationship of ground features. The generated images not only have a high visual realism in the macro sense, but also retain rich texture features in the micro details.

[0040] The present embodiment provides a remote sensing image cloud removal method based on a microwave-guided diffusion model, comprising: obtaining a cloud mask: using a maximum likelihood classification method to classify and identify cloud areas in optical remote sensing images, thereby generating a cloud mask; accurately segmenting the cloud area in the image through the spectral characteristics of the optical image, combined with the prior probability distribution and intra-class variance of the object category; the cloud mask marks the part of the image covered by clouds, and provides an accurate area range for subsequent microwave guidance and image replacement; establishing a data set corresponding to microwaves and visible light, and establishing a diffusion model generated by microwave-guided visible light: collecting remote sensing data with both microwave images and optical images, establishing a paired training data set, ensuring the consistency of microwave images and optical images at the pixel level through spatial registration and time synchronization, and training a diffusion model with microwave images as conditional input based on the data set, so that the model can learn from microwave images. The mapping relationship between microwave images and high-quality visible light images is used. The generated images have both the structural characteristics of microwave images and the texture characteristics of optical images. During the training process, a style transfer module is introduced to extract the texture style characteristics of microwave images through a pre-trained convolutional neural network, and combined with the content characteristics of optical images to ensure the texture consistency and visual quality of the generated images in cloud-occluded areas; Replacement and restoration of cloud-occluded areas: The generated cloud-free optical image will contain complete ground object information, but only the cloud-occluded area is the part that needs to be restored. The cloud mask generated in the first step is used to accurately locate the cloud-covered area in the optical image, and the cloud-free part of the corresponding position of the generated image is cropped out. These cropped parts contain the ground object details in the cloud-covered area, and are generated after the structure is guided by the microwave image, with high-quality texture details and authenticity. After cropping, they are replaced with the corresponding cloud area in the original optical image.

[0041] Experimental results show that compared with traditional cloud removal methods, the microwave-guided combined with style transfer model has significant advantages in ground object restoration accuracy, image continuity and overall style consistency.

[0042] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based device that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

Claims

1. A remote sensing image cloud removal method based on a microwave guided diffusion model, characterized in that: The following steps are involved: S1. Create cloud mask: The maximum likelihood classification method is used to classify and identify cloud areas in optical remote sensing images, thereby generating a cloud mask map. S2. Analyze and extract multi-band microwave features: S21, expanding the pixel values ​​of the two microwave images taken at the same position into a matrix form, and performing standardization processing on each column of data so that the mean is 0 and the variance is 1; S22, calculating the covariance matrix of the standardized matrix, and performing eigenvalue decomposition on it to obtain two corresponding eigenvectors; S23, select the eigenvector corresponding to the maximum eigenvalue, project the original data to the direction of the first principal component PC1, and reshape the projection result into an image form to replace the two microwave images in step S21; wherein the eigenvectors are as follows starting from the original data: , , ,... ; S3. Generate visible light images using microwave-guided diffusion model training: S31, forward diffusion: from the original data At the beginning, the process of transforming the complex real data distribution into a standard Gaussian distribution by gradually adding noise to the data; S32, Reverse Diffusion: From Random Noise To begin, follow the diffusion steps The order of removing noise step by step; at each time step , the diffusion model will be based on the current data Predicting noise components , and use the estimated result of the noise to update the data , thereby approximating the target data distribution; wherein the diffusion model is constructed by a U-Net neural network, and the network architecture of U-Net includes a ControlNet network and a Stable Diffusion network; S33, style transfer: image style transfer based on optimal transmission transfers the color style of the target domain visible light image space to the source domain, where the image space consists of the image plane and the numerical information of the color channel; S4. Replace the cloud map with the microwave generated map: Using the ControlNet network, the microwave band is generated into a visible light image, and the mask image established in step S1 is used to replace the cloud image with the generated part, thereby completing the cloud removal work.

2. The remote sensing image cloud removal method based on microwave guided diffusion model according to claim 1 is characterized in that: The specific steps of S31 are: Forward diffusion from the original data Initially, noise is introduced at each time step t : ， in, is the original data, is the result of forward diffusion after time t, is the noise attenuation coefficient, which determines the ratio of data to noise in each step. is the standard Gaussian distributed noise.

3. The remote sensing image cloud removal method based on microwave guided diffusion model according to claim 2 is characterized in that: In the process of forward diffusion, each moment t is only related to moment t-1, which can be expressed as a Markov process: ， As t increases, is getting closer and closer to pure noise; when , is pure Gaussian noise.

4. The remote sensing image cloud removal method based on microwave guided diffusion model according to claim 1 is characterized in that: The specific steps of S32 are: from arrive In the process, each step is modeled by the U-Net neural network, which learns the noise distribution in the forward diffusion process and learns how to recover the result closer to the original data from the noise data at any time point. The main calculation formula is: in, is the predicted mean, is the predicted variance, is the noise attenuation coefficient in the forward diffusion process, is the cumulative noise attenuation coefficient, is the noise predicted by the neural network.

5. The remote sensing image cloud removal method based on microwave guided diffusion model according to claim 4 is characterized in that: The learning method of the U-Net network includes training the ControlNet network and the Stable Diffusion network; the ControlNet network copies the encoder of the U-Net network into its own network and trains it together with the U-Net network through zero convolution without interference from the generation process; the specific steps are: S321, additional condition input: The microwave image is input into the ControlNet network as an additional condition, and the weight is initialized to zero through zero convolution to ensure that the initial impact of the condition input is controllable and does not interfere with the basic network; S322, conditional guided encoding: The ControlNet network combines the microwave image, the noise image and the time step encoding and passes them to the encoder module; this part consists of multiple Encoder Blocks, each of which includes the following operations: Convolution operation: extract input features and compress spatial resolution; Normalization: Use normalization to standardize the feature distribution; activation Function: Use GELU function to increase nonlinearity; Downsampling: Reduce the size of feature maps through convolution, and change the input map from 64×64 to 32×32, 16×16, and 8×8 in sequence; When training ControlNet, the weights of the U-Net network of Stable Diffusion are frozen to maintain the generation ability of the original model; S323, intermediate feature processing: At each scale, the output of the ControlNet network is fused into the encoder features of the main U-Net through the Skip Connection module, and then a multi-scale feature map that incorporates the conditional input information is output; S324, conditional decoding: the decoder upsamples the low-resolution features through deconvolution; Each decoding block increases the feature resolution from 8×8 to 16×16, then to 32×32, and finally restores it to the original resolution of 64×64. At the same time, the SkipConnection module is used to retain high-resolution local detail information. S325. Loss function: At a given time step t, the text prompt and microwave images , through learning network , used to predict the value added to the noisy image The noise on the , the loss function is expressed as follows: ， is the overall learning objective of the entire diffusion model, which can be directly used to fine-tune the diffusion model.

6. The remote sensing image cloud removal method based on microwave guided diffusion model according to claim 1, characterized in that: The specific steps of S33 are: Assume that the color distribution of the source domain and target domain images are and ,in, represents Gaussian distribution; optimal transmission is determined and A closed-form mapping between ,Right now: ， in is the transmission matrix, and Represent the source domain and target domain samples respectively; determine the optimal mapping through optimal transmission To minimize the distance between the source domain and the target domain distribution, that is: ， in, represents the cost function; the optimal transmission matrix is: 。

Images