A deep learning cloud removal method based on SAR-optical remote sensing image joint

Abstract

The present invention provides a deep learning cloud removal method based on the combination of SAR and optical remote sensing imagery. The specific steps are: generating SAR and corresponding optical remote sensing image datasets, including a training sample dataset, a test sample dataset, and a test sample dataset; building a SAR-optical migration model, training and optimizing the migration model until convergence, inputting the SAR images in the test sample dataset into the migration model to generate pseudo-optical images, and completing SAR-optical migration; building an optical image cloud area reconstruction model, training and optimizing the reconstruction model until convergence, and inputting paired pseudo-optical and cloud-covered images in the test sample dataset into the reconstruction model to generate cloud-free optical images. This method can fully utilize the cloud penetration characteristics of SAR images, providing more realistic reference information for cloud removal methods, improving the reliability of network reconstruction in cloud-covered areas, and completing the task of removing thick clouds from remote sensing images with high efficiency and precision.

Description

Technical field

[0001] The present invention relates to a method for removing thick clouds from remote sensing images based on deep learning combined with SAR and optical remote sensing images. [Background Technology]

[0002] With the development of science and technology, remote sensing technology, as a means of observing the Earth over long distances and over large areas, plays a vital role in geography, land surveying, and most other earth science disciplines. However, due to the inevitable obstruction of observation by clouds, optical remote sensing image data often contains a large amount of missing information. The presence of clouds (especially thick clouds) significantly reduces the effective utilization of remote sensing image data, further affecting downstream industrial applications such as image mosaicking, change detection, and object classification. Given this, image cloud removal is fundamental and critical to the subsequent application of remote sensing data. Therefore, research on thick cloud removal methods in remote sensing imagery is of great significance.

[0003] Traditional remote sensing image cloud removal methods typically utilize multi-temporal optical auxiliary data as a reference to determine the correlation between the auxiliary image and the target cloud-covered image. However, obtaining optical auxiliary data in practical applications is difficult. Auxiliary data with a similar time interval to the target image are often also covered by clouds, while data with a longer time interval exhibit significant differences in land type and spectral information from the target image. Some researchers have also incorporated SAR data into cloud removal methods. While this overcomes the shortcomings of traditional optical imaging methods, SAR data can introduce additional noise and other interfering information.

[0004] Deep learning, a typical machine learning framework developed from traditional neural networks, can extract high-level features from remote sensing imagery and is therefore suitable for thick cloud removal from remote sensing imagery. Generative adversarial networks (GANs) are an advanced and effective network within deep learning, offering significant advantages in image processing. They can directly generate image information using adversarial learning, effectively reducing image distortion and noise. Furthermore, their robustness and generalization capabilities make them promising research prospects in the field of thick cloud removal from remote sensing imagery.

[0005] At present, the thick cloud removal method based on generative adversarial network has high accuracy, but there are still some problems. Its shortcomings are mainly manifested in the following two aspects:

[0006] On the one hand, the network fitting capability is insufficient to be applied to remote sensing images with complex ground objects, resulting in problems such as ground object distortion and noise in the images after cloud removal; on the other hand, the current method is highly dependent on the quality of optical images, and the spatial and spectral information of the images is not fully utilized, resulting in low accuracy of cloud removal images, which is difficult to serve downstream production applications. [Summary of the invention]

[0007] To address the above problems, the present invention provides a deep learning cloud removal method based on the combination of SAR and optical remote sensing images. This method is used to solve the problem that cloud removal methods based on optical images are too dependent on image quality. It fully combines and utilizes the advantages of SAR images and optical images, provides sufficient auxiliary information for the thick cloud removal network model, and improves the efficiency and accuracy of thick cloud removal in remote sensing images.

[0008] The present invention is implemented through the following technical solutions, providing a deep learning cloud removal method based on the combination of SAR and optical remote sensing images, comprising the following steps:

[0009] S1 Data Preprocessing: Input the SAR and optical remote sensing images to be processed, perform geographic coordinate registration, data enhancement and normalization preprocessing, and obtain the SAR-optical remote sensing image dataset corresponding to the geographic coordinates;

[0010] S2 builds a sample dataset for SAR-optical migration: Based on the SAR-optical remote sensing image dataset built in S1, three types of SAR-optical paired migration sample datasets are built, namely: training sample dataset, verification sample dataset, and test sample dataset;

[0011] S3 builds, trains, and adjusts the SAR-optical image migration model: Builds a SAR-optical image migration model, batch-inputs SAR training samples into the SAR-optical image migration model, uses paired optical training samples as guidance, and employs an adaptive moment estimation optimization algorithm to train the migration model. During training, the accuracy of the migration model is evaluated on a validation sample dataset, and the model weights are adjusted and optimized. Model convergence is achieved after multiple complete training sessions.

[0012] S4 generates pseudo-optical images for SAR-optical migration: Based on the migration model trained in S3, the SAR images in the test sample dataset of S2 are input and the SAR images are migrated into RGB three-channel pseudo-optical images as auxiliary data for reconstructing cloud coverage area information;

[0013] S5 builds a sample dataset for cloud area information reconstruction: the reconstruction dataset includes real optical images, pseudo optical images obtained in S4, and randomly simulated cloud cover images. The reconstruction dataset is divided into: cloud area training sample dataset, cloud area verification sample dataset, and cloud area test sample dataset according to the proportion;

[0014] S6: Build, train, and adjust the cloud area information reconstruction model: Construct a cloud area information reconstruction model, input pseudo optical images and simulated cloud cover images into the reconstruction model in batches, use the real optical images as a guide, and use the adaptive moment estimation optimization algorithm to train the migration model. During the training process, evaluate the accuracy of the migration model on the validation sample data set, adjust the optimization model weights, and complete the model convergence after multiple complete training sessions.

[0015] S7 removes thick clouds in the test sample dataset: Based on the reconstruction model trained by S6, it batch-inputs paired pseudo-optical images and cloud cover images from the cloud area test sample dataset to generate cloud-free optical images and complete the thick cloud removal task.

[0016] In particular, the S1 is implemented in the following manner:

[0017] S11 inputs SAR and optical remote sensing images of the same area but different phases, performs georeferencing processing, and generates position-matched SAR-optical image pairs;

[0018] S12 inputs the SAR-optical image data generated by S11 in pairs, crops them into image blocks of size P×P, and performs data enhancement by random angle rotation, proportional scaling, and adding Gaussian noise processing;

[0019] S13 inputs the SAR-optical image data enhanced by S12 in pairs, and uses a normalization method to normalize the image pixel values ​​to the range of 0-1 to obtain a SAR-optical remote sensing image dataset corresponding to the geographic coordinates.

[0020] In particular, the migration sample dataset in S2 includes: SAR images and optical images paired with geographical locations. The three types of SAR-optical paired migration sample datasets are divided into: 60% training sample dataset, 10% validation sample dataset, and 30% test sample dataset.

[0021] In particular, the S3 is implemented in the following manner:

[0022] S31 uses an encoder-decoder structure to build a SAR-optical image migration model. The encoder-decoder structure includes five encoders and their corresponding decoders, with four additional residual connection modules added at the network bottleneck. The encoder is a combination of a convolutional layer that performs downsampling operations, an instance normalization layer, and a leaky rectified linear unit. The decoder contains a deconvolutional layer that performs upsampling operations, an instance normalization layer, and a rectified linear unit. Each encoder is associated with the corresponding decoder through a skip connection.

[0023] S32 selects a SAR-optical paired remote sensing image training sample dataset, batch-inputs it into the established SAR-optical image migration model, and forward-calculates the output value of the model, wherein the batch size is set to B;

[0024] S33 calculates the loss function of the network model and performs backpropagation. The loss function is calculated according to the following formula:

[0025]

[0026] In formula (1), L mae1 is the mean absolute error loss value, N is the number of samples, I fake is the pseudo optical image generated, I real It is a real optical image;

[0027] The S34 optimizer uses an adaptive moment estimation gradient descent algorithm to minimize network loss and update various parameters in the optimized network model;

[0028] After each training iteration of S35, the performance of the migration model in the verification sample data set is accurately evaluated, and the network model parameters are adjusted according to the changes in the model migration accuracy. After multiple adjustments and training, the network model converges.

[0029] In particular, the S5 is implemented in the following manner:

[0030] S51 randomly generates island-shaped cloud masks and overlays them onto real optical images to generate simulated cloud cover images.

[0031] S52 divides the reconstructed dataset into the following proportions: 60% cloud area training sample dataset, 10% cloud area verification sample dataset, and 30% cloud area test sample dataset.

[0032] In particular, the S6 is implemented in the following manner:

[0033] S61 constructs a cloud area information reconstruction model. The reconstruction model adopts a generative adversarial network framework, which consists of a generator and a discriminator. The generator uses the same encoding-decoding structure as the SAR-optical migration model to reconstruct cloud-covered images into cloud-free images. The discriminator model adopts a five-layer convolutional network structure. The first to fourth layers are a combination of convolutional layers, instance normalization, and rectified linear units for extracting image features. The last module of the discriminator is a single convolutional layer for outputting the identification result.

[0034] S62 selects a cloud area information reconstruction training sample dataset, batch-inputs it into the established cloud area information reconstruction model, and forward-calculates the output value of the model, wherein the batch size is set to B;

[0035] S63 calculates the loss function of the network model and back-propagates. The loss function is calculated according to the following formula:

[0036] Loss=λ1L mae2 +λ2L perc +λ3L style +λ4L tv (2),

[0037] In formula (2), Loss is the total loss function, Lmae2 , L perc , L style , L tv are the cloud area mean absolute error loss, perceptual feature loss, perceptual style loss, and total variation loss values, respectively; λ1, λ2, λ3, and λ4 are the weight hyperparameters of the cloud area mean absolute error loss, perceptual feature loss, perceptual style loss, and total variation loss, respectively;

[0038] The S64 training method is to alternate between the generator and the discriminator. As the number of training times increases, the generator and the discriminator will continuously improve their performance in the adversarial learning process until they reach a Nash equilibrium. The optimizer uses an adaptive moment estimation gradient descent algorithm to minimize network loss and update and optimize various parameters in the network model.

[0039] S65 evaluates the accuracy of the verification model. After each training iteration, the performance of the cloud area information reconstruction model in the verification sample data set is accurately evaluated. The network model parameters are adjusted according to the changes in the model reconstruction accuracy. After multiple trainings, the network model converges.

[0040] In particular, the cloud area mean absolute error loss L in S63 mae2 Represented as the true value I gt and the predicted value I pred The sum of absolute differences of is minimized, which is calculated as follows:

[0041]

[0042] In formula (3), M is the number of samples, I gt For a true cloudless image, L pred is the predicted value, i.e., the cloud-removed image generated by the reconstruction model;

[0043] The cloud area perception feature loss L perc By defining a distance measure between activation maps of the pre-trained network, we penalize predictions that are not similar to the label in terms of perceptual features, which is calculated as follows:

[0044]

[0045] In formula (4), φ corresponds to the publicly available VGG-19 pre-trained network. This activation map is also used to calculate the perceptual style loss between the predicted value and the true value in the cloud region of the image. The perceptual style loss of the cloud region is calculated as follows:

[0046]

[0047] In formula (5), G φIt is a Gram matrix constructed by the activation map φ, which is used to constrain the style features between images and reduce the phenomenon of image spectral distortion;

[0048] The total variation loss L tv It is a commonly used constraint regularization term used to maintain the smoothness of the image during the training and optimization process of the reconstruction model, which can effectively reduce the noise generation. It is calculated according to the following formula:

[0049]

[0050] In formula (6), x i，j is the pixel value at (i, j) in the image, and β is used to adjust the degree of total variation.

[0051] The present invention provides a deep learning cloud removal method based on the combination of SAR and optical remote sensing images. The method adopts SAR to optical image migration technology to generate pseudo-optical images, greatly reducing the noise interference of the SAR data itself, while increasing the auxiliary information of the image to ensure the reliability of thick cloud removal. The method builds an image reconstruction model and further processes the rough pseudo-optical image to generate a fine cloud-free image based on the pseudo-optical image, thereby improving the accuracy of thick cloud removal. Compared with the remote sensing image thick cloud removal method based on deep learning, the cloud-removed image has higher accuracy and greater usability.

Brief Description of the Drawings

[0052] Figure 1 This is a flow chart of a deep learning cloud removal method based on SAR-optical remote sensing image combination in the present invention;

[0053] Figure 2 This is a structural diagram of the SAR-optical image migration model built using the method provided by the present invention;

[0054] Figure 3 A comparative diagram of the results of processing a migration dataset using the SAR-optical image migration model trained by the present invention and the results of other migration methods on the same data;

[0055] Figure 4 This is a structural diagram of a cloud area information reconstruction model constructed using the method provided by the present invention;

[0056] Figure 5 This is a comparative diagram of the results of processing a reconstructed data set using the cloud area information reconstruction model trained by the present invention and the results of other cloud removal methods on the same data. [Specific implementation method]

[0057] It should be noted that remote sensing technology is mentioned in the National Natural Science Youth Fund Project "Knowledge Representation and Reuse of Remote Sensing Information Extraction Based on Geographic Ontology: Taking Urban Surface Cover as an Example", National Natural Science Foundation of China, 2021-2023, 42001331; Guangxi Key R&D Program Project "Key Technologies and Application Demonstration of Guangxi Natural Resources Satellite Remote Sensing Intelligent Monitoring", Guike AB22080080. Remote sensing technology needs to be supported by standardized cloud-free images, which shows the importance of the development of remote sensing declouding technology. In view of this, the present invention provides a deep learning declouding method based on the combination of SAR and optical remote sensing images. In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is further described in detail below with reference to the accompanying drawings.

[0058] See also Figure 1 The present invention provides a deep learning cloud removal method based on the combination of SAR and optical remote sensing images. The premise of its application is the existence of SAR image data in the same geographical area as the cloud-covered optical image. The remote sensing image thick cloud removal method adopts a SAR-optical combined remote sensing image thick cloud removal network, which includes two steps: SAR optical migration and cloud area reconstruction. It fully utilizes the cloud penetration characteristics of SAR images, provides more real reference information for the cloud removal method, and effectively removes thick clouds in the image. The specific implementation method is as follows:

[0059] S1 data preprocessing: Sentinel-1 SAR images and Sentinel-2 optical images of the same area were selected as data sources. Geographic coordinate registration, data enhancement, and normalization preprocessing were performed to obtain a SAR-optical remote sensing image dataset corresponding to geographic coordinates. The specific implementation method is as follows:

[0060] S11 inputs SAR and optical remote sensing images of the same area but different phases, performs georeferencing processing, and generates position-matched SAR-optical image pairs;

[0061] S12 inputs the SAR-optical image data generated by S11 in pairs, crops them into image blocks of size P×P, and performs data enhancement by performing random angle rotation with a probability of 20%, scaling, and adding Gaussian noise.

[0062] S13 inputs the SAR-optical image data enhanced by S12 in pairs, and uses the maximum and minimum value normalization method to normalize the image pixel values ​​to the range of 0-1, generating a normalized SAR-optical remote sensing image dataset corresponding to geographic coordinates.

[0063] S2 constructs a sample dataset for SAR-optical migration: Based on the SAR-optical remote sensing image dataset constructed in S1, three types of SAR-optical paired migration sample datasets are constructed. The migration sample datasets include: SAR images and optical images paired with geographical locations. The three types of SAR-optical paired migration sample datasets are divided into: 60% training sample dataset, 10% verification sample dataset, and 30% test sample dataset.

[0064] S3 builds, trains and adjusts the SAR-optical image migration model: Constructs the SAR-optical image migration model such as Figure 2 As shown in the figure, SAR training samples are batch-inputted into the SAR-optical image transfer model. Paired optical training samples are used as guidance, and an adaptive moment estimation optimization algorithm is used to train the transfer model. During the training process, the accuracy of the transfer model is evaluated using a validation sample dataset, and the model weights are adjusted and optimized. After multiple complete training sessions, the model converges. The specific implementation is as follows:

[0065] S31 please refer to Figure 2 , a SAR-to-optical image migration model is built using an encoder-decoder structure consisting of five encoders and their corresponding decoders, with four additional residual connection modules added at the network bottleneck. The encoder is a combination of a convolutional layer that performs downsampling operations, an instance normalization layer, and a leaky rectified linear unit. The decoder contains a deconvolutional layer that performs upsampling operations, an instance normalization layer, and a rectified linear unit. Each encoder is associated with its corresponding decoder via a skip connection.

[0066] S32 selects a SAR-optical paired remote sensing image training sample dataset, batch-inputs it into the established SAR-optical image migration model, and forward-calculates the output value of the model, wherein the batch size is set to B;

[0067] S33 calculates the loss function of the network model and performs backpropagation. The loss function is calculated according to the following formula:

[0068]

[0069] In formula (1), L mea1 is the mean absolute error loss value, N is the number of samples, I fake is the pseudo optical image generated, I real It is a real optical image;

[0070] The S34 optimizer uses an adaptive moment estimation gradient descent algorithm to minimize network loss and update various parameters in the optimized network model;

[0071] S35 evaluates and verifies the model accuracy: After each training iteration, the performance of the migration model in the verification sample data set is accurately evaluated. The network model parameters are adjusted according to the changes in the model migration accuracy. After multiple adjustments and training, the network model converges.

[0072] S4 generates pseudo optical images for SAR-optical migration: Based on the migration model trained in S3, the SAR images in the test sample dataset of S2 are input and the SAR images are migrated into pseudo optical images with RGB three channels as auxiliary data for reconstructing cloud coverage area information. Figure 3 , Figure 3 The results of the popular deep learning transfer method and the method of the present invention on images of buildings, farmland, cultivated land and mountains are shown. Figure 3 It can be seen that the migration model proposed in this invention, due to the introduction of the encoding-decoding structure and the residual connection module, can greatly reduce the noise interference of the SAR data itself in images of various types of land objects compared with other migration methods, while increasing the auxiliary information in the image space and spectrum, which is conducive to improving the reliability of the next step of thick cloud removal.

[0073] S5 constructs a sample dataset for cloud area information reconstruction: the reconstructed dataset includes: real optical images, pseudo optical images obtained in S4, and randomly simulated cloud cover images. The specific implementation is as follows:

[0074] S51 randomly generates island-shaped cloud masks and overlays them onto real optical images to generate simulated cloud cover images.

[0075] S52 divides the reconstructed dataset into the following proportions: 60% cloud area training sample dataset, 10% cloud area verification sample dataset, and 30% cloud area test sample dataset.

[0076] S6 builds, trains and adjusts the cloud area information reconstruction model: builds the cloud area information reconstruction model such as Figure 4 As shown in the figure, pseudo optical images and simulated cloud cover images are input into the reconstruction model in batches and pairs. Using real optical images as guidance, the adaptive moment estimation optimization algorithm is used to train the transfer model. During the training process, the accuracy of the transfer model is evaluated on the validation sample dataset, and the optimization model weights are adjusted. After multiple complete training sessions, the model converges. The specific implementation method is as follows:

[0077] S61 builds a cloud area information reconstruction model such as Figure 4As shown in the figure, the reconstruction model adopts a generative adversarial network framework, which consists of a generator and a discriminator. The generator uses the same encoding-decoding structure as the SAR-optical transfer model to reconstruct cloud-covered images into cloud-free images. The discriminator model adopts a five-layer convolutional network structure. The first to fourth layers are a combination of convolutional layers, instance normalization, and rectified linear units for extracting image features. The last module of the discriminator is a single convolutional layer for outputting the identification results.

[0078] S62 selects a cloud area information reconstruction training sample data set, batch-inputs it into the built cloud area information reconstruction model, and forward-calculates the output value of the model, wherein the batch size is set to B;

[0079] S63 calculates the loss function of the network model and back-propagates. The loss function is calculated according to the following formula:

[0080] Loss=λ1L mae2 +λ2L perc +λ3L style +λ4L tv (2),

[0081] In formula (2), Loss is the total loss function, L mae2 , L perc , L style , L tv are the cloud area mean absolute error loss, perceptual feature loss, perceptual style loss, and total variation loss values, respectively; λ1, λ2, λ3, and λ4 are the weight hyperparameters of the cloud area mean absolute error loss, perceptual feature loss, perceptual style loss, and total variation loss, respectively;

[0082] The cloud area mean absolute error loss L mae2 Represented as the true value I gt and the predicted value I pred The sum of absolute differences of is minimized, which is calculated as follows:

[0083]

[0084] In formula (3), M is the number of samples, I gt For a true cloudless image, L pred is the predicted value, i.e., the cloud-removed image generated by the reconstruction model;

[0085] The cloud area perception feature loss L prec By defining a distance measure between activation maps of the pre-trained network, we penalize predictions that are not similar to the label in terms of perceptual features, which is calculated as follows:

[0086]

[0087] In formula (4), φ corresponds to the publicly available VGG-19 pre-trained network. This activation map is also used to calculate the perceptual style loss between the predicted value and the true value of the image in the cloud area. The perceptual style loss is calculated as follows:

[0088]

[0089] In formula (5), G φ It is a Gram matrix constructed by the activation map φ, which is used to constrain the style features between images and reduce the phenomenon of image spectral distortion;

[0090] The total variation loss L tv It is a commonly used constraint regularization term used to maintain the smoothness of the image during the training and optimization process of the reconstruction model, which can effectively reduce the noise generation. It is calculated according to the following formula:

[0091]

[0092] In formula (6), x i，j is the pixel value at (i, j) in the image, and β is used to adjust the degree of total variation;

[0093] The S64 training method is to alternate between the generator and the discriminator. As the number of training times increases, the generator and the discriminator will continuously improve their performance in the adversarial learning process until they reach a Nash equilibrium. The optimizer uses an adaptive moment estimation gradient descent algorithm to minimize network loss and update and optimize various parameters in the network model.

[0094] S65 evaluates the accuracy of the verification model. After each training iteration, the performance of the cloud area information reconstruction model in the verification sample data set is accurately evaluated. The network model parameters are adjusted according to the changes in the model reconstruction accuracy. After multiple trainings, the network model converges.

[0095] S7 removes thick clouds in the test sample dataset: Based on the reconstruction model trained by S6, it batch-inputs paired pseudo-optical images and cloud cover images from the cloud area test sample dataset to generate cloud-free optical images and complete the thick cloud removal task.

[0096] See also Figure 5 , Figure 5 This is a comparison chart of the cloud removal method of the present invention and the current advanced deep learning cloud removal method. Figure 5 It can be seen that the method provided by the present invention introduces the pseudo-optical image generated by SAR migration as reference information and adopts the cloud area information reconstruction model. Compared with the remote sensing image thick cloud removal method based on deep learning, the cloud area edge transition is more natural, the ground object reconstruction degree is higher, and the cloud removal image has the best visual effect.

[0097] In summary, the method for removing thick clouds from remote sensing images provided by the present invention first produces SAR and corresponding optical remote sensing image datasets, including a training sample dataset, a test sample dataset, and a test sample dataset; secondly, a SAR-optical migration model is built, the migration model is trained and optimized until convergence, and the SAR images in the test sample dataset are input into the migration model to generate pseudo-optical images, thereby completing the SAR-optical migration; thirdly, an optical image cloud area reconstruction model is built, the reconstruction model is trained and optimized until convergence, and the pseudo-optical and cloud cover images in the test sample dataset are input into the reconstruction model in pairs to generate cloud-free optical images, thereby finally completing the thick cloud removal.

Claims

1. A deep learning cloud removal method based on SAR-optical remote sensing image joint, characterized by: The following steps are involved: S1 Data Preprocessing: Input the SAR and optical remote sensing images to be processed, perform geographic coordinate registration, data enhancement and normalization preprocessing, and obtain the SAR-optical remote sensing image dataset corresponding to the geographic coordinates; S2 builds a sample dataset for SAR-optical migration: Based on the SAR-optical remote sensing image dataset built in S1, three types of SAR-optical paired migration sample datasets are built, namely: training sample dataset, verification sample dataset, and test sample dataset; S3 builds, trains, and adjusts the SAR-optical image migration model: Builds a SAR-optical image migration model, batch-inputs SAR training samples into the SAR-optical image migration model, uses paired optical training samples as guidance, and employs an adaptive moment estimation optimization algorithm to train the migration model. During training, the accuracy of the migration model is evaluated on a validation sample dataset, and the model weights are adjusted and optimized. Model convergence is achieved after multiple complete training sessions. S4 generates pseudo-optical images for SAR-optical migration: Based on the migration model trained in S3, the SAR images in the test sample dataset of S2 are input, and the SAR images are migrated into RGB three-channel pseudo-optical images as auxiliary data for reconstructing cloud coverage area information; S4 adopts an encoding-decoding structure to build a SAR-optical image migration model, and the encoding-decoding structure includes: five encoders and their corresponding decoders, and four additional residual connection modules are added at the network bottleneck. The encoder is a combination of a convolutional layer that performs a downsampling operation, an instance normalization layer, and a leakage rectified linear unit. The decoder includes a deconvolutional layer that performs an upsampling operation, an instance normalization layer, and a rectified linear unit. Each encoder is associated with the corresponding decoder through a jump connection; S5 builds a sample dataset for cloud area information reconstruction: the reconstruction dataset includes real optical images, pseudo optical images obtained in S4, and randomly simulated cloud cover images. The reconstruction dataset is divided into: cloud area training sample dataset, cloud area verification sample dataset, and cloud area test sample dataset according to the proportion; S6 builds, trains, adjusts and trains a cloud area information reconstruction model: constructs a cloud area information reconstruction model, batch-paired input of pseudo optical images and simulated cloud cover images into the reconstruction model, uses real optical images as guidance, adopts an adaptive moment estimation optimization algorithm, trains the transfer model, and during the training process, evaluates the accuracy of the transfer model on the validation sample data set, adjusts and optimizes the model weights, and completes the model convergence after multiple complete trainings; S6 constructs a cloud area information reconstruction model, and the reconstruction model adopts a generative adversarial network framework, which consists of a generator and a discriminator, wherein the generator adopts the same encoding-decoding structure as the SAR-optical transfer model to reconstruct the cloud cover image into a cloud-free image, and the discriminator model adopts a five-layer convolutional network structure, wherein the first to fourth layers are a combination of convolutional layers, instance normalization and rectified linear units, which are used to extract image features, and the last module of the discriminator is a single-layer convolutional layer, which is used to output the identification results; S7 removes thick clouds in the test sample dataset: Based on the reconstruction model trained by S6, it batch-inputs paired pseudo-optical images and cloud cover images from the cloud area test sample dataset to generate cloud-free optical images and complete the thick cloud removal task.

2. The deep learning cloud removal method based on SAR-optical remote sensing image combination according to claim 1 is characterized in that: The S1 is specifically implemented according to the following method: S11 inputs SAR and optical remote sensing images of the same area but different phases, performs georeferencing processing, and generates position-matched SAR-optical image pairs; S12 inputs the SAR-optical image data generated by S11 in pairs, crops them into image blocks of size P×P, and performs data enhancement by random angle rotation, proportional scaling, and adding Gaussian noise processing; S13 inputs the SAR-optical image data enhanced by S12 in pairs, and uses a normalization method to normalize the image pixel values ​​to the range of 0-1 to obtain a SAR-optical remote sensing image dataset corresponding to the geographic coordinates.

3. The deep learning cloud removal method based on SAR-optical remote sensing image combination according to claim 1 is characterized in that: The migration sample dataset in S2 includes: SAR images and optical images paired with geographical locations. The three types of SAR-optical paired migration sample datasets are divided into: 60% training sample dataset, 10% validation sample dataset, and 30% test sample dataset.

4. The deep learning cloud removal method based on SAR-optical remote sensing image combination according to claim 1 is characterized in that: The S4 is specifically implemented according to the following method: S41 selects a SAR-optical paired remote sensing image training sample dataset, batch-inputs it into the established SAR-optical image migration model, and forward-calculates the output value of the model, wherein the batch size is set to B; S42 calculates the loss function of the network model and performs backpropagation. The loss function is calculated according to the following formula: （1）， In formula (1), is the mean absolute error loss value, is the number of samples, To generate the pseudo optical image, It is a real optical image; The S43 optimizer uses an adaptive moment estimation gradient descent algorithm to minimize network loss and update various parameters in the optimized network model; After each training iteration of S44, the performance of the migration model in the verification sample data set is accurately evaluated, and the network model parameters are adjusted according to the changes in the model migration accuracy. After multiple adjustments and training, the network model converges.

5. The deep learning cloud removal method based on SAR-optical remote sensing image combination according to claim 1 is characterized in that: The S5 is specifically implemented according to the following method: S51 randomly generates island-shaped cloud masks and overlays them onto real optical images to generate simulated cloud cover images. S52 divides the reconstructed dataset into the following proportions: 60% cloud area training sample dataset, 10% cloud area verification sample dataset, and 30% cloud area test sample dataset.

6. The deep learning cloud removal method based on SAR-optical remote sensing image combination according to claim 4 is characterized in that: The S6 is specifically implemented according to the following method: S61 selects a cloud area information reconstruction training sample dataset, batch-inputs it into the established cloud area information reconstruction model, and forward-calculates the output value of the model, wherein the batch size is set to B; S62 calculates the loss function of the network model and back-propagates. The loss function is calculated according to the following formula: （2）， In formula (2), is the total loss function, They are respectively the cloud area mean absolute error loss, perceptual feature loss, perceptual style loss, and total variation loss value. are the weight hyperparameters of cloud area mean absolute error loss, perceptual feature loss, perceptual style loss, and total variation loss respectively; The S63 training method is to alternate between the generator and the discriminator. As the number of training times increases, the generator and the discriminator will continuously improve their reconstruction and identification performance in the adversarial learning process until they reach a Nash equilibrium. The optimizer uses an adaptive moment estimation gradient descent algorithm to minimize network loss and update and optimize various parameters in the network model. S64 evaluates the accuracy of the verification model. After each training iteration, the performance of the cloud area information reconstruction model in the verification sample data set is accurately evaluated. The network model parameters are adjusted according to the changes in the model reconstruction accuracy. After multiple trainings, the network model converges.

7. The deep learning cloud removal method based on SAR-optical remote sensing image combination according to claim 6 is characterized in that: The mean absolute error loss in the cloud area of ​​S62 Represented as true value and predicted value The sum of absolute differences of is minimized, which is calculated as follows: （3）， In formula (3), is the number of samples, For a true cloudless image, To generate the predicted image, i.e., the cloud-removed image generated by the reconstruction model; The perceptual feature loss By defining a distance measure between activation maps of the pre-trained network, we penalize predictions that are not similar to the label in terms of perceptual features, which is calculated as follows: （4）， In formula (4), Corresponding to the publicly available VGG-19 pre-trained network, this activation map is also used to calculate the perceptual style loss between the predicted and true values ​​of the cloud image. The perceptual style loss calculation formula for the cloud area is as follows: （5）， In formula (5), The activation map The constructed Gram matrix is ​​used to constrain the style features between images and reduce the phenomenon of image spectral distortion; The total variation loss It is a commonly used constraint regularization term used to maintain the smoothness of the image during the training and optimization process of the reconstruction model, which can effectively reduce the noise generation. It is calculated according to the following formula: （6）， In formula (6), is located in the image The pixel value of Used to adjust the degree of total variation.

Images