Density-aware cloud removal method based on optical synthetic aperture radar transducer guidance

Abstract

The present invention discloses a density-aware cloud removal method based on optical synthetic aperture radar converter guidance, constructs a cloud removal model including a first image feature extraction module, a second image feature extraction module, a cloud density perception module, an optical synthetic aperture radar converter module, an optical feature fusion module and an image superposition module, extracts features of a cloud-covered optical image and a synthetic aperture radar image respectively, estimates the cloud density level according to the features of the cloud-covered optical image, transforms and processes the features of the cloud-covered optical image according to the features of the synthetic aperture radar image, fuses the obtained features of the cloud-covered optical image and superimposes them with the original cloud-covered optical image to obtain a cloud-free image; after training the cloud removal model with a preset cloud removal data set, the trained cloud removal module can be used to remove clouds from the cloud-covered optical image according to the synthetic aperture radar image. The present invention can significantly improve the effect of cloud removal.

Description

Technical Field

[0001] The invention belongs to the technical field of remote sensing image processing, and more specifically, relates to a density-aware cloud removal method based on guidance of an optical synthetic aperture radar converter. Background Art

[0002] In the field of remote sensing image processing, cloud cover is a common challenge that affects ground feature extraction and analysis. Since the launch of the Terra and Aqua satellite platforms in 2000 and 2002, respectively, various cloud attribute data of MODIS have been continuously observed. MODIS cloud mask data show that the global cloud coverage is about 67% and the cloud coverage of the land area is about 55%.

[0003] Synthetic aperture radar plays an important role in the cloud removal challenge due to its strong penetration ability and backscatter measurement characteristics. At present, a variety of cloud removal methods based on synthetic aperture radar have been proposed. These methods convert the splicing of multimodal images into cloud-free images through learning. However, existing methods do not consider that the latent structure learned by the model may be different for different cloud density levels, resulting in inconsistent texture and structure, which leads to potential reconstruction failure. Summary of the invention

[0004] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a density-aware cloud removal method based on optical synthetic aperture radar converter guidance, which introduces cloud density perception technology and enhances the cloud removal process based on synthetic aperture radar images, thereby significantly improving the effect of cloud removal.

[0005] In order to achieve the above-mentioned object of the invention, the density-aware cloud removal method based on optical synthetic aperture radar converter guidance of the present invention comprises the following steps:

[0006] S1: Obtain several training samples according to actual needs to construct a cloud removal dataset. Each training sample contains the cloud density levels of cloud-covered optical images, synthetic aperture radar images, cloud-free optical images, and cloud-covered optical images of the same area.

[0007] S2: construct a cloud removal model, including a first image feature extraction module, a second image feature extraction module, a cloud density perception module, an optical synthetic aperture radar converter module, an optical feature fusion module and an image superposition module, wherein:

[0008] The first image feature extraction module is used to extract the cloud cover optical image J opt Perform feature extraction to obtain the cloud cover optical image feature G opt And sent to the cloud density perception module and the optical synthetic aperture radar converter module.

[0009] The second image feature extraction module is used to extract the synthetic aperture radar image J SAR Perform feature extraction to obtain the synthetic aperture radar image feature G SAR and sent to the optical synthetic aperture radar converter module;

[0010] The cloud density perception module is used to detect the cloud cover optical image feature G opt The cloud density is estimated to obtain the cloud density level G label ;

[0011] The optical synthetic aperture radar converter module is used to convert the synthetic aperture radar image features G SAR Cloud cover optical image feature G opt Perform transformation processing to obtain N cloud cover optical image features And output to the optical feature fusion module, i = 1, 2, ..., N; the optical synthetic aperture radar converter module includes N layers of density perception global context interaction modules, where:

[0012] The i-th layer density-aware global context interaction module is used to input cloud cover optical image features and synthetic aperture radar image features Perform interactive processing to obtain cloud cover optical image features and synthetic aperture radar image features Output: Input cloud coverage optical image features of the first layer density-aware global context interaction module Synthetic Aperture Radar Image Features The first N-1 layers of density-aware global context interaction module converts the obtained cloud cover optical image features and synthetic aperture radar image features Output to the next layer of density-aware global context interaction module, i′=1,2,…,N-1, each layer of density-aware global context interaction module obtains the cloud coverage optical image features All are output to the optical feature fusion module;

[0013] The optical feature fusion module is used to combine the optical image features of N clouds. The stitching is then performed and the stitching features are resized to the cloud-covered optical image J opt The size of the feature image J R Send to the image overlay module;

[0014] Image overlay module is used to overlay clouds onto optical images. opt and feature image J R Superimpose to obtain a cloud-free optical image J D =J opt +J R ;

[0015] S3: using the cloud removal data set in step S1 to train the cloud removal model to obtain a trained cloud removal model;

[0016] S4: When a cloud-covered optical image needs to be cloud-removed, the cloud-covered optical image and the corresponding synthetic aperture radar image are input into the trained cloud removal model to obtain a cloud-free optical image.

[0017] The present invention is based on a density-aware cloud removal method guided by an optical synthetic aperture radar converter, and constructs a cloud removal model including a first image feature extraction module, a second image feature extraction module, a cloud density perception module, an optical synthetic aperture radar converter module, an optical feature fusion module and an image superposition module. After feature extraction is performed on a cloud-covered optical image and a synthetic aperture radar image respectively, a cloud density level is estimated according to features of the cloud-covered optical image, features of the cloud-covered optical image are transformed and processed according to features of the synthetic aperture radar image, and the obtained features of the cloud-covered optical image are fused and superimposed with the original cloud-covered optical image to obtain a cloud-free image. After the cloud removal model is trained using a preset cloud removal data set, the trained cloud removal module can be used to remove clouds from the cloud-covered optical image according to the synthetic aperture radar image.

[0018] The present invention has the following beneficial effects:

[0019] 1) The present invention introduces a cloud density perception module and uses different levels of cloud density information to assist the cloud cover optical image in learning the ground texture information covered by clouds in the synthetic aperture radar, thereby improving the cloud removal performance and solving the problem of texture and structure inconsistency caused by the cloud removal task based on the synthetic aperture radar image;

[0020] 2) The present invention adopts a density-aware global context interaction module to combine the cloud cover optical map with synthetic aperture radar image features, which facilitates more detailed and precise cloud removal and ensures the integrity of the underlying surface details while minimizing the introduction of noise and artifacts. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a flowchart of a specific implementation of the density-aware cloud removal method guided by an optical synthetic aperture radar converter according to the present invention;

[0022] Figure 2 is a structural diagram of the cloud removal model in the present invention;

[0023] Figure 3 is a structural diagram of the density-aware global context interaction module in this embodiment. DETAILED DESCRIPTION

[0024] The specific implementation of the present invention is described below in conjunction with the accompanying drawings so that those skilled in the art can better understand the present invention. It should be noted that in the following description, when the detailed description of known functions and designs may dilute the main content of the present invention, these descriptions will be omitted here.

[0025] Example

[0026] Figure 1 FIG. 1 is a flow chart of a specific implementation of the density sensing cloud removal method based on optical synthetic aperture radar converter guidance of the present invention. Figure 1 As shown, the specific steps of the density-aware cloud removal method based on optical synthetic aperture radar converter guidance of the present invention include:

[0027] S101: Get the cloud-free dataset:

[0028] According to actual needs, several training samples are obtained to construct a cloud removal dataset. Each training sample contains the cloud density levels of cloud-covered optical images, synthetic aperture radar images, cloud-free optical images, and cloud-covered optical images of the same area.

[0029] In order to save the cost of determining the cloud density level label, a pixel comparison-based method is used in this embodiment to obtain a rough cloud density level. The specific method is as follows: determine the cloud coverage pixel value range according to actual needs, filter out cloud coverage pixels from the cloud coverage optical image to form an area, and then calculate the proportion of the cloud coverage area to the entire cloud coverage optical image, and judge the cloud density level corresponding to the proportion value according to a preset level range.

[0030] S102: Building a cloud removal model:

[0031] In the present invention, in order to better remove clouds, a cloud removal model based on optical synthetic aperture radar converter guidance and density perception is constructed. Figure 2 is a structural diagram of the cloud removal model in the present invention. Figure 2 As shown, the cloud removal model in the present invention includes a first image feature extraction module (SFE1), a second image feature extraction module (SFE2), a cloud density perception module, an optical synthetic aperture radar converter module, an optical feature fusion module and an image superposition module. Next, each module is described in detail.

[0032] The first image feature extraction module is used to extract the cloud cover optical image J opt Perform feature extraction to obtain the cloud cover optical image feature G opt And sent to the cloud density perception module and the optical synthetic aperture radar converter module.

[0033] The second image feature extraction module is used to extract the synthetic aperture radar image J SARPerform feature extraction to obtain the synthetic aperture radar image feature G SAR And sent to the optical synthetic aperture radar converter module.

[0034] The cloud density perception module is used to detect the cloud cover optical image feature G opt The cloud density is estimated to obtain the cloud density level G label In this embodiment, the cloud density perception module includes a feature expansion module and a classification module, wherein:

[0035] The feature expansion module is used to extract the cloud cover optical image feature G opt Flatten to obtain the two-dimensional feature g opt And sent to the classification module. In this embodiment, the feature expansion module uses a fully connected layer (Fully Connected, FC).

[0036] The classification module is used to classify the two-dimensional feature g opt Classify and use the obtained cloud density category as the cloud density level G label .

[0037] The optical synthetic aperture radar converter module is used to convert the synthetic aperture radar image features G SAR Cloud cover optical image feature G opt Perform transformation processing to obtain N cloud cover optical image features And output to the optical feature fusion module, i=1,2,…,N.

[0038] like Figure 2 As shown, the optical synthetic aperture radar converter module in the present invention includes N layers of density-aware global context interaction modules (DGCI, Density-aware Global Context Interaction), wherein the i-th layer of density-aware global context interaction module is used to input cloud cover optical image features and synthetic aperture radar image features Perform interactive processing to obtain cloud cover optical image features and synthetic aperture radar image features Output. Input cloud coverage optical image features of the first layer density-aware global context interaction module Synthetic Aperture Radar Image Features The first N-1 layers of density-aware global context interaction module converts the obtained cloud cover optical image features and synthetic aperture radar image features Output to the next layer of density-aware global context interaction module, i′=1,2,…,N-1, each layer of density-aware global context interaction module obtains the cloud coverage optical image features All are output to the optical feature fusion module.

[0039] Figure 3 is a structural diagram of the density-aware global context interaction module in this embodiment. Figure 3 As shown, the density-aware global context interaction module in this embodiment includes a radar image processing module, an optical image processing module and a global-local fusion transformation module, wherein:

[0040] The radar image processing module is used to process the synthetic aperture radar image features according to the input Processing to obtain synthetic aperture radar image features The radar image processing module includes K layers of radar image processing units, wherein the jth layer of radar image processing unit is used to process the input synthetic aperture radar image features. Processing to obtain synthetic aperture radar image features Where j = 1, 2, ..., K-1, The first K-1 layer radar image processing unit will obtain the synthetic aperture radar image features Output to the next layer of radar image processing unit, j′=1,2,…,K-1, the Kth layer radar image processing unit will obtain the synthetic aperture radar image features Send it to the global-local fusion transformation module and transform the features As synthetic aperture radar image features Output. Figure 3 As shown, each radar image processing unit includes a convolution layer (conv), a Relu activation layer (Relu), a Swin Transformer layer (STL) and an overlay layer (Add), where:

[0041] The convolutional layer is used to transform the input synthetic aperture radar image features. Perform a convolution operation and send the obtained features to the Relu activation layer.

[0042] The Relu activation layer is used to process the received features using the Relu activation function. Send to the Swin Transformer layer.

[0043] The Swin Transformer layer is used to use a moving window self-attention mechanism to For processing, the specific method of the self-attention mechanism is:

[0044] Using weight matrix The received features Processing is performed to obtain the query features of the synthetic aperture radar image Key Features Sum value characteristics Compute the transformation matrix of a synthetic aperture radar image And sent to the j-th optical image processing unit in the optical image processing module:

[0045]

[0046] Where d is the number of rows of the query feature, and the superscript T represents transposition. represents a learnable relative position encoding.

[0047] Then use the Softmax function to transform the matrix Processing to obtain features And send to the overlay:

[0048]

[0049] Overlays are used to map synthetic aperture radar image features With features Superposition is performed to obtain the characteristics And output.

[0050] The optical image processing module is used to process the cloud cover optical image features according to the input Processing to obtain cloud cover optical image features Similar to the radar image processing module, the optical image processing module includes K layers of optical image processing units, wherein the jth layer of optical image processing units is used to process the cloud cover optical image features according to the input Processing to obtain cloud cover optical image features in The optical image processing unit of the first K-1 layer will obtain the cloud cover optical image features Output to the next layer of optical image processing unit, j′=1,2,…,K-1, the Kth layer optical image processing unit will obtain the cloud cover optical image features Output to the global-local fusion transformation module. Figure 3 As shown, each optical image processing unit includes a convolution layer, a Relu activation layer, a Swin Transformer layer and a stacking layer, where:

[0051] The convolutional layer is used to transform the input cloud cover optical image features Perform a convolution operation and send the obtained features to the Relu activation layer.

[0052] The Relu activation layer is used to process the received features using the Relu activation function. Send to the Swin Transformer layer.

[0053] The Swin Transformer layer is used to use a moving window self-attention mechanism to To process, the specific method of the self-attention mechanism is:

[0054] Using weight matrix The received features Processing is performed to obtain the query features of the cloud cover optical image Key Features Sum value characteristics Compute the transformation matrix of cloud-covered optical images

[0055]

[0056] in, represents a learnable relative position encoding.

[0057] Then the gating function is used to obtain the cloud cover optical image transformation matrix guided by the synthetic aperture radar image

[0058]

[0059] Among them, ⊙ represents the multiplication of corresponding elements, and G() represents the gating function, which is used to adjust the attention weight.

[0060] Then use the Softmax function to transform the matrix Processing to obtain features And send to the overlay:

[0061]

[0062] Overlays are used to cover cloud optical image features With features Superposition is performed to obtain the characteristics And output.

[0063] The global-local fusion transformation module is used to Features Processing to obtain cloud cover optical image characteristics And output, the specific method is:

[0064] Features Feature extraction is performed to obtain feature g. In this embodiment, a series of convolutional layers are used to process the feature For feature extraction, the formula can be expressed as:

[0065]

[0066] Then the split function split() is used to split the feature g to obtain the scaling matrix γ and the offset matrix β:

[0067] (γ,β)=split(g)

[0068] The split function split() is a commonly used function in the neural network toolbox torch, and its specific process will not be repeated here.

[0069] Then the cloud cover optical image characteristics are obtained using the following formula

[0070]

[0071] The optical feature fusion module is used to combine the optical image features of N clouds. The stitching is then performed and the stitching features are resized to the cloud-covered optical image J opt The size of the feature image J R Sent to the image superposition module. Therefore, the feature image J R It can be expressed as:

[0072]

[0073] Among them, [] represents feature concatenation and IR() represents the size transformation function.

[0074] Image overlay module is used to overlay clouds onto optical images. opt and feature image J R Superimpose to obtain cloud-free optical image J D =J opt +J R .

[0075] S103: Training cloud removal model:

[0076] The cloud removal data set in step S101 is used to train the cloud removal model to obtain a trained cloud removal model.

[0077] The setting of the loss function is very important for model training. In this embodiment, the loss function LOSS of the cloud removal model training is calculated using the following method:

[0078] LOSS=λ gen L gen +λ cla L cla

[0079] Among them, L gen represents the generation loss of cloud-free optical images, L cla represents the estimated loss of cloud density level, λ gen ,cla They represent the weights of the corresponding losses respectively.

[0080] In this embodiment, the loss L gen The calculation formula is as follows:

[0081]

[0082] in, represents a true cloud-free optical image and L1() represents the L1 norm, which is used to encourage the model to generate predictions that closely match the ground truth, effectively minimizing the absolute difference between them.

[0083] In this example, the estimated loss L cla Cross entropy loss is used.

[0084] In order to balance the training between accurate cloud classification and effective cloud-free image reconstruction, a dynamic weighting mechanism is adopted in this embodiment, namely, the weight λ gen and λ cla Instead of using a fixed value, it is calculated based on the specific value of the loss. The specific method is:

[0085] The loss ratio r is calculated using the following formula:

[0086]

[0087] Where ε represents a minimum value. In this embodiment, ε = e -6 .

[0088] Then the weight λ is calculated using the following formula gen and λ cla :

[0089]

[0090] This approach not only effectively ensures the optimal balance of training focus, but also enhances the overall performance and reliability of the declouding process, effectively addressing the complexity of different cloud densities.

[0091] S104: Cloud Removal:

[0092] When it is necessary to remove clouds from a cloud-covered optical image, the cloud-covered optical image and the corresponding synthetic aperture radar image are input into the trained cloud removal model to obtain a cloud-free optical image.

[0093] In order to better illustrate the technical effect of the present invention, a specific example is used to experimentally verify the present invention. In this embodiment, the SEN12MS-CR dataset is used for test experiments. The dataset is a multimodal and monotemporal dataset, including multispectral optical satellite observation data, covering cloudy and cloudless conditions. In this embodiment, 175 regions of interest (ROIs) are randomly divided into 165 for training, 10 for verification, and 10 for testing. In order to prevent the overall performance from being biased towards any particular season, images of four different seasons are assigned to the verification set and the test set. The results of cloud removal are evaluated by normalizing data based on peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), spectral angle mapper (SAM) and mean absolute error (MAE). Table 1 is a performance comparison table before and after the cloud density perception module is introduced in the present invention in this embodiment.

[0094] PSNR SSIM SAM MAE w / o Cloud Density Sensing Module 29.08 0.885 7.645 0.027 Experimental Results 29.71 0.885 7.526 0.025

[0095] Table 1

[0096] As shown in Table 1, in the cloud removal task guided by synthetic aperture radar image features, the present invention achieves an improvement of 0.6 in PSNR value and 0.12 in SAM value by introducing a cloud density perception module.

[0097] Although the above describes the illustrative specific embodiments of the present invention to facilitate those skilled in the art to understand the present invention, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations using the concept of the present invention are protected.

Claims

1. A density-aware cloud removal method based on optical synthetic aperture radar transducer guidance, characterized in that: The following steps are involved: S1: Obtain several training samples according to actual needs to construct a cloud removal dataset. Each training sample contains the cloud density levels of cloud-covered optical images, synthetic aperture radar images, cloud-free optical images, and cloud-covered optical images of the same area. S2: construct a cloud removal model, including a first image feature extraction module, a second image feature extraction module, a cloud density perception module, an optical synthetic aperture radar converter module, an optical feature fusion module and an image superposition module, wherein: The first image feature extraction module is used to extract the cloud cover optical image J opt Perform feature extraction to obtain the cloud cover optical image feature G opt and sent to the cloud density sensing module and the optical synthetic aperture radar converter module; The second image feature extraction module is used to extract the synthetic aperture radar image J SAR Perform feature extraction to obtain the synthetic aperture radar image feature G SAR and sent to the optical synthetic aperture radar converter module; The cloud density perception module is used to detect the cloud cover optical image feature G opt The cloud density is estimated to obtain the cloud density level G label ; The optical synthetic aperture radar converter module is used to convert the synthetic aperture radar image features G SAR Cloud cover optical image feature G opt Perform transformation processing to obtain N cloud cover optical image features And output to the optical feature fusion module, i = 1, 2, ..., N; the optical synthetic aperture radar converter module includes N layers of density perception global context interaction modules, where: The i-th layer density-aware global context interaction module is used to input cloud cover optical image features and synthetic aperture radar image features Perform interactive processing to obtain cloud cover optical image features and synthetic aperture radar image features Output: Input cloud coverage optical image features of the first layer density-aware global context interaction module Synthetic Aperture Radar Image Features The first N-1 layers of density-aware global context interaction module converts the obtained cloud cover optical image features and synthetic aperture radar image features Output to the next layer of density-aware global context interaction module, i′=1,2,…,N-1, each layer of density-aware global context interaction module obtains the cloud coverage optical image features All are output to the optical feature fusion module; The optical feature fusion module is used to combine the optical image features of N clouds. The stitching is then performed and the stitching features are resized to the cloud-covered optical image J opt The size of the feature image J R Send to the image overlay module; Image overlay module is used to overlay clouds onto optical images. opt and feature image J R Superimpose to obtain cloud-free optical image J D =J opt +J R ; S3: using the cloud removal data set in step S1 to train the cloud removal model to obtain a trained cloud removal model; S4: When a cloud-covered optical image needs to be cloud-removed, the cloud-covered optical image and the corresponding synthetic aperture radar image are input into the trained cloud removal model to obtain a cloud-free optical image.

2. The density-aware cloud removal method according to claim 1, characterized in that: The cloud density level in step S1 is determined by the following method: The cloud coverage pixel value range is determined according to actual needs, cloud coverage pixels are screened out from the cloud coverage optical image to form an area, and then the proportion of the cloud coverage area to the entire cloud coverage optical image is calculated, and the cloud density level corresponding to the proportion value is determined according to the preset level range.

3. The density-aware cloud removal method according to claim 1, characterized in that: The cloud density perception module in step S2 includes a feature expansion module and a classification module, wherein: The feature expansion module is used to extract the cloud cover optical image feature G opt Flatten to obtain the two-dimensional feature g opt And sent to the classification module; The classification module is used to classify the two-dimensional feature g opt Classify and use the obtained cloud density category as the cloud density level G label .

4. The density-aware cloud removal method according to claim 1, characterized in that: The density-aware global context interaction module in step S2 includes a radar image processing module, an optical image processing module and a global-local fusion transformation module, wherein: The radar image processing module is used to process the synthetic aperture radar image features according to the input Processing to obtain synthetic aperture radar image features The radar image processing module includes K layers of radar image processing units, wherein the jth layer of radar image processing unit is used to process the input synthetic aperture radar image features. Processing to obtain synthetic aperture radar image features Where j = 1, 2, ..., K-1, The first K-1 layer radar image processing unit will obtain the synthetic aperture radar image features Output to the next layer of radar image processing unit, j′=1,2,…,K-1, the Kth layer radar image processing unit will obtain the synthetic aperture radar image features Send it to the global-local fusion transformation module and transform the features As synthetic aperture radar image features Output; Each radar image processing unit includes a convolution layer, a Relu activation layer, a Swin Transformer layer and an overlay layer, where: The convolutional layer is used to transform the input synthetic aperture radar image features. Perform convolution operation and send the obtained features to the Relu activation layer; The Relu activation layer is used to process the received features using the Relu activation function. Send to Swin Transformer layer; The Swin Transformer layer is used to use a moving window self-attention mechanism to For processing, the specific method of the self-attention mechanism is: Using weight matrix The received features Processing is performed to obtain the query features of the synthetic aperture radar image Key Features Sum value characteristics Compute the transformation matrix of a synthetic aperture radar image And sent to the j-th optical image processing unit in the optical image processing module: Where d is the number of rows of the query feature, and the superscript T represents transposition. Represents a learnable relative position encoding; Then use the Softmax function to transform the matrix Processing to obtain features And send to the overlay: Overlays are used to map synthetic aperture radar image features With features Superposition is performed to obtain the characteristics And output; The optical image processing module is used to process the cloud cover optical image features according to the input Processing to obtain cloud cover optical image features The optical image processing module includes K layers of optical image processing units, wherein the jth layer of optical image processing units is used to process the cloud cover optical image features according to the input Processing to obtain cloud cover optical image features in The optical image processing unit of the first K-1 layer will obtain the cloud cover optical image features Output to the next layer of optical image processing unit, j′=1,2,…,K-1, the Kth layer optical image processing unit will obtain the cloud cover optical image features Output to the global-local fusion transformation module; each optical image processing unit includes a convolution layer, a Relu activation layer, a SwinTransformer layer and an overlay layer, where: The convolutional layer is used to transform the input cloud cover optical image features Perform convolution operation and send the obtained features to the Relu activation layer; The Relu activation layer is used to process the received features using the Relu activation function. Send to Swin Transformer layer; The Swin Transformer layer is used to use a moving window self-attention mechanism to To process, the specific method of the self-attention mechanism is: Using weight matrix The received features Processing is performed to obtain the query features of the cloud cover optical image Key Features Sum value characteristics Compute the transformation matrix of cloud-covered optical images in, The representation represents a learnable relative position encoding; Then the gating function is used to obtain the cloud cover optical image transformation matrix guided by the synthetic aperture radar image Among them, ⊙ represents the multiplication of corresponding elements, and G() represents the gating function; Then use the Softmax function to transform the matrix Processing to obtain features And send to the overlay: Overlays are used to cover cloud optical image features With features Superposition is performed to obtain the characteristics And output; The global-local fusion transformation module is used to Features Processing to obtain cloud cover optical image characteristics And output, the specific method is: Features Feature extraction is performed to obtain feature g, and then the segmentation function split() is used to segment feature g to obtain the scaling matrix γ and the offset matrix β: (γ,β)=split(g) Then the cloud cover optical image characteristics are obtained using the following formula 5. The density-aware cloud removal method according to claim 1, characterized in that: The loss function LOSS of the cloud removal model training in step S3 is calculated using the following method: LOSS=λ gen L gen +λ cla L cla Among them, L gen represents the generation loss of cloud-free optical images, L cla represents the estimated loss in cloud density level, λ gen , cla They represent the weights of the corresponding losses respectively.

6. The density-aware cloud removal method according to claim 5, characterized in that: The generation loss of the cloud-free optical image L gen The calculation formula is as follows: in, represents a true cloud-free optical image, and L1() represents the L1 norm.

7. The density-aware cloud removal method according to claim 5, characterized in that: The estimated loss of cloud density level L cla Cross entropy loss is used.

8. The density-aware cloud removal method according to claim 5, characterized in that: The weight λ gen and λ cla The calculation method is: The loss ratio r is calculated using the following formula: Among them, ε represents a minimum value; Then the weight λ is calculated using the following formula gen and λ cla :

Images