A remote sensing cloud removal method and system integrating multi-temporal and multi-modal

Abstract

The application discloses a remote sensing cloud removal method and system integrating multi-time sequence and multi-modal, and belongs to the technical field of remote sensing image cloud removal. Single-time sequence, multi-modal and multi-time sequence image data are obtained from a public data set, and forward diffusion and reverse diffusion processes are constructed. Image degradation is simulated by a state regression stochastic differential equation, cloud image structure information is retained, and a denoising network is used to gradually restore cloud-free images. A multi-time sequence denoising network extracts cross-time sequence features through a shared weight encoder, a grouped channel self-attention, a window self-attention and a time fusion attention module, and enhances the denoising effect. A preconditioning technique and a fast sampling method based on data prediction are further introduced to adaptively scale the image and accelerate reverse sampling, thereby improving the calculation efficiency and sampling accuracy. In the training process, the performance of the test set is quantitatively analyzed, and when the peak signal-to-noise ratio (PSNR), the structural similarity index (SSIM), the mean absolute error (MAE), the spectral angle mapping (SAM) and the learning perceptual image block similarity (LPIPS) reach preset thresholds, a cloud removal model is obtained. The method and system can efficiently remove clouds under single-time sequence, multi-modal and multi-time sequence tasks, restore clear remote sensing images, and significantly improve the image quality and detail retention effect.

Description

Technical Field

[0001] This application relates to the field of remote sensing image cloud removal technology, and in particular to a remote sensing cloud removal method and system that integrates multi-temporal and multi-modal approaches. Background Technology

[0002] Optical remote sensing images possess high resolution, stable geometric features, and clear spatial texture information, enabling them to provide accurate geospatial data for target areas. This makes remote sensing images of significant application value in fields such as urban planning, environmental monitoring, agriculture, forestry, disaster assessment, and climate analysis. However, cloud cover in remote sensing images inevitably affects image quality. Cloud removal is an essential preprocessing step in the application of optical remote sensing images. Currently, scholars both domestically and internationally have proposed various processing methods to address the problem of cloud contamination in remote sensing images, which can be mainly divided into two categories: traditional methods and deep learning methods. Traditional methods rely on detecting the statistical relationship between clouded and cloudless images, or estimating parameters related to physical models. However, under changing and complex weather conditions, the inherent assumptions of these methods do not always match real-world scenarios, resulting in performance inferior to deep learning-based techniques and thus poorer cloud removal results. In recent years, deep learning technology has developed rapidly. These methods utilize deep neural networks to model the complex relationship between clouded and cloudless images, thereby achieving more accurate cloud removal.

[0003] However, existing deep learning methods lack effective multi-temporal processing capabilities, and their cloud removal process fails to adequately preserve image details, resulting in instability and suboptimal image quality. Furthermore, their sampling efficiency is relatively low, and they struggle with complex cloud structures and multimodal, multi-temporal data. Summary of the Invention

[0004] This invention addresses the technical problems existing in the background art by proposing a remote sensing cloud removal method and system that integrates multiple time series and multiple modes.

[0005] To solve the technical problem, the technical solution of the present invention is as follows:

[0006] A remote sensing cloud removal method integrating multi-temporal and multi-modal approaches, the method comprising:

[0007] S1: Construct forward diffusion and backward diffusion processes for the three selected image data. In the forward diffusion process, the cloudless image is gradually perturbed into a noisy cloud image through state regression stochastic differential equations while maintaining the cloud image structure information. In the backward diffusion process, the noisy image obtained by forward diffusion is gradually restored through a denoising network to obtain a preliminary denoised image sequence.

[0008] S2: Construct a multi-temporal denoising network for the initial denoised image sequence obtained in S1. This network includes a shared weight encoder, a grouped channel self-attention module, a window self-attention module, and a temporal fusion attention module. It extracts cross-temporal information from the multi-temporal images and enhances the denoising effect, generating an optimized denoised image sequence.

[0009] S3: Perform preconditioning and fast sampling based on data prediction on the optimized denoised image sequence obtained in S2, adaptively scale the input image, and use the constant variation method to solve the linear and semi-linear equations to accelerate back diffusion, so as to obtain the final denoised image sequence.

[0010] S4: Using the final denoised image sequence obtained in S3 as training input, iteratively train the multi-temporal denoising network and perform quantitative analysis on the test set. When the quantitative index of the test set reaches the preset accuracy, save the network parameters to obtain the finally trained cloud removal model.

[0011] S5: Apply the final trained cloud removal model obtained in S4 to actual remote sensing images to generate a clear sequence of cloudless images.

[0012] Furthermore, three types of image data were selected: training and test sets were selected from the public RICE1 and RICE2 datasets to form single-time-series image data; training and test sets were selected from the public SEN12MS-CR dataset to form multimodal image data; and training and test sets were selected from the public Sen2 MTC New dataset to form multi-time-series image data.

[0013] Furthermore, the forward diffusion process in S1 includes:

[0014] S11: Initial cloud-free image given Compared with cloud image mean By introducing a mean regression term into the drift and diffusion functions, the image state is... The image is gradually perturbed over time, and noise is added to make it gradually approach the mean of the cloud image. Its state changes satisfy the state regression stochastic differential equation:

[0015]

[0016] in, For time-related regression parameters, Where is the diffusion coefficient. For Brownian motion increments;

[0017] S12: Image state during forward diffusion. It can be represented as:

[0018]

[0019] in, To control the hyperparameters of noise intensity, It is random noise. For time-related mean regression parameters, Represents the instantaneous moment during integration;

[0020] S13: Noise image obtained from forward diffusion during the reverse diffusion process. Starting from the initial state, the image structure is gradually restored by denoising, resulting in a preliminary denoised image. Its evolution satisfies the ordinary differential equation:

[0021]

[0022] in, For noise level, It is the diffusion coefficient, which controls the injection intensity of noise; The formula representing the noise prediction of the denoising network is:

[0023]

[0024] in, It is in time At any time on the image Noise prediction, It is a fractional function used in fractional matching methods to estimate the distribution of data;

[0025] S14: Inverse Denoising Network The training objective is to minimize the loss function:

[0026]

[0027] in, It is Gaussian noise. As a conditional input, the loss function enables the denoising network to recover a result close to the original cloudless image. It is the image denoised during the reverse process. It is a noise reduction network. Gaussian noise is used to simulate noise disturbances and improve network robustness. The noise level at the current time step is used as one of the network inputs; the outer layer expectation... The average of all samples in the data distribution is calculated to ensure that the network generalizes across the entire training set; inner layer expectation. Random noise is added to each sample to enhance the network's robustness to noise; minimizing this loss function makes the denoising network... The prediction at each time step should be as close as possible to the original cloudless image. Through repeated iterative reverse denoising processes, a complete and clear cloudless image is finally restored.

[0028] Furthermore, the multi-temporal denoising network in S2 includes the following modules:

[0029] S21: Shared weight encoder module, which encodes each frame of the input multi-temporal image sequence and extracts spatial features, wherein the encoder parameters are shared across all time steps to generate preliminary feature representations across time.

[0030] S22: Grouped Channel Self-Attention Module. This module divides the channel features output by the encoder into multiple groups and calculates channel self-attention for each group, thereby enhancing the interrelationship between channels. The formula for calculating channel attention is:

[0031]

[0032] in, For input features, For the first Group channel characteristics, For channel weighting functions;

[0033] S23: Window self-attention module, introducing a mask into the attention matrix. To restrict the query location to interact only with key / value locations that meet the conditions, the calculation formula is:

[0034]

[0035] Q (Query): Query matrix, representing the query vector for each pixel / location currently being processed; K (Key): Key matrix, the key vector for each pixel / location, used to measure the similarity to the query; V (Value): Value matrix, the value vector for each pixel / location, containing the information to be fused. Calculate the dot product similarity between the query and the key to obtain the attention score matrix; : Scaling factor, where d is the vector dimension, to prevent the dot product value from being too large, which could cause the Softmax gradient to vanish or become unstable;

[0036] Mask matrix Based on the query and key / value location coordinate difference settings, the interaction range under blind spot network conditions is ensured; among them, the mask matrix The formula is as follows:

[0037]

[0038] in, and These are the coordinates of the query and key / value positions;

[0039] S24: Temporal fusion attention module, which fuses image features from different time steps using the generated temporal attention mask to generate a weighted feature map, the formula of which is:

[0040]

[0041] in, To output the jump feature map, For the first Attention mask at each time step, To indicate different heads, Encoder features;

[0042] S25: Decoding module, receives the feature map output by the temporal fusion attention module, and restores it into an optimized cloudless image sequence through a reverse denoising process.

[0043] Furthermore, the preconditioning and rapid sampling based on data prediction in S3 includes the following steps:

[0044] S31: Preconditioning, adaptively scaling the image sequence output by the multi-temporal denoising network, with scaling factors including:

[0045]

[0046] in, The noise level of the target image. The noise level of the cloud image. The covariance between the target image and the cloud image. The length of the image sequence. and For hyperparameters, Indicates the noise level at the current time step;

[0047] S32: Fast sampling processing, solving linear and semi-linear equations using the method of variation of constants to accelerate the backdiffusion process and image status. The calculation formula is:

[0048]

[0049] in, This refers to the image state at the previous time step. , For noise level, , For diffusion parameters, For denoising network prediction, It is random noise. Mean value of cloud image;

[0050] S33: The image sequences processed by S41 and S42 are used as input to the final denoised image sequence for training and testing the multi-temporal denoising network.

[0051] Furthermore, the training and quantitative analysis in S4 includes the following steps:

[0052] S41: Use the final denoised image sequence obtained in S4 as the training input to iteratively train the multi-temporal denoising network, and perform quantitative analysis using the test set after each training round.

[0053] S42: Quantitative analysis metrics include Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), Mean Absolute Error (MAE), Spectral Angle Mapping (SAM), and Learned Perceptual Image Patch Similarity (LPIPS), among which:

[0054] The formula for calculating PSNR is:

[0055]

[0056] in, For the original image, To restore the image, and These are the image width and height, respectively. The maximum pixel value. Indicates row and column indexes;

[0057] The formula for calculating SSIM is:

[0058]

[0059] in, It is a local mean. For local variance, For covariance, It is the stability constant. and It is a partial window for the original image and the restored image.

[0060] The formula for calculating MAE is:

[0061]

[0062] in, For predicted values, For the true value, For the number of data points;

[0063] The formula for calculating SAM is:

[0064]

[0065] in, For the true spectral vector, To reconstruct the spectral vector; Let be the Euclidean norm of the vector.

[0066] The formula for calculating LPIPS is:

[0067]

[0068] in, and For deep networks Layer features, For network layers, Indicates Euclidean distance;

[0069] S43: When the quantitative analysis index on the test set reaches the preset threshold, save the parameters of the multi-temporal denoising network to obtain the finally trained cloud removal model.

[0070] A remote sensing cloud removal system integrating multi-temporal and multi-modal methods, the system being applied to any of the methods described above, the system comprising:

[0071] The data acquisition module is configured to acquire training and test sets from the public RICE1 and RICE2 datasets to form single-time-series image data, acquire training and test sets from the public SEN12MS-CR dataset to form multimodal image data, and acquire training and test sets from the public Sen2 MTC New dataset to form multi-time-series image data.

[0072] The mean regression diffusion construction module is configured to construct forward diffusion and reverse diffusion processes for the image data acquired by the data acquisition module. The forward diffusion process gradually perturbs the cloudless image into a noisy cloud image while maintaining the cloud image structure information through a state regression stochastic differential equation. The reverse diffusion process gradually restores the image structure through a denoising network to generate a preliminary denoised image sequence.

[0073] The multi-temporal denoising network construction module is configured to process the preliminary denoised image sequence generated by the mean regression diffusion construction module, including a shared weight encoder module, a grouped channel self-attention module, a window self-attention module and a temporal fusion attention module, to extract cross-temporal information and enhance the denoising effect, and output an optimized denoised image sequence.

[0074] The preconditioning and fast sampling module is configured to adaptively scale the image sequence output by the multi-temporal denoising network construction module, and accelerate back diffusion by solving the linear and semi-linear equations using the constant variation method to obtain the final denoised image sequence.

[0075] The training and validation module is configured to iteratively train a multi-temporal denoising network using the image sequences obtained by the preconditions and the fast sampling module as input, and perform quantitative analysis on the test set. When the quantitative analysis index reaches a preset threshold, the network parameters are saved to form the final trained cloud removal model.

[0076] This application has the following advantages:

[0077] This invention, by introducing a forward state diffusion process and a mean regression mechanism, effectively preserves structural information in cloud images and maintains pixel-level consistency during cloud removal, thereby significantly improving cloud removal accuracy and image quality. The multi-temporal denoising network can handle dynamic cloud patterns in multi-temporal and multi-modal data, ensuring excellent cloud removal results even in cloud images with significant temporal variations. Fast sampling technology greatly reduces computational costs and improves processing efficiency. Ultimately, the technical solution provided by this invention not only has strong universality, adapting to different types of cloud layers and complex scenes, but also demonstrates high stability and reliability in practical applications, offering a more efficient and accurate solution for cloud removal tasks in remote sensing images. Attached Figure Description

[0078] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0079] Figure 1 This is a flowchart of the remote sensing image cloud removal method based on the mean regression diffusion model in an embodiment of the present invention;

[0080] Figure 2 This is a structural diagram of the mean regression diffusion model constructed in this embodiment of the invention.

[0081] Figure 3 This is a diagram of the multi-temporal denoising network structure constructed in this embodiment of the invention;

[0082] Figure 4 This is a structural diagram of the self-attention module for grouped channels in an embodiment of the present invention;

[0083] Figure 5 This is a structural diagram of the window self-attention module in an embodiment of the present invention;

[0084] Figure 6 This is a structural diagram of the temporal attention fusion module in an embodiment of the present invention;

[0085] Figure 7This is a visualization of the thin cloud before and after cloud removal in an embodiment of the present invention;

[0086] Figure 8 This is a visualization of the thick cloud before and after cloud removal in an embodiment of the present invention;

[0087] Figure 9 These are visualizations of the multimodal cloud removal process before and after in this embodiment of the invention.

[0088] Figure 10 This is a visualization of the effects of cloud removal before and after multiple time series in an embodiment of the present invention. Detailed Implementation

[0089] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0090] Example 1:

[0091] This embodiment proposes a remote sensing cloud removal method integrating multi-temporal and multi-modal approaches, implemented based on a mean regression diffusion model. The specific implementation steps of this method are as follows:

[0092] S1: Select data from the public RICE1 and RICE2 datasets as training and testing sets for single-time-series cloud removal tasks; select data from the public SEN12MS-CR dataset as training and testing sets for multimodal cloud removal tasks; select data from the public Sen2 MTC New dataset as training and testing sets for multi-time-series cloud removal tasks.

[0093] S2: The forward diffusion and backward diffusion processes are constructed using state regression stochastic differential equations and ordinary differential equations;

[0094] S3: Introduce a multi-temporal denoising network as a denoising module for the mean regression diffusion model;

[0095] S4: Improve the training and sampling process through preconditioning techniques and fast sampling methods based on data prediction;

[0096] S5: Train the remote sensing image cloud removal network using the training set and perform quantitative analysis using the test set to evaluate the model's performance in different cloud removal scenarios. When the remote sensing image cloud removal method reaches the preset accuracy on the test set, the final trained remote sensing image cloud removal method is obtained.

[0097] S6: Apply the final remote sensing image cloud removal method to practice to achieve the purpose of removing clouds from remote sensing images.

[0098] Furthermore, in step S2, the specific steps for constructing the forward diffusion and backward diffusion processes using state regression stochastic differential equations and ordinary differential equations are as follows:

[0099] S21: The forward diffusion process is initially given an image containing cloudless data. Heyun Image Furthermore, mean regression terms are introduced into the drift and diffusion functions. The target image is then progressively perturbed, introducing noise and transforming it into a noisy, cloud-covered image. This process preserves the structural information of the cloud image. Finally, through forward diffusion mean regression and noise control, the image state eventually approximates the mean of the cloud image. This ensures structural consistency between the cloud image and the target image, preparing for the reverse denoising process. Specifically, at the beginning of the forward process, the target image (cloud-free image) and the cloud image are defined. The goal is to denoise the cloud-free image (...). Gradually generate a noisy, cloud-covered image. This process is achieved through a state-regression stochastic differential equation. The formula for the diffusion process is:

[0100]

[0101] in, It represents the current image state. It is a cloud image (i.e., a mean image). These are time-dependent regression parameters that control how the image regresses to the mean image. . It is the diffusion coefficient, which controls the intensity of noise injection. This is the increment of standard Brownian motion, representing the randomness of the noise. At this stage, the image state changes from the original cloudless image. Initially, the image gradually regresses to the mean, while noise is gradually introduced during the diffusion process, pushing the image towards the mean of the cloud image.

[0102] During forward diffusion, noise is gradually injected into the image over time, causing the image quality to degrade progressively. This process can be described by the following formula:

[0103]

[0104] in, It is a hyperparameter used to control noise intensity. As time progresses, noise is gradually added to the image, causing it to gradually degenerate into a noisy image. Throughout the diffusion process, the image gradually approaches the mean of the cloud image. Through the mean regression term ( The image retains structural information during degradation and eventually converges to a relatively stable state.

[0105] S22: The back diffusion process is a noise image obtained through forward diffusion. The initial goal was to gradually restore the image's structure through a denoising process and remove clouds, ultimately obtaining a clear, cloudless image. This process is achieved by introducing an ordinary differential equation, the formula of which is:

[0106]

[0107] in, It represents the current image state. It is the mean image of the cloud image. These are time-dependent regression parameters that control how the image regresses to the mean image. . It is the diffusion coefficient, which controls the injection intensity of noise. This is the noise prediction learned in the score matching model, representing the prediction value of the denoising network. Its formula is:

[0108]

[0109] in, It is in time At any time on the image Noise prediction. It is a fractional function used in fractional matching methods to estimate the distribution of data.

[0110] Next, the denoising network ( The role of the denoising network is to predict the denoising result at each time step. In the reverse process, the denoising network adjusts the noise level of the current image based on the noise level (as determined by...). Control) and image input ( The denoising network is optimized using condition c to ultimately recover a clear image. The training process involves minimizing the loss function. The training objective is to enable the denoising network to recover the original cloudless image from a noisy image. The formula for the loss function is as follows:

[0111]

[0112] in, It is an image denoised during the reverse process. It's added noise. It is a denoising network that attempts to predict the denoising result at each time step. The noise level at the current time is used as one of the inputs to the network. The goal of this loss function is to make the denoising network as close as possible to a cloudless image during training. Finally, the reverse process gradually recovers the target cloudless image through repeated iterations of denoising steps.

[0113] Furthermore, in step S3, the specific process of constructing the multi-temporal denoising network is as follows:

[0114] First, the multi-temporal denoising network starts with a set of consecutive image sequences (i.e., multi-temporal images), progressively removing noise from each frame and improving the denoising effect through temporal fusion. Then, through a shared-weight encoder, a grouped channel self-attention module, a window self-attention module, and a temporal fusion module, the multi-temporal images are processed to extract cross-temporal information to enhance the denoising effect. Finally, the network uses the generated attention mask to fuse the images from each temporal step and recovers a clear, cloud-free image through a decoder. Specifically, the input to the multi-temporal denoising network is a set of consecutive image frames, which typically consist of cloud-laden images from multiple time points. For multi-temporal images, the multi-temporal denoising network designs a shared-weight encoder to process each frame, thereby extracting spatial and temporal information from the images at multiple time points. Assume there are... A series of consecutive image frames, each image being... Then, all input image frames are processed by a weighted encoder, and its parameters are shared across all time steps. The shared weight design allows the model to learn shared features across time sequences and reduces the number of parameters during training. Specifically, at each time step... Image The same encoder is provided for processing to obtain the corresponding encoded feature representation.

[0115] The grouped channel self-attention module divides the channels into multiple groups and performs self-attention calculations on each group separately. The number of channels in each group is kept less than the spatial resolution to ensure that the calculation of one group does not affect the spatial information of other groups. Weighted summation of the input features enhances the interrelationships between channels. Assume the input features are... ,in It's about height. It's the width. This refers to the number of channels. The formula for calculating channel attention is:

[0116]

[0117] in, It is the channel weighting function, and it is the first... Group channels, and This represents the concatenation of features across all groups. This grouping strategy ensures that each group has a certain number of channels. Smaller than spatial resolution This avoids blind spot information leakage caused by excessive interaction between channel information and spatial information.

[0118] The window self-attention module adds a mask to the attention matrix. This limits the scope of interaction between the query and the key / value pair. Specifically, the query position only interacts with key / value pairs that have even coordinate relationships, thus simulating the behavior of dilated convolution. Mask matrix The formula is as follows:

[0119]

[0120] in, and These are the coordinates of the query and key / value positions. If the difference between the coordinates of the query and key / value positions is even, the mask value is 0, indicating that interaction between the query and key / value is possible; otherwise, the mask value is negative infinity, indicating that interaction between the query and key / value is blocked. The corresponding window self-attention module calculation formula is:

[0121]

[0122] Among them, the mask matrix The attention matrix is ​​adjusted so that each query location can only interact with matching key / value locations. This mechanism ensures that the interaction range between queries and key / value pairs is strictly limited in the blind spot network, thus maintaining the blind spot requirement while expanding the receptive field.

[0123] Finally, the temporal fusion attention module uses the generated temporal attention mask to fuse images from different time steps. The formula is as follows:

[0124]

[0125] in: It is the output jump feature map, which is obtained from the weighted sum of all temporal input images. It is the first Attention mask generated at each time step To indicate different heads. These are feature maps from the encoder. Finally, the decoder reconstructs a clear, cloudless image from the fused feature maps. The decoder receives attention-weighted feature maps from each time step and denoises them through a reverse process, ultimately reconstructing the cloudless image.

[0126] Furthermore, in step S4, the specific process of improving training and sampling using preconditioning techniques and a fast sampling method based on data prediction is as follows: First, the input image is scaled using a preconditioning module to ensure that the noise level of the image is consistent with the network's expectations. Then, a variational constant method and a simplified sampling method are introduced to accelerate the backdiffusion process, reduce computational costs, and maintain high-quality cloud removal results. Specifically, four factors are introduced. Adaptive scaling of the input and output is performed, and these factors are controlled by four hyperparameters: the variance of the target image, etc. Variance of cloud images Covariance between target image and cloud image and sequence length The formula is:

[0127]

[0128] in: It represents the noise level of the target image (cloud-free image). It represents the noise level of the cloud image. It is the covariance between the target image and the cloud image.

[0129]

[0130]

[0131] Where: is the length of the time-series image (i.e., the length of the image sequence).

[0132]

[0133] The method of variation of constants is used to solve equations of linear and semi-linear structures, thereby quickly finding solutions to ordinary differential equations and improving sampling efficiency. The formula is as follows:

[0134]

[0135] Further, in step S5, the remote sensing image cloud removal network is trained using the training set. After each generation of training, the performance on the test set is compared. When the quantitative analysis results on the test set reach the preset accuracy, i.e., when evaluation indicators such as PSNR, SSIM, MAE, SAM, or LPIPS reach the preset threshold, the training model parameters are saved, and the loss function is constructed.

[0136]

[0137] in, Indicates time steps Noisy images on the screen. It is Gaussian noise. It's a denoising network whose goal is to predict the original cloudless image. . It is the standard deviation of noise. It is a conditional input (e.g., cloud image and optional auxiliary modal image).

[0138] Furthermore, in step S5, a test set is used to test the cloud removal effect of the network. The evaluation indicators for quantitative analysis mainly include peak signal-to-noise ratio, structural similarity index, mean absolute error, spectral angle mapping, and learned perceptual image patch similarity.

[0139] Peak Signal-to-Noise Ratio (PSNR) is a traditional metric for measuring the quality of image restoration, typically used to evaluate the quality of images after compression, denoising, and other processing. A higher PSNR value indicates a more similar restored image to the original image. Its calculation formula is:

[0140]

[0141] in: It is the maximum pixel value of the image, usually 255 (for 8-bit images). It is the mean squared error, and its formula is:

[0142]

[0143] in, It is the original image. This is the restored image. and These are the width and height of the image, respectively.

[0144] SSIM assesses the similarity between two images by comparing the structural information between the original and reconstructed images. Values ​​range from 0 to 1, with values ​​closer to 1 indicating greater similarity. The formula is:

[0145]

[0146] in: and It is a local window for the original image and the restored image. and yes and The mean. and yes and The variance. yes and The covariance. and It is a stability constant used to avoid the case where the denominator is zero.

[0147] MAE is the average of the absolute differences between predicted and actual values. Its formula is:

[0148]

[0149] in: This is a predicted value. It is the actual value. It represents the number of data points.

[0150] SAM measures the angular difference between pixels in an image; a smaller value indicates a higher similarity. The formula is:

[0151]

[0152] in: and These are two images at the 1st... The pixel values ​​of each band. It is the number of bands in the image.

[0153] LPIPS measures the similarity between two images by calculating their distance in the feature space of a deep neural network. Generally, the lower the LPIPS value, the more similar the images are. The formula is:

[0154]

[0155] in: and Through deep networks in the first Features extracted from layers. It refers to the number of layers in the network. This represents the Euclidean distance.

[0156] Example 2:

[0157] This embodiment applies to Embodiment 1 and provides a specific application scenario for a remote sensing cloud removal method that integrates multiple time series and multiple modalities. The application scenario includes the following steps:

[0158] S1: The public RICE1 dataset is primarily used to train the network's ability to remove thin clouds, while the public RICE2 dataset is primarily used to train the network's ability to remove thick clouds. The RICE1 dataset contains 500 data samples, each with a cloudy image and a cloudless image at a resolution of 512×512. These are used as the training and testing sets for this method in an 8:2 ratio, and the images are resized from 512×512 pixels to 256×256 pixels. Subsequently, the pixel values ​​are rescaled to the range [−1, 1].

[0159] The public SEN12MS-CR dataset was primarily used to train the network's multimodal cloud removal capabilities. It contains co-registered multispectral optical images from 13 bands of the Sentinel-2 satellite and SAR images from 2 bands of the Sentinel-1 satellite. The dataset was collected from 169 non-overlapping Regions of Interest (ROIs) across continents, each averaging approximately 52 × 40 km². ROI scenes were segmented into 256 × 256 pixel tiles with 50% spatial overlap. 114,050 images were used for training, 7,176 for validation, and 7,899 for testing. Pixel values ​​for SAR and optical images were cropped to the ranges of [−25, 0] and [0, 10000], respectively. However, pixel values ​​for all images were rescaled to the range of [−1, 1] to achieve symmetric pixel values.

[0160] The public Sen2 MTC New dataset was primarily used to train the network's multi-temporal cloud removal capabilities. It contains co-registered RGB and infrared images across approximately 50 non-overlapping patches. Each patch consists of approximately 70 pairs of cropped 256 × 256 pixel patches with pixel values ​​ranging from 0 to 10,000. The dataset was split into 2,380 training images, 350 validation images, and 687 test images. The pixel values ​​of the images were first scaled to the range [0,1] by dividing by 10,000, and then normalized using a mean of 0.5 and a standard deviation of 0.5. Data augmentation in the training set split included random flipping and 90-degree rotation of every four images.

[0161] S2: As Figure 2 As shown, forward diffusion and backward diffusion processes are constructed using state regression stochastic differential equations and ordinary differential equations.

[0162] S21: In the forward process based on the mean regression diffusion model, the original image Noise is gradually added, eventually generating a noisy image. First, the original image... And noise diffuses through the standard deviation at each step of diffusion Weighting is applied. This is done as the time step increases. As noise increases, the image gradually becomes blurry until it becomes a completely noisy image. Formation. In each diffusion step, the relationship between the image and noise is established through multiplication and the standard deviation function. To adjust, among which The intensity of the noise is controlled. The entire diffusion process will be repeated. Each time, more noise is added until the image is completely noisy. This process is achieved gradually through a diffusion step, ensuring a smooth transition of noise and a final state of complete noise.

[0163] Specifically, at the beginning of the forward process, a target image (cloud-free image) and a cloud image are defined. The target is to obtain the cloud-free image ( Gradually generate a noisy, cloud-covered image. This process is achieved through a state-regression stochastic differential equation. The formula for the diffusion process is:

[0164]

[0165] in, It represents the current image state. It is a cloud image (i.e., a mean image). These are time-dependent regression parameters that control how the image regresses to the mean image. . It is the diffusion coefficient, which controls the intensity of noise injection. This is the increment of standard Brownian motion, representing the randomness of the noise. At this stage, the image state changes from the original cloudless image. Initially, the image gradually regresses to the mean, while noise is gradually introduced during the diffusion process, pushing the image towards the mean of the cloud image.

[0166] During forward diffusion, noise is gradually injected into the image over time, causing the image quality to degrade progressively. This process can be described by the following formula:

[0167]

[0168] in, It is a hyperparameter used to control noise intensity. As time progresses, noise is gradually added to the image, causing it to gradually degenerate into a noisy image. Throughout the diffusion process, the image gradually approaches the mean of the cloud image. Through the mean regression term ( The image retains structural information during degradation and eventually converges to a relatively stable state.

[0169] S22: In the reverse process based on the mean regression diffusion model, first from the noisy cloud image Initially, the original image was gradually recovered through step-by-step denoising. The reverse process removes noise using ordinary differential equations and restores the image using a denoiser. At each step, the denoiser processes the current noisy image, gradually restoring it to a state closer to the original image. The reverse process needs to be repeated. The process is repeated step-by-step to remove noise and restore a clear image. Finally, the reverse process outputs the denoised image. It will gradually approach the original noise-free image. This process is achieved through multiple iterations and gradual denoising, ensuring that noise is effectively removed and image details are restored;

[0170] Specifically, the backdiffusion process is the noise image obtained through forward diffusion. The initial goal was to gradually restore the image's structure through a denoising process and remove clouds, ultimately obtaining a clear, cloudless image. This process is achieved by introducing an ordinary differential equation, the formula of which is:

[0171]

[0172] in, It represents the current image state. It is the mean image of the cloud image. These are time-dependent regression parameters that control how the image regresses to the mean image. . It is the diffusion coefficient, which controls the injection intensity of noise. This is the noise prediction learned in the score matching model, representing the prediction value of the denoising network. Its formula is:

[0173]

[0174] in, It is in time At any time on the image Noise prediction. It is a fractional function used in fractional matching methods to estimate the distribution of data.

[0175] Next, the denoising network ( The role of the denoising network is to predict the denoising result at each time step. In the reverse process, the denoising network adjusts the noise level of the current image based on the noise level (as determined by...). Control) and image input ( The denoising network is optimized using condition c to ultimately recover a clear image. The training process involves minimizing the loss function. The training objective is to enable the denoising network to recover the original cloudless image from a noisy image. The formula for the loss function is as follows:

[0176]

[0177] in, It is an image denoised during the reverse process. It's added noise. It is a denoising network that attempts to predict the denoising result at each time step. The noise level at the current time is used as one of the inputs to the network. The goal of this loss function is to make the denoising network as close as possible to a cloudless image during training. Finally, the reverse process gradually recovers the target cloudless image through repeated iterations of denoising steps.

[0178] S3: As Figure 3 As shown, a multi-temporal denoising network is constructed as a denoising module for the mean regression diffusion model;

[0179] Specifically, in the architecture of the denoising neural network, the entire processing is divided into multiple modules to gradually remove noise from the image and restore a clear image. First, the network receives three inputs: a noisy cloud image, the original cloud image, and auxiliary inputs. These inputs provide different information and context about the image, helping the model better understand and process noise during the denoising process.

[0180] The input image is first segmented into multiple small patches using Patchify. Then, the Embedding operation transforms each image patch into a feature vector suitable for network processing, facilitating subsequent neural network operations. The purpose of this step is to transform the image into a form that is easier to process and understand.

[0181] The image feature maps are processed through a series of HDiT Blocks (High-Dimensional Transformer Blocks). HDiT Blocks 1 to 5 are used to extract temporal features from the image, capturing long-range dependencies through a self-attention mechanism. Each HDiT Block calculates and adjusts weights based on the input image features, thereby extracting a more accurate image representation. To enhance the features, the network merges different features and adjusts their dimensions through a Merge + Proj. operation to accommodate subsequent processing.

[0182] In temporal encoders, shortcut connections are used to directly pass the output of certain layers to subsequent layers, which helps avoid gradient vanishing and accelerates model convergence. Attention masks, on the other hand, help focus on important parts of the image during model training. By assigning different weights to different regions, the model can more effectively handle noise and restore image details.

[0183] The encoded feature maps are fed into the decoder, where the network progressively transforms them into a denoised image. The decoder's role is to gradually restore the image's details and generate the final output image. At the end of image processing, the bottleneck module further compresses and optimizes the feature maps, helping the model integrate information more effectively. This module includes an additional temporal fusion attention module (…). Figure 6The network is specifically designed to handle temporal information, helping it better understand the temporal dependencies in images and improving denoising performance. Throughout the process, multiplication (×) and element-wise addition (⊕) are widely used for feature map weighting, fusion, and feature updates. The mean operator is used to calculate the mean of the feature maps, smoothing the image or providing global statistical information. Furthermore, the network iteratively denoises by repeating the denoising steps N-1 times, with each step gradually approaching the final noise-free image.

[0184] Specifically, the input features of the group channel self-attention module It is a shape of The tensor, in which and It refers to the spatial dimensions of the image (height and width), while It refers to the number of channels. This input feature will be processed by two modules: grouped channel self-attention (...). Figure 4 Self-attention with masked window Figure 5 These two modules are responsible for extracting local and global information, respectively.

[0185] Before performing self-attention calculation, the input features It will depend on the number of channels Classified as There are 1 group, and each group contains 1 group. There are 10 channels. This means that the channels are divided into multiple smaller groups to prevent information leakage. The shape of the feature tensor after each grouping is... Each channel group is processed individually using dilated convolution, an operation designed to increase the receptive field while preserving local information. The outputs of these convolutions are query (Q), key (K), and value (V) matrices. Weighted values ​​for each position are calculated using inter-channel self-attention. The formula for calculating self-attention is as follows:

[0186]

[0187] in This is the feature dimension, used to scale the dot product result. In this way, G-CSA performs weighted aggregation of features within each channel group, enhancing the interaction of global information while avoiding the spatial information leakage problem that may occur in multi-scale architectures.

[0188] The masked window self-attention module first transforms the input features through three 1×1 convolutions. The purpose of the convolution operation is to process the features at each spatial location, transforming them into query (Q), key (K), and value (V). To meet the requirements of blind spot networks, M-WSA introduces a fixed mask matrix 𝑀 to restrict the interaction between the query and the key / value pair. This mask ensures that the query only interacts with keys / value pairs that have even coordinate differences, simulating the receptive field expansion effect of dilated convolution. The design of the mask matrix 𝑀 is as follows:

[0189]

[0190] In this way, the mask matrix blocks invalid interactions between queries and key / value pairs, retaining only valid interactions. The attention calculation formula adjusted by the mask matrix is:

[0191]

[0192] In step S4, the specific process of improving training and sampling using preconditioning techniques and a fast sampling method based on data prediction is as follows: First, the input image is scaled using a preconditioning module to ensure that the noise level of the image is consistent with the network's expectations. Then, a variational constant method and a simplified sampling method are introduced to accelerate the backdiffusion process, reduce computational costs, and maintain high-quality cloud removal results. Specifically, four factors are introduced. Adaptive scaling of the input and output is performed, and these factors are controlled by four hyperparameters: the variance of the target image, etc. Variance of cloud images Covariance between target image and cloud image and sequence length The formula is:

[0193]

[0194] in: It represents the noise level of the target image (cloud-free image). It represents the noise level of the cloud image. It is the covariance between the target image and the cloud image.

[0195]

[0196]

[0197] Where: is the length of the time-series image (i.e., the length of the image sequence).

[0198]

[0199] The method of variation of constants is used to solve equations of linear and semi-linear structures, thereby quickly finding solutions to ordinary differential equations and improving sampling efficiency. The formula is as follows:

[0200]

[0201] In step S5, the remote sensing image cloud removal network is trained using the training set. After each generation of training, the performance on the test set is compared. When the quantitative analysis results on the test set reach the preset accuracy, i.e., when evaluation indicators such as PSNR, SSIM, MAE, SAM, or LPIPS reach the preset threshold, the training model parameters are saved, and the loss function is constructed.

[0202]

[0203] in, Indicates time steps Noisy images on the screen. It is Gaussian noise. It's a denoising network whose goal is to predict the original cloudless image. . It is the standard deviation of noise. It is a conditional input (e.g., cloud image and optional auxiliary modal image).

[0204] Furthermore, in step S5, a test set is used to test the cloud removal effect of the network. The evaluation indicators for quantitative analysis mainly include peak signal-to-noise ratio, structural similarity index, mean absolute error, spectral angle mapping, and learned perceptual image patch similarity.

[0205] Peak Signal-to-Noise Ratio (PSNR) is a traditional metric for measuring the quality of image restoration, typically used to evaluate the quality of images after compression, denoising, and other processing. A higher PSNR value indicates a more similar restored image to the original image. Its calculation formula is:

[0206]

[0207] in: It is the maximum pixel value of the image, usually 255 (for 8-bit images). It is the mean squared error, and its formula is:

[0208]

[0209] in, It is the original image. This is the restored image. and These are the width and height of the image, respectively.

[0210] SSIM assesses the similarity between two images by comparing the structural information between the original and reconstructed images. Values ​​range from 0 to 1, with values ​​closer to 1 indicating greater similarity. The formula is:

[0211]

[0212] in: and It is a local window for the original image and the restored image. and yes and The mean. and yes and The variance. yes and The covariance. and It is a stability constant used to avoid the case where the denominator is zero.

[0213] MAE is the average of the absolute differences between predicted and actual values. Its formula is:

[0214]

[0215] in: This is a predicted value. It is the actual value. It represents the number of data points.

[0216] SAM measures the angular difference between pixels in an image; a smaller value indicates a higher similarity. The formula is:

[0217]

[0218] in: and These are two images at the 1st... The pixel values ​​of each band. It is the number of bands in the image.

[0219] LPIPS measures the similarity between two images by calculating their distance in the feature space of a deep neural network. Generally, the lower the LPIPS value, the more similar the images are. The formula is:

[0220]

[0221] in: and Through deep networks in the first Features extracted from layers. It refers to the number of layers in the network. This represents the Euclidean distance.

[0222] S6: Import the trained model parameters into the network and test the actual cloud removal effect of the network. The resulting visualization comparison before and after cloud removal is shown in the figure below. Figure 7 , Figure 8 , Figure 9 , Figure 10 As shown.

[0223] Example 3:

[0224] This embodiment proposes a remote sensing cloud removal system integrating multi-temporal and multi-modal approaches. The system is used to execute the aforementioned remote sensing cloud removal method integrating multi-temporal and multi-modal approaches. The system includes:

[0225] The training and test set acquisition module selects data from the public RICE1 and RICE2 datasets as training and test sets for single-time-series cloud removal tasks; selects data from the public SEN12MS-CR dataset as training and test sets for multimodal cloud removal tasks; and selects data from the public Sen2 MTC New dataset as training and test sets for multi-time-series cloud removal tasks.

[0226] The mean regression diffusion module is used for forward and backward diffusion processes. It simulates cloud image degradation through state regression stochastic differential equations and optimizes the backward diffusion process through ordinary differential equations (ODEs) to gradually restore image details and sharpness.

[0227] A multi-temporal denoising network construction module is used as a denoising module for the mean regression diffusion model. It optimizes image detail recovery and removes clouds through a temporal self-attention mechanism, thereby improving the cloud removal effect of the image.

[0228] Preconditioning techniques and a fast sampling module are used to optimize the training and sampling process. By preprocessing and a fast sampling method based on data prediction, computational efficiency is improved, sampling time is reduced, and high-quality image recovery is ensured.

[0229] The training and validation module is used to train the remote sensing image cloud removal network using the training set and to perform quantitative analysis using the test set to evaluate the model's performance in different cloud removal scenarios (such as single-time series, single-modality, multi-time series, and multi-modality tasks). When the performance on the test set reaches the preset accuracy, the final trained remote sensing image cloud removal method is obtained.

[0230] The model application module is used to apply the trained remote sensing image cloud removal method to actual remote sensing image data, complete the cloud removal task, and obtain clear images.

[0231] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0232] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A remote sensing cloud removal method integrating multi-temporal and multi-modal approaches, characterized in that, The method includes: S1: Construct forward diffusion and backward diffusion processes for the three selected image data. In the forward diffusion process, the cloudless image is gradually perturbed into a noisy cloud image through state regression stochastic differential equations while maintaining the cloud image structure information. In the backward diffusion process, the noisy image obtained by forward diffusion is gradually restored through a denoising network to obtain a preliminary denoised image sequence. S2: Construct a multi-temporal denoising network for the initial denoised image sequence obtained in S1. This network includes a shared weight encoder, a grouped channel self-attention module, a window self-attention module, and a temporal fusion attention module. It extracts cross-temporal information from the multi-temporal images and enhances the denoising effect, generating an optimized denoised image sequence. S3: Perform preconditioning and fast sampling based on data prediction on the optimized denoised image sequence obtained in S2, adaptively scale the input image, and use the constant variation method to solve the linear and semi-linear equations to accelerate back diffusion, so as to obtain the final denoised image sequence. S4: Using the final denoised image sequence obtained in S3 as training input, iteratively train the multi-temporal denoising network and perform quantitative analysis on the test set. When the quantitative index of the test set reaches the preset accuracy, save the network parameters to obtain the finally trained cloud removal model. S5: Apply the final trained cloud removal model obtained in S4 to actual remote sensing images to generate a clear sequence of cloudless images.

2. The remote sensing cloud removal method integrating multi-temporal and multi-modal approaches according to claim 1, characterized in that, Three types of image data were selected: training and test sets were selected from the public RICE1 and RICE2 datasets to form single-time-series image data; training and test sets were selected from the public SEN12MS-CR dataset to form multimodal image data; and training and test sets were selected from the public Sen2MTC New dataset to form multi-time-series image data.

3. The remote sensing cloud removal method integrating multi-temporal and multi-modal approaches according to claim 1, characterized in that, The forward diffusion process in S1 includes: S11: Initial cloud-free image given Compared with cloud image mean By introducing a mean regression term into the drift and diffusion functions, the image state is... The image is gradually perturbed over time, and noise is added to make it gradually approach the mean of the cloud image. Its state changes satisfy the state regression stochastic differential equation: ， in, For time-related regression parameters, Where is the diffusion coefficient. For Brownian motion increments; S12: Image state during forward diffusion. It can be represented as: ， in, To control the hyperparameters of noise intensity, It is random noise. For time-related mean regression parameters, Represents the instantaneous moment during integration; S13: Noise image obtained from forward diffusion during the reverse diffusion process. Starting from the initial state, the image structure is gradually restored by denoising, resulting in a preliminary denoised image. Its evolution satisfies the ordinary differential equation: ， in, For noise level, It is the diffusion coefficient, which controls the injection intensity of noise; The formula representing the noise prediction of the denoising network is: ， in, It is in time At any time on the image Noise prediction, It is a fractional function used in fractional matching methods to estimate the distribution of data; S14: Inverse Denoising Network The training objective is to minimize the loss function: ， in, It is Gaussian noise. As a conditional input, the loss function enables the denoising network to recover a result close to the original cloudless image. It is the image denoised during the reverse process. It is a noise reduction network. Gaussian noise is used to simulate noise disturbances and improve network robustness. The noise level at the current time step is used as one of the network inputs; the outer layer expectation... The average of all samples in the data distribution is calculated to ensure that the network generalizes across the entire training set; inner layer expectation. Random noise is added to each sample to enhance the network's robustness to noise; minimizing this loss function makes the denoising network... The prediction at each time step should be as close as possible to the original cloudless image. Through repeated iterative reverse denoising processes, a complete and clear cloudless image is finally restored.

4. The remote sensing cloud removal method integrating multi-temporal and multi-modal approaches according to claim 1, characterized in that, The multi-temporal denoising network in S2 includes the following modules: S21: Shared weight encoder module, which encodes each frame of the input multi-temporal image sequence and extracts spatial features, wherein the encoder parameters are shared across all time steps to generate preliminary feature representations across time. S22: Grouped Channel Self-Attention Module. This module divides the channel features output by the encoder into multiple groups and calculates channel self-attention for each group, thereby enhancing the interrelationship between channels. The formula for calculating channel attention is: ， in, For input features, For the first Group channel characteristics, For channel weighting functions; S23: Window self-attention module, introducing a mask into the attention matrix. To restrict the query location to interact only with key / value locations that meet the conditions, the calculation formula is: ， Q (Query): Query matrix, representing the query vector for each pixel / location currently being processed; K (Key): Key matrix, the key vector for each pixel / location, used to measure the similarity to the query; V (Value): Value matrix, the value vector for each pixel / location, containing the information to be fused. Calculate the dot product similarity between the query and the key to obtain the attention score matrix; : Scaling factor, where d is the vector dimension, to prevent the dot product value from being too large, which could cause the Softmax gradient to vanish or become unstable; Mask matrix Based on the query and key / value location coordinate difference settings, the interaction range under blind spot network conditions is ensured; among them, the mask matrix The formula is as follows: ， in, and These are the coordinates of the query and key / value positions; S24: Temporal fusion attention module, which fuses image features from different time steps using the generated temporal attention mask to generate a weighted feature map, the formula of which is: ， in, To output the jump feature map, For the first Attention mask at each time step, To indicate different heads, Encoder features; S25: Decoding module, receives the feature map output by the temporal fusion attention module, and restores it into an optimized cloudless image sequence through a reverse denoising process.

5. The remote sensing cloud removal method integrating multi-temporal and multi-modal processing according to claim 1, characterized in that, The preconditioning and fast sampling based on data prediction in S3 include the following steps: S31: Preconditioning, adaptively scaling the image sequence output by the multi-temporal denoising network, with scaling factors including: ， in, The noise level of the target image. The noise level of the cloud image. The covariance between the target image and the cloud image. The length of the image sequence. and For hyperparameters, Indicates the noise level at the current time step; S32: Fast sampling processing, solving linear and semi-linear equations using the method of variation of constants to accelerate the backdiffusion process and image status. The calculation formula is: ， in, This refers to the image state at the previous time step. , For noise level, , For diffusion parameters, For denoising network prediction, It is random noise. Mean value of cloud image; S33: The image sequences processed by S41 and S42 are used as input to the final denoised image sequence for training and testing the multi-temporal denoising network.

6. The remote sensing cloud removal method integrating multi-temporal and multi-modal approaches according to claim 1, characterized in that, The training and quantitative analysis in S4 includes the following steps: S41: Use the final denoised image sequence obtained in S4 as the training input to iteratively train the multi-temporal denoising network, and perform quantitative analysis using the test set after each training round. S42: Quantitative analysis metrics include Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), Mean Absolute Error (MAE), Spectral Angle Mapping (SAM), and Learned Perceptual Image Patch Similarity (LPIPS), among which: The formula for calculating PSNR is: ， in, For the original image, To restore the image, and These are the image width and height, respectively. The maximum pixel value. Indicates row and column indexes; The formula for calculating SSIM is: ， in, It is a local mean. For local variance, For covariance, It is the stability constant. and It is a partial window for the original image and the restored image. The formula for calculating MAE is: ， in, For predicted values, For the true value, For the number of data points; The formula for calculating SAM is: ， in, For the true spectral vector, To reconstruct the spectral vector, Let be the Euclidean norm of the vector; The formula for calculating LPIPS is: ， in, and For deep networks Layer features, For network layers, Indicates Euclidean distance; S43: When the quantitative analysis index on the test set reaches the preset threshold, save the parameters of the multi-temporal denoising network to obtain the finally trained cloud removal model.

7. A remote sensing cloud removal system integrating multi-temporal and multi-modal characteristics, characterized in that, The system is applied to the method according to any one of claims 1-6, and the system comprises: The data acquisition module is configured to acquire training and test sets from the public RICE1 and RICE2 datasets to form single-time-series image data, acquire training and test sets from the public SEN12MS-CR dataset to form multimodal image data, and acquire training and test sets from the public Sen2 MTC New dataset to form multi-time-series image data. The mean regression diffusion construction module is configured to construct forward diffusion and reverse diffusion processes for the image data acquired by the data acquisition module. The forward diffusion process gradually perturbs the cloudless image into a noisy cloud image while maintaining the cloud image structure information through a state regression stochastic differential equation. The reverse diffusion process gradually restores the image structure through a denoising network to generate a preliminary denoised image sequence. The multi-temporal denoising network construction module is configured to process the preliminary denoised image sequence generated by the mean regression diffusion construction module, including a shared weight encoder module, a grouped channel self-attention module, a window self-attention module and a temporal fusion attention module, to extract cross-temporal information and enhance the denoising effect, and output an optimized denoised image sequence. The preconditioning and fast sampling module is configured to adaptively scale the image sequence output by the multi-temporal denoising network construction module, and accelerate back diffusion by solving the linear and semi-linear equations using the constant variation method to obtain the final denoised image sequence. The training and validation module is configured to iteratively train a multi-temporal denoising network using the image sequences obtained by the preconditions and the fast sampling module as input, and perform quantitative analysis on the test set. When the quantitative analysis index reaches a preset threshold, the network parameters are saved to form the final trained cloud removal model.

Images