A cloud detection method fusing differential attention mechanism and U-Net architecture

Abstract

This invention relates to the field of cloud detection, specifically to a cloud detection method that integrates differential attention mechanism and U-Net architecture. By deeply integrating the differential attention mechanism with the U-Net architecture, this invention sets up multiple cascaded differential attention downsampling modules in the encoder. Each module achieves dynamic dimensionality conversion from image to sequence through a deformable patch embedding layer. The differential attention layer utilizes dual-path Softmax differential operations to eliminate attention common-mode noise, and a topology-aware reconstruction network recovers image features. This effectively suppresses the attention noise generated by the traditional Transformer in cloud detection due to excessive focus on global information. Compared with existing technologies, the recall rate is improved by approximately 12%, and the false positive rate is reduced by approximately 21%, significantly improving the accuracy and reliability of cloud detection for small-sized remote sensing images.

Description

Technical Field

[0001] This invention relates to the field of cloud detection, specifically to a cloud detection method that integrates differential attention mechanism and U-Net architecture. Background Technology

[0002] Current research on cloud detection algorithms mainly falls into two categories: one is the traditional thresholding algorithm, which essentially involves manually classifying pixels in remote sensing images into different categories. While this method is simple to operate and easy to understand, it compares grayscale values ​​pixel by pixel without considering neighborhood relationships. If a single noisy pixel (such as sensor noise) has an abnormally high grayscale value, it will be misidentified as a cloud; conversely, low-grayscale noise points at the cloud edge will be misidentified as ground features, forming "holes." The other category is neural network algorithms, which apply deep learning semantic segmentation algorithms to cloud detection in remote sensing images. By learning cloud features from a large number of labeled cloud-covered images, they can then detect clouds.

[0003] The deep learning-based cloud detection algorithms described above are limited by the size of the receptive field and cannot focus on global information, leaving room for improvement in terms of accuracy, false positives, and false negatives. Some researchers have applied attention mechanisms like Transformer and Vision-Transformer to cloud detection to address the global attention problem, demonstrating high accuracy and low resource consumption in remote sensing image cloud detection tasks. However, the standard Transformer's self-attention mechanism calculates the correlation between all pixel pairs, causing the model to overemphasize the global context while ignoring local details (such as cloud boundaries and thin cloud textures). In the domain of long texts, Transformers tend to allocate only a small portion of attention scores to the correct answer, disproportionately focusing on irrelevant context; these irrelevant attention scores are called "attention noise." Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a cloud detection method that integrates differential attention mechanism and U-Net architecture, solving problems such as attention noise interference, loss of detailed features, and limited model generalization ability in current cloud detection tasks for small-sized multispectral remote sensing images.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a cloud detection method integrating differential attention mechanism and U-Net architecture, the method specifically including the following steps: S1. Acquire the remote sensing image to be detected; S2. Construct and train a cloud detection model for detecting clouds in remote sensing images based on the U-Net network. The cloud detection model includes an encoder, a decoder, and a dynamically adjustable post-processing system. The encoder includes multiple cascaded differential attention downsampling modules for progressively downsampling the input features. Each differential attention downsampling module includes: a deformable patch embedding layer for converting the input four-dimensional image into a three-dimensional sequence and generating positional codes; a differential attention layer for extracting global features from the output features of the deformable patch embedding layer and eliminating noise; and a topology-aware reconstruction network for reconstructing the output features of the differential attention layer into four-dimensional image features. The decoder is used to upsample the deepest feature map output by the encoder level by level, and fuse it with the output features of the corresponding level in the encoder through skip connections to obtain fused features; The dynamically adjustable post-processing system is used to reconstruct the fusion features output by the decoder through the nonlinear coupling of the root mean square normalization layer and the SwiGLU activation function, and to adapt the output images of different scales using a dynamic gain coefficient adjuster to generate cloud detection results. S3. Input the remote sensing image to be detected into the trained cloud detection model to obtain the cloud detection results.

[0006] Preferably, during training, the cloud detection model performs data augmentation on the samples used for training, and the data augmentation method includes at least one of geometric transformation, photometric adjustment and noise injection; The geometric transformation includes at least one of random rotation, horizontal flipping, or vertical flipping; The photometric adjustment includes at least one of random luminance stretching and gamma correction in the HSV color space; The noise injection includes at least one of randomly erasing a portion of the input image, superimposing Gaussian noise, and applying Gaussian blur.

[0007] Preferably, before the features upsampled by the decoder are fused with the output features of the corresponding level in the encoder through skip connections, the feature map output by the corresponding level of the encoder is further subjected to root mean square normalization.

[0008] Preferably, when the deformable patch embedding layer generates position codes, it generates normalized network coordinates to represent the relative positions of pixels based on the three-dimensional sequence, and generates position codes in real time based on the normalized network coordinates to support remote sensing image input at different resolutions.

[0009] Preferably, the encoder further includes a feature reconstruction module, which uses a two-layer convolutional block to perform cross-channel feature scale unification and high-frequency detail reconstruction on the input remote sensing image. The two-layer convolutional block adopts a collaborative architecture of root mean square normalization layer and SwiGLU activation function.

[0010] Preferably, when the topology-aware reconstruction network reconstructs the output features of the differential attention layer into four-dimensional image features, it employs a channel attention-guided cross-dimensional fusion mechanism.

[0011] Preferably, the dynamic gain coefficient regulator in the dynamically adjustable post-processing system is used to adaptively adjust the gain coefficient according to the target scale of the output image and the scale of the input remote sensing image, while maintaining the sharing of model parameters, so as to output an output image with the target scale.

[0012] Preferably, the encoder further includes an adaptive preprocessing module, which employs a cascaded max pooling and channel grouping normalization for coordinated optimization.

[0013] The technical solution also provides a system for implementing a cloud detection method that integrates differential attention mechanism and U-Net architecture. The system includes a processor and a memory, wherein the memory is used to store a computer program, and the computer program is executed by the processor to implement the cloud detection method.

[0014] Compared with existing technologies, this invention provides a cloud detection method that integrates differential attention mechanism and U-Net architecture, which has the following beneficial effects: 1. This invention deeply integrates the differential attention mechanism with the U-Net architecture, setting up multiple cascaded differential attention downsampling modules in the encoder. Each module realizes dynamic dimensionality conversion from image to sequence through a deformable patch embedding layer. The differential attention layer uses dual-path Softmax differential operation to eliminate attention common-mode noise, and the topology-aware reconstruction network recovers image features. This effectively suppresses the attention noise generated by the traditional Transformer in cloud detection due to excessive focus on global information. Compared with the existing technology, the recall rate is improved by about 12%, and the false detection rate is reduced by about 21%, which significantly improves the accuracy and reliability of cloud detection for small-sized remote sensing images.

[0015] 2. This invention solves the problem of inconsistent feature scales between deep and shallow layers caused by direct splicing of skip connections in traditional U-Net by introducing root mean square normalization processing in skip connections, which normalizes the feature maps output by the corresponding layers of the encoder before fusing them with the features of the decoder. At the same time, the deformable patch embedding layer generates normalized network coordinates as position codes in real time according to the size of the input image, enabling the model to flexibly adapt to remote sensing image inputs of different resolutions and enhancing the model's generalization ability to images of different sensors and different scales.

[0016] 3. This invention sets up a feature reconstruction module at the front end of the model and uses a collaborative architecture of root mean square normalization layer and SwiGLU activation function to perform cross-channel feature unification and high-frequency detail reconstruction of multispectral images. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart of the cloud detection model of the present invention; Figure 2 This is a schematic diagram of the differential attention structure of the present invention; Figure 3 This is a processing architecture diagram of the cloud detection model of the present invention; Figure 4 This is a schematic diagram illustrating the results of data augmentation according to the present invention; Figure 5 Radar chart comparing the performance of each model; Figure 6 The training process curves for each model are shown in the graphs. Figure 7 Thermograph for ablation experiment; Figure 8 For comparison, a prediction graph is provided. Figure 9 This is a prediction graph from an ablation experiment. Detailed Implementation

[0018] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. This will allow for a full understanding of how the present application uses technical means to solve technical problems and achieve technical effects, and to facilitate its implementation.

[0019] Those skilled in the art will understand that all or part of the steps in the methods of the following embodiments can be implemented by a program instructing related hardware. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0020] To address the problems of attention noise interference, loss of detailed features, and limited model generalization ability in current cloud detection tasks for small-sized multispectral remote sensing images, this invention provides a cloud detection method that integrates a differential attention mechanism and a U-Net architecture. This method specifically includes the following steps: S1. Acquire the remote sensing image to be detected.

[0021] S2. A cloud detection model for detecting clouds in remote sensing images is constructed and trained based on the U-Net network. The cloud detection model includes an encoder, a decoder, and a dynamically adjustable post-processing system. The encoder includes multiple cascaded differential attention downsampling modules for downsampling the input features step by step. The decoder is used to upsample the deepest feature map output by the encoder step by step and fuse it with the output features of the corresponding level in the encoder through skip connections to obtain fused features.

[0022] The U-Net network was originally a fully convolutional neural network for medical image segmentation. Later, due to its symmetric encoder-decoder structure and skip connections, which performed exceptionally well in few-shot segmentation tasks, it was widely applied in the field of artificial intelligence image segmentation. However, the skip connection structure also introduces low-level features, which may contain noise or irrelevant information. This causes the decoder to need to perform additional computations to filter effective features, affecting training efficiency. The Differential Transformer structure can suppress "attention noise," but this structure is designed for large language model sequence tasks. To make the Differential Transformer applicable to cloud detection tasks, this invention innovatively constructs an end-to-end image processing architecture oriented towards diffusion models. The improved U-Net network structure is thus the cloud detection model, as follows: Figures 1-3 As shown, the improvements mainly include the following three points: (1) An adaptive preprocessing module is proposed. Through the collaborative optimization of cascaded max pooling and channel group normalization (Group-Norm) (GN-BatchFree), the sensitivity of traditional batch normalization (BN) to small sample data is eliminated, while significantly improving the computational efficiency of the model on low-memory devices. Experiments show that when training with a deep model (ResNet-101), when the batch size is 32, the GN validation error rate is only 0.4% higher than BN (22.4% vs 22.0%); while when the batch size is 2, the GN error rate is only 23.0%, which is significantly better than BN's 31.9% (difference of 8.9%).

[0023] (2) The proposed Diff-Transformer core module, namely the differential attention layer, is used to extract global features from the output features of the deformable patch embedding layer and eliminate noise. It innovatively adopts a bidirectional divide-and-conquer coding strategy: the upper layer realizes the dynamic dimensional transformation of the image-sequence (DynPatch Emb) through the deformable patch embedding layer. The Rearrange operation is used to convert the four-dimensional image into a three-dimensional sequence: [B,C,H,W]→[B,H×W,C]. This step treats spatial pixels as sequence elements, forming a sequence structure similar to natural language, and generating normalized network coordinates to represent the relative position of pixels. In addition, the generated position code is generated in real time according to the current normalized network coordinates (H and W) to support arbitrary input sizes (such as images of different resolutions). Its multi-scale convolutional kernel group can adaptively match inputs of different resolutions. The lower layer introduces a topology-aware reconstruction network (TPR-Net), which adopts a cross-dimensional fusion mechanism guided by channel attention to reconstruct the output features of the differential attention layer into four-dimensional image features, successfully solving the problem of dimensional mismatch between Transformer output and image spatial structure.

[0024] Traditional Transformer models tend to overemphasize global information, and this "attention noise" can negatively impact the model's judgment. However, the Differential Transformer model borrows the principle of differential amplifiers and introduces a differential attention mechanism, or differential attention layer. This mechanism eliminates "attention noise" by calculating the difference between two independent attention vectors, amplifying the attention weights of relevant context and enabling the model to focus on key information within the global context. The model structure of the differential attention mechanism is as follows: Figure 2 As shown: First, the differential attention mechanism computes the Q (query value), K (key value), and V (value) tensors, just like the standard attention mechanism. Here, Q, K, and V are obtained by projecting the input tensor X. Then, the Q and K tensors are respectively cut into... The model is processed using tensors, and attention noise is eliminated. Finally, normalization and concatenation are performed. Through these operations, contextual information can be dynamically learned, enhancing the model's training stability and allowing us to focus on the key information we want to focus on.

[0025] (3) Construct a dynamically adjustable post-processing system. In the Diff Transformer layer (differential attention layer), the root mean square layer normalization and the SwiGLU activation function are nonlinearly coupled to reconstruct the fusion features output by the decoder. By introducing a dynamic gain coefficient regulator, while maintaining model parameter sharing, such as maintaining a parameter sharing rate of more than 90%, the gain coefficient is adaptively adjusted according to the target scale of the output image and the scale of the input remote sensing image to output an output image with the target scale. It can flexibly adapt to multi-scale image generation tasks from 96×96 to 1024×1024.

[0026] The following section details the data processing procedure for the cloud detection model. Please refer to [link / reference]. Figure 3 : 1) Feature Reconstruction Module: A feature reconstruction module is included before the encoder. This module uses two-layer convolutional blocks to perform cross-channel feature scale unification and high-frequency detail reconstruction on the input remote sensing image. The two-layer convolutional blocks employ a collaborative architecture of root mean square normalization layers and the SwiGLU activation function. Figure 3 (Blue arrow pointing to the right). The root mean square normalization layer RMSNorm is based on the root mean square scaling mechanism, which enables multispectral features to retain the non-zero mean distribution characteristics while achieving uniform feature scale across channels and reducing the computation time by 6.25%-64%. Its root mean square scaling mechanism effectively suppresses numerical fluctuations in deep convolutions.

[0027] SwiGLU dynamically adjusts the information flow through gating weights, achieving the following in cloud edge detection: ① Thin cloud texture enhancement (activation of high-frequency features when β gating coefficient > 0.7); ② Implicit parameter sharing (reusing projection weights in the gating matrix); ③ Gradient decay compensation (smooth transition characteristics of the Swish function). RMSNorm stabilizes the distribution of intermediate feature maps, while SwiGLU enhances the ability to reconstruct high-frequency details. The synergistic use of these two technologies in two-layer convolutional blocks achieves a triple improvement in efficiency, stability, and expressive power.

[0028] 2) Differential Attention Downsampling Module: Differential Attention Downsampling Module (DownBlock Module) Figure 3 (Orange-red downward arrow) Innovative integration: ① Local receptive field of the convolutional kernel (3×3 depthwise separable convolution); ② Positional bias compensation of the differential attention mechanism. This structure achieves the synergy of local feature extraction and global association modeling (eliminating background noise through differential attention) in four downsampling processes. In particular, during feature processing, differential attention improves the association weights between cloud clusters while reducing the misclassification rate of surface objects by approximately 21%.

[0029] 3) Multi-scale feature fusion path; The U-shaped architecture achieves four-layer cascaded fusion through skip connections: ① After the four-level features at the encoder are normalized by RMSNorm, they are concatenated with the corresponding level features at the decoder. That is, before the features upsampled by the decoder are fused with the output features of the corresponding level in the encoder through skip connections, the feature map output by the corresponding level of the encoder is also subjected to root mean square normalization; ② 3×3 convolution is used for cross-scale feature integration; ③ The top-level fusion module integrates the cross-scale attention of Transformer to establish long-range dependencies between features of different resolutions.

[0030] This architecture achieves a breakthrough in cloud detection through a combination of three strategies: feature recombination, attention purification, and cross-scale fusion. The combination of RMSnorm and SwiGLU, along with a differential mechanism and an improved skip connection, reduces the false detection rate of multispectral features by approximately 21% and improves the recall rate of small targets by approximately 12%.

[0031] S3. Input the remote sensing image to be detected into the trained cloud detection model to obtain the cloud detection results.

[0032] In this embodiment, the cloud detection model is trained using the 95-Cloud dataset, a significant extension of the 38-Cloud dataset designed to address the limitations of insufficient training data size and geographical diversity in remote sensing cloud detection tasks. The training set of this dataset contains 34,701 384×384 pixel image patches from 75 North American-dominated Landsat 8 scenes, covering a wider range of terrain types and significantly enhancing the model's adaptability to regional heterogeneity. The test set uses 9,201 image patches from 20 scenes, ensuring compatibility between the expanded dataset and baseline evaluation standards, and supporting performance comparisons across studies. It features a multispectral data separation storage architecture; it independently provides raw data for the four channels of red (Band4), green (Band3), blue (Band2), and near-infrared (Band5), stored in a separate directory format. This design preserves the spatial independence of multispectral features, facilitating the model's direct extraction of spectral response differences. To address the issues of deep learning models' strong dependence on labeled data and the lack of diversity in public datasets, this embodiment introduces a multi-level data augmentation method based on public datasets, which includes at least one of geometric transformation, photometric adjustment, and noise injection. For example... Figure 4 As shown, a three-dimensional enhancement strategy involving geometric transformation, photometric adjustment, and noise injection effectively expands the coverage of the data distribution. The specific method design is as follows: 1) Geometric transformation: At least one of random rotation (±70°) and horizontal / vertical flip (probability 13%) is used to force the model to learn rotational symmetry and mirror invariance features, which significantly improves the model's adaptability to changes in the target spatial position.

[0033] 2) Luminosity Adjustment: Perform random brightness stretching (adjustment range ±50%) or gamma correction in the HSV color space. (This simulates the imaging characteristics under different light intensities.)

[0034] 3) Noise injection: ① Obstruction interference: Random erasure ( , Image regions (height and width of the input image, respectively); ② Sensor noise: Gaussian noise is superimposed to simulate the noise generated by the sensor under extreme conditions. This operation can further verify the "denoising" effect of the algorithm proposed in this paper; ③ Motion blur: Gaussian blur is used to simulate equipment jitter under abnormal conditions.

[0035] To prevent overfitting during model training, probability prediction is performed before data loading. The probability of using any data augmentation method can be set to 7 / 15, and the probability of a single data augmentation operation is 1 / 15.

[0036] In addition, to verify the denoising performance and effectiveness of the cloud detection model Differential Transformer-UNet (Diff-UNet) in this embodiment under small-scale training, this study selected four representative comparison models: U-Net: a classic encoder-decoder architecture that fuses deep and shallow features through skip connections; SE-Unet: embedding a channel attention module (SENet) to enhance key channel information through feature recalibration; CBAM-Unet: combining channel and spatial dual attention mechanisms (CBAM) to achieve dynamic calibration of multi-dimensional features; and ViT-Unet: introducing a Vision Transformer module to capture long-range dependencies using a self-attention mechanism.

[0037] The parameters and experimental settings for this embodiment are as follows: initial learning rate: 1E-5, end learning rate: 1E-8, training set:validation set = 8:2, Num_worker: 8, patience parameter: 15, decay coefficient: 0.7, random seed: fixed at 42, batch size: 12, training size: 96*96, optimizer type: Adam, loss function: BCE Loss, number of training epochs: 100. The model comparison experimental results are shown in Table 1 below: Diff-UNet innovates with a differential attention mechanism, suppressing common-mode noise through dual-path Softmax interpolation. It includes the classic U-Net and its three attention-enhanced variants (SE-UNet, CBAM-UNet, and ViT-UNet). The test set performance of each model under the same training conditions is shown in Table 1 (all units are %). A visualization comparison of the prediction results can be found here. Figure 5The convergence curve during the training process is shown in the figure. Figure 6 The key findings from the experiment are as follows: 1) The enhancing effect of attention mechanisms on feature learning. The baseline model U-Net showed significant limitations in small-scale training: although the recall rate reached 100%, its overall accuracy was only 31.27%. Combined with metrics such as precision (31.23%) and F1 score (43.48%), this indicates that the model suffers from serious background misclassification (misclassifying non-cloud areas as clouds). In contrast, the improved models that introduced attention mechanisms showed significant performance improvements—SE-UNet (overall accuracy 84.56%) and CBAM-UNet (overall accuracy 76.13%) reduced the false detection rate by 47.7% and 28.1%, respectively, through channel attention and spatial-channel dual attention mechanisms. Table 1 (indicators in rows 2-3) shows that these two models have significant improvements over U-Net in cloud edge recognition and small cloud cluster detection.

[0038] 2) Validation of the advantages of the differential attention mechanism. The ViT-UNet and Diff-UNet models exhibit superior balance across various metrics (Table 1 and...). Figure 5 Among them, Diff-UNet achieved the best performance with an overall accuracy of 94.46% and an F1 score of 90.45%. Of particular note is that Diff-UNet's recall (87.21%) is 9.14 percentage points higher than ViT-UNet (78.07%), while also having a higher precision (83.31% vs 78.96%). This is attributed to its innovative differential attention mechanism (as shown in rows 4-5 of Table 1, Diff-UNet's false positive rate is reduced by approximately 21% compared to ViT-UNet), validating the unique advantage of this mechanism in common-mode noise cancellation.

[0039] 3) Training dynamics and model potential analysis. From the training process curves ( Figure 6 Three learning modes can be observed: ① U-Net converges quickly in the first 10 rounds but then gets stuck in a local optimum, and fails to capture new cloud features in the subsequent 90 rounds of training; ② CBAM-UNet and SE-UNet overfit in the 50th and 70th rounds, respectively, but restart feature learning through a learning rate decay strategy; ③ ViT-UNet and Diff-UNet show a continuous and gradual downward trend in their loss curves, indicating that they still have feature mining potential after 100 rounds of training. This phenomenon is closely related to the long-range dependency modeling capability of the Transformer architecture, especially the differential attention module of Diff-UNet, which exhibits better gradient propagation characteristics.

[0040] To verify the effectiveness of each module in this implementation, four ablation experimental schemes were designed using the controlled variable method, and the results are as follows: Figure 7 As shown in Table 2: Option 1 (Complete Model): DT-UNet network architecture + data augmentation strategy; Option 2 (Denormalization): Remove the RMSNorm normalization and SwiGLU activation function from the downsampling block, but retain the RMSNorm normalization and SwiGLU activation function from the differential Transformer convolution block; Option 3 (Baseline Replacement): Use the traditional U-Net two-layer convolutional structure + retain RMSnorm normalization and SwiGLU activation function; Option 4 (Remove Data Augmentation): Only remove the data augmentation strategy.

[0041] As shown in Table 2 (all units are %), the performance metrics of each scheme on the test set show significant differences, and the following conclusions can be drawn: 1) Validation of the effectiveness of the data augmentation strategy. Comparing Scheme 1 (overall accuracy 94.46%) and Scheme 4 (94.00%), the data augmentation strategy improved the IoU by 4.3 percentage points (69.93%→74.23%) and the F1-score by 2.81% (87.64%→90.45%). This result shows that the dynamic data augmentation strategy presented in this paper can effectively enhance the model's ability to represent multispectral features.

[0042] 2) Adaptability analysis of the normalization module. Compared to Scheme 1, Scheme 2 (de-normalization) maintains an overall accuracy of 95.51%, while improving precision by 2.49% (83.31%→85.80%) and intersection-over-union ratio by 3.03% (74.23%→77.26%). This counterintuitive phenomenon reveals that while the RMSnorm and SwiGLU modules are highly effective in natural language processing, they may disrupt the continuity of local features in small-scale multispectral images (<384 pixels * 384 pixels). This finding provides a new optimization direction for the architectural design of lightweight semantic segmentation models.

[0043] 3) Justification for the necessity of core components. The performance of Scheme 3 (traditional U-Net) model showed a significant collapse: the overall accuracy plummeted to 31.21%, and the F1-score was only 43.41%. However, the unusually high recall (100%) indicated a serious overfitting problem, which may stem from the inability of traditional convolutional layers to effectively model the nonlinear relationships between multispectral channels. This result fully demonstrates the irreplaceable nature of the Differential Transformer module in cross-modal feature fusion.

[0044] Based on the spliced ​​prediction patch results, this study selected five representative groups of samples for visualization ensemble analysis, and constructed model comparison experimental prediction graphs respectively. Figure 8 ) and ablation experiment prediction diagram ( Figure 9 The system is evaluated from two dimensions: visual recognition and spatial consistency.

[0045] 1) Model comparison experiment analysis. Figure 8 The visual comparison results show significant differences in edge detection accuracy and background discrimination ability among different model architectures. Specifically: ① CBAM-UNet and SE-UNet, due to their insufficient attention to spatial features in their attention mechanisms, result in jagged artifacts at predicted boundaries. Although their output metrics show some improvement compared to the U-Net model, they still suffer from spatial continuity defects. ② The U-Net model exhibits overgeneralization in the central mountain region (third sub-image from left to right), misclassifying exposed rock areas as clouds. This contradicts its 100% recall and 31.27% overall precision, highlighting the limitations of the traditional U-Net network architecture in the classification of multispectral complex land cover. ③ In contrast, ViT-UNet and Diff-UNet demonstrate superior pixel-level classification capabilities, with a spatial overlap of 92.8% ± 3.1% between their predicted contours and GT labels. They also maintain clear boundary transition features, especially in the transitional areas between cloudy and cloudless (sub-images 4 and 5).

[0046] 2) Ablation experiment analysis. Figure 9 The ablation experiments shown verify the effectiveness of the algorithm components: Scheme 2 (denormalized) performs best in the cloud detection task, demonstrating the best ability to segment complex cloud systems, and the edge texture transition of the stitched image is the most natural and smooth.

[0047] The above experimental results fully demonstrate the technical superiority of the cloud detection model in this embodiment in small-scale multispectral remote sensing image processing.

[0048] The present invention also provides a cloud detection system that integrates differential attention mechanism and U-Net architecture, comprising: a processor and a memory, wherein the memory is used to store a computer program, and the computer program implements the cloud detection method when executed by the processor.

[0049] The above embodiments provide a detailed description of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A cloud detection method integrating differential attention mechanism and U-Net architecture, characterized in that, The method specifically includes the following steps: S1. Acquire the remote sensing image to be detected; S2. Construct and train a cloud detection model for detecting clouds in remote sensing images based on the U-Net network. The cloud detection model includes an encoder, a decoder, and a dynamically adjustable post-processing system. The encoder includes multiple cascaded differential attention downsampling modules for progressively downsampling the input features. Each differential attention downsampling module includes: a deformable patch embedding layer for converting the input four-dimensional image into a three-dimensional sequence and generating positional codes; a differential attention layer for extracting global features from the output features of the deformable patch embedding layer and eliminating noise; and a topology-aware reconstruction network for reconstructing the output features of the differential attention layer into four-dimensional image features. The decoder is used to upsample the deepest feature map output by the encoder level by level, and fuse it with the output features of the corresponding level in the encoder through skip connections to obtain fused features; The dynamically adjustable post-processing system is used to reconstruct the fusion features output by the decoder through the nonlinear coupling of the root mean square normalization layer and the SwiGLU activation function, and to adapt the output images of different scales using a dynamic gain coefficient adjuster to generate cloud detection results. S3. Input the remote sensing image to be detected into the trained cloud detection model to obtain the cloud detection results.

2. The cloud detection method according to claim 1, characterized in that, During training, the cloud detection model performs data augmentation on the training samples, and the data augmentation method includes at least one of geometric transformation, photometric adjustment and noise injection. The geometric transformation includes at least one of random rotation, horizontal flipping, or vertical flipping; The photometric adjustment includes at least one of random luminance stretching and gamma correction in the HSV color space; The noise injection includes at least one of randomly erasing a portion of the input image, superimposing Gaussian noise, and applying Gaussian blur.

3. The cloud detection method according to claim 1, characterized in that, Before the features upsampled by the decoder are fused with the output features of the corresponding level in the encoder through skip connections, the feature map output by the corresponding level of the encoder is also subjected to root mean square normalization.

4. The cloud detection method according to claim 1, characterized in that, When the deformable patch embedding layer generates position codes, it generates normalized network coordinates to represent the relative positions of pixels based on the three-dimensional sequence, and generates position codes in real time based on the normalized network coordinates to support remote sensing image input at different resolutions.

5. The cloud detection method according to claim 1, characterized in that, The encoder is preceded by a feature reconstruction module, which uses a two-layer convolutional block to perform cross-channel feature scale unification and high-frequency detail reconstruction on the input remote sensing image. The two-layer convolutional block adopts a collaborative architecture of root mean square normalization layer and SwiGLU activation function.

6. The cloud detection method according to claim 1, characterized in that, When the topology-aware reconstruction network reconstructs the output features of the differential attention layer into four-dimensional image features, it adopts a cross-dimensional fusion mechanism guided by channel attention.

7. The cloud detection method according to claim 1, characterized in that, The dynamic gain coefficient regulator in the dynamically adjustable post-processing system is used to adaptively adjust the gain coefficient according to the target scale of the output image and the scale of the input remote sensing image, while maintaining the sharing of model parameters, so as to output an output image with the target scale.

8. The cloud detection method according to claim 1, characterized in that, The encoder is preceded by an adaptive preprocessing module, which employs a collaborative optimization of cascaded max pooling and channel grouping normalization.

9. A cloud detection system integrating differential attention mechanism and U-Net architecture, used to implement the cloud detection method integrating differential attention mechanism and U-Net architecture as described in any one of claims 1-8, characterized in that, It includes a processor and a memory, the memory being used to store a computer program, which, when executed by the processor, implements the cloud detection method as described in any one of claims 1-8.

Images