Lightweight attention cloud detection method based on deep learning

Abstract

The invention relates to the technical field of remote sensing image processing and computer vision, and particularly discloses a light-weight attention cloud detection method based on deep learning, which comprises the following steps of: 1, acquiring remote sensing image data, constructing a deep learning model comprising a plurality of convolutional layers through the remote sensing image data, extracting the features of each convolutional layer; 2, a lightweight attention mechanism is introduced in the feature extraction process, and final attention feature representation is obtained; step 3, carrying out space dimension reduction and channel dimension compression on the input feature map through a space-channel reconstruction convolution module; and step 4, outputting a final cloud detection result, and performing post-processing on the result. According to the lightweight attention cloud detection method based on deep learning, the calculation complexity of the model can be reduced, and the detection precision and calculation efficiency of the cloud layer in the remote sensing image can be improved.

Description

Technical Field

[0001] The present invention relates to the fields of remote sensing image processing and computer vision technology, and in particular to a lightweight attention cloud detection method based on deep learning. Background Art

[0002] Remote sensing images are widely used in meteorological monitoring, environmental protection, land use and other fields. With the development of remote sensing technology, the demand for obtaining high-quality images is increasing. However, the presence of clouds will block surface information and affect the integrity and clarity of image data. This makes cloud detection and removal an important task in remote sensing image processing.

[0003] Traditional cloud detection methods usually rely on features such as spectrum, texture and temperature, and use rule-based methods to segment clouds. These methods often fail to provide high-precision detection results when dealing with complex cloud shapes, different thicknesses and diverse weather conditions. In addition, traditional methods are not robust enough to adapt to the changes in various remote sensing images.

[0004] In recent years, semantic segmentation methods based on deep learning have gradually become the mainstream technology for cloud detection. In particular, the U-Net structure, due to its symmetrical encoding-decoding design, can retain both detail information and global context, and thus performs well in cloud detection. However, the traditional structure of U-Net has some problems, such as high computational overhead, feature redundancy, lack of dynamic attention to important features, etc., which makes it difficult to meet the needs of real-time detection. Summary of the invention

[0005] The present invention aims to solve at least one of the above-mentioned technical problems and provide a lightweight attention cloud detection method based on deep learning, which can reduce the computational complexity of the model and improve the detection accuracy and computational efficiency of clouds in remote sensing images.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is: a lightweight attention cloud detection method based on deep learning, comprising the following steps:

[0007] Step 1. Obtain remote sensing image data, build a deep learning model containing multiple convolutional layers through remote sensing image data, and extract the features of each convolutional layer to obtain the feature map of each convolutional layer. The feature map output by each convolutional layer is expressed as: F out =σ(Conv(F in ,W)+b), where F in is the input feature map, Conv is the convolution operation, W is the convolution kernel weight, b is the bias, and σ is the activation function;

[0008] Step 2. Introduce a lightweight attention mechanism in the feature extraction process, weight the features according to their importance, and send the features to three different convolution branches to generate three different feature representations respectively. Use different weights to extract different features, and then obtain the final attention feature representation by weighted summation of the three different feature representations;

[0009] Step 3. Use the space-channel reconstruction convolution module to reduce the spatial dimension of the input feature map and compress the channel dimension;

[0010] Step 4. Output the final cloud detection results and post-process the results.

[0011] Preferably, in step 1, the deep learning model includes an encoder and a decoder, the encoder downsamples layer by layer, extracts multi-scale features through convolution and pooling operations, and in each convolution layer, an activation function is used to introduce nonlinearity, which is defined as: f(x)=x·tanh(softplus(x)), wherein the softplus function is defined as: softplus(x)=ln(1+e x ); The decoder upsamples layer by layer to restore the spatial resolution and fuse the detailed features in the jump connection; the downsampling formula is: F down =MaxPool(σ(Conv(F in ,W)+b)), the upsampling formula is: F up =Conv(UpSample(F in ),W)+b.

[0012] Preferably, in step 2, the three different feature representations are:

[0013] The formula for weighted summation is:

[0014] Preferably, in step 3, the space-channel reconstruction convolution module includes a space reconstruction unit and a channel reconstruction unit, and the space reconstruction unit is used to reduce the dimension of the input feature map to compress the spatial redundant features, and its operation formula is: sru =f SRU (F in )+F in , where f SRU (·) is a dimensionality reduction function implemented by convolution, which extracts the spatial information of the feature through a parameterized convolution kernel. The channel reconstruction unit is used to compress the channel dimension of the feature map to suppress redundant features and emphasize key features. Its operation formula is: cru =f CRU (F in )+F in.

[0015] Preferably, after compressing the channel dimension of the feature map, a lightweight channel attention mechanism is applied to dynamically adjust the importance of different channels by calculating the weight of each channel. The weight calculation formula is: α i =σ(FC(F cru )), where α i is the weight coefficient for each channel, σ(·) is the Sigmoid activation function, and FC(·) is the channel weighted learning through a small fully connected layer; and the reconstructed feature map is added to the original input feature map to retain the key information to form a new feature representation: F final =F cru α i +F in .

[0016] Preferably, in step 4, post-processing includes noise removal, boundary smoothing and threshold processing, wherein noise removal uses morphological operations to remove small noise points in the segmentation results, boundary smoothing uses image smoothing technology to enhance the coherence of cloud boundaries, and threshold processing thresholds the probability map output by the model to generate a final binary cloud mask.

[0017] Compared with the prior art, the lightweight attention cloud detection method based on deep learning of the present invention has the following beneficial effects:

[0018] (1) Improved the feature expression capability of cloud detection in remote sensing images;

[0019] (2) Reduce computational complexity and be applicable to real-time remote sensing image analysis;

[0020] (3) The detection accuracy of cloud features is enhanced by dynamically focusing on important features. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The specific embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings, wherein:

[0022] Figure 1 A schematic diagram of a flow chart of a lightweight attention cloud detection method based on deep learning of the present invention;

[0023] Figure 2 This is a network architecture diagram of a lightweight attention cloud detection method based on deep learning of the present invention. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0025] It should be noted that when a component is referred to as being "fixed to" another component, it may be directly on the other component or there may be a central component. When a component is considered to be "connected to" another component, it may be directly connected to the other component or there may be a central component at the same time. When a component is considered to be "set on" another component, it may be directly set on the other component or there may be a central component at the same time. When a component is referred to as being "set in the middle", it does not only mean being set in the middle, as long as it is not set at both ends within the range defined by the middle. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only.

[0026] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0027] like Figure 1 to Figure 2 As shown, the present invention discloses a lightweight attention cloud detection method based on deep learning, comprising the following steps:

[0028] Step 1. Obtain remote sensing image data, build a deep learning model with multiple convolutional layers through remote sensing image data, and extract features of each convolutional layer to realize feature extraction network construction. Specifically, by building a deep learning model with multiple convolutional layers, the input image is processed layer by layer. Each convolutional layer will extract different levels of features from the input original image. These features can be divided into the following categories:

[0029] Low-level features: such as edges, textures, colors, etc., are usually extracted in the first few layers of the network.

[0030] Intermediate features: such as shape, patterns of local areas, etc., are usually extracted in the middle layer of the network.

[0031] High-level features: such as objects, semantic information, etc., are usually extracted in the later layers of the network.

[0032] These features are obtained by applying convolution kernels (filters) to the input image in the convolution layer. The feature map (FeatureMap) output by each convolution layer is a feature representation of different spatial regions and scales. The feature map output by each convolution layer is represented as:

[0033] F out =σ(Conv(F in ,W)+b)

[0034] Among them, F in is the input feature map, Conv is the convolution operation, W is the convolution kernel weight, b is the bias, and σ is the activation function.

[0035] In addition, the deep learning model in the present application includes an encoder and a decoder. The encoder downsamples layer by layer, extracts multi-scale features through convolution and pooling operations, and uses an activation function to introduce nonlinearity in each convolution layer, where softplus(x) is a smooth approximate ReLU function, defined as: f(x) = x·tanh(softplus(x)), and the softplus function is defined as: softplus(x) = ln(1+e x ), this smooth curve makes Mish change more gradually when the input tends to negative infinity and positive infinity, instead of a hard inflection point like ReLU. This smooth nonlinear characteristic is particularly suitable for capturing details in images and helps to preserve subtle information of cloud edges in cloud detection tasks.

[0036] The decoder upsamples layer by layer to restore the spatial resolution and fuses detailed features in skip connections, ensuring high-resolution cloud boundary recovery.

[0037] In the above operation, downsampling reduces the spatial resolution of the feature map by using convolution and maximum pooling layers. The formula is: F down =MaxPool(σ(Conv(F in ,W)+b));

[0038] Upsampling gradually restores the spatial resolution of the feature map by using deconvolution or upsampling operations. The formula is: F up =Conv(UpSample(F in ),W)+b.

[0039] Step 2. Introduce a lightweight attention mechanism in the feature extraction process, weight the features according to their importance, and send the features to three different convolution branches to generate three different feature representations respectively. Use different weights to extract different features, and then obtain the final attention feature representation by weighted summation of the three different feature representations.

[0040] Specifically, a lightweight attention mechanism is introduced in the feature extraction process to dynamically focus on important features. The key to the lightweight attention mechanism is to weight the features according to their importance. The input feature tensor is fed into three different convolution branches (called feature branches) to generate three different feature tables.

[0041] T x =Conv(F in ,W x ),

[0042] T y =Conv(F in ,W y ),

[0043] The three different feature representations are: T z =Conv(F in ,W z ), these branches capture different dimensional information of input features. Among them, the convolution branch obtains multi-scale information of features by using different convolution kernel sizes and strides, and uses different weights to extract different features through convolution operations.

[0044] At the same time, the weighted sum of the three feature representations is performed to obtain the final attention feature representation. The importance of the features extracted by each convolutional layer can be evaluated through methods such as self-attention mechanism (Self-Attention) or channel attention mechanism (ChannelAttention). According to the evaluation results, the features are processed separately through three different convolution branches, and each convolution branch produces different feature representations. Usually, each convolution branch uses a different convolution kernel size or convolution method to capture features of different scales or levels.

[0045] For example:

[0046] The first branch is used to extract local details (small-scale features).

[0047] The second branch is used to capture mesoscale regional features.

[0048] The third branch is used to capture global features and consider a wider range of contextual information.

[0049] The output of each branch is weighted and summed according to its importance. These weights are usually learned through training, and the model automatically optimizes these weights through back propagation so that the relevant features of the cloud layer are fully emphasized. The formula for weighted summation is: Among them, α i is the weight coefficient obtained through learning, T iis the feature after being processed by different convolution branches, W i is the convolution kernel of each branch, and T is the final merged feature representation. This weighting method allows the model to focus on more important features during forward propagation, thereby enhancing the overall feature expression capability.

[0050] During network training, the contribution of the lightweight attention mechanism to cloud detection accuracy can be verified by comparing the performance with and without the attention mechanism. Experiments show that the mechanism significantly improves the model's ability to adapt to complex cloud shapes and changes during the feature extraction stage.

[0051] Step 3. Use the space-channel reconstruction convolution module (SCConv) to reduce the spatial dimension of the input feature map and compress the channel dimension. Specifically, the design of the space-channel reconstruction convolution module (SCConv) aims to improve the computational efficiency and model performance by effectively reducing redundant features. Finally, the feature map F after spatial and channel reconstruction out It is obtained by adding the output features of the spatial reconstruction unit and the channel reconstruction unit. The SCConv module is mainly composed of the spatial reconstruction unit (SRU) and the channel reconstruction unit (CRU). The core idea of ​​the SCConv module is to reduce the redundancy of the feature map by processing the spatial and channel dimensions separately. This design enables the model to reduce the consumption of computing resources while retaining important information, thereby adapting to the needs of real-time detection. Among them, the spatial reconstruction unit (SRU) is used to reduce the dimension of the input feature map to compress the spatial redundant features. Its operation formula is:

[0052] F sru =f SRU (F in )+F in

[0053] Among them, f SRU (·) is a dimensionality reduction function implemented by convolution, which extracts the spatial information of features through parameterized convolution kernels.

[0054] The reduced feature map is added to the original input feature map to retain important information in the low-dimensional representation. This skip connection helps prevent information loss while enhancing the model's expressiveness. After feature reconstruction, a nonlinear activation function (Mish) is used to introduce nonlinear transformations to further enhance feature representation.

[0055] The main purpose of the channel reconstruction unit (CRU) is to compress the channel dimension of the feature map, suppress redundant features and emphasize key features. Its operation formula is:

[0056] F cru =f CRU (Fin )+F in

[0057] Here, f CRU (·) 1x1 convolution can be used to compress the number of channels, aiming to reduce the complexity of the feature map.

[0058] Moreover, after channel dimension reduction, a lightweight channel attention mechanism can be applied to dynamically adjust the importance of different channels. After compressing the channel dimension of the feature map, a lightweight channel attention mechanism is applied to dynamically adjust the importance of different channels by calculating the weight of each channel. The weight calculation formula is: α i =σ(FC(F cru ))

[0059] Among them, α i is the weight coefficient for each channel, σ(·) is the Sigmoid activation function, and FC(·) is the channel weighted learning through a small fully connected layer. This mechanism dynamically adjusts the contribution of each channel in the feature map by learning the importance of each channel, thereby improving the recognition accuracy of cloud features. Through the calculated channel weights, the model can focus on important channel features and suppress irrelevant redundant features. This allows the model to focus on more important channel features and add the reconstructed feature map to the original input feature map to retain key information to form a new feature representation: final =F cru α i +F in This process also uses skip connections to improve feature fluidity.

[0060] By comparing the model performance before and after adding the SCConv module in the training and validation stages, the experimental results show that the module significantly reduces the computational complexity in cloud detection tasks while maintaining high-precision detection results. Evaluation indicators such as accuracy and intersection over union (IoU) all show that the SCConv module can effectively improve the performance of cloud detection.

[0061] Step 4. Output the final cloud detection result by combining the feature extraction of the encoder and decoder, the lightweight attention mechanism, and the SCConv module, and post-process the result. Specifically, the post-processing includes noise removal, boundary smoothing, and threshold processing. Noise removal uses morphological operations (opening and closing operations) to remove small noise points in the segmentation results, so as to effectively eliminate isolated noise areas and enhance the overall structure of the cloud layer. Boundary smoothing uses image smoothing technology to enhance the coherence of cloud boundaries, which helps to eliminate jagged edges in the detection results and make the final output cloud mask more natural and realistic. Threshold processing generates the final binary cloud mask by thresholding the probability map output by the model. According to the experimental results, selecting an appropriate threshold (such as 0.5) can effectively distinguish between clouds and non-cloud areas and ensure the accuracy of the detection results.

[0062] A lightweight attention cloud detection method based on deep learning in the present invention has been tested on typical remote sensing image datasets, including ImageNet-1k and PASCAL VOC, and compared with the classic U-Net model. The experimental results show that after adding the lightweight attention mechanism and the SCConv module, the cloud detection accuracy of the model is significantly improved and the computational complexity is significantly reduced. Visual analysis shows that the lightweight attention mechanism can effectively capture the spatial characteristics of the cloud layer, and the SCConv module suppresses redundant features, making the cloud detection results more accurate.

[0063] The above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Any modification or equivalent substitution that does not depart from the spirit and scope of the present invention shall be included in the scope of the technical solution of the present invention.

Claims

1. A lightweight attention cloud detection method based on deep learning, characterized in that: The following steps are involved: Step 1. Obtain remote sensing image data, build a deep learning model containing multiple convolutional layers through remote sensing image data, and extract the features of each convolutional layer to obtain the feature map of each convolutional layer. The feature map output by each convolutional layer is expressed as: F out =σ(Conv(F in ,W)+b), where F in is the input feature map, Conv is the convolution operation, W is the convolution kernel weight, b is the bias, and σ is the activation function; Step 2. Introduce a lightweight attention mechanism in the feature extraction process, weight the features according to their importance, and send the features to three different convolution branches to generate three different feature representations respectively. Use different weights to extract different features, and then obtain the final attention feature representation by weighted summation of the three different feature representations; Step 3. Use the space-channel reconstruction convolution module to reduce the spatial dimension of the input feature map and compress the channel dimension; Step 4. Output the final cloud detection results and post-process the results.

2. According to claim 2, a lightweight attention cloud detection method based on deep learning is characterized in that: In step 1, the deep learning model includes an encoder and a decoder. The encoder downsamples layer by layer, extracts multi-scale features through convolution and pooling operations, and uses an activation function in each convolution layer to introduce nonlinearity, which is defined as: f(x)=x·tanh(softplus(x)), where the softplus function is defined as: softplus(x)=ln(1+e x ); The decoder upsamples layer by layer to restore the spatial resolution and fuse the detailed features in the jump connection; the downsampling formula is: F down =MaxPool(σ(Conv(F in ,W)+b)), the upsampling formula is: F up =Conv(UpSample(F in ),W)+b.

3. According to the light-weight attention cloud detection method based on deep learning in claim 1, it is characterized in that: In step 2, the three different feature representations are: The formula for weighted summation is:

4. According to the light-weight attention cloud detection method based on deep learning in claim 1, it is characterized in that: In step 3, the space-channel reconstruction convolution module includes a space reconstruction unit and a channel reconstruction unit. The space reconstruction unit is used to reduce the dimension of the input feature map to compress the spatial redundant features. Its operation formula is: sru =f SRU (F in )+F in , where f SRU (·) is a dimensionality reduction function implemented by convolution, which extracts the spatial information of the feature through a parameterized convolution kernel. The channel reconstruction unit is used to compress the channel dimension of the feature map to suppress redundant features and emphasize key features. Its operation formula is: cru =f CRU (F in )+F in .

5. According to the light-weight attention cloud detection method based on deep learning in claim 4, it is characterized in that: After compressing the channel dimension of the feature map, a lightweight channel attention mechanism is applied to dynamically adjust the importance of different channels by calculating the weight of each channel. The weight calculation formula is: α i =σ(FC(F cru )), where α i is the weight coefficient for each channel, σ(·) is the Sigmoid activation function, and FC(·) is the channel weighted learning through a small fully connected layer; and the reconstructed feature map is added to the original input feature map to retain the key information to form a new feature representation: F final =F cru α i +F in .

6. According to the light-weight attention cloud detection method based on deep learning in claim 1, it is characterized in that: In step 4, post-processing includes noise removal, boundary smoothing and threshold processing, wherein noise removal uses morphological operations to remove small noise points in the segmentation results, boundary smoothing uses image smoothing technology to enhance the coherence of cloud boundaries, and threshold processing thresholds the probability map output by the model to generate a final binary cloud mask.

Images