Remote Sensing Image Cloud Detection Method and System Based on Bottleneck Attention Module

Abstract

The present invention provides a remote sensing image cloud detection method and system based on a bottleneck attention module, belonging to the technical field of remote sensing image processing. The method steps are as follows: Step 1, obtain and process the sample set; Step 2, build a remote sensing image cloud detection network framework; Step 3, perform iterative training on the remote sensing image cloud detection network; Step 4, output the remote sensing image cloud detection prediction result. The detection system includes four parts: a sample acquisition module, a network framework building module, a training module, and an output module. The remote sensing image cloud detection network constructed by the present invention adopts a bottleneck attention structure, which combines the respective advantages of lightweight convolution, linear reverse residual structure, and coordinate attention mechanism, mainly solving the problems such as reduced feature quantity and decreased accuracy caused by the adoption of lightweight convolution in the network, realizing a cloud detection system with low parameter quantity and low computational complexity, and achieving a lightweight on-orbit cloud detection effect with higher accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image processing, and particularly to a remote sensing image cloud detection method and system based on a bottleneck attention module. Background Art

[0002] Modern remote sensing satellite images have been widely used in people's production and life. However, due to cloud blockage, the available ground object information in the downloaded images in satellite remote sensing is less, or even unusable. With the continuous development of artificial intelligence, cloud detection methods based on deep learning have made new progress in remote sensing image cloud detection technology. Through its powerful feature generalization and expression capabilities, it has become the main research method in many fields of current image processing. Currently, there are mainly three methods for remote sensing cloud detection: rule-based cloud detection methods, machine learning-based cloud detection methods, and deep learning-based cloud detection methods.

[0003] Rule-based cloud detection methods need to set corresponding thresholds according to the image transmission technology of different detectors to achieve cloud removal or achieve cloud removal according to graphics image processing technology. By setting reasonable thresholds or adopting multiple image processing processes, relatively high-precision cloud detection effects can usually be achieved. However, these methods undoubtedly require human experience and complex image processing processes, so there are deficiencies in poor universality and high complexity.

[0004] Machine learning-based cloud detection methods are implemented by combining some machine learning knowledge such as random forest, support vector machine, neural network, etc. with rule-based cloud detection methods. These methods can further improve the cloud detection accuracy. However, the complex process of artificially selecting features requires a lot of energy and time, and cannot meet the on-orbit cloud detection effect of remote sensing satellites.

[0005] Deep learning-based cloud detection methods usually use convolutional neural networks with high depth and large width, and a high-precision remote sensing image cloud detection model can be obtained. However, as the network depth deepens, the number of network parameters and the amount of computation will also increase significantly, seriously affecting the on-orbit cloud detection speed and space occupancy of the remote sensing satellite system. Therefore, how to reduce the amount of computation and the number of parameters of the network has become a difficult problem. Currently, lightweight convolutional neural networks mainly use lightweight convolutions to achieve lightweight effects. However, due to their unique convolution process, lightweight convolutions lead to a reduction in network feature quantities and a significant decrease in accuracy, and cannot achieve high-precision on-orbit cloud detection effects. Summary of the Invention

[0006] Object of the Invention: To propose a remote sensing image cloud detection method based on a bottleneck attention module, and further propose a system for implementing the above method, so as to effectively solve the above problems existing in the prior art.

[0007] First, a remote sensing image cloud detection method based on a lightweight neural network is proposed. The steps of the method are as follows:

[0008] Step 1: Obtain a sample set;

[0009] Step 2: Build a remote sensing image cloud detection network framework;

[0010] Step 3: Iterative training;

[0011] Step 4: Output the prediction results and conduct comparative analysis.

[0012] In a further embodiment of the first aspect, the process of obtaining the sample set further includes:

[0013] Step 1-1, download the required sample set from the China Resources Satellite Application Satellite and remove the samples without clouds or with too little clouds. Use the labelme label making software to label the qualified samples with cloud areas for each sample to make the total sample set N:

[0014] N={(I1,L1),(I2,L2),…,(I N ,L N )

[0015] In the formula, I N represents the nth image, L n Represents the label of the nth image;

[0016] Step 1-2: Randomly select n remote sensing image labels from remote sensing image N to form a training sample set P:

[0017] P={(I1,L1),(I2,L2),…(I n-1 ,L n-1 ),(I n ,L n )

[0018] In the formula, I n represents the nth training sample, L n Represents the label of the nth training image;

[0019] Step 1-3: The remaining Nn remote sensing images and labels form the test sample set T:

[0020] T={(I1,L1),(I2,L2),…(I N-n-1 ,L N-n-1 ),(I N-n ,L N-n )

[0021] In the formula, I N-n represents the Nnth test image, LN-n The label representing the (N - n)-th test image; wherein, in a further embodiment of the first aspect, the process of building the remote sensing image cloud detection network framework further includes:

[0022] Step 2-1: Build a main framework composed of an encoder, a decoder, a bottom connection layer, and a combination module;

[0023] Step 2-2: Define the remote sensing image cloud detection loss function:

[0024] L = L1 + L2

[0025] In the formula, L is the total loss function, L1 is the binary cross-entropy function, and L2 is the set similarity function;

[0026] Among them, the expression of the binary cross-entropy function L1 is as follows:

[0027]

[0028]

[0029] In the formula, T i represents the label of sample i, the positive class is 1, and the negative class is 0. N is the total number of samples. P i is the probability of class i calculated by the Softmax classifier. The main function of Softmax is to calculate the output of the network as the probability distribution of each class. k is the number of classes, Z i , Z j is the predicted output result of classes i and j in the network.

[0030] The expression of the DICE_LOSS (set similarity) function L2 is as follows:

[0031]

[0032] In the formula, TP is the case where the positive sample is predicted as a positive sample by the network. FN is the case where the negative sample is predicted as a positive sample by the network. FP is the case where the negative sample is predicted as a negative sample by the network.

[0033] In a further embodiment of the first aspect, the process of iteratively training the remote sensing image cloud detection network further includes:

[0034] Step 3-1: Adopt the frozen training method. The initial number of iterations is set to 300, and the initial learning rate is set to 10 -4 , when the number of iterations reaches 300, the learning rate is converted to 10 -5 , and the maximum number of iterations is set to 600;

[0035] Step 3-2: Input the training sample set into the remote sensing image cloud detection network in batches for the forward propagation process, and obtain the network prediction result x each time after passing through the intermediate hidden layer.

[0036] Step 3-3: Use the backpropagation algorithm to perform feature learning on the network. Calculate the error ε between the prediction result of the network in each iteration and the corresponding label of the training samples in this batch using the loss function. Then use the Adam optimizer to update the network convolution kernel weight parameters and the intermediate layer connection parameters to reduce the error ε between the network prediction result and the label. Finally, obtain the model weight parameters after 600 iterations.

[0037] In a further embodiment of the first aspect, the process of outputting the remote sensing image cloud detection prediction result further includes:

[0038] Input the image test set m into the trained remote sensing image cloud detection network for prediction to obtain the final model prediction result.

[0039] In the second aspect, a remote sensing image cloud detection system is proposed. The system includes four parts: a sample acquisition module, a network framework construction module, a training module, and an output module. The sample acquisition module is used to acquire the sample set; the network framework construction module is used to construct the remote sensing image cloud detection network framework; the training module is used to perform iterative training on the remote sensing image cloud detection network; the output module is used to output the remote sensing image cloud detection prediction result.

[0040] The image cloud detection network framework constructed by the network framework construction module consists of an encoder, a decoder, a bottom connection layer, and a bottleneck attention module. The encoder contains 4 layers of blocks. Each layer of the encoder contains 2 lightweight convolutional layers of 3×3 and a downsampling layer. At the same time, the input of each layer is the output result of the downsampling of the previous layer. The decoder also contains four layers of blocks. Each layer is 2 lightweight convolutional layers and an upsampling layer. The input of each layer is the output result of the upsampling of the previous layer and is concatenated with the output of the bottleneck attention module of the same layer; the encoder and the decoder are connected through the bottleneck attention module. The bottleneck attention module is a combination of a linear reverse residual and a coordinate attention mechanism. The input of the bottleneck attention module is the output of each layer of the encoder after passing through two lightweight convolutional layers. The bottom connection layer contains 2 lightweight convolutional layers of 3×3. The input is the output result of the downsampling of the fourth layer of the encoder, and the output is connected to each layer of the bottleneck attention module through the upsampling layer to achieve the feature fusion step.

[0041] The downsampling module includes a convolutional layer, a batch normalization layer, a ReLu activation function, and a max pooling layer; the upsampling module includes a convolutional layer, a batch normalization layer, a ReLu activation function, and an upsampling layer; the lightweight module includes a depthwise separable convolution, a ReLu activation function, a Ghost module, and a batch normalization layer. The bottleneck attention includes a linear reverse residual module and a coordinate attention mechanism module. The linear reverse residual includes two pointwise convolutional layers, a lightweight convolutional layer, and a residual module. The coordinate attention mechanism module includes two-direction global max pooling layers, a fully connected layer, a batch normalization layer, a convolutional layer, and a sigmoid function layer.

[0042] In a third aspect, a cloud detection device is proposed, which includes: at least one processor and a memory; the memory stores computer-executable instructions; at least one processor executes the computer-executable instructions stored in the memory, so that at least one processor executes the remote sensing image cloud detection method as described in the first aspect.

[0043] In a fourth aspect, a readable storage medium is proposed, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the remote sensing image cloud detection method as described in the first aspect is implemented.

[0044] Beneficial effects: The remote sensing image cloud detection network constructed by the present invention adopts a bottleneck attention module to solve the problems of reduced feature quantity and decreased accuracy after the network adopts lightweight convolution. At the same time, the bottleneck attention module proposed by the present invention combines the respective advantages of lightweight convolution, linear reverse residual module, and attention mechanism, enabling the network to extract deep features. At the same time, the coordinate attention mechanism decomposes the channel attention into two global average poolings for each channel along different directions. The attention module captures long-term dependencies in one spatial direction and maintains precise position information in the other spatial direction, which helps the network to more accurately locate the object of interest, and at the same time captures long-distance spatial interactions through precise position information, increasing the connection between different channels. Finally, the accuracy of remote sensing image cloud detection is improved. Description of the Drawings

[0045] Figure 1 is the implementation flowchart of the present invention.

[0046] Figure 2 is a schematic diagram of the remote sensing image cloud detection network of the present invention.

[0047] Figure 3 is a schematic diagram of the downsampling module structure constructed by the present invention.

[0048] Figure 4 is a schematic diagram of the upsampling module structure constructed by the present invention.

[0049] Figure 5 It is a schematic diagram of the overall structure of the lightweight convolution module constructed by the present invention.

[0050] Figure 6 It is a schematic diagram of the structure of the combined module constructed by the present invention.

[0051] Figure 7 It is a schematic diagram of the structure of the coordinate attention mechanism constructed by the present invention. Detailed implementation manners

[0052] In the following description, a large number of specific details are given to provide a more thorough understanding of the present invention. However, it is obvious to those skilled in the art that the present invention can be implemented without one or more of these details. In other examples, some technical features well known to those skilled in the art are not described to avoid confusion with the present invention.

[0053] This embodiment proposes a remote sensing image cloud detection method based on a lightweight neural network. This method is implemented by a remote sensing image cloud detection system, which includes four parts: a sample acquisition module, a network framework construction module, a training module, and an output module. The sample acquisition module is used to acquire a sample set; the network framework construction module is used to construct a remote sensing image cloud detection network framework; the training module is used to iteratively train the remote sensing image cloud detection network; the output module is used to output the remote sensing image cloud detection prediction result.

[0054] The image cloud detection network framework constructed by the network framework construction module is composed of an encoder, a decoder, a bottom connection layer, and a bottleneck attention module. The encoder includes 4 layers of building blocks. Each layer of the encoder includes 2 lightweight convolution layers of 3×3 and a downsampling layer. At the same time, the input of each layer is the output result of the downsampling of the previous layer. The decoder also includes four layers of building blocks. Each layer is 2 lightweight convolution layers and an upsampling layer. The input of each layer is the output result of the upsampling of the previous layer and the output of the bottleneck attention module of the same layer is concatenated; the encoder and the decoder are connected through the bottleneck attention module. The bottleneck attention module is a combined module of lightweight convolution, linear reverse residual, and coordinate attention mechanism. The input of the bottleneck attention module is the output of each layer of the encoder after passing through two lightweight convolution layers. The bottom connection layer includes 2 lightweight convolution layers of 3×3. The input is the output result after the downsampling of the fourth layer of the encoder, and the output is connected to the bottleneck attention module of each layer through an upsampling layer to achieve feature fusion.

[0055] The downsampling module includes a convolutional layer, a batch normalization layer, a ReLu activation function, and a max pooling layer; the upsampling module includes a convolutional layer, a batch normalization layer, a ReLu activation function, and an upsampling layer; the lightweight module includes a depthwise separable convolution, a ReLu activation function, a Ghost module, and a batch normalization layer. The bottleneck attention module includes a linear reverse residual module and a coordinate attention mechanism module. The linear reverse residual includes two pointwise convolutional layers and a lightweight convolutional layer. The coordinate attention mechanism module includes two-direction global max pooling layers, a fully connected layer, a batch normalization layer, a convolutional layer, and a sigmoid function layer.

[0056] Specifically, the remote sensing image cloud detection method based on a lightweight neural network includes the following steps, as shown in Figure 1 :

[0057] Step 1: Obtain the training sample set and the test sample set:

[0058] The data set downloads the required sample set from the China Center for Resources Satellite Data and Application, and screens 3,168 remote sensing images with labels and containing a large number of cloud regions from it. First, the 3,168 remote sensing images are composed into a sample set, and the labelme label-making software is used for manual annotation of the cloud-containing regions. Randomly select 2,068 labeled images as the training sample set and input them into the model for iterative training. The remaining 1,100 labeled remote sensing images are used as the test sample set. Among them, the remote sensing images have the same size as the labels and are binary images. The area with a pixel value of 0 in the label corresponds to the classification of background ground objects, and the point with a pixel value of 255 in the label area corresponds to the classification of clouds. Considering the network training problem and reducing the processing time of each scene, and accelerating the learning process of global information, the size of each scene image is reduced to 1320×1200, and the spatial resolution is 160 meters, and the data is normalized to between 0 and 1 during training.

[0059] Step 2: Construct a remote sensing image cloud detection network:

[0060] 2a) As Figure 2The first framework shown consists of an encoder, a decoder, a bottom connection layer, and a bottleneck attention module. The encoder contains 4 layers of blocks, and each layer of the encoder contains 2 lightweight convolutional layers with a convolutional kernel size of 3×3 and a downsampling layer. At the same time, the input of each layer is the output result of the downsampling of the previous layer. The decoder also contains four layers of blocks, each layer being 2 lightweight convolutional layers and an upsampling layer. The input of each layer is the output result of the upsampling of the previous layer and is concatenated with the output of the bottleneck attention module of the same layer. The encoder and the decoder are connected through the bottleneck attention module. The bottleneck attention module is a combination of a linear reverse residual and a coordinate attention mechanism. The input of the combined module is the output of each layer of the encoder after passing through two lightweight convolutional layers. The bottom connection layer contains 2 lightweight convolutional layers with a convolutional kernel size of 3×3. The input is the output result after the downsampling of the fourth layer of the encoder, and the output is connected to each layer of the bottleneck attention module through an upsampling layer to achieve feature fusion.

[0061] The downsampling module is as Figure 3 shown. First, this module contains a convolutional layer, a batch normalization layer, a ReLu activation function, and a max pooling layer. The upsampling module is as Figure 4 shown and contains a convolutional layer, a batch normalization layer, a ReLu activation function, and an upsampling layer. The lightweight convolutional layer is as Figure 5 shown. This module contains a depthwise separable convolutional layer, a ReLu activation function, a Ghost module, and a batch normalization layer. The depthwise separable convolutional layer first uses a convolutional layer with a convolutional kernel size of 3*3 to separate the RGB channels of the image and then performs convolutional operations separately to generate feature maps, and finally synthesizes a new feature map. This operation can greatly reduce the number of parameters and the amount of calculation during the convolution process.

[0062] The Ghost module also consists of two parts. The first part first extracts features from the feature map using a convolutional layer, a batch normalization layer, and a ReLu activation function. Then, the second part performs linear folding mapping on the feature map through another round of convolutional layer, batch normalization layer, and ReLu activation function, and a hyperparameter S is set. By changing the value of the hyperparameter S, the dimension of the linear mapping is changed. Then, a concatenation layer is used to directly concatenate the feature map obtained by the mapping on the feature map obtained by the first part. Through this operation of the dimension reduction convolution method, the computational amount of the network can be reduced.

[0063] The bottleneck attention module is as Figure 6As shown, this module combines the respective advantages of lightweight convolution, linear inverse residual module, and coordinate attention mechanism. The overall structure of the linear inverse residual module includes two pointwise convolution layers and one lightweight convolution layer. This structure first expands and raises the dimension to a high-dimensional space through a pointwise convolution with a kernel size of 1×1. Secondly, a depthwise separable convolution with a kernel size of 3×3 is used for feature extraction. Finally, a pointwise convolution of 1×1 is used to reduce the dimension to a low-dimensional space for operations. This allows the network to extract more deep-level feature information in the high-dimensional space while only increasing the computational complexity slightly. At the same time, to control the scale of dimension transformation, a hyperparameter μ is introduced to adjust the expansion ratio of different dimensions, which is usually set to 6. The overall structure of the coordinate attention mechanism module is as shown in Figure 6 As shown, it includes global max pooling layers, fully connected layers, batch normalization layers, convolutional layers, and Sigmoid function layers in two directions, including convolutional layers, max pooling layers, batch normalization layers, and ReLu activation layers cascaded in sequence. Among them, the max pooling layer can reduce the dimension, remove redundant information, compress features, reduce network complexity, and reduce the computational amount. The convolutional layer is used to extract features and raise or lower the dimension. The addition of the batch normalization layer can avoid the phenomena of gradient dispersion and gradient explosion during the training process. The introduction of the ReLu activation function can increase the generalization ability of the model and also solve the problem of gradient disappearance.

[0064] The formula for the number of parameters of the standard convolution is as follows:

[0065] D K ×D K ×M×N

[0066] The formula for calculating the number of parameters of the depthwise separable convolution is as follows:

[0067] D K ×D K ×M+M×N

[0068] The formula for the number of parameters of the Ghost module is as follows:

[0069]

[0070] The change in the number of parameters between the lightweight convolution and the standard convolution is as follows

[0071]

[0072] The formula for the computational amount of the standard convolution is as follows:

[0073] D F ·D F ·D K ·D K ·M·N

[0074] The formula for the computational amount of the depthwise separable convolution is as follows:

[0075] D K ·D K ·M·D F ·D F +M·N·D F ·D F

[0076] The computational complexity formula of the Ghost module is as follows:

[0077]

[0078] The change in the total computational complexity of the lightweight convolutional kernel standard convolution is as follows:

[0079]

[0080] D K ×D K ×M×N is the formula for calculating the number of parameters of the traditional convolution, D K ×D K ×M+M×N is the formula for calculating the number of parameters of the depth lightweight convolution, D F ·D F ·D K ·D K ·M·N is the formula for calculating the total computational complexity of the traditional convolution, D K ·D K ·M·D F ·D F +M·N·D F ·D F is the formula for calculating the total computational complexity of the depth lightweight.

[0081] Among them, D K represents the convolution kernel size, D F is the feature layer size, M is the number of input channels, N is the number of output channels, and S is a hyperparameter that controls the compression ratio.

[0082] Step 3: Iteratively train the remote sensing image cloud detection network:

[0083] Step 3-1: Adopt the frozen training method. The initial number of iterations is set to 300, and the initial learning rate is set to 10 -4 , when the number of iterations reaches 300, the learning rate is converted to 10 -5 , and the maximum number of iterations is set to 600;

[0084] Step 3-2: Input the training sample set into the remote sensing image cloud detection network in batches for forward propagation, and obtain the network prediction results for each time after passing through the intermediate hidden layer;

[0085] Step 3-3: Use the backpropagation algorithm to perform feature learning on the network. Use the loss function to calculate the error ε between the prediction result of the network for each iteration and the corresponding label of the training samples in this batch. Then use the Adam optimizer to update the weights of the network convolutional kernels and the connection parameters of the intermediate layers, reducing the error ε between the network prediction result and the label. Finally, obtain the model weight parameters after 600 iterations.

[0086] 1) Adopt the frozen training method. Set the initial number of iterations to 300 and the initial learning rate to 10 -4 , when the number of iterations reaches 300, the learning rate is converted to 10 -5 , and set the maximum number of iterations to 600;

[0087] 2) Input the training set samples into the remote sensing image cloud detection network for forward propagation training. Extract multi-scale feature information through continuous downsampling in the encoding module stage. At the same time, obtain the prediction result x of the network for each time after passing through the intermediate hidden layer;

[0088] 3) Use the backpropagation algorithm to perform feature learning on the network. Use the loss function to calculate the error ε between the prediction result of the network for each iteration and the corresponding label of the training samples in this batch. Then use the Adam optimizer to update the weights ω t of the network convolutional kernels and the connection parameters ω t of the intermediate layers, reducing the error ε between the network prediction result and the label. Finally, obtain the model weight parameters after 600 iterations. The update formulas are as follows:

[0089]

[0090]

[0091] where η is the step size, usually η = 1×10-5, ω t+1 , v t+1 respectively represent the update results of ω t , v t , and represents the partial derivative result.

[0092] Step Four: Obtain the remote sensing image cloud detection prediction result:

[0093] Input the remote sensing image test set m into the trained remote sensing image cloud detection network for testing to obtain the final prediction result. The prediction result set represents the cloud detection result image of the probability of each pixel being a cloud. Each pixel in the cloud detection result image represents the detection result of the corresponding pixel in the input image. If the detection is classified as a cloud, its color is set to white, and the corresponding pixel is 255; otherwise, it is the background, set to black, and the corresponding pixel is 0.

[0094] A remote sensing image cloud detection system, which includes four parts: a sample acquisition module, a network framework construction module, a training module, and an output module. The sample acquisition module is used to acquire a sample set; the network framework construction module is used to construct a remote sensing image cloud detection network framework; the training module is used to iteratively train the remote sensing image cloud detection network; the output module is used to output the remote sensing image cloud detection prediction result.

[0095] The image cloud detection network framework constructed by the network framework construction module consists of an encoder, a decoder, a bottom connection layer, and a bottleneck attention module. Among them, the encoder contains 4 layers of blocks, and each layer of the encoder contains 2 lightweight convolutional layers with a convolutional kernel size of 3×3 and a downsampling layer. At the same time, the input of each layer is the output result of the downsampling of the previous layer. The decoder also contains four layers of blocks, each layer is 2 lightweight convolutional layers and an upsampling layer, and the input of each layer is the upsampling output result of the previous layer and the output of the bottleneck attention module of the same layer is concatenated; the encoder and the decoder are connected through the bottleneck attention module. The bottleneck attention module is a combination of a linear reverse residual and a coordinate attention mechanism. The input of the bottleneck attention module is the output of each layer of the encoder after two lightweight convolutional layers. The bottom connection layer contains 2 lightweight convolutional layers with a convolutional kernel size of 3×3, the input is the output result after the downsampling of the fourth layer of the encoder, and the output is connected to the bottleneck attention module of each layer through the upsampling layer to achieve feature fusion.

[0096] The downsampling module includes a convolutional layer, a batch normalization layer, a ReLu activation function, and a max pooling layer; the upsampling module includes a convolutional layer, a batch normalization layer, a ReLu activation function, and an upsampling layer; the lightweight module includes a depthwise separable convolution, a ReLu activation function, a Ghost module, and a batch normalization layer. The bottleneck attention module includes a lightweight convolution, a linear reverse residual module, and a coordinate attention mechanism module. The linear reverse residual includes two pointwise convolutional layers and a lightweight convolutional layer. The coordinate attention mechanism module contains two-direction global max pooling layers, a fully connected layer, a batch normalization layer, a convolutional layer, and a sigmoid function layer.

[0097] Specific experiments:

[0098] In the experiment, two evaluation metrics, the Dice coefficient and the Intersection Over Union (IOU), are used as the evaluation criteria for the binary classification cloud detection experiment. At the same time, in order to quantify the actual improvement effect of the module, the total number of parameters (Total Parameter) and the average prediction speed per image (Speed) are introduced as references.

[0099] The Dice coefficient is defined as follows:

[0100]

[0101] Among them, |X img ∩Y label | represents the intersection between the prediction result and the label. The larger the intersection, the stronger the similarity between the two samples. The numerator is multiplied by 2 to ensure that the final result can be within [0, 1]. The cross ratio is the ratio of the correctly classified pixels of a class to the total actual pixels of that class. The calculation formula is:

[0102]

[0103] Among them, P ii is the correctly classified pixel, P ij is the pixel where the cloud is classified as non-cloud, and P ji is the pixel where the non-cloud is classified as cloud.

[0104] Table 1 Module comparison results

[0105]

[0106] In the above table, U-NetA represents the result of training the Unet model by replacing the standard convolution with lightweight convolution. U-NetB represents the experimental result of introducing a linear reverse residual module after using lightweight convolution in the Unet model. U-NetC represents the experimental result of introducing a coordinate attention mechanism after using lightweight convolution in the Unet model. The method in this paper is the experimental result of jointly training lightweight convolution and the other two modules. At the same time, replacing the standard convolution with lightweight convolution can significantly reduce the number of network parameters, but the accuracy will also decrease accordingly. The roles of the two modules can be demonstrated by introducing the linear reverse residual and the coordinate attention mechanism, and the result after combining the two modules also proves that the combination effect is the best.

[0107] Table 2 Comparison results of cloud detection methods

[0108]

[0109] As can be seen from the above table, the method in this paper is an improvement based on the U-Net method. In the field of remote sensing image cloud detection, the method of the present invention is superior to other semantic segmentation cloud detection methods in terms of speed, accuracy, and the number of parameters. At the same time, on the basis of the original remote sensing cloud detection method, only a small amount of detection speed needs to be sacrificed to achieve a significant reduction in the number of parameters and an improvement in accuracy, which proves the effectiveness of the combined module of the present invention.

[0110] In summary, the present invention proposes a high-precision remote sensing cloud image detection method based on a bottleneck attention module. Specifically, a combined module that combines lightweight convolution, a linear reverse residual module, and a coordinate attention mechanism is invented, which is mainly used to solve the problems such as the reduction of network feature quantity and the decline of accuracy after the network adopts lightweight convolution. The present invention combines the unique bottleneck structure of the linear reverse residual to first increase the dimension to a high-dimensional space, uses lightweight convolution for feature extraction, and then reduces the dimension to the original dimension for convolution operation, avoiding the disadvantages of lightweight convolution and improving the accuracy. Among them, the coordinate attention mechanism improves the network's ability to extract features of the cloud edge details in remote sensing images and enhances the generalization ability of the network. And the bottleneck attention module can be inserted into other network structures and is applicable to all remote sensing image pixel segmentation fields to directly improve its detection accuracy.

[0111] As described above, although the present invention has been shown and described with reference to specific preferred embodiments, it should not be construed as a limitation of the present invention itself. Various changes can be made in its form and details without departing from the spirit and scope of the present invention defined by the appended claims.

Claims

1. A remote sensing image cloud detection method based on a bottleneck attention module, characterized in that The steps include: Step 1: Obtaining and processing sample sets; Step 2: Build the overall framework of the remote sensing image cloud detection network; the overall framework of the remote sensing image cloud detection network includes at least four layers of encoders, each layer of the encoder includes at least two 3×3 lightweight convolutional layers and at least one downsampling layer, and the input of each layer is the output result of the downsampling of the previous layer; At least four layers of decoders, each layer of the decoder includes at least two lightweight convolutional layers and at least one upsampling layer, the input of each layer is the upsampling output result of the previous layer and cascades the output of the combination module of the same layer; the encoder and the decoder are connected through a bottleneck attention module; the combination module is a bottleneck attention module; The bottom layer is the connection layer, which includes at least two 3×3 lightweight convolutional layers. The input is the output result of the fourth layer of the encoder after downsampling. The output is connected to the combination module of each layer through the upsampling layer to achieve feature fusion. A bottleneck attention module is used to connect the encoder and the decoder; the bottleneck attention module is a combination of a linear reversal residual module and a coordinate attention mechanism module, and the input of the bottleneck attention module is the output of each layer of encoder after two lightweight convolution layers; the linear reversal residual module includes two point-by-point convolution layers and one lightweight convolution layer, firstly, the dimension is increased to a high-dimensional space through point-by-point convolution with a convolution kernel size of 1×1, and then a depth-separable convolution with a convolution kernel size of 3×3 is used for feature extraction, and finally, a 1×1 point-by-point convolution is used to reduce the dimension to a low-dimensional space for operation; the coordinate attention mechanism module includes a global maximum pooling layer in two directions, a fully connected layer, a batch normalization layer, a convolution layer, and a Sigmoid function layer; Step 3: Iterative training; Step 4: Output the prediction results and compare them.

2. The remote sensing image cloud detection method according to claim 1, characterized in that Step 1 further comprises: Step 1-1, download the predetermined sample set and remove samples with no cloud or too little cloud. Use labelme label making software to label each qualified sample with a cloud area as a label to make a total sample set N: ； Wherein, I N represents the nth image, and L n represents the label of the nth image; Step 1-2: Randomly select n remote sensing image labels from remote sensing image N to form a training sample set P: ； where I n represents the nth training sample, and L n represents the label of the nth training image; Step 1-3: Combine the remaining N-n remote sensing images and labels to form a test sample set : ； where I N-n represents the (N - n)-th test image, and L N-n represents the label of the (N - n)-th test image; where P >> T.

3. The remote sensing image cloud detection method according to claim 1, characterized in that, Step 2 further includes: Step 2-1, build the main framework consisting of encoder, decoder, bottom connection layer, and combination module; Step 2-2, define the remote sensing image cloud detection loss function: ； where L is the total loss function, is the binary cross-entropy function, is the set similarity function; Among them, the binary cross-entropy function has the following expression: ； ； Where, Ti represents the label of sample i, with the positive class being 1 and the negative class being 0; N is the total number of samples; is the probability of class i calculated by the Softmax classifier. The main function of Softmax is to calculate the output of the network as the probability distribution of each class. k is the number of classes, Z i , Z j is the predicted output result of classes i and j in the network; DICE_LOSS (Set Similarity) function The expression is as follows: ； In the formula, TP is the case where the positive sample is predicted as a positive sample by the network; FN is the case where the negative sample is predicted as a positive sample by the network; FP is the case where the negative sample is predicted as a negative sample by the network.

4. The remote sensing image cloud detection method according to claim 1, wherein, Step 3 further includes: Step 3-1: Adopt the frozen training method, set the initial number of iterations to 300, and set the initial learning rate to 10 -4 , when the number of iterations reaches 300, the learning rate is converted to 10 -5 , and set the maximum number of iterations to 600; Step 3-2: Input the training sample set into the remote sensing image cloud detection network in batches for forward propagation, and obtain each network prediction result x after passing through the intermediate layer; Step 3-3: Use the backpropagation algorithm to perform feature learning on the network, and use the loss function to calculate the error between the prediction result of the network in each iteration and the corresponding label of the training samples in this batch. Then use the Adam optimizer to update the weight parameters of the network convolution kernel and the connection parameters of the intermediate layer, reduce the error ε between the network prediction result and the label, and finally obtain the model weight parameters after 600 iterations.

5. Remote sensing image cloud detection system, characterized in that, The system comprises: A sample acquisition module, used to acquire and process sample sets; Network framework building module, used to build a remote sensing image cloud detection network framework; A training module, used for iterative training of remote sensing image cloud detection network; Output module, used to output the prediction results of remote sensing image cloud detection network; The remote sensing image cloud detection network framework built by the network framework building module further includes: At least four layers of encoders, each layer of the encoder includes at least two 3×3 lightweight convolutional layers and at least one downsampling layer, and the input of each layer is the output result of the downsampling of the previous layer; At least four layers of decoders, each layer of the decoder includes at least two lightweight convolutional layers and at least one upsampling layer, the input of each layer is the output result of the upsampling of the previous layer and the output of the combination module of the same layer is concatenated; the encoder and the decoder are connected through a bottleneck attention module; the combination module is a bottleneck attention module; Bottom-layer connection layer, including at least two 3×3 lightweight convolutional layers, the input is the output result after the fourth layer of the encoder is downsampled, and the output is connected to the combination module of each layer through an upsampling layer to achieve feature fusion; Bottleneck attention module, used to connect the encoder and the decoder; the bottleneck attention module is a combination of a linear reverse residual module and a coordinate attention mechanism module, and the input of the bottleneck attention module is the output of each layer of the encoder after passing through two lightweight convolutional layers; the linear reverse residual module includes two pointwise convolutional layers and one lightweight convolutional layer. First, it is extended and dimension-increased to a high-dimensional space through a pointwise convolution with a kernel size of 1×1. Secondly, a depthwise separable convolution with a kernel size of 3×3 is used for feature extraction. Finally, a 1×1 pointwise convolution is used to reduce the dimension to a low-dimensional space for operation; the coordinate attention mechanism module includes global maximum pooling layers in two directions, a fully connected layer, a batch normalization layer, a convolutional layer, and a Sigmoid function layer.

6. Cloud detection device, characterized in that Including: At least one processor and a memory; The memory stores computer execution instructions; At least one processor executes the computer execution instructions stored in the memory, so that at least one processor executes the remote sensing image cloud detection method according to any one of claims 1 to 4.

7. A readable storage medium, characterized in that, Computer execution instructions are stored in the readable storage medium, and when the processor executes the computer execution instructions, the remote sensing image cloud detection method according to any one of claims 1 to 4 is implemented.

Images