A geoscience prior knowledge guided lightweight remote sensing image cloud detection method, system, device and medium

Abstract

A geoscience prior knowledge guided lightweight remote sensing image cloud detection method, system, device and medium, the method comprises: acquiring a remote sensing image dataset and preprocessing it; constructing an encoder module, a decoder module, a feature extraction module and a feature fusion bridge module; constructing a lightweight cloud detection network model; using the training set to iteratively train the lightweight cloud detection network model, and using the validation set to verify the effect of the lightweight cloud detection network model and calculate the relevant indicators at the interval of the fixed iteration number, selecting the model with the optimal validation process index as the final training model after the iteration training is completed, saving the model parameters as a file, and obtaining the optimal network model file; using the optimal network model file to detect the cloud layer of the required remote sensing image; the system, device and medium are used to realize the invention; the invention solves the problems of large parameter quantity and high computational complexity of mainstream cloud detection neural networks, and improves the precision of cloud detection.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing image processing technology, specifically relating to a lightweight remote sensing image cloud detection method, system, device, and medium guided by prior geoscience knowledge. Background Technology

[0002] Cloud detection in remote sensing images is a crucial task in remote sensing image processing. It provides valuable information for fields such as weather forecasting, climate change, and hydrological cycles, and also offers effective data filtering and quality assessment for other remote sensing applications. Cloud segmentation aims to separate cloud-covered and non-cloud-covered regions in satellite images, thereby obtaining features such as cloud coverage, type, shape, and height. However, cloud detection is also a challenging task. The diverse morphologies, varying scales, blurred boundaries, and severe occlusion of clouds make traditional threshold-based, clustering, and edge detection methods ineffective in handling complex cloud scenes.

[0003] Existing cloud detection methods primarily rely on deep learning, utilizing models such as Convolutional Neural Networks (CNNs) or Fully Convolutional Networks (FCNs) for pixel-level semantic segmentation of satellite images. These methods offer stronger feature extraction and representation capabilities compared to traditional methods, improving the accuracy and efficiency of cloud segmentation to some extent. However, existing deep learning methods also have some drawbacks, such as: the large number of parameters and computational demands of the network models leads to slow operation, making them unsuitable for real-time applications; the network models are not sensitive enough to the frequency features of the input images, resulting in poor generalization ability and difficulty adapting to different cloud types and scenarios; and the network models do not sufficiently fuse multi-scale feature maps and mask maps, resulting in low resolution and accuracy, making it difficult to capture cloud details and boundaries. Furthermore, the utilization of prior geoscientific knowledge in current cloud detection methods is still in its early stages of research, and mining effective geoscientific knowledge and embedding it into the methods remains a challenging area of ​​research in remote sensing.

[0004] Patent application CN110598600A discloses a cloud detection method for remote sensing images based on the UNET neural network. This method achieves high-precision cloud detection in remote sensing images by extracting cloud features using a convolutional neural network based on the UNET network structure. However, due to its large number of parameters, the computational complexity is high, and the requirements for the operating environment are stringent, making it difficult to deploy on low-computing-power, low-power embedded devices. Furthermore, conventional convolutional layers do not effectively extract some remote sensing image features, such as frequency and dark channel values, causing the network model to overlook some important remote sensing image features during training.

[0005] Patent application CN111915592A discloses a deep learning-based cloud detection method for remote sensing images. It incorporates a dark channel sample set and integrates wavelet transform image feature extraction and spatial attention modules based on the UNET network structure, resulting in more accurate cloud feature extraction, improved cloud detection accuracy, and reduced false detection rate. However, the problems of large model size, numerous parameters, and high computational complexity remain unresolved. Summary of the Invention

[0006] To overcome the shortcomings of the prior art, the present invention aims to provide a lightweight remote sensing image cloud detection method, system, device, and medium guided by geoscientific prior knowledge. Through a depthwise separable convolutional encoder and decoder, the number of model parameters can be effectively reduced, thereby lowering the computational requirements of the model deployment environment. A Hadamard product attention layer with linear computational complexity achieves an attention mechanism for multiple dimensions of size and channels with a small number of parameters. A feature extraction module based on Fast Fourier Transform can effectively extract the frequency features of remote sensing images, fully utilizing geoscientific prior knowledge to guide iterative training of the network model and improving the accuracy of cloud detection.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] A lightweight remote sensing image cloud detection method guided by prior geoscience knowledge includes the following steps:

[0009] Step 1: Obtain the remote sensing image dataset and preprocess it to obtain the preprocessed training set and validation set;

[0010] Step 2: Construct the encoder module, decoder module, feature extraction module, and feature fusion bridging module;

[0011] Step 3: The encoder module obtained in Step 2 is used to construct the downsampling sub-network of the network, the decoder module is used to construct the upsampling sub-network, the feature fusion bridging module is used to bridge the layers of the downsampling sub-network and the upsampling sub-network, and the feature extraction module is used as the network model entry point to connect with the downsampling sub-network to construct a lightweight cloud detection network model.

[0012] Step 4: Use the training set obtained in Step 1 to iteratively train the lightweight cloud detection network model constructed in Step 3. After a fixed number of iterations, use the validation set obtained in Step 1 to verify the effect of the lightweight cloud detection network model constructed in Step 3 and calculate relevant indicators. After the iterative training is completed, select the model with the best validation indicators as the final training model and save the model parameters as a file to obtain the optimal network model file.

[0013] Step 5: Use the optimal network model file obtained in Step 4 to perform cloud detection on the required remote sensing image.

[0014] The specific method for step 1 is as follows:

[0015] The acquired remote sensing image is segmented at a specific size and the cloud regions of the slices are manually labeled to obtain multi-channel layer slices and corresponding Ground Truth cloud labels. The slices are then divided into training and validation sets, which are then normalized and their dimensions are transformed.

[0016] The specific method for step 2 is as follows:

[0017] The encoder module is divided into a shallow encoder and a deep encoder. The shallow encoder consists of a depthwise separable convolutional layer, a BN normalization layer, and a ReLU activation layer, with a MaxPool pooling layer connected at the end, used for feature extraction and halving the width and height of the feature map, respectively. The deep encoder consists of a Hadamard product attention layer, a BN normalization layer, a ReLU activation layer, and a pooling layer. The Hadamard product attention layer achieves an attention mechanism for local regions of different dimensions by linearly calculating the feature map through the matrix Hadamard product and reducing the number of parameters. The depthwise separable convolution also effectively reduces the number of parameters in the convolutional layer through the separable structure.

[0018] The decoder module is divided into a shallow decoder and a deep decoder. The shallow decoder consists of a depthwise separable convolutional layer, a BN normalization layer, and a ReLU activation layer, with a deconvolutional layer connected at the end, which are used for feature extraction and restoration of the feature map width and height, respectively. The deep decoder consists of a Hadamard product attention layer, a BN normalization layer, a ReLU activation layer, and a deconvolutional layer. This module serves as the upsampling module for each layer of the encoder in the network structure, and is used to restore the resolution of the feature image.

[0019] The feature extraction module consists of an FFT layer and a dimension adaptation layer. The input feature map is first transformed in dimension by the dimension adaptation layer, and then fed into the FFT layer. It first undergoes a fast Fourier transform, and then the transformed feature map is subjected to an affine transform using learnable weight and bias parameters. The feature map is then restored by an inverse Fourier transform, and finally the dimension is restored by the dimension adaptation layer and output to the outside of the module.

[0020] The feature fusion bridging module integrates the encoder output features of the corresponding layer, the decoder output features of the previous layer, and the single-channel mask feature map of the corresponding scale of the features to achieve multi-scale feature fusion.

[0021] The specific method for step 3 is as follows:

[0022] Step 3.1: Connect the encoder and decoder modules hierarchically, using three layers as a baseline. The first two layers use shallow encoder and decoder modules, and the bottom layer uses deep encoder and decoder modules. Initialize the first-layer encoder with the original image size and the original number of channels of the input image, and output a feature map with half the size and 32 channels. Subsequent encoders are connected end to end and initialized according to the output dimension of the upper layer, outputting feature maps with half the size and doubled channels. The decoders are connected end to end in the same way, and the decoder modules are initialized in reverse according to the input and output parameters of the encoders in the same layer. The shallowest decoder outputs a single-channel feature map, so that the entire network outputs a single-layer mask image with the resolution of the input image. This constitutes the downsampling sub-network and the upsampling sub-network of the network.

[0023] Step 3.2: Initialize the frequency feature extraction module according to the original dimensions of the input image and connect it to the input end of the downsampling sub-network. In this way, the network can learn to adjust the RGB three-channel features to a suitable frequency and perform subsequent inference.

[0024] Step 3.3: Initialize the corresponding layer feature fusion bridging module according to the output dimension of the corresponding layer encoder and the input dimension of the corresponding layer decoder. First, perform a convolution operation with a kernel of 1 on the output of each layer decoder to obtain a mask image of the size of the corresponding layer with a single channel. Input the encoder output, the upper layer decoder output and the mask image into the feature fusion bridging module. The multi-scale output features are fused and superimposed on the upper layer decoder output feature map through the add operation and input into the current layer decoder so that it can participate in subsequent inference operations.

[0025] The specific method for step 4 is as follows:

[0026] Step 4.1: Select AdamW as the parameter optimizer, set the initial learning rate and weight decay factor, and select cosine annealing learning rate to dynamically adjust the learning rate;

[0027] Step 4.2: The performance of the lightweight cloud detection network model constructed in Step 3 is evaluated using the Binary Cross-Entropy Loss (BCE Loss) and Dice Loss functions. The calculation formula for the Binary Cross-Entropy Loss (BCE Loss) is as follows:

[0028] L BCE =-w n [y n ·logx n +(1-y n )·log(1-x n )]

[0029] Where w is the weight of each category, which is 1 by default, x is the model output, and y is the true value;

[0030] The formula for calculating the Dice Loss function is as follows:

[0031]

[0032] Where |X| represents the positive output of the model, |Y| represents the positive true value, and |X∩Y| represents the positive true value, which is the part where both the model output and the true value are positive.

[0033] The model performance is evaluated using a combination of the Binary Cross-Entropy Loss (BCE Loss) and Dice Loss functions, calculated as follows:

[0034] Loss = w BCE ×L BCE +w dice ×L dice

[0035] The weights of BCE Loss and Dice Loss are both set to 1 by default.

[0036] Step 4.3: Set the maximum number of iterations and the batch size per iteration;

[0037] Step 4.4: During each iteration, the training set is input into the lightweight cloud detection network model constructed in Step 3 in batches for calculation. After each calculation, the model output and the corresponding cloud layer labeled slice (ground truth slice) are input into the loss function to evaluate the performance and update the model parameters in reverse. Every fixed number of iterations, the model is evaluated using the validation set, the confusion matrix is ​​calculated, and the performance of various indicators is calculated accordingly. Each validation retains the network model file with the best performance in the iteration history.

[0038] The specific method for step 5 is as follows:

[0039] Step 5.1: Read the network model file that has been fully iterated and trained in Step 4, load the parameters of the network model file into the network model of the application, adjust the network model to inference mode, and fix the parameters of the BN layer.

[0040] Step 5.2: Import the required remote sensing image into the application. The application first segments the required remote sensing image and sets the size of the inner border (padding). The application segments the required remote sensing image with overlapping parts according to the size of the inner border (padding) and organizes the remote sensing image slices into the matrix shape required by the model.

[0041] Step 5.3: Input the remote sensing image slices obtained in Step 5.2 into the network model of the application to obtain the model output; The application restores the output data to a single-channel mask matrix of the original image size according to the segmentation specifications, and discards the padding part of each slice area; After restoring the size, the program sets the values ​​in the mask matrix greater than the threshold to 1 and the values ​​less than the threshold to 0 according to the set threshold of 0-1, and maps them to the value range of [0,255], and then saves it as a single-channel image, where the white area is the cloud pixel area and the black area is the non-cloud pixel area.

[0042] This invention also provides a lightweight remote sensing image cloud detection system guided by prior geoscience knowledge, comprising:

[0043] Dataset preprocessing module: used to acquire remote sensing image datasets and preprocess them to obtain preprocessed training and validation sets;

[0044] The lightweight cloud detection network model building module is used to build the encoder module, decoder module, feature extraction module, and feature fusion bridging module. The encoder module builds the downsampling sub-network of the network, the decoder module builds the upsampling sub-network, the feature fusion bridging module is used to bridge the layers of the downsampling sub-network and the upsampling sub-network, and the feature extraction module is used as the network model entry point to connect with the downsampling sub-network to build the lightweight cloud detection network model.

[0045] The optimal network model file acquisition module is used to iteratively train the lightweight cloud detection network model using the training set, and after a fixed number of iterations, to verify the effect of the lightweight cloud detection network model using the validation set and calculate relevant indicators. After the iterative training is completed, the model with the best indicators in the validation process is selected as the final training model, and the model parameters are saved as a file to obtain the optimal network model file.

[0046] Cloud detection module: Used to detect clouds in the required remote sensing images using the optimal network model file.

[0047] This invention also provides a lightweight remote sensing image cloud detection device guided by prior geoscience knowledge, comprising:

[0048] Memory: A computer program that stores the aforementioned lightweight remote sensing image cloud detection method guided by prior geoscience knowledge, and is a computer-readable device;

[0049] Processor: Used to implement the lightweight remote sensing image cloud detection method guided by prior geoscience knowledge when executing the computer program.

[0050] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, enables the implementation of the aforementioned lightweight remote sensing image cloud detection method guided by prior geoscience knowledge.

[0051] Compared with the prior art, the beneficial effects of the present invention are:

[0052] 1. This invention employs a lightweight deep network structure, using depthwise separable convolution and Hadamard product attention to extract and optimize image features. While maintaining similar performance, this invention significantly reduces the number of parameters and inference time.

[0053] 2. This invention embeds a frequency feature extraction module and a multi-scale feature fusion bridging module to enhance the model's sensitivity to frequency information in remote sensing images and its generalization ability, thereby achieving efficient, accurate and robust cloud detection.

[0054] In summary, this invention significantly lightweights mainstream remote sensing image cloud detection networks by employing depthwise separable convolution and Hadamard product attention mechanisms, reducing the computational power required for operation. Furthermore, it incorporates feature extraction and feature fusion bridging modules, enabling the neural network to leverage prior geoscientific knowledge for learning and iteration, enhancing its sensitivity to frequency features in remote sensing images, and improving cloud detection accuracy. For the need to deploy cloud detection applications on embedded devices, this invention addresses the issues of massive parameter counts and excessive computational complexity in mainstream cloud detection neural networks, making it more compatible with low-computing-power and low-power devices, and further increasing the possibility of real-time on-orbit operation of remote sensing cloud detection applications. Attached Figure Description

[0055] Figure 1 This is a flowchart illustrating an embodiment of the present invention.

[0056] Figure 2 This is a structural diagram of the lightweight cloud detection network model of the present invention.

[0057] Figure 3 This is a structural diagram of the feature extraction module of the present invention.

[0058] Figure 4 This is a structural diagram of the encoder module of the present invention.

[0059] Figure 5 This is a structural diagram of the decoder module of the present invention.

[0060] Figure 6 is a schematic diagram of the cloud detection results of the present invention. In Figure 6(a), it is a grayscale diagram of a remote sensing image. In Figure 6(b), it is a diagram of manually labeled cloud layers. In Figure 6(c), it is the output prediction diagram of the original UNET structure network. In Figure 6(d), it is the output prediction diagram of the lightweight cloud detection network model of the present invention. Detailed Implementation

[0061] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0062] like Figure 1 As shown, a lightweight remote sensing image cloud detection method guided by prior geoscience knowledge includes the following steps:

[0063] Step 1: Obtain the remote sensing image dataset and preprocess it to obtain the preprocessed training set and validation set;

[0064] Step 1.1: Each complete image is cropped into multiple 384*384 image patches to suit the semantic segmentation algorithm based on deep learning. Each image patch has four corresponding spectral channels: red (band 4), green (band 3), blue (band 2) and near-infrared (band 5), as well as corresponding cloud region labels.

[0065] Step 1.2: Read the sample RGB three-color channel image and the corresponding ground truth slices through the PIL library. Before inputting the model, map the color values ​​of each channel from integer values ​​of [0~255] to floating-point values ​​of [0~1], and adjust the dimension from (H,W,C) to (C,H,W) to map the [0,255] of the GT image to a binary matrix of [0,1].

[0066] like Figure 2 As shown, in step 2, the encoder module, decoder module, feature extraction module, and feature fusion bridging module are constructed.

[0067] like Figure 4 As shown, the encoder module is divided into a shallow encoder and a deep encoder. The shallow encoder consists of a depthwise separable convolutional layer, a BN normalization layer, and a ReLU activation layer, with a MaxPool pooling layer connected at the end, used for feature extraction and halving the width and height of the feature map, respectively. The deep encoder consists of a Hadamard product attention layer, a BN normalization layer, a ReLU activation layer, and a pooling layer. The Hadamard product attention layer achieves an attention mechanism for local regions of different dimensions by linearly calculating the feature map through the matrix Hadamard product and reducing the number of parameters. The depthwise separable convolution also effectively reduces the number of parameters in the convolutional layer through the separable structure.

[0068] like Figure 5As shown, the decoder module is divided into a shallow decoder and a deep decoder. The shallow decoder consists of a depthwise separable convolutional layer, a BN normalization layer, and a ReLU activation layer, with a deconvolutional layer connected at the end, which are used for feature extraction and restoration of the feature map width and height, respectively. The deep decoder consists of a Hadamard product attention layer, a BN normalization layer, a ReLU activation layer, and a deconvolutional layer. This module serves as the upsampling module for each layer of the encoder in the network structure, and is used to restore the resolution of the feature image.

[0069] like Figure 3 As shown, the feature extraction module consists of an FFT layer and a dimension adaptation layer. The input feature map is first transformed in dimension by the dimension adaptation layer, and then fed into the FFT layer. It first undergoes a fast Fourier transform, and then the transformed feature map is subjected to an affine transformation using learnable weight and bias parameters. The feature map is then restored by an inverse Fourier transform, and finally the dimension is restored by the dimension adaptation layer and output to the outside of the module.

[0070] The feature fusion bridging module integrates the encoder output features of the corresponding layer, the decoder output features of the previous layer, and the single-channel mask feature map of the corresponding scale of the features, thereby realizing multi-scale feature fusion.

[0071] Step 3: The encoder module obtained in Step 2 is used to construct the downsampling sub-network of the network, the decoder module is used to construct the upsampling sub-network, the feature fusion bridging module is used to bridge the layers of the downsampling sub-network and the upsampling sub-network, and the feature extraction module is used as the network model entry point to connect with the downsampling sub-network to construct a lightweight cloud detection network model.

[0072] Step 3.1: Connect the encoder and decoder modules hierarchically, using three layers as a baseline. The first two layers use shallow encoder and decoder modules, and the bottom layer uses deep encoder and decoder modules. Initialize the first-layer encoder with the original image size and the original number of channels of the input image, and output a feature map with half the size and 32 channels. Subsequent encoders are connected end to end and initialized according to the output dimension of the upper layer, outputting feature maps with half the size and doubled channels. The decoders are connected end to end in the same way, and the decoder modules are initialized in reverse according to the input and output parameters of the encoders in the same layer. The shallowest decoder outputs a single-channel feature map, so that the entire network outputs a single-layer mask image with the resolution of the input image. This constitutes the downsampling sub-network and the upsampling sub-network of the network.

[0073] Step 3.2: Initialize the frequency feature extraction module according to the original dimensions of the input image and connect it to the input end of the downsampling sub-network. In this way, the network can learn to adjust the RGB three-channel features to a suitable frequency and perform subsequent inference.

[0074] Step 3.3: Initialize the corresponding layer feature fusion bridging module according to the output dimension of the corresponding layer encoder and the input dimension of the corresponding layer decoder. First, perform a convolution operation with a kernel of 1 on the output of each layer decoder to obtain a mask image of the size of the corresponding layer with a single channel. Input the encoder output, the upper layer decoder output and the mask image into the feature fusion bridging module. The multi-scale output features are fused and superimposed on the upper layer decoder output feature map through the add operation and input into the current layer decoder so that it can participate in subsequent inference operations.

[0075] Step 4: Select appropriate hyperparameters and loss functions, and use the training set obtained in Step 1 to iteratively train the lightweight cloud detection network model constructed in Step 3. After a fixed number of iterations, use the validation set obtained in Step 1 to verify the effect of the lightweight cloud detection network model constructed in Step 3 and calculate relevant indicators. After the iterative training is completed, select the model with the best validation indicators as the final training model, and save the model parameters as a file to obtain the optimal network model file.

[0076] Step 4.1: Select AdamW as the parameter optimizer, set the initial learning rate to 0.001, the weight decay factor to 0.01, and select cosine annealing learning rate to dynamically adjust the learning rate.

[0077] Step 4.2: The loss function is used to evaluate the performance of the lightweight cloud detection network model constructed in Step 3. In this invention, the binary cross-entropy loss function (BCE Loss), which is commonly used in binary classification problems, and the dice loss function, which performs well in the field of image segmentation, are combined to evaluate the lightweight cloud detection network model constructed in Step 3.

[0078] The formula for calculating the binary cross-entropy loss function (BCE Loss) is as follows:

[0079] L BCE =-w n [y n ·logx n +(1-y n )·log1-x n )]

[0080] Where w is the weight of each category, which is 1 by default, x is the model output, and y is the true value;

[0081] The formula for calculating the Dice Loss function is as follows:

[0082]

[0083] Where |X| represents the positive output of the model, |Y| represents the positive true value, and |X∩Y| represents the positive true value, which is the part where both the model output and the true value are positive.

[0084] The loss function (BCE-Dice Loss) used in this invention combines the binary cross-entropy loss function (BCELoss) and the dice loss function (Dice Loss) to evaluate model performance. Its calculation formula is as follows:

[0085] Loss = w BCE ×L BCE +w dice ×L dice

[0086] The weights of BCE Loss and Dice Loss are both set to 1 by default.

[0087] Step 4.3: Set the maximum number of iterations to 300 and the batch size for a single iteration to 64.

[0088] Step 4.4: During each iteration, the training set is input into the lightweight cloud detection network model constructed in Step 3 in batches for calculation. After each calculation, the model output and the corresponding cloud layer labeled slice (ground truth slice) are input into the loss function to evaluate the performance and update the model parameters in reverse. Every 5 iterations, the model is evaluated using the validation set, the confusion matrix is ​​calculated, and the performance of various indicators is calculated accordingly. Each validation retains the network model file with the best performance in the iteration history.

[0089] Step 5: Use the optimal network model file obtained in Step 4 to perform cloud detection on the required remote sensing image.

[0090] The application first instantiates the network model and loads the model file to import parameters. It then segments and preprocesses any remote sensing satellite image requiring cloud detection. Finally, it inputs the image data into the network to obtain the corresponding cloud prediction results. After binarization, the results are saved as an image, which is the predicted image of the cloud region corresponding to the input image, as detailed below:

[0091] Step 5.1: Read the network model file that has been fully iterated and trained in Step 4, load the parameters of the network model file into the network model of the application, adjust the network model to inference mode, and fix the parameters of the BN layer.

[0092] Step 5.2: Import the required remote sensing image into the application. The application first segments the required remote sensing image and sets the inner border size (padding) of the segmentation, which is 10 pixels by default. The application segments the required remote sensing image with overlapping parts according to the inner border size (padding) and organizes the remote sensing image slices into the matrix shape required by the model.

[0093] Step 5.3: Input the remote sensing image slices obtained in Step 5.2 into the network model of the application to obtain the model output; The application restores the output data to a single-channel mask matrix of the original image size according to the segmentation specifications, and discards the padding part of each slice area; After restoring the size, the program sets the values ​​in the mask matrix greater than the threshold to 1 and the values ​​less than the threshold to 0 according to the set threshold of 0-1, and maps them to the value range of [0,255], and then saves it as a single-channel image, where the white area is the cloud pixel area and the black area is the non-cloud pixel area.

[0094] To address the current limitations of mainstream cloud detection network models in remote sensing images, which involve large parameter and computational loads and are unsuitable for low-power, low-computing-power embedded devices, as well as the tendency of general image segmentation network structures to overlook frequency information in remote sensing images, this invention proposes a remote sensing image cloud detection method and system for low-computing-power devices. This method and system achieve efficient, accurate, and robust cloud detection, suitable for operation in spaceborne environments. This invention integrates the advantages of multiple cloud detection methods, combining quantitative and qualitative approaches, and overcomes the limitations of traditional deep learning models in remote sensing image feature extraction.

[0095] This network not only effectively reduces model complexity and computational load, making it suitable for running on low-computing-power devices, but also leverages frequency information from remote sensing images to improve the accuracy and robustness of cloud detection, overcoming the limitations of traditional deep learning models on remote sensing images. This invention integrates the advantages of multiple cloud detection methods, emphasizing the combination of model compression and quantization with qualitative analysis, providing a more efficient solution for cloud detection tasks in spaceborne environments.

[0096] The core of this invention includes using a lightweight encoder-decoder network and classifier, replacing traditional convolutional modules with depthwise separable convolutional modules, effectively reducing the number of model parameters and computational load, improving the model's inference speed, and maintaining the model's feature extraction capability; implementing an attention mechanism using Hadamard products with linear spatial complexity, enhancing the network model's generalization ability with fewer parameters, and improving the network's feature extraction efficiency; proposing a frequency feature extraction module representing geoscientific prior knowledge, strengthening the network model's sensitivity to frequency information in satellite imagery, and improving the model's ability to extract frequency information features; using multi-scale feature fusion to enhance the model's feature representation ability, improving the accuracy and robustness of cloud detection; combining multiple loss functions to evaluate and iteratively train the network model, extracting and saving the optimal model, preprocessing and inferring arbitrary remote sensing images to obtain mask images of cloud regions, achieving the purpose of cloud segmentation in remote sensing images.

[0097] Experimental Analysis

[0098] Table 1 shows the results obtained from simulation experiments and model calculations. As can be seen from Table 1, the number of parameters in this invention is only 17.6% of that in the UNET structure, which has met the requirements for lightweight design. Under the same CPU computing power test environment, the inference time for (484,3,384,384) data volume is reduced by 4.38 times, while the accuracy (TN+TP ratio) is almost not reduced. The specific effect is shown in Figure 6.

[0099] Table 1 Comparison of Experimental Indicator Models

[0100] UNET 0.939513028 536,729 48m15s Ours 0.935489431 94,513 10m57s

[0101] As shown in Figure 6, Figure 6(a) is a grayscale diagram of a remote sensing image, Figure 6(b) is a diagram of manually labeled clouds, Figure 6(c) is the output prediction diagram of the original UNET structure network, and Figure 6(d) is the output prediction diagram of the lightweight cloud detection network model of this invention. In these diagrams, dark areas represent non-clouds, and light areas represent clouds. It can be seen that even with a 5-fold reduction in the number of parameters, the model performance did not decline, and the prediction effect in high-frequency regions of the feature map is more accurate than the original UNET.

[0102] This invention also provides a lightweight remote sensing image cloud detection system guided by prior geoscience knowledge, comprising:

[0103] Dataset preprocessing module: used to obtain the remote sensing image dataset in step 1 and preprocess it to obtain the preprocessed training set and validation set;

[0104] The lightweight cloud detection network model building module is used to build the encoder module, decoder module, feature extraction module, and feature fusion bridging module in steps 2 and 3. The encoder module obtained in step 2 is used to build the downsampling sub-network of the network, the decoder module is used to build the upsampling sub-network, the feature fusion bridging module is used to bridge the layers of the downsampling sub-network and the upsampling sub-network, and the feature extraction module is used as the network model entry point to connect with the downsampling sub-network to build the lightweight cloud detection network model.

[0105] The optimal network model file acquisition module is used to perform iterative training of the lightweight cloud detection network model built in step 3 using the training set obtained in step 1 in step 4, and to verify the effect of the lightweight cloud detection network model built in step 3 using the validation set obtained in step 1 at fixed iteration intervals and to calculate relevant indicators. After the iterative training is completed, the model with the best indicators in the validation process is selected as the final training model, and the model parameters are saved as a file to obtain the optimal network model file.

[0106] Cloud detection module: Used in step 5 to perform cloud detection on the required remote sensing image using the optimal network model file obtained in step 4.

[0107] This invention also provides a lightweight remote sensing image cloud detection device guided by prior geoscience knowledge, comprising:

[0108] Memory: A computer program that stores the aforementioned lightweight remote sensing image cloud detection method guided by prior geoscience knowledge, and is a computer-readable device;

[0109] Processor: Used to implement the lightweight remote sensing image cloud detection method guided by prior geoscience knowledge when executing the computer program.

[0110] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, enables the implementation of the aforementioned lightweight remote sensing image cloud detection method guided by prior geoscience knowledge.

Claims

1. A lightweight remote sensing image cloud detection method guided by prior geoscience knowledge, characterized in that: Includes the following steps: Step 1: Obtain the remote sensing image dataset and preprocess it to obtain the preprocessed training set and validation set; Step 2: Construct the encoder module, decoder module, feature extraction module, and feature fusion bridging module; The specific method for step 2 is as follows: The encoder module is divided into a shallow encoder and a deep encoder. The shallow encoder consists of a depth-separable convolutional layer, a BN normalization layer, and a ReLU activation layer, and is connected to a MaxPool pooling layer at the end, which are used for feature extraction and halving the width and height of the feature map, respectively. The deep encoder consists of a Hadamard product attention layer, a BN normalization layer, a ReLU activation layer, and a pooling layer. The Hadamard product attention layer achieves the attention mechanism for local regions of different dimensions by linearly calculating the feature map through the matrix Hadamard product and reducing the number of parameters. The depthwise separable convolution also effectively reduces the number of parameters of the convolutional layer through the separable structure. The decoder module is divided into a shallow decoder and a deep decoder. The shallow decoder consists of a depth-separable convolutional layer, a BN normalization layer, and a ReLU activation layer, and is connected to a deconvolutional layer at the end, which are used for feature extraction and restoration of the feature map width and height, respectively. The deep decoder consists of a Hadamard product attention layer, a BN normalization layer, a ReLU activation layer, and a deconvolution layer. This module serves as the upsampling module for each layer of the encoder in the network structure, and is used to restore the resolution of the feature image. The feature extraction module consists of an FFT layer and a dimension adaptation layer. The input feature map is first transformed in dimension by the dimension adaptation layer, and then fed into the FFT layer. It first undergoes a fast Fourier transform, and then the transformed feature map is subjected to an affine transform using learnable weight and bias parameters. The feature map is then restored by an inverse Fourier transform, and finally the dimension is restored by the dimension adaptation layer and output to the outside of the module. The feature fusion bridging module fuses the encoder output features of the corresponding layer, the decoder output features of the previous layer, and the single-channel mask feature map of the corresponding scale of the features to achieve multi-scale feature fusion. Step 3: The encoder module obtained in Step 2 is used to construct the downsampling sub-network of the network, the decoder module is used to construct the upsampling sub-network, the feature fusion bridging module is used to bridge the layers of the downsampling sub-network and the upsampling sub-network, and the feature extraction module is used as the network model entry point to connect with the downsampling sub-network to construct a lightweight cloud detection network model. Step 4: Use the training set obtained in Step 1 to iteratively train the lightweight cloud detection network model constructed in Step 3. After a fixed number of iterations, use the validation set obtained in Step 1 to verify the effect of the lightweight cloud detection network model constructed in Step 3 and calculate relevant indicators. After the iterative training is completed, select the model with the best validation indicators as the final training model and save the model parameters as a file to obtain the optimal network model file. Step 5: Use the optimal network model file obtained in Step 4 to perform cloud detection on the required remote sensing image.

2. The lightweight remote sensing image cloud detection method guided by prior geoscience knowledge according to claim 1, characterized in that: The specific method for step 1 is as follows: The acquired remote sensing image is segmented at a specific size and the cloud regions of the slices are manually labeled to obtain multi-channel layer slices and corresponding Ground Truth cloud labels. The slices are then divided into training and validation sets, which are then normalized and their dimensions are transformed.

3. The lightweight remote sensing image cloud detection method guided by prior geoscience knowledge according to claim 1, characterized in that: The specific method for step 3 is as follows: Step 3.1: Connect the encoder and decoder modules hierarchically, using three layers as a baseline. The first two layers use shallow encoder and decoder modules, and the bottom layer uses deep encoder and decoder modules. Initialize the first-layer encoder with the original image size and the original number of channels of the input image, and output a feature map with half the size and 32 channels. Subsequent encoders are connected end to end and initialized according to the output dimension of the upper layer, outputting feature maps with half the size and doubled channels. The decoders are connected end to end in the same way, and the decoder modules are initialized in reverse according to the input and output parameters of the encoders in the same layer. The shallowest decoder outputs a single-channel feature map, so that the entire network outputs a single-layer mask image with the resolution of the input image. This constitutes the downsampling sub-network and the upsampling sub-network of the network. Step 3.2: Initialize the frequency feature extraction module according to the original dimensions of the input image and connect it to the input end of the downsampling sub-network. In this way, the network can learn to adjust the RGB three-channel features to a suitable frequency and perform subsequent inference. Step 3.3: Initialize the corresponding layer feature fusion bridging module according to the output dimension of the corresponding layer encoder and the input dimension of the corresponding layer decoder. First, perform a convolution operation with a kernel of 1 on the output of each layer decoder to obtain a mask image of the size of the corresponding layer with a single channel. Input the encoder output, the upper layer decoder output and the mask image into the feature fusion bridging module. The multi-scale output features are fused and superimposed on the upper layer decoder output feature map through the add operation and input into the current layer decoder so that it can participate in subsequent inference operations.

4. The lightweight remote sensing image cloud detection method guided by prior geoscience knowledge according to claim 1, characterized in that: The specific method for step 4 is as follows: Step 4.1: Select AdamW as the parameter optimizer, set the initial learning rate and weight decay factor, and select cosine annealing learning rate to dynamically adjust the learning rate; Step 4.2: The performance of the lightweight cloud detection network model constructed in Step 3 is evaluated using the Binary Cross-Entropy Loss (BCE Loss) and Dice Loss functions. The calculation formula for the Binary Cross-Entropy Loss (BCE Loss) is as follows: Where w is the weight of each category, which is 1 by default, x is the model output, and y is the true value; The formula for calculating the Dice Loss function is as follows: Where |X| represents the positive output of the model, |Y| represents the positive true value, and |X∩Y| represents the positive true value, which is the part where both the model output and the true value are positive. The model performance is evaluated using a combination of the binary cross-entropy loss function (BCE Loss) and the dice loss function (Dice Loss), calculated as follows: The weights of BCE Loss and Dice Loss are both set to 1 by default. Step 4.3: Set the maximum number of iterations and the batch size per iteration; Step 4.4: During each iteration, the training set is input into the lightweight cloud detection network model constructed in Step 3 in batches for calculation. After each calculation, the model output and the corresponding cloud layer labeled slice (ground truth slice) are input into the loss function to evaluate the performance and update the model parameters in reverse. Every fixed number of iterations, the model is evaluated using the validation set, the confusion matrix is ​​calculated, and the performance of various indicators is calculated accordingly. Each validation retains the network model file with the best performance in the iteration history.

5. The lightweight remote sensing image cloud detection method guided by prior geoscience knowledge according to claim 1, characterized in that: The specific method for step 5 is as follows: Step 5.1: Read the network model file that has been fully iterated and trained in Step 4, load the parameters of the network model file into the network model of the application, adjust the network model to inference mode, and fix the parameters of the BN layer. Step 5.2: Import the required remote sensing image into the application. The application first segments the required remote sensing image and sets the inner border size (padding). The application segments the required remote sensing image with overlapping parts according to the inner border size (padding) and organizes the remote sensing image slices into the matrix shape required by the model. Step 5.3: Input the remote sensing image slices obtained in Step 5.2 into the network model of the application to obtain the model output; The application restores the output data to a single-channel mask matrix of the original image size according to the segmentation specifications, and discards the padding part of each slice area; After restoring the size, the program sets the values ​​in the mask matrix greater than the threshold to 1 and the values ​​less than the threshold to 0 according to the set threshold of 0-1, and maps them to the value range of [0, 255], and then saves it as a single-channel image, where the white area is the cloud pixel area and the black area is the non-cloud pixel area.

6. A lightweight remote sensing image cloud detection system guided by prior geoscience knowledge as described in any one of claims 1 to 5, characterized in that: include: Dataset preprocessing module: used to acquire remote sensing image datasets and preprocess them to obtain preprocessed training and validation sets; Lightweight cloud detection network model building module: used to build encoder module, decoder module, feature extraction module and feature fusion bridging module; The network constructs a downsampled subnetwork through the encoder module, an upsampled subnetwork through the decoder module, and a feature fusion bridging module to bridge the layers of the downsampled and upsampled subnetworks. The feature extraction module serves as the network model entry point and connects with the downsampled subnetwork to construct a lightweight cloud detection network model. The optimal network model file acquisition module is used to iteratively train the lightweight cloud detection network model using the training set, and after a fixed number of iterations, to verify the effect of the lightweight cloud detection network model using the validation set and calculate relevant indicators. After the iterative training is completed, the model with the best indicators in the validation process is selected as the final training model, and the model parameters are saved as a file to obtain the optimal network model file. Cloud detection module: Used to detect clouds in the required remote sensing images using the optimal network model file.

7. A lightweight remote sensing image cloud detection device guided by prior geoscience knowledge, characterized in that: include: Memory: A computer program for a lightweight remote sensing image cloud detection method guided by prior geoscience knowledge as described in any one of claims 1-5, which is a computer-readable device; Processor: Used to implement the lightweight remote sensing image cloud detection method guided by geoscience prior knowledge as described in any one of claims 1-5 when executing the computer program.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program that, when executed by a processor, enables the implementation of a lightweight remote sensing image cloud detection method guided by prior geoscience knowledge, as described in any one of claims 1-5.

Images