Cloud detection method and device, electronic equipment and computer readable storage medium

Abstract

The invention relates to the technical field of remote sensing image detection, and discloses a cloud detection method and device, electronic equipment and a computer readable storage medium, and the method comprises the steps: inputting a remote sensing image into a pre-trained cloud detection model, and obtaining a first multi-scale feature corresponding to the remote sensing image through an encoder in the cloud detection model; obtaining a plurality of attention vectors corresponding to a plurality of first feature maps in the first multi-scale feature through an attention module in the cloud detection model, and obtaining a second multi-scale feature according to the plurality of attention vectors and the first multi-scale feature; obtaining a preliminary segmentation image of the remote sensing image according to the second multi-scale feature through a decoder in the cloud detection model; obtaining an edge prediction map corresponding to the remote sensing image through a boundary sensing module in the cloud detection model; and obtaining a target segmentation map corresponding to the remote sensing image according to the preliminary segmentation map and the edge prediction map. The problems of fuzzy prediction and boundary fracture of a model in an edge region in the prior art are solved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of remote sensing image detection, and particularly relates to a cloud detection method and device, an electronic device, and a computer readable storage medium. BACKGROUND

[0002] With the continuous development of remote sensing satellite technology, more and more high-resolution optical remote sensing images are widely used in the fields of agriculture, forestry, land use monitoring, ecological environment assessment, and natural disaster emergency response. Accurate detection of clouds in remote sensing images has become an indispensable part of the remote sensing data preprocessing chain.

[0003] Existing remote sensing image cloud detection methods have made some progress, but there are still many technical bottlenecks. Traditional spectral threshold methods rely heavily on manually set band combinations and discrimination rules, have poor generalization ability, and are difficult to adapt to different imaging times, topography, sensor types, or ground cover conditions, especially when dealing with high-reflectivity ground objects, thin clouds, and cloud shadow areas, which are prone to misjudgment or omission. At the same time, such methods lack the ability to understand the semantic level of the image and cannot effectively extract cloud texture, shape, or color information. With the development of deep learning, semantic segmentation neural networks have been gradually applied to cloud detection tasks, and various models have shown strong feature learning ability, but their basic structure is still mainly based on convolutional networks, which are limited by the receptive field and difficult to capture long-range dependency information of large-scale clouds in remote sensing images. There are still deficiencies in the expression of multi-scale clouds and the recognition of cloud boundary areas, and the problems of edge blurring and detail loss often occur.

[0004] The information disclosed in this BACKGROUND section is only for the purpose of increasing the understanding of the background of the present application and should not be taken as an acknowledgment or any form of suggestion that this information forms prior art that is publicly known. SUMMARY

[0005] The present application aims to solve the problem of existing technology that is difficult to capture long-range dependency information of large-scale clouds in remote sensing images, and often appears edge blurring and detail loss, and provides a cloud detection method, device, electronic device, and computer readable storage medium.

[0006] The first aspect of the present application provides a cloud detection method, comprising: inputting a remote sensing image into a pre-trained cloud detection model, obtaining a first multi-scale feature corresponding to the remote sensing image through an encoder in the cloud detection model; obtaining a plurality of attention vectors corresponding to a plurality of first feature maps in the first multi-scale feature through an attention module in the cloud detection model, and obtaining a second multi-scale feature according to the plurality of attention vectors and the first multi-scale feature; obtaining a preliminary segmentation map of the remote sensing image according to the second multi-scale feature through a decoder in the cloud detection model; obtaining an edge prediction map corresponding to the remote sensing image through a boundary perception module in the cloud detection model, wherein the edge prediction map includes a prediction result of each pixel being a cloud edge; and obtaining a target segmentation map corresponding to the remote sensing image according to the preliminary segmentation map and the edge prediction map.

[0007] In an embodiment of the present application, the inputting of the remote sensing image into the pre-trained cloud detection model and the obtaining of the first multi-scale feature corresponding to the remote sensing image through the encoder in the cloud detection model comprises: obtaining a plurality of initial feature maps of a plurality of sizes corresponding to the remote sensing image and a plurality of position encodings corresponding to the plurality of initial feature maps through the encoder; and obtaining the first multi-scale feature according to the plurality of initial feature maps and the plurality of position encodings.

[0008] In an embodiment of the present application, the obtaining of the edge prediction map corresponding to the remote sensing image through the boundary perception module in the cloud detection model comprises: performing gradient-guided processing on the plurality of second feature maps to obtain an edge response map corresponding to the remote sensing image; and performing convolution on the edge response map to generate the edge prediction map.

[0009] In an embodiment of the present application, the gradient-guided processing on the plurality of second feature maps to obtain the edge response map corresponding to the remote sensing image comprises: obtaining the edge response map according to gradient features of second feature maps in a middle deep layer, wherein the resolution of the second feature maps in the middle deep layer is greater than a preset resolution; and the convolution on the edge response map to generate the edge prediction map comprises: performing feature extraction and feature compression on the edge response map through a convolution layer to obtain the edge prediction map.

[0010] In an embodiment of the present application, before the inputting of the remote sensing image into the pre-trained cloud detection model, the method further comprises: performing a preprocessing operation on the remote sensing image, wherein the preprocessing operation comprises at least one of the following: performing radiation scaling on the remote sensing image; performing linear stretching on the remote sensing image; and resampling the remote sensing image according to a preset spatial resolution.

[0011] The second aspect of the present application provides a cloud detection device, comprising: a first processing module configured to input a remote sensing image into a pre-trained cloud detection model, and obtain a first multi-scale feature corresponding to the remote sensing image through an encoder in the cloud detection model; a second processing module configured to obtain a plurality of attention vectors corresponding to a plurality of first feature maps in the first multi-scale feature through an attention module in the cloud detection model, and obtain a second multi-scale feature according to the plurality of attention vectors and the first multi-scale feature; a third processing module configured to obtain a preliminary segmentation map of the remote sensing image according to the second multi-scale feature through a decoder in the cloud detection model; an edge detection module configured to obtain an edge prediction map corresponding to the remote sensing image through a boundary perception module in the cloud detection model, wherein the edge prediction map includes a prediction result of each pixel being a cloud edge; and a fourth processing module configured to obtain a target segmentation map corresponding to the remote sensing image according to the preliminary segmentation map and the edge prediction map.

[0012] In an embodiment of the present application, the first processing module comprises: a first processing submodule configured to obtain a plurality of initial feature maps of a plurality of sizes corresponding to the remote sensing image and a plurality of position encodings corresponding to the plurality of initial feature maps through the encoder; and a second processing submodule configured to obtain the first multi-scale feature according to the plurality of initial feature maps and the plurality of position encodings.

[0013] In an embodiment of the present application, the edge detection module comprises: an edge detection submodule configured to perform gradient-guided processing on the plurality of second feature maps to obtain an edge response map corresponding to the remote sensing image; and a third processing submodule configured to perform convolution on the edge response map to generate the edge prediction map.

[0014] The third aspect of the present application provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the cloud detection method according to the first aspect.

[0015] The fourth aspect of the present application provides a computer-readable storage medium storing a computer program, wherein the computer program is executed by a processor to implement the cloud detection method according to the first aspect.

[0016] Compared with the prior art, the present application achieves the following technical effects:

[0017] The remote sensing image is input into a cloud detection model trained in advance, a first multi-scale feature corresponding to the remote sensing image is obtained through an encoder in the cloud detection model, a plurality of attention vectors corresponding to a plurality of first feature maps in the first multi-scale feature are obtained through an attention module in the cloud detection model, and a second multi-scale feature is obtained according to the plurality of attention vectors and the first multi-scale feature; a preliminary segmentation map of the remote sensing image is obtained according to the second multi-scale feature through a decoder in the cloud detection model, and a target region of different spatial receptive fields is focused; an edge prediction map corresponding to the remote sensing image is obtained through a boundary perception module in the cloud detection model, and the discrimination ability of the model for a cloud boundary region is enhanced; and a target segmentation map corresponding to the remote sensing image is obtained according to the preliminary segmentation map and the edge prediction map. The problems of fuzzy prediction and boundary fracture of the model in the prior art are solved, the cloud edge detection precision is effectively improved, and image details are retained. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 is a flowchart of a cloud detection method according to an embodiment of the present application;

[0019] Figure 2 is a framework diagram of a cloud detection model according to an embodiment of the present application;

[0020] Figure 3 is a framework diagram of an attention module according to an embodiment of the present application;

[0021] Figure 4 is a framework diagram of a cloud detection model according to another embodiment of the present application;

[0022] Figure 5 is a comparison diagram of a remote sensing image and a target segmentation map according to an embodiment of the present application;

[0023] Figure 6 is a diagram of a training sample according to an embodiment of the present application;

[0024] Figure 7 is a diagram of extracting an independent position code according to an embodiment of the present application;

[0025] Figure 8 is a framework diagram of a cloud detection device according to an embodiment of the present application;

[0026] Figure 9 is a framework diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] Unless otherwise defined, all terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. It will be apparent to those skilled in the art that various modifications and variations can be made to the present application without departing from the spirit or scope of the application. Thus, it is intended that the present application cover modifications and variations of this application provided they come within the scope of the appended claims and their equivalents.

[0028] The technical solutions of the present application are described below by specific examples. It should be understood that the one or more steps mentioned in the present application do not exclude other methods and steps before and after the combination steps, or other methods and steps can be inserted between these explicitly mentioned steps. It should also be understood that these examples are only used to illustrate the present application and not to limit the scope of the present application. Unless otherwise stated, the numbering of each method step is only for the purpose of identifying each method step, not to limit the arrangement order of each method or to limit the scope of the implementation of the present application. The change or adjustment of the relative relationship can also be considered as the scope of the implementation of the present application without substantial technical content change.

[0029] The raw materials and instruments used in the examples are not specifically limited in source, and can be purchased on the market or prepared according to the conventional methods well known to those skilled in the art.

[0030] Before introducing the ship detection method in the embodiments of the present application, the background technology of the present application is first described: with the development of deep learning, semantic segmentation type neural networks are gradually applied to cloud detection tasks, such as U-Net, HRNet, DeepLab series models, which show strong feature learning ability, but their basic structure is still mainly based on convolutional networks, which is limited by the receptive field, and it is difficult to capture the long-distance dependency information of large-scale clouds in remote sensing images, and there are still deficiencies in the expression of multi-scale clouds and the identification of cloud boundary areas, and the problems of edge blur and detail loss often occur. In recent years, Segformer (semantic segmentation model) as a new model integrating Transformer structure and semantic segmentation idea performs well in natural image segmentation tasks such as urban street scenes, but when directly applied to remote sensing image scenes, there is still a lack of special modeling of cloud structure, and there are limitations such as insufficient multi-scale feature fusion, weak edge perception ability, and the model is not optimized for remote sensing image characteristics in the prior art.

[0031] To solve the above problems, the present embodiment provides a cloud detection method, as shown in Figure 1 The method can specifically include the following steps:

[0032] S101, inputting a remote sensing image into a cloud detection model pre-trained, and obtaining a first multi-scale feature corresponding to the remote sensing image through an encoder in the cloud detection model;

[0033] In this embodiment, the remote sensing image can be a remote sensing image of the target area acquired in real time, such as an optical remote sensing image taken by a satellite; or it can be a remote sensing image of the target area pre-stored in a preset database. The remote sensing image includes, but is not limited to: optical remote sensing image, radar image, and infrared image.

[0034] The cloud detection model in this embodiment includes an encoder, an attention module, a decoder, a boundary awareness module, and a fusion module. The encoder is used to extract features from the remote sensing image to obtain the first multi-scale features of the remote sensing image. The attention module is used to extract attention vectors between features of different scales and fuse the attention vectors and multi-scale features to obtain the second multi-scale features. The decoder is used to generate preliminary segmentation results based on the second multi-scale features. The boundary awareness module is used to extract the boundary features of the remote sensing image and generate an edge prediction map of the remote sensing image. The fusion module is used to generate the final cloud detection result and generate a target segmentation map based on the second multi-scale features and the edge prediction map.

[0035] In one example, such as Figure 2 As shown, the cloud detection model 20 includes an encoder 210, an attention module 220, a decoder 230, a boundary awareness module 240, and a fusion module 250. The input of the cloud detection model 20 is remote sensing imagery, and the output is a ground truth map of the cloud detection results.

[0036] In this embodiment, the remote sensing image is downsampled in layers by the encoder 210 to obtain corresponding first multi-scale features at different stages. The first multi-scale features include the first feature map obtained by downsampling at each stage, and the resolution of the first feature map at each stage is different.

[0037] S102, through the attention module in the cloud detection model, obtain multiple attention vectors corresponding to multiple first feature maps in the first multi-scale feature, and obtain the second multi-scale feature based on the multiple attention vectors and the first multi-scale feature;

[0038] In this embodiment, the attention module comprises multiple attention channels and multiple attention fusion modules. Each attention channel is connected to an attention fusion module and also to the attention fusion module at the next lower level. The multiple attention channels extract multiple attention vectors corresponding to multiple first feature maps, with each attention channel corresponding to a first feature map layer. The input to each attention channel is a feature map, and the output is an attention vector. The input to the attention fusion module includes both the feature map and the attention vector.

[0039] Specifically, the encoder performs layered downsampling on the remote sensing image to obtain corresponding first multi-scale features at different stages. Multiple attention vectors corresponding to multiple first feature maps within these first multi-scale features are extracted through multiple attention channels. Each attention channel sends its extracted attention vectors to a connected attention fusion module. The attention fusion module fuses the first feature maps and the received attention vectors to obtain a second feature map. Finally, by combining the outputs of multiple attention fusion modules, the second multi-scale feature is obtained.

[0040] In one example, such as Figure 3 As shown, encoder 210 obtains first multi-scale features in four stages by performing layered downsampling on remote sensing images. The first multi-scale features include first feature map 10, first feature map 12, first feature map 14, and first feature map 16, wherein the resolution of the first feature map corresponding to each stage is different. Attention module 220 includes attention channels 2210, 2212, 2214, and 2216, and attention fusion modules 2220, 2222, 2224, and 2226. Attention channel 2210 acquires the attention vector of the first feature map 10 and sends it to attention fusion modules 2220 and 2222; attention channel 2212 acquires the attention vector of the first feature map 12 and sends it to attention fusion modules 2222 and 2224; attention channel 2214 acquires the attention vector of the first feature map 14 and sends it to attention fusion modules 2224 and 2226; attention channel 2216 acquires the attention vector of the first feature map 16 and sends it to attention fusion module 2226. Within the attention modules, each attention fusion module fuses the received attention vector with the first feature map of the corresponding stage to obtain a second feature map. The outputs of multiple attention fusion modules are combined to obtain a second multi-scale feature.

[0041] S103, using the decoder in the cloud detection model, obtains a preliminary segmentation map of the remote sensing image based on the second multi-scale features;

[0042] In specific application scenarios, the fused second multi-scale features are input into the decoder for feature fusion and reconstruction. In one example, the decoder 230 adopts a lightweight structure and effectively aggregates encoded feature information of different scales by combining bilinear upsampling (bilinear interpolation) with pixel-wise convolution to generate a preliminary segmentation map.

[0043] S104, acquire an edge prediction map corresponding to the remote sensing image through a boundary perception module in the cloud detection model, wherein the edge prediction map includes a prediction result of each pixel being a cloud edge.

[0044] To improve the recognition accuracy of cloud edges and detail transition areas in the remote sensing image, the embodiment acquires an edge prediction map corresponding to the remote sensing image through a boundary perception module in the cloud detection model. In the embodiment, the boundary perception module is configured to extract gradient features of cloud edges of the remote sensing image and generate an edge prediction map of the cloud in the remote sensing image. The prediction result includes, but is not limited to, a probability of being a cloud edge.

[0045] In one example, in the cloud detection model, the input of the boundary perception module is the same as the input data of the encoder, both of which are remote sensing images, and the output of the boundary perception module is the edge prediction map, which includes a prediction result of each pixel being a cloud edge.

[0046] In another example, in the cloud detection model as shown in Figure 4 , the input data of the boundary perception module 240 is part of the output data of the encoder 210, the encoder 210 outputs first multi-scale features, the input of the boundary perception module 240 is a middle-deep layer feature (a feature with a resolution lower than a preset resolution and obtained through a middle-late stage downsampling) in the first multi-scale features, and the output of the boundary perception module 240 is an edge prediction map, which includes a prediction result of each pixel being a cloud edge.

[0047] S105, acquire a target segmentation map corresponding to the remote sensing image according to the preliminary segmentation map and the edge prediction map.

[0048] In the embodiment, the edge prediction map and the main segmentation branch output are fused to obtain a target segmentation map corresponding to the remote sensing image. In the target segmentation map, the cloud in the remote sensing image is detected and recognized.

[0049] In one example, as shown in Figure 5 , it is the prediction performance of the cloud detection model in the embodiment on actual remote sensing image data. From left to right in the figure, there are: an original remote sensing image and a target segmentation map. In the figure, white, gray and black represent cloud, cloud shadow and cloudless background, respectively. Figure 5 In the figure, the target segmentation map clearly shows the accurate recognition effect of the model on the cloud, especially in the edge and transition area of the cloud layer. It can be seen that the cloud detection model in the embodiment can accurately restore the outline of the cloud layer and avoid the phenomena of fuzzy boundary, missed detection and misjudgment.

[0050] In the training process of the cloud detection model, the cloud label can be divided into three categories according to the cloud type characteristics of the remote sensing image: cloud, cloud shadow and cloud-free background. The training sample data of the cloud detection model is labeled by manual or pre-set application, and the consistency with the original image size is ensured, and the accuracy of the original image is preserved. As shown in FIG. 13, the first row is the optical remote sensing image original drawing, and the second row is the corresponding cloud type true value map. White, gray and black represent cloud, cloud shadow and cloud-free background respectively. Figure 6

[0051] The training sample set of the cloud detection model in this embodiment can be composed of optical remote sensing images collected in different regions, different urban and rural conditions and different seasons. The collected optical remote sensing images are cropped or compressed to a uniform size, and class balancing processing is performed to ensure that the proportions of each type of cloud, cloud shadow and cloud-free background in the training samples are balanced. The class balancing processing includes but is not limited to expanding the training samples by sampling, screening the samples by random sampling and adjusting the sample weights.

[0052] In some embodiments, before the remote sensing image is input into the cloud detection model that has been pre-trained, the remote sensing image is also subjected to a pre-processing operation, which includes at least one of the following:

[0053] Radiometric calibration is performed on the remote sensing image, specifically, the original digital quantization value (DN value) obtained by the remote sensing sensor is converted into a quantitative process of absolute radiometric brightness or reflectivity, to eliminate the response differences of each detection unit inside the sensor and ensure the consistency of the image radiation.

[0054] Linear stretching is performed on the remote sensing image, specifically, the original remote sensing influence is converted to a target range by a pre-set linear proportional mapping. In addition, the remote sensing image can also be resampled according to a pre-set spatial resolution, for example, the resolution of the remote sensing image is unified from 10 meters to 30 meters.

[0055] In this embodiment, in order to improve the adaptability of the cloud detection model to complex remote sensing scenes, various data enhancement strategies are introduced in the training stage of the cloud detection model, including random flipping, rotation, cropping, color disturbance and Gaussian noise operations. The above enhancement operations are randomly applied to the training samples with a pre-set probability to form a rich high-quality sample set, effectively alleviating the model overfitting problem and significantly improving the generalization performance of the model on unknown area images.

[0056] ​In this embodiment, in order to improve the convergence efficiency and stability of the model, a cosine annealing learning rate scheduling strategy with a preheating mechanism is introduced in the training process. In the early stage of training, a linearly increasing preheating mechanism is used to stabilize the model weight update, and in the middle and late stages of training, a cosine function decreasing strategy is used to gradually reduce the learning rate, effectively improving the model convergence speed and detection accuracy, preventing early training shock and later overfitting. At the same time, an early stopping strategy is used to monitor the loss change of the validation set, and when no significant performance improvement is obtained for consecutive training rounds, the training is automatically stopped, effectively preventing model overfitting.

[0057] In this embodiment, the loss function of the cloud detection model includes a main segmentation loss function and a boundary loss function. The main segmentation loss function adopts a class weighted cross entropy loss function (Weighted-Cross Entropy Loss, WCEL) to reduce the problem of low detection accuracy caused by the imbalance of cloud class samples in the model. The boundary loss function adopts a boundary IoU loss function (Boundary-IoU Loss, BIL) to strengthen the boundary discrimination ability of the boundary segmentation module in the model. The final loss function of the cloud detection model is the weighted sum of the main segmentation loss function and the boundary loss function.

[0058] The main segmentation loss function is:

[0059]

[0060] where C is the number of cloud categories, y i is the true value of the sample, p i is the probability of being predicted as the i-th category, w i is the weight of the i-th category.

[0061] The positive boundary IoU loss function (Boundary-IoU Loss, BIL) is:

[0062]

[0063] where G represents the true value of the sample, P represents the predicted value, G d and P d represent the boundary region with a width of d of G and P respectively.

[0064] The final loss function of the cloud detection model is:

[0065] WBL = λ1·WCEL + λ2BIL

[0066] where λ1 and λ2 are hyperparameter weights that can be dynamically adjusted according to the training target.

[0067] In addition, the cloud detection model in the present application can be integrated into an existing remote sensing image processing platform and geographic information system to provide an accurate cloud detection scheme for massive optical remote sensing data.

[0068] By the cloud detection method proposed in the present embodiment, the remote sensing image is input into the cloud detection model trained in advance, the first multi-scale feature corresponding to the remote sensing image is obtained through the encoder in the cloud detection model, the multiple attention vectors corresponding to the multiple size first feature maps in the first multi-scale feature are obtained through the attention module in the cloud detection model, the multiple attention vectors and the first multi-scale feature are fused to obtain the second multi-scale feature, the preliminary segmentation map of the remote sensing image is obtained according to the second multi-scale feature through the decoder in the cloud detection model, and the target regions of different spatial receptive fields are focused; the edge prediction map corresponding to the remote sensing image is obtained through the boundary perception module in the cloud detection model, and the discrimination ability of the model for the cloud boundary region is enhanced; and the preliminary segmentation map and the edge prediction map are fused to obtain the cloud type true value map corresponding to the remote sensing image, so as to solve the problems of fuzzy edge region prediction and boundary fracture of the model in the prior art, and make the cloud boundary clearer. Through the cloud detection method of the present application, the cloud edge detection precision is effectively improved, and the image details are preserved.

[0069] In some optional embodiments, the above step S101 includes but is not limited to: obtaining, by the encoder, multiple initial feature maps of multiple sizes corresponding to the remote sensing image, and multiple position encodings corresponding to the multiple initial feature maps; and obtaining the first multi-scale feature according to the multiple initial feature maps and the multiple position encodings.

[0070] In the present embodiment, MiT (Mix Vision Transformer) is used as the encoder backbone network in the cloud detection model to realize extraction of the first multi-scale feature. MiT performs hierarchical encoding through multiple stage down-sampling of the remote sensing image, layer-by-layer extracts spatial semantic feature information of the remote sensing image, and introduces independent position encodings matching the initial feature maps at each stage.

[0071] In one example, as shown in Figure 7 the input image size is set to HxW, and the initial feature maps are obtained through hierarchical down-sampling of the encoder at four stages, and the spatial sizes of the initial feature maps are: stage 1: H1xW1, stage 2: H2xW2, stage 3: H3xW3, and stage 4: H4xW4

[0072] After each initial feature map is flattened into a sequence, the dimension is:

[0073]

[0074] wherein X (s) represents the feature embedding of the s-th stage, Ds is the feature dimension of this stage.

[0075] Initialize a set of independent and trainable position encoding vectors for each stage respectively:

[0076]

[0077] That is, each stage corresponds to an independent position encoding matrix, and the dimension is the initial feature dimension of this stage Figure 1 . Then, the position encoding is added to the initial feature map to form the vector representation input to the attention module:

[0078] Z (s) = X (s) + P (s) , s e {1, 2, 3, 4}

[0079] Z (s) as the input of the s-th stage (the first feature map), which retains the spatial position information in the original image and adapts to the semantic modeling process under the current scale.

[0080] In this embodiment, a plurality of initial feature maps of multiple sizes corresponding to the remote sensing image are obtained, and a plurality of position encodings corresponding to the plurality of initial feature maps are obtained. A first multi-scale feature is obtained according to the plurality of initial feature maps and the plurality of position encodings. The expression ability of the model to the spatial structure information in the remote sensing image is maintained, and the modeling precision and context perception ability of the model to the multi-scale target are enhanced.

[0081] In some optional embodiments, the step S104 specifically includes but is not limited to: performing gradient guided processing on the plurality of second feature maps to obtain an edge response map corresponding to the remote sensing image; and performing convolution on the edge response map to generate an edge prediction map.

[0082] The gradient guided processing is performed on the middle-deep layer second feature map of the second multi-scale feature map to extract an edge response region between the cloud body and the background or other categories. Specifically, the gradient features of the plurality of second feature maps are extracted, the gradient features are converted into edge features through a preset algorithm (such as NMS non-maximum suppression, threshold segmentation), and an edge response map is generated. Then, the edge prediction map corresponding to the remote sensing image is generated by processing the edge response map.

[0083] In some optional embodiments, the edge response map is obtained according to the gradient features of the middle-deep layer second feature map, wherein the resolution of the middle-deep layer second feature map is greater than a preset resolution.

[0084] In one example, the middle-deep layer second feature map output by the attention module is set as F, and the Sobel operator is used to calculate the gradient of the middle-deep layer second feature map F:

[0085]

[0086] wherein K x , K y are horizontal and vertical Sobel convolution kernels respectively; G is an edge response map, which is used for subsequent input of a shallow convolution layer to generate a pixel-level edge prediction map.

[0087] The edge prediction map is generated by convolution according to the edge response map, including but not limited to: feature extraction and feature compression of the edge response map by a convolution layer to obtain the edge prediction map.

[0088] The pixel-level edge prediction map is generated by a shallow convolution stack, which uses a small number of convolution layers to gradually extract and compress the edge response map G to obtain an edge probability map (i.e., a prediction of whether each pixel is an edge), wherein the convolution layers include 3x3 convolution, normalization and ReLU nonlinear activation function, specifically including:

[0089] H1 = RELU (BN (W1 G + b1))

[0090] E = sigma (W2 H1 + b2)

[0091] wherein G is an edge response map; BN is a normalization operation; W1 and W2 are shallow convolution kernels (3x3 size); b1 and b2 are bias terms; H1 is an intermediate feature map; sigma is a Sigmoid function, outputting an edge probability of 0-1; and E is a final pixel-level edge prediction map.

[0092] The second aspect of the application also provides a cloud detection device, as shown in Figure 8 The device comprises:

[0093] A first processing module 80 is configured to input a remote sensing image into a pre-trained cloud detection model, and obtain a first multi-scale feature corresponding to the remote sensing image through an encoder in the cloud detection model.

[0094] A second processing module 82 is configured to obtain a plurality of attention vectors corresponding to a plurality of feature maps of different sizes in the first multi-scale feature through an attention module in the cloud detection model, and fuse the plurality of attention vectors and the first multi-scale feature to obtain a second multi-scale feature.

[0095] A third processing module 84 is configured to obtain a preliminary segmentation map of the remote sensing image according to the second multi-scale feature through a decoder in the cloud detection model.

[0096] An edge detection module 86 is configured to acquire an edge prediction map corresponding to the remote sensing image by a boundary perception module in the cloud detection model, wherein the edge prediction map includes a prediction result of each pixel being an edge.

[0097] A fourth processing module 88 is configured to fuse the preliminary segmentation map and the edge prediction map to obtain a cloud type ground truth map corresponding to the remote sensing image.

[0098] Optionally, in the embodiment, the first processing module 80 includes:

[0099] A first processing submodule is configured to acquire a plurality of initial feature maps corresponding to the remote sensing image by the encoder, and a plurality of position encodings corresponding to the plurality of initial feature maps.

[0100] A second processing submodule is configured to acquire the first multi-scale feature according to the plurality of initial feature maps and the plurality of position encodings.

[0101] Optionally, in the embodiment, the edge detection module 86 includes:

[0102] An edge detection submodule is configured to perform gradient-guided processing on the plurality of second feature maps to acquire an edge response map corresponding to the remote sensing image.

[0103] A third processing submodule is configured to perform convolution according to the edge response map to generate the edge prediction map.

[0104] Optionally, in the embodiment, the edge detection submodule includes an acquisition unit configured to acquire the edge response map according to gradient features of a second feature map of a middle deep layer, wherein a resolution of the second feature map of the middle deep layer is greater than a preset resolution.

[0105] The third processing submodule includes a processing unit configured to perform feature extraction and feature compression on the edge response map by a convolution layer to obtain the edge prediction map.

[0106] Optionally, in the embodiment, further includes a preprocessing module configured to perform a preprocessing operation on the remote sensing image before inputting the remote sensing image into the cloud detection model which is pre-trained, and the preprocessing operation includes at least one of the following:

[0107] Radiometric calibration is performed on the remote sensing image.

[0108] Linear stretching is performed on the remote sensing image.

[0109] Resampling is performed on the remote sensing image according to a preset spatial resolution.

[0110] Another embodiment of the present application relates to an electronic device, such as Figure 9 as shown, comprising: at least one processor 901; and a memory 902 connected with the at least one processor 901 in communication; wherein the memory 902 stores instructions executable by the at least one processor 901, and the instructions are executed by the at least one processor 901 to enable the at least one processor 901 to perform the cloud detection method in each of the above embodiments.

[0111] The memory and the processor are connected in a bus mode, and the bus can include any number of interconnected buses and bridges, and the bus connects various circuits of one or more processors and memories together. The bus can also connect various other circuits such as peripheral devices, voltage stabilizers and power management circuits, which are well known in the art, and therefore, further description is not given herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be one element or multiple elements, such as multiple receivers and transmitters, which provide a unit for communicating with various other devices on a transmission medium. The data processed by the processor is transmitted on a wireless medium through the antenna, and further, the antenna also receives data and transmits the data to the processor.

[0112] The processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interface, voltage regulation, power management and other control functions. The memory can be used to store data used by the processor in performing operations.

[0113] Another embodiment of the present application relates to a computer readable storage medium, which stores a computer program. The computer program is executed by the processor to implement the above-mentioned embodiment of the cloud detection method.

[0114] That is, those skilled in the art can understand that all or part of the steps of the above-mentioned embodiment method can be completed by a program instructing related hardware, the program is stored in a storage medium, and includes a plurality of instructions for causing a device (which can be a single chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (Read-Only Memory, ROM for short), a random access memory (Random Access Memory, RAM for short), a magnetic disk or an optical disk, and various storage program codes.

[0115] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the Figure 1 function specified in the flow or flows and / or blocks Figure 1 of the block or blocks.

[0116] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions that are executed on the computer or other programmable apparatus provide steps for implementing the Figure 1 function specified in the flow or flows and / or blocks Figure 1 Figure 1 of the block or blocks.

[0117] The foregoing description of specific exemplary embodiments of the application has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the application to the precise form disclosed, and various modifications and variations are possible in light of the above teachings. It is intended that the scope of the application be limited not with the specific examples presented, but rather by the claims appended hereto.

Claims

1. A cloud detection method, characterized in that, include: The remote sensing image is input into a pre-trained cloud detection model, and the encoder in the cloud detection model is used to obtain the first multi-scale feature corresponding to the remote sensing image. The attention module in the cloud detection model is used to obtain multiple attention vectors corresponding to multiple first feature maps in the first multi-scale feature, and the second multi-scale feature is obtained based on the multiple attention vectors and the first multi-scale feature. The preliminary segmentation map of the remote sensing image is obtained by the decoder in the cloud detection model based on the second multi-scale feature; The edge prediction map corresponding to the remote sensing image is obtained through the boundary perception module in the cloud detection model, wherein the edge prediction map includes the prediction result of each pixel as the cloud edge; Based on the preliminary segmentation map and the edge prediction map, the target segmentation map corresponding to the remote sensing image is obtained.

2. The method according to claim 1, characterized in that, The step of inputting remote sensing images into a pre-trained cloud detection model, and obtaining the first multi-scale features corresponding to the remote sensing images through the encoder in the cloud detection model, includes: The encoder is used to obtain multiple initial feature maps corresponding to the remote sensing image, and multiple location codes corresponding to the multiple initial feature maps; The first multi-scale feature is obtained based on the plurality of initial feature maps and the plurality of position codes.

3. The method according to claim 2, characterized in that, The step of obtaining the edge prediction map corresponding to the remote sensing image through the boundary awareness module in the cloud detection model includes: Gradient-guided processing is performed on the plurality of second feature maps to obtain the edge response map corresponding to the remote sensing image; The edge response map is convolved to generate the edge prediction map.

4. The method according to claim 3, characterized in that, The step of performing gradient-guided processing on the plurality of second feature maps to obtain the edge response map corresponding to the remote sensing image includes: The edge response map is obtained based on the gradient features of the second feature map of the mid-deep layer, wherein the resolution of the second feature map of the mid-deep layer is greater than a preset resolution; The step of generating the edge prediction map by convolution based on the edge response map includes: The edge prediction map is obtained by performing feature extraction and feature compression on the edge response map through convolutional layers.

5. The method according to claim 1, characterized in that, Before inputting the remote sensing image into the pre-trained cloud detection model, the method further includes: The remote sensing image is preprocessed, and the preprocessing operation includes at least one of the following: Radiometric calibration is performed on the remote sensing images; Linear stretching is applied to the remote sensing image; The remote sensing image is resampled according to a preset spatial resolution.

6. A cloud detection device, characterized in that, include: The first processing module is used to input remote sensing images into a pre-trained cloud detection model, and obtain the first multi-scale features corresponding to the remote sensing images through the encoder in the cloud detection model. The second processing module is used to obtain multiple attention vectors corresponding to multiple first feature maps in the first multi-scale feature through the attention module in the cloud detection model, and to obtain the second multi-scale feature based on the multiple attention vectors and the first multi-scale feature. The third processing module is used to obtain a preliminary segmentation map of the remote sensing image based on the second multi-scale feature through the decoder in the cloud detection model. The edge detection module is used to obtain the edge prediction map corresponding to the remote sensing image through the boundary perception module in the cloud detection model, wherein the edge prediction map includes the prediction result of each pixel being the cloud edge; The fourth processing module is used to obtain the target segmentation map corresponding to the remote sensing image based on the preliminary segmentation map and the edge prediction map.

7. The apparatus according to claim 6, characterized in that, The first processing module includes: The first processing submodule is used to obtain multiple initial feature maps corresponding to the remote sensing image and multiple position codes corresponding to the multiple initial feature maps through the encoder. The second processing submodule is used to obtain the first multi-scale feature based on the plurality of initial feature maps and the plurality of position codes.

8. The apparatus according to claim 7, characterized in that, The edge detection module includes: An edge detection submodule is used to perform gradient-guided processing on the plurality of second feature maps to obtain the edge response map corresponding to the remote sensing image. The third processing submodule is used to convolve the edge response map to generate the edge prediction map.

9. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the cloud detection method as described in any one of claims 1 to 5.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the cloud detection method as described in any one of claims 1 to 5.

Images