An improved semantic segmentation method based on unet for cloud and cloud shadow

Abstract

The application aims at a cloud and cloud shadow improved semantic segmentation method based on Unet, comprising: improving the Unet model to obtain a semantic segmentation subnetwork; processing the input cloud image by using the semantic segmentation subnetwork to obtain a semantic segmentation feature map through upsampling; forming four convolution stages, fusing the feature maps obtained by the four convolution stages through 1x1 convolution operation to construct an edge detection subnetwork; processing the input cloud image by using the edge detection subnetwork to obtain an edge detection feature map; and weighting and fusing the input cloud image, the semantic segmentation feature map and the edge detection feature map to output a cloud and cloud shadow segmentation result. The application can effectively improve the recognition accuracy of complex cloud images when the cloud and cloud shadow have more types or shapes.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of remote sensing image processing, and particularly relates to a cloud and cloud shadow improved semantic segmentation method based on Unet, which is especially suitable for application scenarios such as surface feature extraction, climate detection, atmospheric correction and the like. BACKGROUND

[0002] Cloud and cloud shadow detection and segmentation are of great significance in remote sensing image processing, and are of great help to surface feature extraction, climate detection and the like. Cloud and cloud shadow detection can effectively reduce the interference of cloud and cloud shadow on ground object recognition, especially in the fields of vegetation coverage, land use change and water resource monitoring, and can help predict the trend of climate change. At the same time, observing the change of cloud shadow can better evaluate the spatial and temporal distribution of solar energy resources, and provide scientific basis for solar power station site selection and agricultural, industrial and other production activities.

[0003] Traditional cloud and cloud shadow detection methods mainly fall into three categories: one is based on statistics, one is statistical equation method, which uses data samples to build mathematical models, calculates brightness, reflectivity and other parameters to detect cloud layers and identifies cloud shadows according to geometric relationships. The other is the clustering analysis method, which clusters and groups remote sensing image pixels to identify and distinguish cloud layers; the second is based on spectral threshold, which uses the spectral characteristics of single or multi-temporal images to set threshold detection, compares the gray value or reflectivity of each pixel in the image with a predefined threshold, and classifies the pixels as cloud shadow or non-cloud shadow according to the result; the third is based on morphological and texture features, which realizes detection and segmentation according to the typical features of cloud and cloud shadow. However, traditional methods have challenges in detecting different types of clouds and cloud shadows. For example, threshold methods are prone to missed detection or false detection in complex backgrounds, and light reflection can also cause great interference, which requires additional discrimination rules, increasing the complexity and computational load of the algorithm. Affected by factors such as noise interference, the accuracy of the label and the detection efficiency are relatively low, and the detection of the boundary of cloud and cloud shadow is very rough, and the edge information extraction capability is not high.

[0004] With the continuous progress of deep learning technology, semantic segmentation network is introduced into remote sensing image processing. CNN-based network model performs well in image classification tasks and lays the foundation for completing pixel-level classification tasks, i.e. semantic segmentation. Some researchers first introduced the concept of fully convolutional neural network (FCN) and realized image pixel classification through the improvement of CNN network, achieving the effect of image segmentation. Subsequently, someone introduced the encoder-decoder structure and realized pixel-level classification through maximum pooling index transmission. By designing a U-shaped network (Unet) based on encoder and decoder, and establishing a skip connection between the encoder and the decoder, the details of the image are effectively preserved. UNet++ further optimizes and improves the UNet structure in medical image segmentation, effectively enhances the feature expression ability of the network with the help of nested and dense connection modules. Some researchers also proposed an image segmentation method called DeepLab, which uses a deep convolutional neural network (DCNN) structure, expands the receptive field of each network layer with various network layers and atrous convolution, and refines the feature information with a fully connected random field (CRF) to achieve accurate semantic segmentation of images. However, these networks often ignore the interaction between local and global features, making it difficult to accurately understand information when there are complex features or interference noise.

[0005] The invention disclosed in patent publication CN114943876A discloses a multi-level semantic fusion cloud and cloud shadow detection method, device and storage medium, which uses a residual network (ResNet) as the backbone network, combines the encoder-decoder structure, and proposes a cloud image segmentation method containing a multi-branch residual context semantic module, a multi-scale convolution sub-channel attention module and a feature fusion up-sampling module. However, this invention focuses on cloud edge and thin cloud detection, and when there are many types or shapes of clouds and cloud shadows in the cloud image, the edge detection of this patent relies on the feature extraction of the main network, lacks independent branches, and may damage the integrity of different features, and the boundary segmentation is not sharp enough, resulting in jagged or discontinuous segmentation of cloud shadow edges, thereby affecting the effect of cloud shadow segmentation, and the error and omission rates are high; at the same time, the fusion method used in this patent may also cause information loss in the above-mentioned complex background, and the low-contrast texture may be smoothed by the pooling operation during high-level feature fusion, resulting in the loss of texture information in the thin cloud area, and if the fusion does not fully separate the semantic and edge channels, the model may misjudge the ground object shadow as a cloud shadow. SUMMARY

[0006] The application aims to provide an improved Unet-based cloud and cloud shadow semantic segmentation method.

[0007] To achieve the above technical purposes, the application adopts the following technical scheme:

[0008] The application discloses an improved Unet-based cloud and cloud shadow semantic segmentation method, which comprises the following steps:

[0009] S1, replace the down-sampling of the Unet model with an improved ASPP spatial pyramid pooling module, and embed an attention mechanism module in the skip connection to construct a semantic segmentation subnetwork; the semantic segmentation subnetwork is used to process the input cloud image, and semantic segmentation feature maps are obtained through up-sampling;

[0010] S2, connect four convolutional layers in sequence to form four convolutional stages, add a 1x1 convolutional operation to each convolutional stage, then use a deconvolutional operation to restore the feature maps of the last three convolutional stages to the same size as the input image, and finally fuse the feature maps obtained by the four convolutional stages through a 1x1 convolutional operation to construct an edge detection subnetwork; the edge detection subnetwork is used to process the input cloud image to obtain edge detection feature maps;

[0011] S3, the input cloud image, semantic segmentation feature maps and edge detection feature maps are weighted and fused, and the fused features are processed through a Sigmoid activation function layer to output cloud and cloud shadow segmentation results.

[0012] Further, in step S2, the improved ASPP spatial pyramid pooling module comprises a first semantic convolutional layer, a second semantic convolutional layer, a third semantic convolutional layer, a fourth semantic convolutional layer, an average pooling layer, a maximum pooling layer, a semantic feature fusion layer and a 1x1 convolutional layer.

[0013] The first semantic convolutional layer, the second semantic convolutional layer, the third semantic convolutional layer and the fourth semantic convolutional layer respectively extract features from the input cloud image; the maximum pooling layer performs a maximum operation on the local region of the input cloud image to extract local detail features including the edges and textures of the cloud shadow in the input cloud image; and the average pooling layer performs an average operation on the local region of the input cloud image to extract global context features including the overall shape of the cloud and the distribution of the cloud and the cloud shadow in the input cloud image.

[0014] Further, the first semantic convolutional layer uses a 1x1 convolutional kernel to extract features of the input cloud image, and extracts preliminary features of the cloud image; the second semantic convolutional layer uses a 3x3 convolutional kernel and sets a hole rate of 6 to capture large-range context information in the cloud image; the third semantic convolutional layer uses a 3x3 convolutional kernel and increases the hole rate to 12 to capture larger-range context information in the cloud image, and extracts large-scale features between clouds and cloud shadows in the cloud image; the fourth semantic convolutional layer uses a 3x3 convolutional kernel and increases the hole rate to 18 to capture the largest-range context information in the cloud image as global features between clouds and cloud shadows in the cloud image; the semantic feature fusion layer concatenates feature maps from different convolutional layers and pooling layers to generate more rich feature representations by integrating local details and global context information; and the 1x1 convolutional layer further convolves the feature map fused by the semantic feature fusion layer to adjust the number of channels of the feature map and finally generates an output feature map for subsequent tasks.

[0015] Further, in step S2, the edge detection sub-network includes a first edge convolutional layer, a second edge convolutional layer, a third edge convolutional layer, a fourth edge convolutional layer and an edge feature fusion layer connected in sequence, and four 1x1 convolutional layers connected with the first convolutional layer, the second convolutional layer, the third convolutional layer and the fourth convolutional layer, respectively.

[0016] Each of the first edge convolutional layer, the second edge convolutional layer, the third edge convolutional layer and the fourth edge convolutional layer is inserted with a pooling layer, and the first edge convolutional layer, the second edge convolutional layer, the third edge convolutional layer and the fourth edge convolutional layer sequentially extract features of the input cloud image to obtain cloud shadow edge features of different scales, and reduce the resolution of the feature map through the respective corresponding pooling layer.

[0017] The output feature of the first edge convolutional layer is obtained after 1x1 convolutional operation to obtain the first convolution stage feature map; the output features of the second edge convolutional layer, the third edge convolutional layer and the fourth edge convolutional layer are respectively obtained after 1x1 convolutional operation, and then the corresponding convolution stage feature map is restored to the same scale of the input cloud image by using the deconvolution operation, to obtain the second convolution stage feature map, the third convolution stage feature map and the fourth convolution stage feature map, respectively.

[0018] The edge feature fusion layer is a 1x1 convolutional layer, which fuses the first convolution stage feature map, the second convolution stage feature map, the third convolution stage feature map and the fourth convolution stage feature map to obtain an edge detection feature map containing edge information of cloud shadow regions of different ranges.

[0019] Further, the first edge convolution layer, the second edge convolution layer, the third edge convolution layer and the fourth edge convolution layer all adopt 3*3 convolution layers, and the channel numbers are 64, 128, 256 and 512 respectively.

[0020] The step S3 further comprises:

[0021] The input cloud image , the semantic segmentation feature map and the edge detection feature map are spliced and fused to obtain a fused feature map .

[0022] .

[0023] The fused feature map contains the sum of the channel numbers of the input cloud image , the semantic segmentation feature map and the edge detection feature map .

[0024] The global average value and the maximum value of each feature channel of the fused feature map are obtained by performing average pooling and maximum pooling operations on each feature channel of the fused feature map :

[0025] .

[0026] .

[0027] wherein i represents the index of the feature channel, .

[0028] The weight of each feature channel is assigned by multiplication calculation to extract effective feature information; the multiplication calculation formula is as follows:

[0029] .

[0030] wherein represents the feature value of the i-th channel of the weighted fused feature map; represents the feature value of the k-th channel of the feature map; represents the weight coefficient of the i-th channel; and N represents the feature channel number.

[0031] Compared with the prior art, the present application has the following beneficial effects:

[0032] The Unet-based cloud and cloud shadow improved semantic segmentation method of the application, for complex cloud images when the types or shapes of clouds and cloud shadows are more, proposes a new edge detection and feature fusion method, effectively splices the original image, semantic segmentation features and edge detection features, uses concat fusion to splice the semantic features and edge features in the channel dimension, completely retains the original information of both, retains the integrity of the original features, and reduces isolated false detection in edge detection. The application deploys an attention mechanism and an improved ASPP hollow spatial pyramid pooling module in the semantic segmentation subnetwork, so that the model can adaptively adjust the feature expression of the cloud and cloud shadow feature map, improve the model's ability to capture key information, and capture rich multi-scale information at the local feature level. Compared with ordinary convolution operation, the number of parameters is effectively reduced; the fusion of edge features and semantic features effectively improves the model's ability to extract edge features, and obtains clearer and more complete image target edge information. The application combines the advantages of different subnetworks intuitively, and also avoids information loss that may be caused by early fusion, especially in complex backgrounds, the complementarity of edges and semantics is stronger, and the misjudgment and omission rates are low. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 is the Unet-based cloud and cloud shadow improved semantic segmentation diagram designed by the application;

[0034] Figure 2 is the CBAM diagram designed by the application;

[0035] Figure 3 is the improved ASPP diagram designed by the application;

[0036] Figure 4 is the edge detection subnetwork diagram designed by the application;

[0037] Figure 5 is the feature fusion module diagram designed by the application.

[0038] Figure 6 is the recognition result diagram of the application, wherein (a) is a cloud image over a large piece of vegetation covered land in a remote sensing image data set, (e) is the experimental result diagram corresponding to (a); (b) is a cloud image over a coast in a remote sensing image data set, (f) is the experimental result diagram corresponding to (b); (c) is a cloud image over a barren mountain in a remote sensing image data set, (g) is the experimental result diagram corresponding to (c); (d) is a broken and scattered cloud image over the ocean in a remote sensing image data set, and (h) is the experimental result diagram corresponding to (d). DETAILED DESCRIPTION

[0039] The embodiments of the application are further described in detail below with reference to the accompanying drawings.

[0040] The application discloses an improved semantic segmentation method based on a cloud and cloud shadow Unet, and the method comprises the following steps:

[0041] S1, replace the down-sampling of the Unet model with an improved ASPP spatial pyramid pooling module, embed an attention mechanism module in the skip connection, and construct a semantic segmentation subnetwork; the input cloud image is processed by using the semantic segmentation subnetwork, and a semantic segmentation feature map is obtained through up-sampling;

[0042] S2, sequentially connect four convolutional layers to form four convolutional stages, add a 1x1 convolutional operation to each convolutional stage, then restore the feature maps of the last three convolutional stages to the same scale of the input image by using a deconvolution operation, and finally fuse the feature maps obtained by the four convolutional stages through a 1x1 convolutional operation to construct an edge detection subnetwork; the input cloud image is processed by using the edge detection subnetwork to obtain an edge detection feature map;

[0043] S3, the input cloud image, the semantic segmentation feature map and the edge detection feature map are weighted and fused, and the fused features are processed through a Sigmoid activation function layer to output a cloud and cloud shadow segmentation result.

[0044] As shown in Figure 1 , the input of the model is divided into three parts: a semantic segmentation subnetwork, an edge detection subnetwork and a feature fusion module. The main implementation steps include: replacing the down-sampling of the semantic segmentation subnetwork with an improved ASPP spatial pyramid pooling module, embedding a CBAM (channel and spatial attention module) in the skip connection, and obtaining a semantic segmentation feature map through up-sampling; obtaining the contour feature information of the input image through four consecutive convolutional layers in the edge detection subnetwork to obtain an edge detection feature map; weighting and fusing the input feature map, the semantic segmentation feature map and the edge detection feature map through the feature fusion module, and finally outputting an image through Sigmoid.

[0045] As shown in Figure 2 , the CBAM is a widely used adaptive feature importance learning mechanism in convolutional neural networks, which can improve the network performance. When a feature map is given, the module will independently infer the attention weight value along two independent dimensions in turn, and then multiply the generated attention weight map with the input feature map for adaptive refinement. The mechanism is composed of a channel attention module and a spatial attention module: the channel attention module learns and highlights the importance of each channel in feature extraction through weighted average processing; the spatial attention module learns and emphasizes the importance of each spatial position in feature extraction through weighted average processing, effectively highlighting the target region and suppressing irrelevant information. The formula is as follows:

[0046] ;

[0047] ;

[0048] ;

[0049] ;

[0050] wherein, represents the input feature map; represents a Sigmoid function; represents the feature map after channel attention; represents the input feature map of spatial attention; represents the feature map after spatial attention; represents the weight feature vector output after CBAM fusion attention operation.

[0051] Unet network can capture features at different levels through the encoder-decoder structure, but its ability to extract multi-scale information is relatively limited. In the encoder stage, it mainly focuses on the extraction of local information, while in the decoder stage, although upsampling and feature fusion are performed, it may still not be able to effectively integrate local information and global information, resulting in insufficient use of receptive field and affecting the understanding of the overall structure of the image and the target.

[0052] As a means to effectively expand the receptive field of ordinary convolution kernels, the dilated convolution has always been favored by researchers in the segmentation field. Its ability lies in significantly expanding the receptive field of the convolution kernel without increasing the model parameter quantity and computational complexity. By adding intervals to the ordinary convolution kernel to form a dilated convolution, the convolution kernel can cover a larger range without changing the size of the convolution kernel or the number of layers. The relationship between the range of the dilated convolution kernel and its dilated rate is as follows:

[0053] ;

[0054] wherein, r is the dilated rate; is the range of the initial convolution kernel; is the actual receptive field size of the dilated convolution.

[0055] However, the original ASPP lacks balanced processing of local and global features, resulting in insufficient extraction of some information, so an Avgpool and a Maxpool are added to the ASPP module in this paper:

[0056] a) AvgPool (average pooling): By averaging the local region of the feature map, it can retain more global context information, which helps the network to capture the overall structure of the image.

[0057] b) MaxPool (Max Pooling): By performing a maximum operation on the local regions of the feature map, more local detail information can be preserved, which helps the network to capture the edge, texture and other detailed features of the image.

[0058] By combining AvgPool and MaxPool, the module can simultaneously preserve global context information and local detail information, thereby achieving a better balance in the feature extraction process. As shown in Figure 3 the improved ASPP spatial pyramid pooling module includes a first semantic convolutional layer, a second semantic convolutional layer, a third semantic convolutional layer, a fourth semantic convolutional layer, an average pooling layer, a maximum pooling layer, a semantic feature fusion layer and a 1x1 convolutional layer; the first semantic convolutional layer, the second semantic convolutional layer, the third semantic convolutional layer, the fourth semantic convolutional layer respectively extract features from the input cloud image; the maximum pooling layer performs a maximum operation on the local region of the input cloud image, and extracts the local detail features of the input cloud image including the edge and texture of the cloud shadow; the average pooling layer performs an average operation on the local region of the input cloud image, and extracts the global context features of the input cloud image including the overall shape of the cloud and the distribution of the cloud and cloud shadow. Specifically, the first semantic convolutional layer uses a 1x1 convolutional kernel to extract features from the input cloud image, and extracts the preliminary features of the cloud image; the second semantic convolutional layer uses a 3x3 convolutional kernel and sets the hole rate to 6 to capture the context information in a larger range of the cloud image; the third semantic convolutional layer uses a 3x3 convolutional kernel and increases the hole rate to 12 to capture the context information in a larger range of the cloud image, and extracts the large-scale features between the cloud and the cloud shadow in the cloud image; the fourth semantic convolutional layer uses a 3x3 convolutional kernel and increases the hole rate to 18 to capture the context information in the largest range of the cloud image as the global features between the cloud and the cloud shadow in the cloud image; the semantic feature fusion layer concatenates the feature maps from different convolutional layers and pooling layers, integrates local details and global context information, and generates more rich feature representations; the 1x1 convolutional layer performs further convolution operation on the feature map fused by the semantic feature fusion layer, adjusts the number of channels of the feature map, and finally generates the output feature map for subsequent tasks . This combination can enhance the network's ability to extract multi-scale features, especially when dealing with complex segmentation tasks, it can better cope with targets of different scales.

[0059] In the edge detection sub-network, the convolutional layers are divided into four stages, and a pooling layer is inserted between each of the four convolutional stages to reduce the dimension of the information extracted by the convolution. After adding a 1x1 convolution operation in each convolutional stage, the feature map is restored to the same scale as the input image using the deconvolution operation to ensure the same scale as the input image. Finally, the feature maps obtained from the four stages are fused through a 1x1 convolution operation. This can effectively obtain the outline of the image and obtain clearer and more complete image target edge information, as shown in Figure 4 The edge detection sub-network includes a first edge convolutional layer, a second edge convolutional layer, a third edge convolutional layer, a fourth edge convolutional layer, and an edge feature fusion layer connected in sequence, and four 1x1 convolutional layers connected with the first convolutional layer, the second convolutional layer, the third convolutional layer, and the fourth convolutional layer, respectively; a pooling layer is inserted in each of the first edge convolutional layer, the second edge convolutional layer, the third edge convolutional layer, and the fourth edge convolutional layer, and the first edge convolutional layer, the second edge convolutional layer, the third edge convolutional layer, and the fourth edge convolutional layer sequentially extract features from the input cloud image to obtain cloud shadow edge features of different scales, and reduce the resolution of the feature map through the corresponding pooling layer; the output feature of the first edge convolutional layer is obtained after a 1x1 convolution operation to obtain a first convolution stage feature map; the output features of the second edge convolutional layer, the third edge convolutional layer, and the fourth edge convolutional layer are respectively obtained after a 1x1 convolution operation, and then the feature maps of the corresponding convolution stages are restored to the same scale as the input cloud image using the deconvolution operation, to obtain a second convolution stage feature map, a third convolution stage feature map, and a fourth convolution stage feature map, respectively; the edge feature fusion layer is a 1x1 convolutional layer that fuses the first convolution stage feature map, the second convolution stage feature map, the third convolution stage feature map, and the fourth convolution stage feature map to obtain an edge detection feature map containing edge information of cloud shadow regions of different ranges. Preferably, the first edge convolutional layer, the second edge convolutional layer, the third edge convolutional layer, and the fourth edge convolutional layer all use 3x3 convolutional layers, and the number of channels is 64, 128, 256, and 512, respectively.

[0060] 1) Input feature map

[0061] The input feature map is represented as I, and its size is HxWxC, where H is the height, W is the width, and C is the number of channels.

[0062] 2) The network is divided into four convolutional stages, each stage containing convolutional and pooling operations, and the calculation formula is as follows:

[0063] ;

[0064] wherein, is the result of the convolutional operation in each stage, represents the number of network layers, ; represents the layer convolution kernel size; represents the layer convolution output channel number; represents the pooling kernel size;

[0065] 3) After each convolution stage, add a 1x1 convolution operation to further extract features and reduce the number of channels, as follows:

[0066] ;

[0067] wherein, is the stage convolution result after being pooled;

[0068] 4) In order to restore the feature map to the same scale as the input image, use the deconvolution operation to up-sample the feature map of each stage, as follows:

[0069] ;

[0070] wherein, is the deconvolution result, is the deconvolution kernel size;

[0071] 5) The feature maps of the four stages are fused by a 1x1 convolution operation to obtain the final feature map:

[0072] ;

[0073] wherein, is the final output feature map of the edge detection subnetwork;

[0074] When the semantic segmentation subnetwork and the edge detection subnetwork extract semantic features and edge features respectively, the two kinds of features are fused. The concat fusion method can completely retain the original information of each feature map involved in fusion, without losing features, so a feature fusion module based on concat fusion is designed to weight and fuse the two kinds of features, as shown in Figure 5 .

[0075] 1) The feature map

[0076] The input feature map of this module is divided into three parts, namely the original image , the edge detection subnetwork output feature map and the semantic segmentation subnetwork output feature map .

[0077] 2) Concatenate the input feature map, semantic feature map, and edge feature map to obtain a fused feature map :

[0078] .

[0079] 3) Perform average pooling and maximum pooling operations on the fused feature map to obtain the global average and maximum values of each feature channel, respectively:

[0080] ;

[0081] ;

[0082] where i represents the index of the feature channel, In semantic segmentation, a feature channel refers to the number of channels in a feature map output by each layer of a convolutional neural network (CNN). Each channel represents a specific feature that is extracted from the input image through convolutional operations. In this step, the feature channel refers to each channel in the fused feature map When performing average pooling and maximum pooling operations, each channel is processed individually to calculate the global average and maximum values of each channel. The feature map is concatenated from the input cloud image, semantic segmentation feature map, and edge detection feature map, with a channel count equal to the sum of the three. Each channel corresponds to a specific feature, which may be an edge, texture, or higher-level feature.

[0083] 4) Assign weights to each feature channel through multiplication to highlight important features and suppress unimportant features, thereby extracting effective feature information. The multiplication formula is as follows:

[0084] ;

[0085] where, represents the feature value of the i-th channel in the weighted fused feature map; represents the feature value of the i-th channel in the feature map; represents the weight coefficient of the i-th channel; represents the number of feature channels. By assigning weights to each feature channel, the result becomes more robust.

[0086] ​​​Experiments are performed on the GF1_WHU dataset, which contains different backgrounds such as natural water, artificial features, water, and mixed ground, covers various typical cloud classes such as thick clouds, thin clouds, and broken clouds, and the like. The experiments show that the model has good segmentation effect on different backgrounds and different cloud classes, and part of the cloud images are as shown in Figure 6

[0087] Although preferred embodiments of the application have been described, those skilled in the art will be able to make additional changes and modifications to the embodiments once they have the basic inventive concept. Therefore, the appended claims are intended to be interpreted as including all the preferred embodiments and all the changes and modifications falling within the scope of the application.

[0088] Obviously, those skilled in the art can make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application also intends to include these modifications and variations.​

Claims

1. A Unet-based cloud and cloud shadow improved semantic segmentation method, characterized in that, The method comprises the following steps: S1, replacing the down-sampling of the Unet model with an improved ASPP spatial pyramid pooling module, while embedding an attention mechanism module in the skip connection, to construct a semantic segmentation sub-network; processing the input cloud image using the semantic segmentation sub-network, and obtaining a semantic segmentation feature map through up-sampling; S2, connecting four convolutional layers in sequence to form four convolutional stages, adding a 1x1 convolutional operation to each convolutional stage, then restoring the feature maps of the last three convolutional stages to the same scale as the input image using a deconvolution operation, and finally fusing the feature maps obtained from the four convolutional stages through a 1x1 convolutional operation to construct an edge detection sub-network; processing the input cloud image using the edge detection sub-network to obtain an edge detection feature map; S3, weighting and fusing the input cloud image, the semantic segmentation feature map, and the edge detection feature map, and then processing the fused features through a Sigmoid activation function layer to output a cloud and cloud shadow segmentation result; The improved ASPP spatial pyramid pooling module comprises a first semantic convolutional layer, a second semantic convolutional layer, a third semantic convolutional layer, a fourth semantic convolutional layer, an average pooling layer, a maximum pooling layer, a semantic feature fusion layer, and a 1x1 convolutional layer; The first semantic convolutional layer, the second semantic convolutional layer, the third semantic convolutional layer, and the fourth semantic convolutional layer respectively extract features from the input cloud image; the maximum pooling layer performs a maximum operation on the local region of the input cloud image to extract local detail features including the edges and textures of the cloud shadow in the input cloud image; and the average pooling layer performs an average operation on the local region of the input cloud image to extract global context features including the overall shape of the cloud and the distribution of the cloud and cloud shadow in the input cloud image; The first semantic convolutional layer uses a 1x1 convolutional kernel to extract preliminary features from the input cloud image; the second semantic convolutional layer uses a 3x3 convolutional kernel with a dilation rate of 6 to capture large-scale context information in the cloud image; the third semantic convolutional layer uses a 3x3 convolutional kernel with a dilation rate of 12 to capture larger-scale context information in the cloud image and extract large-scale features between the cloud and the cloud shadow in the cloud image; the fourth semantic convolutional layer uses a 3x3 convolutional kernel with a dilation rate of 18 to capture the largest-scale context information in the cloud image as the global features between the cloud and the cloud shadow in the cloud image; the semantic feature fusion layer concatenates the feature maps from different convolutional layers and pooling layers to integrate local details and global context information and generate more rich feature representations; and the 1x1 convolutional layer performs further convolutional operations on the feature map fused by the semantic feature fusion layer to adjust the number of channels of the feature map and finally generate an output feature map for subsequent tasks. 2.The Unet-based cloud and cloud shadow improved semantic segmentation method of claim 1, wherein, In step S2, the edge detection sub-network comprises a first edge convolution layer, a second edge convolution layer, a third edge convolution layer, a fourth edge convolution layer and an edge feature fusion layer connected in sequence, and four 1x1 convolution layers connected with the first convolution layer, the second convolution layer, the third convolution layer and the fourth convolution layer, respectively; Each of the first edge convolution layer, the second edge convolution layer, the third edge convolution layer and the fourth edge convolution layer is inserted with a pooling layer, the first edge convolution layer, the second edge convolution layer, the third edge convolution layer and the fourth edge convolution layer sequentially perform feature extraction on the input cloud image to obtain cloud shadow edge features of different scales, and reduce the resolution of the feature map through the respective corresponding pooling layer; The output feature of the first edge convolution layer is obtained after 1x1 convolution operation to obtain a first convolution stage feature map; the output features of the second edge convolution layer, the third edge convolution layer and the fourth edge convolution layer are respectively obtained after 1x1 convolution operation, and then the feature maps of the corresponding convolution stage are restored to the same scale of the input cloud image by using deconvolution operation to obtain a second convolution stage feature map, a third convolution stage feature map and a fourth convolution stage feature map, respectively. The edge feature fusion layer is a 1x1 convolution layer, which fuses the first convolution stage feature map, the second convolution stage feature map, the third convolution stage feature map and the fourth convolution stage feature map to obtain an edge detection feature map containing edge information of cloud shadow regions of different ranges. 3.The Unet-based cloud and cloud shadow improved semantic segmentation method of claim 2, wherein, The first edge convolution layer, the second edge convolution layer, the third edge convolution layer and the fourth edge convolution layer all adopt 3x3 convolution layers, and the channel numbers are 64, 128, 256 and 512, respectively. 4.The Unet-based cloud and cloud shadow refinement semantic segmentation method of claim 1, wherein, Step S3 further comprises: The input cloud image I, the semantic segmentation feature map f out and the edge detection feature map F out are spliced and fused to obtain a fused feature map F fused : F fused = concat(I, F out , f out ); the fused feature map F fused The number of included feature channels is the sum of the channel numbers of the input cloud image I, the semantic segmentation feature map f out , and the edge detection feature map F out . The average pooling and maximum pooling operations are performed on each feature channel of the fused feature map F fused , and the global average value and the maximum value of each feature channel of the feature map F fused are obtained respectively: F Avg (i) = AvgPool(F fused (i)); F Max (i) = MaxPool(F fused (i)); Wherein, i represents the index of the feature channel, i = 1, 2, …, N; Each feature channel is assigned a weight by multiplication calculation to extract effective feature information; the multiplication calculation formula is as follows: where y i represents the feature value of the i-th channel of the weighted fused feature map; x k represents the feature value of the k-th channel of the feature map; ω i represents the weight coefficient of the i-th channel; and N represents the number of feature channels.

Images