A remote sensing image cloud detection method based on spectral feature guidance and space spectrum convolution

Abstract

A remote sensing image cloud detection method based on spectral feature guidance and spatial-spectral convolution includes: generating a multispectral remote sensing image training set and a test set; constructing a remote sensing image cloud detection network based on spectral feature guidance and spatial-spectral convolution, wherein the remote sensing image cloud detection network based on spectral feature guidance and spatial-spectral convolution includes sequentially cascaded encoding and decoding modules, wherein the encoding module includes a shortwave infrared cloud index guidance module and multiple sequentially cascaded spatial-spectral feature fusion modules; training the remote sensing image cloud detection network based on spectral feature guidance and spatial-spectral convolution using the training set; and inputting the test set into the trained remote sensing image cloud detection network for cloud detection. This invention extracts and fuses spectral features and spatial features respectively, effectively extracting and fusing multi-spectral remote sensing data, and achieving decoupling of spatial correlation and channel correlation in remote sensing data.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a cloud detection method for remote sensing images based on spectral feature guidance and spatial-spectral convolution. Background Technology

[0002] Cloud occlusion in remote sensing images is a long-standing and unavoidable problem. The obstruction of ground targets by clouds poses significant challenges to Earth observation tasks (such as land monitoring and disaster early warning). Cloud detection algorithms for remote sensing images are crucial for monitoring cloud cover and subsequent cloud removal, reducing the negative impact of cloud occlusion and improving the utilization rate of remote sensing images. With the advancement of remote sensing satellite technology, multispectral remote sensing data is becoming increasingly readily available, further increasing the demand for cloud detection technology and driving the development of cloud detection algorithms.

[0003] Patent application CN115761484A discloses a cloud detection method and apparatus based on remote sensing images. The method comprises the following steps: First, acquiring raw multispectral data and dividing it into training and testing sets; second, constructing a deep neural network including a convolutional feature extraction layer, a data image embedding layer, a self-attention feature extraction layer, an upsampling layer, and a cloud edge post-processing layer; third, training the constructed deep neural network using the divided training set as input; and fourth, using the trained deep neural network to perform cloud detection on the multispectral remote sensing images.

[0004] Patent application CN 115861833A discloses a remote sensing image cloud detection method with a dual-branch structure. The implementation steps of the method are as follows: First, obtain a processed training sample image set; second, construct a remote sensing image cloud detection network including a dual-branch encoder and decoder; third, train the remote sensing image cloud detection network using the training sample image set; fourth, use the trained remote sensing image cloud detection network to perform cloud detection on the remote sensing image.

[0005] However, existing technologies have the following main drawbacks:

[0006] First, remote sensing images have many spectral bands. Existing methods treat all spectral band data uniformly. However, the remote sensing information contained in different spectral bands varies greatly. Existing methods do not fully analyze and utilize the spectral information.

[0007] Second, clouds in remote sensing images and common bright ground features such as ice and snow have a high degree of similarity. Existing methods cannot distinguish between clouds and ice and snow well, and may detect ice and snow as clouds, resulting in a large number of false detections. Summary of the Invention

[0008] To overcome the shortcomings of the prior art, the present invention aims to provide a cloud detection method for remote sensing images based on spectral feature guidance and spatial-spectral convolution. By designing a shortwave infrared cloud index, the method utilizes the reflectivity differences between clouds and snow in different bands to distinguish between them. Furthermore, the shortwave infrared cloud index, a spectral feature, enhances the differentiation between clouds and snow. Simultaneously, a multi-scale spatial-spectral convolution is designed to acquire multi-scale spatial and spectral information, fully utilizing the spectral band data of the remote sensing image. The present invention extracts and fuses spectral and spatial features separately, effectively extracting and fusing multi-spectral remote sensing data, thus decoupling spatial and channel correlations in the remote sensing data.

[0009] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0010] A cloud detection method for remote sensing images based on spectral feature guidance and spatial-spectral convolution includes the following steps:

[0011] Step 1: Generate a training set and a test set of multispectral remote sensing images;

[0012] Step 2: Construct a remote sensing image cloud detection network based on spectral feature guidance and spatial-spectral convolution. The remote sensing image cloud detection network based on spectral feature guidance and spatial-spectral convolution includes a sequentially cascaded encoding module and a decoding module. The encoding module includes a shortwave infrared cloud index guidance module and a series of sequentially cascaded spatial-spectral feature fusion modules.

[0013] Step 3: Use the training set generated in Step 1 to train the remote sensing image cloud detection network based on spectral feature guidance and spatial-spectral convolution constructed in Step 2.

[0014] Step 4: Input the test set generated in Step 1 into the remote sensing image cloud detection network trained in Step 3 to perform cloud detection.

[0015] The specific method for step 1 is as follows:

[0016] Step 1.1: Select at least 100 multispectral remote sensing images with a resolution of A*A to form a sample set;

[0017] Step 1.2: Randomly select 70%-90% of the images in the sample set of Step 1.1 as the training set, and the remaining 10%-30% as the test set.

[0018] The specific method for step 2 is as follows:

[0019] Step 2.1: Build a shortwave infrared cloud index guidance module, whose structure is as follows: cloud index calculation layer, cloud index feature calculation layer, and feature fusion layer;

[0020] The cloud index calculation layer calculates the cloud index according to the following formula:

[0021]

[0022] Among them, SI x，y This represents the cloud index value of the pixel in row x and column y. This represents the reflectance value of the pixel in the x-th row and y-th column at the 2.2-micron wavelength. This represents the reflectance value of the pixel in the x-th row and y-th column at the 1.4-micron wavelength. This represents the reflectance value of the pixel in the x-th row and y-th column at 1.6 micrometer wavelength, where min indicates the minimum value.

[0023] Step 2.2, build the spatial-spectral feature fusion module;

[0024] Construct M spatial spectral feature fusion modules with identical structures, where M≥2, and their structures are: first spatial convolutional layer, second spatial convolutional layer, third spatial convolutional layer, first inter-spectral convolutional layer, second inter-spectral convolutional layer, third inter-spectral convolutional layer, and fourth inter-spectral convolutional layer.

[0025] The convolution dilation rate of the first spatial convolutional layer is set to 1, the convolution dilation rate of the second spatial convolutional layer is set to 3, the convolution dilation rate of the third spatial convolutional layer is set to 5, and the kernel size of the first to fourth interspectral convolutional layers is set to 1×1.

[0026] Step 2.3: The shortwave infrared cloud index guidance module built in Step 2.1 and the M spatial spectral feature fusion modules built in Step 2.2 are cascaded sequentially to form an encoding module;

[0027] Step 2.4: Construct the decoding module, whose structure consists of a first convolutional layer, a second convolutional layer, a third convolutional layer, and a fourth convolutional layer; the kernel size of the first to fourth convolutional layers is set to 3×3.

[0028] Step 2.5: The encoding module built in Step 2.3 and the decoding module built in Step 2.4 are cascaded sequentially to form a remote sensing image cloud detection network based on spectral feature guidance and spatial-spectral convolution.

[0029] The specific method for step 3 is as follows:

[0030] The binary images in the training set generated in step 1 are input into the remote sensing image cloud detection network based on spectral feature guidance and spatial spectral convolution constructed in step 2. The parameters of each layer of the cloud detection network are iteratively updated using the backpropagation gradient descent method until the loss function of the cloud detection network converges, thus obtaining the trained remote sensing image cloud detection network.

[0031] The specific method for step 4 is as follows:

[0032] Step 4.1: Input the test set from Step 1.2 into the remote sensing image cloud detection network trained in Step 3, and output the predicted probability map corresponding to yuqi2;

[0033] Step 4.2: Divide the predicted probability map output in Step 4.1 into thresholds of 0.5. Mark pixels with a predicted probability greater than or equal to 0.5 as cloud pixels and pixels with a predicted probability less than 0.5 as non-cloud pixels, thus completing the cloud detection of the remote sensing image.

[0034] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0035] 1. This invention incorporates a shortwave infrared cloud index guidance module, which can distinguish between clouds and snow by utilizing the reflectivity differences between clouds and snow in different bands. The guidance module enhances the network's extraction and learning of the differential features between clouds and snow, overcoming the problem in existing technologies where it is difficult to distinguish between clouds and snow, resulting in the detection of large areas of snow and ice as cloud areas. This allows the invention to maintain good performance even when performing cloud detection on remote sensing images containing a large amount of snow and ice.

[0036] 2. This invention, by designing a multi-scale spatial-spectral feature fusion module, can extract and fuse multi-scale spatial features and inter-spectral features, thereby making better use of the spectral band information of multispectral remote sensing images. This overcomes the problem of treating spectral band data uniformly in the prior art, enabling this invention to make full use of spatial and spectral information, thereby improving the performance of cloud detection.

[0037] In summary, by designing a shortwave infrared cloud index guidance module and a spatial-spectral feature fusion module, this invention enables the network to learn the reflection features of spectral bands to achieve more refined segmentation and separate spatial feature extraction from spectral feature extraction. It also decouples the cross-channel correlation and spatial correlation mapping of feature maps, and features comprehensive detection and full utilization of remote sensing image data rich in spectral data. Attached Figure Description

[0038] Figure 1 This is a flowchart of the present invention.

[0039] Figure 2 It is the atmospheric apparent reflectance of common land cover types.

[0040] Figure 3 This is a schematic diagram of the cloud index feature guidance module of the present invention.

[0041] Figure 4 This is a structural diagram of the spatial spectrum feature fusion module of the present invention.

[0042] Figure 5 This is a diagram of the encoding and decoding feature fusion module of the present invention.

[0043] Figure 6 This is the overall network structure diagram of the algorithm of this invention.

[0044] Figure 7 This is a schematic diagram of the remote sensing image cloud detection network structure of the present invention. Detailed Implementation

[0045] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0046] This invention can be used to segment and remove cloud-containing portions of remote sensing images.

[0047] like Figure 1 As shown, a cloud detection method for remote sensing images based on spectral feature guidance and spatial-spectral convolution includes the following steps:

[0048] Step 1: Generate a training set and a test set of multispectral remote sensing images;

[0049] Step 1.1: Select at least 100 multispectral remote sensing images with a resolution of 384×384 to form a sample set;

[0050] Step 1.2: Randomly select 80% of the images from the sample set in Step 1.1 as the training set, and the remaining 20% ​​as the test set.

[0051] Step 2: Construct a remote sensing image cloud detection network based on spectral feature guidance and spatial-spectral convolution. The remote sensing image cloud detection network based on spectral feature guidance and spatial-spectral convolution includes a sequentially cascaded encoding module and a decoding module. The encoding module includes a shortwave infrared cloud index guidance module and a series of sequentially cascaded spatial-spectral feature fusion modules.

[0052] Shortwave infrared cloud index guidance module

[0053] like Figure 2 As shown, different types of land features (such as snow, desert, and vegetation) and clouds exhibit significant differences in atmospheric apparent reflectance across different spectral bands. This difference in reflectance can enhance the differentiation of visually difficult-to-distinguish pixels (such as snow and clouds). The atmospheric apparent reflectance of common land feature types is shown below. Figure 2 As shown in the figure, the reflectance differences between common ground features and clouds vary significantly across different wavelengths. Ice / snow and clouds exhibit similar reflectance transformation characteristics with relatively small differences in the wavelength range below 1.43 micrometers. Therefore, it is difficult to distinguish ice / snow from clouds using only spectral information below 1.43 micrometers. However, in the wavelength range above 1.43 micrometers, the reflectance of ice / snow decreases considerably, while clouds retain relatively high reflectance. Therefore, enhancing the spectral information above 1.43 micrometers can help differentiate clouds from ice / snow, avoiding missed detections and false detections.

[0054] Observation of reflectance spectra revealed that in the SWIR-Cirrus band at a wavelength of 1.4 μm, the reflectance values ​​for all categories were low and similar, while in the SWIR bands at 1.6 and 2.2 μm, the cloud pixel values ​​were approximately 0.2 higher than those for other categories. Therefore, similar to the Normalized Difference Vegetation Index (NDDI), this invention proposes a Shortwave Infrared Cloud Index (SWIR-Index) based on spectral reflectance characteristics, as shown in Equation 1, and uses this index for prior guidance of spectral features.

[0055] SWIR_Index=SWIR1.6+SWIR2.2SWIRCirrus (Formula 1)

[0056] Among them, S WIR 1.6 SWIR represents the apparent atmospheric reflectance in the 1.6 micrometer SWIR band. 2.2 SWIR represents the atmospheric apparent reflectance in the SWIR band at 2.2 micrometers. Cirrus This represents the atmospheric apparent reflectance in the SWIR-Cirrus band at 1.4 micrometers.

[0057] By calculating the shortwave infrared cloud index value for each pixel in the remote sensing image space, a shortwave infrared cloud index feature map of the image can be obtained. Thresholding of this feature map can also achieve coarse cloud segmentation. However, the thresholding method is not precise enough and has poor results. Therefore, this invention combines the prior knowledge of the shortwave infrared cloud index and designs a feature guidance module. This module uses the shortwave infrared cloud index features to guide the network's feature learning, allowing the network to learn the reflectance characteristics of the spectral bands, thereby achieving more precise segmentation.

[0058] like Figure 3 As shown, this invention uses an attention mechanism to design the cloud index feature guidance module. The attention mechanism is a highly effective feature guidance method. The structure of the cloud index feature guidance module is as follows: Figure 3 As shown, the shortwave infrared cloud index is first calculated on the original multispectral remote sensing image, resulting in a shortwave infrared cloud index feature map. Then, the feature matrix is ​​transposed and multiplied by itself to calculate the autocorrelation matrix of the shortwave infrared cloud index feature matrix. The autocorrelation matrix contains the correlation of each pixel with respect to the global context, which can be viewed as the importance weight of the shortwave infrared cloud index feature map at spatial pixels. This importance weight is then multiplied point-by-point with the input features to guide the input features. Finally, two 3×3 convolutions are used for feature learning, integrating the shortwave infrared cloud index spectral feature into the network.

[0059] Step 2.1: Build a shortwave infrared cloud index guidance module, whose structure is as follows: cloud index calculation layer, cloud index feature calculation layer, and feature fusion layer;

[0060] The cloud index calculation layer calculates the cloud index according to the following formula:

[0061]

[0062] Among them, SI x，y This represents the cloud index value of the pixel in row x and column y. This represents the reflectance value of the pixel in the x-th row and y-th column at the 2.2-micron wavelength. This represents the reflectance value of the pixel in the x-th row and y-th column at the 1.4-micron wavelength. This represents the reflectance value of the pixel in the x-th row and y-th column at 1.6 micrometer wavelength, where min indicates the minimum value.

[0063] Spatial Spectral Feature Fusion Module

[0064] Convolutional neural networks (CNNs) perform local perception through the kernels of convolutional blocks, starting with local perception and ultimately combining these local parts to form the overall picture of the observed object. This process involves information exchange in both spatial and channel dimensions. However, unlike the three-channel images of natural images, remote sensing images have a large number of spectral bands with significant differences, and simply using convolutional blocks would introduce a lot of redundant information. Moreover, cross-channel correlations in remote sensing images are also important, and the mapping of spatial features and spectral features in the network should be separated. If ordinary 2D convolution is still used, the mapping of spatial and spectral features will be confused, and certain interference information will be introduced. Based on this, this invention designs a spatial-spectral feature fusion module to separate spatial feature extraction and spectral feature extraction, thereby decoupling the cross-channel correlation and spatial correlation mapping of feature maps. Specifically, the convolution method is changed, treating each channel of the feature map as an independent group, and performing spatial dimension convolution on each channel separately for feature extraction.

[0065] like Figure 4The diagram shows the structure of the spatial-spectral feature fusion module. First, the input feature map is grouped by channel, with each channel forming a separate group. Within each group, convolutions with dilation rates of {1, 3, 5} and a kernel size of 3×3 are performed. In this process, only spatial information of a single channel is extracted, without any cross-channel information interaction. Using multiple dilated convolutions in parallel allows for the extraction of information across multiple spatial scales without increasing the number of network parameters. After extracting spatial information, channel-dimensional convolutions are used to fuse inter-spectral information. The outputs of dilated convolutions with different dilation rates can be considered as semantic features at different spatial scales. During inter-spectral information fusion, these semantic features at different spatial scales are fused step-by-step through addition. Then, 1×1 convolutions are used to avoid spatial information interaction and only perform inter-spectral information fusion. This step-by-step addition method allows the receptive field of the feature map to gradually increase from small to large without adding additional spatial information. At the final output, each level is added again, followed by spatial and channel-dimensional convolutions to obtain the final module output. The spatial-spectral feature fusion module fuses and separates spatial and spectral information, which can avoid interference from spectral information when extracting spatial information, thereby making better use of spatial features under different channels.

[0066] Step 2.2, build the spatial-spectral feature fusion module;

[0067] Construct M spatial spectral feature fusion modules with identical structures, where M≥2, and their structures are: first spatial convolutional layer, second spatial convolutional layer, third spatial convolutional layer, first inter-spectral convolutional layer, second inter-spectral convolutional layer, third inter-spectral convolutional layer, and fourth inter-spectral convolutional layer.

[0068] The convolution dilation rate of the first spatial convolutional layer is set to 1, the convolution dilation rate of the second spatial convolutional layer is set to 3, the convolution dilation rate of the third spatial convolutional layer is set to 5, and the kernel size of the first to fourth interspectral convolutional layers is set to 1×1.

[0069] Step 2.3: The shortwave infrared cloud index guidance module built in Step 2.1 and the M spatial spectral feature fusion modules built in Step 2.2 are cascaded sequentially to form an encoding module;

[0070] Step 2.4: Construct the decoding module, whose structure consists of a first convolutional layer, a second convolutional layer, a third convolutional layer, and a fourth convolutional layer; the kernel size of the first to fourth convolutional layers is set to 3×3.

[0071] Step 2.5: The encoding module built in Step 2.3 and the decoding module built in Step 2.4 are cascaded sequentially to form a remote sensing image cloud detection network based on spectral feature guidance and spatial-spectral convolution.

[0072] Encoding and decoding feature fusion module

[0073] Different feature layers contain different information distributions. Shallow features contain rich spatial structure information, but their scale is small and their global contextual semantic information is relatively weak. Deep features contain rich semantic information and can effectively locate salient features, but their scale is large and they lack spatial detail information. End-to-end semantic segmentation networks extract deep semantic feature information through top-down convolutional layers, and then use bottom-up convolutional decoding modules for image restoration and segmentation. At the decoding end, the layer-by-layer convolutional upsampling operations cause the decoding process to lose guidance from high-level semantic information, leading to a degraded network performance. Therefore, a common operation is to skip the feature map at the encoding end to the decoding end to supplement semantic information, such as UNet (RONNEBERGER O, FISCHER P, BROX TU-net: Convolutional networks for biomedical image segmentation[C] / / Medical ImageComputing and Computer-Assisted Intervention–MICCAI 2015:18th InternationalConference,Munich,Germany,October 5-9,2015,Proceedings,Part III 18.Springer,2015:234-241.) and RS-Net (XIE F,SHI M,SHI Z,et al.Multilevel cloud detectioninremote sensing images based on deep learning[J].IEEE Journal of SelectedTopics in Applied Earth Observations and Remote Sensing,2017,10(8):3631-3640.) and other networks. While simply stitching the feature map from the encoding end to the decoding end can make up for the missing semantic information, it also introduces other redundant information.

[0074] To compensate for missing semantic information while avoiding the introduction of redundant information, this invention designs an encoding-decoding feature fusion module. This module does not simply concatenate the feature maps from the encoding end; instead, it weights and fuses the feature maps from the encoding and decoding ends. Through network learning, weights are automatically calculated, and the features from the encoding end are fused with those from the decoding end according to specific weights. The structure of the encoding-decoding feature fusion module is as follows: Figure 5 As shown.

[0075] When fusing feature maps from the encoding and decoding ends, a certain weighting is required because the two feature maps have different impacts on the decoding end. This weighted feature map is... Figure 5 The weight λ is related to the two input feature maps and is learned through network training. Therefore, when calculating the weight λ, the two input feature maps are used to learn the weights. First, the two input feature maps are added together, at which point it can be assumed that the weights of the two feature maps have the same influence. Then, spatial and channel weights are extracted from the added feature map, and the importance weights of each channel dimension and spatial dimension are obtained through convolution learning. Finally, the spatial weights and channel weights are weighted by matrix multiplication, and after batch normalization and sigmoid activation, the weight of each pixel is obtained, which is the required weight feature map λ. The final output calculation of the module is shown in Equation 2:

[0076]

[0077] Where X represents the feature map from the encoder, Y represents the feature map from the decoder, λ represents the weighted feature map generated through self-learning, and Z represents the final output of the module. This indicates that corresponding pixels are multiplied. The weights of the feature maps at the encoding and decoding ends are complementary, and the final fused output feature map is obtained through comprehensive calculation.

[0078] Step 3: Use the training set generated in Step 1 to train the remote sensing image cloud detection network based on spectral feature guidance and spatial-spectral convolution constructed in Step 2.

[0079] The specific method for step 3 is as follows:

[0080] The binary images in the training set generated in step 1 are input into the remote sensing image cloud detection network based on spectral feature guidance and spatial spectral convolution constructed in step 2. The parameters of each layer of the cloud detection network are iteratively updated using the backpropagation gradient descent method until the loss function of the cloud detection network converges, thus obtaining the trained remote sensing image cloud detection network.

[0081] Step 4: Input the test set generated in Step 1 into the remote sensing image cloud detection network trained in Step 3 to perform cloud detection.

[0082] Step 4.1: Input the test set from Step 1.2 into the remote sensing image cloud detection network trained in Step 3, and output the predicted probability map corresponding to yuqi2;

[0083] Step 4.2: Divide the predicted probability map output in Step 4.1 into thresholds of 0.5. Mark pixels with a predicted probability greater than or equal to 0.5 as cloud pixels and pixels with a predicted probability less than 0.5 as non-cloud pixels, thus completing the cloud detection of the remote sensing image.

[0084] Overall network structure

[0085] like Figure 6 The diagram shows the overall structure of the model in this invention. The overall network framework remains an encoding-decoding structure. The encoder and decoder are symmetrical encoding-decoding structures. To obtain multi-scale information, a downsampling operation is performed after each encoding block, and the downsampled block is then used for encoding. The encoding network has a total of four layers and four downsampling operations, ultimately obtaining the encoded feature map. During the decoding process, the feature map is upsampled level by level, eventually restoring the spatial size of the image to the same size as the input. Finally, convolution and the sigmoid activation function are used to obtain the final prediction probability, which is then thresholded to obtain the final cloud detection result.

[0086] Spectral band information can help distinguish pixels in remote sensing images that are subjectively visually very similar, such as clouds and snow, because these objects have similar characteristics in the visible light band but significant differences in other bands. To fully extract and utilize these differences, this invention proposes a shortwave infrared cloud index and designs a feature-guided module. To fully utilize the spectral features of remote sensing images, such as... Figure 6 As shown, in the final overall network structure, a shortwave infrared cloud index guidance module is added to the first stage of the encoding end, using spectral band information to guide shallow feature extraction. During the encoding stage, spectral information that can distinguish between clouds and bright ground features such as snow and ice is used for guidance learning, enabling the network to pay more attention to spectral features and reducing false detections of snow and ice areas.

[0087] For multispectral remote sensing images, spectral band information is also crucial, highlighting the characteristics of ground cover types in different bands. In existing end-to-end semantic segmentation networks, since most processed images are three-channel natural images, they typically use ordinary convolutional modules for spatial and channel feature extraction, integrating spatial and channel extraction. However, this invention focuses on multispectral remote sensing images with a large number of spectral bands. To achieve decoupling and fusion of spatial and spectral information extraction, this invention proposes a spatial-spectral feature fusion module, realizing the fusion of multi-scale spatial feature extraction and spectral band information. To enhance the network's learning of spatial and spectral features, such as... Figure 6As shown, this invention uses a spatial-spectral feature fusion module as the encoding block of the encoder, fully extracting spatial-spectral characteristics during the encoding stage. This allows the encoder to pay more attention to joint spatial-spectral features, not only focusing on features that can distinguish between clouds and non-clouds spatially, but also enhancing spectral bands that can reflect the differences between clouds and ground features.

[0088] The decoding process of the encoder-decoder network has a significant impact on the final detection result. The encoder performs deep encoding on the feature map to extract deep semantic information, and the decoder interprets this information. During interpretation, it also relies on information supplemented by the encoder; therefore, the encoder-decoder structure usually contains skip connections to use encoder features to supplement information during the decoding process. This invention proposes a decoding fusion module with self-learning weights. By automatically learning and extracting the fusion weights of the feature maps at both the encoder and decoder ends, the network can automatically select the proportion of features from both ends in the information interpretation process. This improves the autonomy of information interpretation, enhances decoding efficiency and effectiveness, and improves overall network performance.

[0089] This invention proposes a shortwave infrared cloud index based on the difference in atmospheric apparent reflectance between clouds and ground objects in remote sensing images. A feature guidance module is designed using this index, employing spectral features to guide the network's feature learning, making the network focus more on the spectral band characteristics that can distinguish clouds from ground objects, especially clouds and snow. This invention also designs a spatial-spectral feature fusion module and an encoding-decoding feature fusion module. The spatial-spectral feature fusion module fuses the multi-scale spatial and inter-spectral characteristics of the feature maps, while self-learning weights are used to fuse the feature maps from the encoding and decoding ends. A weighted approach is used to compensate for information loss during the decoding process. This invention effectively improves cloud detection performance, especially demonstrating excellent performance in snow and ice scenarios, and effectively reduces false detections.

[0090] like Figure 7 As shown, the requirements can be met even without removing the shortwave infrared cloud index guidance module and spatial-spectral convolution, but the performance of cloud detection will be significantly reduced. This is because after the shortwave infrared cloud index guidance module and spatial-spectral convolution, the spectral difference features and multi-scale spatial features of the image are extracted more fully, and removing these two modules will lead to insufficient feature extraction.

Claims

1. A method for cloud detection of remote sensing images based on spectral feature guided and spatial-spectral convolution, characterized in that: Includes the following steps: Step 1: Generate a training set and a test set of multispectral remote sensing images; Step 2: Construct a remote sensing image cloud detection network based on spectral feature guidance and spatial-spectral convolution. The remote sensing image cloud detection network based on spectral feature guidance and spatial-spectral convolution includes a sequentially cascaded encoding module and a decoding module. The encoding module includes a shortwave infrared cloud index guidance module and a series of sequentially cascaded spatial-spectral feature fusion modules. The specific method for step 2 is as follows: Step 2.1: Build a shortwave infrared cloud index guidance module, whose structure is as follows: cloud index calculation layer, cloud index feature calculation layer, and feature fusion layer; The cloud index calculation layer calculates the cloud index according to the following formula: wherein, SI x，y represents the cloud index value of the pixel point in the xth row and yth column, represents the reflectivity value of the pixel point in the xth row and yth column at the 2.2-micron wave band, represents the reflectivity value of the pixel point in the xth row and yth column at the 1.4-micron wave band, represents the reflectivity value of the pixel point in the xth row and yth column at the 1.6-micron wave band, and min represents taking the minimum value. Step 2.2, build the spatial-spectral feature fusion module; Construct M spatial spectral feature fusion modules with identical structures, where M≥2, and their structures are: first spatial convolutional layer, second spatial convolutional layer, third spatial convolutional layer, first inter-spectral convolutional layer, second inter-spectral convolutional layer, third inter-spectral convolutional layer, and fourth inter-spectral convolutional layer. The convolution dilation rate of the first spatial convolutional layer is set to 1, the convolution dilation rate of the second spatial convolutional layer is set to 3, the convolution dilation rate of the third spatial convolutional layer is set to 5, and the kernel size of the first to fourth interspectral convolutional layers is set to 1×1. Step 2.3: The shortwave infrared cloud index guidance module built in Step 2.1 and the M spatial spectral feature fusion modules built in Step 2.2 are cascaded sequentially to form an encoding module; Step 2.4: Construct the decoding module, whose structure consists of a first convolutional layer, a second convolutional layer, a third convolutional layer, and a fourth convolutional layer; the kernel size of the first to fourth convolutional layers is set to 3×3. Step 2.5, the feature maps of the encoding module and the feature maps of the decoding module are fused by weighting, the weights are automatically calculated through learning of the network, the features of the encoding module are fused with the decoding module according to a certain weight, when the feature maps (X) from the encoding module and the feature maps (Y) from the decoding module are fused, a certain weight is used, a weight feature map (lambda) related to the input feature map is obtained through network training and learning, the calculation process is as follows: firstly, the two input feature maps are added, then the spatial and channel weights of the added feature maps are extracted respectively, the importance weights of each channel dimension and spatial dimension are obtained through convolution learning, finally, the spatial weight and the channel weight are weighted in the form of matrix multiplication, after batch normalization and Sigmoid function activation, the weight of each pixel point is obtained, that is, the required weight feature map lambda; the final output feature (Z) of the module is obtained through the formula is calculated, wherein the corresponding pixels are multiplied, through the weight complement of the feature map of the encoding module and the feature map of the decoding module, the fused output feature map is finally calculated and integrated, and the remote sensing image cloud detection network based on spectral feature guidance and space spectrum convolution is formed. Step 3: Use the training set generated in Step 1 to train the remote sensing image cloud detection network based on spectral feature guidance and spatial-spectral convolution constructed in Step 2. Step 4: Input the test set generated in Step 1 into the remote sensing image cloud detection network trained in Step 3 to perform cloud detection.

2. The remote sensing image cloud detection method based on spectral feature guidance and hyperspectral convolution according to claim 1, characterized in that: The specific method for step 1 is as follows: Step 1.1: Select at least 100 multispectral remote sensing images with a resolution of A*A to form a sample set; Step 1.2: Randomly select 70%-90% of the images in the sample set of Step 1.1 as the training set, and the remaining 10%-30% as the test set.

3. The remote sensing image cloud detection method based on spectral feature guidance and spatial-spectral convolution according to claim 1, characterized in that: The specific method for step 3 is as follows: The binary images in the training set generated in step 1 are input into the remote sensing image cloud detection network based on spectral feature guidance and spatial spectral convolution constructed in step 2. The parameters of each layer of the cloud detection network are iteratively updated using the backpropagation gradient descent method until the loss function of the cloud detection network converges, thus obtaining the trained remote sensing image cloud detection network.

4. The remote sensing image cloud detection method based on spectral feature guidance and spatial-spectral convolution according to claim 1, characterized in that: The specific method for step 4 is as follows: Step 4.1: Input the test set from Step 1.2 into the remote sensing image cloud detection network trained in Step 3, and output the predicted probability map corresponding to yuqi2; Step 4.2: Divide the predicted probability map output in Step 4.1 into thresholds of 0.

5. Mark pixels with a predicted probability greater than or equal to 0.5 as cloud pixels and pixels with a predicted probability less than 0.5 as non-cloud pixels, thus completing the cloud detection of the remote sensing image.

Images