Hyperspectral remote sensing image cloud detection method based on space-spectrum dual-coding U-shaped network

Abstract

The invention provides a hyperspectral remote sensing image cloud detection method based on a space-spectrum dual-coding U-shaped network, and relates to the technical field of remote sensing image processing, and the method comprises the steps: collecting a hyperspectral image, carrying out the preprocessing and manual marking, and obtaining a cloud detection data set; constructing a hyperspectral cloud detection model based on the space-spectrum dual-coding U-shaped network; training a hyperspectral cloud detection model through the marked cloud detection data set; and obtaining a to-be-detected hyperspectral image and inputting the to-be-detected hyperspectral image into the trained hyperspectral cloud detection model to obtain a cloud detection result. Through the hyperspectral cloud detection model, the recognition accuracy and efficiency of the hyperspectral image are improved, and the problem that a current cloud detection method only pays attention to spatial information features and ignores spectral discrimination information extraction is solved.

Description

Technical Field

[0001] This application relates to the technical field of remote sensing image processing, and in particular to a hyperspectral remote sensing image cloud detection method based on an air-spectrum dual-coded U-shaped network. Background Technique

[0002] In recent years, hyperspectral remote sensing has developed rapidly, providing new data sources for industrial applications. Hyperspectral data has typical characteristics of integrating spectrum and image, and continuous spectrum. Each pixel corresponds to an almost continuous spectral curve, which can obtain fine diagnostic features of ground objects and greatly promotes the rapid development of remote sensing applications.

[0003] Like traditional optical satellites, hyperspectral satellites are inevitably affected by cloud occlusion during the imaging process. The presence of clouds will cover some ground objects, resulting in a decline in image quality, a reduction in the utilization rate of remote sensing data, and a serious hindrance to remote sensing earth observation. Therefore, in the application of remote sensing images, cloud detection, as a key step in image data preprocessing, its detection efficiency and accuracy have an important impact on remote sensing image interpretation and industrial applications.

[0004] Hyperspectral remote sensing covers multiple electromagnetic wave bands such as ultraviolet, visible light, near-infrared, short-wave infrared, and mid-infrared, effectively realizing the integration of spectrum and image. It has the characteristics of high spectral resolution, wide spectral range, and multiple imaging bands, and has great advantages in various industrial fields. However, the complex internal structure and rich spatial and spectral information of hyperspectral data bring greater challenges to cloud detection research.

[0005] The cloud detection task mainly uses the spectral, frequency, and spatial texture characteristics of clouds to distinguish clear sky pixels and cloud pixels in remote sensing images through certain technical means, which can be regarded as a binary segmentation task. The cloud detection method based on physical rules analyzes the reflectance differences of clouds in visible light, short-wave infrared, and thermal infrared bands and the spectral change rules of multiple bands, and sets thresholds as judgment criteria to distinguish clouds and cloud-free surfaces. This method is relatively simple and easy to implement, has high interpretability, and performs well under specific environmental conditions. However, its detection effect is limited by the sensor model and threshold parameters, and it is unstable and has poor universality in complex environments.

[0006] With the rapid development of machine learning theory, classical machine learning algorithms such as support vector machines, artificial neural networks, and random forests have gradually been introduced into the field of remote sensing image cloud detection. The cloud detection method based on shallow machine learning mines the spatial patterns of images, better extracts discriminative features, constructs a classifier, and distinguishes clouds from the background. This method improves the utilization rate of remote sensing information and effectively improves the cloud detection accuracy. However, it cannot mine the deep semantic information of images, is difficult to adapt to complex cloud changes, usually has limited detection accuracy, high uncertainty, and is difficult to adapt to complex scenes. Summary of the Invention

[0007] The object of the present invention is to provide a hyperspectral remote sensing image cloud detection method based on a spatial-spectral dual-encoding U-shaped network to solve the problems of limited detection accuracy, high uncertainty, and difficulty in adapting to complex scenarios existing in existing hyperspectral cloud detection methods.

[0008] The above object of the present application is achieved through the following technical solutions:

[0009] S1: Collect hyperspectral images, perform preprocessing and manual annotation to obtain a cloud detection data set;

[0010] S2: Construct a hyperspectral cloud detection model based on a spatial-spectral dual-encoding U-shaped network;

[0011] S3: Train the hyperspectral cloud detection model with the labeled cloud detection data set;

[0012] S4: Obtain the hyperspectral image to be detected and input it into the trained hyperspectral cloud detection model to obtain the cloud detection result.

[0013] Optionally, step S1 includes:

[0014] S11: Collect hyperspectral images with different cloud coverages, different underlying surface types, and different cloud types;

[0015] S12: Use ENVI data image processing software to perform preprocessing and manual annotation on the hyperspectral images, and label them as two categories: cloud and non-cloud;

[0016] S13: Select, crop, and block the hyperspectral images and their labels to obtain image blocks with a size of 256×256, and construct a cloud detection data set;

[0017] S14: Perform data augmentation on the cloud detection data set; the data augmentation includes: flipping, rotating, and scaling.

[0018] Optionally, step S2 includes:

[0019] The hyperspectral cloud detection model includes: an encoder and a decoder;

[0020] The encoder includes: a preliminary feature encoding module, a spatial feature encoding branch, a spectral feature encoding branch, and a splicing module;

[0021] The preliminary feature encoding module is a two-dimensional convolutional layer with a size of 3×3;

[0022] The spatial feature encoding branch includes: 4 spatial feature encoding modules and 4 pooling layers with a pooling window size of 2×2;

[0023] The spatial feature encoding module includes: a first spatial encoding branch and a second spatial encoding branch; the first spatial encoding branch and the second spatial encoding branch are connected in parallel; the first spatial encoding branch includes: a first encoding unit and a second encoding unit; the first encoding unit is connected to the second encoding unit;

[0024] The first encoding unit includes: a 3×3 convolutional layer, a batch normalization layer, and a ReLU activation function;

[0025] The second encoding unit includes: a 3×3 convolutional layer, a batch normalization layer, and a ReLU activation function;

[0026] The third encoding unit includes: a 1×1 convolutional layer, a batch normalization layer, and a ReLU activation function;

[0027] The spectral feature encoding branch includes: 4 spectral feature encoding modules and 4 pooling layers with a pooling window size of 2×2;

[0028] The spectral feature encoding module includes: a first spectral encoding branch and a second spectral encoding branch;

[0029] The first spectral encoding branch and the second spectral encoding branch are connected in parallel;

[0030] The first spectral encoding branch includes: a first Reshape unit, a first spectral convolutional layer, a second spectral convolutional layer, a second Reshape unit, and a 1×1 two-dimensional convolution;

[0031] The first Reshape unit, the first spectral convolutional layer, the second spectral convolutional layer, the second Reshape unit, and the 1×1 two-dimensional convolution of the first spectral encoding branch are connected in sequence;

[0032] The first spectral convolutional layer includes: a one-dimensional convolution with a kernel size of 3, a batch normalization unit, and a ReLU activation function unit;

[0033] The second spectral convolutional layer includes: a one-dimensional convolution with a kernel size of 3, a batch normalization unit, and a ReLU activation function unit;

[0034] The output features of the spatial feature encoding branch and the output features of the spectral feature encoding branch are connected to a splicing module;

[0035] The splicing module is used to splice the output feature map of the spatial feature encoding branch and the output feature map of the spectral feature encoding branch to obtain spliced features;

[0036] The decoder includes: a convolutional layer with a residual structure and an upsampling unit;

[0037] The decoder is used to decode the spliced features to obtain the final cloud detection result.

[0038] Optionally, step S3 includes:

[0039] The training of the hyperspectral cloud detection model is guided by a joint loss, and its specific calculation method is as follows:

[0040]

[0041]

[0042] In the formula, and respectively represent the generated cloud mask and the label ground truth. l J , l BCE and l SSIM represent the joint loss, the BCE loss, and the SSIM loss; α is the weight of the BCE loss; μ L , σ L and are the mean and standard deviation of L and respectively, is the covariance of L and ; c1 and c2 are constants; the value range of the cloud mask is 0 to 1, and the larger the value, the greater the probability that the pixel belongs to the cloud.

[0043] Optionally, the Adam optimizer is used to train the hyperspectral cloud detection model.

[0044] Optionally, step S4 includes:

[0045] Input the hyperspectral image to be detected into the trained hyperspectral cloud detection model to obtain a cloud prediction probability image;

[0046] Through a preset cloud probability threshold, perform binary segmentation on the cloud prediction probability image to obtain the cloud detection result, specifically as follows:

[0047] Pixels in the cloud prediction probability image greater than the cloud probability threshold are determined to be clouds, and pixels less than or equal to the cloud probability threshold are determined to be non-clouds, to obtain the final binary cloud detection result.

[0048] An electronic device includes a processor, a memory, a user interface, and a network interface. The memory is used to store instructions. The user interface and the network interface are used to communicate with other devices. The processor is used to execute the instructions stored in the memory so that the electronic device executes a hyperspectral remote sensing image cloud detection method based on an air-spectral dual-coding U-shaped network.

[0049] A computer-readable storage medium stores instructions that, when executed, perform a hyperspectral remote sensing image cloud detection method based on an air-spectrum dual-encoding U-shaped network.

[0050] The beneficial effects brought by the technical solution provided in this application are as follows:

[0051] Collect hyperspectral images and perform preprocessing and annotation to construct a sample data set and a sample label set for hyperspectral image cloud detection; divide the hyperspectral cloud detection data set into a training set and a test set; based on a hyperspectral cloud detection model of an air-spectrum dual-encoding U-shaped network, the dual-encoding branches of the hyperspectral cloud detection model simultaneously extract spatial features and spectral features in the image, thereby improving the discrimination ability of the model; the spatial encoding branch adopts a spatial feature encoding branch based on 2D convolution to effectively capture the spatial information of the image; the spectral encoding branch adopts a spectral feature encoding branch based on 1D convolution, focusing on extracting the spectral information of the image; jointly fuse the spatial and spectral features extracted by the dual-encoding branches, and generate the final cloud detection result after being processed by the decoding module; solve the problem that the current cloud detection method only focuses on spatial information features and ignores the extraction of spectral discrimination information, and realize the generation of a hyperspectral image cloud detection model with high recognition accuracy and efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The following will further illustrate this application in conjunction with the drawings. In the drawings:

[0053] Figure 1 are the figures in the embodiments of this application;

[0054] Figure 2 is the model structure diagram in the embodiments of this application;

[0055] Figure 3 is the schematic diagram of the SEM structure of the spatial feature encoding branch based on 2D convolution in the embodiments of this application;

[0056] Figure 4 is the schematic diagram of the SpEM of the spectral feature encoding branch in the embodiments of this application;

[0057] Figure 5 is the schematic diagram of the electronic device structure in the embodiments of this application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0058] For a clearer understanding of the technical features, objectives, and effects of this application, the specific embodiments of this application will now be described in detail with reference to the drawings.

[0059] The embodiments of this application provide a hyperspectral remote sensing image cloud detection method based on an air-spectrum dual-encoding U-shaped network.

[0060] Please refer toFigure 1 , Figure 1 This is the S diagram of a hyperspectral remote sensing image cloud detection method based on an air-spectrum dual-coded U-shaped network in an embodiment of the present application, including:

[0061] S1: Collect hyperspectral images, perform preprocessing and manual annotation to obtain a cloud detection dataset;

[0062] S2: Construct a hyperspectral cloud detection model based on an air-spectrum dual-coded U-shaped network;

[0063] S3: Train the hyperspectral cloud detection model with the labeled cloud detection dataset;

[0064] Among them, the initial hyperspectral cloud detection model includes: a spatial encoding branch, which uses a spatial feature encoding branch (Spatial Encoding Module, SEM) based on 2D convolution and focuses on extracting the spatial information of the image; a spectral encoding branch, which uses a spectral feature encoding branch (Spectral Encoding Module, SpEM) based on 1D convolution and is used to extract the rich spectral information of the image; the spatial and spectral features extracted by the joint dual branches are fused to effectively improve the discrimination ability of the model, and the spatial resolution is gradually restored through the processing of the decoding module, and the high-dimensional features extracted by the encoder part are decoded into the output image required for cloud detection;

[0065] As an embodiment, after testing with the test set, through the set segmentation threshold, binary threshold segmentation is performed on the cloud probability image, and the image blocks are stitched into a whole-scene cloud mask image, and finally the training results are evaluated by calculating the preset indicators; the preset indicators include: overall accuracy, recall rate, precision rate, F1 index, and Jaccard index.

[0066] S4: Obtain the hyperspectral image to be detected and input it into the trained hyperspectral cloud detection model to obtain the cloud detection result.

[0067] As an embodiment, hyperspectral images are collected and preprocessed and labeled to construct a sample data set and a sample label set for hyperspectral image cloud detection; the hyperspectral cloud detection data set is divided into a training set and a test set; based on the U-Net network, a Spatial-Spectral Dual-Encoding U-Shaped Cloud Detection Network (SSDE-UNet) is constructed, and the training set is used to train the SSDE-UNet; the SSDE-UNet includes: a dual-encoding branch that simultaneously extracts spatial features and spectral features in the image, thereby improving the discrimination ability of the model; the spatial encoding branch uses a Spatial Encoding Module (SEM) based on 2D convolution to effectively capture the spatial information of the image; the spectral encoding branch uses a Spectral Encoding Module (SpEM) based on 1D convolution to focus on extracting the spectral information of the image; the spatial and spectral features extracted by the joint dual-encoding branch are fused, and after being processed by the decoding module, the final cloud detection result is generated; the training result is evaluated according to the test set to obtain a hyperspectral cloud detection model. Through the construction of the SSDE-UNet, the problem that the current cloud detection method only focuses on spatial information features and ignores the extraction of spectral discrimination information is solved, and a hyperspectral image cloud detection model with high recognition accuracy and efficiency is generated.

[0068] Step S1 includes:

[0069] S11: Collect hyperspectral images with different cloud coverages, different underlying surface types, and different cloud types.

[0070] In the embodiment of the present invention, by covering a variety of cloud coverages, a variety of underlying surface types, and cloud types, the sample diversity is increased to ensure the representativeness of the hyperspectral cloud detection data set and the effectiveness of the hyperspectral cloud detection model. Among them, the cloud coverage range is 10% to 80%, the underlying surface types include 6 types: barren, forest, grassland / crop, ice / snow, city, and water area, and the cloud types include large thick clouds, thin clouds with weak transparency and visual effects, and stippled broken clouds, etc.

[0071] S12: Use ENVI data image processing software to preprocess and manually label the hyperspectral images, and label them into two categories: cloud and non-cloud.

[0072] In the embodiment of the present invention, to ensure the representativeness of the data set, hyperspectral images with different cloud amounts, different underlying surface types, and different cloud types are selected, and ENVI data image processing software is used to preprocess the images. The preprocessed image samples are manually labeled to obtain corresponding cloud mask labels, and the cloud detection data set is constructed.

[0073] In the embodiments of the present invention, the sample diversity is increased by covering a variety of cloud coverages, a variety of underlying surface types, and cloud types to ensure the representativeness of the hyperspectral cloud detection dataset and the effectiveness of the hyperspectral cloud detection model. Among them, the cloud coverage ranges from 10% to 80%, the underlying surface types include 6 types: barren, forest, grassland / crop, ice / snow, city, and water area, and the cloud types include large thick clouds, thin clouds with weak transparency and visual effects, and stippled broken clouds, etc.

[0074] S13: Select, crop, and block process the hyperspectral image and its label to obtain image blocks with a size of 256×256, and construct a cloud detection dataset;

[0075] S14: Perform data augmentation on the cloud detection dataset; the data augmentation includes: flipping, rotating, and scaling.

[0076] As an embodiment, a hyperspectral cloud detection dataset is constructed using the preprocessed hyperspectral image, and the corresponding sample label set is obtained through manual annotation. The "cloud" pixels are assigned a value of 255, and the "non-cloud" pixels are assigned a value of 0. Then, the hyperspectral image and label after data augmentation are selected, cropped, and block processed to obtain the final hyperspectral cloud detection dataset. The hyperspectral cloud detection dataset is divided into a training set and a test set, and both the training set and the test set contain different underlying surface types, different cloud types, and different cloud coverages to increase the representativeness of the dataset.

[0077] Step S2 includes:

[0078] The hyperspectral cloud detection model includes: an encoder and a decoder;

[0079] The encoder includes: a preliminary feature encoding module, a spatial feature encoding branch, a spectral feature encoding branch, and a splicing module;

[0080] The preliminary feature encoding module is a two-dimensional convolutional layer with a size of 3×3;

[0081] The spatial feature encoding branch includes: 4 spatial feature encoding modules and 4 pooling layers with a pooling window size of 2×2;

[0082] The spatial feature encoding module includes: a first spatial encoding branch and a second spatial encoding branch; the first spatial encoding branch and the second spatial encoding branch are connected in parallel; the first spatial encoding branch includes: a first encoding unit and a second encoding unit; the first encoding unit is connected to the second encoding unit;

[0083] The first encoding unit includes: a 3×3 convolutional layer, a batch normalization layer, and a ReLU activation function;

[0084] The second encoding unit includes: a 3×3 convolutional layer, a batch normalization layer, and a ReLU activation function;

[0085] The third encoding unit includes: a 1×1 convolutional layer, a batch normalization layer, and a ReLU activation function;

[0086] The spectral feature encoding branch includes: 4 spectral feature encoding modules and 4 pooling layers with a pooling window size of 2×2;

[0087] The spectral feature encoding module includes: a first spectral encoding branch and a second spectral encoding branch;

[0088] The first spectral encoding branch and the second spectral encoding branch are in parallel;

[0089] The first spectral encoding branch includes: a first Reshape unit, a first spectral convolutional layer, a second spectral convolutional layer, a second Reshape unit, and a 1×1 two-dimensional convolution;

[0090] The first Reshape unit, the first spectral convolutional layer, the second spectral convolutional layer, the second Reshape unit, and the 1×1 two-dimensional convolution of the first spectral encoding branch are sequentially connected;

[0091] The first spectral convolutional layer includes: a one-dimensional convolution with a kernel size of 3, a batch normalization unit, and a ReLU activation function unit;

[0092] The second spectral convolutional layer includes: a one-dimensional convolution with a kernel size of 3, a batch normalization unit, and a ReLU activation function unit;

[0093] As an embodiment, as Figure 4 shown, in the spectral encoding branch, each spectral feature extraction convolutional block is mainly composed of two layers of one-dimensional convolution. For the input feature map, first, the Reshape unit operation is performed to complete the transformation of the feature map dimension, and then it is input to the convolutional layer for feature extraction. Each convolutional layer contains a one-dimensional convolution with a kernel size of 3, followed by batch normalization BN and ReLU activation function processing. The feature map obtained after two convolutional operations is converted to three dimensions, and a 1×1 two-dimensional convolution with a kernel size of 1×1 is used to complete the conversion of the number of channels. At the same time, a residual connection is introduced in the convolutional block. The input feature is dimension-expanded using a 1×1 two-dimensional convolution, and the output feature map is added to the feature map after two convolutional operations to obtain the final output result.

[0094] The output feature of the spatial feature encoding branch and the output feature of the spectral feature encoding branch are connected to a splicing module;

[0095] The splicing module is used to splice the output feature maps of the spatial feature encoding branch and the spectral feature encoding branch to obtain the spliced features;

[0096] The decoder includes: a convolutional layer with a residual structure and an upsampling unit;

[0097] The decoder is used to decode the spliced features to obtain the final cloud detection result.

[0098] The present application provides an embodiment as follows. The hyperspectral image in the training set is input into the constructed hyperspectral cloud detection model. The main network architecture is a deep convolutional U-Net architecture, which consists of two components: an encoder and a decoder. The encoding part extracts features from the input image data through a series of convolutional and pooling operations, and converts them into high-level semantic information and inputs it into the decoding part. The decoding part performs upsampling to map the features extracted by the encoding part to the original image space to generate a cloud mask with the same size as the input image patch.

[0099] The present application provides an embodiment as follows. In the encoding structure, the hyperspectral image patch is input into the network. First, it undergoes a 2D convolution with a convolutional kernel size of 3×3 for preliminary feature mapping, and the output feature map of this convolutional layer is respectively input into two encoding branches: spatial and spectral. The spatial encoding branch includes four layers of spatial feature encoding branches (Spatial EncodingModule, SEM) and four pooling layers with a pooling window size of 2×2. 2D convolution is used to extract the spatial information in the hyperspectral image. After each convolutional block and pooling layer, the size of the feature map is reduced to half of the original, while the number of channels of the feature map increases to twice the original. The spectral encoding branch consists of four layers of spectral feature encoding branches (Spectral Encoding Module, SpEM) and four pooling layers with a pooling window size of 2×2. Here, 1D convolution is used to extract the spectral information in the image. Similarly, after each convolutional block and pooling layer, the feature map size is halved and the number of channels is doubled. The spatial and spectral features output by the double encoding branches are jointly spliced and input into the decoding part.

[0100] The present application provides an embodiment as follows. Four upsampling layers are used in the decoder, and each layer consists of a convolution with a residual structure and an upsampling operation. To maintain a low number of parameters, 2D convolution is uniformly used to complete the feature map decoding process. In addition, skip connections are used in the network to propagate the beneficial information in the encoding stage to the decoding stage, so as to more accurately restore the spatial distribution of clouds. Finally, a predicted cloud probability map is obtained through a 1×1 convolution and a Sigmoid function, and a final cloud mask is generated through an appropriate threshold.

[0101] As an example, the design of the spatial-spectral dual-encoding feature extraction branch is mainly based on the structural features of 2D convolution and 1D convolution. Different convolution blocks are stacked to simultaneously extract the spatial features and spectral features in the hyperspectral image, making full use of the complementarity of spatial and spectral information, thereby effectively improving the discriminative ability of the model and the cloud detection performance. The spatial feature encoding branch SEM based on 2D convolution is introduced. By making full use of the spatial information of the image, it can more accurately capture the positional relationships of ground objects in different regions of the image and information such as the edges and shapes of clouds, thereby improving the accuracy and precision of cloud detection.

[0102] As an example, the convolutional layer in the CNN performs operations by sliding the convolution kernel or filter on the input image, effectively capturing the spatial structure information in the image. In the two-dimensional convolutional layer, the convolution kernel is a two-dimensional matrix, which performs pixel-by-pixel multiplication and summation operations with the input image and slides along the spatial dimension direction of the image to obtain the output feature map. The spatial feature encoding branch gradually extracts the image feature information by stacking two convolutional layers and performing residual connections.

[0103] For the input feature map, it first undergoes two convolutional operations. Each convolutional operation uses a two-dimensional convolution kernel with a size of 3×3, and is followed by batch normalization BN and ReLU activation function processing after each two-dimensional convolution. At the same time, to optimize network training, residual connections of the input feature map are introduced in the module. The residual connection uses a two-dimensional convolutional layer with a convolution kernel size of 1×1 to expand the dimension of the input feature map, and adds it to the feature map obtained through two convolutional operations to retain the original information of the input feature map, thereby better utilizing the detailed information of the input features. Finally, the output feature map of this convolutional block is obtained.

[0104] Step S3 includes:

[0105] The training sample data and corresponding sample label data of the hyperspectral remote sensing image are batch-input into the hyperspectral remote sensing image cloud detection network based on the spatial-spectral dual-encoding U-shaped network constructed in step S2. After calculation, the cloud prediction probability of the training sample is output, the network loss function is calculated, and backpropagation is performed.

[0106] The training of the hyperspectral cloud detection model is guided by the joint loss, and its specific calculation method is as follows:

[0107]

[0108] In the formula, and respectively represent the generated cloud mask and the label ground truth, l J , l BCE and l SSIM represent the joint loss, BCE loss, and SSIM loss; α is the weight of the BCE loss; μL , σ L and are the mean and standard deviation of respectively, is the covariance of L and ; c1 and c2 are constants; the cloud mask ranges from 0 to 1, and the larger the value, the greater the probability that the pixel belongs to the cloud.

[0109] The Adam optimizer is used to train the hyperspectral cloud detection model.

[0110] As an example, set the initial learning rate, select a learning rate decay strategy, adjust the training parameters and the optimizer, perform accuracy verification during the training process, evaluate the accuracy of each training of the model using the validation sample dataset, adjust the network model parameters according to the evaluation accuracy and loss value of the validation set, and record the accuracy and loss value of each training of the network model, so as to obtain the best network parameter model.

[0111] Step S4 includes:

[0112] Input the hyperspectral image to be detected into the trained hyperspectral cloud detection model to obtain a cloud prediction probability image;

[0113] Perform binary segmentation on the cloud prediction probability image through a preset cloud probability threshold to obtain a cloud detection result, specifically as follows:

[0114] Pixels in the cloud prediction probability image greater than the cloud probability threshold are determined as clouds, and pixels less than or equal to the cloud probability threshold are determined as non-clouds, obtaining the final binary cloud detection result.

[0115] In the embodiment of the present invention, the cloud segmentation threshold is set to 0.5. In the binary threshold segmentation, pixels with a cloud probability value greater than 0.5 in the test result are segmented as clouds and assigned a value of 1, and pixels less than 0.5 are segmented as non-clouds and assigned a value of 0, finally obtaining a binary cloud mask image. The cloud mask image blocks are spliced in sequence to output the cloud detection result corresponding to the entire scene image.

[0116] Select the hyperspectral cloud detection dataset constructed in step 1 for cloud detection experiments, and select the RS-Net and CloudX-Net models as comparison methods. The quantitative evaluation of their cloud detection performance is shown in the table.

[0117] Methods Precision Recall OA F1-score Jaccard CloudX-Net 86.09 81.61 86.90 80.15 70.17 RS-Net 91.71 76.12 88.13 79.18 69.65 SSDE-UNet 89.82 79.46 88.31 81.33 71.10

[0118] As can be seen from the table, the method proposed in the present invention has achieved the best results on this dataset, and has been significantly improved compared with other methods in most metrics. Among them, the recall rate is 79.46%, the overall precision reaches 88.31%, the F1-Score is 81.33%, and the Jacard is 71.10%. Compared with the sub-optimal method CloudX-Net, there are obvious improvements of 1.18% and 0.93% in F1-Score and Jaccard respectively, which proves the effectiveness and superiority of the method proposed in the present invention.

[0119] This application also discloses an electronic device. Referring to Figure 5 , Figure 5 is a schematic structural diagram of an electronic device disclosed in an embodiment of the present application. The electronic device 500 may include: at least one processor 501, at least one network interface 504, a user interface 503, a memory 505, and at least one communication bus 502.

[0120] Among them, the communication bus 502 is used to realize the connection and communication between these components.

[0121] Among them, the user interface 503 may include a display screen. Optionally, the user interface 503 may further include a standard wired interface and a wireless interface.

[0122] Among them, the network interface 504 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface).

[0123] This application also discloses a computer-readable storage medium. The computer-readable storage medium stores multiple instructions, and the instructions are suitable for being loaded by a processor to execute the above-mentioned method for detecting clouds in hyperspectral remote sensing images based on an air-spectrum dual-coding U-shaped network.

[0124] The above are only exemplary embodiments of the present disclosure, and the scope of the present disclosure cannot be limited by this. That is, any equivalent changes and modifications made in accordance with the teachings of the present disclosure still fall within the scope covered by the present disclosure.

[0125] This application aims to cover any variations, uses, or adaptive changes of the present disclosure. These variations, uses, or adaptive changes follow the general principles of the present disclosure and include common general knowledge or conventional technical means in the technical field not recorded in the present disclosure. The description and embodiments are only regarded as exemplary, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A hyperspectral remote sensing image cloud detection method based on a spatial-spectral dual-coding U-type network, characterized in that: The method comprises the following steps: S1: Collect hyperspectral images, preprocess and manually annotate them to obtain cloud detection dataset; S2: Construct a hyperspectral cloud detection model based on spatial-spectral dual-coding U-type network; S3: Train the hyperspectral cloud detection model using the annotated cloud detection dataset; S4: Obtain the hyperspectral image to be detected and input it into the trained hyperspectral cloud detection model to obtain the cloud detection result.

2. The method for detecting cloud from hyperspectral remote sensing images based on a spatial-spectral dual-coding U-type network as claimed in claim 1, characterized in that: Step S1 includes: S11: Collect hyperspectral images with different cloud coverage, different underlying surface types, and different cloud types; S12: Use ENVI data image processing software to pre-process and manually annotate the hyperspectral images into two categories: cloud and non-cloud; S13: Select, crop and block the hyperspectral images and their labels to obtain image blocks of size 256×256 and construct a cloud detection dataset; S14: performing data enhancement on the cloud detection dataset; the data enhancement includes: flipping, rotating, and scaling.

3. The method for detecting cloud from hyperspectral remote sensing images based on a spatial-spectral dual-coding U-type network as claimed in claim 1, characterized in that: Step S2 includes: The hyperspectral cloud detection model includes: an encoder and a decoder; The encoder includes: a preliminary feature encoding module, a spatial feature encoding branch, a spectral feature encoding branch and a splicing module; The preliminary feature encoding module is a 2D convolutional layer of size 3×3; The spatial feature encoding branch includes: 4 spatial feature encoding modules and 4 pooling layers with a pooling window size of 2×2; The spatial feature coding module includes: a first spatial coding branch and a second spatial coding branch; the first spatial coding branch and the second spatial coding branch are connected in parallel; the first spatial coding branch includes: a first coding unit and a second coding unit; the first coding unit is connected to the second coding unit; The first encoding unit includes: a 3×3 convolutional layer, a batch normalization layer, and a ReLU activation function; The second encoding unit includes: a 3×3 convolutional layer, a batch normalization layer, and a ReLU activation function; The third encoding unit includes: a 1×1 convolution layer, a batch normalization layer and a ReLU activation function; The spectral feature encoding branch includes: 4 spectral feature encoding modules and 4 pooling layers with a pooling window size of 2×2; The spectral feature encoding module includes: a first spectral encoding branch and a second spectral encoding branch; The first spectrum encoding branch is connected in parallel with the second spectrum encoding branch; The first spectral encoding branch includes: a first Reshape unit, a first spectral convolution layer, a second spectral convolution layer, a second Reshape unit and a 1×1 two-dimensional convolution; The first Reshape unit and the first spectral convolution layer of the first spectral encoding branch, The second spectral convolution layer, the second Reshape unit, and the 1×1 two-dimensional convolution are connected in sequence; The first spectral convolution layer includes: a one-dimensional convolution with a convolution kernel size of 3, a batch normalization unit, and a ReLU activation function unit; The second spectral convolution layer includes: a one-dimensional convolution with a convolution kernel size of 3, a batch normalization unit, and a ReLU activation function unit; A module for connecting and splicing the output features of the spatial feature encoding branch and the output features of the spectral feature encoding branch; The splicing module is used to splice the output feature map of the spatial feature encoding branch and the output feature map of the spectral feature encoding branch to obtain a splicing feature; The decoder includes: a convolutional layer with a residual structure and an upsampling unit; The decoder is used to decode the concatenated features to obtain the final cloud detection results.

4. The method for detecting cloud from hyperspectral remote sensing images based on a spatial-spectral dual-coding U-type network as claimed in claim 1, characterized in that: Step S3 includes: The joint loss is used to guide the training of the hyperspectral cloud detection model, and its specific calculation method is: In the formula, and Represent the generated cloud mask and label truth value respectively, l J , l BCE and l SSIM represents the joint loss, BCE loss and SSIM loss; α is the weight of BCE loss; μ L , σ L and They are and The mean and standard deviation of For L and covariance of; c1 and c2 are constants; cloud mask The value range is 0 to 1. The larger the value, the greater the probability that the pixel belongs to the cloud.

5. The method for detecting cloud from hyperspectral remote sensing images based on a spatial-spectral dual-coding U-type network as claimed in claim 1, characterized in that: The Adam optimizer is used to train the hyperspectral cloud detection model.

6. The method for detecting cloud from hyperspectral remote sensing images based on a spatial-spectral dual-coding U-type network as claimed in claim 1, characterized in that: Step S4 includes: The hyperspectral image to be detected is input into the trained hyperspectral cloud detection model to obtain a cloud prediction probability image; The cloud prediction probability image is binarized and segmented using the preset cloud probability threshold to obtain the cloud detection results, as follows: The pixels in the cloud prediction probability image that are greater than the cloud probability threshold are judged as clouds, and the pixels that are less than or equal to the cloud probability threshold are judged as non-clouds, and the final binarized cloud detection result is obtained.

7. An electronic device, characterized in that: It includes a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes the method as described in any one of claims 1-6.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed by a computer, the method according to any one of claims 1 to 6 is executed.

Images