Remote sensing image cloud detection method and device based on deep semantic segmentation

The deep semantic segmentation model addresses the high error rates in FY-2 satellite cloud detection by improving sample imbalance and enhancing cloud boundary extraction, achieving significant accuracy improvements for cloud identification.

CN114092801BActive Publication Date: 2024-12-03NAT SATELLITE METEOROLOGICAL CENT
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202111265605.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-10-28
Publication Date
2024-12-03
Estimated Expiration
2041-10-28

AI Technical Summary

Technical Problem

Existing cloud detection methods for FY-2 satellite data suffer from high error rates and inability to accurately identify broken clouds, leading to uncertainties in time and space scales, which hampers the effective utilization of historical satellite data for weather and climate analysis.

Method used

A remote sensing image cloud detection method based on deep semantic segmentation, utilizing a deep semantic segmentation model with encoding and decoding structures, convolution operations, and an improved loss function to address the imbalance of positive and negative samples, enhancing the extraction of cloud boundaries and contextual information.

Benefits of technology

The method achieves improved cloud detection accuracy, particularly for broken clouds and boundaries, with a kappa coefficient of about 0.53 and overall recognition accuracy enhanced by 6-10% compared to channel threshold methods, effectively extracting cloud boundaries and reducing misclassification.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114092801B_ABST
    Figure CN114092801B_ABST
Patent Text Reader

Abstract

A remote sensing image cloud detection method and device based on deep semantic segmentation, through deviation normalization, maps the remote sensing image pixel values ​​outside the preset spectral channel effective value range to the range of [0,1], and converts the mapped remote sensing image pixel values ​​into image pixel values ​​with a value range of 0 to 255; a deep semantic segmentation model is constructed, and the deep semantic segmentation model includes convolution operation, activation operation, pooling operation, feature fusion, transposed convolution operation and logistic regression operation. The present invention specifically designs a deep semantic segmentation model. In order to aggregate the contextual information of clouds in different regions, the jump connection from high-level features to low-level is increased, and the number of downsampling in the encoding process is increased to ensure that enough feature information is extracted; the loss function is improved to achieve effective weighting of a small number of positive samples, and the cloud boundary can be effectively extracted, especially for the identification of broken clouds and the boundary between clouds and clear sky, more channel threshold methods have significant advantages.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of environmental monitoring, and in particular relates to a remote sensing image cloud detection method and device based on deep semantic segmentation. Background Art

[0002] Fengyun-2 (FY-2) meteorological satellite is my country's first generation of geostationary meteorological satellites. Since the launch of FY-2A in 1997, FY-2H has been successfully launched in 2018. At present, all batches of this series of satellites have been launched, totaling 8 satellites. The remaining 3 are in operation and are planned to be gradually replaced by the new generation of geostationary meteorological satellite Fengyun-4. Relying on the construction of the FY-2 satellite ground application system engineering, my country built a fully functional integrated satellite remote sensing data sharing system in 2004, which has accumulated more than ten years of geostationary meteorological satellite data and provided data services to domestic and foreign users. The FY-2 satellite data has a high observation frequency and a wide coverage, providing valuable historical data for scientific research on climate change, ecological environment change, etc. in my country.

[0003] With the widespread application of deep learning technology in the field of computer vision, image classification, object recognition and semantic segmentation tasks have achieved major breakthroughs. Unlike traditional machine learning, which relies on manual feature extraction, deep learning can automatically extract image features and has the characteristics of transferability, providing new ideas for the reprocessing of FY-2 historical data. In recent years, deep learning technology has begun to be used for cloud detection of satellite remote sensing data with higher resolution and more spectral channels.

[0004] The cloud detection results of FY-2 data are the basis for forming various quantitative remote sensing products. Whether it is weather and climate analysis based on cloud images, or inversion of various atmospheric and surface parameters based on cloud removal, it is necessary to accurately identify clouds in remote sensing images. Related studies have shown that the cloud detection error of FY-2 is large. The cloud detection results of FY-2E and FY-2G are judged as having clouds at a ratio of 44.98% and 40.46% respectively, and broken clouds cannot be shown; the uncertainty of clouds in time and space scales leads to the ever-changing characteristics of satellite cloud images. Therefore, cloud detection has always been the focus and difficulty of meteorological satellite remote sensing research and quantitative application. It is of great significance to improve the level and accuracy of automatic observation of clouds and realize the reprocessing of massive satellite historical data. Summary of the invention

[0005] To this end, the present invention provides a remote sensing image cloud detection method and device based on deep semantic segmentation, which can aggregate the contextual information of clouds in different regions, increase the jump connection from high-level features to low-level features, improve the loss function to address the serious imbalance problem of positive and negative samples in network training, and effectively extract the boundaries of clouds.

[0006] In order to achieve the above object, the present invention provides the following technical solution: a remote sensing image cloud detection method based on deep semantic segmentation, comprising:

[0007] Data preprocessing: Through deviation standardization, the remote sensing image pixel values ​​outside the preset spectral channel valid value range are mapped to the range of [0,1], and the mapped remote sensing image pixel values ​​are converted into image pixel values ​​with a value range of 0 to 255;

[0008] Model building: construct a deep semantic segmentation model, which includes convolution operation, activation operation, pooling operation, feature fusion, transposed convolution operation and logistic regression operation;

[0009] The deep semantic segmentation model has an encoding structure and a decoding structure:

[0010] The encoding structure is composed of a plurality of repeated first convolutional units, wherein the first convolutional unit uses a rectified linear unit activation function and a maximum pooling operation for downsampling;

[0011] In the decoding structure, the feature map is upsampled, a transposed convolution operation is performed by a second convolution unit, and a feature map output by a corresponding layer in the cascade coding structure is convolved by a third convolution unit to map the feature map to a preset size;

[0012] The fourth convolution unit is used to perform convolution operation to map the feature vector into a probability vector for each pixel, and the pixel-level classification result is obtained by selecting the position of the maximum value in the probability vector corresponding to each pixel.

[0013] As a preferred solution of the remote sensing image cloud detection method based on deep semantic segmentation, the first convolution unit includes two repeated 3×3 convolution kernels, and each convolution kernel in the first convolution unit includes a 2×2 maximum pooling operation with a step size of 2 for downsampling;

[0014] During each downsampling process, the number of feature channels is doubled.

[0015] As a preferred solution for the remote sensing image cloud detection method based on deep semantic segmentation, the second convolution unit is a 2×2 convolution kernel, and a transposed convolution operation is performed by the second convolution unit to restore half of the feature space resolution and reduce the number of feature channels by half.

[0016] As a preferred solution for the remote sensing image cloud detection method based on deep semantic segmentation, the third convolution unit uses two 3×3 convolution kernels to perform convolution operations. The convolution kernel in the third convolution unit uses a rectified linear unit activation function to map the feature map to a preset size.

[0017] As a preferred solution for the remote sensing image cloud detection method based on deep semantic segmentation, the fourth convolution unit uses a 1×1 convolution kernel to perform a convolution operation to map the feature vector into a probability vector, and obtains a pixel-level classification result by selecting the position of the maximum value in the probability vector corresponding to each pixel.

[0018] As a preferred solution of the remote sensing image cloud detection method based on deep semantic segmentation, in the convolution operation, the input vector is subjected to convolution and transposed convolution operations inside the encoder and decoder respectively;

[0019] In the feature fusion, the low-level feature vectors are concatenated with the corresponding high-level feature vectors into a new tensor, and the high-level semantic information and the low-level local feature information are fused.

[0020] As a preferred solution of the remote sensing image cloud detection method based on deep semantic segmentation, the transposed convolution operation includes the following steps:

[0021] Given a step length s, add s-1 zeros after each element in the input feature map along the direction of the step length to obtain the expanded feature map x;

[0022] According to the convolution rule, calculate the position L and number of padding 0s, reverse L up and down and left and right, and then fill x with 0s to get x';

[0023] Reverse the convolution kernel, that is, perform recursive operations in the up, down, left, and right directions;

[0024] Take x' as input and perform a convolution operation with a step size of s.

[0025] As the preferred solution for the remote sensing image cloud detection method based on deep semantic segmentation, the training sample size, batch size, number of iterations and learning rate are set, the Adam Optimizer is used as the optimizer, and the improved version of focalloss is used as the loss function to train the deep semantic segmentation model.

[0026] As a preferred solution for the remote sensing image cloud detection method based on deep semantic segmentation, at the beginning of each iterative training of the deep semantic segmentation model, a batch of training samples is selected, and the training samples are input into the deep semantic segmentation model to obtain a batch of detection results. The loss value between the output value and the true value is calculated by focal loss, the loss value is optimized using the BP algorithm, and the model parameters are updated in the opposite direction of the loss value gradient.

[0027] The present invention also provides a remote sensing image cloud detection device based on deep semantic segmentation, which adopts the above remote sensing image cloud detection method based on deep semantic segmentation, including:

[0028] The data preprocessing module is used to map the remote sensing image pixel values ​​outside the preset spectral channel valid value range to the range of [0,1] through deviation standardization, and convert the mapped remote sensing image pixel values ​​into image pixel values ​​with a value range of 0 to 255;

[0029] A model building module, used to build a deep semantic segmentation model, wherein the deep semantic segmentation model includes convolution operation, activation operation, pooling operation, feature fusion, transposed convolution operation and logistic regression operation;

[0030] The deep semantic segmentation model has an encoding structure and a decoding structure:

[0031] The encoding structure is composed of a plurality of repeated first convolutional units, wherein the first convolutional unit uses a rectified linear unit activation function and a maximum pooling operation for downsampling;

[0032] In the decoding structure, the feature map is upsampled, a transposed convolution operation is performed by a second convolution unit, and a feature map output by a corresponding layer in the cascade coding structure is convolved by a third convolution unit to map the feature map to a preset size;

[0033] The fourth convolution unit is used to perform convolution operation to map the feature vector into a probability vector for each pixel, and the pixel-level classification result is obtained by selecting the position of the maximum value in the probability vector corresponding to each pixel.

[0034] The invention has the following advantages: through deviation standardization, remote sensing image pixel values ​​outside the preset spectral channel effective value range are mapped into the range of [0,1], and the mapped remote sensing image pixel values ​​are converted into image pixel values ​​in the value range of 0 to 255; a deep semantic segmentation model is constructed, and the deep semantic segmentation model includes convolution operation, activation operation, pooling operation, feature fusion, transposed convolution operation and logistic regression operation; the deep semantic segmentation model has a coding structure and a decoding structure: the coding structure is composed of a plurality of repeated first convolution units, and the first convolution unit uses a rectified linear unit activation function and a maximum pooling operation for downsampling; in the decoding structure, a feature map is upsampled, a transposed convolution operation is performed by a second convolution unit, and a feature map output by a corresponding layer in a cascade coding structure is convoluted by a third convolution unit to map the feature map to a preset size; a fourth convolution unit is used to perform a convolution operation, a feature vector is mapped into a probability vector, and a pixel-level classification result is obtained by selecting the position of the maximum value in the probability vector corresponding to each pixel. The present invention specifically designs a deep semantic segmentation model. In order to aggregate the contextual information of clouds in different areas, the jump connections from high-level features to low-level features are added, and the number of downsampling times in the encoding process is increased to ensure that enough feature information is extracted. In view of the serious imbalance problem of positive and negative samples in network training, the loss function is improved to achieve effective weighting of a small number of positive samples, which can effectively extract the boundaries of clouds, especially for the identification of broken clouds and the boundaries between clouds and clear sky, and the multi-channel threshold method has significant advantages. It has a certain robustness and is less affected by the misjudged categories in the training samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the implementation methods of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the implementation methods or the description of the prior art. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other implementation drawings can be derived from the provided drawings without creative work.

[0036] The structures, proportions, sizes, etc. illustrated in this specification are only used to match the contents disclosed in the specification so as to facilitate understanding and reading by persons familiar with the technology. They are not used to limit the conditions under which the present invention can be implemented, and therefore have no substantial technical significance. Any structural modification, change in proportion or adjustment of size shall still fall within the scope of the technical contents disclosed in the present invention without affecting the effects and purposes that can be achieved by the present invention.

[0037] Figure 1 A schematic diagram of a deep semantic segmentation model used in a remote sensing image cloud detection method based on deep semantic segmentation provided in an embodiment of the present invention;

[0038] Figure 2 A flow chart of model training in a remote sensing image cloud detection method based on deep semantic segmentation provided in an embodiment of the present invention;

[0039] Figure 3 Schematic diagram of a remote sensing image cloud detection device based on deep semantic segmentation provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0040] The following is a description of the implementation of the present invention by specific embodiments. People familiar with the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] Example 1

[0042] In Example 1 of the present invention, the data sources used by the remote sensing image cloud detection method based on deep semantic segmentation are as follows:

[0043] The FY-2E / VISSR L1 data from January to December 2010 are used as training samples, and the Aqua / MODIS MYD35 data from the same period are used as label samples. The auxiliary data include 3 categories:

[0044] (1) The geographic information data required for the spatiotemporal matching of the above two types of data, namely, the nominal projection latitude and longitude lookup table of FY-2 and Aqua / MODIS MYD03 data.

[0045] (2) To facilitate result comparison and error analysis, the corresponding products of the multi-channel threshold cloud detection algorithm for FY-2 business applications were collected and matched.

[0046] (3) The corresponding Aqua MODIS L1B data are selected for some typical samples to generate true color synthetic remote sensing images as auxiliary reference for manual identification.

[0047] See also Figure 1 and Figure 2 The remote sensing image cloud detection method based on deep semantic segmentation of the present invention comprises:

[0048] Data preprocessing: Through deviation standardization, the remote sensing image pixel values ​​outside the preset spectral channel valid value range are mapped to the range of [0,1], and the mapped remote sensing image pixel values ​​are converted into image pixel values ​​with a value range of 0 to 255;

[0049] Model building: construct a deep semantic segmentation model, which includes convolution operation, activation operation, pooling operation, feature fusion, transposed convolution operation and logistic regression operation;

[0050] The deep semantic segmentation model has an encoding structure and a decoding structure:

[0051] The encoding structure is composed of a plurality of repeated first convolutional units, wherein the first convolutional unit uses a rectified linear unit activation function and a maximum pooling operation for downsampling;

[0052] In the decoding structure, the feature map is upsampled, a transposed convolution operation is performed by a second convolution unit, and a feature map output by a corresponding layer in the cascade coding structure is convolved by a third convolution unit to map the feature map to a preset size;

[0053] The fourth convolution unit is used to perform convolution operation to map the feature vector into a probability vector for each pixel, and the pixel-level classification result is obtained by selecting the position of the maximum value in the probability vector corresponding to each pixel.

[0054] Specifically, during the data preprocessing process, the FY-2E L1 China region data (spatial resolution of 5km) and the MYD35 cloud detection product are transformed into equal longitude and latitude projections, and the obtained projection data are matched using the nearest neighbor method to obtain registration data with a spatial error of less than 10 kilometers (2 pixels) and a time error of less than 15 minutes. Samples with poor quality are removed with reference to the quality control code of the MYD35 data. The registration data is cut into 256×256 tile data again to improve training efficiency. The holdout method is used to randomly select 60% of the samples from the tile data as training data sets, 20% of the samples as validation data sets, and the remaining 20% ​​of the samples as test data sets, and there is no overlap between the data sets in terms of geographical location.

[0055] In this embodiment, the first convolution unit includes two repeated 3×3 convolution kernels, and each convolution kernel in the first convolution unit includes a 2×2 maximum pooling operation with a step size of 2 for downsampling; in each downsampling process, the number of feature channels is doubled. The second convolution unit is a 2×2 convolution kernel, and the transposed convolution operation is performed by the second convolution unit to restore half of the feature space resolution and reduce the number of feature channels by half. The third convolution unit uses two 3×3 convolution kernels for convolution operation, and the convolution kernel in the third convolution unit uses a rectified linear unit activation function to map the feature map to a preset size. The fourth convolution unit uses a 1×1 convolution kernel for convolution operation to map the feature vector into a probability vector for each pixel.

[0056] Specifically, the deep semantic segmentation model consists of an encoding structure and a decoding structure.

[0057] Among them, the encoding structure follows the typical convolutional network structure. The first convolution unit of the encoding structure consists of two repeated 3×3 convolution kernels (padded convolution), and both use the rectified linear unit (ReLU) activation function and a 2×2 maximum pooling operation with a step size of 2 for down sampling. In each downsampling step, the number of feature channels is doubled.

[0058] In the decoding structure, each step includes upsampling the feature map; then the second convolution unit uses a 2×2 convolution kernel to perform a transpose convolution operation to restore half of the feature space resolution and reduce the number of feature channels by half; then the feature map output by the corresponding layer in the cascade coding structure; and the third convolution unit uses two 3×3 convolution kernels to perform convolution operations, and both use the rectified linear unit activation function to map the feature map to a 256×256×4 feature map, that is, each cloud pixel corresponds to four classification feature values. Since there is a missing problem of boundary pixels in each convolution operation, it is necessary to crop the feature map. In the last layer, the fourth convolution unit uses a 1×1 convolution kernel to perform a convolution operation to map the feature vector into a probability vector of size 256×256×4. By selecting the position of the maximum value in the probability vector corresponding to each pixel, the pixel-level classification result is obtained.

[0059] In this embodiment, during the convolution operation, the input vector is convolved and transposed convolutionally performed inside the encoder and decoder, respectively. Represents the feature matrix x i,j,k , the nth node in the filter has a corresponding filter weight of w i,j,k , b n represents the bias parameter corresponding to the nth output value, then The convolution value g(n) is:

[0060]

[0061] Where H, W, and C represent the length, width, and number of convolution kernels, respectively.

[0062] Specifically, in the pooling operation, low-level feature information can be merged, the amount of computational data can be reduced, and the receptive field of high-level filters can be expanded. Given a feature matrix x of size H×W×C, a pooling kernel of size F×F (unpadded pooling), and a pooling step length s, the output of the maximum pooling operation is:

[0063] F(i,j)=max(G(i,j))

[0064] Where G(i,j) represents the pooling area of ​​size F×F in x

[0065] Specifically, in the feature fusion operation, the low-level feature vector is spliced ​​with the corresponding high-level feature vector into a new tensor, which can fuse the high-level semantic information and the low-level local feature information, thereby achieving accurate and detailed cloud feature extraction. In the deep semantic segmentation model proposed in the present invention, the feature map output by each upsampling is channel-connected in series with the feature map of the same scale output by the downsampling part.

[0066] In this embodiment, the transposed convolution operation includes the following steps:

[0067] Given a step length s, add s-1 zeros after each element in the input feature map along the direction of the step length to obtain the expanded feature map x.

[0068] According to the convolution rule, calculate the position L and number of padding 0s, reverse L up and down and left and right, and then fill x with 0s to get x';

[0069] Reverse the convolution kernel, that is, perform recursive operations in the up, down, left, and right directions;

[0070] Take x' as input and perform a convolution operation with a step size of s.

[0071] Specifically, in the logistic regression operation, the output values ​​of multiple neurons are mapped to between 0 and 1, which can be used as a probability description of multi-classification prediction. Given the output value Z of the i-th neuron in the output layer i , then Z i The probability value S i for:

[0072]

[0073] Where N is the number of neurons in the output layer, that is, the number of categories.

[0074] In this embodiment, the training sample size, batch size, number of iterations and learning rate are set, the optimizer uses AdamOptimizer, and the loss function uses an improved version of focal loss to perform deep semantic segmentation model training.

[0075] Specifically, in view of the complexity of cloud detection, the present invention proposes an improved algorithm for a given predicted value y′∈[10-5,1-10-5] and its true label y:

[0076]

[0077] In the formula, w a =a·(1-y′) γ y, w b =(1-a)·(y′) γ (1-y), where γ is used to reduce the loss of easy-to-classify samples so that the model pays more attention to difficult samples, and α is used to balance the uneven ratio of positive and negative samples.

[0078] In this embodiment, at the beginning of each iterative training of the deep semantic segmentation model, a batch of training samples is selected, and the training samples are input into the deep semantic segmentation model to obtain a batch of detection results. The loss value between the output value and the true value is calculated by focalloss, the loss value is optimized using the BP algorithm, and the model parameters are updated in the opposite direction of the loss value gradient.

[0079] Specifically, the training sample size is set to 256×256×5, the batch size is 16, the number of iterations is 500000, the learning rate is 0.001, the optimizer is Adam Optimizer (β1=0.9, β2=0.999, ε=10-8), and the loss function is the improved version of focal loss with parameters γ=2, α=0.25.

[0080] At the beginning of each iterative training, a batch of training samples is first selected. Then, the training samples are input into the deep semantic segmentation model (forward propagation algorithm) to obtain a batch of detection results. Again, the loss between the predicted value and the true value is calculated based on the improved focalloss method. Finally, the BP algorithm is used to optimize the loss, and the model parameters are updated according to the opposite direction of the loss gradient. The training process is iterated until certain stopping conditions are reached.

[0081] In the present invention, the deep semantic segmentation model makes full use of the existing cloud detection products with better accuracy, and can realize fast and automatic cloud detection and classification without artificially designing classification features. Since MODIS is used as a label, the four-classification of cloud detection is realized, which is easier to be consistent with the cloud detection classification of Fengyun-3, Fengyun-4 and other foreign satellites. In view of the serious imbalance of positive (suspected clear sky and suspected cloud) and negative (clear sky and cloud) samples in model training, the loss function is improved to achieve effective weighting of a small number of positive samples, which can better extract the boundary of the cloud. The deep semantic segmentation model can achieve about 75% of the accuracy of MODIS four-classification with only 5km channel information of FY2E VISSR, and the kappa coefficient reaches about 0.53; the overall recognition accuracy is improved by 6%-10% compared with the channel threshold method, and some samples can be significantly improved by more than 20%, especially for the recognition of broken clouds and the boundary between clouds and clear sky. The channel threshold method has significant advantages.

[0082] In summary, the present invention maps the remote sensing image pixel values ​​outside the preset spectral channel effective value range to the range of [0,1] through deviation standardization, and converts the mapped remote sensing image pixel values ​​into image pixel values ​​in the value range of 0 to 255; constructs a deep semantic segmentation model, and the deep semantic segmentation model includes convolution operation, activation operation, pooling operation, feature fusion, transposed convolution operation and logistic regression operation; the deep semantic segmentation model has a coding structure and a decoding structure: the coding structure is composed of a plurality of repeated first convolution units, and the first convolution unit uses a rectified linear unit activation function and a maximum pooling operation for downsampling; in the decoding structure, the feature map is upsampled, and the second convolution unit is used to perform a transposed convolution operation, and the feature map output by the corresponding layer in the cascade coding structure is then convolved by the third convolution unit to map the feature map to a preset size; the fourth convolution unit is used to perform a convolution operation, and the feature vector is mapped into a probability vector of each pixel, and the pixel-level classification result is obtained by selecting the position of the maximum value in the probability vector corresponding to each pixel. The present invention specifically designs a deep semantic segmentation model. In order to aggregate the contextual information of clouds in different areas, the jump connections from high-level features to low-level features are added, and the number of downsampling times in the encoding process is increased to ensure that enough feature information is extracted. In view of the serious imbalance problem of positive and negative samples in network training, the loss function is improved to achieve effective weighting of a small number of positive samples, which can effectively extract the boundaries of clouds, especially for the identification of broken clouds and the boundaries between clouds and clear sky, and the multi-channel threshold method has significant advantages. It has a certain robustness and is less affected by the misjudged categories in the training samples.

[0083] Example 2

[0084] See also Figure 3Embodiment 2 of the present invention provides a remote sensing image cloud detection device based on deep semantic segmentation, using the remote sensing image cloud detection method based on deep semantic segmentation of embodiment 1 or any possible implementation thereof, including:

[0085] The data preprocessing module 1 is used to map the remote sensing image pixel values ​​outside the preset spectral channel effective value range to the range of [0,1] through deviation standardization, and convert the mapped remote sensing image pixel values ​​into image pixel values ​​with a value range of 0 to 255;

[0086] Model building module 2, used to build a deep semantic segmentation model, the deep semantic segmentation model includes convolution operation, activation operation, pooling operation, feature fusion, transposed convolution operation and logistic regression operation;

[0087] The deep semantic segmentation model has an encoding structure and a decoding structure:

[0088] The encoding structure is composed of a plurality of repeated first convolutional units, wherein the first convolutional unit uses a rectified linear unit activation function and a maximum pooling operation for downsampling;

[0089] In the decoding structure, the feature map is upsampled, a transposed convolution operation is performed by a second convolution unit, and a feature map output by a corresponding layer in the cascade coding structure is convolved by a third convolution unit to map the feature map to a preset size;

[0090] The fourth convolution unit is used to perform convolution operation to map the feature vector into a probability vector for each pixel, and the pixel-level classification result is obtained by selecting the position of the maximum value in the probability vector corresponding to each pixel.

[0091] In this embodiment, the first convolution unit includes two repeated 3×3 convolution kernels, and each convolution kernel in the first convolution unit includes a 2×2 maximum pooling operation with a step size of 2 for downsampling;

[0092] During each downsampling process, the number of feature channels is doubled.

[0093] In this embodiment, the second convolution unit is a 2×2 convolution kernel, and a transposed convolution operation is performed by the second convolution unit to restore half of the feature space resolution and reduce the number of feature channels by half.

[0094] In this embodiment, the third convolution unit uses two 3×3 convolution kernels to perform convolution operations, and the convolution kernel in the third convolution unit uses a rectified linear unit activation function to map the feature map to a preset size.

[0095] In this embodiment, the fourth convolution unit uses a 1×1 convolution kernel to perform a convolution operation to map the feature vector into a probability vector, and obtains a pixel-level classification result by selecting the position of the maximum value in the probability vector corresponding to each pixel.

[0096] In this embodiment, in the convolution operation, the input vector is subjected to convolution and transposed convolution operations inside the encoder and decoder respectively;

[0097] In the feature fusion, the low-level feature vectors and the corresponding high-level feature vectors are concatenated into a new tensor, and the high-level semantic information and the low-level local feature information are fused.

[0098] As a preferred solution of the remote sensing image cloud detection method based on deep semantic segmentation, the transposed convolution operation includes the following steps:

[0099] Given a step length s, add s-1 zeros after each element in the input feature map along the direction of the step length to obtain the expanded feature map x.

[0100] According to the convolution rule, calculate the position L and number of padding 0s, reverse L up and down and left and right, and then fill x with 0s to get x';

[0101] Reverse the convolution kernel, that is, perform recursive operations in the up, down, left, and right directions;

[0102] Take x' as input and perform a convolution operation with a step size of s.

[0103] In this embodiment, the training sample size, batch size, number of iterations and learning rate are set, the optimizer uses AdamOptimizer, and the loss function uses an improved version of focal loss to perform deep semantic segmentation model training.

[0104] In this embodiment, at the beginning of each iterative training of the deep semantic segmentation model, a batch of training samples is selected, and the training samples are input into the deep semantic segmentation model to obtain a batch of detection results. The loss value between the output value and the true value is calculated by focalloss, the loss value is optimized using the BP algorithm, and the model parameters are updated in the opposite direction of the loss value gradient.

[0105] It should be noted that the information interaction, execution process, etc. between the modules / units of the above-mentioned device are based on the same concept as the method embodiment in Example 1 of the present application, and the technical effects they bring are the same as those of the method embodiment of the present application. For specific contents, please refer to the description in the method embodiment shown in the previous part of the present application, and will not be repeated here.

[0106] Example 3

[0107] Embodiment 3 of the present invention provides a computer-readable storage medium, in which a program code of a remote sensing image cloud detection method based on deep semantic segmentation is stored. The program code includes instructions for executing the remote sensing image cloud detection method based on deep semantic segmentation of Embodiment 1 or any possible implementation thereof.

[0108] The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0109] Example 4

[0110] Embodiment 4 of the present invention provides an electronic device, comprising a processor, wherein the processor is coupled to a storage medium, and when the processor executes instructions in the storage medium, the electronic device executes the remote sensing image cloud detection method based on deep semantic segmentation of Embodiment 1 or any possible implementation thereof.

[0111] Specifically, the processor can be implemented by hardware or by software. When implemented by hardware, the processor can be a logic circuit, an integrated circuit, etc.; when implemented by software, the processor can be a general-purpose processor implemented by reading software codes stored in a memory. The memory can be integrated in the processor or can be located outside the processor and exist independently.

[0112] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center.

[0113] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0114] Although the present invention has been described in detail above by general description and specific embodiments, it is obvious to those skilled in the art that some modifications or improvements can be made to the present invention. Therefore, these modifications or improvements made without departing from the spirit of the present invention all belong to the scope of protection claimed by the present invention.

Claims

1. A remote sensing image cloud detection method based on deep semantic segmentation, characterized in that: include: Data preprocessing: Through deviation standardization, the remote sensing image pixel values ​​outside the preset spectral channel valid value range are mapped to the range of [0,1], and the mapped remote sensing image pixel values ​​are converted into image pixel values ​​with a value range of 0 to 255; Model building: construct a deep semantic segmentation model, which includes convolution operation, activation operation, pooling operation, feature fusion, transposed convolution operation and logistic regression operation; The deep semantic segmentation model has an encoding structure and a decoding structure: The encoding structure is composed of a plurality of repeated first convolutional units, wherein the first convolutional unit uses a rectified linear unit activation function and a maximum pooling operation for downsampling; In the decoding structure, the feature map is upsampled, a transposed convolution operation is performed by a second convolution unit, and a feature map output by a corresponding layer in the cascade coding structure is convolved by a third convolution unit to map the feature map to a preset size; The fourth convolution unit is used to perform convolution operation to map the feature vector into a probability vector of each pixel, and the pixel-level classification result is obtained by selecting the position of the maximum value in the probability vector corresponding to each pixel; The first convolution unit includes two repeated 3×3 convolution kernels, and each convolution kernel in the first convolution unit includes a 2×2 maximum pooling operation with a step size of 2 for downsampling; In each downsampling process, the number of feature channels is doubled; The second convolution unit is a 2×2 convolution kernel, and the transposed convolution operation is performed by the second convolution unit to restore half of the feature space resolution and reduce the number of feature channels by half; The third convolution unit uses two 3×3 convolution kernels to perform convolution operations, and the convolution kernel in the third convolution unit uses a rectified linear unit activation function to map the feature map to a preset size; The fourth convolution unit uses a 1×1 convolution kernel to perform a convolution operation to map the feature vector into a probability vector, and obtains a pixel-level classification result by selecting the position of the maximum value in the probability vector corresponding to each pixel; In the convolution operation, the input vector is convolved and transposed convolutionally performed inside the encoder and decoder respectively; In the feature fusion, the low-level feature vectors are concatenated with the corresponding high-level feature vectors into a new tensor, fusing the high-level semantic information and the low-level local feature information; The transposed convolution operation includes the following steps: Given a step length s, add s-1 zeros after each element in the input feature map along the direction of the step length to obtain the expanded feature map x; According to the convolution rule, calculate the position L and number of padding 0s, reverse L up and down and left and right, and then fill x with 0s as a whole to get x , ; Reverse the convolution kernel, that is, perform recursive operations in the up, down, left, and right directions; x , As input, a convolution operation with a stride of s is performed.

2. The remote sensing image cloud detection method based on deep semantic segmentation according to claim 1 is characterized in that: The training sample size, batch size, number of iterations and learning rate are set. The Adam Optimizer is used as the optimizer and the improved focalloss is used as the loss function to train the deep semantic segmentation model.

3. The remote sensing image cloud detection method based on deep semantic segmentation according to claim 2 is characterized in that: At the beginning of each iterative training of the deep semantic segmentation model, a batch of training samples is selected, and the training samples are input into the deep semantic segmentation model to obtain a batch of detection results. The loss value between the output value and the true value is calculated by focalloss, the loss value is optimized using the BP algorithm, and the model parameters are updated according to the opposite direction of the loss value gradient.

4. A remote sensing image cloud detection device based on deep semantic segmentation, adopting the remote sensing image cloud detection method based on deep semantic segmentation according to any one of claims 1 to 3, characterized in that: include: The data preprocessing module is used to map the remote sensing image pixel values ​​outside the preset spectral channel valid value range to the range of [0,1] through deviation standardization, and convert the mapped remote sensing image pixel values ​​into image pixel values ​​with a value range of 0 to 255; A model building module is used to build a deep semantic segmentation model, wherein the deep semantic segmentation model includes convolution operation, activation operation, pooling operation, feature fusion, transposed convolution operation and logistic regression operation; The deep semantic segmentation model has an encoding structure and a decoding structure: The encoding structure is composed of a plurality of repeated first convolutional units, wherein the first convolutional unit uses a rectified linear unit activation function and a maximum pooling operation for downsampling; In the decoding structure, the feature map is upsampled, a transposed convolution operation is performed by a second convolution unit, and a feature map output by a corresponding layer in the cascade coding structure is convolved by a third convolution unit to map the feature map to a preset size; The fourth convolution unit is used to perform convolution operation to map the feature vector into a probability vector for each pixel, and the pixel-level classification result is obtained by selecting the position of the maximum value in the probability vector corresponding to each pixel.

Citation Information

Patent Citations

  • Image Semantic Segmentation Method Based on Deep Full Convolutional Network and Conditional Random Field

    AU2020103901A4

  • road extraction method based on remote sensing images and deep learning

    CN109800736A

  • Remote sensing image cloud detection method based on multi-scale fusion semantic segmentation network

    CN110119728A

  • Image segmentation model training method and device, image segmentation method and device, equipment and medium

    CN111932546A

  • Lightweight semantic segmentation method for high-resolution remote sensing image

    CN112183360A