Unpaired Weakly Supervised Cloud Detection Method and System Based on Markov Discriminator

Abstract

The present invention relates to the field of remote sensing image processing, and specifically discloses a non-paired weakly supervised cloud detection method and system based on a Markov discriminator, including: acquiring cloudy remote sensing images and cloudless remote sensing images and using them as a training set to construct a cloud detection network, selecting a first sample from the cloudy remote sensing images and inputting it into the cloud detection network for inaccurate supervision to generate a rough confidence map; selecting a second sample from the cloudless remote sensing images and inputting it into the cloud detection network for inaccurate supervision, and correcting the rough confidence map by learning the prior features of the second sample, then performing backpropagation to update the network parameters of the cloud detection network to obtain a trained cloud detection network, determining a final confidence map according to the trained cloud detection network, and determining a binary cloud detection result based on the final confidence map. The above solution in the present invention solves the problems that paired supervision must use paired labels and the segmentation accuracy is not high due to insufficient pixel-level labels.

Description

Technical Field

[0001] The present invention relates to the field of remote sensing image processing, and in particular to an unpaired weakly supervised cloud detection method and system based on a Markov discriminator. Background Art

[0002] With the rapid development of remote sensing technology, using remote sensing images to obtain geospatial information has great research significance and application value. However, remote sensing images are easily affected by cloud cover, resulting in partial information loss, affecting the results of subsequent target recognition or detection tasks. In order to solve this problem, many cloud detection methods have been proposed in recent years, which can be roughly divided into two categories: one is a graphics method based on traditional mathematical and statistical analysis, and the other is a data-driven deep learning algorithm. Among them, the graphics method based on traditional mathematical and statistical analysis can be roughly divided into three categories: methods based on hyperspectral information, methods based on thresholds, and methods based on machine learning classification. The method based on hyperspectral information uses the multi-band information of hyperspectral remote sensing images to detect cloud areas, which is highly dependent on the sensor model. Similar assumptions cannot be used to generalize to different sensors, and this method is not suitable for processing RGB remote sensing images; the threshold-based methods include single threshold, automatic cloud cover assessment (ACCA), dual-channel dynamic threshold, channel comprehensive operation, Fmask cloud detection algorithm, etc. These methods have a common problem that they are easily affected by the geographical environment. The method based on machine learning classification does not need to determine the threshold of the image, and has been increasingly used in cloud area detection in recent years. However, machine learning-based methods require manual selection of favorable features, making it difficult for classical machine learning-based methods to more efficiently extract higher-level semantic information from remote sensing images. As the scene becomes more complex, the accuracy of machine learning-based methods will also decrease.

[0003] The detection of cloud areas using data-driven deep learning algorithms does not require manual selection of cloud features, but rather uses convolutional neural networks (CNNs) to automatically extract cloud features. Many remote sensing cloud detection methods based on deep learning divide remote sensing images into multiple remote sensing image blocks, model the cloud detection task as an image classification problem, and then use CNNs to extract image features to achieve cloud detection. This type of method is more accurate than graphics methods based on traditional mathematical and statistical analysis, but it can cause large errors when remote sensing images contain both clouds and no clouds.

[0004] In addition, most deep learning models are trained using supervised learning. Such models are highly dependent on cloud remote sensing images and their corresponding label values, and require a large amount of human resources to obtain pixel-level fine label values. The acquisition cost of paired, pixel-level manually labeled label images is extremely high, almost becoming a bottleneck problem restricting the development of such methods.

[0005] To alleviate the problem of low segmentation accuracy caused by insufficient pixel-level labels, many cloud detection methods based on weakly supervised learning have been continuously proposed. A large number of cloud-containing remote sensing images for training can be generated through the Generative Adversarial Network (GAN), thus avoiding a large amount of manual annotation. The interleaved perception autoencoder can integrate heterogeneous information and improve the joint segmentation performance of the two types of data without relying on label data. The above methods have alleviated the problem of over-reliance on label values to a certain extent, but still cannot meet the current demand for obtaining geospatial information through cloud detection. Summary of the Invention

[0006] The purpose of the present invention is to provide a non-paired weakly supervised cloud detection method and system based on a Markov discriminator, which can get rid of the problems of paired supervision that must use paired labels and low image segmentation accuracy caused by insufficient pixel-level labels, and obtain more accurate cloud detection results.

[0007] To achieve the above purpose, the present invention provides the following solutions:

[0008] A non-paired weakly supervised cloud detection method based on a Markov discriminator, the cloud detection method comprising:

[0009] Step 1: Obtain cloud-containing remote sensing images and cloud-free remote sensing images, and use the cloud-containing remote sensing images and cloud-free remote sensing images as a training set;

[0010] Step 2: Set cloud detection network parameters;

[0011] Step 3: Construct a cloud detection network based on the cloud detection network parameters;

[0012] Step 4: Pre-define the label of the cloud-containing remote sensing image as all 0, and pre-define the label of the cloud-free remote sensing image as all 1;

[0013] Step 5: Select m cloud-containing images from the cloud-containing remote sensing images as the first sample and input them into the cloud detection network, perform inaccurate supervision with all labels being 0, calculate the first cross-entropy loss, and generate a rough confidence map;

[0014] Step 6: Select m cloudless images from the cloudless remote sensing images as the second sample, input them into the cloud detection network, perform inaccurate supervision with all labels being 1, calculate the second cross-entropy loss, and correct the rough confidence map by learning the prior features of the second sample;

[0015] Step 7: Perform backpropagation based on the first cross-entropy loss and the second cross-entropy loss to update the network parameters of the cloud detection network;

[0016] Step 8: Repeat Step 2 - Step 7 until the cloud detection network is trained, and obtain the trained cloud detection network;

[0017] Step 9: Input the cloudy remote sensing image to be detected into the trained cloud detection network to obtain the final confidence map;

[0018] Step 10: Determine the binary cloud detection result based on the final confidence map.

[0019] Preferably, the cloud detection network is a spectral normalization Markov discriminator, and the Markov discriminator specifically includes: a first convolutional layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer, a fifth convolutional layer, a first spectral normalization layer, a second spectral normalization layer, a third spectral normalization layer, a fourth spectral normalization layer, and a fifth spectral normalization layer; the first convolutional layer, the first spectral normalization layer, the second convolutional layer, the second spectral normalization layer, the third convolutional layer, the third spectral normalization layer, the fourth convolutional layer, the fourth spectral normalization layer, the fifth convolutional layer, and the fifth spectral normalization layer are connected in sequence.

[0020] Preferably, the first spectral normalization layer, the second spectral normalization layer, the third spectral normalization layer, the fourth spectral normalization layer, and the fifth spectral normalization layer all adopt the following formula:

[0021]

[0022] where W is the weight parameter matrix of the current layer network, and σ (W) is the maximum singular value of the weight parameter matrix.

[0023] Preferably, the loss function of the first cross-entropy loss is:

[0024]

[0025] where D sn represents the spectral normalization Markov discriminator, C represents the cloudy remote sensing images of the mini-batch samples, represents the probability that the first sample is sampled from the cloudy remote sensing image.

[0026] Preferably, the loss function of the second cross-entropy loss is:

[0027]

[0028] Among them, D sn represents the spectral normalization Markov discriminator, N represents the cloud-free remote sensing map of a small batch of samples, indicating the probability that the second sample is sampled from a cloud-free remote sensing image.

[0029] Preferably, the training set is unlabeled cloud regions and unpaired cloud-covered remote sensing images and cloud-free remote sensing images.

[0030] Preferably, the determining the binary cloud detection result based on the final confidence map specifically includes the following steps:

[0031] Obtain the optimal threshold of the foreground image and the background image;

[0032] Binarize the confidence map according to the optimal threshold to obtain the cloud detection result.

[0033] Preferably, the obtaining the optimal threshold of the foreground image and the background image specifically includes the following steps:

[0034] Calculate the variances of the foreground image and the background image: g = w0 × w1 × (u0 - u1) 2 ; where, the proportion of foreground points in the image is w0, and the average gray level is u0; the proportion of background points in the image is w1, and the average gray level is u1;

[0035] Take the gray value corresponding to the maximum variance of the foreground image and the background image as the optimal threshold of the foreground image and the background image.

[0036] Preferably, the binarizing the confidence map according to the optimal threshold to obtain the cloud detection result specifically adopts the following formula:

[0037]

[0038] where, f(x, y) is the confidence map, and T is the optimal threshold of the foreground image and the background image.

[0039] Based on the above method in the present invention, the present invention also provides a non-paired weakly supervised cloud detection system based on a Markov discriminator, and the cloud detection system includes:

[0040] An image acquisition and division module, configured to acquire cloud-covered remote sensing images and cloud-free remote sensing images, and use the cloud-covered remote sensing images and cloud-free remote sensing images as a training set;

[0041] A cloud detection network parameter determination module, configured to set cloud detection network parameters;

[0042] A cloud detection network construction module, configured to construct a cloud detection network based on the cloud detection network parameters;

[0043] A label definition module, configured to pre-define the label of the cloud-containing remote sensing image as all 0s, and pre-define the label of the cloud-free remote sensing image as all 1s;

[0044] A first cross-entropy loss calculation and rough confidence map generation module, configured to select m cloud-containing images from the cloud-containing remote sensing images as the first samples to input into the cloud detection network, perform inaccurate supervision with all labels being 0, calculate the first cross-entropy loss, and generate a rough confidence map;

[0045] A second cross-entropy loss calculation and rough confidence map correction module, configured to select m cloud-free images from the cloud-free remote sensing images as the second samples to input into the cloud detection network, perform inaccurate supervision with all labels being 1, calculate the second cross-entropy loss, and correct the rough confidence map by learning the prior features of the second samples;

[0046] A cloud detection network parameter update module, configured to perform backpropagation based on the first cross-entropy loss and the second cross-entropy loss to update the network parameters of the cloud detection network;

[0047] A loop training module, configured to repeat the cloud detection network parameter determination module, the cloud detection network construction module, the label definition module, the first cross-entropy loss calculation and rough confidence map generation module, the second cross-entropy loss calculation and rough confidence map correction module, and the cloud detection network parameter update module until the cloud detection network is trained, and obtain a trained cloud detection network;

[0048] A test module, configured to input the cloud-containing remote sensing image to be detected into the trained cloud detection network to obtain a final confidence map;

[0049] A cloud detection result acquisition module, configured to determine a binary cloud detection result based on the final confidence map.

[0050] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0051] The present invention provides a non-paired weakly supervised cloud detection method and system based on a Markov discriminator. By obtaining cloud-containing remote sensing images and cloud-free remote sensing images, and using the cloud-containing remote sensing images and cloud-free remote sensing images as a training set, setting cloud detection network parameters and constructing a cloud detection network, predefining the label of the cloud-containing remote sensing image as all 0s, and predefining the label of the cloud-free remote sensing image as all 1s, selecting m cloud-containing images from the cloud-containing remote sensing images as the first sample and inputting them into the cloud detection network, performing inaccurate supervision with all labels being 0, calculating the first cross-entropy loss, and generating a rough confidence map, selecting m cloud-free images from the cloud-free remote sensing images as the second sample and inputting them into the cloud detection network, performing inaccurate supervision with all labels being 1, calculating the second cross-entropy loss, and correcting the rough confidence map by learning the prior features of the second sample, performing backpropagation based on the first cross-entropy loss and the second cross-entropy loss to update the network parameters of the cloud detection network, repeating the training process until the cloud detection network is trained, obtaining the trained cloud detection network, inputting the cloud-containing remote sensing image to be detected into the trained cloud detection network, obtaining the final confidence map, and determining the binary cloud detection result based on the final confidence map. This application does not require a large amount of label data for pixel-by-pixel marking of whether it is a cloud, and only completes training through cloud-containing remote sensing images and cloud-free remote sensing images, solving the problems of paired supervision that must use paired labels and low segmentation accuracy due to insufficient pixel-level labels, and obtaining a more accurate cloud detection result.

[0052] In addition, based on the Markov discriminator, this application classifies the cloud-containing remote sensing image and the cloud-free remote sensing image respectively to obtain a confidence map, then uses threshold segmentation to convert the probability distribution map into a mask composed of 0s and 1s to obtain a fine detection result, and the weakly supervised training method can reduce the network cost of constructing the cloud detection and obtain a more accurate cloud detection network model. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the embodiments. Obviously, the following described drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0054] Figure 1 It is a flowchart of a non-paired weakly supervised cloud detection method based on a Markov discriminator provided by the present invention;

[0055] Figure 2 It is a schematic structural diagram of a non-paired weakly supervised cloud detection system based on a Markov discriminator provided by the present invention;

[0056] Figure 3Schematic diagram of cloud detection network training and testing in this embodiment;

[0057] Figure 4 Schematic diagram of cloud detection results in this embodiment. Detailed implementation manners

[0058] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0059] The purpose of the present invention is to provide a non-paired weakly supervised cloud detection method and system based on a Markov discriminator, which can solve the problems in the prior art that the image segmentation accuracy is not high, the cloud detection method in remote sensing images lacks pixel-level labels and relies too much on pixel-level labels, resulting in inaccurate cloud remote sensing image detection.

[0060] To make the above objects, features, and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific implementation manners.

[0061] Referring to the accompanying drawings Figure 1 , the present invention provides a non-paired weakly supervised cloud detection method based on a Markov discriminator. The cloud detection method includes:

[0062] Step 1: Obtain cloud-containing remote sensing images and cloud-free remote sensing images, and use the cloud-containing remote sensing images and cloud-free remote sensing images as a training set. In this embodiment, the numbers of cloud-containing remote sensing images and cloud-free remote sensing images in the training set respectively account for 20% of the total data.

[0063] Specifically, the training set is cloud-containing remote sensing images and cloud-free remote sensing images that are unlabeled in the cloud region and non-paired. The non-paired remote sensing images are randomly selected cloud-containing remote sensing images and cloud-free remote sensing images that have not been paired and labeled.

[0064] Specifically, for the convenience of network training, the cloud remote sensing images can be segmented through a sliding window to obtain multiple remote sensing images.

[0065] Step 2: Set cloud detection network parameters. In this embodiment, the cloud detection network parameters are shown in Table 1.

[0066] Table 1 Cloud detection network parameters

[0067] Layer Kernel Input Output Stride Padding Conv_1 4×4 3 64 2 1 Conv_2 4×4 64 128 2 1 Conv_3 4×4 128 256 2 1 Conv_4 4×4 256 512 2 1 Conv_5 4×4 512 1 2 1

[0068] Step 3: Construct a cloud detection network based on the cloud detection network parameters.

[0069] Specifically, refer to the attached drawings Figure 3 , the attached drawings Figure 3 are the schematic diagrams of the cloud detection network training and testing in this embodiment. The cloud detection network is a spectral normalization Markov discriminator. The Markov discriminator specifically includes:

[0070] The first convolutional layer, the second convolutional layer, the third convolutional layer, the fourth convolutional layer, the fifth convolutional layer, the first spectral normalization layer, the second spectral normalization layer, the third spectral normalization layer, the fourth spectral normalization layer, and the fifth spectral normalization layer; the first convolutional layer, the first spectral normalization layer, the second convolutional layer, the second spectral normalization layer, the third convolutional layer, the third spectral normalization layer, the fourth convolutional layer, the fourth spectral normalization layer, the fifth convolutional layer, and the fifth spectral normalization layer are connected in sequence. In this embodiment, the activation functions of the first convolutional layer, the second convolutional layer, the third convolutional layer, and the fourth convolutional layer are LeakyRelu, the fifth convolutional layer is an s-type activation function, and the optimization algorithm is Adam. The number of iterations is 200 times, and the learning rate is 0.0001.

[0071] Refer to the attached drawings Figure 3 , the Markov discriminator maps the remote sensing image into an N×N matrix. Each position in the matrix represents the probability of the true or false of the corresponding position in the image. Among them, the dimension of the Markov discriminator is the same as that of the image to be measured.

[0072] In this embodiment, the dataset of the cloud detection network is a preset label, not a label that pairwise annotates the cloud area pixel by pixel. Therefore, the training of the network may be unstable. The spectral norm normalization strategy in the SN-GAN network is adopted to improve the training stability of the discriminator, that is, by adding a spectral normalization layer behind each convolutional layer to make the discriminator satisfy Lipschitz continuity.

[0073] Specifically, the first spectral normalization layer, the second spectral normalization layer, the third spectral normalization layer, the fourth spectral normalization layer, and the fifth spectral normalization layer all adopt the following formula to improve the network training stability:

[0074]

[0075] where W is the weight parameter matrix of the current layer network, and σ (W) is the maximum singular value of the weight parameter matrix.

[0076] Step 4: Pre-define the label of the cloud-containing remote sensing image as all 0, and the label of the cloud-free remote sensing image as all 1.

[0077] Specifically, the cloud detection network provides inaccurate supervision information for cloud-covered remote sensing images, that is, setting the entire label of a cloud-covered remote sensing image to 0, which essentially assumes that the entire image is covered by clouds, and setting the entire label of a cloud-free remote sensing image to 1, actually specifying the prior information of the cloud-free area.

[0078] Step 5: Select m cloud-covered images from the cloud-covered remote sensing images as the first sample {c (1) , ……… c (m)} and input them into the cloud detection network for inaccurate supervision with all labels being 0, calculate the first cross-entropy loss, and generate a rough confidence map.

[0079] Among them, the loss function of the first cross-entropy loss is:

[0080]

[0081] Among them, D sn represents the spectral normalization Markov discriminator, C represents the cloud-covered remote sensing images of the mini-batch samples, represents the probability that the first sample is sampled from the cloud-covered remote sensing image.

[0082] Step 6: Select m cloud-free images from the cloud-free remote sensing images as the second sample {f (1) , ……… f (m)} and input them into the cloud detection network for inaccurate supervision with all labels being 1, calculate the second cross-entropy loss, and correct the rough confidence map by learning the prior features of the second sample, so that the values of the cloud-free areas in the rough confidence map tend to be 1.

[0083] Among them, the loss function of the second cross-entropy loss is:

[0084]

[0085] Among them, D sn represents the spectral normalization Markov discriminator, N represents the cloud-free remote sensing images of the mini-batch samples, represents the probability that the second sample is sampled from the cloud-free remote sensing image.

[0086] Determine the total loss function of the Markov discriminator through the first cross-loss function and the second cross-loss function, and the specific expression is as follows:

[0087]

[0088] Among them, λ1 and λ2 are weight coefficients.

[0089] In this embodiment, m = 1, λ1 = 1, λ2 = 2 are set, and the sizes of the cloud-covered images and cloud-free images are both 512px * 512px.

[0090] Step 7: Based on the first cross-entropy loss and the second cross-entropy loss, perform backpropagation to update the network parameters of the cloud detection network.

[0091] Specifically, update the cloud detection network parameter θ by reducing its stochastic gradient.

[0092] Step 8: Repeat Step 2 - Step 7 until the cloud detection network is trained, and obtain the trained cloud detection network.

[0093] Specifically, let the number of iterations be E max , when the number of training times for the first iteration is T max , the training set is trained completely. When the number of training times for the E max -th iteration is T max , the cloud detection network is trained completely. In this embodiment, set E max = 20, T max = 20.

[0094] In this embodiment, the cloud detection network adopts the weak supervision training method, and the training set used in each iteration is the same.

[0095] Specifically, referring to the training part in the attached figure Figure 3 , input the cloudy image and cloudless image with a size of 512px * 512px into the cloud detection network for training. Each time of training, first downsample the size to 225px * 225px, and then upsample it to 512px * 512px to be consistent with the size of the image to be trained.

[0096] Step 9: Input the cloudy remote sensing image to be detected into the trained cloud detection network to obtain the final confidence map.

[0097] Specifically, referring to the testing part in the attached figure Figure 3 , input the cloudy remote sensing image to be detected, that is, the side view in Figure 3 , with a size of 512px * 512px, into the trained cloud detection network for detection. During the detection process, first downsample the size of the detection image to 225px * 225px, and then upsample it to 512px * 512px to obtain the final confidence map for easy observation.

[0098] Step 10: Determine the binary cloud detection result based on the final confidence map.

[0099] The specific steps are as follows:

[0100] Step 10.1: Obtain the optimal thresholds of the foreground image and the background image.

[0101] Step 10.2: Binarize the confidence map according to the optimal threshold to obtain a cloud detection result.

[0102] Wherein, obtaining the optimal threshold value of the foreground image and the background image in step 10.1 specifically includes the following steps:

[0103] Calculate the variance of the foreground image and the background image: g = w0×w1×(u0-u1) 2 ; Among them, the number of foreground points accounts for w0 of the image, and the average grayscale is u0; the number of background points accounts for w1 of the image, and the average grayscale is u1;

[0104] The gray value corresponding to the maximum variance of the foreground image and the background image is used as the optimal threshold value of the foreground image and the background image.

[0105] Specifically, the variance formula of the foreground image and the background image is obtained by combining the following formulas:

[0106] u=w0×u0+w1×u1

[0107] g=w0×(u0-u) 2 +w1×(u1-u) 2

[0108] Among them, the proportion of foreground points in the image is w0, the average grayscale is u0, the proportion of background points in the image is w1, the average grayscale is u1, and the total average grayscale of the image is u.

[0109] When the variance g between the foreground image and the background image is the largest, the difference between the foreground image and the background image is the largest. At this time, the grayscale value of the entire image is the optimal threshold between the foreground image and the background image. The optimal threshold between the foreground image and the background image is defined as T.

[0110] In step 10.2, the confidence map is segmented using the OTSU threshold method to obtain the cloud detection result. The OTSU threshold method is a method that uses the maximum inter-class variance to automatically determine the threshold, and is a global binarization algorithm. Step 10.2 specifically uses the following formula:

[0111]

[0112] Among them, g(x,y) represents the cloud detection result, f(x,y) is the confidence map, and T is the optimal threshold of the foreground image and the background image.

[0113] For specific cloud detection results, please refer to the attached figure Figure 4 , the left half is the image to be tested, and the right half is the cloud detection result finally obtained in this application.

[0114] Specifically, the value at each position on the final confidence map represents the probability that the corresponding point in the image to be measured is a cloud.

[0115] See the attached drawings Figure 2 , based on the method mentioned above, the present application also provides a non-paired weakly supervised cloud detection system based on a Markov discriminator. The cloud detection system includes: an image acquisition and division module, a cloud detection network parameter determination module, a cloud detection network construction module, a label definition module, a first cross-entropy loss calculation and rough confidence map generation module, a second cross-entropy loss calculation and rough confidence map correction module, a cloud detection network parameter update module, a cyclic training module, a test module, and a cloud detection result acquisition module.

[0116] Among them, the image acquisition and division module 11 is used to acquire cloud-containing remote sensing images and cloud-free remote sensing images, and use the cloud-containing remote sensing images and cloud-free remote sensing images as the training set.

[0117] The cloud detection network parameter determination module 12 is used to set the cloud detection network parameters.

[0118] The cloud detection network construction module 13 is used to construct a cloud detection network based on the cloud detection network parameters;

[0119] The label definition module 14 is used to pre-define the label of the cloud-containing remote sensing image as all 0s, and the label of the cloud-free remote sensing image as all 1s.

[0120] The first cross-entropy loss calculation and rough confidence map generation module 15 is used to select m cloud-containing images from the cloud-containing remote sensing images as the first samples and input them into the cloud detection network. With the labels all being 0 for inaccurate supervision, calculate the first cross-entropy loss, and generate a rough confidence map.

[0121] The second cross-entropy loss calculation and rough confidence map correction module 16 is used to select m cloud-free images from the cloud-free remote sensing images as the second samples and input them into the cloud detection network. With the labels all being 1 for inaccurate supervision, calculate the second cross-entropy loss, and correct the rough confidence map by learning the prior features of the second samples.

[0122] The cloud detection network parameter update module 17 is used to perform backpropagation based on the first cross-entropy loss and the second cross-entropy loss to update the network parameters of the cloud detection network.

[0123] The cyclic training module 18 is used to repeat the cloud detection network parameter determination module, the cloud detection network construction module, the label definition module, the first cross-entropy loss calculation and rough confidence map generation module, the second cross-entropy loss calculation and rough confidence map correction module, and the cloud detection network parameter update module until the cloud detection network is trained, and a trained cloud detection network is obtained.

[0124] The test module 19 is used to input the cloudy remote sensing image to be detected into the trained cloud detection network to obtain the final confidence map.

[0125] The cloud detection result acquisition module 20 is used to determine the binary cloud detection result based on the final confidence map.

[0126] In summary, the non-paired weakly supervised cloud detection method and system based on the Markov discriminator in the present invention have the following beneficial effects:

[0127] 1. In this application, cloudy remote sensing images and cloudless remote sensing images are used as the training set, without the need for pixel-level label data, solving the problems of paired supervision that must use paired labels and low segmentation accuracy due to insufficient pixel-level labels, obtaining more accurate cloud detection results, and having a lower operation cost.

[0128] 2. Based on the Markov discriminator, this application classifies by separately inputting cloudy remote sensing images and cloudless remote sensing images to obtain the confidence map, and then uses threshold segmentation to convert the probability distribution map into a mask composed of 0 and 1, obtaining a more accurate cloud detection network model.

[0129] In this specification, each embodiment is described in a progressive manner. The key point of each embodiment is to illustrate the differences from other embodiments. The same or similar parts among the embodiments can be referred to each other. For the system disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method part.

[0130] Specific examples are used in this article to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A non-paired weakly supervised cloud detection method based on a Markov discriminator, characterized in that The cloud detection method includes: Step 1: Obtain cloud-covered remote sensing images and cloud-free remote sensing images, and use the cloud-covered remote sensing images and cloud-free remote sensing images as a training set; Step 2: Set cloud detection network parameters; Step 3: Construct a cloud detection network based on the cloud detection network parameters; Step 4: Pre-define the labels of the cloud-covered remote sensing images as all 0s, and pre-define the labels of the cloud-free remote sensing images as all 1s; Step 5: Select m cloud-covered images from the cloud-covered remote sensing images as the first samples and input them into the cloud detection network. Perform inaccurate supervision with all labels being 0, calculate the first cross-entropy loss, and generate a rough confidence map; Step 6: Select m cloud-free images from the cloud-free remote sensing images as the second samples and input them into the cloud detection network. Perform inaccurate supervision with all labels being 1, calculate the second cross-entropy loss, and correct the rough confidence map by learning the prior features of the second samples; Step 7: Perform backpropagation based on the first cross-entropy loss and the second cross-entropy loss to update the network parameters of the cloud detection network; Step 8: Repeat Step 2 - Step 7 until the cloud detection network is trained, and obtain a trained cloud detection network; Step 9: Input the cloud-covered remote sensing image to be detected into the trained cloud detection network to obtain a final confidence map; Step 10: Determine the binary cloud detection result based on the final confidence map.

2. The unpaired weakly supervised cloud detection method based on a Markov discriminator according to claim 1, wherein The cloud detection network is a spectral normalization Markov discriminator. The Markov discriminator specifically includes: A first convolutional layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer, a fifth convolutional layer, a first spectral normalization layer, a second spectral normalization layer, a third spectral normalization layer, a fourth spectral normalization layer, and a fifth spectral normalization layer; the first convolutional layer, the first spectral normalization layer, the second convolutional layer, the second spectral normalization layer, the third convolutional layer, the third spectral normalization layer, the fourth convolutional layer, the fourth spectral normalization layer, the fifth convolutional layer, and the fifth spectral normalization layer are connected in sequence.

3. The non-paired weakly supervised cloud detection method based on a Markov discriminator according to claim 2, wherein The first spectral normalization layer, the second spectral normalization layer, the third spectral normalization layer, the fourth spectral normalization layer, and the fifth spectral normalization layer all adopt the following formula: Among them, W is the weight parameter matrix of the current layer network, and σ (W) is the maximum singular value of the weight parameter matrix.

4. The unpaired weakly supervised cloud detection method based on a Markov discriminator according to claim 2, wherein The loss function of the first cross-entropy loss is: Among them, D sn represents the spectral normalized Markov discriminator, and C represents the cloud-covered remote sensing images of the mini-batch samples. indicates the probability that the first sample is sampled from the cloud-covered remote sensing images.

5. The method for unpaired weakly supervised cloud detection based on a Markov discriminator according to claim 2, wherein The loss function of the second cross-entropy loss is: Among them, D sn represents the spectral normalized Markov discriminator, N represents the cloud-free remote sensing images of the mini-batch samples, indicates the probability that the second sample is sampled from cloud-free remote sensing images.

6. The unpaired weakly supervised cloud detection method based on a Markov discriminator according to claim 1, characterized in that The training set is cloud-covered remote sensing images and cloud-free remote sensing images that are unlabeled in the cloud region and not paired.

7. The unpaired weakly supervised cloud detection method based on a Markov discriminator according to claim 1, characterized in that The determining the binary cloud detection result based on the final confidence map specifically includes the following steps: Obtain the optimal threshold for the foreground image and the background image; Perform binaryzation on the confidence map according to the optimal threshold to obtain the cloud detection result.

8. The unpaired weakly supervised cloud detection method based on a Markov discriminator according to claim 7, characterized in that The obtaining the optimal threshold for the foreground image and the background image specifically includes the following steps: Calculate the variance of the foreground image and the background image: g = w0 × w1 × (u0 - u1) 2 ; where, the proportion of foreground points in the image is w0, and the average gray level is u0; the proportion of background points in the image is w1, and the average gray level is u1; Take the gray value corresponding to the maximum variance of the foreground image and the background image as the optimal threshold for the foreground image and the background image.

9. The non-paired weakly supervised cloud detection method based on a Markov discriminator according to claim 8, wherein The performing binaryzation on the confidence map according to the optimal threshold to obtain the cloud detection result specifically adopts the following formula: where f(x, y) is the confidence map, and T is the optimal threshold for the foreground image and the background image.

10. A non-paired weakly supervised cloud detection system based on a Markov discriminator, characterized in that The cloud detection system includes: An image acquisition and division module, which is used to acquire cloud-covered remote sensing images and cloud-free remote sensing images, and use the cloud-covered remote sensing images and cloud-free remote sensing images as a training set; A cloud detection network parameter determination module, which is used to set cloud detection network parameters; A cloud detection network construction module, which is used to construct a cloud detection network based on the cloud detection network parameters; A label definition module, which is used to pre-define the label of the cloud-covered remote sensing image as all 0s, and pre-define the label of the cloud-free remote sensing image as all 1s; A first cross-entropy loss calculation and rough confidence map generation module, which is used to select m cloud-covered images from the cloud-covered remote sensing images as the first sample and input them into the cloud detection network, perform inaccurate supervision with all labels being 0, calculate the first cross-entropy loss, and generate a rough confidence map; A second cross-entropy loss calculation and rough confidence map correction module, which is used to select m cloud-free images from the cloud-free remote sensing images as the second sample and input them into the cloud detection network, perform inaccurate supervision with all labels being 1, calculate the second cross-entropy loss, and correct the rough confidence map by learning the prior features of the second sample; A cloud detection network parameter update module, which is used to perform backpropagation based on the first cross-entropy loss and the second cross-entropy loss to update the network parameters of the cloud detection network; A cyclic training module, which is used to repeat the cloud detection network parameter determination module, the cloud detection network construction module, the label definition module, the first cross-entropy loss calculation and rough confidence map generation module, the second cross-entropy loss calculation and rough confidence map correction module, and the cloud detection network parameter update module until the cloud detection network is trained, and obtain a trained cloud detection network; A test module, which is used to input the cloud-covered remote sensing image to be detected into the trained cloud detection network to obtain a final confidence map; A cloud detection result acquisition module, which is used to determine a binary cloud detection result based on the final confidence map.

Images