Cloud detection method for multi-temporal optical remote sensing images based on deep low-rank network

Abstract

The present invention provides a multi-temporal remote sensing image cloud detection method based on a deep low-rank network, which utilizes the physical characteristics of low-rank spatiotemporal spectrum of multi-temporal remote sensing images, learns to reconstruct clear sky cloudless background features in a data-driven manner, and constructs a feature extraction network in stages in combination with the ideas of twin difference and non-local attention, respectively extracts and fuses single-scene spatial spectrum features and multi-temporal differential features, and intelligently realizes multi-temporal remote sensing image cloud coverage detection, and successfully overcomes the problems of lack of existing multi-temporal cloud detection algorithms based on deep learning and weak interpretability. The present invention applies a deep low-rank network model to multi-temporal Landsat remote sensing image cloud detection. Landsat‑8 remote sensing image experiments show that compared with existing remote sensing image cloud detection methods, the multi-temporal remote sensing image cloud detection method based on a deep low-rank network is faster and more accurate, and has strong stability for different scenes and different cloud coverage conditions.

Description

Technical Field

[0001] The present invention relates to the field of cloud detection in multi-temporal optical remote sensing images, and in particular to an automatic cloud detection method for multi-temporal optical remote sensing images based on a deep low-rank network, which combines the low-rank physical prior of multi-temporal images with the feature expression ability of neural networks to realize intelligent multi-temporal remote sensing image cloud coverage detection. Background Art

[0002] Optical remote sensing images are a necessary means for earth resource exploration and ecological environment monitoring. They have the excellent characteristics of short imaging period and wide coverage. They can provide rich information about ground objects, and at the same time intuitively reflect the spatial distribution of ground objects and record the reflection spectrum characteristics of ground objects. They are widely used in many fields such as environmental monitoring, geological survey, urban planning, disaster assessment, etc., and provide effective support for various major strategic decisions.

[0003] Common optical remote sensing satellites include Landsat series satellites, Sentinel series satellites, domestic GF series satellites, QuickBird satellites, MODIS satellites, Hyperion satellites, etc. There are many types of optical satellites. However, according to the global cloud cover data released by the International Satellite Cloud Climate Program (ISCCP), the cloud coverage rate of the earth's surface is about 66%, and the cloud pollution problem in optical remote sensing images is inevitable and very serious. Therefore, the development of high-precision remote sensing image cloud detection methods is conducive to restoring the quality of remote sensing images and improving image utilization. It is of great significance and value for the subsequent development and application of remote sensing images in my country's population, environment, and geography.

[0004] In view of the problem of cloud detection in optical remote sensing images, domestic and foreign scholars have conducted a lot of research and developed a variety of classic remote sensing image cloud detection algorithms. According to the difference in the number of images used, it can be divided into single-view image cloud detection algorithm and multi-temporal image cloud detection algorithm. Single-view image cloud detection algorithm is the most studied and widely used cloud detection method. This method constructs a cloud detection model based on different hypothetical models and prior knowledge, combined with the physical mechanism and statistical characteristics of cloud formation. In contrast, the basic principle of multi-temporal cloud detection algorithm is to comprehensively utilize the spatiotemporal spectral characteristics of remote sensing images and detect clouds and fog based on the difference information between multi-temporal images, that is, cloud pixels can be identified based on the cloud-free background by the difference between the actual observation value and the cloud-free value. Multi-temporal cloud detection algorithm introduces time dimension information on the basis of single-view image, and can usually generate more accurate cloud masks using image information from the same location at different times.

[0005] Existing multi-temporal cloud detection algorithms have made significant progress in recent years. First, an iterative cloud optimization transform algorithm (IHOT) based on the HOT algorithm was proposed. This method successfully overcomes the problem of confusion between bright surface and clouds by introducing cloud-free reference images for regression iteration. In addition, an automatic time series analysis algorithm (ATSA) was proposed for multi-temporal optical image cloud detection. This algorithm can detect cloud pixels through the cloud index of the time series, which is suitable for the situation where the frequency of cloud-free observations in a certain area is low. Furthermore, a multi-temporal integrated cloud z-score algorithm (MTICZ) was proposed for cloud and cloud shadow detection, which detects the location of clouds by analyzing the cloud index of the target image and the time series. Secondly, a model combining U-net deep convolutional neural network and long short-term memory network (LSTM) was also proposed for cloud detection in MSG / SEVIRI images of time series, and achieved high-precision detection results.

[0006] Although a lot of research has been done on cloud detection methods for multi-temporal optical remote sensing images, there are still some key issues that have not been resolved. These include: (1) Existing traditional multi-temporal cloud detection algorithms rely on specific thresholds and lack universality in scenarios with different underlying surfaces and cloud types; (2) Deep learning multi-temporal cloud detection algorithms are underdeveloped and do not take into account the intrinsic physical properties of multi-temporal images, resulting in a lack of interpretability.

[0007] Therefore, considering the low-rank characteristics of the spatiotemporal spectrum of multi-temporal images, a multi-temporal remote sensing image cloud detection method based on a deep low-rank network is proposed, which is very meaningful. Summary of the invention

[0008] The purpose of the present invention is to solve the shortcomings of the existing multi-temporal cloud detection algorithms based on deep learning, which are scarce and weak in interpretability. A multi-temporal remote sensing image cloud detection method based on a deep low-rank network is provided. The method utilizes the physical characteristics of the low-rank spatiotemporal spectrum of multi-temporal remote sensing images to learn and reconstruct the background features of clear sky without clouds in a data-driven manner. A phased feature extraction network is constructed by combining the ideas of twin differences and non-local attention to extract and fuse single-scene spatial spectrum features and multi-temporal differential features, thereby intelligently realizing multi-temporal remote sensing image cloud coverage detection.

[0009] The technical solution of the present invention provides a multi-temporal remote sensing image cloud detection method based on a deep low-rank network, comprising the following steps:

[0010] Step 1, extracting the spatial spectrum features of clouds in the original image to be detected;

[0011] Step 2, extracting low-rank background features of multi-temporal images;

[0012] Step 3: Using the multi-temporal cloud detection network, the original image and the background features obtained in step 2 are input simultaneously. The twin differential structure with shared weights is adopted, combined with the idea of ​​non-local attention, to extract the change information of the original image and background features, and obtain cloud features based on temporal changes, i.e., multi-temporal change feature maps.

[0013] In step 4, the cloud mask is predicted by integrating the spatial spectrum features of the cloud in step 1 and the cloud features based on phase changes in step 3. The network is trained according to the loss function until the training converges and all network parameters reach the optimal state, thus obtaining the final prediction map.

[0014] Moreover, in step 1, the spatial spectrum features of the cloud in the original image to be detected are extracted by designing a single-scene cloud detection network, and the specific implementation method is as follows;

[0015] Let the given original single-view image be Where w and h represent the length and width of the image, and c represents the number of bands. The input of the single-scene cloud detection network is The output is f represents the channel dimension of the output feature map. The single scene cloud detection network is mainly composed of a contraction path and an expansion path. Among them, the contraction path is composed of six stacked encoding modules. The encoding module is divided into two branches, the left branch is the feature extraction branch, which automatically extracts features based on 2D convolutions of two 3*3 convolution kernels. Each convolution layer is usually connected to a ReLU activation function to perform nonlinear mapping on the original function; the right branch is the context retention branch, which uses 1*1 2D convolution to extract the most important feature information, and uses the identity mapping operation to accelerate the back propagation of the gradient flow while retaining low-level feature information. After all features are aggregated, the 2*2 maximum pooling operation is finally used to downsample the features and reduce network parameters to avoid overfitting. Through this encoder module, multi-scale features of the image can be automatically extracted during network training. The expansion path is formed by stacking five decoding modules. In the decoding module, a deconvolution operation is first used to gradually restore the size of the original image. Then, two convolution operations are used to extract the feature information that combines low-level and high-level features. A shortcut connection operation is used to help the network preserve and utilize the contextual information learned from the early layers, so that the network can capture more cloud features. At the same time, the training process is accelerated by preventing the gradient vanishing phenomenon during the network back propagation. Based on this decoding module, the network can fuse multi-level features while stabilizing training and gradually restore the size of the original image. In addition, the jump connection between the contraction branch and the expansion branch of the single-scene cloud detection network can realize the fusion of shallow features and deep features, which helps to generate more accurate cloud masks. In the encoding module and the decoding module, the specific expression of the core convolution operation and nonlinear activation operation is shown in formula (1):

[0016]

[0017] Among them, F in and F out Represents the feature map of input and output. K is the convolution kernel and B is the bias matrix. The symbol * represents the convolution operation. max(0,*) represents the nonlinear activation operation.

[0018] Moreover, in step 2, a low-rank feature extraction module is designed to extract low-rank background features of multi-temporal images. The input of the low-rank feature extraction module is the multi-temporal image, which is denoted as t represents the number of time phases. The output of the low-rank feature extraction module is a low-rank feature map of the same size as the input, that is, a low-rank feature background map with sparse noise removed. First, multiple 2D convolutional layers are stacked together, and the channel dimension is downsampled by 8 times, that is, the rank reduction ratio of the low-rank feature extraction module is 8, and the matrix is ​​obtained after dimensional transformation. Where k is the size of the upper rank. Then, use the same convolution operation and dimension transformation operation, and perform matrix transposition operation to obtain Similarly, for the original matrix After 1*1 convolution and dimension transformation, we get pass and Matrix multiplication to obtain a matrix Considering the unbounded nature of the convolution operation, U (i) = {u (i,1) ,...u (i,wh)}, and V (i) = {v (i,1) ,...v (i,k)}, Normalization is beneficial to the stability of network training. (i,j) As the semantic meaning of the basis, it is normalized using the l2 norm to retain its directionality, as shown in the following formula:

[0019]

[0020] Here, ε is set to 10 -6 Avoid divisors being 0. For the coefficient matrix u (i,j) Use the softmax function to normalize and get the corresponding probability of each basis, as shown in the following formula:

[0021]

[0022] Finally, the low-rank features can be obtained through the matrix and After the low-rank features are transformed into matrix dimensions, the reconstructed low-rank feature graph is obtained. Finally, after a 1*1 convolution, the reconstructed clear sky and cloudless background features can be obtained.

[0023] Moreover, in step 3, the low-rank background feature map obtained by the low-rank feature extraction module is With the original image feature map The two images are input into a multi-temporal cloud detection network based on a twin difference structure to extract differential feature information, that is, to compare the difference between the original image and the low-rank background feature map and extract the cloud mask from the change information. The specific steps are as follows:

[0024] The original image and low-rank background features are extracted through weight-sharing convolutional layers and pooling layers, and the feature maps of the same size extracted from each layer are differentially operated. Then, starting from the deepest differential feature map, upsampling is performed based on the deconvolution block, and the upsampled feature map is connected with the differential feature map of the previous layer, so as to gradually restore the original image size using multi-scale differential semantic features, while retaining the semantic features of the deep layer of the network and the detailed information of the shallow layer. In addition, the global context module (GCB) is introduced after the three differential feature maps in the deeper layer. Its main function is to capture the relationship between long-distance pixels, breaking through the limitation of the traditional convolution operation that only has a local receptive field. For input feature maps of size C*H*W, the specific operation of the GCB module is to first use 1*1 convolution and softmax function to obtain attention weights, and then perform attention pooling to obtain C*1*1 global context features; then use 1x1 convolution for feature conversion to obtain channel dependencies. In order to reduce the number of parameters, a bottleneck form is used for feature transformation, which increases the difficulty of optimization in this process. Therefore, layer normalization is added in front of the ReLU activation function to reduce the difficulty of optimization and improve the generalization of the network as a regularization; finally, the C*1*1 global context features are aggregated to the features of each position for global context modeling. Finally, the network outputs a multi-phase change feature map

[0025] Moreover, in step 4, the input is a multi-phase differential feature map And single scene spatial spectrum feature map Finally, the final prediction graph is obtained by cascading convolution operations and adding sigmoid activation functions.

[0026] In addition, the present invention uses binary cross entropy loss to optimize all network parameters in steps 1 to 4, and the loss is expressed as shown in the following formula.

[0027]

[0028] Where N represents the number of pixels. represents the true label of the i-th pixel, which takes a value of 0 or 1, where 0 represents a non-cloud pixel and 1 represents a cloud pixel. (i) Represents the probability that the i-th pixel predicted by the network is a cloud pixel. Save the model when the loss is optimized to the lowest state.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] The present invention proposes a multi-temporal remote sensing image cloud detection method based on a deep low-rank network, which solves the problem that most existing multi-temporal cloud detection methods rely on physical thresholds and are not universal, and that there are few multi-temporal methods based on deep learning and lack interpretability. The low-rank feature extraction module constrains the separation of clouds and backgrounds during network training, reconstructs background features, and intelligently extracts deep change feature information of clouds based on improved convolution blocks, combined with the spatial spectrum feature information of the target image, to achieve automated multi-temporal image cloud detection. Compared with existing remote sensing image cloud detection methods, the multi-temporal remote sensing image cloud detection method based on a deep low-rank network is faster and more accurate, and has strong robustness and stability for different scenes and different cloud coverage conditions. Therefore, the multi-temporal remote sensing image cloud detection method based on a deep low-rank network not only has very important academic value, but also has important practical significance. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is the overall flow chart of the present invention;

[0032] Figure 2 It is a single scene cloud detection network structure diagram of the present invention;

[0033] Figure 3 It is a structural diagram of the coding module in the single scene cloud detection network of the present invention;

[0034] Figure 4 It is a structural diagram of a decoding module in a single-view cloud detection network of the present invention;

[0035] Figure 5 It is a low-rank feature extraction module diagram of the present invention;

[0036] Figure 6 This is a diagram of the multi-temporal cloud detection network structure of the present invention. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solution and advantages of the present invention more clear, the following is a further detailed description of a deep low-rank network cloud detection method according to an embodiment of the present invention in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0038] In response to the problems of lack of multi-temporal cloud detection algorithms based on deep learning and weak interpretability, the present invention proposes a multi-temporal remote sensing image cloud detection method based on deep low-rank network. The method utilizes the physical characteristics of low-rank spatiotemporal spectrum of multi-temporal remote sensing images to learn and reconstruct the background features of clear sky without clouds in a data-driven manner. The method combines the ideas of twin difference and non-local attention to construct a staged feature extraction network, which respectively extracts and fuses single-scene spatial spectrum features and multi-temporal differential features, and intelligently realizes cloud cover detection from multi-temporal remote sensing images.

[0039] The present invention can be implemented by computer software technology. Figure 1 The specific steps of the method for detecting deep low-rank network cloud in an image of an embodiment are described in detail.

[0040] Step 1: Design a single cloud detection network, such as Figure 2-4 As shown, the original image to be detected is input to extract the spatial spectrum characteristics of the cloud;

[0041] Step 2: Design a low-rank feature extraction module, such as Figure 5 As shown, multi-temporal images are input into the module, and the low-rank background features of the multi-temporal images are learned in the network based on the matrix decomposition and reconstruction process;

[0042] Step 3: Design a multi-temporal cloud detection network, such as Figure 6 As shown in the figure, the original image and the background features obtained in step 2 are input simultaneously, and the twin difference structure with shared weights is adopted, combined with the idea of ​​non-local attention, to extract the change information of the original image and background features, and obtain cloud features based on phase changes;

[0043] In step 4, the cloud mask is predicted by integrating the spatial spectrum features of the cloud in step 1 and the cloud features based on phase changes in step 3. The network is trained according to the loss function until the training converges and all network parameters reach the optimal state, thus obtaining the final prediction map.

[0044] Moreover, in step 1, the given original single-view image is recorded as Where w and h represent the length and width of the image, and c represents the number of bands. The input of the single-scene cloud detection network is The output is f represents the channel dimension of the output feature map. The single scene cloud detection network is mainly composed of a contraction path and an expansion path. Among them, the contraction path is composed of six stacked encoding modules. The encoding module is divided into two branches, the left branch is the feature extraction branch, which automatically extracts features based on 2D convolutions of two 3*3 convolution kernels. Each convolution layer is usually connected to a ReLU activation function to perform nonlinear mapping on the original function; the right branch is the context retention branch, which uses 1*1 2D convolution to extract the most important feature information, and uses the identity mapping operation to accelerate the back propagation of the gradient flow while retaining low-level feature information. After all features are aggregated, the 2*2 maximum pooling operation is finally used to downsample the features and reduce network parameters to avoid overfitting. Through this encoder module, multi-scale features of the image can be automatically extracted during network training. The expansion path is formed by stacking five decoding modules. In the decoding module, a deconvolution operation is first used to gradually restore the size of the original image. Then, two convolution operations are used to extract the feature information that combines low-level and high-level features. A shortcut connection operation is used to help the network preserve and utilize the contextual information learned from the early layers, so that the network can capture more cloud features. At the same time, the training process is accelerated by preventing the gradient vanishing phenomenon during the network back propagation. Based on this decoding module, the network can fuse multi-level features while stabilizing training and gradually restore the size of the original image. In addition, the jump connection between the contraction branch and the expansion branch of the single-scene cloud detection network can realize the fusion of shallow features and deep features, which helps to generate more accurate cloud masks. In the encoding module and the decoding module, the specific expression of the core convolution operation and nonlinear activation operation is shown in formula (1):

[0045]

[0046] Among them, F in and F out Represents the feature map of input and output. K is the convolution kernel and B is the bias matrix. The symbol * represents the convolution operation. max(0,*) represents the nonlinear activation operation.

[0047] The image in the embodiment is Landsat data, the image size is 512×512, and contains 4 bands, namely red, green, blue and near infrared bands. The final output spatial spectrum feature of the single scene cloud detection network

[0048] In step 2, the input of the low-rank feature extraction module is a multi-temporal image, denoted as t represents the number of time phases. The output of the low-rank feature extraction module is a low-rank feature map of the same size as the input, that is, a low-rank feature background map with sparse noise removed. First, multiple 2D convolutional layers are stacked together, and the channel dimension is downsampled by 8 times, that is, the rank reduction ratio of the low-rank feature extraction module is 8, and the matrix is ​​obtained after dimensional transformation. Where k is the size of the upper rank. Then, use the same convolution operation and dimension transformation operation, and perform matrix transposition operation to obtain Similarly, for the original matrix After 1*1 convolution and dimension transformation, we get pass and Matrix multiplication to obtain a matrix Considering the unbounded nature of the convolution operation, U (i) = {u (i,1) ,...u (i,wh)}, and V (i) = {v (i,1) ,...v (i,k)}, Normalization is beneficial to the stability of network training. (i,j) As the semantic meaning of the basis, it is normalized using the l2 norm to retain its directionality, as shown in the following formula:

[0049]

[0050] Here, ε is set to 10 -6 Avoid divisors being 0. For the coefficient matrix u (i,j) Use the softmax function to normalize and get the corresponding probability of each basis, as shown in the following formula:

[0051]

[0052] Finally, the low-rank features can be obtained through the matrix and After the low-rank features are transformed into matrix dimensions, the reconstructed low-rank feature graph is obtained. Finally, after a 1*1 convolution, the reconstructed clear sky and cloudless background features can be obtained.

[0053] The multi-phase image in the embodiment includes 16 phases, that is, t is 16. The input image block After the feature is reduced in rank, the rank k is set to 8. In addition, ε is set to 10 -6 Avoid being divided by 0. The final output has a clear sky with no clouds. Keep the same dimensions as the original image.

[0054] In step 3, the low-rank background feature map obtained by the low-rank feature extraction module is With the original image feature map The two images are input into a multi-temporal cloud detection network based on a twin difference structure to extract differential feature information, that is, to compare the difference between the original image and the low-rank background feature map and extract the cloud mask from the change information. The specific steps are as follows:

[0055] The original image and low-rank background features are extracted through weight-sharing convolutional layers and pooling layers, and the feature maps of the same size extracted from each layer are differentially operated. Then, starting from the deepest differential feature map, upsampling is performed based on the deconvolution block, and the upsampled feature map is connected with the differential feature map of the previous layer, so as to gradually restore the original image size using multi-scale differential semantic features, while retaining the semantic features of the deep layer of the network and the detailed information of the shallow layer. In addition, the global context module (GCB) is introduced after the three differential feature maps in the deeper layer. Its main function is to capture the relationship between long-distance pixels, breaking through the limitation of the traditional convolution operation that only has a local receptive field. For input feature maps of size C*H*W, the specific operation of the GCB module is to first use 1*1 convolution and softmax function to obtain attention weights, and then perform attention pooling to obtain C*1*1 global context features; then use 1x1 convolution for feature conversion to obtain channel dependencies. In order to reduce the number of parameters, a bottleneck form is used for feature transformation, which increases the difficulty of optimization in this process. Therefore, layer normalization is added in front of the ReLU activation function to reduce the difficulty of optimization and improve the generalization of the network as a regularization; finally, the C*1*1 global context features are aggregated to the features of each position for global context modeling. Finally, the network outputs a multi-phase change feature map

[0056] In the embodiment, the original image is input simultaneously and background characteristics Feature differentiation is performed through weight sharing of the twin network, and the multi-phase change feature map is finally output Single scene spatial spectrum characteristics The dimensions remain consistent.

[0057] In step 4, the input is multi-phase differential features and single scene spatial spectrum characteristics Finally, the final prediction graph is obtained by cascading convolution operations and adding sigmoid activation functions. In addition, the present invention uses binary cross entropy loss to optimize all network parameters in steps 1 to 4, and the loss is expressed as shown in the following formula.

[0058]

[0059] Where N represents the number of pixels. represents the true label of the i-th pixel, which takes a value of 0 or 1, where 0 represents a non-cloud pixel and 1 represents a cloud pixel. (i) Represents the probability that the i-th pixel predicted by the network is a cloud pixel. Save the model when the loss is optimized to the lowest state.

[0060] In this embodiment, the input features are and Final output binary prediction map The entire network training is implemented under the Keras deep learning framework. The batch size in the network training process is set to 4, the optimization method used is the Adam gradient descent method, the initial learning rate of the model training is set to 1e-4, and the adaptive learning rate decay strategy is adopted during the training process. The decay rate is 0.7 and the patience factor is 15. This strategy continues until the learning rate reaches a value of 1e-8. In specific implementation, those skilled in the art can adjust the hyperparameters of the network according to the specific images used.

[0061] The embodiments of the present invention use Landsat remote sensing images, but are not limited to Landsat remote sensing images. For other multi-temporal optical remote sensing images, regardless of the cloud pollution intensity, image spatial resolution or number of bands, it has wide versatility and is less restricted by objective factors. The experimental results of Landsat-8 remote sensing images show that the overall accuracy of this method reaches 96.4%, and the kappa coefficient is 0.93. Compared with classic cloud detection algorithms such as Fmask, it effectively reduces the false detection rate of highlighted surfaces, improves cloud detection accuracy, and realizes intelligent cloud detection that is universal for multiple scenarios.

[0062] It should be noted and understood that various modifications and improvements can be made to the present invention described in detail above without departing from the spirit and scope of the present invention required by the appended claims. Therefore, the scope of the technical solution claimed is not limited by any specific exemplary teaching given.

Claims

1. A multi-temporal remote sensing image cloud detection method based on a deep low-rank network, characterized in that: The following steps are involved: Step 1, extracting the spatial spectrum features of clouds in the original image to be detected; In step 1, a single-scene cloud detection network is designed to extract the spatial spectrum features of clouds in the original image to be detected. The specific implementation method is as follows; The given original single-view image is recorded as Where w and h represent the length and width of the image, c represents the number of bands, and the input of the single-scene cloud detection network is The output is f represents the output feature map P single The channel dimension of The single scene cloud detection network consists of a contraction path and an expansion path. The contraction path is composed of six stacked encoding modules. The encoding module is divided into two branches, the left branch is the feature extraction branch, which automatically extracts features based on 2D convolutions of two 3×3 convolution kernels. Each convolution layer is connected to a ReLU activation function to perform nonlinear mapping on the original function; the right branch is the context retention branch, which uses 1×1 2D convolution to extract feature information, and uses the identity mapping operation to accelerate the back propagation of the gradient flow while retaining low-level feature information. After the features are aggregated, the 2×2 maximum pooling operation is finally used to achieve feature downsampling and reduce network parameters to avoid overfitting; the encoder module automatically extracts multi-scale features of the image; The expansion path is formed by stacking five decoding modules. In the decoding module, a deconvolution operation is first used to gradually restore the size of the original image. Then, two convolution operations are used to extract feature information. A shortcut connection operation is used to help the network preserve and utilize contextual information learned from early layers, so that the network can capture more cloud features and accelerate the training process by preventing the gradient from disappearing during the network back propagation. Based on the decoding module, the network can fuse multi-level features while stabilizing training and gradually restore the size of the original image. Step 2, extracting low-rank background features of multi-temporal images; Step 3: Using the multi-temporal cloud detection network, the original image and the background features obtained in step 2 are input simultaneously. The twin differential structure with shared weights is adopted, combined with the idea of ​​non-local attention, to extract the change information of the original image and background features, and obtain cloud features based on temporal changes, i.e., multi-temporal change feature maps. In step 4, the cloud spatial spectrum features in step 1 and the cloud features based on phase changes in step 3 are integrated to predict cloud masks. The network is trained according to the loss function until the training converges and the network parameters reach the optimal value, thus obtaining the final prediction map.

2. The multi-temporal remote sensing image cloud detection method based on a deep low-rank network as claimed in claim 1, characterized in that: The jump connection between the contraction path and the expansion path of the single-scene cloud detection network can realize the fusion of shallow features and deep features, which helps to generate more accurate cloud masks. In the encoding module and decoding module, the specific expression of the core convolution operation and nonlinear activation operation is shown in formula (1): Among them, F in and F out Represents the feature map of input and output, K is the convolution kernel, B is the bias matrix, the symbol * represents the convolution operation, and max represents the non-linear activation operation.

3. The multi-temporal remote sensing image cloud detection method based on deep low-rank network as claimed in claim 1, characterized in that: In step 2, a low-rank feature extraction module is designed to extract low-rank background features of multi-temporal images. The input of the low-rank feature extraction module is the multi-temporal image, denoted as Where w and h represent the length and width of the image, c represents the number of bands, and t represents the number of phases. The output of the low-rank feature extraction module is a low-rank feature map of the same size as the input, that is, a low-rank feature background map with sparse noise removed. The processing process of designing the low-rank feature extraction module is as follows; First, multiple 2D convolutional layers are stacked together, and the channel dimension is downsampled by 8 times, that is, the rank reduction ratio of the low-rank feature extraction module is 8, and the matrix is ​​obtained after dimensional transformation Where k is the size of the upper rank; then, use the same convolution operation and dimension transformation operation, and perform matrix transposition operation to obtain Similarly, for the original matrix After 1×1 convolution and dimension transformation, we get pass and Matrix multiplication to obtain a matrix Considering the unbounded nature of the convolution operation, in this low-rank feature extraction module and After normalization, we get and It is conducive to the stability of network training; finally, the low-rank features are obtained through the matrix and The product of is: After the low-rank features are transformed into matrix dimensions, the reconstructed low-rank feature graph is obtained. Finally, after a 1×1 convolution, the reconstructed clear sky and cloudless background features can be obtained.

4. The multi-temporal remote sensing image cloud detection method based on a deep low-rank network as claimed in claim 3, characterized in that: In order to keep v (i,j) As the semantic meaning of the basis, it is normalized using the l2 norm to retain its directionality, as shown in the following formula: Here, ε is set to 10 -6 Avoid the divisor being 0; for the coefficient matrix u (i,j) Use the softmax function to normalize and get the corresponding probability of each basis, as shown in the following formula:

5. The multi-temporal remote sensing image cloud detection method based on deep low-rank network as claimed in claim 1, characterized in that: In step 3, the low-rank background feature map obtained by the low-rank feature extraction module is With the original image feature map The two images are input together into a multi-temporal cloud detection network based on a twin difference structure to extract differential feature information, that is, to compare the difference between the original image and the low-rank background feature map, and extract the cloud mask from the change information, where w and h represent the length and width of the image, and c represents the number of bands; the specific steps are as follows: The original image and low-rank background features are extracted through weight-sharing convolutional layers and pooling layers, and the feature maps of the same size extracted from each layer are differentially processed. Then, upsampling is performed based on the deconvolution block starting from the deepest differential feature map, and the upsampled feature map is connected with the differential feature map of the previous layer, so as to gradually restore the original image size using multi-scale differential semantic features, while retaining the semantic features of the deep layer of the network and the detailed information of the shallow layer. The global context module GCB is introduced after the three differential feature maps in the deeper layer. Its function is to capture the relationship between long-distance pixels and break through the limitation of the traditional convolution operation that only has a local receptive field. For the input feature map of size C×H×W, the specific operation of the GCB module is to first use 1×1 convolution and softmax function to obtain the attention weight, and then perform attention pooling to obtain the global context feature of size C×1×1. Subsequently, 1×1 convolution is used to transform the features to obtain the inter-channel dependency. In order to reduce the number of parameters, the bottleneck form is used for feature transformation, which increases the difficulty of optimization in this process. Therefore, layer normalization is added before the ReLU activation function to reduce the difficulty of optimization and improve the generalization of the network as a regularizer. Finally, the global context features of size C×1×1 are aggregated to the features of each position for global context modeling. Finally, the multi-temporal cloud detection network outputs a multi-temporal change feature map. f' represents the feature map P multi The channel dimension.

6. The multi-temporal remote sensing image cloud detection method based on deep low-rank network according to claim 1, characterized in that: In step 4, the cloud spatial spectrum features in step 1 and the cloud features based on phase changes in step 3 are integrated, and finally the final prediction map is obtained through cascade convolution operation and sigmoid activation function. Where w and h represent the length and width of the image.

7. The multi-temporal remote sensing image cloud detection method based on deep low-rank network as claimed in claim 1, characterized in that: The binary cross entropy loss is used to optimize the network parameters in steps 1 to 4. The loss is expressed as follows: Where N represents the number of pixels. represents the true label of the i'th pixel, which takes a value of 0 or 1, where 0 represents a non-cloud pixel and 1 represents a cloud pixel. (i') Represents the probability that the i'th pixel predicted by the network is a cloud pixel. When the loss is optimized to the minimum state, the model is saved.

Images