Remote sensing image anti-cloud interference change detection method based on comparative learning

Abstract

The invention discloses a remote sensing image anti-cloud interference change detection method based on comparative learning. The method comprises the following steps: constructing a sample set; training a feature extractor by adopting the sample set to obtain a trained feature extractor; constructing a change detection network by adopting a trained feature extractor, a feature processing module, a space attention module, a converter module, a channel attention module, a connection module and a classifier; the two detection images are input into a change detection network, the shooting time of the two detection images is different, the shooting areas of the two detection images are the same, and the change detection network outputs a prediction change graph. The global features and the local features of the image are fused through the change detection network constructed by the trained feature extractor, and the efficiency and the precision are improved by adopting the space attention module and the channel attention module, so that quick and accurate detection can be realized even in a remote sensing image change detection task with cloud interference, and the detection efficiency is improved. And the robustness during remote sensing image change detection is improved.

Description

Technical Field

[0001] The present invention relates to the field of remote sensing image change detection, and more specifically, to a remote sensing image anti-cloud and fog interference change detection method based on contrast learning. Background Art

[0002] Bi-temporal change detection is a key task in remote sensing, which involves comparing and identifying changes between registered remote sensing images of the same area in different phases. It has wide applications in disaster assessment, urban planning, agricultural surveys, resource management, and environmental monitoring. Pseudo-change is a long-standing problem in the field of change detection due to factors such as complex textures, seasonal changes, climate change, and changing needs. Pseudo-change refers to the phenomenon that unchanged areas in remote sensing images of different phases are mistakenly identified as changed areas due to other differences. Typical causes of pseudo-change can be divided into color changes, temporary objects, shadow shapes, and cloud interference. These pseudo-change problems cause shadows and projection differences in unchanged areas to be mistakenly detected as changed areas. Among them, pseudo-change caused by thin cloud interference is a type of problem that deserves special attention. Existing remote sensing image change detection datasets generally do not introduce the influence of thin cloud interference, resulting in most change detection methods created based on these datasets not considering the influence of thin cloud interference.

[0003] Traditional methods based on manual feature variation can achieve good results in simple scenarios, but usually perform poorly in complex scenarios.

[0004] Algorithms based on deep convolutional neural networks (CNNs) perform better. Deep convolutional neural networks are widely used to extract discriminative local features in change detection, including classic convolutional neural networks and their extended architectures, such as ResNet (residual network) and UNet (image segmentation network).

[0005] Compared with pure convolutional neural networks, algorithms based on the Transformer structure have achieved impressive results in change detection tasks by effectively modeling global context information through the encoder-decoder architecture. However, it is limited by the limitations of the Transformer itself and has limited use of local features.

[0006] Algorithms that combine CNN and Transformer have been developed, such as BIT-CD (change detection model), which has achieved good performance in change detection tasks. However, the performance of change detection networks based on this idea still has a lot of room for improvement, especially in the problem of pseudo-changes caused by cloud interference.

[0007] The Chinese patent application document (application number: 201410441207.2, application date: 2014.09.01) discloses a remote sensing image change detection method. After inputting the remote sensing images before and after the change, it is necessary to determine whether the remote sensing image is foggy, and then perform recognition after defogging the foggy remote sensing image. This method requires a lot of processing and calculation in the process of image judgment and defogging operations, and the processing efficiency is low.

[0008] Based on this, there is an urgent need to provide a method that can resist cloud and fog interference and realize change detection accurately and quickly. Summary of the invention

[0009] In view of this, the present invention provides a remote sensing image anti-cloud and fog interference change detection method based on contrast learning.

[0010] The present invention provides a remote sensing image anti-cloud and fog interference change detection method based on contrast learning, comprising:

[0011] Constructing a sample set, the sample set comprising remote sensing images and foggy images, the number of the remote sensing images is at least two, any two of the remote sensing images are different, the remote sensing images correspond to the foggy images one by one, and the foggy images are obtained from the corresponding remote sensing images;

[0012] Using the sample set to train a feature extractor to obtain a trained feature extractor;

[0013] Build a change detection network, including:

[0014] Constructing a subnetwork, including: providing the trained feature extractor to receive an input image, extracting a 32×32×32 feature map, a 32×64×64 feature map and a 32×128×128 feature map according to the input image, upsampling the 32×32×32 feature map to obtain an upsampled feature, and downsampling the 32×128×128 feature map to obtain a downsampled feature; connecting the input end of a feature processing module to the output end of the trained feature extractor to fuse the 32×64×64 feature map, the upsampled feature and the downsampled feature to obtain a fused feature map; connecting the input end of a spatial attention module to the output end of the feature processing module, connecting the input end of a converter module to the output end of the spatial attention module, and connecting the input end of a channel attention module to the output end of the converter module, so as to process the fused feature map in sequence through the spatial attention module, the converter module and the channel attention module to obtain a final feature map;

[0015] providing two of said sub-networks;

[0016] The input end of the connection module is connected to the output end of the channel attention module of the two sub-networks respectively to obtain the two final feature maps, and the two final feature maps are connected in pairs along the channel dimension to obtain a total map;

[0017] Connecting the input end of the classifier to the output end of the connection module to output a predicted change map according to the overall map;

[0018] Two detection images are provided, wherein the two detection images are taken at different times and in the same area, and the two detection images are input into the change detection network as the two input images. The input end of the trained feature extractor in the change detection network corresponds one-to-one to the input image, and the change detection network outputs the corresponding predicted change map.

[0019] Optionally, constructing the sample set includes:

[0020] The cloud layer simulation image is obtained by combining Perlin noise with fractal Brownian motion.

[0021] Providing at least two of the remote sensing images;

[0022] Each of the remote sensing images is fused with the cloud simulation image to obtain the foggy image corresponding to the remote sensing image;

[0023] All the remote sensing images and all the foggy images constitute the sample set.

[0024] Optionally, the remote sensing image is fused with the cloud simulation image to obtain the foggy image corresponding to the remote sensing image, which is calculated in the following manner:

[0025] I(x)=J(x)t(x)+A(1-t(x))

[0026] Wherein, J(x) is the remote sensing image, t(x) is a parameter representing the light that is not scattered and reaches the camera, A is the cloud simulation image, and I(x) is the foggy image corresponding to the remote sensing image.

[0027] Optionally, using the sample set to train the feature extractor to obtain the trained feature extractor includes:

[0028] Providing two of the feature extractors;

[0029] Randomly select two images from the sample set, the selected two images are any two remote sensing images, or the selected two images are any two foggy images, or one of the two images is any one remote sensing image and the other is any one foggy image;

[0030] Input the two selected images into the two feature extractors, the feature extractors correspond to the images one by one, the feature extractors output feature vectors according to the input images, and obtain the two feature vectors;

[0031] Determine whether there is a corresponding relationship between the two selected images, if there is a corresponding relationship between the two images, set the label value to 0, if there is no corresponding relationship between the two images, set the label value to 1;

[0032] Calculate a loss function based on the two feature vectors and the label value;

[0033] The feature extractor is trained according to the loss function.

[0034] Optionally, the loss function is calculated according to the two feature vectors and the label value in the following manner:

[0035] loss=(1-label)euclidean(f1,f2) 2 +label(1-euclidena(f1,f2)) 2

[0036] Among them, loss is the loss value, label is the label value, euclidean(f1, f2) is the normalized Euclidean distance between the two feature vectors, f1 is the feature vector corresponding to one of the selected images, and f2 is the feature vector corresponding to the other selected image.

[0037] Optionally, the fused feature map is processed sequentially by the spatial attention module, the converter module, and the channel attention module to obtain the final feature map, including:

[0038] The spatial attention module processes the fused feature map to obtain a variable feature map;

[0039] The converter module and the channel attention module process the variable feature map to obtain the final feature map.

[0040] Optionally, the spatial attention module processes the fused feature map to obtain the variable feature map, which is calculated in the following manner:

[0041]

[0042] in, is the fusion feature map, σ is the activation function, f is the convolution kernel operation, Maxpool is the maximum pooling operation, Avgpool is the average pooling operation, is the two-dimensional spatial attention, is element-wise multiplication, is the variable characteristic diagram.

[0043] Optionally, the converter module and the channel attention module process the variable feature map to obtain the final feature map, which is calculated as follows:

[0044]

[0045] M c =MLP(M i )

[0046]

[0047] Among them, Maxpool is the maximum pooling operation, is the variable feature map, T() is the converter module processing, ⊕ is element-by-element addition, MLP is multi-layer perception processing, is element-wise multiplication, is the final feature map.

[0048] Compared with the prior art, the remote sensing image anti-cloud and fog interference change detection method based on contrast learning provided by the present invention achieves at least the following beneficial effects:

[0049] The present invention provides a remote sensing image anti-cloud and fog interference change detection method based on contrastive learning, including constructing a sample set; using the sample set to train a feature extractor to obtain a trained feature extractor; using the trained feature extractor, feature processing module, spatial attention module, converter module, channel attention module, connection module and classifier to construct a change detection network; inputting two detection images into the change detection network, the two detection images are shot at different times and in the same area, and the change detection network outputs a predicted change map. The feature extractor is trained using a sample set containing remote sensing images and foggy images to obtain a trained feature extractor, so that the trained feature extractor has a good ability to resist cloud interference; the change detection network constructed using the trained feature extractor fuses the global features and local features of the image, and uses a spatial attention module and a channel attention module to improve efficiency and accuracy, so that even in the remote sensing image change detection task with cloud interference, fast and accurate detection can be achieved, and the robustness of remote sensing image change detection is improved.

[0050] Of course, any product implementing the present invention does not necessarily need to achieve all of the technical effects described above at the same time.

[0051] Further features and advantages of the present invention will become apparent from the following detailed description of exemplary embodiments of the present invention with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.

[0053] Figure 1 It is a flow chart of a remote sensing image anti-cloud and fog interference change detection method based on contrast learning provided by the present invention.

[0054] Figure 2 It is a structural diagram of a change detection network.

[0055] Figure 3 It is another flow chart of the remote sensing image anti-cloud and fog interference change detection method based on contrast learning provided by the present invention.

[0056] Figure 4 It is an effect diagram that simulates clouds in remote sensing images.

[0057] Figure 5 It is a flowchart of contrastive learning.

[0058] Figure 6 A schematic diagram of the spatial attention module.

[0059] Figure 7 This is a schematic diagram of the channel attention module.

[0060] In the figure: 1. Subnetwork; 2. Trained feature extractor; 3. Feature processing module; 4. Spatial attention module; 5. Converter module; 6. Channel attention module; 7. Connection module; 8. Classifier. DETAILED DESCRIPTION

[0061] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present invention unless otherwise specifically stated.

[0062] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.

[0063] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.

[0064] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0065] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0066] Example 1

[0067] Combination Figure 1 and Figure 2 , Figure 1 It is a flow chart of a remote sensing image anti-cloud and fog interference change detection method based on contrast learning provided by the present invention. Figure 2 1 is a structural diagram of a change detection network, which is used to illustrate a specific embodiment of the remote sensing image anti-cloud and fog interference change detection method based on contrast learning provided by the present invention, including:

[0068] S101: construct a sample set, the sample set includes remote sensing images and foggy images, the number of remote sensing images is at least two, any two remote sensing images are different, the remote sensing images and the foggy images correspond one to one, and the foggy images are obtained from the corresponding remote sensing images;

[0069] S102: training a feature extractor using the sample set to obtain a trained feature extractor;

[0070] S103: Build a change detection network (ACCDNet), including:

[0071] S1031: Constructing a subnetwork, including: providing a trained feature extractor for receiving an input image, extracting a 32×32×32 feature map, a 32×64×64 feature map, and a 32×128×128 feature map according to the input image, upsampling the 32×32×32 feature map to obtain an upsampled feature, and downsampling the 32×128×128 feature map to obtain a downsampled feature; connecting an input end of a feature processing module to an output end of the trained feature extractor, for fusing the 32×64×64 feature map, the upsampled feature, and the downsampled feature to obtain a fused feature map; connecting an input end of a spatial attention module to an output end of the feature processing module, connecting an input end of a converter module to an output end of the spatial attention module, and connecting an input end of a channel attention module to an output end of the converter module, for processing the fused feature map in sequence through the spatial attention module, the converter module, and the channel attention module to obtain a final feature map;

[0072] S1032: Provide two sub-networks;

[0073] S1033: Connecting the input end of the connection module to the output end of the channel attention module of the two sub-networks respectively to obtain two final feature maps, and connecting the two final feature maps in pairs along the channel dimension to obtain a total map;

[0074] S1034: connecting the input end of the classifier to the output end of the connection module to output a predicted change graph according to the overall graph;

[0075] S104: Provide two detection images, which are taken at different times but in the same area, and input the two detection images as two input images into a change detection network. The input end of the trained feature extractor in the change detection network corresponds one-to-one to the input image, and the change detection network outputs a corresponding predicted change map.

[0076] It should be noted that in this embodiment, the feature extractor uses ResNet-50, which can perform multi-scale feature extraction on the input image. Of course, it is not limited to this, and the feature extractor can be selected according to actual needs. Figure 2 In this embodiment, the two detection images are respectively denoted as I1 and I2, the trained feature extractor is pre-trainedResNet-50, the spatial attention module is SAM, the converter module is Transformer, the converter module includes an encoder encoder and a decoder decoder, the channel attention module is CAM, and the classifier is Classifier. In step S1032, two sub-networks are provided to construct a twin network structure, and the weights are shared between the two sub-networks so that the two sub-networks have the same weights.

[0077] It can be understood that for the input image I i (i∈{1,2})∈R 3×H×W , use the pre-trained ResNet-50 backbone to extract feature maps of three different scales, respectively and is a 32×32×32 feature map, is a 32×64×64 feature map, is a 32×128×128 feature map. Upsampling obtains upsampling features, Downsampling obtains downsampled features, and upsampled features, downsampled features and get is the fusion feature map. Input into the spatial attention module SAM is the variable characteristic diagram. Input into the converter module Transformer and the channel attention module CAM in turn, and get The final feature maps of the same scale from the two sub-networks are connected in pairs along the channel dimension and input into the classifier to obtain the predicted change map.

[0078] The present embodiment provides a remote sensing image anti-cloud and fog interference change detection method based on contrastive learning, including constructing a sample set; using the sample set to train a feature extractor to obtain a trained feature extractor; using the trained feature extractor, feature processing module, spatial attention module, converter module, channel attention module, connection module and classifier to construct a change detection network; inputting two detection images into the change detection network, the two detection images are shot at different times and in the same area, and the change detection network outputs a predicted change map. The feature extractor is trained using a sample set containing remote sensing images and foggy images to obtain a trained feature extractor, so that the trained feature extractor has a good ability to resist cloud interference; the change detection network constructed using the trained feature extractor fuses the global features and local features of the image, and uses the spatial attention module and the channel attention module to improve efficiency and accuracy, so that even in the remote sensing image change detection task with cloud interference, fast and accurate detection can be achieved, and the robustness of remote sensing image change detection can be improved.

[0079] Example 2

[0080] Combination Figure 2 , Figure 3 , Figure 4 , Figure 5 , Figure 6 and Figure 7 , Figure 3 is another flow chart of the remote sensing image anti-cloud and fog interference change detection method based on contrast learning provided by the present invention, Figure 4 It is an effect diagram of simulating clouds in remote sensing images. Figure 5 It is a flowchart of contrastive learning. Figure 6 is a schematic diagram of the spatial attention module. Figure 7 : is a schematic diagram of a channel attention module to illustrate another specific embodiment of the remote sensing image anti-cloud and fog interference change detection method based on contrast learning provided by the present invention, including:

[0081] S201: construct a sample set, the sample set includes remote sensing images and foggy images, the number of remote sensing images is at least two, any two remote sensing images are different, the remote sensing images and the foggy images correspond one to one, and the foggy images are obtained from the corresponding remote sensing images.

[0082] S202: Using the sample set to train a feature extractor to obtain a trained feature extractor, including:

[0083] S2021: Provides two feature extractors;

[0084] S2022: randomly selecting two images from the sample set, where the selected two images are any two remote sensing images, or the selected two images are any two foggy images, or one of the two selected images is any remote sensing image and the other is any foggy image;

[0085] S2023: input the two selected images into two feature extractors, the feature extractors correspond to the images one by one, and the feature extractors output feature vectors according to the input images to obtain two feature vectors;

[0086] S2024: Determine whether there is a corresponding relationship between the two selected images. If there is a corresponding relationship between the two images, set the label value to 0; if there is no corresponding relationship between the two images, set the label value to 1;

[0087] S2025: Calculate the loss function based on the two feature vectors and the label value, as follows:

[0088] loss=(1-label)euclidean(f1,f2) 2 +label(1-euclidean(f1,f2)) 2

[0089] Where loss is the loss value, label is the label value, euclidean(f1, f2) is the normalized Euclidean distance between two feature vectors, f1 is the feature vector corresponding to one selected image, and f2 is the feature vector corresponding to another selected image;

[0090] S2026: Train the feature extractor according to the loss function.

[0091] S203: Building a change detection network, including:

[0092] S2031: constructing a subnetwork, including: providing a trained feature extractor for receiving an input image, extracting a 32×32×32 feature map, a 32×64×64 feature map and a 32×128×128 feature map according to the input image, upsampling the 32×32×32 feature map to obtain an upsampled feature, and downsampling the 32×128×128 feature map to obtain a downsampled feature; connecting an input end of a feature processing module to an output end of the trained feature extractor, for fusing the 32×64×64 feature map, the upsampled feature and the downsampled feature to obtain a fused feature map; connecting an input end of a spatial attention module to an output end of the feature processing module, connecting an input end of a converter module to an output end of the spatial attention module, and connecting an input end of a channel attention module to an output end of a converter module, for processing the fused feature map in sequence through the spatial attention module, the converter module and the channel attention module to obtain a final feature map;

[0093] S2032: Provide two sub-networks;

[0094] S2033: Connecting the input end of the connection module to the output end of the channel attention module of the two sub-networks respectively to obtain two final feature maps, and connecting the two final feature maps in pairs along the channel dimension to obtain a total map;

[0095] S2034: Connecting the input end of the classifier to the output end of the connection module to output a predicted change graph based on the overall graph.

[0096] S204: providing two detection images, wherein the two detection images are shot at different times but in the same shooting area, and inputting the two detection images as two input images into a change detection network, wherein the input end of a trained feature extractor in the change detection network corresponds one-to-one to the input image, and the change detection network outputs a corresponding predicted change map.

[0097] It should be noted that in step S201, a sample set is constructed, including: using Perlin noise in combination with fractal Brownian motion to obtain a cloud simulation image; providing at least two remote sensing images; each remote sensing image is fused with the cloud simulation image to obtain a foggy image corresponding to the remote sensing image; all remote sensing images and all foggy images constitute a sample set. Perlin noise is a type of noise that can produce continuous and smooth random values. It is suitable for simulating various phenomena in nature, such as the undulations of mountains, the shape of clouds, the dynamic changes of flames, etc. It has the characteristics of smoothness, multidimensionality, and predictability. In this embodiment, Perlin noise of different frequencies is combined with fractal Brownian motion to simulate the interference effect of thin clouds in nature. Refer to Figure 4 , Figure 4The LEVIR-CD dataset is composed of remote sensing images, and the LEVIR-CD-cloud dataset is composed of foggy images corresponding to the remote sensing images in the LEVIR-CD dataset. The size of the foggy image is 1024×1024 pixels.

[0098] Specifically, the remote sensing image is fused with the cloud simulation image to obtain the foggy image corresponding to the remote sensing image, which is calculated as follows:

[0099] I(x)=J(x)t(x)+A(1-t(x))

[0100] Among them, J(x) is the remote sensing image, t(x) is a parameter, representing the light that is not scattered and reaches the camera, A is the ambient light, representing the cloud simulation image, I(x) is the foggy image corresponding to the remote sensing image, and x = (x, y) is the image coordinate.

[0101] For each remote sensing image in the LEVIR-CD dataset, a cloud layer simulation operation with random initial values ​​is performed to generate the corresponding foggy image. Four remote sensing images and the corresponding foggy images are randomly selected for reference. Figure 4 ,Depend on Figure 4 It can be seen that the combination of Perlin noise and fractal Brownian motion can better simulate the effect of thin cloud interference.

[0102] Reference Figure 5 In step S202, the feature extractor is trained using the sample set, and the trained feature extractor is based on contrastive learning. The remote sensing image and its corresponding foggy image are used as positive sample pairs, and the other combinations are used as negative sample pairs. When the input image is a positive sample pair (Positive Pair), let label = 0, that is, the label value is 0; when the input image is a negative sample pair (Negative Pair), let label = 1, that is, the label value is 1. In this way, in the constructed loss function, when a positive sample pair is input, the smaller the difference between the extracted feature vectors f1 and f2, the smaller the total loss function; when a negative sample pair is input, the larger the difference between the extracted feature vectors f1 and f2, the smaller the total loss function, thereby achieving the effect of maximizing the similarity between positive sample pairs and maximizing the difference between negative sample pairs.

[0103] It can be understood that in step S2031, the feature extractor removes the initial fully connected layer as the backbone to extract multi-scale features from the input images I1 and I2. The feature extractor ResNet backbone includes five main blocks, including a 7×7 convolutional layer and four residual blocks. For ease of understanding, these five blocks are called Conv1, Res2, Res3, Res4 and Res5. Res3 and Res4 perform downsampling with a step size of 2. For the input bi-temporal image I i(i∈{1,2})∈R 3×H×W , extract three feature maps of different scales from Res2, Res3 and Res5, respectively. and ).

[0104] In step S2031, the fused feature map is processed by the spatial attention module, the converter module and the channel attention module in sequence to obtain a final feature map, including:

[0105] The spatial attention module processes the fused feature map to obtain the variable feature map;

[0106] The converter module and channel attention module process the variable feature map to obtain the final feature map.

[0107] The spatial attention module processes the fused feature map to obtain the variable feature map, which is calculated as follows:

[0108]

[0109] in, is the fusion feature map, σ is the activation function, f is the convolution kernel operation, Maxpool is the maximum pooling operation, Avgpool is the average pooling operation, is the two-dimensional spatial attention, is element-wise multiplication, It is a variable feature map.

[0110] The converter module and the channel attention module process the variable feature map to obtain the final feature map, which is calculated as follows:

[0111]

[0112] M c =MLP(M i )

[0113]

[0114] Among them, Maxpool is the maximum pooling operation, is the variable feature map, T() is the converter module processing, ⊕ is the element-by-element addition, MLP is the multi-layer perception processing, is element-wise multiplication, is the final feature map.

[0115] Among them, after the extracted features are mixed, the feature map obtained Through the subsequent spatial attention module SAM. The spatial attention module SAM automatically emphasizes the important information related to the feature map in terms of position. Figure 6, SAM uses two-dimensional spatial attention In each Element-wise multiplication is implemented on . The meaningful features related to position changes are given greater weights. In this way, the spatial attention module SAM can effectively highlight the changing areas of the dual-phase image and suppress the features of irrelevant areas. Perform average-pooling and max-pooling operations along the channel dimension, and then concatenate the results of the pooling operations to generate

[0116] Reference Figure 7 , after passing through the spatial attention module SAM, we get the variable feature map The final feature map is then generated through the converter module Transformer and the channel attention module CAM The transformer module Transformer utilizes encoder and decoder blocks, and the transformer module Transformer is plug-and-play in the change detection network provided in this embodiment. In this embodiment, the spatial attention module SAM and the transformer module Transformer are used to model spatial context information and global context information, respectively. The channel attention module CAM models channel context information by highlighting the channels related to the change. Figure 7 As shown, for image I i , multiple features share the same channel attention M c . To calculate channel attention, we first use the element-wise summation method to fuse the feature maps of the same scale of the two sub-branches, and then apply the max-pooling operation along the spatial dimension of the fused result. Next, we use element-wise summation again to merge the multi-scale results of the max-pooling operation, and pass the fused result through a multi-layer perception (MLP) to obtain channel attention. The MLP consists of a full convolutional layer, a ReLU activation function, a full convolutional layer, and a Sigmoid activation function, where Figure 7 Maxpool stands for max-pooling.

[0117] The change detection network provided in this embodiment fuses the global and local features of the image, and adopts spatial and channel attention to improve efficiency and accuracy. Perlin noise is used to simulate natural thin cloud interference and added to the classic change dataset to generate a new change detection dataset with thin cloud interference, so that the change detection network has more superior performance in the remote sensing image change detection task with cloud interference, especially thin cloud interference.

[0118] Example 3

[0119] Reference Figure 2 Based on the same inventive concept, the present invention also provides a remote sensing image anti-cloud and fog interference change detection network based on contrast learning, comprising:

[0120] Two identical sub-networks 1, sub-network 1 includes a trained feature extractor 2, a feature processing module 3, a spatial attention module 4, a converter module 5 and a channel attention module 6, the input end of the feature processing module 3 is connected to the output end of the trained feature extractor 2, the input end of the spatial attention module 4 is connected to the output end of the feature processing module 3, the input end of the converter module 5 is connected to the output end of the spatial attention module 4, and the input end of the channel attention module 6 is connected to the output end of the converter module 5;

[0121] A connection module 7, wherein the input end of the connection module 7 is respectively connected to the output end of the channel attention module 6 of the two sub-networks 1;

[0122] A classifier 8 , an input end of the classifier 8 is connected to an output end of the connection module 7 .

[0123] It should be noted that the trained feature extractor 2 is used to receive an input image, extract a 32×32×32 feature map, a 32×64×64 feature map and a 32×128×128 feature map according to the input image, upsample the 32×32×32 feature map to obtain an upsampled feature, and downsample the 32×128×128 feature map to obtain a downsampled feature; the feature processing module 3 is used to fuse the 32×64×64 feature map, the upsampled feature and the downsampled feature to obtain a fused feature map; the spatial attention module 4, the converter module 5 and the channel attention module 6 are used to process the fused feature map in sequence through the spatial attention module 4, the converter module 5 and the channel attention module 6 to obtain a final feature map; the connection module 7 is used to obtain two final feature maps, connect the two final feature maps in pairs along the channel dimension to obtain a total map; the classifier 8 is used to output a predicted change map based on the total map.

[0124] It can be understood that the remote sensing image anti-cloud interference change detection network based on contrastive learning constructed using the trained feature extractor 2 fuses the global features and local features of the image, and uses the spatial attention module 4 and the channel attention module 6 to improve efficiency and accuracy. Even in the remote sensing image change detection task with cloud interference, it can achieve fast and accurate detection, thereby improving the robustness of remote sensing image change detection.

[0125] It can be seen from the above embodiments that the remote sensing image anti-cloud and fog interference change detection method based on contrast learning provided by the present invention achieves at least the following beneficial effects:

[0126] The present invention provides a remote sensing image anti-cloud and fog interference change detection method based on contrastive learning, including constructing a sample set; using the sample set to train a feature extractor to obtain a trained feature extractor; using the trained feature extractor, feature processing module, spatial attention module, converter module, channel attention module, connection module and classifier to construct a change detection network; inputting two detection images into the change detection network, the two detection images are shot at different times and in the same area, and the change detection network outputs a predicted change map. The feature extractor is trained using a sample set containing remote sensing images and foggy images to obtain a trained feature extractor, so that the trained feature extractor has a good ability to resist cloud interference; the change detection network constructed using the trained feature extractor fuses the global features and local features of the image, and uses a spatial attention module and a channel attention module to improve efficiency and accuracy, so that even in the remote sensing image change detection task with cloud interference, fast and accurate detection can be achieved, and the robustness of remote sensing image change detection is improved.

[0127] Although some specific embodiments of the present invention have been described in detail by way of example, it will be appreciated by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present invention. It will be appreciated by those skilled in the art that the above embodiments may be modified without departing from the scope and spirit of the present invention. The scope of the present invention is defined by the appended claims.

Claims

1. A remote sensing image anti-cloud and fog interference change detection method based on contrastive learning, characterized in that: include: Constructing a sample set, the sample set comprising remote sensing images and foggy images, the number of the remote sensing images is at least two, any two of the remote sensing images are different, the remote sensing images correspond to the foggy images one by one, and the foggy images are obtained from the corresponding remote sensing images; Using the sample set to train a feature extractor to obtain a trained feature extractor; Build a change detection network, including: Constructing a subnetwork, including: providing the trained feature extractor to receive an input image, extracting a 32×32×32 feature map, a 32×64×64 feature map and a 32×128×128 feature map according to the input image, upsampling the 32×32×32 feature map to obtain an upsampled feature, and downsampling the 32×128×128 feature map to obtain a downsampled feature; connecting the input end of a feature processing module to the output end of the trained feature extractor to fuse the 32×64×64 feature map, the upsampled feature and the downsampled feature to obtain a fused feature map; connecting the input end of a spatial attention module to the output end of the feature processing module, connecting the input end of a converter module to the output end of the spatial attention module, and connecting the input end of a channel attention module to the output end of the converter module, so as to process the fused feature map in sequence through the spatial attention module, the converter module and the channel attention module to obtain a final feature map; providing two of said sub-networks; The input end of the connection module is connected to the output end of the channel attention module of the two sub-networks respectively to obtain the two final feature maps, and the two final feature maps are connected in pairs along the channel dimension to obtain a total map; Connecting the input end of the classifier to the output end of the connection module to output a predicted change map according to the overall map; Two detection images are provided, wherein the two detection images are taken at different times and in the same area, and the two detection images are input into the change detection network as the two input images. The input end of the trained feature extractor in the change detection network corresponds one-to-one to the input image, and the change detection network outputs the corresponding predicted change map.

2. The remote sensing image anti-cloud and fog interference change detection method based on contrast learning according to claim 1 is characterized in that: Constructing the sample set includes: The cloud layer simulation image is obtained by combining Perlin noise with fractal Brownian motion. Providing at least two of the remote sensing images; Each of the remote sensing images is fused with the cloud simulation image to obtain the foggy image corresponding to the remote sensing image; All the remote sensing images and all the foggy images constitute the sample set.

3. The remote sensing image anti-cloud and fog interference change detection method based on contrast learning according to claim 2 is characterized in that: The remote sensing image is fused with the cloud layer simulation image to obtain the foggy image corresponding to the remote sensing image, which is calculated in the following manner: I(x)=J(x)t(x)+A(1-t(x)) Wherein, J(x) is the remote sensing image, t(x) is a parameter representing the light that is not scattered and reaches the camera, A is the cloud simulation image, and I(x) is the foggy image corresponding to the remote sensing image.

4. The remote sensing image anti-cloud and fog interference change detection method based on contrast learning according to claim 1 is characterized in that: Using the sample set to train the feature extractor to obtain the trained feature extractor includes: Providing two of the feature extractors; Randomly select two images from the sample set, the selected two images are any two remote sensing images, or the selected two images are any two foggy images, or one of the two images is any one remote sensing image and the other is any one foggy image; Input the two selected images into the two feature extractors, the feature extractors correspond to the images one by one, the feature extractors output feature vectors according to the input images, and obtain the two feature vectors; Determine whether there is a corresponding relationship between the two selected images, if there is a corresponding relationship between the two images, set the label value to 0, if there is no corresponding relationship between the two images, set the label value to 1; Calculate a loss function based on the two feature vectors and the label value; The feature extractor is trained according to the loss function.

5. The remote sensing image anti-cloud and fog interference change detection method based on contrast learning according to claim 4 is characterized in that: The loss function is calculated based on the two feature vectors and the label value in the following manner: loss=(1-label)euclidean(f1,f2) 2 +label(1-euclidean(f1,f2)) 2 Among them, loss is the loss value, label is the label value, euclidean(f1, f2) is the normalized Euclidean distance between the two feature vectors, f1 is the feature vector corresponding to one of the selected images, and f2 is the feature vector corresponding to the other selected image.

6. The remote sensing image anti-cloud and fog interference change detection method based on contrast learning according to claim 1 is characterized in that: Processing the fused feature map sequentially through the spatial attention module, the converter module, and the channel attention module to obtain the final feature map, including: The spatial attention module processes the fused feature map to obtain a variable feature map; The converter module and the channel attention module process the variable feature map to obtain the final feature map.

7. The remote sensing image anti-cloud and fog interference change detection method based on contrast learning according to claim 6 is characterized in that: The spatial attention module processes the fused feature map to obtain the variable feature map, which is calculated as follows: in, is the fusion feature map, σ is the activation function, f is the convolution kernel operation, Maxpool is the maximum pooling operation, Avgpool is the average pooling operation, is the two-dimensional spatial attention, is element-wise multiplication, is the variable characteristic diagram.

8. The remote sensing image anti-cloud and fog interference change detection method based on contrast learning according to claim 6 is characterized in that: The converter module and the channel attention module process the variable feature map to obtain the final feature map, which is calculated as follows: M c =MLP(M i ) Among them, Maxpool is the maximum pooling operation, is the variable feature map, T() is the converter module processing, is element-by-element addition, MLP is multi-layer perception processing, is element-wise multiplication, is the final feature map.

Images