Remote sensing image change detection system based on collaborative similarity learning

Abstract

The application discloses a remote sensing image change detection system based on cooperative similarity learning, which comprises a backbone network, a similarity fusion module, a multi-scale information fusion structure, an attention fusion module and a segmentation head, wherein the backbone network is used for extracting feature maps of two original images of the same place at different times respectively; the similarity fusion module is used for fusing the two feature maps to obtain a fusion feature map containing change information; the multi-scale information fusion structure is used for obtaining different scale feature maps by using the fusion feature map containing change information; the attention fusion module is used for fusing the different scale feature maps by using an attention mechanism to obtain an attention fusion feature map; and the segmentation head is used for obtaining a change detection result. The application constructs the attention fusion module, improves performance and representation ability by fusing multi-layer features, constructs the similarity fusion module, obtains change features by learning invariant features, and reduces misjudgment caused by uneven distribution of positive and negative samples.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing image change detection technology, specifically relating to a remote sensing image change detection system based on collaborative similarity learning. Background Technology

[0002] Change detection in remote sensing images is a hot research topic in the field of remote sensing image processing. Automatic change detection has gained widespread attention and played a crucial role in promoting the application and development of remote sensing images. Remote sensing image change detection involves analyzing the differences between two phases to distinguish differences between images of the same area at different times. It is a fundamental research direction in remote sensing image analysis with wide applications, such as monitoring land use, tracking urban changes, and detecting deforestation. Deep learning has greatly advanced remote sensing image change detection by extracting discriminative hierarchical features. However, due to the increasing number of high-resolution remote sensing images with rich spatial details but limited spectral information, using traditional backbone networks leads to blurred boundaries between different semantics of hierarchical features. This explains why most false alarms in the final prediction are distributed around change boundaries. Therefore, finding discriminative and effective feature representations is crucial.

[0003] Traditional methods typically rely on image processing techniques to extract low-level feature information, such as edges, contours, and texture details. These techniques often struggle to adapt to complex change detection scenarios in remote sensing images. However, breakthroughs in deep neural networks have demonstrated significant success in numerous tasks. Notably, several deep learning architectures have been widely used for change detection: Deep Belief Networks (DBNs), U-net, Siamese Convolutional Networks, and Changer. Since high-level semantic information is crucial for understanding complex changing scenes, applying deep learning methods to change detection has become a trend.

[0004] Change detection in remote sensing images requires pixel-level prediction. Fully convolutional neural networks (CNNs), used for dense prediction tasks, have been widely applied to the design of deep learning-based change detection methods. These methods primarily employ similar CNN structures, generating difference maps or change vectors through deep feature comparisons. Previously, CNNs demonstrated their ability to extract effective semantic features, making them a popular choice for change detection (CD) in remote sensing images. However, due to the inherent locality of convolutional operations, they struggle to capture general temporal and spatial information. Advances in Transformers in Natural Language Processing (NLP) have prompted researchers to extend them to various computer vision tasks. A series of Transformer models have been developed in computer vision, including Vision Transformer (ViT), Twins, DeiT, SegFormer, and the Twin Transformer. In recent years, Transformers have demonstrated outstanding performance in remote sensing, with their long-range modeling capabilities being fully utilized.

[0005] Most of these networks are modifications of networks originally proposed for single-image semantic segmentation. Transforming these networks to detect changes in bi-temporal images often presents significant challenges. Early fusion methods lack informative deep features from the individual original images. Later fusion methods involve the problem of low-feature representations of the original images and the fusion of heterogeneous features.

[0006] In summary, existing methods for detecting changes in remote sensing images have the following problems: (1) Remote sensing images have the problem of large intra-class differences and small inter-class differences. The former is mainly caused by differences in viewpoint, target deformation, light intensity and scale changes; the latter is mainly caused by different categories having the same underlying features. At the same time, due to the uneven distribution of samples, the probability of positive and negative samples being misclassified is greatly increased, which is manifested as outliers and many barbs on the boundary.

[0007] (2) CNN-based methods cannot establish long-distance dependencies on global information between feature regions, cannot fully explore the contour edges and details of the image, and ignore the multi-scale, multi-directional and multi-resolution representation of features in the frequency domain.

[0008] (3) When different feature layers are fused, semantic differences will occur. For cross-layer fusion of features from non-adjacent stages, there will be a certain degree of insufficient feature fusion extraction. After fusion, information from a certain feature layer will be ignored, resulting in incomplete fused features. Summary of the Invention

[0009] To address the aforementioned problems in the existing technology, this invention provides a remote sensing image change detection system based on collaborative similarity learning. The technical problem to be solved by this invention is achieved through the following technical solution: This invention provides a remote sensing image change detection system based on collaborative similarity learning, comprising a backbone network, a similarity fusion module, a multi-scale information fusion structure, an attention fusion module, and a segmentation head, wherein... The backbone network is used to input two original images of the same location at different times, and to extract the feature maps of the two original images respectively; The similarity fusion module is used to fuse the feature maps of the two original images to obtain a fused feature map containing change information; The multi-scale information fusion structure is used to obtain feature maps of different scales by utilizing the fused feature map containing change information; The attention fusion module is used to fuse the feature maps of different scales using an attention mechanism to obtain an attention fusion feature map; The segmentation head is used to perform change detection on the attention fusion feature map to obtain change detection results.

[0010] In one embodiment of the present invention, the backbone network is a pre-trained Swing Transformer network.

[0011] In one embodiment of the present invention, the similarity fusion module includes a splicing block, a cosine similarity block, a variation feature block, and a fusion block, wherein, The stitching block is used to stitch together the feature maps of the two original images to obtain a stitched feature map; The cosine similarity block is used to perform cosine similarity processing on the feature maps of the two original images to obtain cosine similarity feature maps; The change feature block is used to obtain the change feature map between the feature maps of the two original images; The fusion block is used to perform feature fusion on the spliced ​​feature map, the cosine similarity feature map, and the change feature map to obtain a fused feature map that includes change information.

[0012] In one embodiment of the present invention, the changing feature block is specifically used for: The first processing result is obtained by multiplying the Q-key value q1 of the first feature map and the K-key value k2 of the second feature map in the feature maps of the two original images by the softmax function. The V key value v2 of the second feature map is multiplied by the first processing result to obtain the second processing result; The second feature map is obtained by subtracting the second processing result from the second feature map pixel by pixel.

[0013] In one embodiment of the present invention, a 1×1 convolutional layer is further included after the stitched block to perform convolution processing on the stitched feature map, so that the number of channels of the stitched feature map is adjusted to be the same as that of the original image; A tile function is set after the cosine similarity block to perform calculations on the cosine similarity feature map, so that the number of channels of the cosine similarity feature map is adjusted to be the same as that of the original image.

[0014] In one embodiment of the present invention, the multi-scale information fusion structure includes a feature pyramid network and a self-attention module, wherein, The downsampling branch of the feature pyramid network obtains a set of feature maps sequentially through downsampling. C 1 ，C 2 ，C 3 ， C 4}, and the feature map obtained from the last downsampling. C 4. Input to the self-attention module; The output of the self-attention module is fed into the upsampling branch of the feature pyramid network, and feature maps of different scales are obtained through downsampling. P 1, P 2, P 3, P 4}.

[0015] In one embodiment of the present invention, the attention fusion module is specifically used for: The largest-scale feature map among the feature maps of different scales output by the multi-scale information fusion structure. P 1. Input is processed in a self-attention unit to obtain the output result; Residual feature maps in the feature maps of different scales P 2, P 3, P The Q-key values ​​of 4 are concatenated and dimensionality reduced; The data obtained from the dimensionality reduction process is compared with the feature map. P 2, P 3, P The K key values ​​of 4 are multiplied by a dot to obtain three image data after dot multiplication, and the softmax function is then applied to the three image data respectively. The processed three image data and feature maps P 2, P 3, PThe V key value of 4 is multiplied by a dot, and the three image data after the dot product are added pixel by pixel before upsampling. Finally, the upsampled image is added pixel by pixel to the output of the self-attention unit to obtain the attention fusion feature.

[0016] In one embodiment of the present invention, the remote sensing image change detection system based on collaborative similarity learning further includes a collaborative loss function, the expression of which is: , in, Represents cross-entropy loss, λ 0 represents the weight of the cross-entropy loss. Indicates loss on the dice. λ 1 represents the weight of the loss from the dice roll. Indicates the height of the original image. This indicates the width of the original image.

[0017] In one embodiment of the present invention, the expression for the cross-entropy loss is:

[0018] in, This represents the coordinates of each pixel in the input image. This indicates the prediction result for the image.

[0019] In one embodiment of the present invention, the expression for the dice loss is:

[0020] in, This is to avoid the denominator being zero.

[0021] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention constructs an attention fusion module for a remote sensing image change detection system based on collaborative similarity learning. This module is used for feature fusion across layers, improving the model's performance and representational ability by fusing multi-layer features. The fusion method is based on self-attention and considers feature interactions between adjacent layers. This invention also constructs a similarity fusion module for two-stage feature fusion. By learning invariant features, it obtains changed features, significantly reducing misjudgments caused by the imbalanced distribution of positive and negative samples in change detection.

[0022] 2. This invention utilizes a Swing transformer network as the backbone network to establish long-range dependencies on global information. A feature pyramid network is then connected to the backbone network, and a self-attention mechanism is embedded in the last layer of the feature pyramid network to deeply extract features after downsampling in the last layer. During network training, a combined approach of main loss and auxiliary loss is used. Since the decoder is a simple linear layer, adding an auxiliary loss function increases the difficulty of network training but improves network performance.

[0023] 3. This invention optimizes the model training process by utilizing multiple scales and adding auxiliary loss, fully leveraging the spatial feature information in the image. Simultaneously, it applies an attention mechanism to cross-layer feature fusion, reducing significant information loss during feature fusion and thus compensating for the limited spectral information in remote sensing images. By extracting unchanged features to obtain changed features and focusing on the similarity between features from two time phases, it not only significantly reduces the prediction probability of outliers but also results in smoother learned boundary contour features and lower false alarm rates.

[0024] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0025] Figure 1 This is a block diagram of a remote sensing image change detection system based on collaborative similarity learning provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of a remote sensing image change detection system based on collaborative similarity learning provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of a similarity fusion module provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of an attention fusion module provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the visualization results of change detection in remote sensing images using different methods. Detailed Implementation

[0026] To further illustrate the technical means and effects adopted by the present invention to achieve the intended purpose, the following detailed description of the remote sensing image change detection system based on collaborative similarity learning proposed in accordance with the present invention is provided in conjunction with the accompanying drawings and specific embodiments.

[0027] The foregoing and other technical contents, features, and effects of the present invention will be clearly presented in the following detailed description of specific embodiments in conjunction with the accompanying drawings. Through the description of the specific embodiments, a more in-depth and concrete understanding can be gained of the technical means and effects adopted by the present invention to achieve its intended purpose. However, the accompanying drawings are for reference and illustration only and are not intended to limit the technical solutions of the present invention.

[0028] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations are intended to cover non-exclusive inclusion, such that an article or apparatus comprising a list of elements includes not only those elements but also other elements not expressly listed. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the article or apparatus that includes said element.

[0029] This invention proposes a remote sensing image change detection method based on collaborative similarity learning, which mainly addresses two issues: First, the limited spectral information of remote sensing images leads to blurred boundaries between different semantics; second, due to the randomness and disparity in the distribution and quantity of positive and negative samples, the prediction results contain outliers.

[0030] This invention provides a remote sensing image change detection system based on collaborative similarity learning. Please refer to [link to relevant documentation]. Figure 1 and Figure 2 The remote sensing image change detection system includes a backbone network 101, a similarity fusion module 102, a multi-scale information fusion structure 103, an attention fusion module 104, and a segmentation head 105. The backbone network 101 is used to input two original images of the same location at different times and extract feature maps from each image. The similarity fusion module 102 fuses the feature maps of the two original images to obtain a fused feature map containing change information. The multi-scale information fusion structure 103 uses the fused feature map containing change information to obtain feature maps at different scales. The attention fusion module 104 uses an attention mechanism to fuse feature maps at different scales to obtain an attention fused feature map. The segmentation head 105 performs change detection on the attention fused feature map to obtain the change detection result.

[0031] In this embodiment, the backbone network 101 is a pre-trained Siamese transformer network, and the input original image is normalized to a uniform size of 224×224. The backbone network 101 is a weight-shared Siamese network, in which the window size is set to 7, and the patch size in each window is 4.

[0032] Subsequently, the backbone network 101 extracts the feature maps of the two original images and feeds them into the similarity fusion module (SFM) 102. The SFM 102 fuses these two feature maps, converting them into a single change feature map. This module maps the two change features between the two temporal images (i.e., two original images of the same location at different times) into a single feature. It is worth noting that if only unidirectional change (building reconstruction or demolition) is considered, only the image or feature at the second time point contains information about the change.

[0033] A known prior knowledge is that the changed regions in an image are significantly smaller than the unchanged regions. Based on this premise, we can learn the unchanged regions and compare them with the original image to obtain the changed regions. Simply put, it's about transforming a known invariant region into an unknown changed region. Transforming the object of study makes it easier to extract the semantic information of changes in the image and better identify the fine-grained attributes of the changed object.

[0034] Please see Figure 3 , Figure 3 This is a schematic diagram of the structure of a similarity fusion module provided in an embodiment of the present invention. The similarity fusion module 102 of this embodiment includes a concatenation block, a cosine similarity block, a variation feature block, and a fusion block. The concatenation block concatenates the feature maps of two original images to obtain a concatenated feature map; the cosine similarity block performs cosine similarity processing on the feature maps of two original images to obtain a cosine similarity feature map; the variation feature block obtains a variation feature map between the feature maps of two original images; and the fusion block performs feature fusion on the concatenated feature map, the cosine similarity feature map, and the variation feature map to obtain a fused feature map including variation information.

[0035] It should be noted that the original image is assumed to have C channels. in Then the number of channels in the feature map of the original image is also C. in The number of channels in the stitched feature map after the stitching blocks is 2C. in Therefore, a 1×1 convolutional layer is added after the stitched block to perform convolution processing on the stitched feature map, so that the number of channels of the stitched feature map is adjusted to be the same as that of the original image.

[0036] Similarly, the number of channels in the cosine similarity feature map after the cosine similarity block is 1. A tile function is also set after the cosine similarity block to perform calculations on the cosine similarity feature map, so that the number of channels in the cosine similarity feature map is adjusted to be the same as that of the original image.

[0037] Furthermore, such as Figure 3 As shown, the change feature block in this embodiment is specifically used for: multiplying the Q key value q1 of the first feature map and the K key value k2 of the second feature map in the feature maps of the two original images by a dot product and then performing a softmax function operation to obtain a first processing result; multiplying the V key value v2 of the second feature map by the first processing result to obtain a second processing result; and subtracting the second processing result from the second feature map to obtain a change feature map.

[0038] Subsequently, the stitched feature maps, cosine similarity feature maps, and variation feature maps with the same channels are fused to obtain a fused feature map that includes variation information. This fusion method aims not only to obtain rich semantics and a large receptive field, but also to more accurately capture the variation features between biphasic images.

[0039] The processing procedure of this similarity fusion module can also be described as follows:

[0040] in, These represent two original images of the same location at different times. This represents the first feature map output by the backbone network. This represents the second feature map output by the backbone network. This represents the output of the similarity fusion module. This indicates the processing procedure of the similarity fusion module.

[0041] The similarity fusion module transforms bi-temporal features through three branches: concatenation blocks, cosine similarity blocks, and variable feature blocks. This generates continuously evolving perceptual semantics. Finally, the features from the three branches are added together. After passing through the similarity fusion module, the network not only achieves the fusion of two types of features but also possesses richer spatial features and perceptual semantics.

[0042] Further, please continue to see Figure 1 The multi-scale information fusion structure 103 in this embodiment includes a feature pyramid network and a self-attention module. The downsampling branch of the feature pyramid network obtains a set of feature maps sequentially through downsampling. C 1 ，C 2 ，C 3 ，C 4}, and the feature map obtained from the last downsampling. C4. Input is fed into the self-attention module; the output of the self-attention module is fed into the upsampling branch of the feature pyramid network, and feature maps of different scales are obtained through downsampling. P 1, P 2, P 3, P 4}.

[0043] Specifically, in this embodiment, a set of feature maps obtained through downsampling is represented as { C 1, C 2, C 3, C 4}, where, C 4 is also the feature map input from the attention module, and the feature map output by the feature pyramid network is { P 1, P 2, P 3, P 4}. Figure 1 {4, 8, 16, 32} represent different downsampling rates. C The downsampling rate of 4 is relatively large, therefore these features are closely related to high-level semantics. Experience shows that fusing features from different resolutions is more effective than using only... C 1. Good. Here, simple addition is used to complete the fusion operation. This multi-scale information fusion structure 103 can be formulated as follows:

[0044] in, Y This represents the output of the similarity fusion module. Indicates an upsampling operation. This represents the self-attention mechanism.

[0045] Subsequently, the Attention Fusion Module (AFM) 104 combines images of different sizes from the four paths to facilitate the formation of comprehensive features across multiple scales. See also... Figure 4 , Figure 4 This is a schematic diagram of an attention fusion module provided in an embodiment of the present invention. Specifically, the attention fusion module 104 is used to: input the largest-scale feature map P1 from the feature maps of different scales output by the multi-scale information fusion structure 103 into the attention unit for processing, and obtain the output result; and process the remaining feature maps from the feature maps of different scales. P 2, P 3, P The Q-key values ​​q2, q3, and q4 of the input feature map are concatenated and dimensionality reduced. Specifically, a 1×1 convolutional layer is added after the concatenated block to reduce the number of feature channels in the concatenated block to be the same as the input feature map. The dimensionality-reduced data is then compared with the feature map. P 2, P 3, PThe K key values ​​k2, k3, and k4 of 4 are multiplied by a dot to obtain three image data sets. Then, the softmax function is applied to each of these three image data sets. Finally, the three image data sets are compared with the feature map. P 2, P 3, P The V key values ​​v2, v3, and v4 of 4 are multiplied by a dot, and the three image data after the dot product are added pixel by pixel before upsampling. Finally, the upsampled image is added pixel by pixel to the output of the self-attention unit to obtain the attention fusion feature.

[0046] Specifically, the attention fusion module 104 improves the model's performance and representational power by fusing features from multiple layers. The fusion method is based on a self-attention mechanism and considers feature interactions between adjacent layers. We believe that deeper features contain more useful information, while features from other layers primarily focus on the similarity between layers. Therefore, for the first layer, a self-attention operation is performed directly. For the other three layers, the features generated by the three layers are first processed... splice them together, and then use the corresponding layers. and Perform self-attention exercises.

[0047] like Figure 4 As shown, the input features of the attention fusion module 104 are P 1 (Part 1) P 2. P 3 and P 4 (Lower), Z 1 represents the output after the self-attention unit. For the feature map... P 2. P 3 and P 4. Obtain the corresponding key-value pairs through feature transformation. QKVs , respectively q 2. q 3 and q 4. Simultaneously obtain k 2 、k 3 、k 4 and v 2 、v 3 、v 4. First, splicing. q 2 and q 3 and q 4 to obtain Q The number of channels in the feature image increases from C in Added to 3C in Then Q After being fed into a 1×1 convolutional layer and undergoing convolution, a channel number of C is obtained. in of q Subsequently, q respectively with k2. k 3 and k Multiply by 4 to obtain three feature maps; then use the softmax function to operate on the three feature maps and output the result. w 2. w 3 and w 4. w i and v i ( i = 2，3，4 The values ​​of the dot products between the two elements are added together to obtain the result. Z 2; Finally, Z 2. After upsampling, the output of the self-attention unit is compared with the output of the self-attention unit. Z 1. Perform pixel-by-pixel addition to obtain the final fused feature map. Z .

[0048] The calculation formula for attention fusion module 104 is as follows:

[0049] in, Indicates an upsampling operation. This indicates a convolution operation.

[0050] Finally, the segmentation head 105 directly divides the feature map fused by the attention fusion module 104 into two categories: positive samples (pixels that have changed) and negative samples (pixels that have not changed), to obtain the final change detection result. In the final change detection result map, pixels that have changed are marked as 1, corresponding to black dots, and pixels that have not changed are marked as 0, corresponding to white dots.

[0051] This invention proposes two innovative feature fusion modules: a similarity fusion module (SFM) and an attention fusion module (AFM). These modules enhance the network's ability to accurately detect changes in remote sensing images. The similarity fusion module performs two-stage feature fusion, learning changed features from invariant features. The attention fusion module fuses multi-scale features using a layer-by-layer fusion strategy, considering inter-layer interactions. This invention connects the feature pyramid network to the pre-trained backbone network and then integrates the self-attention module after the final downsampling layer of the feature pyramid network. This design fully extracts the global features after downsampling. Compared to the Pyramid Pooling Module (PPM), using the self-attention module significantly reduces the number of parameters and computational cost. Furthermore, a pixel shuffle operation is chosen during upsampling of the feature pyramid network, resulting in a slight improvement in both recall and accuracy.

[0052] Furthermore, this embodiment aims to focus on and emphasize edge features in the image. Therefore, a bi-branch loss function (cooperative loss function) is introduced, with each branch assigned a different weight. This loss function utilizes the complementary learning ability of shallow and deep features, thus reducing the likelihood of undetected changes in the prediction results. This improves the model's recall and increases the probability of predicting positive samples (change regions). The bi-branch function plays a synergistic role during model training. Positive samples in the image receive more attention during detection, and the second loss branch is added after the third layer downsampling of the multi-scale information fusion structure 103.

[0053] Specifically, since backpropagation cannot propagate to shallow layers, when the network is deep and the loss is propagated from back to front, the gradient may be zero, making the model difficult to train. Therefore, the branch added in this embodiment can serve as a regular constraint, forcing intermediate layer features to have detection capabilities. Furthermore, the added branch is beneficial for learning feature edges. First, feature information is easily lost during sampling. Second, the number of positive and negative samples in binary change detection is unbalanced. To mitigate these problems, the collaborative loss function in this embodiment selects dice loss and cross entropy loss as minimization objectives.

[0054] Specifically, the expression for cross-entropy loss is:

[0055] in, Represents cross-entropy loss, This represents the coordinates of each pixel in the input image. This represents the prediction result for the image. Indicates the height of the original image. This indicates the width of the original image.

[0056] The formula for dice loss is:

[0057] in, Indicates the height of the input image. Indicates the width of the input image. In order to avoid the denominator being zero, in this embodiment, The value is 10 -7 .

[0058] The overall collaborative loss function is expressed as follows:

[0059] in, λ 0 represents the weight of the main loss (cross-entropy loss). λ1 is the weight of the auxiliary loss (dice loss).

[0060] In summary, the remote sensing image change detection system in this embodiment includes four key modules. The first is a similarity fusion module, designed for dual-temporal image fusion. The second is an attention fusion module for multi-layer feature fusion. The third is a multi-scale information fusion structure, aimed at learning multi-scale change feature maps. The fourth is a loss function, which focuses more on learning positive samples.

[0061] Please see Figure 5 , Figure 5 This is a schematic diagram of the visualization results of change detection in remote sensing images using different methods. Figure 5 The red areas represent regions incorrectly predicted as positive samples, and the green areas represent regions incorrectly predicted as negative samples. The first two rows are dual-temporal images, the third row is the label, and the fourth row is the prediction result. The first five columns are the results from the first dataset, Levir_CD, and the last five columns are the results from the second dataset, S2Looking. Figure 5 The results shown demonstrate that the remote sensing image change detection system of this invention can detect positive samples (red areas) with a high probability, effectively solving the problems of scale diversity in changed areas and high false alarm rates at change boundaries. It has application value in solving the problem of building change detection in actual production.

[0062] Furthermore, Table 1 shows the experimental results of different detection methods on two datasets for change detection, with evaluation metrics including F1 score, IoU (Intersection over Union), recall, and precision. As can be seen from Table 1, the remote sensing image change detection system of this embodiment achieves state-of-the-art results on two open datasets used for change detection, and this remote sensing image change detection system of this embodiment was tested without incorporating any data augmentation methods.

[0063] Table 1. Results of different models on the two datasets

[0064] Experiments have demonstrated that the remote sensing image change detection system of this invention can capture more accurate region boundary information and perform more precise segmentation of positive and negative samples. This invention effectively preserves image details and achieves accurate segmentation results.

[0065] In summary, remote sensing image change detection can be applied to urban planning and monitoring. By detecting changes in urban land use, we can better understand the dynamics of urban development and provide decision support for urban planning. Secondly, remote sensing images can be used to monitor urban environmental pollution, traffic conditions, and other issues, providing data support for urban management. Remote sensing image change detection can quickly and accurately assess the impact of natural disasters. For example, by comparing pre-disaster and post-disaster remote sensing images, we can assess the extent of damage to buildings, roads, and other infrastructure caused by disasters such as earthquakes and floods, providing guidance for rescue and reconstruction efforts. Overall, remote sensing image change detection has broad application prospects and can provide important data support and decision-making basis for various fields. With the continuous development and improvement of remote sensing technology, the application scope and accuracy of remote sensing image change detection will continue to improve.

[0066] The remote sensing image change detection system in this embodiment constructs an attention fusion module for cross-layer feature fusion. By fusing multi-layer features, it improves the model's performance and representational ability. The fusion method is based on self-attention and considers feature interactions between adjacent layers. This invention also constructs a similarity fusion module for two-stage feature fusion. By learning from invariant features, it obtains changed features, significantly reducing misjudgments caused by the imbalanced distribution of positive and negative samples in change detection. A Swing transformer network is used as the backbone network to establish long-distance dependencies on global information. A feature pyramid network is then connected to the backbone network, and a self-attention mechanism is embedded in the last layer of the feature pyramid network to deeply extract features after downsampling from the last layer. During network training, a combined approach of main loss and auxiliary loss is used. Since the decoder is a simple linear layer, adding an auxiliary loss function increases the difficulty of network training but improves network performance.

[0067] The remote sensing image change detection system of this invention optimizes the model training process by utilizing multi-scale methods and adding auxiliary loss, making full use of spatial feature information in the image. Simultaneously, it applies an attention mechanism to cross-layer feature fusion, reducing significant information loss during feature fusion and thus compensating for the limited spectral information of remote sensing images. By extracting unchanged features to obtain changed features and focusing on the similarity between features from two time phases, it not only significantly reduces the prediction probability of outliers but also results in smoother learned boundary contour features and lower false alarm rates.

[0068] In the several embodiments provided by this invention, it should be understood that the apparatus and methods disclosed in this invention can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of modules is merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0069] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated module can be implemented in hardware or in the form of hardware plus software functional modules.

[0070] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A remote sensing image change detection system based on collaborative similarity learning, characterized in that, It includes a backbone network (101), a similarity fusion module (102), a multi-scale information fusion structure (103), an attention fusion module (104), and a segmentation head (105), among which, The backbone network (101) is used to input two original images of the same location at different times, and to extract the feature maps of the two original images respectively; The similarity fusion module (102) is used to fuse the feature maps of the two original images to obtain a fused feature map containing change information; The multi-scale information fusion structure (103) is used to obtain feature maps of different scales by utilizing the fusion feature map containing change information; The attention fusion module (104) is used to fuse the feature maps of different scales using an attention mechanism to obtain an attention fusion feature map; The segmentation head (105) is used to perform change detection on the attention fusion feature map to obtain change detection results; The similarity fusion module (102) includes a splicing block, a cosine similarity block, a variation feature block, and a fusion block, wherein, The stitching block is used to stitch together the feature maps of the two original images to obtain a stitched feature map; The cosine similarity block is used to perform cosine similarity processing on the feature maps of the two original images to obtain cosine similarity feature maps; The change feature block is used to obtain the change feature map between the feature maps of the two original images; The fusion block is used to perform feature fusion on the spliced ​​feature map, the cosine similarity feature map, and the change feature map to obtain a fused feature map that includes change information; The attention fusion module (104) is specifically used for: The feature map of the largest scale among the feature maps of different scales output by the multi-scale information fusion structure (103) P 1. Input is processed in a self-attention unit to obtain the output result; Residual feature maps in the feature maps of different scales P 2, P 3, P The Q-key values ​​of 4 are concatenated and dimensionality reduced; Dimensionally reduced data and feature maps P 2, P 3, P The K key values ​​of 4 are multiplied by a dot to obtain three image data after dot multiplication, and the softmax function is then applied to the three image data respectively. The processed three image data and feature maps P 2, P 3, P The V key value of 4 is multiplied by a dot, and the three image data after the dot product are added pixel by pixel before upsampling. Finally, the upsampled image is added pixel by pixel to the output of the self-attention unit to obtain the attention fusion feature.

2. The remote sensing image change detection system based on collaborative similarity learning according to claim 1, characterized in that, The backbone network (101) is a pre-trained swin transformer network.

3. The remote sensing image change detection system based on collaborative similarity learning according to claim 1, characterized in that, The change feature block is specifically used for: The first processing result is obtained by multiplying the Q-key value q1 of the first feature map and the K-key value k2 of the second feature map in the feature maps of the two original images by the softmax function. The V key value v2 of the second feature map is multiplied by the first processing result to obtain the second processing result; The second feature map is obtained by subtracting the second processing result from the second feature map pixel by pixel.

4. The remote sensing image change detection system based on collaborative similarity learning according to claim 3, characterized in that, A 1×1 convolutional layer is added after the stitched block to perform convolution processing on the stitched feature map, so that the number of channels of the stitched feature map is adjusted to be the same as that of the original image; A tile function is set after the cosine similarity block to perform calculations on the cosine similarity feature map, so that the number of channels of the cosine similarity feature map is adjusted to be the same as that of the original image.

5. The remote sensing image change detection system based on collaborative similarity learning according to claim 3, characterized in that, The multi-scale information fusion structure (103) includes a feature pyramid network and a self-attention module. in, The downsampling branch of the feature pyramid network obtains a set of feature maps sequentially through downsampling. C 1 ，C 2 ，C 3 ，C 4}, and the feature map obtained from the last downsampling. C 4. Input to the self-attention module; The output of the self-attention module is fed into the upsampling branch of the feature pyramid network, and feature maps of different scales are obtained through downsampling. P 1, P 2, P 3, P 4}.

6. The remote sensing image change detection system based on collaborative similarity learning according to claim 1, characterized in that, It also includes a collaborative loss function, the expression of which is: ， in, Represents cross-entropy loss, λ 0 represents the weight of the cross-entropy loss. Indicates loss on the dice. λ 1 represents the weight of the loss from the dice roll. Indicates the height of the original image. This indicates the width of the original image.

7. The remote sensing image change detection system based on collaborative similarity learning according to claim 6, characterized in that, The expression for the cross-entropy loss is: in, This represents the coordinates of each pixel in the input image. This indicates the prediction result for the image.

8. The remote sensing image change detection system based on collaborative similarity learning according to claim 7, characterized in that, The expression for the dice loss is: in, This is to avoid the denominator being zero.

Images