Change detection method based on semantic prior and fusion space positioning

Abstract

The invention discloses a change detection method based on semantic prior and fusion space positioning, and relates to the technical field of image processing, and the method comprises the steps: firstly, carrying out the pre-training of a BPNet network on a supervised remote sensing extraction task, and fully utilizing the prior knowledge in the remote sensing extraction task; and then fine tuning is performed on the pre-trained BPNet network, a twinning network structure sharing weight is adopted to expand an encoder and decoder structure, a space consistency attention module is introduced, and feature information extraction and fusion are performed on the dual-tense image by combining a self-attention mechanism and a large convolution kernel decomposition technology so as to obtain a dual-tense image. Therefore, remote sensing image target change information is obtained. Therefore, by adopting the change detection method based on semantic prior and fusion space positioning, through the space consistency attention module, the depth encoder can be helped to expand an effective receptive field, the modeling capability of a long-distance dependency relationship is enhanced, and meanwhile, double-tense change detail information can be effectively captured, and a more accurate feature map can be obtained.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a change detection method based on semantic priori and fusion spatial positioning. Background Art

[0002] Change detection (CD) aims to identify ground changes by analyzing multi-temporal remote sensing images taken at different times over the same geographic area. Binary change detection (BCD) refers to the process of identifying changed objects of interest given known binary labels. Building change detection is a binary change detection task focused on buildings and is widely used in various fields such as urban development monitoring, disaster assessment, and land use planning.

[0003] The development of CD technology is closely related to the progress of earth observation technology, the evolution of information technology (IT), and the rise of artificial intelligence. Initially, there were two main methods for traditional CD: pixel-based methods and object-based methods. The pixel-based change detection (PBCD) method mainly obtains the change map by performing operations pixel by pixel, and then determines the change area by setting a threshold. The object-based change detection (OBCD) method uses spectral, texture, and spatial context information to capture changes at the object level. However, these methods still face some challenges. Although the PBCD method attaches importance to the semantic information of each pixel, it ignores the spatial relationship between pixels. The OBCD method is very sensitive to the choice of object segmentation algorithm, and lacks effective modeling of pixel semantics, making it difficult to distinguish pseudo changes. In addition, these methods show limited adaptability in the face of scene changes, need to rely on a lot of manual adjustments, and their accuracy is also easily questioned.

[0004] With the emergence of large-scale remote sensing data and the rapid development of deep learning technology, deep learning has shown great potential in the change detection task in the remote sensing field. The fully convolutional twin network is considered to be the first deep learning method for change detection, which has opened up the wide application of deep learning in this field. Current change detection methods include CNN-based methods (such as Changer, SGSLN), Transformer-based methods (such as ChangeFormer, BiT), and state-space model-based methods (such as RSM-CD). Although these deep learning methods have achieved good results, their performance improvement is limited by the lack of high-quality annotated data. In addition, in the task of building change detection, it often requires a lot of human resources to obtain and accurately register remote sensing images of the same area. Due to differences in sensors or changes in environmental conditions, the registration process may produce errors, which in turn limits the number of available image pairs. This data scarcity easily leads to the performance bottleneck of deep learning algorithms. In contrast, the datasets for single-temporal tasks are much larger, such as the WHU building dataset, the Massachusetts building dataset, the Inria building dataset, and natural image datasets (such as ImageNet and MSCOCO). This imbalance in data size directly affects the training effect of the model in the field of change detection. Therefore, how to effectively utilize data from similar tasks has become a key issue in current research.

[0005] Pre-training on large-scale datasets can provide the model with rich prior knowledge of remote sensing images, thereby improving its performance when fine-tuning on the subsequent target dataset. This is an effective strategy that can alleviate the problem of scarce data in the target domain. For example, some studies have optimized model performance by performing supervised pre-training on large-scale remote sensing scene classification datasets such as MillionAID. Other studies have performed multi-task pre-training on the SAMRS dataset, covering scene classification, semantic segmentation, instance segmentation, and rotated object detection. There are also studies that use unsupervised pre-training methods, such as using the MAE model on the MillionAID or fMoW Sentinel multispectral datasets. In addition, the domain gap between remote sensing images and natural images will also limit the change detection performance. Traditional pre-training datasets lack semantic features from a remote sensing perspective (such as buildings, roads, and trees), making it difficult to migrate to remote sensing tasks. It can be seen that existing studies have not fully utilized semantic segmentation image data in the same visual domain as the building change detection task in the pre-training stage. How to extend these models to support dual-input functions is still a problem worth exploring.

[0006] Therefore, there is an urgent need for a method that integrates the two stages of pre-training and fine-tuning to adapt to multi-temporal image change recognition, improve the sensitivity of capturing the temporal relationship of bi-temporal images, and solve the problem of data scarcity, providing an effective framework for processing complex remote sensing tasks. Summary of the invention

[0007] The purpose of the present invention is to provide a change detection method based on semantic prior and fused spatial positioning. By pre-training on an extracted data set with highly similar change detection features to the target task, the network can extract prior knowledge of the corresponding features from it, so that it can more effectively cope with the challenge of data sparsity in the change detection task.

[0008] To achieve the above object, the present invention provides a change detection method based on semantic prior and fusion spatial positioning, comprising:

[0009] Firstly, the BPNet network is pre-trained on the supervised remote sensing extraction task to make full use of the prior knowledge in the remote sensing extraction task;

[0010] Then, the pre-trained BPNet network is fine-tuned, and the encoder and decoder structures are expanded using a twin network structure with shared weights. A spatial consistency attention module is introduced. By combining the self-attention mechanism and large convolution kernel decomposition technology, feature information of the dual-temporal images is extracted and fused to obtain the target change information of the remote sensing image.

[0011] Preferably, both the encoder and the decoder of the BPNet network use a feature enhancement module for feature extraction, and the expression of the feature enhancement module is:

[0012]

[0013] In the formula, represents the input features of the feature enhancement module, represents the feature after expansion through the first 1×1 convolution, represents the output features of the feature enhancement module, n represents the temporal state of the feature map, ⊕ represents element-level addition, SE represents the self-attention mechanism, and DWConv 3×3 They represent the depth convolution with a convolution kernel of 3×3, Conv 1×1 Represents a 1×1 convolution.

[0014] Preferably, the spatial consistency attention module includes an ADLKA module and a TFU module: the ADLKA module is used to capture broader contextual information and fuse change information; the TFU module is used to retain the temporal information in the bi-temporal features.

[0015] Preferably, the expression output by the ADLKA module is:

[0016]

[0017] In the formula, represents the mixed bi-temporal information features after passing through the TFU module, represents the attention map output by the ADLKA module, Indicates that the convolution kernel is a k2×k2 depth dilated convolution, Indicates that the convolution kernel is k1×k1 deep convolution, Conv 1×1 Represents point-wise convolution.

[0018] Preferably, the TFU module processes each sample independently using layer normalization, expressed as follows:

[0019]

[0020] T l =GELU(LN(X s ));

[0021] In the formula, Represent the input features of the first and second tenses respectively, represents the features reduced by 1×1 convolution, represents the time-fused bi-temporal features, concat represents channel connection, LN represents layer normalization, and GELU represents the activation function.

[0022] Preferably, a multi-scale fusion segmentation head is introduced after the encoder, and the feature map extracted from each layer is convolved to generate a prediction map of the corresponding scale. The obtained prediction maps of each layer are adjusted to the original image size through bilinear interpolation, and fused to generate a multi-scale prediction map.

[0023] Preferably, deep supervision is introduced into the fine-tuned network to generate multi-scale masks by calculating the loss function at each layer of the network. The expression of the loss function is:

[0024]

[0025] In the formula, l i represents the loss function of the i-th layer, Bce and Dice represent the binary cross loss function and similarity coefficient loss function respectively, and λ i Represents the loss function coefficients of different layers, y, are the true label and the predicted label respectively.

[0026] Therefore, the present invention adopts the above-mentioned change detection method based on semantic prior and fusion spatial positioning, which has the following technical effects:

[0027] (1) The “pre-training + fine-tuning” technique is adopted. First, the generalization ability and performance of the model are significantly improved through pre-training. Then, based on the pre-trained model, additional training is performed on a specific data set through fine-tuning to adapt it to specific tasks or fields, while reducing the high cost and overfitting risk of training the model from scratch.

[0028] (2) The encoder and decoder structures are expanded by using a twin network structure with shared weights to solve the adaptation problem of dual image inputs. At the same time, a spatial consistency attention module is introduced to enhance the modeling capability of long-distance dependencies through large convolution kernels, thereby improving model performance. By fusing the features in the encoder branch, a feature map containing change information is generated, thereby better capturing the details of dual temporal changes.

[0029] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 BPNet network structure diagram in an embodiment of a change detection method based on semantic prior and fusion spatial positioning;

[0031] Figure 2 It is a diagram of a fine-tuning network structure in an embodiment of a change detection method based on semantic prior and fusion spatial positioning;

[0032] Figure 3 Schematic diagram of each module in the fine-tuning network in an embodiment of a change detection method based on semantic prior and fusion spatial positioning, wherein (a) is a SCAM module, (b) is an ADLKA module, (c) is a TFU module, and (d) is a LFEM module;

[0033] Figure 4 It is a Grad-CAM heat map visualization result corresponding to the LKA module in the embodiment of the change detection method based on semantic prior and fusion spatial positioning, wherein (a) is the first temporal image, (b) is the first temporal feature map of the first layer of the encoder, (c) is the first temporal feature map of the second layer of the encoder, (d) is the first temporal feature map of the third layer of the encoder, (e) is the first temporal feature map of the fourth layer of the encoder, (f) is the first temporal feature map of the fifth layer of the encoder, (g) is the second temporal image, (h) is the second temporal feature map of the first layer of the encoder, (k) is the second temporal feature map of the second layer of the encoder, (l) is the second temporal feature map of the third layer of the encoder, (m) is the second temporal feature map of the fourth layer of the encoder, and (n) is the second temporal feature map of the fifth layer of the encoder;

[0034] Figure 5It is a Grad-CAM heat map visualization result corresponding to the SCAM module in the embodiment of the change detection method based on semantic prior and fusion spatial positioning, wherein (a) is the first temporal image, (b) is the first temporal feature map of the first layer of the encoder, (c) is the first temporal feature map of the second layer of the encoder, (d) is the first temporal feature map of the third layer of the encoder, (e) is the first temporal feature map of the fourth layer of the encoder, (f) is the first temporal feature map of the fifth layer of the encoder, (g) is the second temporal image, (h) is the second temporal feature map of the first layer of the encoder, (k) is the second temporal feature map of the second layer of the encoder, (l) is the second temporal feature map of the third layer of the encoder, (m) is the second temporal feature map of the fourth layer of the encoder, and (n) is the second temporal feature map of the fifth layer of the encoder;

[0035] Figure 6 is a change detection result of scene 1 in an embodiment of a change detection method based on semantic prior and fusion spatial positioning, (a) is a first temporal image, (b) is a second temporal image, (c) is a real change map, (d) is a change detection result of a baseline network, (e) is a change detection result after superimposing an LKA module, and (f) is a change detection result after superimposing a SCAM module;

[0036] Figure 7 is the change detection result of scene 2 in the embodiment of the change detection method based on semantic prior and fusion spatial positioning, (a) is the first temporal image, (b) is the second temporal image, (c) is the real change map, (d) is the change detection result of the baseline network, (e) is the change detection result after superimposing the LKA module, and (f) is the change detection result after superimposing the SCAM module;

[0037] Figure 8 This is the change detection result of scene three in an embodiment of the change detection method based on semantic prior and fusion spatial positioning, (a) is the first temporal image, (b) is the second temporal image, (c) is the real change map, (d) is the change detection result of the baseline network, (e) is the change detection result after superimposing the LKA module, and (f) is the change detection result after superimposing the SCAM module. DETAILED DESCRIPTION

[0038] The present invention can be explained in more detail by the following examples. The purpose of disclosing the present invention is to protect all changes and improvements within the scope of the present invention. The present invention is not limited to the following examples.

[0039] In this embodiment, the change of a building is taken as the detection target to illustrate the framework and effect of the change detection method based on semantic prior and fusion spatial positioning, as follows:

[0040] Building change detection can be seen as a specific extension of the building extraction task, and its core goal is to selectively identify and extract those buildings that have changed by analyzing bi-temporal images, while ignoring those that have not changed. In order to design an effective building change detection model, it is crucial to first let the network learn the prior knowledge of building extraction. This strategy enables the model to transform change detection into a binary classification problem after mastering the basic ability of building extraction, that is, to determine whether the extracted building has changed. To this end, in this embodiment, the BPNet network is pre-trained on the supervised building extraction task to make full use of the prior knowledge in the building extraction task.

[0041] Among them, the BPNet network is a classic U-shaped structure, such as Figure 1 As shown in the figure, it includes a five-stage encoder, a five-stage decoder, a Van module, and a multi-scale fusion module attached to the decoder stage. The specific design is as follows:

[0042] (1) Encoder and decoder structure: The encoder and decoder are composed of two LFEMs respectively. The first LFEM is used to adjust the dimension of the input features, while the second LFEM focuses on extracting deep features. Before the encoder, a preliminary feature extraction module (stem) consisting of 3×3 convolution, batch normalization, and SiLU activation function is first used to expand the feature dimension. Between each encoder, double downsampling is performed through max pooling. In the decoder stage, upsampling is performed through bilinear interpolation. This parameter-free operation can effectively reduce the number of parameters and computation of the model.

[0043] (2) Integration of attention modules: In the U-shaped structure, a Van Block is configured between the symmetrical encoder and decoder to fuse the feature map of the encoder stage with the upsampled feature map of the decoder stage through skip connections. The input of each decoder stage is the sum of the upsampled feature map of the previous stage and the symmetrical encoder feature map processed by the Van Block.

[0044] (3) Processing and output of feature maps: After each decoder stage, the output feature maps are saved, and five saliency feature maps are finally obtained. These feature maps are first processed by the first layer of the multi-scale module and the sigmoid activation function is applied; then, these processed feature maps are concatenated using channels and further processed by the second layer of the multi-scale module, and finally the final output is generated by the sigmoid function.

[0045] In order to adapt the pre-trained model to cope with specific downstream tasks, fine-tuning transfer learning technology is required. However, since the change detection task involves the special data structure of dual image input, it is not applicable to directly fine-tune the semantic segmentation model trained on a single image. It is necessary to make specific adjustments based on the characteristics of the dual image input to ensure the effectiveness and accuracy of the model in processing change detection tasks. Based on this, in this embodiment, a change enhancement network (CAN) is proposed. In the building detection task, it is assumed that the representation module F o With fixed capacity, it consists of K layers L k , k=1…K, and the hidden feature of each layer is c k ∈R nk , where n k is the number of units in the kth layer. Let W k represents the weight matrix between the kth layer and the k-1th layer, and includes the bias term, then c k =f(W k c k-1 ), where f(·) is a nonlinear function, such as SiLU activation function. The fine-tuning network of this embodiment is implemented by adding k Then build a new layer: c , thus forming a variation-enhanced representation module F c . : c It can be viewed as a change detection adaptation layer that allows new combinations of existing operations and avoids drastic modifications to pre-trained layers to adapt to new tasks. In the layer: c The expression is c c =f(W c c k ) is expressed as, where W c Presentation Layer c With L k The weight matrix between c is the newly generated feature.

[0046] In summary, based on the traditional fine-tuning, two important improvements are made to the BPNet network structure to meet the specific needs of bi-temporal image input:

[0047] First, the encoder and decoder structures are expanded. To adapt to the input of dual-temporal images in the change detection task, this embodiment adopts a twin network structure with shared weights, allowing the same encoder and decoder to simultaneously process image inputs at two different time points and ensure that both use the same pre-trained weights. The assumption followed by this method is that the t1 and t2 images come from the same distribution, that is, By sharing weights, the Siamese network is able to extract consistent feature representations between two time points. Specifically, when the encoder and decoder networks receive images from t1 and t2 as input, respectively, the generated feature representation is c 1 k =F o (t 1 ), c 2 k =F o (t 2 ). In this way, the consistency of the bi-temporal images in the feature space is ensured, thereby improving the performance of the model in the change detection task.

[0048] Second, introduce bi-temporal information processing. In the change detection task, in order to effectively capture the bi-temporal relationship, it is not enough to rely on the pre-trained network alone. The spatially consistent inductive bias is introduced into the model to increase the capacity of the model. At the same time, the Van module of the pre-trained network BPNet is replaced by the SCAM module to achieve the fusion of bi-temporal information; in addition, in order to deal with the channel mismatch problem caused by the expansion of the twin network, the fusion block is introduced after the decoder to continue to reduce the feature dimension. These two items are used as the change detection adaptation layer L c ,Specifically, the fusion block generates a richer feature representation by combining the ,features at time points t1 and t2 together, and reduces the feature ,dimension to one half after channel concatenation to match the number of channels of the ,multiscale fusion segmentation head.

[0049] Through the above two processing operations, the stability of the model structure can be guaranteed, and at the same time, the dual-temporal information processing capability can be effectively introduced to achieve more efficient change detection. Although the model has been expanded and some new units have been added, the overall parameter amount and computational complexity are still low, and the cost of complete fine-tuning is within an acceptable range. Through the building extraction model after CAN expansion, the design has rich multi-scale features and low calculation and parameter amount. The pre-trained weights of the building extraction network can be imported into the change detection architecture. Only a very small amount of parameters need to be introduced, so that the change detection network can learn more robust building features and improve performance. In addition, the building extraction model after CAN expansion needs to process multi-scale information, so deep supervision is introduced to improve the prediction accuracy of the model. Deep supervision generates multi-scale mask results by calculating the loss function at each layer of the network, which helps to improve the performance of the model at different scales. Specifically, the design of the loss function takes into account the loss of each level and can be expressed as:

[0050]

[0051] In the formula, l irepresents the loss function of the i-th layer, Bce and Dice represent the binary cross loss function and similarity coefficient loss function respectively, and λ i Represents the loss function coefficients of different layers, y, are the true label and the predicted label respectively.

[0052] In the change detection task, in order to effectively capture the bi-temporal information, the model needs to introduce appropriate inductive bias. However, the traditional twin network structure can only process the features of each time point independently, which means that each encoder-decoder branch can only focus on the data of a single time point, ignoring the change association between the two temporal states. Based on this, the spatial consistency attention module (SCAM module) is introduced to generate a feature map containing change information by fusing the features in the encoder branch, so as to better capture the details of the bi-temporal changes. In addition, the SCAM module also combines the advantages of the self-attention mechanism and large convolution kernels, and uses large kernel convolution decomposition technology to generate more accurate attention feature maps.

[0053] like Figure 3 As shown in Figure 1, the SCAM module includes an ADLKA module and a TFU module: the ADLKA module can capture a wider range of contextual information and fuse change information without destroying the integrity of the bitemporal features; the TFU module is designed to retain the temporal information in the bitemporal features.

[0054] First, the ADLKA module outputs the attention map, which is expressed as follows:

[0055]

[0056] In the formula, Represents the mixed bi-temporal information features after passing through the TFU module; Represents the attention map output by the ADLKA module; Denote depth dilated convolution and depth convolution respectively, which are used to extract context features. In this embodiment, k1=5, k2=5, d=3 are set to balance performance and computational complexity; Conv 1×1 Represents point-wise convolution, which is used to fuse local and contextual features to generate attention maps.

[0057] Furthermore, since it is observed that the changed areas in the images at two time points are spatially consistent, it is assumed that the change positions in the bi-temporal images are spatially consistent, and the inductive bias for change detection is incorporated into the model. At this time, the same attention map can act on the features of two time points at the same time, and the expression is as follows:

[0058]

[0059] In the formula, Represents the input features of two temporal states, Represents element-wise product.

[0060] Then, features are extracted through batch normalization (BN), 1×1 convolution, GELU activation function, and FFN module to complete the model training and inference process.

[0061] In addition, since the bitemporal features incorporate temporal information, directly using BN may lead to confusion of temporal information and cause large fluctuations during training. To this end, the TFU module uses LN to normalize each sample independently to better retain the temporal information, thereby reducing fluctuations during training. The specific expression is as follows:

[0062]

[0063] X s =Conv 1×1 (X m );

[0064] T l =GELU(LN(X s ));

[0065] In the formula, represents the characteristics after channel connection, represents the features reduced by 1×1 convolution, represents the time-fused bi-temporal features, LN represents layer normalization, GELU represents activation function, and concat represents channel connection.

[0066] As the model size increases, its deployment difficulty also increases. Based on this, the feature enhancement module (LFEM) is adopted to maintain computational and parameter efficiency while improving the model's feature extraction capabilities. LFEM is similar to the mobile backward bottleneck convolution (MBConv) module, including a 1×1 convolution for dimension expansion, followed by a 3×3 depthwise convolution for cross-channel interaction, and the introduction of the SE module for weighted processing of feature maps; finally, the feature map is mapped to the target output channel through a 1×1 convolution, and the specific expression is as follows:

[0067]

[0068]

[0069] In the formula, represents the input features, represents the feature after expansion through the first 1×1 convolution, represents the output feature, n=1,2 represents the first and second tense states, and ⊕ represents element-wise addition. Each convolutional layer is followed by batch normalization and SiLU activation function. When the input channel is equal to the output channel, that is, C in =C out , residual connections are used to shorten the gradient back propagation path, and Droppath is used to randomly skip the network path to achieve random depth of the network, so as to reduce the complexity of the network and prevent overfitting of the model.

[0070] In addition, in order to achieve multi-scale prediction output and solve the problem that back propagation is difficult to reach the shallow layer, the multi-scale fusion segmentation head (MSFSH) is introduced. 3×3 convolution is used in each decoder layer of the network to generate a prediction map of the corresponding scale, and the prediction map is adjusted to the original image size through bilinear interpolation. Finally, 1×1 convolution is used to fuse the multi-scale prediction map to generate the final feature map output.

[0071] Verification Example 1

[0072] The building extraction model after CAN expansion is recorded as Uchanger model. Uchanger design has rich multi-scale features and low calculation and parameter amount. In this embodiment, two instances of Uchanger are provided by using different filter number configurations: the normal version Uchanger-base (2.782M) and the relatively small version Uchanger-small (0.593M). The detailed configuration files are shown in Table 1.

[0073] Table 1 Detailed configuration files of two instances of the Uchanger model

[0074]

[0075] This verification example uses the existing FC-EF, FC-Siam-Conc, FC-Siam-Diff, DTCDSCN, ChangeFormer, IFN, BiT, ChangerEx, SGSLN / 512, CGNet, C2Fnet, and SRCNet change detection methods to compare and analyze with the change detection method of this embodiment, as shown in Table 2. It can be seen that the methods Uchanger_small and Uchanger_base of this embodiment both achieved high F1 scores of 92.45% and 92.87%, respectively, which shows that the method of this embodiment has advantages in accuracy and stability in identifying change areas, and also shows that it has strong performance when processing complex scenes.

[0076] Table 2 Results of different change detection methods on the LEVIR-CD dataset

[0077] method Accuracy Recall F1 score FC-EF 86.91 80.17 83.40 FC-Siam-Conc 91.99 76.77 83.69 FC-Siam-Diff 89.53 83.31 86.31 DTCDSCN 88.53 86.83 87.67 ChangeFormer 92.59 89.68 91.11 IFN 94.02 82.93 88.13 BiT 91.97 88.62 90.26 ChangerEx 92.97 90.61 91.77 SGSLN / 512 93.07 91.61 92.33 CGNet 93.15 90.90 92.01 C2FNet 93.69 90.04 91.83 SRCNet 92.63 91.72 92.24 Uchanger_small 92.93 91.96 92.45 Uchanger_base 93.18 92.61 92.87

[0078] In other verification examples, the method of this embodiment performs well on the LEVIR-CD+, S2looking, and CDD datasets. In particular, for the situation where the two-phase images in rural areas in the S2Looking dataset have large style differences, the method of this embodiment uses an improved change detection inductive bias to capture changes more accurately and efficiently, and can better adapt to the complex and changing scenes in the dataset, thereby achieving more significant performance improvement on the S2Looking dataset.

[0079] In another verification example, the method of this embodiment is compared with the existing model in terms of parameter number, computational complexity and accuracy. Compared with the second-ranked model, the method of this embodiment achieves an F1 score of 92.45 using only 0.607M parameters (a decrease of 90%) and 6.053MACs (a decrease of 47%) in computational complexity, surpassing other change detection models in performance with lower theoretical parameter and computational complexity.

[0080] Ablation experiment

[0081] In this embodiment, the MSFSH module aims to maximize the use of feature information at different scales to achieve more accurate building change detection, which can not only better capture key subtle changes, but also effectively enhance the learning of shallow features by shortening the path of gradient backpropagation, reduce the impact of gradient vanishing or explosion problems, and thus improve the overall performance of the model. The SCAM module is a key component for enhancing the model's receptive field and bi-temporal information interaction. The core purpose of the CAN strategy training model is to introduce prior knowledge of building extraction, thereby improving the performance of the model in change detection tasks.

[0082] Table 3 Ablation experimental results of each module on the LEVIR-CD dataset

[0083]

[0084] In order to verify the effectiveness of the proposed module, an ablation experiment was conducted on the LEVIR-CD dataset. It can be seen that the model using MSFSH improves the F1 score by 0.38% compared with the baseline model; this result significantly shows that the multi-scale fusion segmentation head plays an important role and necessity in improving the accuracy and efficiency of the model, as shown in Table 3.

[0085] Compared with the second group of experiments, it was also found that only introducing large convolution kernels (the third group of experiments) significantly improved the recall rate (increased by 1.41%), but caused the accuracy rate to drop synchronously (reduced by 1.37%). Compared with the third group of experiments, when adding the dual-temporal interaction function on the basis of the large convolution kernel (the fourth group of experiments), only 0.066M parameters were added, but MACs was reduced by 0.11G, and both the recall rate and the accuracy rate were significantly improved, and finally the F1 score was improved by 0.57% compared with the baseline model. This shows that model design not only needs to enhance the capabilities of a single branch, but also must pay attention to the interactivity between different branches, so as to improve the performance comprehensively.

[0086] In addition, through the fourth and fifth group of experiments, it can be seen that after introducing the prior knowledge of building extraction, the model performance is significantly improved after fine-tuning on the change detection dataset, and the recall rate and F1 score are increased by 0.64% and 0.33% respectively. These results fully demonstrate the effectiveness of the prior knowledge of building extraction in the change detection task. By introducing this prior knowledge, the model not only enhances the ability to capture target features, but also improves the accuracy and stability in complex change detection scenarios, effectively improving the model performance.

[0087] On the other hand, Grad-CAM is also used to perform heat map analysis on the decoder layer output of the network stacked with LKA modules and SCAM modules, as shown in Figure 4 and Figure 5 As shown in the figure. It can be seen that the LKA module pays more attention to the building and its surroundings more evenly, but cannot distinguish between changed and unchanged buildings. The SCAM module can not only accurately mark the area to be built, but also pay special attention to the changed buildings, while ignoring the unchanged parts. These results show that the SCAM module has a significant advantage in capturing the characteristics of changes, thereby greatly improving the overall performance of the model. In addition, in the analysis of the contribution of the SCAM module to the expansion of the receptive field, it is found that: (1) the decoder layer tends to focus on local features; (2) as the resolution of the decoder feature map decreases, the effective receptive fields (ERFs) become wider; (3) the SCAM module helps the deep decoder expand the ERFs. This means that the SCAM module can enhance the modeling ability of long-distance dependencies through large convolution kernels, thereby improving the performance of the model.

[0088] In another verification example, the LEVIR-CD dataset was used to verify the effect of the SCAM module in target change detection in different scenarios. When dealing with large-scale targets, the SCAM model performs well in detecting building change areas, especially in detail and edge processing. Figure 6 For scenes blocked by trees, SCAM shows higher robustness and can effectively prevent false detection of changing buildings, such as Figure 7In addition, in complex scenes, SCAM shows great advantages in locating changing buildings, such as Figure 8 shown.

[0089] Therefore, the present invention adopts the above-mentioned change detection method based on semantic prior and fusion spatial positioning, which can reduce the complexity of the network, prevent overfitting, realize the effective fusion of dual-temporal images, better capture timing information, and thus improve the accuracy of target change detection.

[0090] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A change detection method based on semantic prior and fusion spatial positioning, characterized in that: include: Firstly, the BPNet network is pre-trained on the supervised remote sensing extraction task to make full use of the prior knowledge in the remote sensing extraction task; Then, the pre-trained BPNet network is fine-tuned, and the encoder and decoder structures are expanded using a twin network structure with shared weights. A spatial consistency attention module is introduced. By combining the self-attention mechanism and large convolution kernel decomposition technology, feature information of the dual-temporal images is extracted and fused to obtain the target change information of the remote sensing image.

2. The change detection method based on semantic prior and fusion spatial positioning according to claim 1, characterized in that: The encoder and decoder of the BPNet network both use feature enhancement modules for feature extraction. The expression of the feature enhancement module is: In the formula, represents the input features of the feature enhancement module, represents the feature after expansion through the first 1×1 convolution, represents the output features of the feature enhancement module, n represents the temporal state of the feature map, represents element-wise addition, SE represents self-attention mechanism, DWConv 3×3 They represent the depth-separable convolution with a convolution kernel of 3×3, Conv 1×1 Represents a 1×1 convolution.

3. The change detection method based on semantic prior and fusion spatial positioning according to claim 1, characterized in that: The spatial consistency attention module includes the ADLKA module and the TFU module: the ADLKA module is used to capture broader contextual information and fuse change information; the TFU module is used to retain the temporal information in the bi-temporal features.

4. The change detection method based on semantic prior and fusion spatial positioning according to claim 3 is characterized in that: The expression of the ADLKA module output is: In the formula, represents the mixed bi-temporal information features after passing through the TFU module, represents the attention map output by the ADLKA module, Indicates that the convolution kernel is a k2×k2 depth dilated convolution, Conv represents a depthwise convolution with a convolution kernel of k1×k1. 1×1 Represents point-wise convolution.

5. The change detection method based on semantic prior and fusion spatial positioning according to claim 3, characterized in that: The TFU module uses layer normalization to process each sample independently, as shown below: T l =GEL(LN(X s )); In the formula, Represent the input features of the first and second tenses respectively, represents the features reduced by 1×1 convolution, represents the time-fused bi-temporal features, concat represents channel connection, LN represents layer normalization, and GELU represents the activation function.

6. The change detection method based on semantic prior and fusion spatial positioning according to claim 1, characterized in that: A multi-scale fusion segmentation head is introduced after the encoder to convolve the feature maps extracted from each layer to generate prediction maps of the corresponding scale. The prediction maps of each layer are adjusted to the size of the original image through bilinear interpolation and fused to generate multi-scale prediction maps.

7. The change detection method based on semantic prior and fusion spatial positioning according to claim 1, characterized in that: In the fine-tuned network, deep supervision is introduced to generate multi-scale masks by calculating the loss function at each layer of the network. The expression of the loss function is: In the formula, l i represents the loss function of the i-th layer, Bce and Dice represent the binary cross loss function and similarity coefficient loss function respectively, and λ i Represents the loss function coefficients of different layers, y, are the true label and the predicted label respectively.

Images