A remote sensing image change detection method and system based on transformer and dense feature fusion

Abstract

The application discloses an image processing technical field and a remote sensing image change detection method and system based on a Transformer and dense feature fusion. First, a backbone network based on ResNeT18 is used for preliminary feature extraction, then two feature maps extracted are subjected to semantic Tokenize to generate two groups of tokens sequences, the sequences are spliced and input into a Transformer encoder, interaction between different heads is carried out under the premise of guaranteeing multi-head diversity through an improved multi-head attention, the network is helped to model the global information of an image in a token-based space, then tokens rich in context information are segmented and input into a twin Transformer decoder, are projected back to a pixel space, two enhanced feature maps are obtained, and a dense feature fusion module is used to help the network to generate a change region and enhance edge information. Finally, a change result map is generated through feature fusion and a shallow convolutional network. The method can improve the performance of the network and improve the change detection precision.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a method and system for detecting changes in remote sensing images based on Transformer and dense feature fusion. Background Technology

[0002] Change detection in remote sensing imagery involves quantitatively analyzing and determining the characteristics and processes of surface changes from remote sensing data from different periods. High-resolution remote sensing images, due to varying imaging conditions and the complex and diverse nature of objects within the scene, can exhibit different spectral characteristics for objects with the same semantic concept at different times and spatial locations. This presents a significant challenge for change detection. To accurately identify changed regions in complex scenes, change detection models need to recognize high-level semantic information about these regions, distinguishing genuine changes from complex, seemingly unrelated areas of change.

[0003] By utilizing self-attention to model the semantic relationships between pixel pairs in spatiotemporal space, better detection results can be achieved. In 2020, Hao Chen et al. from Beihang University proposed STANet, a novel Siamese network based on metric and spatiotemporal attention. It divides the network into different scales, embeds self-attention within each small region to model long-distance dependencies between pixels, and uses a batch-balanced contrastive loss function to alleviate the problem of sample imbalance, helping the network extract features rich in contextual information. However, the model has very low computational efficiency, high computational complexity, and its computational complexity increases quadratically with the number of pixels.

[0004] Many current change detection methods based on deep convolutional neural networks fail to effectively capture long-distance dependencies between pixels. Against this backdrop, Hao Chen et al. from Beihang University proposed the Bitemporal Image Transformer (BIT) detection algorithm, achieving good results in change detection tasks. However, because each head of the multi-head attention mechanism in the Transformer is only responsible for a subset of the input token sequence, when the channel dimension of the subset is small, the dot product of the query and key cannot form an information function, leading to a decrease in network performance. Furthermore, BIT's direct subtraction feature fusion method not only easily damages the feature structure but may also introduce noise interference. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a remote sensing image change detection method and system based on Transformer and dense feature fusion, so that the final change detection results of dual-temporal multispectral remote sensing images are more accurate.

[0006] To achieve the above objectives, the technical solution adopted by this invention is: a remote sensing image change detection method based on Transformer and dense feature fusion, comprising:

[0007] a. For the registered dual-temporal optical remote sensing images, preliminary feature extraction is first performed using a backbone network based on a convolutional neural network (CNN). The backbone network uses a ResNeT18 network based on improved residual blocks.

[0008] b. Use a semantic tokenizer to generate corresponding token groups from the image features in step a, that is, the image features can be represented by token groups.

[0009] c. Contextual information is modeled in the token-based spatiotemporal space using an improved Transformer encoder. Then, the tokens rich in contextual information are decoded by a Transformer decoder and reprojected into the pixel space to obtain the refined original features.

[0010] d. Feature maps enhanced by the Transformer decoder The DenseFusion Module (DFM) improves upon the direct subtraction and absolute value fusion method by integrating multiple features to help the network make better decisions and improve the model's robustness.

[0011] e. During the training phase, a binary cross-entropy function is selected as the network's loss function, and the model parameters are optimized by minimizing the loss function. The prediction phase of the network employs a shallow convolutional network, where the classifier consists of two 3×3 convolutional layers with batch normalization. The output channels of the convolutional layers are 32 and 2 respectively. The classifier is followed by a Softmax function to output a probability map.

[0012] In step a, the improved ResNeT18 residual block uses the original ResNet, which has five stages, each of which performs a 2x downsampling. To reduce the loss of spatial detail information in the image, feature extraction uses only the first four stages of ResNeT18, and the stride in the last two stages of ResNeT18 is set to 1. Then, a pointwise convolution with 32 output channels is added to the end of ResNet to reduce the dimensionality of the image, and the output feature map is obtained by upsampling.

[0013] Step b employs a Siamese semantic tokenizer to enable the two-phase images to share semantic concepts, transforming the initially extracted feature maps into compact semantic tokens. In NLP, a tokenizer divides the input sentence into other elements, i.e., words or phrases, and represents them using token vectors. Similarly, this invention uses a semantic tokenizer to transform the initially extracted feature maps into visualized words, with each word corresponding to a token vector. The semantic tokenizer learns a set of spatial attention mappings, transforming the feature maps into compact token sets.

[0014] set up These are the input feature maps corresponding to two temporal remote sensing images, where H, W, and C correspond to the height, width, and channel dimension of the feature map, respectively. There are two sets of tokens, where L is the size of the token vocabulary. The specific process is as follows: First, process the feature map X... i Each pixel on (i = 1, 2) Pointwise convolution is used to obtain L semantic groups, each representing a semantic concept. Then, a spatial attention map is calculated using the Softmax function for each semantic group with dimension HW. Finally, X is calculated using the attention map. i The weighted average sum of the pixels yields a compact vocabulary of size L, i.e., semantic tokensT. i .

[0015] T i =(A i ) T X i =(σ(Φ(X) i ;W))) T X i (1)

[0016] Where Φ represents the input image X i Perform pointwise convolution with kernel W, where σ represents the Softmax function.

[0017] The Transformer encoder and decoder in step c include:

[0018] The encoder typically consists of N identical layers, each containing two sub-layers with residual structures. The first sub-layer comprises Multi-Head Attention (MSA) and Layer Normalization (LN), while the second sub-layer consists of a Feedforward Network (FFN) and LN. The decoder has a similar structure to the encoder, but with a third sub-layer, CrossAttention, which interacts with the encoder. Conventionally, each head is only responsible for a subset of the input tokens. Especially when the token embedding dimension is small, the dot product of query and key cannot form an information function, degrading network performance. This invention employs an improved MSA:

[0019] Instance normalization is applied to the dot product matrix after the Softmax function to reconstruct the diversity capability of multi-head attention. In this way, the interactions between different heads are modeled while preserving multi-head diversity; the mathematical expression is:

[0020]

[0021] In the formula, Q, K, and V represent the query matrix, key matrix, and value matrix, respectively; IN represents instance normalization, which calculates the H and W dimensions separately, helping the network converge while maintaining the independence of each image instance and channel; Conv(·) is a standard 1×1 convolution operation with cross-channel information interaction, used to model the interaction between different heads, achieving the effect that each head can depend on all keys and queries for attention; dk is the vector dimension of query and key.

[0022] The Transformer encoder consists of N E The system consists of improved multi-head attention and multilayer perceptron (MLP). Unlike the Post-LN structure used in the standard transformer, this invention uses a Pre-LN structure, which places the layer normalization LN before the MSA and MLP, respectively. This can help make the network training process more stable and converge faster.

[0023] In the Transformer, the input to the self-attention of each layer l is in the form of a triplet, i.e., (Q, K, V), which passes through the input of layer (l-1). The calculation is as follows:

[0024] Q = T (l-1) W q (3)

[0025] K = T (l-1) W k(4)

[0026] V = T (l-1) W v (5)

[0027] In the formula Let Q, K, and V be the parameters of the linear projection layer, and d be the channel dimension.

[0028] The core idea of ​​the Transformer encoder is multi-head attention. Multi-head attention first executes multiple independent attention heads in parallel, then concatenates the outputs of these independent attention heads, and finally projects the result to obtain the final output. Its advantage is that it can jointly process information from different positions in different representation subspaces. The j-th head... j The calculation formula is as follows:

[0029]

[0030] In the formula These are the parameters of the linear projection layer corresponding to the j-th Q, K, V.

[0031] The token sequence is obtained by concatenating multiple heads of the ESA and then passing them through the MLP neural network in the second sub-layer.

[0032] cb. This invention uses a twin Transformer decoder to extract feature maps from each input image. Unlike the standard Transformer decoder, the Transformer decoder used consists of N... D It consists of multi-head cross-attention (MA) layers and MLP blocks, eliminating multi-head attention and thus avoiding X. i The extensive computation between pixels, and the layer normalization location, employs a Pre-LN structure similar to the encoder. In the Transformer encoder's MSA, the query, key, and value come from the same input sequence; in the Transformer decoder, the query comes from the input image features X. i The keys and values ​​come from the encoder's output, i.e. Each pixel of the input image can be represented by a compact combination of semantic tokens, given an input feature sequence X. i Using pixels as queries and tokens as keys, the Transformer decoder combines each pixel with a token group. The relationship between them yields a precise feature map.

[0033] In step d, the Dense Feature Module (DFM) consists of two branches: a sum branch and a difference branch. Each branch contains two streams, and the weights of the two streams within the branch are shared. The module uses the sum branch to obtain enhanced edge information and the difference branch to generate varied regions. All convolutional operations use 3×3 convolutional kernels, and batch normalization (BN) is applied after the final convolutional layer. Furthermore, due to the presence of residual connections in the dense connections, the last two features in each stream can be considered as residuals of the previous feature, which to some extent corrects the previous feature, making the new feature map more aligned.

[0034] The binary cross-loss function in step e is calculated using the following formula:

[0035]

[0036] In the formula, BceLoss is the binary crossover loss. y is the predicted value, and y is the actual value. The greater the difference between the predicted and actual values, the greater the loss.

[0037] A method and system for detecting changes in remote sensing images based on Transformer and dense feature fusion, comprising a processor and a storage device, wherein the storage device stores multiple instructions for the processor to load and execute the steps of any of the methods described above.

[0038] Compared with existing technologies, the beneficial effects achieved by this invention are as follows: The remote sensing image change detection method and system based on Transformer and dense feature fusion described in this invention first uses a ResNeT18-based backbone for preliminary feature extraction. Then, the two extracted feature maps are semantically tokenized to generate two sets of token sequences. These sequences are concatenated and input into the Transformer encoder. Through improved multi-head attention, interactions between different heads are performed while ensuring multi-head diversity, helping the network model the global information of the image in a token-based space. Next, the tokens rich in contextual information are segmented and input into a Siamese Transformer decoder, which is then reprojected back into the pixel space to obtain two enhanced feature maps. Finally, a dense feature fusion module helps the network generate change regions and enhance edge information. This method makes the final change detection results of dual-temporal remote sensing images more reliable, robust, and accurate. Attached Figure Description

[0039] Figure 1 This is an overall block diagram of a remote sensing image change detection method and system based on Transformer and dense feature fusion according to an embodiment of the present invention.

[0040] Figure 2 This is an ESA (Improved MSA) structure diagram in a remote sensing image change detection method based on Transformer and dense feature fusion according to an embodiment of the present invention.

[0041] Figure 3 This is a schematic diagram of the Dense Feature Fusion (DFM) module in a remote sensing image change detection method based on Transformer and dense feature fusion according to an embodiment of the present invention.

[0042] Figure 4 This refers to the LEVER optical remote sensing image change dataset and detection results used in this embodiment of the invention. Figure 4 (a), (b), (c), (d), and (e) represent the image before the change, the image after the change, the changed reference image, the detection results of the FC-Siam-Conc algorithm, the detection results of STANet, and the detection results of this invention, respectively. Detailed Implementation

[0043] A remote sensing image change detection method based on Transformer and dense feature fusion mainly includes: First, preliminary feature extraction is performed using a ResNeT18-based backbone network. Then, the two extracted feature maps are semantically tokenized to generate two sets of token sequences. These sequences are concatenated and input into a Transformer encoder. Through improved multi-head attention, interactions between different heads are performed while maintaining multi-head diversity, helping the network model global information of the image in a token-based space. Next, the tokens rich in contextual information are segmented and input into a Siamese Transformer decoder, which reprojects them back into the pixel space to obtain two enhanced feature maps. A dense feature fusion module helps the network generate change regions and enhance edge information. Finally, a change result map is generated after feature fusion and a shallow convolutional network. The proposed method improves network performance and increases the accuracy of change detection.

[0044] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0045] like Figure 1 The diagram shown illustrates the implementation flow of the remote sensing image change detection method based on Transformer and dense feature fusion described in this invention, including the following steps:

[0046] Step 1: Input two registered optical remote sensing images of the same area but at different times, denoted as I. (1) and I(2) Considering that residual structures can solve the gradient vanishing problem caused by the increasing number of network layers, this invention initially uses a ResNet18 network as the backbone network for feature extraction, and improves upon it. The original ResNet18 has five stages, with each stage performing a 2x downsampling. In order to reduce the loss of spatial detail information in the image, feature extraction only uses the first four stages of ResNet18, and the stride in the last two stages of ResNet18 is set to 1. Then, a pointwise convolution with 32 output channels is added to the end of ResNet to reduce the dimensionality of the image, and the output feature map is obtained by upsampling.

[0047] Step 2: A Siamese semantic tokenizer is used to enable the two-phase images to share semantic concepts, transforming the initially extracted feature maps into compact semantic tokens. In NLP, a tokenizer divides the input sentence into other elements, i.e., words or phrases, and represents them with token vectors. Similarly, this invention uses a semantic tokenizer to transform the initially extracted feature maps into visualized words, with each word corresponding to a token vector. A set of spatial attention mappings is learned through the semantic tokenizer, transforming the feature maps into compact token sets.

[0048] set up These are the feature maps processed in step 1 for the two corresponding temporal remote sensing images. H, W, and C correspond to the height, width, and channel dimension of the feature map, respectively. There are two sets of tokens, where L is the size of the token vocabulary. The specific process is as follows: First, process the feature map X... i Each pixel on (i = 1, 2) Pointwise convolution is used to obtain L semantic groups, each representing a semantic concept. Then, a spatial attention map is calculated using the Softmax function for each semantic group with dimension HW. Finally, X is calculated using the attention map. i The weighted average sum of the pixels yields a compact vocabulary of size L, i.e., semantic tokensT. i .

[0049] T i =(A i ) T X i =(σ(Φ(X) i ;W))) T X i (1)

[0050] In the formula, Φ represents the input image X. i Perform pointwise convolution with kernel W, where σ represents the Softmax function.

[0051] Step 3: Build the encoder and decoder of the Transformer respectively.

[0052] Step 3.1: Constructing an improved multi-head attention ESA in the Transformer encoder

[0053] An encoder typically consists of N identical layers, and each layer contains two sub-layers with residual structures. The first sub-layer consists of MSA and LN, and the second sub-layer consists of FFN and LN. The decoder has a similar structure to the encoder, but it contains a third sub-layer with Cross Attention, which interacts with the encoder.

[0054] Multi-head attention (MSA) has two drawbacks: ① The computational cost increases quadratically with the number or dimensionality of tokens, resulting in significant computational and inference overhead during training; ② Each head is only responsible for a subset of the input tokens, especially when the token embedding dimension is small, the dot product of query and key cannot form an information function, reducing network performance. To balance efficiency and accuracy, the BIT network sets the encoder depth (i.e., the number of Transformer layers) to 1 and the decoder depth to 8. Unlike standard Transformer implementations, the Transformer decoder consists of ND Multi-head Cross-Attention (MA) and MLP blocks, removing MSA to avoid extensive inter-pixel computation.

[0055] Based on this, the present invention improves upon the second drawback of MSA mentioned above, as shown in the appendix. Figure 2 As shown. Conv(·) is a standard 1×1 convolution operation with cross-channel information interaction. It is used to model the interaction between different heads, achieving the effect that each head of attention can depend on all keys and queries. However, this weakens the joint processing capability of multi-head attention for different locations of different representation subsets. Initially, instance normalization (IN) was used for image style transfer. Its normalization calculation method differs from batch normalization (BN) and layer normalization (LN). It is independent of the batch dimension and channel dimension. It calculates the H and W dimensions separately, which helps the network converge while maintaining the independence of each image instance and channel. The improved MSA used in this invention is called ESA. It adds instance normalization to the dot product matrix after the Softmax function to reconstruct the diversity capability of multi-head attention. In this way, the interaction between different heads is modeled while maintaining the diversity of multi-head attention. The mathematical expression is:

[0056]

[0057] In the formula, Q, K, and V represent the query matrix, K represents the key matrix, and V represents the value matrix, respectively; IN represents instance normalization, which calculates the H and W dimensions separately, helping the network converge while maintaining the independence of each image instance and channel; Conv(·) is a standard 1×1 convolution operation with cross-channel information interaction, used to model the interaction between different heads, achieving the effect that each attention head can depend on all keys and queries; d k It is the vector dimension of query and key.

[0058] Step 3.2: Build the Transformer encoder

[0059] The Transformer encoder consists of N E The system consists of an ESA and a multilayer perceptron (MLP). Unlike the Post-LN structure used in the standard transformer, this invention uses a Pre-LN structure, which places the layer normalization (LN) before the ESA and MLP. This can help make the network training process more stable and the convergence speed faster.

[0060] In the Transformer, the input to the self-attention of each layer l is in the form of a triplet, i.e., (Q, K, V), which passes through the input of layer (l-1). The calculation is as follows:

[0061] Q = T (l-1) W q (3)

[0062] K = T (l-1) W k (4)

[0063] V = T (l-1) W v (5)

[0064] In the formula Let Q, K, and V be the parameters of the linear projection layer, and d be the channel dimension.

[0065] The core idea of ​​the Transformer encoder is multi-head attention. Multi-head attention first executes multiple independent attention heads in parallel, then concatenates the outputs of these independent attention heads, and finally projects the result to obtain the final output. Its advantage is that it can jointly process information from different positions in different representation subspaces. The j-th head... j The calculation formula is as follows:

[0066]

[0067] In the formula These are the parameters of the linear projection layer corresponding to the j-th Q, K, V.

[0068] The token sequence is obtained by concatenating multiple heads of the ESA and then passing them through the MLP neural network in the second sub-layer.

[0069] Step 3.3: Build the Transformer decoder

[0070] A twin Transformer decoder is used to extract feature maps from each input image. Unlike the standard Transformer decoder, the Transformer decoder used consists of N... D It consists of multi-head cross-attention (MA) layers and MLP blocks, eliminating multi-head attention and thus avoiding X. i The extensive computation between pixels, and the layer normalization location, employs a Pre-LN structure similar to the encoder. In the Transformer encoder's MSA, the query, key, and value come from the same input sequence; in the Transformer decoder, the query comes from the input image features X. i The keys and values ​​come from the encoder's output, i.e. Each pixel of the input image can be represented by a compact combination of semantic tokens, given an input feature sequence X. i Using pixels as queries and tokens as keys, the Transformer decoder combines each pixel with a token group. The relationship between them yields a precise feature map.

[0071] Step 4: Construct Dense Fusion Module (DFM)

[0072] Appendix Figure 3 This is a schematic diagram of the DFM module, consisting of two branches: a sum branch and a difference branch. The input to the DFM module is the feature Z output from the dual-temporal image Transformer decoder. (1) and Z (2) Each branch contains two streams, and the weights of the two streams within a branch are shared. The module uses the summation branch to obtain enhanced edge information and the difference branch to generate varied regions. All convolutional operations use 3×3 convolutional kernels, and batch normalization (BN) is applied after the final convolutional layer. In addition, due to the presence of residual connections in the dense connections, the last two features in each stream can be regarded as residuals of the previous feature, which to some extent corrects the previous feature and makes the new feature map more aligned.

[0073] Step 5: During the network training phase, a binary classification crossover loss function is used, calculated as follows:

[0074]

[0075] In the formula, BceLoss is the binary crossover loss. y is the predicted value, and y is the actual value. The greater the difference between the predicted and actual values, the greater the loss.

[0076] Step 6: After normalizing the two multispectral images from different time periods, input them into the trained Transformer network to classify them into changed and unchanged categories.

[0077] The invention will be further described in detail below with specific experimental data. The experimental data used in this experiment is the LEVER change detection data, with images cropped to 256×256. The experiment is based on the PyTorch framework, with a batch size of 8, epochs of 200, a learning rate of 0.01, linear decay to 0, and stochastic gradient descent (SGD) as the optimizer. In the experiment, the semantic token length L was set to 4, the Transformer encoder had 1 layer, the decoder had 8 layers, and the number of heads in the MSA and MA was set to 8, with each head having a channel dimension of 8. A shallow convolutional network was used to identify change regions during the prediction phase, and a binary cross-entropy loss function was used to optimize the network parameters during the training phase.

[0078] To verify the effectiveness of the present invention, the change detection method of the present invention is compared with the following change detection method:

[0079] (1) The STANet detection method proposed by Hao Chen et al. [“A Spatial-Temporal Attention-Based Method and a New Dataset for Remote Sensing Image Change Detection.(Remote Sensing, 2020, 12(10)]

[0080] (2) The BIT detection method (GAN) proposed by Hao Chen et al. [The detection method proposed by Hao Chen et al. in the article "Remotesensing image change detection with transformers. (IEEE Transactions on Geoscience and Remote Sensing, 2022, 60)]

[0081] (4) The method of the present invention.

[0082] Detection performance was assessed using precision, recall, and F1 score. F1 is a comprehensive evaluation index for change detection; a value closer to 1 indicates better performance of the change detection method. The detection results are shown in Table 1.

[0083] Table 1. Performance of change detection results (%) on the LEVIR dataset

[0084]

[0085] As shown in Table 1, the detection method proposed in this invention has the highest F1 score, which is closer to 1 than the other three detection algorithms. Furthermore, the Precision and Recall scores of this invention are the highest among the compared algorithms, also closer to 1. In summary, the change detection algorithm of this invention outperforms the other three detection methods, indicating that the change detection method proposed in this invention is effective.

[0086] Figure 4 (a) and (b) are the before and after temporal images in the LEVER dataset. Figure 4 (c) is a reference diagram for change detection. Figure 4 (d), (e), and (f) represent the change detection results of the STANet algorithm, the BIT algorithm, and the present invention, respectively. Figure 4 Based on the comparison of the reference images, the detection effect of the algorithm proposed in this invention is the best in terms of visual effect.

[0087] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0088] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0089] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0090] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0091] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A remote sensing image change detection method based on Transformer and dense feature fusion, characterized by: a. For the registered dual-temporal optical remote sensing images, preliminary feature extraction is first performed using a convolutional neural network (CNN) backbone network, where the backbone network is a ResNet18 network based on improved residual blocks. b. Use a semantic tokenizer to generate corresponding token groups from the image features extracted in step a, that is, the image features are represented by token groups; c. By improving the multi-head attention MSA in the Transformer encoder to ESA, contextual information is modeled in the token-based spatiotemporal space. Then, the Transformer decoder decodes the tokens rich in contextual information and reprojects them into the pixel space to obtain the refined original features. d. Feature maps enhanced by the Transformer decoder The Dense Fusion Module (DFM) improves the direct subtraction and absolute value fusion method by integrating multiple features to help the network make better decisions and improve the robustness of the model. The DFM module consists of sum branches and difference branches, each branch contains two streams, and the network weights of the two streams within each branch are shared. e. During the training phase, the binary cross-entropy function is selected as the loss function of the network, and the model parameters are optimized by minimizing the loss function. During the prediction phase of the network, a shallow convolutional network is used, in which the classifier consists of two 3×3 convolutional layers with batch normalization. The output channels of the convolutional layers are 32 and 2, respectively. The classifier is followed by the Softmax function to output the change probability map.

2. The remote sensing image change detection method based on Transformer and dense feature fusion according to claim 1, characterized in that, The improved residual block ResNet18 model in step a aims to increase spatial detail information in the image, thereby providing better features for subsequent change detection. In the original five stages of ResNet18, the last stage is discarded, and feature extraction only uses the first four stages of ResNet18. The stride in the last two stages of ResNet18 is set to 1. Then, a pointwise convolution with 32 output channels is added to the end of ResNet to reduce the dimensionality of the image, and the output feature map is obtained by upsampling.

3. The remote sensing image change detection method based on Transformer and dense feature fusion according to claim 1, characterized in that, The semantic tokenizer in step b is a twin semantic tokenizer, which enables the two-phase images to share semantic concepts and transforms the initially extracted feature map into compact semantic tokens. In NLP, the tokenizer divides the input sentence into other elements, namely words or phrases, and represents them with token vectors. The semantic tokenizer is used to transform the initially extracted feature map into visualized words, with each word corresponding to a token vector. Through the semantic tokenizer, a set of spatial attention mappings is learned to transform the feature map into a compact set of tokens. set up These are the input feature maps corresponding to two temporal remote sensing images, where H, W, and C correspond to the height, width, and channel dimension of the feature map, respectively. There are two sets of tokens, where L is the size of the token vocabulary; the specific process is as follows: first, process the feature map X... i Each pixel on Pointwise convolution is used to obtain L semantic groups, each representing a semantic concept. Then, a spatial attention map is calculated using the Softmax function for each semantic group with dimension HW. Finally, X is calculated using the attention map. i The weighted average sum of the pixels yields a compact vocabulary of size L, i.e., semantic tokens T. i Its calculation is expressed as follows: T i =(A i ) T X i =(σ(Φ(X i ;W))) T X i (1) Where Φ represents the input image X i Perform pointwise convolution with kernel W, where σ represents the Softmax function.

4. The remote sensing image change detection method based on Transformer and dense feature fusion according to claim 1, characterized in that, The encoder and decoder construction in step c includes: ca, Constructing an improved multi-head attention ESA After applying the Softmax function to the standard multi-head attention algorithm, instance normalization is added to the dot product matrix to reconstruct the diversity capabilities of multi-head attention. This approach models the interactions between different heads while preserving the diversity of the multi-head system. The mathematical expression is as follows: In the formula, Q, K, and V represent the query matrix, K represents the key matrix, and V represents the value matrix, respectively; IN represents instance normalization, which calculates the H and W dimensions separately, helping the network converge while maintaining the independence of each image instance and channel; Conv(·) is a standard 1×1 convolution operation with cross-channel information interaction characteristics, used to model the interaction between different heads, achieving the effect that attention depends on all keys and queries for each head; d k It is the vector dimension of query and key; cb. Constructing a Transformer encoder based on an improved multi-head attention ESA. The Transformer encoder consists of N E It consists of layer-modified multi-head attention and multi-layer perceptron (MLP). Unlike the Post-LN structure used in the standard transformer, it adopts a Pre-LN structure, which places layer normalization (LN) before the MSA and MLP, to make the network training process more stable and converge faster. In the Transformer, the input to the self-attention of each layer l is in the form of a triplet, i.e., (Q, K, V), which is passed through the input tokens of layer (l-1). The calculation is as follows: Q=T (l-1) W q (3) K=T (l-1) W k (4) V=T (l-1) W v (5) In the formula Let Q, K, and V be the parameters of the linear projection layer, and d be the channel dimension. The core idea of ​​the Transformer encoder is multi-head attention. Multi-head attention first executes multiple independent attention heads in parallel, then concatenates the outputs of these independent attention heads, and finally projects the result to obtain the final output. Its advantage is that it can jointly process information from different positions in different representation subspaces. The j-th head... j The calculation formula is as follows: In the formula For the parameters of the linear projection layer corresponding to the j-th Q, K, V; The token sequence is obtained by splicing together multiple heads of ESA and then passing through the MLP neural network in the second sub-layer. cc, Building a Transformer Decoder A twin Transformer decoder is used to extract feature maps from each input image. Unlike the standard Transformer decoder, the Transformer decoder used consists of N... D It consists of a multi-head cross-attention layer (MCA) and an MLP block, eliminating the self-attention layer and thus avoiding X. i The large amount of computation between pixels is handled by a Pre-LN structure, similar to the encoder, for layer normalization. In the Transformer encoder's MSA, the query, key, and value come from the same input sequence; in the Transformer decoder, the query comes from the input image feature X. i The keys and values ​​come from the encoder's output, i.e. Each pixel of the input image is represented by a compact combination of semantic tokens, and the input is a feature sequence X. i Using pixels as queries and tokens as keys, the Transformer decoder combines each pixel with a token group. The relationship between them yields a precise feature map.

5. The remote sensing image change detection method based on Transformer and dense feature fusion according to claim 1, characterized in that, The DFM module utilizes summation branches to obtain enhanced edge information and difference branches to generate regions of variation. All convolutional operations use 3×3 convolutional kernels, and batch normalization (BN) is used after the last convolutional layer. In addition, due to the presence of residual connections in dense connections, the last two features in each stream are regarded as residuals of the previous feature, which can correct the previous feature and make the new feature map more aligned.

6. The remote sensing image change detection method based on Transformer and dense feature fusion according to claim 1, characterized in that, During the training phase, for change detection in both the change and non-change categories, a binary cross-loss function is used, calculated as follows: In the formula, BceLoss is the binary crossover loss. y is the predicted value, and y is the actual value. The greater the difference between the predicted and actual values, the greater the loss.

7. A deep learning-based multispectral remote sensing image change detection system, characterized in that, It includes a processor and a storage device, wherein the storage device stores a plurality of instructions for the processor to load and execute the steps of the method according to any one of claims 1 to 6.

Images