Remote sensing image change detection method based on difference enhancement Swin Transform

Abstract

The invention discloses a remote sensing image change detection method based on difference enhancement Swin Transform. The method comprises the following steps: firstly, constructing a backbone network based on a twin neural network architecture for extracting depth features of different levels of a dual-temporal remote sensing image; secondly, a difference enhancement module (DEM) is provided and used for enhancing difference feature learning between the dual-temporal remote sensing images on different levels; then, a Swin Transform module (STM) is introduced, global context information is modeled, and more change details are captured while a network acceptance domain is expanded; finally, a feature weight fusion module (FWFM) is designed to effectively aggregate the multi-level feature difference graph. An attention mechanism is integrated in the difference enhancement module provided by the invention, so that the difference feature expression of the dual-temporal remote sensing image can be effectively improved; meanwhile, the designed feature weight fusion module can effectively solve the problems of semantic loss, uneven contribution degrees of different levels of features and the like during remote sensing image change detection.

Description

Technical Field

[0001] The invention belongs to the field of image processing, and in particular relates to a remote sensing image change detection method based on difference enhancement Swin Transformer. Background Art

[0002] As an important task in the field of remote sensing, change detection aims to capture the difference information of images of the same area at different times and mark the pixel areas that have changed. The core of change detection is to find the difference information. In early studies, simple geometric operations (such as image difference, image ratio, regression analysis, etc.) were usually used to obtain difference maps. Although these methods are simple, they are greatly affected by noise and it is difficult to extract complete changes. Later, in order to improve these shortcomings, people proposed difference map generation methods based on image transformation, such as iterative weighted multivariate analysis, change vector analysis, principal component analysis, independent component analysis, etc. Most of these methods rely on artificial features, which are relatively inefficient and not accurate enough.

[0003] In recent years, deep learning theory has developed rapidly and has become a very successful method in the field of machine learning. Due to the powerful feature extraction and network discrimination capabilities of deep learning, more complex and deep features in images can be extracted, and the network can be quickly optimized by back propagation. For this reason, change detection methods based on deep learning have emerged. This type of method can not only improve the accuracy of change detection, but also be more intelligent and automated.

[0004] Publication No. CN116403123A discloses a remote sensing image change detection method, including image acquisition, model pre-training, pyramid feature extraction, change comparison network, and difference feature map processing. First, the original remote sensing image is input into a pre-trained neural network to obtain a feature difference map; then, the difference features are calculated through an inner product operation. This method mainly utilizes a convolutional neural network for change detection and has achieved certain results. However, due to the problems of limited network receptive field and loss of global context information in the convolutional neural network structure adopted, there are many deficiencies in the accuracy and robustness of detection.

[0005] Fang et al. published a paper titled "SNUNet-CD: A Densely Connected Siamese Network for Change Detection of VHR Images" in IEEE Geoscience and Remote Sensing Letters in 2022. This method constructs a densely connected twin network for change detection, and designs a full-scale dense connection method to reduce feature loss during downsampling, achieving certain results. However, this method focuses on the extraction of deep features and ignores shallow fine-grained information, resulting in uncertainty in change detection on tiny parts such as edge pixels.

[0006] In summary, the general remote sensing image change detection methods have many limitations, mainly manifested in:

[0007] (1) The receptive field of convolutional neural networks is limited: Existing remote sensing image change detection methods usually use convolutional neural networks for feature extraction. However, the convolution operation has limitations in the receptive field, resulting in unclear recognition of small changes. At the same time, the image features at different levels extracted by the network are not well connected, resulting in weak feature expression capabilities, which in turn leads to low detection accuracy.

[0008] (2) The contribution of features at different levels to the discrimination of difference features is unbalanced: Many existing change detection networks can generate features at different levels, but during the downsampling process, they fail to reasonably consider the imbalanced contribution of features at different levels to the discrimination of difference features, resulting in the inability to completely and accurately detect the changed area.

[0009] (3) The network training parameters are large: In order to obtain better performance, many existing change detection networks continue to deepen the number of network layers, but ignore the cost of network training, which easily leads to overfitting in the training process. Summary of the invention

[0010] Purpose of the invention: In view of the problems existing in the prior art, the present invention provides a remote sensing image change detection method based on difference enhancement SwinTransformer. The difference enhancement module designed in this method incorporates an attention mechanism, which can effectively improve the difference feature expression of dual-phase remote sensing images; at the same time, the feature weight fusion module designed can effectively solve the problems of semantic loss and uneven contribution of features at different levels in remote sensing image change detection.

[0011] Technical solution: To achieve the purpose of the present invention, the technical solution adopted by the present invention is: a remote sensing image change detection method based on difference enhancement SwinTransformer, the specific steps are as follows:

[0012] (1) Construct a remote sensing image change detection dataset, and divide the dataset into a training set Train, a validation set Valid, and a test set Test;

[0013] (2) Construct a Swin Transformer network model DEST based on difference enhancement, which mainly consists of four parts: a backbone network based on the twin neural network architecture, a difference enhancement module (DEM), a Swin Transformer module (STM), and a feature weight fusion module (FWFM);

[0014] (3) The backbone network based on the twin neural network architecture in step (2) is mainly used to extract deep features at different levels of dual-temporal remote sensing images;

[0015] (4) The difference enhancement module (DEM) in step (2) is mainly used to enhance the difference feature learning between dual-temporal remote sensing images at different levels;

[0016] (5) The Swin Transformer module (STM) in step (2) is mainly used to model global context information, capturing more change details while expanding the network receptive field;

[0017] (6) The feature weight fusion module (FWFM) in step (2) is mainly used to effectively aggregate multi-level feature difference maps;

[0018] (7) Use the training set to train the DEST network, and use the validation set to evaluate the performance of the model during the training process to obtain the optimal model;

[0019] (8) The test set is input into the trained optimal model, and the unchanged areas and changed areas are classified and identified to obtain the final change detection results.

[0020] In step (1), the data set construction and division method is as follows:

[0021] (1.1) Building a remote sensing change detection image dataset (j=1,2), and make the corresponding label set Where N represents N pairs of dual-temporal remote sensing images. represents the i-th remote sensing image sample at the j-th phase;

[0022] (1.2) Divide the data set into a training set Train, a validation set Valid, and a test set Test according to a certain ratio. N1+N2+N3=N.

[0023] Among them, the Swin Transformer network model DEST based on difference enhancement in step (2) mainly includes four parts: a backbone network based on a twin neural network architecture, a difference enhancement module (DEM), a Swin Transformer module (STM), and a feature weight fusion module (FWFM).

[0024] Among them, in step (3), the method of constructing a backbone network based on the twin neural network architecture and performing deep feature extraction is as follows:

[0025] (3.1) Constructing the input layer, in which the dual-temporal remote sensing image is cut into several image blocks of size 256×256×3;

[0026] (3.2) Construct convolution layer 1, design a 3×3 convolution kernel in this layer, set the stride value to 1, set the padding mode to SAME, set the activation function to Relu, and input a feature map of 256×256×24;

[0027] (3.3) Construct convolution layer 2, design a 3×3 convolution kernel in this layer, set the stride value to 1, set the padding mode to SAME, and set the activation function to Relu; at the same time, set the pooling layer Maxpool with a size of 2 and a stride of 2, and output a feature map of 128×128×48;

[0028] (3.4) Construct convolution layer 3, design a 3×3 convolution kernel in this layer, set the stride value to 1, set the padding mode to SAME, and set the activation function to Relu; at the same time, set the pooling layer Maxpool with a size of 2 and a stride of 2, and output a feature map of 64×64×96;

[0029] (3.5) Construct convolution layer 4, design a 3×3 convolution kernel in this layer, set the stride value to 1, set the padding mode to SAME, and set the activation function to Relu; at the same time, set the pooling layer Maxpool with a size of 2 and a stride of 2, and output a feature map of 32×32×192;

[0030] (3.6) Construct convolution layer 5, design a 3×3 convolution kernel in this layer, set the stride value to 1, set the padding mode to SAME, and set the activation function to Relu; at the same time, set the pooling layer Maxpool with a size of 2 and a stride of 2, and output a feature map of 16×16×384.

[0031] In step (4), the method of constructing a difference enhancement module (DEM) and enhancing the difference feature learning between dual-temporal remote sensing images at different levels is as follows:

[0032] (4.1) The DEM module consists of two branches: the upper branch uses convolution operations to detect areas with significant changes; the lower branch uses pooling operations as an attention mechanism to further capture the difference information of small changes;

[0033] (4.2) Assuming X1 and X2 are the two inputs of the DEM module, after the upper branch processing, the upper branch difference map X dif_1 , and its calculation process is expressed as follows:

[0034]

[0035] Among them, conv 3×3 represents a 3×3 convolutional layer, represents pixel-by-pixel subtraction operation, ⊙ represents pixel-by-pixel multiplication operation, Indicates a cascade operation;

[0036] (4.3) After X1 and X2 are processed by the lower branch, the lower branch difference graph X can be obtained. dif_2 , and its calculation process is expressed as follows:

[0037]

[0038] Among them, GAP_w represents the global average pooling operation along the width direction, GAP_h represents the global average pooling operation along the height direction, and X w and X h is the corresponding pooling result;

[0039] (4.4) Substitute the X obtained in step (4.2) dif_1 and X obtained in step (4.3) dif_2 The fusion is performed to obtain the final difference map diff, and the process can be expressed as follows:

[0040]

[0041] Among them, conv 1×1 represents a 1×1 convolutional layer, Represents a pixel-by-pixel addition operation;

[0042] (4.5) The dual-phase feature maps output by steps (3.3)-(3.6) are respectively input into the corresponding difference enhancement modules (DEM), and four difference feature enhancement maps can be obtained, which are represented by diff1, diff2, diff3, and diff4 respectively.

[0043] In step (5), the method of using the Swin Transformer module (STM) to model the global context information is as follows:

[0044] (5.1) The difference feature enhancement graph diff4 obtained in step (4.5) is input into STM to obtain the difference feature enhancement graph diff4' after global context modeling.

[0045] Among them, in step (6), the method of designing a feature weight fusion module (FWFM) for fusing multi-level feature difference maps is as follows:

[0046] (6.1) The feature weight fusion module (FWFM) consists of two parts: the basic fusion module (BFM) and the interactive fusion module (IFM);

[0047] (6.2) The difference feature enhancement graphs diff1, diff2, diff3 obtained in step (4.5) and the difference feature enhancement graph diff4' after global context modeling obtained in step (5.1) are respectively input into the basic fusion module (BFM) and the interactive fusion module (IFM) to obtain the corresponding outputs output1 and output2 respectively;

[0048] (6.3) Output1 and output2 are fused to obtain the final output of FWFM. The specific process is as follows:

[0049]

[0050] Among them, conv 3×3 represents a 3×3 convolutional layer, Represents a pixel-by-pixel addition operation.

[0051] In step (7), the method of using the training set to train the DEST network is as follows:

[0052] (7.1) The cross entropy function is used as the first loss function of the entire network:

[0053] L bce =-ylog2(y′)-(1-y)log2(1-y′)

[0054] Among them, y and y′ represent the true label and the predicted label;

[0055] (7.2) In order to alleviate the impact of sample imbalance, the dice loss function is used as the second loss function of the entire network:

[0056]

[0057] Where N represents the number of pixels in the image, y i and i ′ represents the true label and predicted label of the i-th pixel;

[0058] (7.3) Combining the two loss functions, we can construct the overall loss function of the network:

[0059] L=L bce +L dice

[0060] In step (8), the test set is input into the trained optimal model, and the Sigmoid function is used to classify and identify the unchanged area and the changed area to obtain the change detection result.

[0061] Beneficial effects: The present invention adopts the above technical solution and has the following beneficial effects:

[0062] (1) In the extracted bi-temporal features of the same level, the method of the present invention adopts a difference enhancement module (DEM) fused with an attention mechanism to interact between bi-temporal features, which can effectively capture feature difference information; at the same time, due to the use of the attention mechanism, features of different levels can be weighted, so that the model can pay more attention to information related to the current task.

[0063] (2) The method of the present invention introduces an improved Swin Transformer module (STM) to perform context modeling of the image on the basis of reducing the amount of computation, thereby enhancing the ability to detect image changes and reducing the impact of irrelevant information on change judgment. Compared with the original Swin Transformer, STM can generate a feature image with the same size and the same number of channels as the input feature map, which reduces the parameters and computation of the network to a certain extent.

[0064] (3) The feature weight fusion module (FWFM) designed by the method of the present invention strengthens the information interaction between feature maps of different sizes during the upsampling process, which can solve the problem of uneven contribution of features at different levels to change detection, and prevent the network from losing information during downsampling and upsampling. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 It is the overall block diagram of the method of the present invention;

[0066] Figure 2 Schematic diagram of the designed difference enhancement module (DEM) structure;

[0067] Figure 3 Schematic diagram of the designed feature weight fusion module (FWFM) structure. DETAILED DESCRIPTION

[0068] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0069] The technical solution of the present invention is further described in detail as follows:

[0070] (1) Construct a remote sensing image change detection dataset, and divide the dataset into a training set Train, a validation set Valid, and a test set Test;

[0071] (1.1) Building a remote sensing change detection image dataset (j=1,2), and make the corresponding label set Where N represents N pairs of dual-temporal remote sensing images. Represents the i-th remote sensing image sample at the j-th time phase. The present invention uses the public dataset DSIFN-CD for remote sensing image change detection to conduct experiments. The dataset contains changes in various land cover objects, a total of 394 image pairs, and the size is 512×512 pixels. In order to avoid overfitting, the dataset is enhanced and pruned into non-overlapping blocks of 256×256 pixels;

[0072] (1.2) Divide the data set into a training set Train, a validation set Valid, and a test set Test according to a certain ratio. N1+N2+N3=N. In the present invention, the number of samples in the training set, validation set and test set are 3600, 340 and 48 respectively.

[0073] (2) Figure 1 As shown in the figure, a Swin Transformer network model DEST based on difference enhancement is constructed, which mainly includes four parts: a backbone network based on the twin neural network architecture, a difference enhancement module (DEM), a Swin Transformer module (STM), and a feature weight fusion module (FWFM).

[0074] (3) Build a backbone network based on the twin neural network architecture and perform deep feature extraction:

[0075] (3.1) Constructing the input layer, in which the dual-temporal remote sensing image is cut into several image blocks of size 256×256×3;

[0076] (3.2) Construct convolution layer 1, design a 3×3 convolution kernel in this layer, set the stride value to 1, set the padding mode to SAME, set the activation function to Relu, and input a feature map of 256×256×24;

[0077] (3.3) Construct convolution layer 2, design a 3×3 convolution kernel in this layer, set the stride value to 1, set the padding mode to SAME, and set the activation function to Relu; at the same time, set the pooling layer Maxpool with a size of 2 and a stride of 2, and output a feature map of 128×128×48;

[0078] (3.4) Construct convolution layer 3, design a 3×3 convolution kernel in this layer, set the stride value to 1, set the padding mode to SAME, and set the activation function to Relu; at the same time, set the pooling layer Maxpool with a size of 2 and a stride of 2, and output a feature map of 64×64×96;

[0079] (3.5) Construct convolution layer 4, design a 3×3 convolution kernel in this layer, set the stride value to 1, set the padding mode to SAME, and set the activation function to Relu; at the same time, set the pooling layer Maxpool with a size of 2 and a stride of 2, and output a feature map of 32×32×192;

[0080] (3.6) Construct convolution layer 5, design a 3×3 convolution kernel in this layer, set the stride value to 1, set the padding mode to SAME, and set the activation function to Relu; at the same time, set the pooling layer Maxpool with a size of 2 and a stride of 2, and output a feature map of 16×16×384.

[0081] (4) Figure 2 As shown in the figure, a difference enhancement module (DEM) is constructed, and the difference feature learning between dual-temporal remote sensing images is enhanced at different levels:

[0082] (4.1) The DEM module consists of two branches: the upper branch uses convolution operations to detect areas with significant changes; the lower branch uses pooling operations as an attention mechanism to further capture the difference information of small changes;

[0083] (4.2) Assuming X1 and X2 are the two inputs of the DEM module, after the upper branch processing, the upper branch difference map X dif_1 , and its calculation process is expressed as follows:

[0084]

[0085] Among them, conv 3×3 represents a 3×3 convolutional layer, represents pixel-by-pixel subtraction operation, ⊙ represents pixel-by-pixel multiplication operation, Indicates a cascade operation;

[0086] (4.3) After X1 and X2 are processed by the lower branch, the lower branch difference graph X can be obtained. dif_2 , and its calculation process is expressed as follows:

[0087]

[0088] Among them, GAP_w represents the global average pooling operation along the width direction, GAP_h represents the global average pooling operation along the height direction, and X w and X h is the corresponding pooling result;

[0089] (4.4) Substitute the X obtained in step (4.2) dif_1 and X obtained in step (4.3) dif_2 The fusion is performed to obtain the final difference map diff, and the process can be expressed as follows:

[0090]

[0091] Among them, conv 1×1 represents a 1×1 convolutional layer, Represents a pixel-by-pixel addition operation;

[0092] (4.5) The dual-phase feature maps output by steps (3.3)-(3.6) are respectively input into the corresponding difference enhancement modules (DEM), and four difference feature enhancement maps can be obtained, which are represented by diff1, diff2, diff3, and diff4 respectively.

[0093] (5) Use the Swin Transformer module (STM) to model global context information:

[0094] (5.1) The difference feature enhancement graph diff4 obtained in step (4.5) is input into STM to obtain the difference feature enhancement graph diff4' after global context modeling.

[0095] (6) Figure 3 As shown, a feature weight fusion module (FWFM) is designed to fuse multi-level feature difference maps:

[0096] (6.1) The feature weight fusion module (FWFM) consists of two parts: the basic fusion module (BFM) and the interactive fusion module (IFM);

[0097] (6.2) The difference feature enhancement graphs diff1, diff2, diff3 obtained in step (4.5) and the difference feature enhancement graph diff4' after global context modeling obtained in step (5.1) are respectively input into the basic fusion module (BFM) and the interactive fusion module (IFM) to obtain the corresponding outputs output1 and output2 respectively;

[0098] (6.3) Output1 and output2 are fused to obtain the final output of FWFM. The specific process is as follows:

[0099]

[0100] Among them, conv 3×3 represents a 3×3 convolutional layer, Represents a pixel-by-pixel addition operation.

[0101] (7) Use the training set to train the DEST network:

[0102] (7.1) The cross entropy function is used as the first loss function of the entire network:

[0103] L bce =-ylog2(y′)-(1-y)log2(1-y′)

[0104] Among them, y and y′ represent the true label and the predicted label;

[0105] (7.2) In order to alleviate the impact of sample imbalance, the dice loss function is used as the second loss function of the entire network:

[0106]

[0107] Where N represents the number of pixels in the image, y i and i ′ represents the true label and predicted label of the i-th pixel;

[0108] (7.3) Combining the two loss functions, we can construct the overall loss function of the network:

[0109] L=L bce +L dice

[0110] (8) The test set is input into the trained optimal model, and the Sigmoid function is used to classify and identify the unchanged areas and changed areas to obtain the change detection results.

[0111] The present invention uses Python to conduct experiments in the NVIDIA RTX 4090 graphics card and Pytorch framework. During the training process, DEST uses stochastic gradient descent as the optimization algorithm, with a momentum size of 0.9 and a weight decay of 5×10 -4 The initial learning rate is 1×10 -3 , and decayed by a cosine annealing strategy.

[0112] The present invention uses two different remote sensing image change detection methods to compare with the proposed method, and the selected comparison algorithms are:

[0113] A change detection method with dense connections of convolutional layers proposed by Fang et al. in “SNUNet-CD: A Densely Connected Siamese Network for Change Detection of VHR Images. IEEE Geoscience Remote Sensing Letter, 2022”, referred to as Method 1.

[0114] The method proposed by Chen et al. in "Remote Sensing Image Change Detection with Transformers. IEEE Geoscience Remote Sensing Letter, 2022", referred to as Method 2.

[0115] Table 1 is a performance comparison of the three methods on the high-resolution remote sensing scene image public dataset DSIFN-CD. Precision (P), Recall (R), and F1 indicators are the most commonly used remote sensing image change detection performance metrics. The larger its value, the better the performance. It can be seen that the method proposed in this invention has the best performance for remote sensing image change detection.

[0116] Table 1 Comparison of classification performance of three methods on DSIFN-CD dataset

[0117] method P(％) R(％) F1(％) Method 1 79.65 89.21 84.15 Method 2 84.23 75.83 79.80 Method of the present invention 85.72 83.54 84.62

Claims

1. A remote sensing image change detection method based on difference enhanced Swin Transformer, characterized by: The following steps are involved: (1) Construct a remote sensing image change detection dataset, and divide the dataset into a training set Train, a validation set Valid, and a test set Test; (2) Construct a Swin Transformer network model DEST based on difference enhancement, which mainly consists of four parts: a backbone network based on the twin neural network architecture, a difference enhancement module (DEM), a Swin Transformer module (STM), and a feature weight fusion module (FWFM); (3) The backbone network based on the twin neural network architecture in step (2) is mainly used to extract deep features at different levels of dual-temporal remote sensing images; (4) The difference enhancement module (DEM) in step (2) is mainly used to enhance the difference feature learning between dual-temporal remote sensing images at different levels; (5) The Swin Transformer module (STM) in step (2) is mainly used to model global context information, capturing more change details while expanding the network receptive field; (6) The feature weight fusion module (FWFM) in step (2) is mainly used to effectively aggregate multi-level feature difference maps; (7) Use the training set to train the DEST network, and use the validation set to evaluate the performance of the model during the training process to obtain the optimal model; (8) The test set is input into the trained optimal model, and the unchanged areas and changed areas are classified and identified to obtain the final change detection results.

2. According to the remote sensing image change detection method based on difference enhancement Swin Transformer of claim 1, in step (1), the data set construction and division method is as follows: (1.1) Building a remote sensing change detection image dataset (j=1,2), and make the corresponding label set in, N represents N pairs of dual-temporal remote sensing images. represents the i-th remote sensing image sample at the j-th phase; (1.2) Divide the data set into a training set Train, a validation set Valid, and a test set Test according to a certain ratio. N1+N2+N3=N.

3. According to the remote sensing image change detection method based on difference enhancement Swin Transformer of claim 1, in step (3), the method of constructing a backbone network based on a twin neural network architecture and performing deep feature extraction is as follows: (3.1) Constructing the input layer, in which the dual-temporal remote sensing image is cut into several image blocks of size 256×256×3; (3.2) Construct convolution layer 1, design a 3×3 convolution kernel in this layer, set the stride value to 1, set the padding mode to SAME, set the activation function to Relu, and input a feature map of 256×256×24; (3.3) Construct convolution layer 2, design a 3×3 convolution kernel in this layer, set the stride value to 1, set the padding mode to SAME, and set the activation function to Relu; at the same time, set the pooling layer Maxpool with a size of 2 and a stride of 2, and output a feature map of 128×128×48; (3.4) Construct convolution layer 3, design a 3×3 convolution kernel in this layer, set the stride value to 1, set the padding mode to SAME, and set the activation function to Relu; at the same time, set the pooling layer Maxpool with a size of 2 and a stride of 2, and output a feature map of 64×64×96; (3.5) Construct convolution layer 4, design a 3×3 convolution kernel in this layer, set the stride value to 1, set the padding mode to SAME, and set the activation function to Relu; at the same time, set the pooling layer Maxpool with a size of 2 and a stride of 2, and output a feature map of 32×32×192; (3.6) Construct convolution layer 5, design a 3×3 convolution kernel in this layer, set the stride value to 1, set the padding mode to SAME, and set the activation function to Relu; at the same time, set the pooling layer Maxpool with a size of 2 and a stride of 2, and output a feature map of 16×16×384.

4. According to the remote sensing image change detection method based on difference enhancement Swin Transformer of claim 1, in step (4), the method of constructing a difference enhancement module (DEM) and enhancing the difference feature learning between dual-temporal remote sensing images at different levels is as follows: (4.1) The DEM module consists of two branches: the upper branch uses convolution operations to detect areas with significant changes; the lower branch uses pooling operations as an attention mechanism to further capture the difference information of small changes; (4.2) Assuming X1 and X2 are the two inputs of the DEM module, after the upper branch processing, the upper branch difference map X dif_1 , and its calculation process is expressed as follows: in, conv 3×3 represents a 3×3 convolutional layer, represents pixel-by-pixel subtraction operation, ⊙ represents pixel-by-pixel multiplication operation, Indicates a cascade operation; (4.3) After X1 and X2 are processed by the lower branch, the lower branch difference graph X can be obtained. dif_2 , and its calculation process is expressed as follows: Among them, GAP_w represents the global average pooling operation along the width direction, GAP_h represents the global average pooling operation along the height direction, and X w and X h is the corresponding pooling result; (4.4) Substitute the X obtained in step (4.2) dif_1 and X obtained in step (4.3) dif_2 The fusion is performed to obtain the final difference map diff, and the process can be expressed as follows: Among them, conv 1×1 represents a 1×1 convolutional layer, Represents a pixel-by-pixel addition operation; (4.5) The dual-phase feature maps output by steps (3.3)-(3.6) are respectively input into the corresponding difference enhancement modules (DEM), and four difference feature enhancement maps can be obtained, which are represented by diff1, diff2, diff3, and diff4 respectively.

5. According to the remote sensing image change detection method based on difference enhanced Swin Transformer of claim 1, in step (5), the method of using Swin Transformer module (STM) to model the global context information is as follows: (5.1) The difference feature enhancement graph diff4 obtained in step (4.5) is input into STM to obtain the difference feature enhancement graph diff4' after global context modeling.

6. According to the remote sensing image change detection method based on difference enhancement Swin Transformer of claim 1, in step (6), a feature weight fusion module (FWFM) is designed to fuse the multi-level feature difference maps as follows: (6.1) The feature weight fusion module (FWFM) consists of two parts: the basic fusion module (BFM) and the interactive fusion module (IFM); (6.2) The difference feature enhancement graphs diff1, diff2, diff3 obtained in step (4.5) and the difference feature enhancement graph diff4' after global context modeling obtained in step (5.1) are respectively input into the basic fusion module (BFM) and the interactive fusion module (IFM) to obtain the corresponding outputs output1 and output2 respectively; (6.3) Output1 and output2 are fused to obtain the final output of FWFM. The specific process is as follows: in, conv 3×3 represents a 3×3 convolutional layer, Represents a pixel-by-pixel addition operation.

7. According to the remote sensing image change detection method based on difference enhancement Swin Transformer of claim 1, in step (7), the method of training the DEST network using the training set is as follows: (7.1) The cross entropy function is used as the first loss function of the entire network: L bce =-ylog2(y′)-(1-y)log2(1-y′) in, y and y′ represent the true label and predicted label; (7.2) In order to alleviate the impact of sample imbalance, the dice loss function is used as the second loss function of the entire network: Where N represents the number of pixels in the image, y i and i ′ represents the true label and predicted label of the i-th pixel; (7.3) Combining the two loss functions, we can construct the overall loss function of the network: L=L bce +L dice

Images