Remote sensing image change detection method based on deformable attention network

Abstract

The application discloses a remote sensing image change detection method based on a deformable attention network, designs a global context module based on a deformable attention mechanism, combines two double-time feature fusion modes of feature splicing and element subtraction, enhances the interaction of double-time features, enables the attention module to focus on the change area of interest, and fully learns the global context by using a sparse space sampling strategy and a remote relationship modeling capability; a difference enhancement module of the application guides the fusion of global features and low-level features through an attention mechanism, further generates a difference change graph containing rich change information, and enables the difference change graph to better complete detection. The application achieves good effects on a remote sensing image change detection dataset.

Description

Technical Field

[0001] The present invention relates to remote sensing image processing technology, and in particular to a remote sensing image change detection method based on a deformable attention network. Background Art

[0002] With the growing demand for military and civilian applications, the study of optical remote sensing imagery has become increasingly important. Remote sensing change detection uses multi-source remote sensing imagery covering the same surface area over different time periods to identify changes in that area. Its goal is to identify interesting changes while filtering out irrelevant interference. It is a key branch of remote sensing image analysis and has been widely applied in forest cover mapping, urban expansion, ecosystem monitoring, land use, disaster damage assessment (such as landslides and earthquakes), and environmental monitoring.

[0003] Due to limitations in imaging technology, early remote sensing images had low resolution, and each pixel often contained multiple distinct objects, such as trees, roads, and grass. In recent years, with the development of sensors, including satellite and aerial sensors, both temporal and spatial resolution have gradually increased, increasing the ability to acquire multi-temporal remote sensing imagery. The availability of high-resolution, multi-source, and multi-temporal remote sensing imagery has opened up new avenues for regularly monitoring ecological degradation and analyzing land cover changes.

[0004] Change detection based on high-resolution optical remote sensing imagery faces numerous challenges, such as scale variations, different imaging orientations, and cluttered backgrounds. High-resolution optical remote sensing images have distinct geometric structures, complex texture information, and limited spectral information, leading to spectral variations, loss of spatial information, and errors in the geometric registration of dual-temporal images. Due to seasonal variations and lighting conditions, objects with the same semantic meaning can exhibit different spectral characteristics at different times, causing spurious changes. Effectively identifying high-level semantic information related to changes of interest in a scene and distinguishing between real changes and complex, unrelated changes are currently key issues in change detection in high-resolution optical remote sensing imagery.

[0005] Existing change detection methods for remote sensing images can be divided into two categories: traditional change detection methods and deep learning-based change detection methods. Traditional change detection methods for remote sensing images lack generalization capabilities across scenes and are unable to model high-level contextual features. In recent years, the development of deep learning has significantly improved change detection performance. Most change detection methods are based on convolutional neural networks. While these methods improve change detection accuracy, they still fall short in modeling long-range dependencies due to the inherent limitations of convolution. Compared to their convolutional neural network counterparts, Transformer-based models have a larger receptive field and excel at modeling long-range dependencies, achieving superior results. However, simply expanding the receptive field can also lead to several issues. First, using dense attention can result in excessive memory and computational costs. Second, features can be affected by irrelevant portions outside the region of interest, making small objects invisible. Furthermore, unlike previous methods that use only a single concatenation or subtraction operation to fuse dual temporal features, leaving only one operation is less adaptable to capturing temporal differences and often fails to achieve good results. Summary of the Invention

[0006] Purpose of the invention: The purpose of the present invention is to address the deficiencies in the prior art and to provide a remote sensing image change detection method based on a deformable attention network. The present invention can well detect the change area of ​​interest without the need for a complex dual-temporal feature fusion method, and obtain a change map with better accuracy and higher confidence.

[0007] Technical solution: A remote sensing image change detection method based on a deformable attention network of the present invention comprises the following steps:

[0008] Step S1: Input a pair of dual-time series remote sensing images (images before and after the change), use two shared weight feature extraction networks to process the dual-time series features of the two dual-time series remote sensing images respectively, to obtain the multi-layer features of the two dual-time series remote sensing images, and then record the multi-layer features of the first dual-time series remote sensing image as The multi-layer features of the second dual-time series remote sensing image are recorded as

[0009] Step S2: For the high-level features in the multi-layer features obtained in step S1 and as well as and Use the encoder of the corresponding feature extraction network to perform feature fusion ( and Fusion, and Fusion), the two fused features are input into the global context module (the global context module uses deformable attention to enhance the change area information), and the global feature F is obtained. global ; The feature fusion methods here include feature concatenation and element subtraction in sequence;

[0010] Step S3: For the low-level features in the multi-level features obtained in step S1 and as well as and Use the encoder of the corresponding feature extraction network to perform feature fusion to obtain the fused low-level features ( and Fusion, and Fusion), the obtained fusion low-level features are combined with the global features F global The two are input together into the difference enhancement module, which uses the attention mechanism to guide the interaction of the fusion of low-level features and global features, outputs the enhanced detail features and obtains the new shallow features.

[0011] The feature fusion methods here include feature concatenation and element subtraction in sequence;

[0012] Step S4: decode the features of each layer enhanced in step S2 and step S3 through the decoder to obtain the final change map S final , and then train the network model with the binary cross entropy BCE loss function through the labeled true value.

[0013] In order to obtain more discriminative features, the feature extraction network in step S1 adopts the Swin-B backbone network in the Transformer with shared weights; the number of channels of the extracted multi-layer features are 128, 256, 512 and 1024 respectively.

[0014] In order to better model long-distance dependencies, mine the complementary information of features in different fusion methods and obtain global features, the global context module in step S2 models long-range dependencies in a sparse manner and obtains more comprehensive global context information, thereby strengthening the global understanding of the visual scene during the model learning process; the global context module enhances the high-level features in the multi-layer features (that is, the third-layer features and the fourth-layer features). ) is as follows:

[0015] First, the encoded high-level features and Channel splicing and fusion feature and Perform element-wise subtraction fusion to obtain High-level features and Channel splicing and fusion High-level features and Perform element-wise subtraction fusion to obtain And the fusion features and Input to the global context module, and Perform channel-level splicing and fusion to obtain feature F C , and then use the attention mechanism to the feature F C Selective recombination is performed to obtain the recombination feature X3. and Perform the same operation to obtain X4; at the same time, the fusion feature and After query mapping, queries C3 and C4 are generated respectively, and the fusion features and Generate queries S3 and S4 through query mapping;

[0016] Then, the sampling network is used to learn the query C3, C4, S3 and S4 to generate the offset of the change area of ​​interest, where the sampling network includes two depth-wise separable convolutions, one activation function and one convolution layer;

[0017] Then, the bilinear interpolation method is used to sample the recombined features X3 and X4 according to the offset to obtain the features of their important areas, and the deformed key is obtained by mapping. and corresponding values and keys and corresponding values After the deformation value key value Move to important areas;

[0018] At the same time, query C3 and the transformed key value Input the multi-head attention module together for feature capture, and combine the query S3 with the deformed key value The two captured features are fused together by element addition to obtain the third layer of enhanced features. At the same time, the query C4 is combined with the deformed key value Input the multi-head attention module together for feature capture, and combine the query S4 with the deformed key value The two captured features are input into the multi-head attention module for feature capture, and the two captured features are fused element by element to obtain the fourth layer of enhanced features;

[0019] Finally, the enhanced features of the third layer and the fourth layer are concatenated and then the convolution layer is used to reduce the dimension to obtain the global feature F. global .

[0020] The global context module of the present invention adopts interactive learning of two adjacent branch features to capture the correlation between features of different fusion methods to obtain a more comprehensive feature expression; it uses deformable attention sparse modeling to model long-range dependencies and mine the discriminative information of features in different fusion methods, which helps to focus on clearer structural information, effectively fuse features, and enhance the change area.

[0021] The deformable attention mechanism can enhance high-level features, but the low-level features have a large resolution. The calculation of dot product and bilinear interpolation in the deformable attention will increase the computational overhead. Therefore, step S3 uses the difference enhancement module to use spatial attention and channel attention to aggregate the global information of the global features and the detailed information of the low-level features to generate a difference map containing rich information. The specific method is as follows:

[0022] First, the encoded low-level features (also known as shallow features) and Channel splicing and fusion Low-level features and Element-wise subtraction fusion Low-level features and Channel splicing and fusion Layer Features and Element-wise subtraction fusion The features and Input to the difference enhancement module, which consists of three branches;

[0023] The first branch will feature and Perform convolution dimensionality reduction to obtain and Using parallel spatial attention and channel attention and Adaptively select important channels and spatial regions, perform weighted summation of channel attention and spatial attention, and obtain a mixed feature vector;

[0024] The second branch first converts the global feature F obtained in step S2 into global Upsample to a suitable resolution and then combine with the reduced dimension features and Add them together and then use spatial attention to filter key areas to obtain more discriminative features;

[0025] The third branch will feature and Convolution and spatial attention processing are performed in sequence. Through spatial attention, the information with more texture in the difference features is highlighted, and the feature information with less importance and more noise is suppressed, thereby obtaining a new feature map.

[0026] Finally, the three new feature maps obtained from the three branches are fused through element-level addition to extract the common area features and obtain new shallow features.

[0027] Furthermore, the decoder in step S4 processes the enhanced features to obtain the final change map S final The specific process is:

[0028] If 1≤i<4,

[0029] If i=4, then

[0030]

[0031] in, It is a feature enhanced by the global context module or the difference enhancement module. It is the decoder feature of the corresponding layer. Conv represents a layer of 3*3 convolution and ReLU activation function to achieve nonlinear activation. The i+1 layer feature is kept consistent with the i layer feature size through upsampling operation. σ represents the Sigmoid activation function.

[0032] Furthermore, the network model is trained using the binary cross entropy (BCE) loss function with the labeled true values.

[0033] Given the final change graph and the true value Y={Y n |n=1,...,T}, where n represents the nth pixel and T is the final change map S final The total number of pixels;

[0034] The binary cross entropy BCE loss function is:

[0035]

[0036] Beneficial effects: Based on the pseudo-change problem in change detection, the present invention designs a global context module. Based on deformable attention, it combines sparse spatial sampling strategies and long-range relationship modeling capabilities to fully learn the complementary information of features in different fusion methods, thereby enhancing global context information. This module combines two dual-time series feature fusion methods, feature splicing and element subtraction, to jointly learn the offsets of keys and values, enabling the attention module to focus on the change areas of interest. In addition, the difference enhancement module guides the fusion of global features and low-level features through the attention mechanism to obtain a more discriminative feature representation, and can generate a difference change map containing rich change information. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a schematic diagram of the overall process of the present invention;

[0038] Figure 2 Schematic diagram of the global context module and difference enhancement module structure of the present invention;

[0039] Figure 3 Schematic diagram of visual comparison in the embodiment. DETAILED DESCRIPTION

[0040] The technical solution of the present invention is described in detail below, but the protection scope of the present invention is not limited to the embodiments.

[0041] The Transformer-based network model in the prior art usually uses dense attention, which will lead to excessive computational costs. The features will be affected by irrelevant parts beyond the area of ​​interest and will be invisible to small objects. In addition, previous methods only use a single splicing or subtraction operation to perform dual-time series feature fusion, which has poor adaptability. Based on these two problems, the present invention designs a global context module and a difference feature module. The global context module is based on deformable attention and can adaptively adjust the receptive field, better model long-distance dependencies, pay attention to the spatial position of effective information in the feature map, combine the two fusion methods, mine the complementary information of features of different fusion methods, and obtain global features. The difference enhancement module uses three branches and uses the attention module to guide the shallow features and global features of the two fusion methods to fuse and generate a feature map with rich details.

[0042] like Figure 1 As shown, the remote sensing image change detection method based on deformable attention network of the present invention includes the following steps:

[0043] Step S1: Input a pair of dual-time series remote sensing images, use two shared weight feature extraction networks to process the dual-time series features of the two dual-time series remote sensing images respectively, to obtain the multi-layer features of the two dual-time series remote sensing images, and then record the multi-layer features of the first dual-time series remote sensing image as The multi-layer features of the second dual-time series remote sensing image are recorded as

[0044] Step S2: For the high-level features in the multi-layer features obtained in step S1 and as well as and Use the encoder of the corresponding feature extraction network to perform feature fusion, input the two fused features into the global context module, and obtain the global feature F global ; The feature fusion methods here include feature concatenation and element subtraction in sequence;

[0045] Step S3: For the low-level features in the multi-level features obtained in step S1 and as well as and Use the encoder of the corresponding feature extraction network to perform feature fusion to obtain fused low-level features, and combine the obtained fused low-level features with the global features F global The two are input together into the difference enhancement module, which uses the attention mechanism to guide the interaction of low-level features and global features to obtain new shallow features. The feature fusion methods here include feature concatenation and element subtraction.

[0046] Step S4: decode the features of each layer enhanced in steps S2 and S3 through the decoder to obtain the final change map S final , and then train the network model with the binary cross entropy BCE loss function through the labeled true value.

[0047] like Figure 1 and Figure 2 As shown, the overall network model of this embodiment consists of the following components: an encoder, a global context module, a difference enhancement module, and a decoder. The encoder extracts multi-layer features from the dual temporal images. The global context module extracts contextual information from features of different fusion methods, enhances the feature representation of the changed regions, and generates high-quality global features. The difference enhancement module fuses global features with low-level features to add detail to the change map. Finally, the decoder predicts the final segmentation map.

[0048] In step S2, the global context module enhances the high-level features in the multi-layer features. The specific method is as follows:

[0049] First, the encoded high-level features and Channel splicing and fusion feature and Perform element-wise subtraction fusion to obtain High-level features and Channel splicing and fusion High-level features and Perform element-wise subtraction fusion to obtain And the fusion features and Input to the global context module, and Perform channel-level splicing and fusion to obtain feature F C , and then use the attention mechanism to the feature F C Selective recombination is performed to obtain the recombination feature X3. and Perform the same operation to obtain X4; at the same time, the fusion feature and After query mapping, queries C3 and C4 are generated respectively, and the fusion features and The query mapping generates queries S3 and S4. Then, the sampling network is used to learn queries C3, C4, S3 and S4 to generate the offset of the change area of ​​interest. The sampling network includes two depth-wise separable convolutions, one activation function and one convolution layer. Then, the bilinear interpolation method is used to sample the reconstructed features X3 and X4 according to the offset to obtain the features of their important areas. The deformed key is obtained by mapping. and corresponding values and keys and corresponding values After the deformation value key value Move to the important area; at the same time, query C3 and the deformed key value Input the multi-head attention module together for feature capture, and combine the query S3 with the deformed key value The two captured features are fused together by element addition to obtain the third layer of enhanced features. At the same time, the query C4 is combined with the deformed key value Input the multi-head attention module together for feature capture, and combine the query S4 with the deformed key value The two captured features are input into the multi-head attention module for feature capture, and the elements of the two captured features are fused by element addition to obtain the fourth layer enhanced features; finally, the third layer enhanced features and the fourth layer enhanced features are first channel-wise spliced ​​and then the convolution layer is used for dimensionality reduction to obtain the global feature F global .

[0050] In step S3 above, the difference enhancement module uses spatial attention and channel attention to aggregate the global information of global features and the detailed information of low-level features to generate a difference map containing rich information. The specific method is as follows:

[0051] First, the encoded low-level features and Channel splicing and fusion Low-level features and Element-wise subtraction fusion Low-level features and Channel splicing and fusion Layer Features and Element-wise subtraction fusion The features and Input to the difference enhancement module, which consists of three branches;

[0052] The first branch will feature and Perform convolution dimensionality reduction to obtain and Using parallel spatial attention and channel attention and Adaptively select important channels and spatial regions, perform weighted summation of channel attention and spatial attention, and obtain hybrid features;

[0053] The second branch first converts the global feature F obtained in step S2 into global Upsample to a suitable resolution and then combine with the reduced dimension features and Add them together and then use spatial attention to filter key areas to obtain more discriminative features;

[0054] The third branch will feature and Convolution and spatial attention processing are performed in sequence to obtain new features;

[0055] Finally, the new features obtained from the three branches are fused through element-level addition to obtain new shallow features.

[0056] In this embodiment, for the multi-layer feature of the first dual-time series remote sensing image, The second multi-layer feature of dual-time series remote sensing images The features after channel concatenation and element subtraction fusion in step S2 and step 3 are and i represents the i-th layer, and the values ​​of i are 1, 2, 3 and 4 respectively.

[0057] Among them, the high-level feature and Based on deformable attention combined with channel splicing and element subtraction to enhance the interaction between dual temporal features and capture deep temporal difference information, the network can more accurately locate the changing area of ​​interest. and Perform feature concatenation and element subtraction to obtain the corresponding fusion feature, then concatenate the fusion feature channels, and use the channel attention mechanism for selective reorganization to obtain the reorganized feature X i ∈R H ×W×C , since high-level features (third-layer features and fourth-layer features) are processed here, i=3,4.

[0058]

[0059]

[0060]

[0061] Among them, Conv represents a layer of 3*3 convolution; Concat represents splicing; CA represents the channel attention mechanism; |*| represents the absolute value operation.

[0062] like Figure 2 As shown, the fusion features and After query mapping, query C is generated and the fusion features are and After query mapping, query S is generated. In this embodiment, query C and query S are denoted as q ci and q di (i=3,4).

[0063] For the recombinant feature X i ∈R H×W×C , generate a unified reference point grid p∈R for it HG×WG×2 , the grid size is obtained by sampling the input feature map size with a sampling factor r, that is, HG = H / r, WG = W / r. To obtain the offset of each reference point, the offset network is obtained from two query features q ci and q di The offset network consists of two branches, using two depth-wise separable convolutions to learn two queries respectively. The outputs are added together and then passed through the activation function and convolution layer to obtain the output offset.

[0064]

[0065] p 1i=DWConv(q ci ),p 2i =DWConv(q di ), i=3,4

[0066] Δp i =Conv(GELU(p 1i +p 2i )), i=3,4

[0067] Among them, DWConv represents depth-wise separable convolution, W * is the mapping matrix; GELU represents the activation function

[0068] After obtaining the above offset, the bilinear interpolation method is used to sample the recombined features X3 and X4 according to the offset to obtain the features of their important areas, and the deformed key is obtained by mapping Sum

[0069]

[0070]

[0071]

[0072]

[0073]

[0074] Where W * is the mapping matrix; φ uses bilinear interpolation; Attention represents multi-head attention.

[0075] In this embodiment, for the multi-layer feature of the first dual-time series remote sensing image, The second multi-layer feature of dual-time series remote sensing images The features after channel concatenation and element subtraction fusion in step S2 and step 3 are and i represents the i-th layer, and the values ​​of i are 1, 2, 3 and 4 respectively.

[0076] Among them, the low-level characteristics and Obtained by feature concatenation and element subtraction fusion and Since the processing object at this time is the low-level features (first-level features and second-level features), i = 1, 2, thus we can get:

[0077]

[0078]

[0079]

[0080]

[0081]

[0082]

[0083] SA stands for spatial attention; CA stands for channel attention.

[0084] The decoder in step S4 above processes the enhanced features and obtains the final change map S final The specific process is:

[0085] If 1≤i<4,

[0086] If i=4, then

[0087]

[0088] in, It is a feature enhanced by the global context module or the difference enhancement module. It is the decoder feature of the corresponding layer. Conv represents a layer of 3*3 convolution and ReLU activation function to achieve nonlinear activation. The i+1 layer feature is kept consistent with the i-th layer feature size through upsampling operation. σ represents the Sigmoid activation function.

[0089] The network model is trained using the binary cross entropy BCE loss function through the labeled true value;

[0090] Given the final change graph and the true value Y={Y n |n=1,...,T}, where n represents the nth pixel and T is the final change map S final The total number of pixels;

[0091] The binary cross entropy BCE loss function is:

[0092]

[0093] Example:

[0094] This example uses publicly available remote sensing image change detection datasets: DSIFN-CD, LEVIR-CD, and WHU-CD. The LEVIR-CD dataset, proposed in 2020, collects images from 20 different cities and regions in Texas, USA, covering most human habitats. It contains 637 pairs of 1024×1024 pixel high-resolution images. The data is divided into 256×256 pixel blocks without overlap, resulting in 7120 / 1024 / 2048 image pairs for training / validation / testing, respectively. The WHU-CD dataset records building changes following the 2011 earthquake in New Zealand. It consists of a pair of 32,507×15,354 pixel high-resolution aerial images. The data is divided into 256×256 pixel blocks without overlap, and randomly divided into 5,949 training sets, 742 validation sets, and 743 test sets. The DSIFN-CD dataset contains 14,400 training samples, 1,360 validation samples, and 192 test samples.

[0095] This example uses Pytorch to implement the network and trains it on two Titan XP GPUs. For input, all image pairs are resized to 384×384. During training, this example uses Adaptive Moment Estimation (Adam) to optimize the network with a batch size of 8 and trains for 100 rounds. The initial learning rate of the network is set to 10 -4 , multiplied by 0.2 every 25 rounds.

[0096] To facilitate quantitative evaluation, this embodiment adopts five widely used indicators.

[0097] (1) Precision is used to measure the proportion of correctly found positive samples, reflecting the ability of the algorithm to correctly predict positive samples. It is defined as:

[0098] Precision = TP / (TP+FP)

[0099] (2) Recall is used to measure the ratio of the number of positive samples found among all positive samples, reflecting the ability of the algorithm to correctly predict the full range of positive samples. It is defined as:

[0100] Recall = TP / (TP+FN)

[0101] (3) F1-score is the weighted harmonic mean of precision and recall. The formula for F1-score is:

[0102]

[0103] (4) IoU, defined as:

[0104] IOU=TP / (TP+FP+FN)

[0105] (5) OA overall accuracy is used to indicate the proportion of correct samples to the total samples. The overall accuracy can be used to show the overall accuracy of the samples. It is defined as:

[0106] OA=(TP+TN) / (TP+TN+FP+FN)

[0107] Example:

[0108] Compare the technical solution of the present invention with other existing technologies.

[0109] This embodiment compares the network of the technical solution of the present invention with nine other existing technical solutions, namely FC-EF, FC-Siam-Di, FC-Siam-Conc, DTCDSCN, STANct, IFNet, SNUNet, BIT, and ChangeFormer.

[0110] Quantitative comparison:

[0111] Table 1 shows the specific comparative test results of this embodiment. This embodiment uses F1-score, Precision, Recall, Intersection over Union (IoU), and OA to evaluate the corresponding change maps on three datasets. Across all evaluation metrics, the present invention achieves optimal performance. Compared to the suboptimal method, the performance improvements on the three datasets are 5.7, 4.6, 6.8, 9.6, and 1.2 percentage points, respectively, in F1-score, Precision, Recall, IoU, and OA.

[0112] Table 1 Schematic diagram of comparison of prediction indicators

[0113]

[0114] Qualitative evaluation:

[0115] like Figure 3 As shown, this embodiment is compared with the final change graph of other technical solutions. This implementation selected 8 pairs of challenging samples. It can be found from Figure 3 that the technical solution of the present invention is more accurate in segmenting the changed area. In the challenge of small targets, most methods will lose part of the object, but the present invention will not have this problem (columns 5 and 6). In the challenge of cluttered scenes, most methods are more susceptible to background noise, but the technical solution of the present invention can avoid this type of problem very well (columns 7 and 8).

[0116] As can be seen from the above examples, the present invention can produce more refined change prediction results. Furthermore, the features integrated by the present invention's global context information module contain richer global context information. The present invention's difference enhancement module fuses global and low-level features to further highlight target change areas and suppress the influence of redundant background noise. As a result, the final change map produced by the present invention is more accurate and has a higher confidence level.

Claims

1. A remote sensing image change detection method based on deformable attention network, characterized in that: The following steps are involved: Step S1: Input a pair of dual-time series remote sensing images, use two shared weight feature extraction networks to process the dual-time series features of the two dual-time series remote sensing images respectively, to obtain the multi-layer features of the two dual-time series remote sensing images, and then record the multi-layer features of the first dual-time series remote sensing image as The multi-layer features of the second dual-time series remote sensing image are recorded as Step S2: For the high-level features in the multi-layer features obtained in step S1 and as well as and Use the encoder of the corresponding feature extraction network to perform feature fusion, input the two fused features into the global context module, and obtain the global feature F global The feature fusion methods here include feature concatenation and element subtraction. The specific method of using the global context module to enhance the high-level features in the multi-layer features is as follows: First, the encoded high-level features and Channel splicing and fusion feature and Perform element-wise subtraction fusion to obtain High-level features and Channel splicing and fusion High-level features and Perform element-wise subtraction fusion to obtain And the fusion features and Input to the global context module, and Perform channel-level splicing and fusion to obtain feature F C , and then use the attention mechanism to the feature F C Selective recombination is performed to obtain the recombination feature X3. and Perform the same operation to obtain X4; at the same time, the fusion feature and After query mapping, queries C3 and C4 are generated respectively, and the fusion features and Generate queries S3 and S4 through query mapping; Then, the sampling network is used to learn the query C3, C4, S3 and S4 to generate the offset of the change area of ​​interest, where the sampling network includes two depth-wise separable convolutions, one activation function and one convolution layer; Then, the bilinear interpolation method is used to sample the recombined features X3 and X4 according to the offset to obtain the features of their important areas, and the deformed key is obtained by mapping. and corresponding values and keys and corresponding values After the deformation value key value Move to important areas; At the same time, query C3 and the transformed key value Input the multi-head attention module together for feature capture, and combine the query S3 with the deformed key value The two captured features are fused together by element addition to obtain the third layer of enhanced features. At the same time, the query C4 is combined with the deformed key value Input them into the multi-head attention module for feature capture, and combine the query S4 with the deformed value The two captured features are input into the multi-head attention module for feature capture, and the two captured features are fused element by element to obtain the fourth layer of enhanced features; Finally, the enhanced features of the third layer and the fourth layer are concatenated and then the convolution layer is used to reduce the dimension to obtain the global feature F. global ; Step S3: For the low-level features in the multi-level features obtained in step S1 and as well as and Use the encoder of the corresponding feature extraction network to perform feature fusion to obtain fused low-level features, and combine the obtained fused low-level features with the global features F global The two are input together into the difference enhancement module, which uses the attention mechanism to guide the interaction of low-level features and global features to obtain new shallow features. The feature fusion methods here include feature concatenation and element subtraction. Step S4: decode the features of each layer enhanced in steps S2 and S3 through the decoder to obtain the final change map S final , and then train the network model with the binary cross entropy BCE loss function through the labeled true value.

2. The remote sensing image change detection method based on deformable attention network according to claim 1 is characterized in that The feature extraction network in step S1 adopts the Swin-B backbone network in the Transformer with shared weights; the number of channels of the extracted multi-layer features are 128, 256, 512 and 1024 respectively.

3. The remote sensing image change detection method based on deformable attention network according to claim 1, characterized in that The step S3 uses the difference enhancement module to use spatial attention and channel attention to aggregate the global information of the global features and the detailed information of the low-level features to generate a difference map containing rich information. The specific method is: First, the encoded low-level features and Channel splicing and fusion Low-level features and Element-wise subtraction fusion Low-level features and Channel splicing and fusion Layer Features and Element-wise subtraction fusion The features and Input to the difference enhancement module, which consists of three branches; The first branch will feature and Perform convolution dimensionality reduction to obtain and Using parallel spatial attention and channel attention and Adaptively select important channels and spatial regions, perform weighted summation of channel attention and spatial attention, and obtain hybrid features; The second branch first converts the global feature F obtained in step S2 into global Upsample to a suitable resolution and then combine with the reduced dimension features and Add them together and then use spatial attention to filter key areas to obtain more discriminative features; The third branch will feature and Convolution and spatial attention processing are performed in sequence to obtain new features; Finally, the new features obtained from the three branches are fused through element-level addition to obtain new shallow features.

4. The remote sensing image change detection method based on deformable attention network according to claim 1 is characterized in that The decoder in step S4 processes the enhanced features to obtain the final change map S final The specific process is: If 1≤i<4, If i=4, then in, It is a feature enhanced by the global context module or the difference enhancement module. It is the decoder feature of the corresponding layer. Conv represents a layer of 3*3 convolution and ReLU activation function to achieve nonlinear activation. The i+1 layer feature is kept consistent with the i layer feature size through upsampling operation. σ represents the Sigmoid activation function.

5. The remote sensing image change detection method based on deformable attention network according to claim 1 is characterized in that The network model is trained using the binary cross entropy BCE loss function through the labeled true value; Given the final change graph and the true value Y={Y n |n=1,...,T}, where n represents the nth pixel and T is the final change map S final The total number of pixels; The binary cross entropy BCE loss function is:

Images