Remote sensing image change detection method based on progressive attention difference feature enhancement network

Abstract

The application discloses a remote sensing image change detection method based on a progressive attention-based difference feature enhancement network. Previous methods mainly use attention mechanisms and dense convolution iterations for fine-grained matching. However, these methods do not fully utilize the rich semantic information of high-resolution images, leading to deviations in overall semantic understanding. In addition, we noticed that since deep learning models are based on local receptive fields for prediction, boundary positioning is prone to inaccuracy. The application first proposes a progressive attention-based difference feature enhancement network to study the relationship between global information and difference information. A twin network is designed to enhance the difference features of global features, which uses fusion information and regional correlation characteristics to mine more complete visual semantics and avoid understanding deviations. A progressive attention module is proposed to restore valuable but often overlooked boundary information of the image. The application achieves effective results through a large number of experiments on LEVIR-CD.

Description

Technical Field

[0001] This invention pertains to methods for change detection and relates to two technical fields: computer vision and remote sensing. Background Technology

[0002] In the field of remote sensing, change detection (CD) is one of the main research topics. Change detection in remote sensing images is a technique that processes multi-temporal remote sensing image data to obtain information on the spatial and temporal distribution changes of ground features in the same area at different times. With the development of deep learning, change detection in remote sensing images has made great progress. Modern change detection (CD) uses deep convolutions with strong discriminative power and has achieved success in practice. Nevertheless, change detection in high-resolution remote sensing images remains challenging. Due to the complexity and diversity of objects in a scene, even objects with the same semantic concept may exhibit different spectral characteristics at different times and spatial locations, which also affects the accuracy of change detection.

[0003] A common approach in existing models is to extract features from biphasic images using Siamese networks and then merge them into a single deep learning network. Differential features are then generated based on these processed biphasic image features for classification. Before deep learning, direct classification-based methods made significant progress in colorimetric analysis (CD), such as change vector analysis (VVA), which is powerful in extracting pixel-level features. With the rapid increase in image resolution, remote sensing images record more object details, and many works have improved CD performance by extracting more discriminative features. However, these methods still have some problems when analyzing change regions. The FCN model uses the U-net model to detect new building regions. However, the FCN model performs poorly because it ignores spatial information and changes in objects at different scales. SNUNet focuses on handling multi-scale features to handle scale changes of objects through ECAM (Integrated Channel Attention Module). However, this ECAM only considers channel attention, ignoring spatial relationships between pixels, resulting in the detection of some unexpected regions with seasonal vegetation changes in the generated change maps.

[0004] Contextual modeling across spatial and temporal dimensions is crucial for identifying changes in interest in high-resolution remote sensing images. Therefore, research focuses on increasing the receptive field (RF) of models by adding more convolutional layers, using dilated convolutions, and applying attention mechanisms. This helps models acquire more contextual information and improves model accuracy. Unlike purely convolution-based methods, which rely on a limited receptive field size, attention-based methods (such as channel attention, spatial attention, and self-attention) are more effective at modeling global information. They can analyze and process images globally to better capture spatial and semantic correlations, improving model performance and accuracy. Most existing methods still struggle to link distant concepts in space and time. This is because they either apply attention individually to each temporal image to enhance its features or simply use attention to reweight and fuse bitemporal features in the channel or spatial dimensions, failing to capture deeper spatiotemporal correlations. Some studies have achieved good results by utilizing self-attention mechanisms to simulate semantic relationships between arbitrary pixels in space and time. However, these methods are computationally inefficient and require high computational complexity, as the computational complexity quadratically increases with the number of pixels.

[0005] Furthermore, Guo et al. used a fully convolutional Siamese network with contrast loss to measure regions of change. Zhang et al. proposed a deep supervised image fusion network for CDs. Some studies also focus on object-specific CDs. For example, Liu et al. proposed a deep Siamese convolutional network with dual task constraints to construct CDs. Jiang et al. proposed an attention-guided Siamese network based on pyramid features to construct CDs. Lei et al. proposed a hierarchical paired channel fusion network for street view CDs. These methods have achieved great success in CD feature learning. However, these methods have limited global representation capabilities, typically focusing only on local regions of the changing object. We found that transformers are strong in extracting global features. Therefore, unlike previous work, we leverage the advantages of transformers to explore the relationship between global and differential information, proposing a new learning framework that fully utilizes the semantic information of high-resolution images to achieve more discriminative feature representations. Summary of the Invention

[0006] The purpose of this invention is to address the problem that current remote sensing image change detection methods based on stacked convolution rarely consider the relationship between global and difference information in biphasic images, neglecting the importance of feature interactions. Furthermore, as remote sensing image resolution increases, the rich semantic information contained in high-resolution images is not fully utilized.

[0007] The technical solution adopted by the present invention to solve the above-mentioned technical problems is as follows:

[0008] S1. Construct a progressive attention module to obtain global features and differential features based on the features of the input biphasic image;

[0009] S2. Utilize semantic extractors, transformer encoders, and decoders to extract deeper contextual features;

[0010] S3. Construct a feature fusion module to guide and enhance differential features based on global features;

[0011] S4. Combine the networks in S1 and S3 to construct a differential feature enhancement network architecture based on progressive attention;

[0012] The progressive attention module mines more complete visual features based on the correlation and differences between biphasic images, enhances global contextual information by utilizing channel-level attention, and improves learning ability through residuals, thereby increasing the utilization rate of boundary information. We describe the detailed operation below:

[0013] Given biphasic image features F t1 , Given C=32, H=64, and W=64, generate F through the following three steps. c' :

[0014] F c' =ReLU(BN(Conv([F t1 ,F t2 ]))) (1)

[0015] Conv is a 3x3 convolution kernel that reduces the dimensionality of the concatenated features.

[0016] BN() refers to normalization, and ReLU() is the activation function.

[0017] By F c' The pooled feature F is obtained p1 :

[0018] F p1 =GAP(F c' )+GMP(F c' (2)

[0019] GAP() is the global average pooling function, and GMP() is the global max pooling function.

[0020] Another F p Using a 1x1 convolution kernel and activation function, and then with F... c' After multiplying and performing residual connection, the output F of the asymptotic attention module is obtained. c :

[0021] F c =F c' *σ(Conv(F p1 )+F c' (3)

[0022] Where σ() is the Sigmoid function.

[0023] The following describes feature subtraction of a biphasic image and processing it using a progressive attention module to obtain F. d The steps are the same as above:

[0024] F d' =ReLU(BN(Conv(F) t1 -F t2 (4)

[0025] F p2 =GAP(F d' )+GMP(F d' (5)

[0026] F d =F d' *σ(Conv(F p2 )+F d' (6)

[0027] This method transforms the two inputs into summation and difference features of a biphasic image, recovering valuable but often overlooked boundary information of the image; it also improves interdependence and enhances feature representation capabilities through a channel attention mechanism.

[0028] The deep feature extraction module leverages the superior long-range feature extraction capabilities of the semantic extractor, transformer encoder, and decoder to obtain deeper contextual features, making full use of the rich semantic information of high-resolution images.

[0029] The summation feature F obtained by the asymptotic attention module c Features F of the difference feature image d The input is fed into the deep feature extraction module, first passing through the semantic generator; simultaneously, F c and F d It is also fed into the cross-attention of the transformer decoder to obtain features with more global context, and the output of the semantic generator is represented as T. c and T d It is given by the following formula:

[0030] T c =(A c ) T F c (7)

[0031] T d =(A d ) T F d (8)

[0032] Where A c and A d They are F c F d Obtained through a 3x3 convolution, (A c ) T and (A) d ) T This represents the transpose of a matrix.

[0033] T is obtained through pointwise convolution. c With T d Global tags and difference tags, T c , C = 32, L = H * W.

[0034] Then generate T c and T d As input to the transformer encoder, it generates labels with more global context and boundary information, T ce and T de :

[0035] [T ce ,I de ] = TransformerEncoder(T c ,I d (9)

[0036] F c F d T ce and I de The input is fed into the transformer decoder to obtain the deep global features F. cd Deep difference feature F dd :

[0037] F cd =TransformerDecoder(F c ,T ce (10)

[0038] F dd =TransformerDecoder(F d ,T de (11)

[0039] Where F cd , C = 32, H = 64, W = 64.

[0040] The differential feature enhancement module interacts with deep global features and deep differential features, integrating different cues from dual-temporal image features. This is achieved by using global image features F... o Modulate the differential image features F in a way that relies on prior knowledge. p The fusion differential features F of fine-grained local changes in the enhanced dual-temporal images were obtained. x :

[0041] F o =Conv(ReLU(Conv(F) cd (12)

[0042] F p =Conv(ReLU(Conv(F) dd (13)

[0043] F x =F p *(F o +1)+F p (14)

[0044] Where F o and F p Enhanced differential features F are obtained by using spatial attention through two 1x1 convolutions. x Finally, the classification results are obtained.

[0045] The remote sensing image transformation detection method based on a differential feature enhancement network with progressive attention includes a progressive attention module, a deep feature extraction module, and a differential feature enhancement module.

[0046] Compared with existing technologies, the beneficial effects of this invention are:

[0047] 1. This invention proposes a novel differential feature enhancement network based on progressive attention for remote sensing image classification (CD). It employs global and differential features from dual-temporal images for guidance and interaction to achieve comprehensive semantic understanding and fusion, thereby improving the expressive power of differential images and increasing classification accuracy.

[0048] 2. This invention proposes a progressive attention module to mine richer visual features and designs channel attention to enhance the global and differential information of images. Both are used to improve the overall semantics of images and improve the accuracy of image classification. Attached Figure Description

[0049] Figure 1 This is a schematic diagram of the structure of a remote sensing image change detection method based on a differential feature enhancement network with progressive attention.

[0050] Figure 2 This is a schematic diagram of the progressive attention module.

[0051] Figure 3 This is a schematic diagram of a semantic extractor.

[0052] Figure 4 This is a schematic diagram of a transformer encoder and decoder model with global context information.

[0053] Figure 5 This is a schematic diagram of the differential feature enhancement module.

[0054] Figure 6 This figure compares the results of the change detection model based on the differential feature enhancement network of progressive attention with other change detection models on the dataset.

[0055] Figure 7 This is a visualization of the change detection results. Detailed Implementation

[0056] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this patent.

[0057] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0058] Figure 1 This is a schematic diagram of the structure of a differential feature enhancement network based on progressive attention. (Example) Figure 1 As shown, the entire change detection framework mainly consists of three parts: a progressive attention module, a deep feature extraction module, and a differential feature enhancement module.

[0059] Figure 2 This is a schematic diagram of a progressive attention module. (Example) Figure 2 As shown, the progressive attention module mines more complete visual features based on the correlation and differences between biphasic images, enhances global contextual information by utilizing channel-level attention, and improves learning ability through residuals, thereby increasing the utilization rate of boundary information. We describe the detailed operation below:

[0060] Given biphasic image features F t1 , Given C=32, H=64, and W=64, generate F through the following three steps. c' :

[0061] Fc' =ReLU(BN(Conv([F t1 ,F t2 ]))) (1)

[0062] Conv is a 3x3 convolution kernel that reduces the dimensionality of the concatenated features.

[0063] BN() refers to normalization, and ReLU() is the activation function.

[0064] By F c' The pooled feature F is obtained p1 :

[0065] F p1 =GAP(F c' )+GMP(F c' (2)

[0066] GAP() is the global average pooling function, and GMP() is the global max pooling function.

[0067] Another F p Using a 1x1 convolution kernel and activation function, and then with F... c' After multiplying and performing residual connection, the output F of the asymptotic attention module is obtained. c :

[0068] F c =F c' *σ(Conv(F p1 )+F c' (3)

[0069] Where σ() is the Sigmoid function.

[0070] The following is the F obtained by processing the subtraction features of biphasic images through a progressive attention module. d The steps are the same as above:

[0071] F d' =ReLU(BN(Conv(F) t1 -F t2 (4)

[0072] F p2 =GAP(F d' )+GMP(F d' (5)

[0073] F d =F d' *σ(Conv(F p2 )+F d' (6)

[0074] This method transforms the two inputs into summation and difference features of a biphasic image, recovering valuable but often overlooked boundary information of the image; it also improves interdependence and enhances feature representation capabilities through a channel attention mechanism.

[0075] Figure 3 and Figure 4 This is a schematic diagram of a deep feature extraction model. The deep feature extraction module leverages the superior long-range feature extraction capabilities of the semantic extractor, transformer encoder, and decoder to obtain deeper contextual features, fully utilizing the rich semantic information of high-resolution images.

[0076] like Figure 3 As shown, the summation feature F obtained by the asymptotic attention module c Features F of the difference feature image d The input is fed into the deep feature extraction module, first passing through the semantic generator; simultaneously, F c and F d It is also fed into the cross-attention of the transformer decoder to obtain features with more global context, and the output of the semantic generator is represented as T. c and T d It is given by the following formula:

[0077] T c =(A c ) T F c (7)

[0078] T d =(A d ) T F d (8)

[0079] Where A c and A d They are F c F d It was obtained through a 3x3 convolution, (A c ) T and (A) d ) T This represents the transpose of a matrix.

[0080] T is obtained through pointwise convolution. c With T d Global tags and difference tags, T c , C = 32, L = H * W.

[0081] like Figure 4 As shown, the generated T c and T dAs input to the transformer encoder, it generates labels with more global context and boundary information, T ce and T de :

[0082] [T ce ,I de ] = TransformerEncoder(T c ,I d (9)

[0083] F c F d T ce and I de The input is fed into the transformer decoder to obtain the deep global features F. cd Deep difference feature F dd :

[0084] F cd =TransformerDecoder(F c ,T ce (10)

[0085] F dd =TransformerDecoder(F d ,T de (11)

[0086] Where F cd , C = 32, H = 64, W = 64.

[0087] Figure 5 The differential feature enhancement module interacts with deep global features and deep differential features, integrating different cues from dual-temporal image features. This is achieved by using global image features F... o Modulate the differential image features F in a way that relies on prior knowledge. p The fusion differential features F of fine-grained local changes in the enhanced dual-temporal images were obtained. x :

[0088] F o =Conv(ReLU(Conv(F) cd (12)

[0089] F p =Conv(ReLU(Conv(F) dd (13)

[0090] F x =F p *(F o +1)+Fp (14)

[0091] Where F o and F p Enhanced differential features F are obtained by using spatial attention through two 1x1 convolutions. x Finally, the classification results are obtained.

[0092] Figure 6 This image compares the results of a remote sensing image change detection model based on a differential feature enhancement network with other change detection models on the LEVIR-CD dataset. Figure 6 As shown, the transformation detection results of the differential feature enhancement network based on progressive attention are more accurate than those of other models.

[0093] Figure 7 This is a visualization of the change detection model. (Example:) Figure 7 As shown, on the LEVIR-CD dataset, given any pair of remote sensing images of the same region from different periods, the differential feature enhancement network model based on progressive attention can derive the corresponding changes.

[0094] This invention proposes a differential feature enhancement network based on progressive attention for change detection. A progressive attention module is introduced to mine the similarity and difference features of biphasic images to preserve more complete semantic information. Furthermore, a feature difference enhancement module is added to modulate the difference image features using global image features as prior knowledge, resulting in fused differential features of fine-grained local changes in the biphasic images. With the help of the proposed global feature enhancement differential feature strategy, the similarity and difference information between biphasic images, as well as the autocorrelation of image regions, are unified to achieve comprehensive semantic understanding and fusion. Extensive experiments on LEVIR-CD demonstrate that the model achieves excellent results in change detection. In future work, we will continue to explore how to better learn the semantic information of high-resolution remote sensing images and effectively integrate it for change detection.

[0095] Finally, the details of the above examples of the present invention are merely illustrative of the invention. Any modifications, improvements, and substitutions to the above embodiments by those skilled in the art should be included within the scope of protection of the claims of the present invention.

Claims

1. A remote sensing image change detection method based on a differential feature enhancement network with progressive attention, characterized in that, The method includes the following steps: S1. Construct a progressive attention module to obtain global features and differential features based on the features of the input biphasic image; S2. Extract contextual features using a deep feature extraction module that includes a semantic extractor, a Transformer encoder, and a Transformer decoder; S3. Construct a differential feature enhancement module to enhance differential features based on global features; S4. Combine the modules in S1, S2 and S3 to construct a differential feature enhancement network architecture based on progressive attention; The specific process of S1 is as follows: The progressive attention module mines visual features based on the correlation and differences between biphasic images, enhances global contextual information using channel-level attention, and improves learning ability through residual connections, thereby increasing the utilization rate of boundary information. The specific operation is as follows: Given biphasic image features Given C=32, H=64, and W=64, generate the following steps: : ) (1) Where Conv is a 3x3 convolution kernel that reduces the dimensionality of the concatenated features; BN() refers to normalization, and ReLU() is the activation function; Depend on Obtain the pooled features : (2) Where GAP() is the global average pooling function and GMP() is the global max pooling function; Again Using a 1x1 convolution kernel and activation function, and then... The output of the asymptotic attention module is obtained after multiplication and residual connection. : (3) Where σ() is the Sigmoid function; Feature subtraction is performed on the biphasic image, and then processed through a progressive attention module to obtain... The steps are as follows: ) (4) (5) (6) The specific process of S2 is as follows: Summation features obtained by the progressive attention module and differences The data is input into the deep feature extraction module and first passes through the semantic generator; simultaneously, and It is also fed into the cross-attention of the transformer decoder to obtain features with more global context, and the output of the semantic generator is represented as and It is given by the following formula: (7) (8) in and They are , It was obtained through a 3x3 convolution. and Represents the transpose of a matrix; The results are obtained through point-by-point convolution. and As the initial global label and initial difference label C=32, L=H*W; then generate the initial labels. and Input the transformer encoder to perform global context enhancement and generate enhancement tags. and : (9) Will , , The input is fed into the transformer decoder to obtain deep global features. Differential features with deep layers : (10) (11) in C=32, H=64, W=64; The specific process of S3 is as follows: The differential feature enhancement module interacts with deep global features and deep differential features, integrating different cues from biphasic image features, and using global image features... Modulate deep differential features based on prior knowledge. The fusion difference features of fine-grained local changes in the enhanced biphasic image were obtained. : (12) * + (13) Enhanced differential features are obtained by utilizing spatial attention. Finally, the classification results are obtained.

Images