A multi-level difference feature grouping fusion high-resolution change detection method
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHONGQING UNIV
- Filing Date
- 2025-03-11
- Publication Date
- 2026-06-02
Smart Images
Figure CN120219958B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of deep learning technology and relates to a high-resolution change detection method based on multi-level differential feature grouping and fusion. Background Technology
[0002] Remote sensing change detection identifies areas of change by comparing remote sensing images of the same geographic region from different periods, providing crucial information for land cover change observation. With the widespread use of high-resolution satellites such as QuickBird, IKONOS, and WorldView, change detection technology has been further advanced and is widely applied in fields such as urban planning, environmental monitoring, disaster assessment, and resource management. Therefore, obtaining high-precision change prediction maps is essential for accurately interpreting observed scenarios.
[0003] Traditional change detection methods typically rely on pixel-level difference analysis, identifying changed regions by comparing remote sensing images from different time points. Common methods include principal component analysis (PCA), independent component analysis (ICA), mixture conditional random field (MFR) models, and support vector machines (SVMs). However, these methods depend on manually designed features and are often affected by factors such as noise, image registration accuracy, and illumination variations in complex scenes, limiting the accuracy and reliability of the detection results. In recent years, with the continuous development of deep learning, change detection methods based on convolutional neural networks (CNNs) have been widely applied. However, these methods struggle to effectively capture global information, making it difficult to maintain structural integrity when dealing with large-scale changed regions. To address this issue, researchers have applied Transformers to change detection tasks, further improving detection accuracy by effectively aggregating contextual information. However, these methods only use Transformers as single-phase feature extractors, failing to fully exploit the change information between two-phase images. Furthermore, existing high-resolution change detection methods based on Transformers still need improvement in their ability to integrate features at different levels, failing to fully utilize the correlation and complementarity between features at different levels, resulting in limited performance in complex scenes. Summary of the Invention
[0004] In view of this, the purpose of this invention is to provide a high-resolution change detection method based on multi-level differential feature grouping and fusion. This method extracts multi-level features from two temporal remote sensing images using a differential global Transformer (DiffFormer), incorporating differential information while capturing global information to generate richer feature representations and enhance the model's perceptual capabilities. Subsequently, a Spatial Change Awareness Module (SCAM) is introduced to enhance the model's ability to perceive complex shapes, avoiding ambiguity or misjudgment when processing details or boundary regions. Finally, to efficiently fuse the extracted multi-level differential features, a novel feature grouping and fusion paradigm is introduced, enabling the model to simultaneously focus on local details and global semantic information, thereby achieving high-precision change detection.
[0005] To achieve the above objectives, the present invention provides the following technical solution:
[0006] A high-resolution change detection method based on multi-level differential feature grouping and fusion is provided. The method includes the following steps: S1: preprocessing the high-resolution images used as the training set to obtain training samples; S2: inputting the training samples into the multi-level differential feature grouping and fusion network for training; S3: after training, performing change detection on the test samples to obtain the results.
[0007] Further, in step S1, the two temporal high-resolution images used as the training set are preprocessed. First, image registration is performed to ensure alignment between the two temporal images; then, atmospheric correction, radiometric correction, and geometric correction are performed to eliminate atmospheric effects and radiometric differences under different sensors and shooting conditions, and to ensure the geometric accuracy of the images; then, the images are cropped using a 256×256 non-overlapping window, and data augmentation operations are performed, including random flipping, random rescaling, Gaussian blur, and random color dithering, where the random rescaling ratio ranges from 0.8 to 1.2; finally, the images are normalized to ensure that the images are comparable at the same scale.
[0008] Furthermore, in step S2, the training samples are input into a multi-stage DiffFormer to extract rich multi-scale features. The DiffFormer consists of repeated consecutive blocks. In the initial block, the features of two time phases are first normalized by layers, and then a difference operation is performed to calculate the difference operator. The normalized features of the corresponding time phases and the difference operator are then input into the Differential Attention Mechanism (DW-MSA) for processing, as shown in the following formula:
[0009]
[0010]
[0011] in, , The input features are two-phase features, and LN is the layer normalization. For difference operators, This indicates the characteristics of the DW-MSA output.
[0012] Then, the processed two-phase features are subtracted again, and then input into DW-MSA for further processing along with the corresponding phase features to obtain the residuals. These residuals are then added to the input features, as shown in the following formula:
[0013]
[0014]
[0015] in, , The processed two-phase characteristics, For difference operators, This indicates the characteristics after addition.
[0016] Finally, the summed features undergo another layer normalization operation, then pass through a multilayer perceptron, using Gaussian error linear units as non-linear activation functions, and are added to the summed features, as shown in the following formula:
[0017]
[0018] MLP stands for Multilayer Perceptron. This indicates the characteristics of the initial block output.
[0019] The features output from the initial block are input into subsequent blocks. The second attention operation in subsequent blocks uses the Global Differential Attention (GDW-MSA) mechanism. This mechanism introduces a global query component to compensate for the shortcomings of local attention without increasing the computational cost. The formula is as follows:
[0020]
[0021]
[0022]
[0023] in, This represents the characteristics of the initial block output. This indicates the characteristics of the DW-MSA output of subsequent blocks. , These are the difference operators for DW-MSA and GDW-MSA in subsequent blocks, respectively. For global query components, This represents the characteristics after GDW-MSA summation. This indicates the characteristics of the output of subsequent blocks.
[0024] Furthermore, in DW-MSA, the input features are first transformed into query values through a linear layer, and the difference operator is transformed into key values and attribute values, as shown in the following formula:
[0025]
[0026] in, Indicates input features, This represents the difference operator, where Q, K, and V represent the obtained query value, key value, and attribute value, respectively, and W... q W k W v These represent the projected weights of the query value, key value, and attribute value, respectively.
[0027] Then, after dividing the input into h heads along the channel dimension, window attention is calculated using non-overlapping windows. The calculated attention values are then concatenated along the channel dimension to obtain the final output of DW-MSA, as shown in the following formula:
[0028]
[0029] in, , , This represents the query value, key value, and attribute value of the nth header. represents the attention of the nth head, softmax() represents the activation function that maps the value to the interval [0,1], d represents the dimension of each head, [] represents the concatenation operation on the channel dimension, and A represents the attention value obtained by DW-MSA.
[0030] Unlike DW-MSA, GDW-MSA introduces a global query component, adding attention weights guided by the global query when calculating the attention for each head, as shown in the following formula:
[0031]
[0032] in, Indicates the global query component. This represents the attention of the nth head in GDW-MSA.
[0033] Furthermore, in GDW-MSA, the global query component is pre-computed by the global query generator at each stage and shared across all blocks to interact with local key-value pairs. First, the height and width of the input feature map are transformed into the height and width of a local window through a repeated feature matching module. Then, through reshaping and repetition, a global query component with the same resolution as the local query is obtained. The feature matching module mainly includes convolutional layers and a compression activation module, applying the GELU non-linear activation function and combining residual connections to enhance feature transfer efficiency. Max pooling layers are also used for downsampling to achieve progressive feature dimensionality reduction, as shown in the following formula:
[0034]
[0035] Where X is the input feature map, FM(X) is the feature map obtained after passing through the feature matching module, MaxPool() represents the max pooling operation, and SE represents the compressed activation module. , These represent convolution operations with kernel sizes of 1×1 and 3×3, respectively.
[0036] In the compressed activation module, compression, activation, and multiplication feature fusion operations are used to adjust the importance of each channel, thereby enhancing the module's representational ability and reducing computational complexity. The formula is as follows:
[0037]
[0038] Where X is the input feature map, SE(X) is the feature map obtained after the compressed activation module, and AvgPool() represents the average pooling operation. Indicates a linear layer. This represents the Sigmoid activation function, and res() represents the integer operation.
[0039] Furthermore, the feature maps before and after the first-stage DiffFormer extraction are input into SCAM. In this module, the feature maps first undergo preliminary feature transformation through two 3×3 convolutional layers and a ReLU activation function. Then, feature enhancement is performed through multi-scale expanded deformable convolutional blocks. The enhanced feature map is concatenated with the input feature map along the channel dimension, and then passed through another 3×3 convolutional layer to generate residual features. Finally, the residual features are added to the original input feature map to optimize the features, as shown in the following formula:
[0040]
[0041] Where X is the input feature map, SCAM(X) is the feature map obtained after passing through the spatial variation perception module, and MDDC is a multi-scale dilated deformable convolutional block.
[0042] In the multi-scale dilated deformable convolutional block, the input feature map is processed by a 1×1 convolutional layer and three dilated deformable convolutional blocks with kernel size of 3×3 and dilatation rates of 3, 6, and 12, respectively. The feature maps processed by each branch are concatenated along the channel dimension to form the output of the module, as shown in the following formula:
[0043]
[0044] Where X is the input feature map, and MDDC(X) is the feature map obtained after passing through a multi-scale dilated deformable convolutional block. , , These represent expanded deformable convolutional blocks with a kernel size of 3×3 and a void ratio of 3, 6, and 12, respectively.
[0045] Within each dilated deformable convolutional block, two dilated convolutional layers with 3×3 kernels and a ReLU activation function are first used to expand the receptive field. Next, the transformed feature map is added to the input feature map to ensure no loss of original information details. Finally, after processing through a 3×3 convolutional layer, a deformable convolutional layer is introduced to dynamically adjust the position and shape of the convolutional kernel, thereby more accurately adapting to the complex and irregular contour features of the changing region. The formula is as follows:
[0046]
[0047] Where X is the input feature map. This represents a dilated convolution operation with a kernel size of 3×3 and a dilation rate of 3. This represents deformable convolution operation.
[0048] Furthermore, a multi-scale difference feature map is obtained by performing a difference operation on the two-phase multi-scale features extracted by the multi-stage DiffFormer and processed by SCAM, as shown in the following formula:
[0049]
[0050] in, and Indicates two different time phases. , These represent the features extracted by DiffFormer in the i-th stage of different time phases. This represents the feature map obtained by the feature difference between the two phases after the DiffFormer in the i-th stage. , These represent the features processed by SCAM at different time points. This represents the feature map obtained by the feature difference between the two phases after SCAM processing in the i-th stage.
[0051] Furthermore, low-level difference features , , The input is fed into the low-level feature fusion module for adaptive fusion. In the low-level feature fusion module, the first step is to... Projecting onto a convolutional module consisting of convolutional layers, batch normalization layers, and ReLU activation functions, and then... , To add them together, use the following formula:
[0052]
[0053] Where F is the additive feature map, This represents the convolutional module; then, the summed features are enhanced separately through channel and spatial attention modules, and the enhanced features are summed again. Finally, a weight map is generated using the sigmoid activation function, as shown in the following formula:
[0054]
[0055] Where F is the additive feature map and w is the weight map. Let represent the Sigmoid activation function, CA represent the channel attention module, and SA represent the spatial attention module; and let the weight map be compared with the input features respectively. , Perform a multiplication operation, and then multiply the resulting features with the input features. , The features are added together and then subjected to a 1×1 convolution to generate the final fused features, as shown in the following formula:
[0056]
[0057] Where w is the weighted graph, Represents a 1×1 convolution. This is the output feature map of the low-level feature fusion module.
[0058] In the channel attention module, the input feature map is first subjected to max pooling and average pooling operations based on length and width, respectively. Then, the pooled features are fed into two multilayer perceptrons and summed. After being processed by the sigmoid activation function to obtain weighted coefficients, the weighted coefficients are multiplied element-wise with the input feature map to generate the final output of the module, as shown in the following formula:
[0059]
[0060] Where CA(X) is the feature map obtained after passing through the channel attention module, AvgPool(X) and MaxPool(X) represent the feature maps obtained after average pooling and max pooling operations, respectively, and MLP represents a two-layer multilayer perceptron.
[0061] In the spatial attention module, the input feature map is first subjected to channel-based max pooling and average pooling operations, respectively. Then, the pooled features are concatenated along the channel dimension. Next, a 7×7 convolution is used for dimensionality reduction. Finally, after processing with the sigmoid activation function, the feature map is multiplied with the input feature map to obtain the final output of the module, as shown in the following formula:
[0062]
[0063] Where SA(X) is the feature map obtained after passing through the spatial attention module. This indicates a convolution operation with a kernel size of 7×7.
[0064] Furthermore, high-level differences will be considered. , , The input features are then fused in the high-level feature fusion module. In the high-level feature fusion module, the input features... First, the data is processed by a convolutional module, and then the resolution is enhanced by an upsampling module consisting of convolutional layers. The upsampled features are then compared with another feature. After concatenation along the channel dimension, and then processing through the same convolutional and upsampling modules, it is combined with another feature. The features are then concatenated, and finally, the resolution of the resulting features is adjusted by a convolutional module. After processing by a normalization-based attention module, the final fused features are output, as shown in the following formula:
[0065]
[0066] Where U represents the upsampling module, and NAM represents the normalization-based attention module. This is the output feature map of the high-level feature fusion module.
[0067] Furthermore, the high- and low-level fused features are input into the feature merging module for effective interaction, forming a more comprehensive feature representation. In the feature merging module, the low-level fused features are first passed through an average pooling layer and two 1×1 convolutional layers, then multiplied element-wise with themselves, and finally processed by the Softmax activation function to obtain the enhanced low-level fused features, as shown in the following formula:
[0068]
[0069] in, This represents a two-layer 1×1 convolution operation. To enhance low-level fusion features.
[0070] Simultaneously, the high-level fusion features are input into dilated convolutional modules with kernel sizes of 3×3 and dilation rates of 1, 3, and 5, respectively. The features output by each module are concatenated along the channel dimension, and then integrated through a 1×1 convolutional module to reduce the number of channels and enhance the expressive power of the features, ultimately obtaining the enhanced high-level fusion features, as shown in the following formula:
[0071]
[0072] in, , , These represent dilated convolutional modules with a kernel size of 3×3 and dilation rates of 1, 3, and 5, respectively. This represents a 1×1 convolutional module. To enhance the high-level fusion features.
[0073] Finally, the enhanced low-level fusion features are processed through a 1×1 convolution and then multiplied with the enhanced high-level fusion features. The result of the matrix multiplication is then normalized using the Softmax activation function and added to the high-level fusion features, as shown in the following formula:
[0074]
[0075] in, These are low-level fusion features resulting from the interaction.
[0076] The same procedure is performed on the enhanced high-level fusion features to obtain Finally, the obtained features are concatenated along the channel dimension and integrated using a 1×1 convolution module. This integrated feature is then fed into a normalization-based attention module to generate the final merged feature. Finally, it passes through an upsampling module and a classification layer to obtain the final change detection result, as shown in the following formula:
[0077]
[0078] in, For the high-level fusion features after interaction, C represents the classification layer. This is the final result of the change detection.
[0079] Furthermore, the loss function for the overall network is set as follows:
[0080]
[0081] Where X represents the predicted change detection result, and Y represents the ground truth label of a given sample. A loss function is calculated, and the model parameters of the change detection framework are optimized based on the loss function and the backpropagation process. After training, a trained change detection framework is obtained. The trained change detection framework is then used to discriminate input samples, and a change detection result map is output.
[0082] The beneficial effects of this invention are as follows:
[0083] The Multi-Level Differential Feature Grouping Fusion Network (MDGF) proposed in this invention effectively extracts and integrates multi-level, multi-scale salient feature information, enhancing the model's understanding of local details and global semantics, and improving the accuracy and robustness of change detection in complex changing regions. This invention proposes a novel Transformer structure (DiffFormer) that simultaneously introduces change information and global information into the attention mechanism, effectively modeling long-distance interdependencies between different objects and significantly enhancing the feature representation of change information. This invention also proposes a Spatial Change Awareness Module (SCAM) that can capture the overall structure of changing regions and better adapt to the complex shapes and edges of these regions. Finally, this invention proposes a novel multi-level feature fusion method that adaptively combines spatial detail information and rich semantic information, thereby improving the model's performance in recognizing changes of interest in complex scenes. Experimental results on three publicly available high-resolution change detection datasets demonstrate that the proposed MDGF outperforms state-of-the-art high-resolution change detection methods.
[0084] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description
[0085] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:
[0086] Figure 1 This is a flowchart of the method of the present invention;
[0087] Figure 2 A diagram of the Multilevel Differential Feature Grouping Fusion Network (MDGF) for high-resolution change detection;
[0088] Figure 3Here are the structural diagrams of the Differential Global Transformer (DiffFormer) of the present invention, where (a) is the structural diagram of a continuous block in DiffFormer, (b) is the structural diagram of DW-MSA, and (c) is the structural diagram of GDW-MSA.
[0089] Figure 4 This is a structural diagram of the global query generator of the present invention;
[0090] Figure 5 This is a structural diagram of the Spatial Change Sensing Module (SCAM) of the present invention;
[0091] Figure 6 This is a structural diagram of the low-level feature fusion module of the present invention;
[0092] Figure 7 This is a structural diagram of the high-level feature fusion module of the present invention;
[0093] Figure 8 This is a structural diagram of the feature merging module of the present invention;
[0094] Figure 9 These are several instances from the LEVIR-CD dataset, where the Image1 column is the image before the change, the Image2 column is the image after the change, and the MDGF column is the change detection result of the method described in this invention;
[0095] Figure 10 Visualization results for different methods on the LEVIR-CD dataset, where the methods include FC-EF, FC-Siam-Di, FC-Siam-Conc, DTCDSCN, SNUNet, BIT, ChangeFormer, RDPNet, MDGF, and GT columns are ground truth maps. Detailed Implementation
[0096] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings.
[0097] Figure 1The flowchart of the method of this invention is shown in Figure 2. This invention provides a multi-level differential feature grouping and fusion method for high-resolution change detection. As shown in the figure, in the image preprocessing stage, radiometric correction, geometric correction and other methods are used to quickly achieve image registration and perform data augmentation operations. The deep learning network used for change detection is shown in Figure 2. It can effectively extract and integrate multi-scale salient feature information, improving the accuracy and robustness of change detection. The network consists of a differential global Transformer (DiffFormer), a spatial change awareness module (SCAM), and a new feature grouping and fusion paradigm. First, the dual-temporal high-resolution image obtains multi-scale differential features through DiffFormer and the spatial change awareness module designed for low-level features. Then, a new feature grouping and fusion paradigm is introduced to efficiently fuse local details and global semantic information, thereby achieving high-precision change detection. The differential global Transformer (DiffFormer) designed in this invention is used to effectively model long-distance interdependencies and significantly enhance the feature representation of change information. The spatial change awareness module (SCAM) designed in this invention is used to capture the overall structure of the change region and better adapt to the complex shape of the change region. The feature grouping and fusion paradigm designed in this invention adaptively combines spatial detail information and rich semantic information to improve the model's performance in recognizing changes of interest in complex scenes. Specifically, the technical solution of this invention includes the following:
[0098] 1. High-resolution image preprocessing: The two temporal high-resolution images used as the training set are preprocessed. First, image registration is performed to ensure alignment between the two temporal images. Next, atmospheric correction, radiometric correction, and geometric correction are performed to eliminate atmospheric effects and radiometric differences under different sensors and shooting conditions, and to ensure the geometric accuracy of the images. Then, the images are cropped using a 256×256 non-overlapping window, and data augmentation operations are performed, including random flipping, random rescaling, Gaussian blur, and random color dithering, where the random rescaling ratio ranges from 0.8 to 1.2. Finally, the images are normalized to ensure that the images are comparable at the same scale.
[0099] 2. Input the training samples into a multi-stage DiffFormer to extract rich multi-scale features. The DiffFormer consists of repeating consecutive blocks, such as... Figure 3 As shown in (a), in the initial block, the two-phase features are first normalized by the layer, and then the difference operator is calculated by the difference operation. The normalized features of the corresponding phases and the difference operator are input into the differential attention mechanism DW-MSA for processing, as shown in the following formula:
[0100]
[0101]
[0102] in, , The input features are two-phase features, and LN is the layer normalization. For difference operators, This indicates the characteristics of the DW-MSA output.
[0103] Then, the processed two-phase features are subtracted again, and then input into DW-MSA for further processing along with the corresponding phase features to obtain the residuals. These residuals are then added to the input features, as shown in the following formula:
[0104]
[0105]
[0106] in, , The processed two-phase characteristics, For difference operators, This indicates the characteristics after addition.
[0107] Finally, the summed features undergo another layer normalization operation, then pass through a multilayer perceptron, using Gaussian error linear units as non-linear activation functions, and are added to the summed features, as shown in the following formula:
[0108]
[0109] MLP stands for Multilayer Perceptron. This indicates the characteristics of the initial block output.
[0110] The features output from the initial block are input into subsequent blocks. The second attention operation in subsequent blocks uses the Global Differential Attention (GDW-MSA) mechanism. This mechanism introduces a global query component to compensate for the shortcomings of local attention without increasing the computational cost. The formula is as follows:
[0111]
[0112]
[0113]
[0114] in, This represents the characteristics of the initial block output. This indicates the characteristics of the DW-MSA output of subsequent blocks. , These are the difference operators for DW-MSA and GDW-MSA in subsequent blocks, respectively. For global query components, This represents the characteristics after GDW-MSA summation. This indicates the characteristics of the output of subsequent blocks.
[0115] 3. A schematic diagram of DW-MSA is shown below. Figure 3 As shown in (b), in DW-MSA, the input features are first transformed into query values through a linear layer, and the difference operator is transformed into key values and attribute values, as shown in the following formula:
[0116]
[0117] in, Indicates input features, This represents the difference operator, where Q, K, and V represent the obtained query value, key value, and attribute value, respectively, and W... q W k W v These represent the projected weights of the query value, key value, and attribute value, respectively.
[0118] Then, after dividing the input into h heads along the channel dimension, window attention is calculated using non-overlapping windows. The calculated attention values are then concatenated along the channel dimension to obtain the final output of DW-MSA, as shown in the following formula:
[0119]
[0120] in, , , This represents the query value, key value, and attribute value of the nth header. represents the attention of the nth head, softmax() represents the activation function that maps the value to the interval [0,1], d represents the dimension of each head, [] represents the concatenation operation on the channel dimension, and A represents the attention value obtained by DW-MSA.
[0121] A schematic diagram of GDW-MSA is shown below. Figure 3 As shown in (c), unlike DW-MSA, GDW-MSA introduces a global query component, which adds attention weights guided by the global query when calculating the attention of each head, as shown in the following formula:
[0122]
[0123] in, Indicates the global query component. This represents the attention of the nth head in GDW-MSA.
[0124] 4. In GDW-MSA, the global query component is pre-computed at each stage by a global query generator and shared across all blocks to interact with local key-value pairs. A schematic diagram of the global query generator is shown below. Figure 4 As shown, the input feature map's height and width are first transformed into the height and width of a local window through a repeated feature matching module. Then, a global query component with the same resolution as the local query is obtained through reshaping and repetition. The feature matching module mainly includes convolutional layers and a compression activation module, which applies the GELU nonlinear activation function and combines residual connections to enhance feature transfer efficiency. Max pooling layers are also used for downsampling to achieve gradual feature dimensionality reduction, as shown in the following formula:
[0125]
[0126] Where X is the input feature map, FM(X) is the feature map obtained after passing through the feature matching module, MaxPool() represents the max pooling operation, and SE represents the compressed activation module. , These represent convolution operations with kernel sizes of 1×1 and 3×3, respectively.
[0127] In the compressed activation module, compression, activation, and multiplication feature fusion operations are used to adjust the importance of each channel, thereby enhancing the module's representational ability and reducing computational complexity. The formula is as follows:
[0128]
[0129] Where X is the input feature map, SE(X) is the feature map obtained after the compressed activation module, and AvgPool() represents the average pooling operation. Indicates a linear layer. This represents the Sigmoid activation function, and res() represents the integer operation.
[0130] 5. Input the feature maps before and after the first-stage DiffFormer extraction into SCAM respectively. A schematic diagram of SCAM is shown below. Figure 5 As shown, in this module, the feature map first undergoes preliminary feature transformation through two 3×3 convolutional layers and a ReLU activation function. Then, it is enhanced through multi-scale dilated deformable convolutional blocks. The enhanced feature map is concatenated with the input feature map along the channel dimension, and then passed through another 3×3 convolutional layer to generate residual features. Finally, the residual features are added to the original input feature map to optimize the features, as shown in the following formula:
[0131]
[0132] Where X is the input feature map, SCAM(X) is the feature map obtained after passing through the spatial variation perception module, and MDDC is a multi-scale dilated deformable convolutional block.
[0133] In the multi-scale dilated deformable convolutional block, the input feature map is processed by a 1×1 convolutional layer and three dilated deformable convolutional blocks with kernel size of 3×3 and dilatation rates of 3, 6, and 12, respectively. The feature maps processed by each branch are concatenated along the channel dimension to form the output of the module, as shown in the following formula:
[0134]
[0135] Where X is the input feature map, and MDDC(X) is the feature map obtained after passing through a multi-scale dilated deformable convolutional block. , , These represent expanded deformable convolutional blocks with a kernel size of 3×3 and a void ratio of 3, 6, and 12, respectively.
[0136] Within each dilated deformable convolutional block, two dilated convolutional layers with 3×3 kernels and a ReLU activation function are first used to expand the receptive field. Next, the transformed feature map is added to the input feature map to ensure no loss of original information details. Finally, after processing through a 3×3 convolutional layer, a deformable convolutional layer is introduced to dynamically adjust the position and shape of the convolutional kernel, thereby more accurately adapting to the complex and irregular contour features of the changing region. The formula is as follows:
[0137]
[0138] Where X is the input feature map. This represents a dilated convolution operation with a kernel size of 3×3 and a dilation rate of 3. This represents deformable convolution operation.
[0139] 6. Perform a difference operation on the two-phase multi-scale features extracted by the multi-stage DiffFormer and processed by SCAM to obtain a multi-scale difference feature map, as shown in the following formula:
[0140]
[0141] in, and Indicates two different time phases. , These represent the features extracted by DiffFormer in the i-th stage of different time phases. This represents the feature map obtained by the feature difference between the two phases after the DiffFormer in the i-th stage. , These represent the features processed by SCAM at different time points. This represents the feature map obtained by the feature difference between the two phases after SCAM processing in the i-th stage.
[0142] 7. Low-level difference features , , The input is fed into a low-level feature fusion module for adaptive fusion. For example... Figure 6 As shown, in the low-level feature fusion module, firstly... Projecting onto a convolutional module consisting of convolutional layers, batch normalization layers, and ReLU activation functions, and then... , To add them together, use the following formula:
[0143]
[0144] Where F is the additive feature map, This represents the convolutional module; then, the summed features are enhanced separately through channel and spatial attention modules, and the enhanced features are summed again. Finally, a weight map is generated using the sigmoid activation function, as shown in the following formula:
[0145]
[0146] Where F is the additive feature map and w is the weight map. Let represent the Sigmoid activation function, CA represent the channel attention module, and SA represent the spatial attention module; and let the weight map be compared with the input features respectively. , Perform a multiplication operation, and then multiply the resulting features with the input features. , The features are added together and then subjected to a 1×1 convolution to generate the final fused features, as shown in the following formula:
[0147]
[0148] Where w is the weighted graph, Represents a 1×1 convolution. This is the output feature map of the low-level feature fusion module.
[0149] In the channel attention module, the input feature map is first subjected to max pooling and average pooling operations based on length and width, respectively. Then, the pooled features are fed into two multilayer perceptrons and summed. After being processed by the sigmoid activation function to obtain weighted coefficients, the weighted coefficients are multiplied element-wise with the input feature map to generate the final output of the module, as shown in the following formula:
[0150]
[0151] Where CA(X) is the feature map obtained after passing through the channel attention module, AvgPool(X) and MaxPool(X) represent the feature maps obtained after average pooling and max pooling operations, respectively, and MLP represents a two-layer multilayer perceptron.
[0152] In the spatial attention module, the input feature map is first subjected to channel-based max pooling and average pooling operations, respectively. Then, the pooled features are concatenated along the channel dimension. Next, a 7×7 convolution is used for dimensionality reduction. Finally, after processing with the sigmoid activation function, the feature map is multiplied with the input feature map to obtain the final output of the module, as shown in the following formula:
[0153]
[0154] Where SA(X) is the feature map obtained after passing through the spatial attention module. This indicates a convolution operation with a kernel size of 7×7.
[0155] 8. High-level difference characteristics , , The input is then fed into a high-level feature fusion module for final fusion. For example... Figure 7 As shown, in the high-level feature fusion module, the input features First, the data is processed by a convolutional module, and then the resolution is enhanced by an upsampling module consisting of convolutional layers. The upsampled features are then compared with another feature. After concatenation along the channel dimension, and then processing through the same convolutional and upsampling modules, it is combined with another feature. The features are then concatenated, and finally, the resolution of the resulting features is adjusted by a convolutional module. After processing by a normalization-based attention module, the final fused features are output, as shown in the following formula:
[0156]
[0157] Where U represents the upsampling module, and NAM represents the normalization-based attention module. This is the output feature map of the high-level feature fusion module.
[0158] 9. Input the high- and low-level fused features into the feature merging module for effective interaction, forming a more comprehensive feature representation. For example... Figure 8 As shown, in the feature merging module, the low-level fused features are first passed through an average pooling layer and two 1×1 convolutional layers, then multiplied element-wise with themselves, and finally processed by the Softmax activation function to obtain the enhanced low-level fused features, as shown in the following formula:
[0159]
[0160] in, This represents a two-layer 1×1 convolution operation. To enhance low-level fusion features.
[0161] Simultaneously, the high-level fusion features are input into dilated convolutional modules with kernel sizes of 3×3 and dilation rates of 1, 3, and 5, respectively. The features output by each module are concatenated along the channel dimension, and then integrated through a 1×1 convolutional module to reduce the number of channels and enhance the expressive power of the features, ultimately obtaining the enhanced high-level fusion features, as shown in the following formula:
[0162]
[0163] in, , , These represent dilated convolutional modules with a kernel size of 3×3 and dilation rates of 1, 3, and 5, respectively. This represents a 1×1 convolutional module. To enhance the high-level fusion features.
[0164] Finally, the enhanced low-level fusion features are processed through a 1×1 convolution and then multiplied with the enhanced high-level fusion features. The result of the matrix multiplication is then normalized using the Softmax activation function and added to the high-level fusion features, as shown in the following formula:
[0165]
[0166] in, These are low-level fusion features resulting from the interaction.
[0167] The same procedure is performed on the enhanced high-level fusion features to obtain Finally, the features obtained from the above two steps are concatenated along the channel dimension and integrated using a 1×1 convolution module. This integrated feature is then fed into a normalization-based attention module to generate the final merged feature. Finally, it passes through an upsampling module and a classification layer to obtain the final change detection result, as shown in the following formula:
[0168]
[0169] in, For the high-level fusion features after interaction, C represents the classification layer. This is the final result of the change detection.
[0170] 10. The loss function for the entire network is set as follows: Where X represents the predicted change detection result, and Y represents the ground truth label of a given sample. A loss function is calculated, and the model parameters of the change detection framework are optimized based on the loss function and the backpropagation process. After training, a trained change detection framework is obtained. The trained change detection framework is then used to discriminate input samples, and a change detection result map is output.
[0171] like Figure 9 The experimental results of the MDGF change detection network described in this invention on an open-source high-resolution dataset show that changed regions are detected very well. The detection performance of this invention can be further illustrated through comparative experiments. On the LEVIR-CD dataset, the method of this invention is compared with other existing methods such as FC-EF, FC-Siam-Di, FC-Siam-Conc, DTCDSCN, SNUNet, BIT, ChangeFormer, and RDPNet. Precision, Recall, F1 score, Intersection over Union (IoU), and Overall Accuracy are calculated respectively. A higher precision indicates a higher proportion of correct predictions among all positive results; a higher recall indicates a higher proportion of correctly predicted positive results; and a higher F1 score indicates a better overall evaluation. Table 1 shows the values of various indicators for change detection results from different methods.
[0172] Table 1. Comparison of MDGF with various methods on the LEVIR-CD dataset.
[0173]
[0174] It can be seen that the method of the present invention achieves the best accuracy on this dataset. Figure 10 Visualized detection results of the above methods are presented, demonstrating that the method described in this invention outperforms other high-resolution image building change detection methods. The proposed method can accurately locate the position of changed regions and has advantages over other methods in adapting to complex backgrounds.
[0175] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications should be covered within the scope of the claims of the present invention.
Claims
1. A high-resolution change detection method based on multi-level differential feature grouping and fusion, characterized in that: The method includes the following steps: S1: Preprocess the high-resolution images used as the training set to obtain training samples; S2: Input the training samples into the multi-level differential feature grouping and fusion network for training; S3: After training is completed, perform change detection on the test samples to obtain the results; In step S1, the two-phase high-resolution images used as the training set are preprocessed. First, image registration is performed to ensure alignment between the two phase images. Next, atmospheric correction, radiometric correction, and geometric correction are performed to eliminate atmospheric effects and radiometric differences under different sensors and shooting conditions, and to ensure the geometric accuracy of the images. Then, the images are cropped using a 256×256 non-overlapping window, and data augmentation operations are performed, including random flipping, random rescaling, Gaussian blur, and random color dithering, where the random rescaling ratio ranges from 0.8 to 1.
2. Finally, the images are normalized to ensure that the images are comparable at the same scale. In step S2, the training samples are input into a multi-stage DiffFormer to extract rich multi-scale features. The DiffFormer consists of repeated consecutive blocks. In the initial block, the features of two time phases are first normalized by layers, and then the difference operator is calculated by difference operation. The normalized features of the corresponding time phases and the difference operator are input into the differential attention mechanism DW-MSA for processing, as shown in the following formula: in, , The input features are two-phase features, and LN is the layer normalization. For difference operators, This indicates the characteristics of the DW-MSA output; Then, the processed two-phase features are subtracted again, and then input into DW-MSA for further processing along with the corresponding phase features to obtain the residuals. These residuals are then added to the input features, as shown in the following formula: in, , The processed two-phase characteristics, For difference operators, Indicates the characteristics after addition; Finally, the summed features undergo another layer normalization operation, then pass through a multilayer perceptron, using Gaussian error linear units as non-linear activation functions, and are added to the summed features, as shown in the following formula: MLP stands for Multilayer Perceptron. This represents the characteristics of the initial block output; The features output from the initial block are input into subsequent blocks. The second attention operation in subsequent blocks uses the Global Differential Attention (GDW-MSA) mechanism. This mechanism introduces a global query component to compensate for the shortcomings of local attention without increasing the computational cost. The formula is as follows: in, This represents the characteristics of the initial block output. This indicates the characteristics of the DW-MSA output of subsequent blocks. , These are the difference operators for DW-MSA and GDW-MSA in subsequent blocks, respectively. For global query components, This represents the characteristics after GDW-MSA summation. This indicates the characteristics of the output of subsequent blocks; In DW-MSA, the input features are first transformed into query values through a linear layer, and the difference operator is transformed into key values and attribute values. Then, the input is divided into h heads in the channel dimension, and window attention is calculated using non-overlapping windows. The calculated attention is then concatenated in the channel dimension. Unlike DW-MSA, GDW-MSA introduces a global query component, which adds attention weights guided by the global query when calculating the attention of each head.
2. The high-resolution change detection method based on multi-level differential feature grouping and fusion according to claim 1, characterized in that: In DW-MSA, the formulas for calculating query values, key values, and attribute values are as follows: in, Indicates input features, This represents the difference operator, where Q, K, and V represent the obtained query value, key value, and attribute value, respectively, and W... q W k W v These represent the projected weights of the query value, key value, and attribute value, respectively. The final output of DW-MSA is obtained using the following formula: in, , , This represents the query value, key value, and attribute value of the nth header. represents the attention of the nth head, softmax() represents the activation function that maps the value to the interval [0,1], d represents the dimension of each head, [] represents the concatenation operation on the channel dimension, and A represents the attention value obtained by DW-MSA; The formula for calculating GDW-MSA is as follows: in, Indicates the global query component. This represents the attention of the nth head in GDW-MSA; In GDW-MSA, the global query component is pre-computed by the global query generator at each stage and shared among all blocks to interact with local key-value pairs. First, a repeated feature matching module transforms the height and width of the input feature map into the height and width of a local window. Then, through reshaping and repetition, a global query component with the same resolution as the local query is obtained. The feature matching module mainly includes convolutional layers and a compressed activation module, applying the GELU non-linear activation function and combining residual connections to enhance feature transfer efficiency. Max pooling layers are also used for downsampling to achieve progressive feature dimensionality reduction, as shown in the following formula: Where X is the input feature map, FM(X) is the feature map obtained after passing through the feature matching module, MaxPool() represents the max pooling operation, and SE represents the compressed activation module. , These represent convolution operations with kernel sizes of 1×1 and 3×3, respectively. In the compressed activation module, compression, activation, and multiplication feature fusion operations are used to adjust the importance of each channel, thereby enhancing the module's representational ability and reducing computational complexity. The formula is as follows: Where X is the input feature map, SE(X) is the feature map obtained after the compressed activation module, and AvgPool() represents the average pooling operation. Indicates a linear layer. This represents the Sigmoid activation function, and res() represents the integer operation.
3. The high-resolution change detection method based on multi-level differential feature grouping and fusion according to claim 2, characterized in that: The feature maps before and after extraction by the first-stage DiffFormer are input into SCAM. In this module, the feature maps first undergo preliminary feature transformation through two 3×3 convolutional layers and a ReLU activation function. Then, feature enhancement is performed through multi-scale dilated deformable convolutional blocks. The enhanced feature map is concatenated with the input feature map along the channel dimension, and then passed through a 3×3 convolutional layer to generate residual features. Finally, the residual features are added to the original input feature map to optimize the features, as shown in the following formula: Where X is the input feature map, SCAM(X) is the feature map obtained after passing through the spatial variation perception module, MDDC is a multi-scale dilated deformable convolutional block, and ReLU is the activation function; In the multi-scale dilated deformable convolutional block, the input feature map is processed by a 1×1 convolutional layer and three dilated deformable convolutional blocks with kernel size of 3×3 and dilatation rates of 3, 6, and 12, respectively. The feature maps processed by each branch are concatenated along the channel dimension to form the output of the module, as shown in the following formula: Where X is the input feature map, and MDDC(X) is the feature map obtained after passing through a multi-scale dilated deformable convolutional block. , , These represent expanded deformable convolutional blocks with a kernel size of 3×3 and dilatations of 3, 6, and 12, respectively. Within each dilated deformable convolutional block, two dilated convolutional layers with 3×3 kernels and a ReLU activation function are first used to expand the receptive field. Next, the transformed feature map is added to the input feature map to ensure no loss of original information details. Finally, after processing through a 3×3 convolutional layer, a deformable convolutional layer is introduced to dynamically adjust the position and shape of the convolutional kernels, thereby more accurately adapting to the complex and irregular contour features of the changing region. The formula is as follows: Where X is the input feature map. This represents a dilated convolution operation with a kernel size of 3×3 and a dilation rate of 3. This represents deformable convolution operation.
4. The high-resolution change detection method based on multi-level differential feature grouping and fusion according to claim 3, characterized in that: The multi-scale difference feature map is obtained by performing a difference operation on the two-phase multi-scale features extracted by the multi-stage DiffFormer and processed by SCAM, as shown in the following formula: in, and Indicates two different time phases. , These represent the features extracted by DiffFormer in the i-th stage of different time phases. This represents the feature map obtained by the feature difference between the two phases after the DiffFormer in the i-th stage. , These represent the features processed by SCAM at different time points. This represents the feature map obtained by the feature difference between the two phases after SCAM processing in the i-th stage; Low-level difference features , , The input is fed into the low-level feature fusion module for adaptive fusion. In the low-level feature fusion module, the first step is to... Projecting onto a convolutional module consisting of convolutional layers, batch normalization layers, and ReLU activation functions, and then... , To add them together, use the following formula: Where F is the additive feature map, This represents the convolutional module; then, the summed features are enhanced separately through channel and spatial attention modules, and the enhanced features are summed again. Finally, a weight map is generated using the sigmoid activation function, as shown in the following formula: Where F is the additive feature map and w is the weight map. Let represent the Sigmoid activation function, CA represent the channel attention module, and SA represent the spatial attention module; and let the weight map be compared with the input features respectively. , Perform a multiplication operation, and then multiply the resulting features with the input features. , The features are added together and then subjected to a 1×1 convolution to generate the final fused features, as shown in the following formula: Where w is the weighted graph, Represents a 1×1 convolution. This is the output feature map of the low-level feature fusion module; In the channel attention module, the input feature map is first subjected to max pooling and average pooling operations based on length and width, respectively. Then, the pooled features are fed into two multilayer perceptrons and summed. After being processed by the sigmoid activation function to obtain weighted coefficients, the weighted coefficients are multiplied element-wise with the input feature map to generate the final output of the module, as shown in the following formula: Where CA(X) is the feature map obtained after passing through the channel attention module, AvgPool(X) and MaxPool(X) represent the feature maps obtained after average pooling and max pooling operations, respectively, and MLP represents a two-layer multilayer perceptron. In the spatial attention module, the input feature map is first subjected to channel-based max pooling and average pooling operations, respectively. Then, the pooled features are concatenated along the channel dimension. Next, a 7×7 convolution is used for dimensionality reduction. Finally, after processing with the sigmoid activation function, the feature map is multiplied with the input feature map to obtain the final output of the module, as shown in the following formula: Where SA(X) is the feature map obtained after passing through the spatial attention module. This indicates a convolution operation with a kernel size of 7×7.
5. The high-resolution change detection method based on multi-level differential feature grouping and fusion according to claim 4, characterized in that: High-level difference characteristics , , The input features are then fused in the high-level feature fusion module. First, the data is processed by a convolutional module, and then the resolution is enhanced by an upsampling module consisting of convolutional layers. The upsampled features are then compared with another feature. After concatenation along the channel dimension, and then processing through the same convolutional and upsampling modules, it is combined with another feature. The features are then concatenated, and finally, the resolution of the resulting features is adjusted by a convolutional module. After processing by a normalization-based attention module, the final fused features are output, as shown in the following formula: Where U represents the upsampling module, and NAM represents the normalization-based attention module. This is the output feature map of the high-level feature fusion module.
6. The high-resolution change detection method based on multi-level differential feature grouping and fusion according to claim 5, characterized in that: The high- and low-level fused features are input into the feature merging module for effective interaction, forming a more comprehensive feature representation. In the feature merging module, the low-level fused features are first passed through an average pooling layer and two 1×1 convolution layers, then multiplied element-wise with themselves, and finally processed by the Softmax activation function to obtain the enhanced low-level fused features, as shown in the following formula: in, This represents a two-layer 1×1 convolution operation. To enhance low-level fusion features; Simultaneously, the high-level fusion features are input into dilated convolutional modules with kernel sizes of 3×3 and dilation rates of 1, 3, and 5, respectively. The features output by each module are concatenated along the channel dimension, and then integrated through a 1×1 convolutional module to reduce the number of channels and enhance the expressive power of the features, ultimately obtaining the enhanced high-level fusion features, as shown in the following formula: in, , , These represent dilated convolutional modules with a kernel size of 3×3 and dilation rates of 1, 3, and 5, respectively. This represents a 1×1 convolutional module. To enhance high-level fusion features; Finally, the enhanced low-level fusion features are processed through a 1×1 convolution and then multiplied with the enhanced high-level fusion features. The result of the matrix multiplication is then normalized using the Softmax activation function and added to the high-level fusion features, as shown in the following formula: in, These are low-level fusion features resulting from the interaction. The same procedure is performed on the enhanced high-level fusion features to obtain Finally, the obtained features are concatenated along the channel dimension and integrated using a 1×1 convolution module. This integrated feature is then fed into a normalization-based attention module to generate the final merged feature. Finally, it passes through an upsampling module and a classification layer to obtain the final change detection result, as shown in the following formula: in, For the high-level fusion features after interaction, C represents the classification layer. This is the final change detection result; The loss function for the overall network is set as follows: Where X represents the predicted change detection result, Y represents the ground truth label of the given sample, the loss function is calculated, the model parameters of the change detection framework are optimized based on the loss function and the backpropagation process, and after training, the trained change detection framework is obtained; the trained change detection framework is used to discriminate the input samples and output the change detection result map.