Remote sensing image change detection method and device based on multi-scale difference feature fusion

Abstract

The present application relates to a kind of remote sensing image change detection method and equipment based on multiscale difference feature fusion, belong to remote sensing image processing technical field.The present application is by designing multi-attention feature enhancement module to double time-phase image and its difference information are adaptively weighted and enhanced, while using cross-scale residual fusion module to the feature from different scales is unified fusion, global change feature after fusion is input into classification head and is carried out change detection, the accurate capture of the detail and overall structure of the change region of ground object is realized, to effectively improve the accuracy and robustness of change detection, and also can obtain better detection performance under the condition of less training sample.

Description

Technical Field

[0001] This invention relates to a method and device for detecting changes in remote sensing images based on multi-scale difference feature fusion, belonging to the field of remote sensing image processing technology. Background Technology

[0002] Remote sensing image change detection aims to identify and extract land cover change information by analyzing the spectral and spatial differences between dual-temporal hyperspectral images of the same area. This technology has been widely applied in fields such as disaster assessment, land cover change analysis, and geographic information systems (GIS).

[0003] Hyperspectral remote sensing images possess irreplaceable advantages in detecting ground cover changes due to their rich spectral information. However, they also face challenges such as high-dimensional data redundancy, spectral noise interference, and insufficient detection accuracy of traditional methods when dealing with complex backgrounds and subtle changes. Traditional methods rely on statistical and algebraic operations (such as spectral differencing and principal component analysis) or shallow feature transformations, which suffer from sensitivity to predefined parameters, poor noise resistance, and difficulty in capturing complex nonlinear changes. While deep learning-based convolutional neural networks and Transformer methods have achieved good results in extracting global and local information, they still have shortcomings in handling multi-scale changes, capturing subtle change features, and comprehensively considering spectral and spatial context information: 1) Single-scale features are difficult to adapt to the detection needs of ground cover changes of different sizes; 2) Traditional attention mechanisms are insufficient for feature difference mining in dual-temporal images and difference images; 3) The correlation between multi-scale features is not fully modeled, leading to missed detection of small targets and blurred boundaries. Therefore, how to fully utilize the rich information of hyperspectral images while achieving efficient fusion of features at different scales and fine-grained change information extraction remains a key technical problem that urgently needs to be solved. Summary of the Invention

[0004] The purpose of this invention is to address the problems of insufficient multi-scale information, incomplete capture of subtle changes, and low robustness of traditional methods in existing hyperspectral remote sensing image change detection technologies, and to provide a remote sensing image change detection method based on multi-scale difference feature fusion.

[0005] The technical solution adopted in this invention is as follows:

[0006] The remote sensing image change detection method based on multi-scale difference feature fusion includes the following steps:

[0007] S1. Extract multi-layer preliminary features from the dual-temporal hyperspectral images respectively;

[0008] S2. The extracted dual-temporal preliminary features and their differential features are respectively input into a multi-attention feature enhancement module for feature enhancement. In the multi-attention feature enhancement module, channel attention enhancement is performed on the dual-temporal preliminary features, spatial attention enhancement is performed on the differential features, and then the enhanced features are obtained by fusing them through feature weighting.

[0009] S3. The enhanced features at different scales output by each multi-attention feature enhancement module are input into the cross-scale residual fusion module for fusion. The cross-scale residual fusion module stitches the enhanced features from different scales after uniform size adjustment, and then uses the residual connection structure to add and fuse the information at different scales to form a comprehensive global change feature.

[0010] S4. Input the fused global change features into the classification head for change detection.

[0011] In the above method, step S1 uses depthwise separable convolution for preliminary feature extraction.

[0012] The difference features mentioned in step S2 are obtained by element-wise subtraction of the initial features from the two temporal phases; the channel attention enhancement first utilizes global average pooling to extract representative elements F of the features in each channel. avg Subsequently, channel attention weights W are obtained through a fully connected layer and the sigmoid function. Then, the weight vector W is multiplied element-wise with the initial features of the two temporal phases to obtain the features enhanced by the channel features. The spatial attention enhancement involves performing average pooling and max pooling on the channel dimension for the input difference features to obtain the spatial representation F of the feature map. avg and F max , will F avg and F max The data is concatenated, then convolved, and finally the normalized spatial attention matrix F is obtained through the sigmoid function. atention Utilizing spatial attention F atention The differential features of the input are multiplied pixel by pixel to obtain the spatially enhanced features.

[0013] In step S2, multi-source features are fused using a feature weighting method, calculated as follows:

[0014] F out =λ1F1'+λ2F”+λ3F2',

[0015] Where λ1, λ2, and λ3 are adjustable weight hyperparameters used to balance the contributions of attention from different dimensions, F1' and F2' are features enhanced by channel features, and F' is a feature enhanced by spatial features. out The enhanced features obtained through fusion.

[0016] The formula for cross-scale residual fusion in step S3 is:

[0017] F concat =[US(F out_1 ), F out_2 DS(F) out_3 )],

[0018] F find =F concat +US(F out_1 )+F out_2 +DS(F out_3 ),

[0019] Among them, F out_i For each enhanced feature, F concat For feature concatenation, US(·) represents upsampling, DS(·) represents downsampling, and F final This represents the global change characteristics after the final cross-scale residual fusion.

[0020] In step S4, the classification head is mapped to the change category channel through a 1×1 convolution, and the Softmax function is used to normalize and classify each pixel, finally outputting a high-precision change detection map.

[0021] Another objective of this invention is to provide a remote sensing image change detection device based on multi-scale difference feature fusion, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the remote sensing image change detection method based on multi-scale difference feature fusion as described above.

[0022] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the remote sensing image change detection method based on multi-scale difference feature fusion as described above.

[0023] The beneficial effects of this invention are:

[0024] This invention enhances the spectral and spatial features of dual-temporal images and difference images through a channel-spatial attention mechanism, and dynamically focuses on key change regions using feature weighting. This effectively addresses the limitations of traditional attention mechanisms in modeling feature differences and improves sensitivity to complex nonlinear changes. Furthermore, by fusing multi-resolution features from different levels of the encoder through cross-scale residual connections, this invention integrates global semantic information while preserving original details, significantly enhancing the model's ability to detect changes in land features of different sizes and avoiding missed detections of small targets caused by single-scale features. In addition, this invention effectively alleviates boundary ambiguity problems and improves the internal compactness and edge integrity of change regions through complementary fusion of multi-scale features and preservation of residual information. The overall solution of this invention introduces a multi-attention feature enhancement module and a cross-scale residual fusion module. While fully extracting the dual-temporal features and their differences in hyperspectral images, it achieves efficient complementarity and information interaction between multi-scale features. It effectively integrates the global spectral-spatial features and multi-scale difference information of dual-temporal hyperspectral images, significantly improving the accuracy and robustness of change detection. It is particularly suitable for detecting changes in small targets in complex scenes and analyzing the boundary integrity of changes in large-scale land cover. It can also achieve superior detection performance under conditions of limited training samples. It achieves efficient collaboration of multi-source features with a lightweight design, and has important application value in fields such as agricultural management, disaster assessment, and land use monitoring. Attached Figure Description

[0025] Figure 1 This is a model network framework diagram of the method of the present invention;

[0026] Figure 2 This is a schematic diagram of the multi-attention feature enhancement module structure of the present invention;

[0027] Figure 3 This is a schematic diagram of the cross-scale residual fusion module structure of the present invention. Detailed Implementation

[0028] The present invention will be further described below with reference to specific embodiments.

[0029] Example 1: A remote sensing image change detection method based on multi-scale difference feature fusion, comprising the following steps:

[0030] S1. Extract multi-layer preliminary features from the dual-temporal hyperspectral images:

[0031] Input dual-temporal hyperspectral images, and extract preliminary spectral-spatial features through convolutional layers, such as... Figure 1As shown, the input of this invention is a pair of bi-temporal remote sensing images, representing the same geographical area at two different time points. Let the input bi-temporal images be X1 and X2, respectively. These images are used to capture surface change information, that is, to detect changes that occur between the two time points. This invention uses depthwise separable convolution for preliminary feature extraction. Depthwise separable convolution improves feature learning efficiency while reducing computational resource consumption. The difference features are obtained by element-wise subtraction of the features from the bi-temporal images. The initially extracted features serve as input for subsequent steps. The above steps can be formulated as follows:

[0032] F pre =DWConv(X1),

[0033] F post =DWConv(X2),

[0034] F diff =F pre -F post ,

[0035] Where DWConv(·) represents a depthwise separable convolution operation.

[0036] S2. The extracted preliminary features and differential features of each dual-temporal layer are input into a multi-attention feature enhancement module for feature enhancement. The multi-attention feature enhancement module performs channel attention enhancement on the preliminary features of the dual-temporal layers and spatial attention enhancement on the differential features. Then, the enhanced features are obtained by fusing them through feature weighting.

[0037] This invention has three levels of features. The input to each multi-attention feature enhancement module is the dual-temporal image features and difference features of its own level. The differences between the multi-attention feature enhancement modules at different levels are only in the input; the operations performed are the same. The following multi-attention feature enhancement steps take the input F of the first level as an example. pre F post F diff Let's take an example to illustrate.

[0038] like Figure 2 As shown, the multi-attention feature enhancement module of the present invention has three inputs, namely the dual-temporal preliminary feature F. pre F post and differential features F diff .

[0039] This invention addresses the initial dual-phase feature F (representative feature F). pre F post Enhance channel attention, such as Figure 2 As shown, firstly, global average pooling is used to extract representative elements F of the features in each channel. avgThen, channel attention weights W are obtained through a fully connected layer and the sigmoid function. The weight vector W is then multiplied element-wise with the original feature map F to dynamically adjust the contribution of each channel. The above steps can be formulated as follows:

[0040] F avg =GlobalAvgPool(F),

[0041] W=σ(FC(F avg ),

[0042] F′=F×W,

[0043] Where GlobalAvgPool(·) represents the global average pooling operation, FC(·) represents the fully connected operation, σ(·) represents the Sigmoid function, and F' represents the feature enhanced by the channel feature.

[0044] This invention enhances spatial attention for differential features, such as Figure 2 As shown, for input feature F diff Spatial attention first performs average pooling and max pooling along the channel dimension to efficiently obtain the spatial representation F of the feature map. avg and F max The spatial representation F of the feature map avg and F max The data is concatenated, then convolved, and finally the normalized spatial attention matrix F is obtained through the sigmoid function. attention Utilizing spatial attention F attention With input features F diff Pixel-by-pixel multiplication is performed to obtain the weighted feature F. The above steps can be formulated as follows:

[0045] F avg =AvgPool(F diff ),

[0046] F max =MaxPool(F diff ),

[0047] F attention =σ(Conv([F avg F max ])),

[0048] F"=F diff ×F attention

[0049] Where AvgPool(·) represents the channel average pooling operation, MaxPool(·) represents the channel max pooling operation, Conv(·) represents the convolution operation, σ(·) represents the Sigmoid function, and F” represents the feature enhanced by spatial features.

[0050] After obtaining the multi-source features enhanced by the attention mechanism, this invention fuses the multi-source features through feature weighting, calculated as follows:

[0051] F out =λ1F1 ' +λ2F”+λ3F2 ' ,

[0052] Where λ1, λ2, and λ3 are adjustable weight hyperparameters used to balance the contributions of attention from different dimensions, F1' and F2' are features enhanced by channel features, and F' is a feature enhanced by spatial features. out The enhanced features obtained through fusion.

[0053] The multi-attention feature enhancement module introduces multiple attention mechanisms to adaptively weight and finely adjust the features of the input hyperspectral image. This effectively highlights key spectral and spatial information while suppressing redundancy and noise interference, thereby strengthening the discriminative features reflecting changing regions in the image. This mechanism not only redistributes the importance of different features in the channel dimension but also captures local change information in the spatial dimension, making the overall feature representation more comprehensive and refined. This provides a solid and highly discriminative foundation for subsequent multi-scale feature fusion and change detection.

[0054] S3. The enhanced features at different scales output by each multi-attention feature enhancement module are input into the cross-scale residual fusion module for fusion. The cross-scale residual fusion module stitches together the enhanced features from different scales after uniform size adjustment, and then uses the residual connection structure to add and fuse the information from different scales to form a comprehensive global change feature:

[0055] like Figure 3 As shown, this invention obtains a spliced ​​feature F by uniformly adjusting the size of multi-attention enhancement features from different scales. concat By aligning multi-scale features through convolution, and then using residual connection structures to add and fuse information from different scales, the integrity and detailed information of the original features are preserved, forming a comprehensive global variation feature. The above process can be formulated as follows:

[0056] F concat =[US(F out_1 ), F out_2 DS(F) out_3 )],

[0057] F final =F concat +US(F out_1 )+F out_2 +DS(F out_3 ),

[0058] Among them, F out_i For each enhanced feature, F concat For feature concatenation, US(·) represents upsampling, DS(·) represents downsampling, and F final This represents the global change characteristics after the final cross-scale residual fusion.

[0059] The cross-scale residual fusion module effectively integrates feature information from different scales. It unifies the size and aligns the information of multi-scale features, and fuses features at different levels (such as shallow high-resolution detail features, mid-level medium semantic features, and deep low-resolution global features) after unifying their resolution. At the same time, it uses residual connections to preserve the original feature information and avoid information loss during the fusion process. This enhances the model's ability to capture information changes at various scales, improves detection accuracy and robustness, and complements the multi-attention feature enhancement stage. The multi-attention feature enhancement module focuses on feature enhancement in key regions, while the cross-scale residual fusion module ensures the effective integration of multi-scale information.

[0060] S4. Input the fused global change features into the classification head for change detection:

[0061] The final global change feature F is obtained. final Then, the classification head maps the final features to the channel representing the number of change categories, generating change detection results. The classification head maps to the change category channel through a 1×1 convolution and uses the Softmax function to normalize and classify each pixel, ultimately outputting a high-precision change detection map, achieving accurate identification and location of ground feature changes. Let the number of change categories be K, and the classification head maps the final global change features f through a 1×1 convolution. final Perform channel mapping to obtain feature f class ∈R K×H×W The mapped features f class For classification of change categories, change detection results are generated through pixel-by-pixel classification operations. The above algorithm can be formulated as follows:

[0062] f class [:,i,j]=C1(f final )

[0063] P i,j (k)=Softmax(f class [:i,j]),

[0064]

[0065] Here, C1(·) represents a 1×1 convolution, the Softmax(·) function normalizes the classification result of each pixel into a probability distribution, ensuring the sum is 1, and the argmax(·) function takes the category corresponding to the highest probability. The predicted categories of all pixels are combined into a complete change detection map. exist In this context, the value of each pixel represents the category of change it describes.

[0066] Example 2: A remote sensing image change detection device based on multi-scale difference feature fusion, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the remote sensing image change detection method based on multi-scale difference feature fusion as described in Example 1 above.

[0067] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the remote sensing image change detection method based on multi-scale difference feature fusion as described in Embodiment 1 above.

[0068] The above is a further description of the present invention in conjunction with typical embodiments, and the scope of protection of the present invention is not limited thereto.

Claims

1. A remote sensing image change detection method based on multi-scale difference feature fusion, characterized in that, The steps include the following: S1. Extracting multi-layer preliminary features from the dual-time high-spectrum images respectively; S2. Inputting the extracted dual-time preliminary features and their difference features into a multi-attention feature enhancement module respectively for feature enhancement, performing channel attention enhancement on the dual-time preliminary features and spatial attention enhancement on the difference features in the multi-attention feature enhancement module, and then fusing the enhanced features through feature weighting; S3. Inputting the enhanced features of different scales output by the multi-attention feature enhancement modules into a cross-scale residual fusion module for fusion, adjusting the enhanced features from different scales to a unified size for splicing, and then adding and fusing the information of different scales through a residual connection structure to form comprehensive global change features; S4. Inputting the fused global change features into a classification head for change detection.

2. The remote sensing image change detection method based on multi-scale difference feature fusion according to claim 1, characterized in that, Step S1 adopts deep separable convolution for preliminary feature extraction.

3. The remote sensing image change detection method based on multi-scale difference feature fusion according to claim 1, characterized in that, The difference features in step S2 are obtained by element-by-element subtraction of the dual-time preliminary features.

4. The remote sensing image change detection method based on multi-scale difference feature fusion according to claim 1, characterized in that, The channel attention enhancement in step S2 first extracts a representative element F of the feature in each channel by using a global average pooling operation avg , then obtains a channel attention weight W by using a full connection and a Sigmoid function, and then obtains the feature after the channel feature enhancement by element-wise multiplication of the weight vector W and the dual-phase preliminary feature. The spatial attention enhancement described above involves performing average pooling and max pooling on the channel dimension of the input differential features to obtain the spatial representation F of the feature map. avg and F max The spatial representation F of the feature map avg and F max The data is concatenated, then convolved, and finally the normalized spatial attention matrix F is obtained through the sigmoid function. atention Utilizing spatial attention F atention The differential features of the input are multiplied pixel by pixel to obtain the spatially enhanced features.

5. The remote sensing image change detection method based on multi-scale difference feature fusion according to claim 1, characterized in that, In step S2, the multi-source features are fused through feature weighting, and the calculation method is as follows: F out = λ1F'1+ λ2F" + λ3F'2, wherein λ1, λ2, λ3 are adjustable weight hyperparameters for balancing the contribution of different dimensional attentions, F1', F2' are features enhanced by channel feature enhancement, F" is a feature enhanced by spatial feature enhancement, F is a feature enhanced by both channel feature enhancement and spatial feature enhancement out is the fused enhanced feature.

6. The remote sensing image change detection method based on multi-scale difference feature fusion according to claim 1, characterized in that, In step S3, the cross-scale residual fusion formula is: F concat = [US(F out_1 ), F out_2 , DS(F out_3 )], F find = F concat + US(F out_1 ) + F out_2 + DS(F out_3 ), where F out_i is the enhanced feature for each scale, F concat is the concatenation feature, US(·) represents the up-sampling operation, DS(·) represents the down-sampling operation, F final represents the final global change feature after the cross-scale residual fusion.

7. The remote sensing image change detection method based on multi-scale difference feature fusion according to claim 1, characterized in that, In step S4, the classification head is mapped to the change category channel through 1×1 convolution, and the Softmax function is used for normalized classification of each pixel point, and finally a high-precision change detection map is output.

8. A remote sensing image change detection device based on multi-scale difference feature fusion, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the remote sensing image change detection method based on multi-scale difference feature fusion according to any one of claims 1-7.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the remote sensing image change detection method based on multi-scale difference feature fusion according to any one of claims 1-7.

Images