Remote sensing image change detection method and equipment based on multi-scale difference feature fusion

Abstract

The invention relates to a remote sensing image change detection method and equipment based on multi-scale difference feature fusion, and belongs to the technical field of remote sensing image processing. According to the invention, a multi-attention feature enhancement module is designed to carry out adaptive weight adjustment and enhancement on a dual-time-phase image and differential information thereof, meanwhile, a cross-scale residual fusion module is adopted to carry out unified fusion on features from different scales, and fused global change features are input to a classification head to carry out change detection. According to the method, the details and the overall structure of the ground object change area are accurately captured, so that the accuracy and the robustness of change detection are effectively improved, and better detection performance can be obtained under the condition that the number of training samples is small.

Description

Technical Field

[0001] The present invention relates to a remote sensing image change detection method and device based on multi-scale difference feature fusion, belonging to the technical field of remote sensing image processing. Background Art

[0002] Remote sensing image change detection aims to identify and extract information about land cover changes by analyzing the spectral and spatial differences between dual-temporal hyperspectral images of the same area. This technology has been widely used in disaster assessment, land cover change analysis, and geographic information systems (GIS).

[0003] Hyperspectral remote sensing images offer irreplaceable advantages in ground change detection due to their rich spectral information. However, they also face challenges such as high-dimensional data redundancy, spectral noise interference, and the inadequate detection accuracy of traditional methods when dealing with complex backgrounds and subtle changes. Traditional methods rely on statistical and algebraic operations (such as spectral differencing and principal component analysis) or shallow feature transformations, resulting in sensitivity to predefined parameters, poor noise immunity, and difficulty capturing complex nonlinear changes. While deep learning-based convolutional neural networks and Transformer methods have achieved promising results in extracting global and local information, they still have shortcomings in handling multi-scale variations, capturing subtle change features, and comprehensively considering spectral and spatial context. 1) Single-scale features are difficult to adapt to the needs of detecting changes in objects of varying sizes; 2) traditional attention mechanisms are inadequate in exploiting feature differences between bi-temporal and difference images; and 3) the correlations between multi-scale features are not fully modeled, leading to missed detection of small objects and blurred boundaries. Therefore, how to efficiently fuse features across different scales and extract detailed change information while fully leveraging the rich information in hyperspectral images remains a key technical challenge that urgently needs to be overcome. Summary of the Invention

[0004] The purpose of the present invention is to provide a remote sensing image change detection method based on multi-scale difference feature fusion to address the problems of insufficient multi-scale information, incomplete capture of subtle changes and low robustness of traditional methods in the existing hyperspectral remote sensing image change detection technology.

[0005] The technical solution adopted by the present invention is:

[0006] The remote sensing image change detection method based on multi-scale difference feature fusion includes the following steps:

[0007] S1. Extract multiple layers of preliminary features from dual-phase hyperspectral images;

[0008] S2. The extracted bi-phase preliminary features and their difference features are input into a multi-attention feature enhancement module for feature enhancement. The module performs channel-wise attention enhancement on the bi-phase preliminary features and spatial attention enhancement on the difference features. These features are then fused using a weighted feature approach to obtain the enhanced features.

[0009] S3. The enhanced features at different scales output by each multi-attention feature enhancement module are fed into the cross-scale residual fusion module for fusion. The cross-scale residual fusion module resizes the enhanced features from different scales and then concatenates them. It then uses the residual connection structure to add and fuse the information at different scales to form a comprehensive global change feature.

[0010] S4. Input the fused global change features into the classification head for change detection.

[0011] In the above method, step S1 uses depthwise separable convolution to perform preliminary feature extraction.

[0012] The difference feature described in step S2 is obtained by element-by-element subtraction of the bi-phase preliminary features; the channel attention enhancement first uses the global average pooling operation to extract the representative element F of the feature in each channel avg , then the channel attention weight W is obtained through full connection and Sigmoid function, and then the weight vector W is multiplied element by element with the dual-phase preliminary feature to obtain the channel feature enhanced feature; the spatial attention enhancement is to perform average pooling and maximum pooling on the channel dimension for the input difference feature, and obtain the spatial representation F of the feature map avg and F max , F avg and F max Splicing is performed, followed by convolution operation, and the normalized spatial attention matrix F is obtained through the Sigmoid function atention , using spatial attention F atention Multiply the input difference features pixel by pixel to obtain features enhanced by spatial features.

[0013] In step S2, multi-source features are fused by feature weighting, and the calculation method is as follows:

[0014] F out =λ1F1'+λ2F"+λ3F2',

[0015] Among them, λ1, λ2, and λ3 are adjustable weight hyperparameters used to balance the contribution of attention in different dimensions. F1' and F2' are features enhanced by channel features, and F' is a feature enhanced by spatial features. out is the enhanced feature obtained by fusion.

[0016] The cross-scale residual fusion formula in step S3 is:

[0017] F concat =[US(F out_1 ), F out_2 , DS(F out_3 )],

[0018] F find =F concat +US(F out_1 )+F out_2 +DS(F out_3 ),

[0019] Among them, F out_i is the enhanced feature, F concat is the concatenated feature, US(·) represents the upsampling operation, DS(·) represents the downsampling operation, and F final Represents the global change characteristics after the final cross-scale residual fusion.

[0020] In step S4, the classification head is mapped to the change category channel through 1×1 convolution, and the Softmax function is used to normalize and classify each pixel, and finally a high-precision change detection map is output.

[0021] Another object of the present invention is to provide a remote sensing image change detection device based on multi-scale difference feature fusion, including a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the program, the remote sensing image change detection method based on multi-scale difference feature fusion as described above is implemented.

[0022] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the remote sensing image change detection method based on multi-scale difference feature fusion as described above.

[0023] The beneficial effects of the present invention are:

[0024] The present invention enhances the spectral and spatial features of dual-phase images and difference images respectively through a channel-spatial attention mechanism, and dynamically focuses on key change areas by using feature weighting, effectively solving the limitation of traditional attention mechanism in insufficient modeling of feature differences and improving sensitivity to complex nonlinear changes. At the same time, the present invention fuses multi-resolution features of different levels of encoders through cross-scale residual connections, integrating global semantic information while retaining the original details, significantly enhancing the model's ability to detect changes in objects of different sizes, and avoiding missed detection of small targets caused by single-scale features. In addition, the present invention effectively alleviates the boundary ambiguity problem and improves the internal compactness and edge integrity of the change area through the complementary fusion of multi-scale features and the retention of residual information. The overall solution of the present invention introduces a multi-attention feature enhancement module and a cross-scale residual fusion module. While fully extracting the dual-phase features and difference information of the hyperspectral image, it achieves efficient complementarity and information interaction between multi-scale features, effectively integrates the global spectral-spatial features and multi-scale difference information of the dual-phase hyperspectral image, and significantly improves the accuracy and robustness of change detection. It is particularly suitable for small target change detection in complex scenes and boundary integrity analysis of large-scale land object changes. It can achieve better detection performance even with fewer training samples, and realizes efficient coordination of multi-source features under lightweight design. It has important application value in agricultural management, disaster assessment, land use monitoring and other fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a model network framework diagram of the method of the present invention;

[0026] Figure 2 This is a schematic diagram of the structure of the multi-attention feature enhancement module of the present invention;

[0027] Figure 3 Schematic diagram of the cross-scale residual fusion module structure of the present invention. DETAILED DESCRIPTION

[0028] The present invention is further described below with reference to specific embodiments.

[0029] Example 1 A remote sensing image change detection method based on multi-scale difference feature fusion includes the following steps:

[0030] S1. Extract multiple layers of preliminary features from dual-phase hyperspectral images:

[0031] Input the dual-phase hyperspectral image and extract preliminary spectral-spatial features through the convolution layer, such as Figure 1As shown, the input of the present invention is a pair of dual-phase remote sensing images, each representing the same geographical area at two different time points. Let the input dual-phase images be X1 and X2 respectively. These images are used to capture surface change information, that is, to detect changes that occur between two moments. The present invention uses depthwise separable convolution for preliminary feature extraction. While improving feature learning efficiency, depthwise separable convolution can reduce computing resource consumption. The difference features are obtained by element-by-element subtraction of the dual-phase image features. The preliminary extracted features serve as input for subsequent steps. The above steps can be formulated as follows:

[0032] F pre =DWConv(X1),

[0033] F post =DWConv(X2),

[0034] F diff =F pre -F post ,

[0035] Here, DWConv(·) represents the depthwise separable convolution operation.

[0036] S2. The extracted bi-phase preliminary features and their difference features of each layer are input into a multi-attention feature enhancement module for feature enhancement. The multi-attention feature enhancement module performs channel attention enhancement on the bi-phase preliminary features and spatial attention enhancement on the difference features. The enhanced features are then fused using feature weighting to obtain the following:

[0037] The embodiment of the present invention has three levels of features. The input of each multi-attention feature enhancement module is the dual-phase image features and difference features of the current level. The multi-attention feature enhancement modules of different levels differ only in the input, and the operations performed are the same. The following multi-attention feature enhancement steps are based on the input F of the first level. pre 、F post 、F diff Take this as an example to illustrate.

[0038] like Figure 2 As shown, the multi-attention feature enhancement module of the present invention has three inputs, namely, the bi-phase preliminary feature F pre 、F post and the difference feature F diff .

[0039] The present invention is for the dual-phase preliminary feature F (representative feature F pre 、F post ) to enhance channel attention, such as Figure 2 As shown, firstly, the global average pooling operation is used to extract the representative element F of the feature in each channel avg, then the channel attention weight W is obtained through full connection and Sigmoid function, and then the weight vector W is multiplied element by element with the original feature map F to dynamically adjust the contribution of each channel. The above steps can be formulated as:

[0040] F avg =GlobalAvgPool(F),

[0041] W=σ(FC(F avg ),

[0042] F′=F×W,

[0043] Among them, GlobalAvgPool(·) represents the global average pooling operation, FC(·) represents the fully connected operation, σ(·) represents the Sigmoid function, and F' represents the feature enhanced by channel features.

[0044] The present invention enhances spatial attention to difference features, such as Figure 2 As shown, for the input feature F diff , spatial attention first performs average pooling and maximum pooling in the channel dimension, and efficiently obtains the spatial representation F of the feature map avg and F max , the feature map space is represented by F avg and F max Splicing is performed, followed by convolution operation, and the normalized spatial attention matrix F is obtained through the Sigmoid function attention , using spatial attention F attention With input feature F diff Multiply pixel by pixel to get the weighted feature F. The above steps can be formulated as:

[0045] F avg =AvgPool(F diff ),

[0046] F max =MaxPool(F diff ),

[0047] F attention =σ(Conv([F avg , F max ])),

[0048] F"=F diff ×F attention

[0049] Among them, AvgPool(·) represents the channel average pooling operation, MaxPool(·) represents the channel maximum pooling operation, Conv(·) represents the convolution operation, σ(·) represents the Sigmoid function, and F” represents the feature after spatial feature enhancement.

[0050] After obtaining the multi-source features enhanced by the attention mechanism, the present invention fuses the multi-source features by feature weighting, and the calculation method is as follows:

[0051] F out =λ1F1 ' +λ2F”+λ3F2 ' ,

[0052] Among them, λ1, λ2, and λ3 are adjustable weight hyperparameters used to balance the contribution of attention in different dimensions. F1' and F2' are features enhanced by channel features, and F' is a feature enhanced by spatial features. out is the enhanced feature obtained by fusion.

[0053] The multi-attention feature enhancement module introduces multiple attention mechanisms to adaptively weight and fine-tune the input hyperspectral image features. This effectively highlights key spectral and spatial information while suppressing redundancy and noise interference, thereby strengthening the discriminative features that reflect change areas in the image. This mechanism not only redistributes the importance of different features in the channel dimension, but also captures local change information in the spatial dimension, making the overall feature expression more comprehensive and refined, providing a solid and highly recognizable foundation for subsequent multi-scale feature fusion and change detection.

[0054] S3. The enhanced features of different scales output by each multi-attention feature enhancement module are input into the cross-scale residual fusion module for fusion. The cross-scale residual fusion module concatenates the enhanced features from different scales after resizing them to a uniform size. It then uses the residual connection structure to add and fuse the information of different scales to form a comprehensive global change feature:

[0055] like Figure 3 As shown in the figure, the present invention adjusts the multi-attention enhancement features from different scales to a uniform size to obtain the splicing feature F concat , multi-scale features are aligned through convolution, and then the residual connection structure is used to add and fuse information of different scales to maintain the integrity and detail information of the original features and form a comprehensive global change feature. The above process can be formulated as:

[0056] F concat =[US(F out_1 ), F out_2 , DS(F out_3 )],

[0057] F final =F concat +US(F out_1 )+F out_2 +DS(F out_3 ),

[0058] Among them, F out_i is the enhanced feature, F concat is the concatenated feature, US(·) represents the upsampling operation, DS(·) represents the downsampling operation, and F final Represents the global change characteristics after the final cross-scale residual fusion.

[0059] The cross-scale residual fusion module effectively integrates feature information from different scales. It unifies the size and aligns the information of multi-scale features, unifies the resolution of features at different levels (such as shallow high-resolution detail features, middle-level medium semantic features, and deep low-resolution global features), and uses residual connections to maintain the original feature information to avoid information loss during the fusion process, thereby enhancing the model's ability to capture information changes at each scale, improving detection accuracy and robustness. At the same time, it complements the multi-attention feature enhancement stage, that is, the multi-attention feature enhancement module focuses on feature enhancement in key areas, while the cross-scale residual fusion module ensures the effective integration of multi-scale information.

[0060] S4. Input the fused global change features into the classification head for change detection:

[0061] Get the final global change feature F final Finally, the classification head is used to map the final features to the number of channels of the change categories to generate the change detection results. The classification head is mapped to the change category channel through 1×1 convolution, and the Softmax function is used to normalize and classify each pixel point, and finally a high-precision change detection map is output to achieve accurate recognition and positioning of ground object changes. Assuming the number of change categories is K, the classification head uses 1×1 convolution to map the final global change feature f final Perform channel mapping to obtain feature f class ∈R K×H×W , the mapped feature f class Applied to the classification of change categories, the change detection results are generated through pixel-by-pixel classification operations. The above algorithm can be formulated as:

[0062] f class [:, i, j] = C1(f final )

[0063] P i,j (k)=Softmax(f class [:i,j]),

[0064]

[0065] Among them, C1(·) represents a 1×1 convolution, the Softmax(·) function normalizes the classification result of each pixel into a probability distribution to ensure that the sum is 1, and the argmax(·) function represents the category corresponding to the maximum probability. The predicted categories of all pixels are combined into a complete change detection map exist In , the value of each pixel represents the change category it describes.

[0066] Example 2 A remote sensing image change detection device based on multi-scale difference feature fusion includes a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the program, it implements the remote sensing image change detection method based on multi-scale difference feature fusion as described in Example 1 above.

[0067] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the remote sensing image change detection method based on multi-scale difference feature fusion as described in Example 1 above.

[0068] The above is a further description of the present invention in conjunction with typical embodiments, and the protection scope of the present invention is not limited thereto.

Claims

1. A remote sensing image change detection method based on multi-scale difference feature fusion, characterized by: The steps are as follows: S1. Extract multiple layers of preliminary features from dual-phase hyperspectral images; S2. The extracted bi-phase preliminary features and their difference features are input into a multi-attention feature enhancement module for feature enhancement. The module performs channel-wise attention enhancement on the bi-phase preliminary features and spatial attention enhancement on the difference features. These features are then fused using a weighted feature approach to obtain the enhanced features. S3. The enhanced features at different scales output by each multi-attention feature enhancement module are fed into the cross-scale residual fusion module for fusion. The cross-scale residual fusion module resizes the enhanced features from different scales and then concatenates them. It then uses the residual connection structure to add and fuse the information at different scales to form a comprehensive global change feature. S4. Input the fused global change features into the classification head for change detection.

2. The remote sensing image change detection method based on multi-scale difference feature fusion according to claim 1 is characterized in that: Step S1 uses depthwise separable convolution for preliminary feature extraction.

3. The remote sensing image change detection method based on multi-scale difference feature fusion according to claim 1 is characterized in that: The difference features described in step S2 are obtained by element-by-element subtraction of the bi-phase preliminary features.

4. The remote sensing image change detection method based on multi-scale difference feature fusion according to claim 1 is characterized in that: The channel attention enhancement described in step S2 first uses the global average pooling operation to extract the representative elements F of the features in each channel avg , then the channel attention weight W is obtained through full connection and Sigmoid function, and then the weight vector W is multiplied element by element with the bi-phase preliminary feature to obtain the feature enhanced by the channel feature; The spatial attention enhancement is to perform average pooling and maximum pooling on the channel dimension for the input difference features, and obtain the spatial representation F of the feature map avg and F max , the feature map space is represented by F avg and F max Splicing is performed, followed by convolution operation, and the normalized spatial attention matrix F is obtained through the Sigmoid function atention , using spatial attention F atention Multiply the input difference features pixel by pixel to obtain features enhanced by spatial features.

5. The remote sensing image change detection method based on multi-scale difference feature fusion according to claim 1 is characterized in that: In step S2, multi-source features are fused by feature weighting, and the calculation method is as follows: F out =λ1F1'+λ2F”+λ3F2', Among them, λ1, λ2, and λ3 are adjustable weight hyperparameters used to balance the contribution of attention in different dimensions. F1' and F2' are features enhanced by channel features, and F' is a feature enhanced by spatial features. out is the enhanced feature obtained by fusion.

6. The remote sensing image change detection method based on multi-scale difference feature fusion according to claim 1 is characterized in that: The cross-scale residual fusion formula in step S3 is: F concat =[US(F out_1 ),F out_2 ,DS(F out_3 )], F final =F concat +US(F out_1 )+F out_2 +DS(F out_3 ), Among them, F out_i is the enhanced feature, F concat is the concatenated feature, US(·) represents the upsampling operation, DS(·) represents the downsampling operation, and F final Represents the global change characteristics after the final cross-scale residual fusion.

7. The remote sensing image change detection method based on multi-scale difference feature fusion according to claim 1 is characterized in that: In step S4, the classification head is mapped to the change category channel through 1×1 convolution, and the Softmax function is used to normalize and classify each pixel, and finally a high-precision change detection map is output.

8. A remote sensing image change detection device based on multi-scale difference feature fusion, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the remote sensing image change detection method based on multi-scale difference feature fusion as described in any one of claims 1 to 7 is implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the remote sensing image change detection method based on multi-scale difference feature fusion as described in any one of claims 1 to 7 is implemented.

Images