A remote sensing image change detection algorithm based on multi-dimensional feature collaborative modeling

Abstract

The application discloses a remote sensing image change detection algorithm based on multi-dimensional feature collaborative modeling. The steps are as follows: image block preprocessing and division of remote sensing image change detection public reference data set; a remote sensing image change detection model based on multi-dimensional feature collaborative modeling is constructed, for the output from the feature extraction stage, the main branch inputs it into a double-branch feature difference modeling module to model the difference of each layer of double-time feature pairs to form multi-scale difference features, and then inputs it into a boundary protection feature calibration module to strengthen the edge, and the enhanced features are input into a decoder to generate a detection result; the auxiliary branch inputs it into a style difference feature alignment module; after the model is trained, the test set is tested. On the four evaluation indexes of precision, recall, F1 score and intersection over union, compared with the existing change detection method FC-Siam-Conv, it is respectively increased by 3.72%, 6.93%, 5.18% and 8.38%; compared with the existing change detection method BIT, it is respectively increased by 1.63%, 1.11%, 1.38% and 2.32%.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image change detection, and in particular to a remote sensing image change detection algorithm based on multi-dimensional feature collaborative modeling. Background Technology

[0002] Remote sensing technology refers to a comprehensive technology that utilizes sensor equipment installed on platforms such as satellites, aircraft, and drones to acquire electromagnetic wave information reflected or radiated by objects on the Earth's surface or in the atmosphere, and processes and analyzes this data to identify, monitor, and interpret features and changes on the Earth's surface. As a branch of remote sensing technology, change detection primarily focuses on the same geographical area on the Earth's surface, aiming to capture the changing characteristics of land cover information over time. With the development of remote sensing imaging technology, change detection of remote sensing images has been widely applied in many important fields, including land use and planning management, natural disaster assessment and response, crop growth analysis, early warning and detection of geological disasters such as landslides, and dynamic monitoring of the ecological environment. Change detection typically uses pixel-level classification to identify areas of change between two temporal images at different times. Each image pixel is assigned a binary label, where "0" indicates that the area has not changed between the two time points, and "1" indicates that the area has changed. It is worth noting that although some areas may visually exhibit obvious changes, these changes may not necessarily belong to the type of change the mission is interested in. For example, seasonal changes such as tree growth, crop rotation, or changes in building shadow positions, while real changes, are semantically irrelevant to the target of change detection. With the accelerating pace of urbanization and the continuous evolution of the ecological environment, the surface landscape exhibits unprecedented complexity. This complexity is reflected not only in the diversification of land cover types but also in the higher frequency and more complex morphologies of land cover changes. Simultaneously, thanks to the continuous improvement of optical sensor capabilities, remote sensing images have significantly enhanced spatial, spectral, and temporal resolution, resulting in richer details of land features. Against this backdrop, change detection in remote sensing images faces even more severe challenges.

[0003] The closest approach to existing technologies: The differences between multi-temporal images are more complex and diverse, exacerbating the interference of pseudo-changes and further obscuring the discernibility of genuine changes. Significant spectral and textural differences often exist between pairs of dual-temporal images acquired at different times. These interferences, caused by non-changing factors such as illumination variations, atmospheric disturbances, sensor differences, or registration errors, are easily misidentified as changed areas by existing change detection models, resulting in numerous pseudo-changes. Furthermore, genuine changed areas often exhibit weak change characteristics due to complex backgrounds, small target scales, or subtle changes, making them difficult to fully perceive and distinguish during the training of existing change detection models, and easily overlooked or misidentified as unchanged areas. In other words, existing change detection methods still lack a unified and effective solution for simultaneously and accurately perceiving changed areas and suppressing interfering pseudo-changes. Simultaneously, due to the lack of effective modeling of ground feature boundaries in existing change detection methods, feature aliasing easily occurs at the boundaries of adjacent ground features, and multiple changing objects are easily misidentified as a single entity, especially in dense target areas where boundary adhesion and detail blurring are particularly prominent. Therefore, how to accurately highlight the real change region and its boundary structure while suppressing spurious change interference remains a key problem that urgently needs to be solved in the field of change detection. Summary of the Invention

[0004] To overcome the shortcomings of existing change detection methods in perceiving changed regions, suppressing interference and pseudo-changes, and addressing boundary adhesion, this invention provides a remote sensing image change detection algorithm based on multi-dimensional feature collaborative modeling. This algorithm solves the problem of accurately highlighting real changed regions and their boundary structures while suppressing pseudo-change interference.

[0005] To achieve the aforementioned objectives, the technical solution adopted by this invention includes:

[0006] A remote sensing image change detection algorithm based on multi-dimensional feature collaborative modeling includes the following steps:

[0007] Step 1: Perform block preprocessing on the images in the public benchmark dataset for remote sensing image change detection, and divide the dataset;

[0008] Step 2: Construct a remote sensing image change detection model based on multi-dimensional feature collaborative modeling. The model consists of the following parts:

[0009] 2-1 Feature Extraction Backbone Network;

[0010] 2-2 Two-branch feature difference modeling module;

[0011] 2-3 Boundary Protection Feature Calibration Module;

[0012] 2-4 Style Difference Feature Alignment Module;

[0013] 2-5 decoder;

[0014] The feature maps obtained in the feature extraction stage are divided into two paths: a main branch and an auxiliary branch, including the following steps: The main branch directly inputs the dual-temporal feature map pairs from each level of the feature extraction stage into the dual-branch feature difference modeling module to perform difference modeling to form multi-scale difference features; the multi-scale difference features are input into the boundary protection feature calibration module to enhance the feature representation capability of change edges; the enhanced features are uniformly input into the decoder stage to generate the final change detection result; the auxiliary branch inputs the feature maps obtained in the feature extraction stage into the style difference feature alignment module to achieve feature distribution alignment, which is used to alleviate style differences between cross-temporal images;

[0015] Step 3: Using Nvidia GPUs, train the constructed network model in batches using the created training set. During training, save the network model with the minimum training loss and continuously optimize the model using the error backpropagation algorithm. A loss function that combines the main branch and auxiliary branch is designed during training. The main branch is designed with a main loss for the change detection task to optimize pixel-level classification performance. The auxiliary branch is designed with a style alignment loss to guide the alignment of shallow style information with deep features and to alleviate cross-temporal feature distribution shift.

[0016] Step 4: Test the remote sensing images in the test set using the model saved during training to obtain the result images of remote sensing change detection.

[0017] Step 2 uses the Mix Transformer from the SegFormer architecture as the backbone network for feature extraction.

[0018] Step 2, the dual-branch feature difference modeling module, combines feature subtraction branch and feature concatenation branch, and achieves complementarity between the two through feature fusion, and consists of the following steps:

[0019] 2-2-1) The feature subtraction branch performs element-wise subtraction on the two-phase feature maps to obtain preliminary differential features. The SimAM attention mechanism is introduced to emphasize regions of significant change. The resulting attention map performs element-wise weighting on the differential features to enhance the response of regions of change and generate the feature subtraction branch result. This attention map is also used to perform element-wise weighting and residual addition on the original two-phase features to generate an enhanced feature map containing information of significant changes.

[0020] 2-2-2) The feature concatenation branch performs 1×1 convolutions on the enhanced feature maps containing significant changes in the two time phases to generate three sets of features: Query, Key, and Value. Channel attention enhancement is applied to the Query feature. Through a cross-learning mechanism, the Query of one time phase is matched with the Key of another time phase, thereby constructing a bidirectional semantic correspondence. After obtaining the cross-time phase similarity matrix, the matrix is ​​used to multiply the Value feature of the other time phase and then linearly transformed by 1×1 convolution. Finally, it is added to the residual of the original feature to form a semantically aligned enhanced feature. Finally, the semantically aligned enhanced features of the two time phases are concatenated and processed by 3×3 convolution, batch normalization, and ReLU activation to output the feature concatenation branch result.

[0021] 2-2-3) The feature subtraction branch and the feature concatenation branch are added element-wise, and then transformed to the frequency domain space through ReLU and Fast Fourier Transform. In the frequency domain space, a global group filter is introduced, and the features obtained by adding the features of the two branches element-wise are processed by a linear layer and the result of global average pooling to generate channel attention weights. The frequency domain features after filtering and enhancement are then restored to the spatial domain through inverse Fast Fourier Transform. The frequency domain enhanced features are added element-wise with the features of the first two branches, and after normalization and convolution processing, the final multi-scale difference features are output.

[0022] Step 2, the boundary protection feature calibration module, introduces an offset calibration mechanism based on boundary enhancement. This mechanism calculates the offset between adjacent layers of multi-layer differential features, remaps deep features, and corrects shallow features layer by layer. This mitigates the misalignment of semantic and positional information while protecting the boundary. Specifically, it includes the following steps:

[0023] 2-3-1) To enhance the shallow features, a 1×1 convolution is used to compress the multi-channel features, and a boundary probability map is obtained through Sigmoid activation. Then, a two-dimensional 5×5 Gaussian convolution kernel is used for smoothing, and a high-frequency residual structure similar to a Laplacian pyramid is constructed by combining downsampling and upsampling operations. By differentiating the results before and after smoothing, the high-frequency edge components are obtained, and they are multiplied element-wise with the original features and the residuals are added to obtain the shallow features after detail enhancement.

[0024] 2-3-2) Perform offset calibration between adjacent layers, concatenate the shallow features of detail enhancement with the adjacent upsampled deep features, and introduce the SE module to perform channel weighting on the salient region to obtain the fused features. After 3×3 convolution, a semantic offset map is generated. Based on this offset map, grid sampling operation is used to perform pixel-level mapping on the deep features to achieve spatial alignment of deep and shallow features. The aligned deep features are added to the shallow detail enhancement features to obtain the fused features.

[0025] 2-3-3) The above shallow feature detail enhancement and offset calibration strategy between adjacent layers are applied iteratively across multiple levels. That is, deep features are upsampled sequentially and fused with the enhanced features of the previous stage. An offset map is generated through convolution and spatial mapping is performed to gradually complete the alignment and boundary enhancement of multi-layer features. The final fused feature is a boundary enhancement feature map that fuses three layers of features and has undergone offset calibration.

[0026] 2-3-4) The deepest differential features are compressed and upsampled to the same size as the boundary enhancement feature map through 1×1 convolution, and then concatenated with it. After 1×1 convolution and Sigmoid activation, a normalized attention map is generated. This attention map is applied to feature maps of different scales after being downsampled layer by layer.

[0027] The style difference feature alignment module described in step 2 performs feature alignment at both the shallow and deep levels; the outputs of the first and second layers of the feature extraction backbone network are selected as shallow features, and the outputs of the third and fourth layers are selected as deep features.

[0028] 2-4-1) In the shallow stage, instance normalization is used to standardize the shallow features so that the mean and variance of each channel feature are consistent, thereby reducing the distribution differences between images caused by different imaging conditions; then the covariance matrix of the two phase features is calculated, and statistical alignment of shallow features is achieved by minimizing the difference between covariances.

[0029] 2-4-2) In the deep stage, the deep features are flattened into a point set representation. and The feature size is H×W×C, N=H×W represents the number of feature points, and each point x i y j This represents a C-dimensional eigenvector at a spatial location, originating from the unit sphere S. C-1 Randomly sample L directions Projecting the high-dimensional distribution onto a one-dimensional plane yields the corresponding projected representation. and And in each direction θ l The projected values ​​are sorted and matched point by point to calculate the one-dimensional p-order Wasserstein distance in that direction.

[0030]

[0031] Where {x (i)}, {y (i)} respectively represent the directions θ lThe projected values ​​are sorted in ascending order; the energy function f(x) = exp(x) is defined, and the Wasserstein distance in each direction is exponentially amplified; for the l-th direction, the weighting coefficient is w. l for:

[0032]

[0033] The loss function for merging the main branch and auxiliary branch in step 3 is:

[0034] L = L cd +λ sfa L sfa ;

[0035] Among them, L cd Detecting the principal loss by changing the main branch, L sfa To assist in the style alignment loss of the branch, L sfa To adjust the coefficients that affect the style alignment loss of auxiliary branches.

[0036] The main loss for change detection in the main branch employs a combination of binary cross-entropy loss and dice loss, where the binary cross-entropy L... bce The loss is used for pixel-level binary classification supervision, while the dice loss L... dice This focuses on the matching degree of the foreground change region, effectively alleviating the class imbalance problem. The two are defined as follows:

[0037] L cd =λ bce L bce +λ dice L dice ;

[0038]

[0039] Where, λ bce To adjust the coefficient of the binary cross-entropy loss, λ dice To adjust the coefficients for the dice loss, N represents the total number of pixels, and t and p represent the true label and the predicted result, respectively; for each pixel i, t i ∈{0,1} represents the true category, where 0 indicates no change and 1 indicates change; the predicted probability is the probability that pixel i is classified as changed, i.e., p i For change, and 1-p i No change.

[0040] The style alignment loss function L of the auxiliary branch sfa It involves domain alignment of feature maps at different scale levels within the feature extraction backbone; the loss is composed of the shallow feature alignment loss L. sa L in deep feature alignment lossda It consists of two parts:

[0041] L sfa =L sa +L da ;

[0042] Among them, the shallow feature alignment loss L sa By comparing the feature covariance matrices of the same image at different time points, the consistency of shallow style representation is constrained, as shown below:

[0043]

[0044] in, and Let represent the covariance matrix of the i-th shallow feature map;

[0045] The deep feature alignment loss L da The method employs a direction-weighted Wasserstein distance approach, which uses one-dimensional distance fusion after multi-directional projection to measure the difference in semantic feature distribution. This is defined as follows:

[0046]

[0047] in, <X (j) θ l >, <Y (j) θ l > represent the distribution of the j-th biphase feature along the l-th projection direction; w l These are the directional normalized weights calculated based on the energy function f(x) = exp(x); This represents the p-th order one-dimensional Wasserstein distance.

[0048] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0049] First, this invention can effectively distinguish between real changes and noise under multiple pseudo-change interferences, significantly improving the detection accuracy of change regions. Second, through refined feature representation and boundary modeling, the model can accurately characterize real change regions with complex structures and clear boundaries, ensuring the integrity and precision of the detection results. Furthermore, to address the style drift problem caused by factors such as illumination and imaging angle between images across time periods, the model designs feature alignment and style adaptation mechanisms to mitigate feature mismatch and improve the matching reliability between images from different time periods. Overall, this model not only improves the accuracy and robustness of change detection but also takes into account fine-grained boundary representation and feature consistency across time periods. In terms of precision, recall, F1 score, and intersection-union ratio (IU / IU), it achieves improvements of 3.72%, 6.93%, 5.18%, and 8.38% respectively compared to the existing change detection method FC-Siam-Conv; and improvements of 1.63%, 1.11%, 1.38%, and 2.32% respectively compared to the existing change detection method method bit-in-time (BIT). Attached Figure Description

[0050] Figure 1 This is a diagram showing the overall structure of a remote sensing image change detection model based on multi-dimensional feature collaborative modeling.

[0051] Figure 2 Structure diagram of the module for modeling differences in two-branch features;

[0052] Figure 3 Structure diagram of the boundary protection feature calibration module;

[0053] Figure 4 The diagram shows the change detection effect of the present invention, where (a), (b), and (c) are phase 1 images of different examples, (a1), (b1), and (c1) are phase 2 images of different examples, and (a2), (b2), and (c2) are the change detection results of different examples.

[0054] Figure 5 The change detection results of existing change detection methods are shown in the following figures: (a), (b), and (c) are phase 1 images of different examples, (a1), (b1), and (c1) are phase 2 images of different examples, (a3), (b3), and (c3) are the change detection results of the existing method FC-Siam-Conc in different examples, and (a4), (b4), and (c4) are the change detection results of the existing method BIT in different examples. Detailed Implementation

[0055] The embodiments described below are exemplary descriptions of key experimental evidence and are not intended to limit the core content and application scope of this invention due to the amount of evidence. It should be noted that all the accompanying drawings and corresponding descriptions merely illustrate the concept, principles, and representative experimental evidence of the disclosed embodiments of this invention. Where the chain of evidence is complete, it is unnecessary to show all the specific details and extended details of the various embodiments listed in this invention.

[0056] Unless otherwise defined, the technical terms used in the following embodiments have the same meaning as commonly understood by those skilled in the art to which this invention pertains.

[0057] The main processing steps of the remote sensing image change detection model based on multi-dimensional feature collaborative modeling of this invention include:

[0058] Step 1: Data Preprocessing

[0059] Due to the limitations of GPU memory resources, the images in the public benchmark dataset for remote sensing image change detection are preprocessed by segmentation. The original dual-temporal image pairs are all cropped into image blocks of size 256×256. The preprocessed dataset is then divided into training set, validation set and test set.

[0060] Step 2: Construct a remote sensing image change detection model based on multi-dimensional feature collaborative modeling

[0061] The model mainly consists of the following parts: a feature extraction backbone network, a two-branch feature difference modeling module, a boundary protection feature calibration module, a style difference feature alignment module, and a decoder.

[0062] In the feature extraction stage, the Mix Transformer in the SegFormer architecture is selected as the backbone network to construct multi-scale and highly expressive feature representations. The obtained feature maps are then processed by the backbone branch and auxiliary branches. The backbone branch directly feeds the dual-temporal feature map pairs from each level of the feature extraction stage into the Dual-branch Feature Difference Modeling Module (DFDMM), which consists of a subtraction branch and a concatenation branch, to achieve efficient modeling of difference information and further construct multi-scale difference features. Finally, the multi-scale difference features are input into the Boundary Preservation Feature Optimization Module (BPFOM) to enhance and preserve boundary features during the discrimination of changing regions. Next, these enhanced features are uniformly input into the decoder stage, which consists of a feature fusion module and a prediction head. First, the decoder receives the multi-scale difference feature maps after boundary enhancement and performs uniform spatial scale adjustments through the MLP and upsampling modules to obtain feature representations with consistent dimensions. Subsequently, the features from the four scales are concatenated along the channel dimension to form a high-dimensional fusion representation. This fusion is then further processed by an MLP to integrate cross-scale feature information, generating a unified difference feature map. This difference feature map serves as input to the prediction head, undergoes upsampling, and is used for classification, ultimately outputting a binary change detection result. The auxiliary branch inputs the feature map obtained during feature extraction into the Style-Difference Feature Alignment Module (SFAM) to align feature distributions and mitigate style differences between temporal images.

[0063] The overall structure diagram of the model is as follows Figure 1 As shown.

[0064] 2-1 Feature Extraction Backbone Network

[0065] In the feature extraction stage, the Mix Transformer from the SegFormer architecture is selected as the backbone network to construct multi-scale and highly expressive feature representations. The model input is dual-temporal remote sensing images, and the encoder adopts a weight-sharing mechanism, that is, the two temporal images are extracted through encoder branches with identical structures and shared parameters. In the initial stage, MiT divides the input image into multiple sub-regions using overlapping image patch embedding and extracts basic features through convolutional operations. Unlike traditional Transformers that heavily rely on positional encoding, MiT naturally incorporates positional information into the structure through convolutional receptive fields, thus possessing stronger scale adaptability and resolution transfer robustness. The feature extraction backbone consists of four progressively advancing Transformer encoding stages, each outputting a feature map at a corresponding scale, namely 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the original resolution. With the help of an efficient self-attention mechanism, the backbone network can capture subtle changes in local regions at shallow layers and model cross-regional semantic dependencies at deep layers, thereby enhancing the response capability to long-distance changes. In particular, the sequence compression strategy introduced by MiT significantly reduces the computational complexity of the self-attention mechanism. Through hierarchical design and convolutional fusion mechanism, MiT effectively ensures the information integrity of multi-scale features, laying a solid feature foundation for the accurate identification of subsequent change regions.

[0066] 2-2 Two-branch feature difference modeling module

[0067] After feature extraction, the construction of the difference feature map plays a crucial role in the final change detection result. Most existing methods use feature subtraction, feature concatenation, or feature fusion to characterize the differences between two temporal images. However, these methods mainly rely on linear comparisons of local spatial features, making it difficult to fully depict the deep semantic relationships and change patterns across time phases. Especially when there is significant interference in the two temporal images, this coarse-grained difference modeling strategy often weakens the true change information and even misclassifies non-changed areas as changed areas. To more effectively capture change information, a dual-branch feature difference modeling module was designed, combining the advantages of feature subtraction in emphasizing spatial changes with the advantages of feature concatenation in preserving temporal information, and achieving complementarity through feature fusion. The module structure diagram is shown below. Figure 2 As shown.

[0068] The 2-2-1 feature subtraction branch first performs element-wise subtraction on the dual-temporal feature maps to obtain preliminary difference features, explicitly highlighting regions of change. Then, based on these difference features, the SimAM attention mechanism is introduced. This is a lightweight attention module based on energy function optimization that adaptively generates a 3D attention weight map without additional parameters to emphasize regions of significant change. The resulting attention map then performs element-wise weighting on the difference features, enhancing the response to regions of change, resulting in the output E of the feature subtraction branch. sub Next, the attention map is used to weight the original two-phase features element-wise and sum the residuals, which can enhance change-sensitive regions while maintaining the original semantic structure, thereby generating an enhanced feature map containing significant change information.

[0069] The 2-2-2 feature concatenation branch aims to capture the semantic dependencies between features across two time phases. First, 1×1 convolutions are performed on the enhanced feature maps containing significant changes from both time phases, generating three sets of features: Query, Key, and Value. To further improve the robustness of semantic modeling, attention enhancement is applied to the channels of the generated Query features from both time phases. Specifically, global average pooling and global max pooling are used to extract global channel descriptions, which are then input into a shared two-layer multilayer perceptron. After Sigmoid activation, channel saliency weights are obtained. Next, a cross-learning mechanism is used to match the Query from one time phase with the Key from another, thus constructing a bidirectional semantic correspondence and obtaining similarity matrices S1 and S2. After obtaining the two cross-time phase similarity matrices, the Value feature from one time phase is multiplied by the similarity matrix of another time phase, and then linearly transformed using a 1×1 convolution. Finally, this is added to the residual of the original feature from that time phase to form the semantically aligned enhanced feature. and Let the original characteristic of phase 1 be E. pre The query obtained after convolution of enhanced features containing significant change information is Q. pre Key is K pre Value is V pre The original feature of phase 2 is E post The query obtained after convolution of enhanced features containing significant change information is Q. post Key is K post Value is V post Global average pooling is AvgPool, global max pooling is MaxPool, the sigmoid activation function is σ, the multilayer perceptron is MLP, and the softmax function is Softmax. The above process can be represented as follows:

[0070]

[0071] Finally, the enhanced features with semantic alignment between the two time phases are concatenated, and after processing with 3×3 convolution, batch normalization, and ReLU activation, the feature concatenation branch result E is output. add .

[0072] 2-2-3 Performs bi-branch feature fusion. To fully integrate the outputs of the feature subtraction branch and the feature concatenation branch, element-wise addition is performed, and ReLU activation is used to introduce non-linear expressive power. Subsequently, a Fast Fourier Transform is used to transform the fused features into frequency domain spatial features F. merge This allows for the capture of a wider range of global variation patterns. In the frequency domain, a global group filter is introduced. Multiple learnable filter banks are designed through group convolution. The features obtained by element-wise addition of two branches are processed through a linear layer and global average pooling to generate channel attention weights, adaptively fusing different frequency responses. The frequency domain features enhanced by filtering are then restored to the spatial domain through inverse fast Fourier transform to obtain F′. merge Let G be the filtering operation, and the inverse fast Fourier transform be IFFT. The above process can be represented as follows:

[0073] F′ merge =IFFT(G(E) sub +E add )⊙F merge )

[0074] Finally, the frequency domain enhancement features are added element-wise to the output features of the first two branches. After normalization and convolution, the final multi-scale difference feature E is output. i (i = 1, 2, 3, 4) to achieve high-precision modeling of dual-temporal changes.

[0075] 2-3 Boundary Protection Feature Calibration Module

[0076] Boundary information is crucial for accurately characterizing changing regions, but most existing change detection models do not explicitly model boundary features, often leading to blurred boundaries. This is especially problematic when dealing with small targets or fine-grained changes, where the lack of boundary information can easily cause false positives or false negatives. To maximize the preservation of boundary position information during forward propagation and mitigate noise interference from shallow features, a boundary protection calibration module is designed. Building upon boundary enhancement, an offset calibration mechanism is introduced: calculating the offset between adjacent layers of multi-layer differential features, remapping deep features, and correcting shallow features layer by layer, thereby protecting boundaries while alleviating the misalignment of semantic and positional information. The input features of this module are the multi-scale differential features output from the dual-branch feature differential modeling module. The module structure diagram is shown below. Figure 3 As shown.

[0077] In the 2-3-1 boundary enhancement stage, taking the adjacent first two layer difference feature maps E1 and E2 as examples, to address the issue of rich boundary details but high noise in shallow features, detail enhancement is first performed on the shallow features. Multi-channel features are compressed using a 1×1 convolution, and a boundary probability map is obtained through Sigmoid activation. To prevent excessive edge response, a two-dimensional 5×5 Gaussian convolution kernel is used for smoothing, and a high-frequency residual structure similar to a Laplacian pyramid is constructed by combining downsampling and upsampling operations. By differencing the results before and after smoothing, high-frequency edge components in the image can be effectively preserved. This approach highlights boundaries and texture details. Let the downsampling operation be Dn, the upsampling operation be Up, the Gaussian convolution be Ga, the Sigmoid operation be σ, and the 1×1 convolution be Conv. 1×1 The above process can be represented as:

[0078]

[0079] Subsequently, high-frequency edge components Element-wise multiplication with the original feature E1 and summing the residuals yields the shallow features with enhanced details. This enhances the detail representation at the boundaries of changing regions and suppresses noise interference.

[0080] 2-3-2 Adjacent Layer Migration Calibration Stage: First, E2 is upsampled to the same spatial size as E1 using bilinear interpolation, denoted as... The shallow features enhanced with detail are concatenated with adjacent upsampled deep features, and a SE module is introduced to perform channel weighting on salient regions to obtain fused features. To better preserve boundary position information in the encoder and progressively calibrate target information to reduce noise interference, semantic offsets are extracted between adjacent feature layers. The fused features are then convolved with a 3×3 layer to generate a semantic offset map, which is used to predict the spatial displacement relationship between features. Based on this offset map, a grid sampling operation is used to perform pixel-level mapping on the upsampled deep features according to the offset map, achieving spatial alignment of shallow and deep features. The aligned deep features are then presented. Shallow features with enhanced detail The fusion feature I1 is obtained by adding them together.

[0081] In the 2-3-3 multi-stage feature alignment and supplementation stage, the aforementioned shallow feature detail enhancement and offset calibration strategies between adjacent layers are iteratively applied across multiple levels. This involves sequentially upsampling deep features and fusing them with the enhanced features from the previous stage. An offset map is generated through convolution and spatially mapped, gradually completing the alignment and boundary strengthening of multi-layer features. The resulting fused feature not only possesses a higher resolution structural response at the boundaries but also achieves cross-layer consistency at the semantic level. The final output feature I2 is the boundary-enhanced feature map that fuses the three-layer differential features E1, E2, and E3 and has undergone offset calibration, which can be used to generate the final saliency map.

[0082] In the 2-3-4 feature supplementation and global correction stage, since deep features have strong global perception capabilities but weak boundary representation, a feature supplementation mechanism is introduced to improve spatial localization accuracy. First, the deepest differential feature E4 is compressed using a 1×1 convolution, then upsampled to the same size as the boundary enhancement feature I2. The two are then concatenated and followed by a 1×1 convolution and sigmoid activation to generate a normalized attention map E. att .

[0083] The attention map, after layer-by-layer downsampling, is applied to the multi-scale differential feature map E. i This enables layer-by-layer completion and fine-grained enhancement based on boundary-semantic relationships. Combined with the channel-adaptive weighting of the SE module, the model can achieve multi-layer feature E... i The process enhances the response to key information and generates the final multi-layered feedback feature D. i Let DS represent the downsampling operation, SE represent the SE module, and E represent the E module. i For multi-scale difference features, the above process can be represented as:

[0084] D i =SE((DS(E) att )+1)⊙E i )

[0085] 2-4 Style Difference Feature Alignment Module

[0086] In remote sensing image change detection tasks, the input dual-temporal images are affected by external factors such as illumination conditions, cloud cover, and imaging angle, resulting in significant differences in visual style between images at different time points. These factors not only cause changes in the spectral curves of the images but also lead to shifts in the feature distribution of the same category across different time phases, introducing spurious change interference and reducing the model's discriminative performance. Drawing on the concept of domain adaptation, a style difference feature alignment module was designed after the feature extraction backbone network to reduce inter-domain shifts caused by non-semantic factors, performing feature alignment at both shallow and deep levels. The outputs of the first and second layers of the feature extraction backbone network were selected as shallow features, while the outputs of the third and fourth layers were selected as deep features.

[0087] In the shallow layer stage (2-4-1), the module standardizes the features using instance normalization, ensuring that the mean and variance of features across all channels are consistent, thereby mitigating distribution differences between images caused by varying imaging conditions. Subsequently, the covariance matrix of features from two different time phases is calculated, and statistical alignment of shallow features is achieved by minimizing the difference between covariances. This approach effectively suppresses interference from illumination, color, and texture while preserving structural information, making the feature distribution statistically consistent and providing a stable input foundation for subsequent semantic learning.

[0088] 2-4-2 In the deep feature stage, the feature representation becomes more abstract and semantic, and traditional covariance constraints are no longer sufficient to capture its complex distribution shift. Therefore, energy-driven orientation-aware slice Wasserstein distance (IS-EBSW) is introduced as a deep feature alignment metric. First, the feature map is treated as a sample distribution, and the deep features are flattened into a point set representation. and The feature size is H×W×C, N=H×W represents the number of feature points, and each point x i y j Let S represent the C-dimensional feature vector at a spatial location. Since directly calculating the Wasserstein distance in high-dimensional space is costly, a method based on the unit sphere S... C-1 Randomly sample L directions Projecting the high-dimensional distribution onto a one-dimensional plane yields the corresponding projected representation. and And in each direction θ l By sorting the projected values ​​and matching them point by point, the one-dimensional p-order Wasserstein distance in that direction can be calculated efficiently.

[0089]

[0090] Where {x (i)},{y (i)} respectively represent the directions θ l The projection values ​​are sorted in ascending order. However, not every projection direction is equally important; some directions are more discriminative in representing changes and should be given a greater weight in terms of loss contribution. An energy function f(x) = exp(x) is defined to exponentially amplify the Wasserstein distance in each direction, forming an exponentially perceptive direction-sensitive scheduling. For the l-th direction, its weighting coefficient is w. l for:

[0091]

[0092] By using this direction-weighted distribution metric, the module can flexibly align the distribution of high-dimensional semantic features in key directions, enabling the model to capture structural consistency while distinguishing semantic variations.

[0093] Step 3: Model Training

[0094] Using Nvidia GPUs, the constructed network model is trained in batches using the created training set. During training, the network model with the minimum training loss is saved, and the model is continuously optimized using the error backpropagation algorithm. At the same time, data augmentation techniques such as random flipping, scaling, cropping, and Gaussian blur are used to improve the model's robustness and generalization ability in different scenarios.

[0095] During training, a loss function that combines the main branch and auxiliary branches was designed. The main branch design incorporates the main loss L, specifically tailored for the change detection task. cd The auxiliary branch is used to optimize pixel-level classification performance; the auxiliary branch is used to guide the style alignment loss L between shallow style information and deep features. sfa This is used to mitigate the shift in feature distribution across time phases, as follows:

[0096] L = L cd +λ sfa L sfa ;

[0097] Where, λ sfa To adjust the coefficients that affect the style alignment loss of auxiliary branches.

[0098] The main loss for detecting changes in the main branch employs a combination of binary cross-entropy loss and dice loss, where the binary cross-entropy L... bce The loss is used for pixel-level binary classification supervision, while the dice loss L... dice This focuses on the matching degree of the foreground change region, effectively alleviating the class imbalance problem. The two are defined as follows:

[0099] L dc =λ bce L bce +λ dice L dice ;

[0100]

[0101] Where, λ bce To adjust the coefficient of the binary cross-entropy loss, λ dice To adjust the coefficients for the dice loss, N represents the total number of pixels, and t and p represent the true label and the predicted result, respectively. For each pixel i, t... i∈{0,1} represents the true category, where 0 indicates no change and 1 indicates change. The predicted probability is expressed as the probability that pixel i is classified as changed, i.e., p. i For change, and 1-p i No change.

[0102] Style difference feature alignment loss L in auxiliary branches sfa This is achieved by aligning feature maps at different scale levels within the feature extraction backbone into their respective domains. The loss is comprised of the shallow feature alignment loss L. sa L in deep feature alignment loss da It consists of two parts:

[0103] L sfa =L sa +L da ;

[0104] Among them, the shallow feature alignment loss L sa By comparing the feature covariance matrices of the same image at different time points, the consistency of shallow style representation is constrained, as shown below:

[0105]

[0106] in, and Let represent the covariance matrix of the i-th shallow feature map.

[0107] The deep feature alignment loss L da The method employs a direction-weighted Wasserstein distance approach, which uses one-dimensional distance fusion after multi-directional projection to measure the difference in semantic feature distribution. This is defined as follows:

[0108]

[0109] in, <X (j) θ l >, <Y (j) θ l > represent the distribution of the j-th biphase feature along the l-th projection direction; w l These are the directional normalized weights calculated based on the energy function f(x) = exp(x); This represents the p-th order one-dimensional Wasserstein distance.

[0110] For training parameter settings, the optimizer uses stochastic gradient descent, with an initial learning rate of 1e-3, a momentum coefficient of 0.99, and a weight decay coefficient of 5e-4. Cosine annealing is used to adjust the learning rate. The batch size and number of training epochs are 8 and 200, respectively. No data augmentation is used during the testing phase; evaluation is performed directly on the original test set data.

[0111] Step 4: Test the test set.

[0112] By testing the test set with the model saved during training, the corresponding remote sensing image change detection result image can be obtained. For example... Figure 4 As shown, this is the remote sensing image change detection result corresponding to the remote sensing image change detection algorithm based on multi-dimensional feature collaborative modeling of the present invention. Figure 5 The image shown is the result of remote sensing image change detection corresponding to the existing method, which can intuitively demonstrate the advantages of the present invention in improving the remote sensing image change detection method.

[0113] Table 1 shows the quantitative results, which clearly demonstrate the superior performance of this invention in specific evaluation metrics. Compared to the existing change detection method FC-Siam-Conv, the precision, recall, F1 score, and intersection-over-union ratio (IoU) are improved by 3.72%, 6.93%, 5.18%, and 8.38%, respectively; and compared to the existing change detection method BIT, the accuracy is improved by 1.63%, 1.11%, 1.38%, and 2.32%, respectively.

[0114] Table 1

[0115]

[0116] It should be understood that the above embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. For those skilled in the art, various modifications or substitutions can be made without departing from the inventive concept, and all such modifications or substitutions should be considered within the scope of protection of this invention.

Claims

1. A remote sensing image change detection algorithm based on multi-dimensional feature collaborative modeling, characterized in that, Includes the following steps: Step 1: Perform block preprocessing on the images in the public benchmark dataset for remote sensing image change detection, and divide the dataset; Step 2: Construct a remote sensing image change detection model based on multi-dimensional feature collaborative modeling. The model consists of the following parts: 2-1 Feature extraction backbone network; 2-2 Two-branch feature difference modeling module; 2-3 Boundary Protection Feature Calibration Module; 2-4 Style Difference Feature Alignment Module; 2-5 decoder; The feature maps obtained in the feature extraction stage are divided into two paths: a main branch and an auxiliary branch, including the following steps: the main branch directly inputs the dual-temporal feature maps from each level of the feature extraction stage into the dual-branch feature difference modeling module to perform difference modeling to form multi-scale difference features; the multi-scale difference features are input into the boundary protection feature calibration module to enhance the feature expression capability of change edges; the enhanced features are uniformly input into the decoder stage to generate the final change detection result; The auxiliary branch inputs the feature map obtained in the feature extraction stage into the style difference feature alignment module to achieve feature distribution alignment, which is used to alleviate style differences between images across time phases. Step 3: Using Nvidia GPUs, train the constructed network model in batches using the created training set. During training, save the network model with the minimum training loss and continuously optimize the model using the error backpropagation algorithm. A loss function that combines the main branch and auxiliary branch is designed during training. The main branch is designed with a main loss for the change detection task to optimize pixel-level classification performance. The auxiliary branch is designed with a style alignment loss to guide the alignment of shallow style information with deep features and to alleviate cross-temporal feature distribution shift. Step 4: Test the remote sensing images in the test set using the model saved during training to obtain the result images of remote sensing change detection.

2. The remote sensing image change detection algorithm based on multi-dimensional feature collaborative modeling according to claim 1, characterized in that, Step 2 uses the Mix Transformer from the SegFormer architecture as the backbone network for feature extraction.

3. The remote sensing image change detection algorithm based on multi-dimensional feature collaborative modeling according to claim 1, characterized in that, Step 2, the dual-branch feature difference modeling module, combines feature subtraction branch and feature concatenation branch, and achieves complementarity between the two through feature fusion, and consists of the following steps: 2-2-1) The feature subtraction branch performs element-wise subtraction on the two-phase feature maps to obtain preliminary differential features. The SimAM attention mechanism is introduced to emphasize regions of significant change. The resulting attention map performs element-wise weighting on the differential features to enhance the response of regions of change and generate the feature subtraction branch result. This attention map is also used to perform element-wise weighting and residual addition on the original two-phase features to generate an enhanced feature map containing information of significant changes. 2-2-2) The feature concatenation branch performs 1×1 convolutions on the enhanced feature maps containing significant changes in both time phases to generate three sets of features: Query, Key, and Value. Channel attention enhancement is applied to the Query feature, and through a cross-learning mechanism, the Query of one time phase is matched with the Key of another time phase, thereby constructing a bidirectional semantic correspondence. After obtaining the cross-time phase similarity matrix, the matrix is ​​used to multiply the Value feature of the other time phase, and a linear transformation is performed through 1×1 convolution. Finally, it is added to the residual of the original feature to form a semantically aligned enhanced feature. Finally, the semantically aligned enhanced features of the two time phases are concatenated, and after 3×3 convolution, batch normalization, and ReLU activation, the feature concatenation branch result is output. 2-2-3) The feature subtraction branch and the feature concatenation branch are added element by element, and then transformed to the frequency domain space through ReLU and fast Fourier transform. In the frequency domain space, a global group filter is introduced, and the feature obtained by combining the element-wise addition of the two branch features is processed by a linear layer and the result of global average pooling to generate channel attention weights. The frequency domain features after filtering and enhancement are then restored to the spatial domain through inverse fast Fourier transform. The frequency domain enhancement features are added element-wise to the features from the first two branches, and after normalization and convolution, the final multi-scale difference features are output.

4. The remote sensing image change detection algorithm based on multi-dimensional feature collaborative modeling according to claim 1, characterized in that, Step 2, the boundary protection feature calibration module, introduces an offset calibration mechanism based on boundary enhancement. This mechanism calculates the offset between adjacent layers of multi-layer differential features, remaps deep features, and corrects shallow features layer by layer. This mitigates the misalignment of semantic and positional information while protecting the boundary. Specifically, it includes the following steps: 2-3-1) To enhance the shallow features, a 1×1 convolution is used to compress the multi-channel features, and a boundary probability map is obtained through Sigmoid activation. Then, a two-dimensional 5×5 Gaussian convolution kernel is used for smoothing, and a high-frequency residual structure similar to a Laplacian pyramid is constructed by combining downsampling and upsampling operations. By differentiating the results before and after smoothing, the high-frequency edge components are obtained, and they are multiplied element-wise with the original features and the residuals are added to obtain the shallow features after detail enhancement. 2-3-2) Perform offset calibration between adjacent layers, concatenate the shallow features of detail enhancement with the adjacent upsampled deep features, and introduce the SE module to perform channel weighting on the salient region to obtain the fused features. After 3×3 convolution, a semantic offset map is generated. Based on this offset map, grid sampling operation is used to perform pixel-level mapping on the deep features to achieve spatial alignment of deep and shallow features. The aligned deep features are added to the shallow detail enhancement features to obtain the fused features. 2-3-3) The above shallow feature detail enhancement and offset calibration strategy between adjacent layers are applied iteratively across multiple levels. That is, deep features are upsampled sequentially and fused with the enhanced features of the previous stage. An offset map is generated through convolution and spatial mapping is performed to gradually complete the alignment and boundary enhancement of multi-layer features. The final fused feature is a boundary enhancement feature map that fuses three layers of features and has undergone offset calibration. 2-3-4) The deepest differential features are compressed and upsampled to the same size as the boundary enhancement feature map through 1×1 convolution, and then concatenated with it. After 1×1 convolution and Sigmoid activation, a normalized attention map is generated. This attention map is then downsampled layer by layer and applied to feature maps of different scales.

5. The remote sensing image change detection algorithm based on multi-dimensional feature collaborative modeling according to claim 1, characterized in that, The style difference feature alignment module described in step 2 performs feature alignment at both the shallow and deep levels. The outputs of the first and second layers of the feature extraction backbone network are selected as shallow features, and the outputs of the third and fourth layers are selected as deep features. 2-4-1) In the shallow stage, instance normalization is used to standardize the shallow features so that the mean and variance of each channel feature are consistent, thereby reducing the distribution differences between images caused by different imaging conditions; then the covariance matrix of the two phase features is calculated, and statistical alignment of shallow features is achieved by minimizing the difference between covariances. 2-4-2) In the deep stage, the deep features are flattened into a point set representation. and The feature size is H×W×C, N=H×W represents the number of feature points, and each point x i y j This represents a C-dimensional eigenvector at a spatial location, originating from the unit sphere S. C-1 Randomly sample L directions Projecting the high-dimensional distribution onto a one-dimensional plane yields the corresponding projected representation. and And in each direction θ l The projected values ​​are sorted and matched point by point to calculate the one-dimensional p-order Wasserstein distance in that direction. Where {x (i) }, {y (i) } respectively represent the directions θ l The projected values ​​are sorted in ascending order; the energy function f(x) = exp(x) is defined, and the Wasserstein distance in each direction is exponentially amplified; for the l-th direction, the weighting coefficient is w. l for:

6. The remote sensing image change detection algorithm based on multi-dimensional feature collaborative modeling according to claim 1, characterized in that, The loss function for merging the main branch and auxiliary branch in step 3 is: L=L cd +λ sfa L sfa ; Among them, L cd Detecting the principal loss by changing the main branch, L sfa For the style alignment loss of the auxiliary branch, λ sfa To adjust the coefficients that affect the style alignment loss of auxiliary branches.

7. The remote sensing image change detection algorithm based on multi-dimensional feature collaborative modeling according to claim 6, characterized in that, The main loss for detecting changes in the main branch employs a combination of binary cross-entropy loss and dice loss, where the binary cross-entropy L... bce The loss is used for pixel-level binary classification supervision, while the dice loss L... dice This focuses on the matching degree of the foreground change region, effectively alleviating the class imbalance problem. The two are defined as follows: L cd =λ bce L bce +λ dice L dice ; Where, λ bce To adjust the coefficient of the binary cross-entropy loss, λ dice To adjust the coefficients for the dice loss, N represents the total number of pixels, and t and p represent the true label and the predicted result, respectively; for each pixel i, t i ∈{0,1} represents the true category, where 0 indicates no change and 1 indicates change; the predicted probability is the probability that pixel i is classified as changed, i.e., p i For change, and 1-p i No change.

8. The remote sensing image change detection algorithm based on multi-dimensional feature collaborative modeling according to claim 6, characterized in that, The style alignment loss function L of the auxiliary branch sfa It involves domain alignment of feature maps at different scale levels within the feature extraction backbone; the loss is composed of the shallow feature alignment loss L. sa L in deep feature alignment loss da Two parts composition: L sfa =L sa +L da ; Among them, the shallow feature alignment loss L sa By comparing the feature covariance matrices of the same image at different time points, the consistency of shallow style representation is constrained, as shown below: in, and Let represent the covariance matrix of the i-th shallow feature map; The deep feature alignment loss L da The method employs a direction-weighted Wasserstein distance approach, which uses one-dimensional distance fusion after multi-directional projection to measure the difference in semantic feature distribution. This is defined as follows: in, <X (j) θ l >, <Y (j) θ l > represent the distribution of the j-th biphase feature along the l-th projection direction; w l These are the directional normalized weights calculated based on the energy function f(x) = exp(x); This represents the p-th order one-dimensional Wasserstein distance.

Images